Ensemble Interval Prediction for Solar Photovoltaic Power Generation

Abstract

In recent years, solar photovoltaic power generation has emerged as an essential means of energy supply. The prediction of its active power is not only conducive to cost saving but can also promote the development of solar power generation industry. However, it is challenging to obtain an accurate and high-quality interval prediction of active power. Based on the data set of desert knowledge in the Australia solar center in Australia, firstly, we have compared twelve interval prediction methods based on machine learning. Secondly, six ensemble methods, namely Ensemble-Mean, Ensemble-Median (Ensemble-Med), Ensemble-Envelop (Ensemble-En), Ensemble-Probability averaging of endpoints and simple averaging of midpoints (Ensemble-PM), Ensemble-Exterior trimming (Ensemble-TE), and Ensemble-Interior trimming (Ensemble-TI) are used to combine forecast intervals. The result indicates that Ensemble-TE is the best method. Additionally, compared to other methods, Ensemble-TE ensures the prediction interval coverage probability for confidence levels of 95%, 90%, 85%, and 80% as 0.960, 0.920, 0.873, and 0.824, respectively, using 15-min level data. Meanwhile, the narrower prediction interval normalized averaged width is calculated for the same confidence levels as 0.066, 0.045, 0.035, and 0.028, respectively. In addition, higher Winkler score and smaller coverage width-based criterion are obtained, representing high-quality intervals. We have calculated smaller mean prediction interval center deviation, which is approximately 0.044. Thus, the above demonstrates that this study obtains the prediction interval with better performance compared to other existing methods.

1. Introduction

1.1. Motivation and Incitement

With the depletion of traditional non-renewable energy sources and increasingly serious problems of environmental pollution and climate warming, it is vital to find suitable renewable sources of energy, advocate the low-carbon economy, and achieve carbon neutralization as soon as possible. Solar photovoltaic power generation, which uses clean energy, is a good way to promote the sustainable development of energy [1]. As a type of new energy, solar energy is considered to be abundant, widely distributed, and environmentally friendly. Therefore, solar photovoltaic power generation has been widely recognized all over the world and occupies an essential position in the new energy area [2]. The penetration of solar energy in power systems around the world has also been increased [3]. However, it is limited by certain physical factors, such as daylight change, local climate, and geographical location [4]. Meanwhile, active power is volatile and uncertain [5], which not only influences the operation and maintenance of solar photovoltaic power generation negatively, but also brings great challenges to its prediction. Overestimation of active power will increase the cost of power supply system, while underestimation will lead to insufficient power consumption in the whole society, thus affecting people’s normal life. Therefore, it is extremely necessary to find an effective and accurate power prediction method for solar photovoltaic power generation.

1.2. Literature Review and Research Gaps

The prediction of solar photovoltaic power generation mainly consists of two parts: point prediction and interval prediction. They are also known as deterministic prediction and probabilistic prediction with different forms of prediction results. Specifically, the point prediction implies the use of multiple variables to train the regression algorithm and obtain a series of single-point prediction values. Currently, many studies have researched point prediction in solar photovoltaic power generation. Wang et al. (2018) [6] used gated recurrent unit (GRU) to predict the photovoltaic power generation. Benali et al. (2019) [7] utilized random forests (RF) and artificial neural network (ANN) models to predict and compare solar power output. Dash et al. (2021) [8] used empirical wavelet transform (EWT) and robust minimum variance random vector functional link network (RRVFLN) in short term solar power forecasting. Elsaraiti et al. (2022) [9] demonstrated that the LSTM network could provide reliable information for the photovoltaic power forecast. Shedbalkar et al. (2022) [10] proposed using the Bayesian linear regression to solve solar power generation forecasting problem. Elizabeth et al. (2022) [11] proposed a multi-step convolutional neural network (CNN) Stacked LSTM technique to predict short-term solar power. In addition, there are dozens of related machine learning methods for point prediction of solar photovoltaic power generation [4,5]. However, the single-point prediction obviously cannot meet the researchers’ requirements for the reliability of power generation prediction. The uncertainty and error range of the predicted value are considered essential as well. Therefore, it is important to study and develop the interval prediction of the solar photovoltaic power generation.

In contrast, interval prediction possesses more research value, since it provides more uncertain information to quantify the extent to which we can trust the predictions of the model. The purpose of the interval prediction is to obtain the forecast interval C for real value y and a given confidence level $\alpha$ to ensure:

$$P(y\in C)\ge \alpha .$$

It can not only quantify the uncertainty of the predicted value and estimate more accurately, but also facilitate relevant staff to regulate and plan for the entire solar photovoltaic power generation system. In recent years, many related studies have been reported. Almeida et al. (2015) [12] used quantile regression forests (QRF) to construct prediction intervals. Ni et al. (2017) [13] proposed a method based on the extreme learning machine (ELM) and lower upper bound estimation (LUBE) to construct a reliable solar energy prediction interval. Huang et al. (2017) [14] used the k-nearest neighbor to obtain the interval of solar energy prediction. Pan et al. (2021) [15] used attention mechanism-gated recurrent unit-kernel density estimation (A-GRU-KDE) to predict and estimate the interval of solar power. Wang et al. (2021) [16] combined LSTM and Gaussian process regression (GPR) to obtain a reliable interval estimation in solar power generation. Ramkumar et al. (2021) [17] proposed a model of solar photovoltaic power interval forecast based on an online sequential extreme learning machine with a forgetting mechanism (FOS-ELM) algorithm. Li et al. (2022) [18] proposed a method for interval forecasting day-ahead solar power generation based on extreme gradient boosting (XGBoost) and KDE. Chen et al. (2022) [19] proposed a method using the density peak clustering improved by the kernel Mahalanobis distance (KMDDPC) combined with the multivariate kernel density estimation (MKDE) method to obtain the prediction intervals in four seasons. The methods mentioned above have their respective advantages and their prediction accuracy is also different. To combine the advantages of a variety of methods and further improve the prediction accuracy, the ensemble method is proposed, which also have the opportunity to obtain prediction intervals with higher quality.

The multi-model ensemble method, i.e., the model combination method, is the average of estimates or forecasts from multiple models that are different from each other through suitable weights [20], which can not only fully utilize the results of each method but also obtain more stable results [21]. Zhang et al. (2011) [20] chose a variety of weight selection criteria, such as smoothed-Akaike information criterion (S-AIC), smoothed-bayesian information criterion (S-BIC), and optimal weight selection method to average the model. Zhang et al. (2011) [22] used the focused information criterion (FIC) and frequentist model average (FMA) in generalized additive partial linear models (GAPLMs) to study model selection and model averaging. Gaba et al. (2017) [23] proposed the probability averaging of endpoints and simple averaging of midpoints (PM) to combine interval prediction results. The above studies reveal that the ensemble method can further improve the performance of the results of each method.

1.3. Major Contribution and Organization

This study proposes an ensemble interval prediction for solar photovoltaic power generation that obtains prediction intervals with higher quality than other methods. The main contributions of this paper can be stated as follows:

1.

Twelve latest interval prediction methods with good results are compared simultaneously in this paper in order to understand their advantages and applicability, and use them effectively. To the best of our knowledge, the existing literature does not focus on the horizontal comparison for these methods.

2.

Six ensemble methods of prediction interval are used for a combination of prediction intervals of the aforementioned partial methods. The focus is on the advantages of those methods to obtain more reliable and stable interval prediction results.

3.

Compared with the previous prediction results, the ensemble or combination method of interval described in this study further improves the quality of interval prediction: the prediction interval coverage probability (PICP) under different confidence levels is close to the given confidence level; it has a smaller prediction interval normalized averaged width (PINAW), i.e., narrower and more accurate standardized average interval width; it has smaller coverage width-based criterion (CWC) and higher Winkler score, that is, the interval holds a higher comprehensive quality; and it has a smaller mean prediction interval center deviation (MPICD), that is, the actual value is closer to the midpoint of the prediction interval.

The rest of this paper is organized as follows. Section 2 provides a detailed description of the methods of point prediction, methods of interval prediction, and ensemble methods used in this study. In Section 3, the data sources, data preprocessing, and performance evaluation indexes used in this study are presented. In Section 4, the experimental results of point prediction, interval prediction, and interval prediction using ensemble method are described and compared, and the best-combined interval forecasting method is identified. Finally, the main conclusions and contributions of this paper along with future research prospects are described in Section 5.

2. Methodology

This section mainly includes three parts: point prediction, which is used to obtain the deterministic prediction of active power, method of interval prediction, which is used to get the probability prediction of active power, and ensemble method of interval prediction, which is used to combine the constructed intervals to obtain higher quality intervals. Specifically, the flow chart describing the three kinds of methods is illustrated in Figure 1.

2.1. Point Prediction

2.1.1. Random Forest (RF)

RF is an ensemble algorithm that was first proposed by Breiman [24]. Its base learner is CART, and the ensemble method is bagging [25].

The objective of bagging is to use the bootstrap [26] method to resample the original training samples to gain a large number of independent sample subsets. Then, the average or mode of their fitting values is computed to average the model with high difference and small deviation to reduce the variance of the model. Based on the binary decision tree, the objective of the CART algorithm is to recursively bisect each input feature to divide the input space into finite elements, where we predict the probability distribution, including generation and pruning.

RF combines the idea of CART and bagging. Then, it introduces random feature selection to decorrelate the tree to improve bagging. RF randomly samples without replacement rather than selecting all of the original training samples. In addition, it randomly selects partial features for the split points on each decision tree. Generally, the number of features (k) opted is approximately the square root of the number of features (p) [27], i.e., $k\approx \sqrt{p}$. Compared with the single regression tree, RF can avoid the negative influence of noise, outliers, and overfitting and achieve more accurate and robust results.

2.1.2. Gated Recurrent Unit (GRU)

GRU [28] is a special RNN [29], whose basic structure is similar to that of RNN. More specifically, it uses the current input $x_t$ and previous hidden state $h_{t-1}$ to obtain the current result $y_t$ and current hidden state $h_t$ as the outputs.

Unlike the LSTM network with three gates, which also belongs to RNN, GRU has only two gates, including an update gate and a reset gate. The network structure of GRU is depicted in Figure 2.

In Figure 2, the update gate $r_t$ is used to control and filter the extent to which the state information of the previous time is brought into the current state. The larger the value of $r_t$, the more state information of the previous time is retained. The expression of $r_t$ is as follows:

$$r_t=\sigma \left(W_r[h_{t-1},x_t]\right),$$

where $\sigma$ is the sigmoid function and W represents the corresponding parameters to be trained (similarly hereinafter). The reset gate $z_t$ controls the amount of information of the previous state recorded to the current candidate set ${\tilde{h}}_t$. The smaller the value of $z_t$, the less information about the previous state is recorded. $z_t$ and ${\tilde{h}}_t$ are expressed as follows:

$$z_t=\sigma \left(W_z[h_{t-1},x_t]\right),$$

$${\tilde{h}}_t=tanh\left(W_{\tilde{h}}[r_t\ast h_{t-1},x_t]\right),$$

where ∗ is the Hadamard product. The expression of the new hidden state $h_t$ and current output result $y_t$ is as follows:

$$h_t=(1-z_t)\ast h_{t-1}+z_t\ast {\tilde{h}}_t,$$

$$y_t=\sigma \left(W_o{h}_t\right).$$

In general, both LSTM and CRU filter and retain important features and information obtained from various gate functions. Both of them can maintain long-term memory and alleviate the gradient disappearance problem, while GRU has one less gate function than LSTM, which leads to a more compact and simpler structure [30]. Therefore, we chose GRU for point prediction.

2.1.3. Gradient Boosting Regression Tree (GBRT)

Similar to RF, GBRT [31] with CART as its base learner is also an ensemble learning algorithm. Nonetheless, the ensemble method used in GBRT is different, and is known as boosting. Unlike bagging, boosting does not require resampling but generates multiple trees sequentially based on the modified values of the original data. Each tree learns the residuals between the results obtained by all previous trees and the real values to further update the residuals to gradually reduce them. GBRT provides accurate prediction and strong generalization ability without having high requirements on data. In this study, the mean and median of active power are considered as the prediction targets, and MSE and pinball loss (also known as weighted absolute deviations [32]) are selected as the scoring criteria to choose hyperparameters and then predict the active power.

In addition to the methods described above, ridge regression [33] and NGB are also used in this study for point prediction. Particularly, NGB will be introduced in Section 2.2, since it can output the results of point and interval prediction simultaneously.

2.2. Interval Prediction

2.2.1. Kernel Density Estimation (KDE)

KDE [34] is a commonly used nonparametric method that uses adjacent samples to estimate the density function at a certain point, and then estimates the probability density of the whole sample. The expression of KDE estimation is as follows:

$$\widehat{f}\left(x\right)=\frac{1}{nh}\sum _{i=1}^{n}K\left(\frac{x-x_i}{h}\right),$$

where $K\left(x\right)$ represents kernel function and h represents bandwidth. In this study, the Gaussian kernel is selected for KDE, and its expression is as follows:

$$K\left(x\right)=\frac{1}{\sqrt{2\pi }}{e}^{-\frac{x^2}{2}}.$$

The commonly used methods to determine the optimal bandwidth include the cross validation and adaptive bandwidth rule of thumb. In this study, we choose the 5-fold cross validation to select the bandwidth. Then we determine the KDE estimation of the sample probability density function. Next, we obtain the KDE estimation of the cumulative distribution function ${\widehat{F}}_h\left(x\right)={\int }_{-\infty }^x{\widehat{f}}_h\left(t\right)dt=\frac{1}{nh}{\sum }_{i=1}^n{\int }_{-\infty }^{x}K\left(\frac{t-x_i}{h}\right)dt$ to the pth quantile ${\widehat{Q}}_p\left(h\right)$ using ${\widehat{F}}_h\left(x\right)$, which can be expressed as follows:

$${\widehat{Q}}_p\left(h\right)=\sup \{x:{\widehat{F}}_h\left(x\right)\le p\}.$$

Finally, for new samples, given the confidence level $\alpha$, the constructed prediction interval C is described as follows:

$$C=[{\widehat{Q}}_{(1-\alpha )/2}\left(h\right),{\widehat{Q}}_{(1+\alpha )/2}\left(h\right)].$$

2.2.2. Natural Gradient Boosting (NGB)

NGB is a gradient boosting regression method that predicts the conditional probability distribution of the target variable to construct the prediction interval. Different from the general gradient boosting method, NGB is solved on the statistical manifold where the probability distribution is located (see [35] for the definition) and applied to multi-parameter distribution. More specifically, it includes basic learner $f^{\left(k\right)}$, parameterized conditional probability distribution $P_{\theta }\left(y\right|x)$, and appropriate scoring rules S. Scoring rules are used to calibrate the difference between model results and real values. If a scoring rule satisfies the following formula:

$$E_{y\sim Q}\left[S(Q,y)\right]\le E_{y\sim Q}\left[S(P,y)\right],$$

where P is the predicted distribution and Q is the real distribution, then it is called an appropriate scoring rule. The maximum likelihood estimation (MLE) is one of the common scoring rules, which can induce Kullback-Leibler Divergence. Choosing MLE as the scoring rule, the gradient for the prediction parameters $\theta$ on conditional probability density function $P_{\theta }\left(y\right|x)$ is recorded as ${\nabla }_{\theta }L(\theta ,y)$ and the natural gradient of each step is recorded as $g_i^k$, which can be obtained by solving the corresponding optimization problem. The expression of $g_i^k$ is as follows:

$$g_i^k=I_L{\left({\theta }_i^{k-1}\right)}^{-1}{\nabla }_{\theta }L({\theta }_i^{k-1},y),$$

where $I_L\left(\theta \right)$ is the Fisher information brought by the observed value of $P_{\theta }$, which can be written as:

$$I_L\left(\theta \right)=E_{y\sim P_{\theta }}\left[{\nabla }_{\theta }L(\theta ,y){\nabla }_{\theta }L{(\theta ,y)}^{\mathrm{T}}\right].$$

In the process of model learning, a group of basic learners is trained using natural gradient and input values at each stage. In this study, normal distribution is used; therefore, the parameters include mean $\mu$ and standard deviation $\sigma$, indicating that the corresponding basic learners are $f_{\mu }^{\left(k\right)}$ and $f_{\sigma }^{\left(k\right)}$. Then, scale as follows and update the parameter $\theta$:

$$\theta ={\theta }^{\left(0\right)}-\eta \sum _{k=1}^n{\rho }^{\left(k\right)}{f}^{\left(k\right)}\left(x\right),$$

where ${\rho }^{\left(k\right)}$ and $f^{\left(k\right)}\left(x\right)$ are the scaling coefficient and the basic learner of each stage, respectively, and $\eta$ is the learning rate. Finally, we obtained two sets of parameter values $\widehat{\mu }$ and $\widehat{\sigma }$ in the fitted normal distribution, that is, given a new sample x, $y|x\sim N(\widehat{\mu }\left(x\right),\widehat{\sigma }\left(x\right))$. Then, under the confidence level $\alpha$, the corresponding quantiles ${\mathsf{\Phi }}^{-1}((1-\alpha )/2)$ and ${\mathsf{\Phi }}^{-1}((1+\alpha )/2)$ are calculated and prediction interval C is constructed as follows:

$$C=[\widehat{\mu }\left(x\right)+\widehat{\sigma }\left(x\right){\mathsf{\Phi }}^{-1}((1-\alpha )/2),\widehat{\mu }\left(x\right)+\widehat{\sigma }\left(x\right){\mathsf{\Phi }}^{-1}((1+\alpha )/2)].$$

2.2.3. Jackknife+-after-bootstrap (J+ab)

J+ab [36] is a method for constructing prediction intervals in which the bootstrap and the jackknife+ [37] methods are used to construct prediction intervals. In addition, jackknife+ is a modified version of the jackknife method. To obtain the prediction interval using the J+ab method, firstly, the training dataset is resampled using bootstrap. Then, the regression model $\widehat{\mu }$ is trained to obtain the predicted value, which will be aggregated. The ensemble method described in this paper is mean. Next, we calculate the absolute value of the leave-one-out residual of each sample $R_i=|y_i-{\widehat{\mu }}_{(-i)}\left(x_i\right)|$, where ${\widehat{\mu }}_{(-i)}\left(x_i\right)$ is the predicted value of the regression model that is not trained by the sample i. It is also the output of the regression model trained by other subsets resampled in the bootstrap, which does not contain the sample i. Finally, for the new samples, the prediction interval C is constructed using $R_i$. The expression of C is as follows:

$$C=[q_{\alpha }^{-}({\widehat{\mu }}_{(-i)}\left(x\right)-R_i),q_{\alpha }^{+}({\widehat{\mu }}_{(-i)}\left(x\right)+R_i)],$$

where $q_{\alpha }^{-}\left(x\right)$ and $q_{\alpha }^{+}\left(x\right)$ represent the upper and lower $\alpha$th quantiles of x at the confidence level $\alpha$, respectively.

J+ab can be combined with any regression method for prediction since there is no requirement for the underlying regression algorithm. Therefore, we combine J+ab with Ridge, multi-layer perceptron (MLP), and RF for interval prediction, respectively.

2.2.4. Random Forest-Out-Of-Bag (RF-OOB)

Based on RF, RF-OOB [38] is a method that uses out-of-bag (OOB) [39] samples to calculate the error of OOB prediction and then obtains the empirical distribution to establish the prediction interval. After establishing the RF model, we can use the OOB sample corresponding to each point to calculate the prediction error given by $D_i=y_i-{\widehat{y}}_i^{\ast }$, $i=1,2,\dots ,n$, where ${\widehat{y}}_i^{\ast }$ represents the prediction value of the OOB samples. The original sample and prediction error are both independent and identically distributed. Therefore, we can establish the empirical distribution D of the prediction error, and then construct the prediction interval C for the new sample y. The expression of C is as follows:

$$C=[\widehat{y}+D_{[n,(1-\alpha )/2]},\widehat{y}+D_{[n,(1+\alpha )/2]}],$$

where $\alpha$ is the confidence level, $D_{[n,p]}$ is the pth quantile of D, and $\widehat{y}$ is the predicted value. Thus, we can get the prediction interval obtained by RF-OOB at a given confidence level.

2.2.5. Split Conformal-Random Forest (SC-RF)

SC-RF is a combination of split conformal (SC) [40] and RF. It uses split samples to separate the model fitting part from the subsequent sorting part and has no requirements for the underlying method. It can be combined with any regression method for prediction, and the computational cost is far less than the full conformal prediction [41].

SC-RF first divides the training samples into two parts $S_1$ and $S_2$ with the same sample size. $S_1$ is used as the training set to build the RF model $\widehat{\mu }$ and $S_2$ is used as the test set to generate the predicted value of the RF model. Then, the absolute value of the residual $R_i=|y_i-\widehat{\mu }\left(x_i\right)|,i\in S_2$ is obtained. Next, we calculate the kth value d in $R_i$; here $k=(n/2+1)\alpha$ and $\alpha$ is the confidence level. Finally, for the new sample x, we construct the prediction interval C, which can be expressed as follows:

$$C=[\widehat{\mu }\left(x\right)-d,\widehat{\mu }\left(x\right)+d].$$

It can be proven [40] that the prediction interval C satisfies $\alpha \le P(Y\in C)\le \alpha +2/(n+2)$, which assures that the coverage of C reaches the given confidence level $\alpha$.

2.2.6. Quantile Regression Forests (QRF)

Combined with RF and quantile regression [42], QRF [32] is a nonparametric conditional quantile method that is capable of predicting any quantile to establish a prediction interval. Compared with RF, which only retains the mean value of all observations in the leaves, QRF retains all values and evaluates the conditional distribution based on this information. The conditional distribution expression of the target variable when X is given as follows:

$$F\left(y\right|X=x)=P(Y\le y|X=x)=E\left(I\right(Y\le y\left)\right|X=x),$$

where $I(·)$ is the indicative function. Then, the weighted average of the observed values of $I(Y\le y)$ is used to approximate $E\left(I\right(Y\le y\left)\right|X=x)$ to obtain the estimation of the conditional distribution $\widehat{F}$, whose expression is as follows:

$$\widehat{F}\left(y\right|X=x)=\sum _{i=1}^n{w}_i\left(x\right)I(Y\le y),$$

$$w_i\left(x\right)=\frac{1}{k}\sum _{t=1}^k{w}_i(x,{\theta }_t),$$

$$w_i(x,{\theta }_t)=\frac{X_i\in R_l(x,\theta )}{\#\{j:X_j\in R_l(x,\theta )\}},$$

where $\theta$ is the parameter, $R_l(x,\theta )$ is the leaf node, $\#\{j:X_j\in R_l(x,\theta )\}$ indicates the number of samples belonging to $R_l(x,\theta )$, $w_i(x,{\theta }_t)$ is the weight vector, the sum of which is 1, and $w_i\left(x\right)$ is the weight in RF representing the average weight vector of k decision trees. After that, the estimation of pth quantile ${\widehat{Q}}_p\left(x\right)$ is obtained, whose expression is as follows:

$${\widehat{Q}}_p\left(x\right)=\sup \{y:\widehat{F}\left(y\right|X=x)\le p\}.$$

Finally, for the new sample x with the confidence level $\alpha$, the constructed prediction interval C is as follows:

$$C=[{\widehat{Q}}_{(1-\alpha )/2}\left(x\right),{\widehat{Q}}_{(1+\alpha )/2}\left(x\right)].$$

QRF provides a complete conditional distribution of the target variable when x is given and is widely used in the field of machine learning to estimate quantiles and obtain prediction intervals.

2.3. Ensemble Method of Interval Prediction

In this section, we introduce several ensemble methods for prediction intervals to combine a variety of constructed prediction intervals to obtain better prediction intervals. Specifically, six ensemble methods are introduced in this study, namely Ensemble-Mean, Ensemble-Med, Ensemble-En, Ensemble-TE, Ensemble-TI, and Ensemble-PM. It is assumed that m prediction intervals corresponding to n real values have been obtained, which is expressed as follows:

$$[L_{ij},U_{ij}],i=1,2,\dots ,n,j=1,2,\dots ,m,$$

Then, the final ensemble interval is as follows:

$$[L_i,U_i]=f\left([L_{ij},U_{ij}]\right),i=1,2,\dots ,n,j=1,2,\dots ,m,$$

where $f\left(x\right)$ represents the ensemble method of m prediction intervals.

Ensemble-Mean means that the ensemble interval is the mean value of m intervals, and the formula is given by:

$$L_i=\frac{1}{m}\sum _{j=1}^m{L}_{ij},U_i=\frac{1}{m}\sum _{j=1}^m{U}_{ij}.$$

Ensemble-Mean is one of the most commonly used ensemble methods, which is simple and easy to understand, and can represent the centralized trend of intervals.

Ensemble-Med represents the median value of m intervals, and the expression is as follows:

$$L_i=media{n}_j\left(L_{ij}\right),U_i=media{n}_j\left(U_{ij}\right).$$

Ensemble-Med has good stability and is not easily to be affected by extreme interval values. Therefore, some intervals with large fluctuations will be excluded.

Ensemble-En means that the lower bound of ensemble intervals takes the minimum value of the lower bound of prediction intervals, and the upper bound of ensemble intervals takes the maximum value of the upper bound of prediction intervals. In other words, the ensemble interval surrounds all the prediction intervals. The formula is given by:

$$L_i={\min }_j\left(L_{ij}\right),U_i={\max }_j\left(U_{ij}\right).$$

Ensemble-En may improve the coverage of the prediction interval. However, it may also make the interval too wide.

Ensemble-TE means to delete the outermost $2k$ points in the prediction intervals and then compute average, which is expressed as follows [23]:

$$k=\left\{\begin{array}{cc}0,\hfill & 1\le m\le 3,\hfill \\ 1,\hfill & 4\le m\le 7,\hfill \\ 2,\hfill & 8\le m\le 11,\hfill \\ 3,\hfill & 12\le m,\hfill \end{array}\right.$$

$$L_i=\frac{1}{m-k}\sum _{j=1}^{m-k}{L}_{ij}^{(-k_{\min })},U_i=\frac{1}{m-k}\sum _{j=1}^{m-k}{U}_{ij}^{(-k_{\max })},$$

where $L_{ij}^{(-k_{\min })}$ represents the lower bound of the remaining prediction intervals after deleting the minimum k values and $U_{ij}^{(-k_{\max })}$ represents the upper bound of the remaining prediction intervals after deleting the maximum k values. Ensemble-TE not only removes some outliers in the prediction intervals, making it easier to obtain a narrow ensemble interval, but also uses the advantage of mean value, representing the centralized trend of the remaining prediction interval. Thus, the entire information is used as fully as possible without being affected by larger extreme values.

Ensemble-TI means that $2k$ points at the innermost part of the prediction interval are deleted and then averaged. The expression is as follows:

$$L_i=\frac{1}{m-k}\sum _{j=1}^{m-k}{L}_{ij}^{(-k_{\max })},U_i=\frac{1}{m-k}\sum _{j=1}^{m-k}{U}_{ij}^{(-k_{\min })},$$

where $L_{ij}^{(-k_{\max })}$ represents the lower bound of the remaining prediction intervals after deleting the maximum k values and $U_{ij}^{(-k_{\min })}$ represents the upper bound of the remaining prediction intervals after deleting the minimum k values. Ensemble-TI retains the information on the outside of the prediction intervals and deletes the innermost information, which may help to appropriately improve the coverage of interval prediction.

Ensemble-PM [23] means that the ensemble interval is obtained by simply averaging the endpoints of the prediction interval after probability averaging, and it is assumed that the prediction interval of each point is established based on normal distribution. If $Y_i\sim N(\mu ,\sigma )$, then $Z_i=\frac{Y_i-\mu }{\sigma }\sim N(0,1)$, and the pth quantile is expressed, as follows:

$$P(Y_i\le y)=P(Z_i\le \frac{y-\mu }{\sigma })=\mathsf{\Phi }\left(\frac{y-\mu }{\sigma }\right)=p.$$

Therefore, the pth quantile on $N(\mu ,\sigma )$ can be expressed as [43]:

$$y=\mu +\sigma {\mathsf{\Phi }}^{-1}\left(p\right).$$

Then, average the quantile values corresponding to the ensemble interval on m cumulative distribution functions and make it equal to the quantile of the endpoint of the interval at the given confidence level $\alpha$; here, $p=(1-\alpha )/2,(1+\alpha )/2$, and the upper and lower bounds of the ensemble interval satisfy the following formula:

$$\frac{1}{m}\sum _{j=1}^m{F}_{ij}\left(L_i\right)=(1-\alpha )/2,\frac{1}{m}\sum _{j=1}^m{F}_{ij}\left(U_i\right)=(1+\alpha )/2,$$

where $F_{ij}$ is the cumulative distribution function of the jth prediction interval at the ith point. Ensemble-PM makes full use of the probability information of the prediction interval.

3. Data and Evaluation Indexes

3.1. Data Description and Processing

In this section, we introduce the data sources used in this study and how we process the data, including the selection of the period of data, the selection of variables, display of variable correlation, data preprocessing, and division of training, validation, and test sets.

The data used in this study are from the desert knowledge Australia solar center (DKASC) in Alice Springs, Australia at http://dkasolarcentre.com.au/locations/alice-springs (last access: 5 September 2022). This platform provides historical data queries, real-time displays of solar power generation, and relevant meteorological information on power station systems of various manufacturers. In this study, we select the data of the 31st site established in 2013. The time range is from 1 April 2014 to 31 October 2015 with an interval of 15 min and 5 min. The aim of this study is to predict the active power of solar energy. Since there is no solar energy at night to generate power, we limit the time range of each day to 5:30–19:00. The opted variables include three parts. First is the physical variables, including wind speed, weather temperature (Celsius), weather relative humidity, global horizontal radiation, and diffuse horizontal radiation. Second is the time variables, including hour and month. To ensure continuity, the month is mapped to cosine and sine values of the month, which are represented as month_cos and month_sin. The respective expressions are as follows:

$$\mathrm{month}\_\cos =cos(2\pi ·\mathrm{month}/12),\mathrm{month}\_\sin =sin(2\pi ·\mathrm{month}/12).$$

Considering the autocorrelation of the solar active power, in the third part, we choose four lagged values t-15, t-30, t-45, and t-60 of active power for data with an interval of 15 min, which respectively corresponds to the power values 15, 30, 45, and 60 min before the current time. Then, we calculate the missing proportion of each variable and find that only active power is missing, which is 0.03%. Because the missing proportion is very low, it can be deleted. Next, we draw the heatmap of Pearson correlation coefficients between the relevant variables of the model, as demonstrated in Figure 3. Correspondingly, we select four lagged values for data with an interval of 5 min and do the same process above. Given that the results are similar to that of the 15-min interval, we only illustrate the latter there.

Figure 3 reflects that the active power has a strong linear correlation with the global horizontal radiation and four lagged values t-15, t-30, t-45, and t-60. It also has a weak linear correlation with diffuse horizontal radiation, wind speed, weather temperature (Celsius), and weather relative humidity. In particular, it seems to have a very weak linear correlation with month_sin, month_cos, and hour. However, this only represents linear correlation. Further modeling and analysis are required for other correlations. In terms of the division of data set, we regard the period from 1 April 2014 to 30 June 2015 as the training set, 1 July 2015 to 31 August 2015 as the validation set, and 1 September 2015 to 31 October 2015 as the test set.

3.2. Performance Evaluation Indexes

3.2.1. Evaluation Indexes for Point Prediction

In this section, we introduce five indexes to evaluate the accuracy and fitting degree of the point prediction, including mean absolute error (MAE), root mean square error (RMSE), fitting coefficient (${\mathrm{R}}^2$), mean absolute percentage error (MAPE), and symmetric mean absolute percentage error (SMAPE), which are used to evaluate the difference between the predicted results of the model and real values. The specific expressions of the five indicators are as follows:

$$\mathrm{MAE}=\frac{1}{n}{\displaystyle \sum _{i=1}^n}\left|y_i-{\widehat{y}}_i\right|,$$

$$\mathrm{RMSE}=\sqrt{\frac{1}{n}\sum _{i=1}^n{\left(y_i-{\widehat{y}}_i\right)}^2},$$

$${\mathrm{R}}^2=\frac{{\sum }_{i=1}^n{\left({\widehat{y}}_i-\overline{y}\right)}^2}{{\sum }_{i=1}^n{\left(y_i-\overline{y}\right)}^2},$$

$$\mathrm{MAPE}=\frac{1}{n}\sum _{i=1}^n\left|\frac{y_i-{\widehat{y}}_i}{y_i}\right|\times 100\%,$$

$$\mathrm{SMAPE}=\frac{1}{n}\sum _{i=1}^n\left|\frac{y_i-{\widehat{y}}_i}{\left|y_i\right|+\left|{\widehat{y}}_i\right|}\right|\times 100\%,$$

where $\overline{y}=\frac{1}{n}{\sum }_{i=1}^n{y}_i$ represents the mean of the actual value, $y_i$ represents the actual value, ${\widehat{y}}_i$ represents the predicted value, and n is the sample size. MAE, RMSE, MAPE, and SMAPE measure the degree to which the predicted value deviates from the real value. The smaller the value of the four indexes, the closer the predicted value is to the real value and the stronger is the prediction ability of the model. ${\mathrm{R}}^2$ measures the fitting degree of the predicted value to the real value; that is, the extent of the proportion of variance of real value is explained by the predicted value of the model. The closer ${\mathrm{R}}^2$ is to 1, the better is the fitting effect of the model.

3.2.2. Evaluation Indexes for Interval Prediction

To measure the quality of prediction interval, we select five indicators, including PICP, PINAW, CWC, Winkler score, and MPICD to evaluate the reliability, accuracy, and relative position with the real value of the prediction interval.

PICP represents the proportion of the predicted interval covering the real value, which is used to measure the reliability of the interval. The prediction interval is recorded as $[L_i,U_i],i=1,2,\dots ,n$, and the expression of PICP is as follows:

$$\mathrm{PICP}=\frac{1}{n}\sum _{i=1}^{n}I(y_i\in [L_i,U_i]),$$

where $I(·)$ is the indicative function, which means $I(y_i\in [L_i,U_i])=1$ if $y_i\in [L_i,U_i]$, otherwise $I(y_i\in [L_i,U_i])=0$. Larger PICP ensures that more real values are covered and the prediction interval is more reliable. Generally speaking, for a given confidence level $\alpha$, PICP should be as close as possible to $\alpha$ when it is greater than or equal to $\alpha$ [44].

PINAW refers to the relative average width of the prediction interval after standardization, which is used to measure the accuracy of the interval. The expression of PINAW is as follows:

$$\mathrm{PINAW}=\frac{1}{nR}\sum _{i=1}^n(U_i-L_i),$$

where $R=\max \left(y_i\right)-\min \left(y_i\right)$ represents the range of data, which is used to further standardize the average width of the interval to exclude the width change caused by the variation of real value. Smaller PINAW results in a narrow relative prediction range and more accurate prediction interval. If PINAW is too large, it may easily close to PICP. However, the relative prediction range will be too wide to capture the detailed changes.

However, the two indicators mentioned above are contradictory, which means that they consider only one characteristic of the prediction interval. When the PICP is large enough, PINAW is also large. When the PINAW is small, the PICP often fails to meet the confidence level. Thus, it is difficult to comprehensively evaluate the interval quality. Therefore, we also select the CWC and Winkler score to comprehensively evaluate the reliability and accuracy of the prediction interval.

CWC will punish the prediction interval of PICP that does not reach the confidence level accordingly, and the expression is as follows:

$$\mathrm{CWC}=\mathrm{PINAW}\left(1+I\left(\mathrm{PICP}<\alpha \right)e^{-\eta \left(\mathrm{PICP}-\alpha \right)}\right),$$

where $\alpha$ represents the confidence level and $\eta$ represents the penalty factor, which is used to scale the difference between PICP and $\alpha$. When the PICP of the interval reaches the confidence level, CWC will be similar to PINAW. When PICP does not reach the confidence level, the corresponding penalty will be made to make the CWC larger. Therefore, smaller CWC better ensures the quality of the prediction interval. In this study, penalty factors 25 and 50 are opted for evaluation. However, it is noted that the excessive penalty of CWC results in part from the value being too small; thus, the penalty factor we chose is 25.

Winkler score [45] specifically penalizes the interval that does not cover the real value of each point. The expressions are as follows:

$$w_i^{\left(\alpha \right)}=U_i^{\left(\alpha \right)}-L_i^{\left(\alpha \right)},$$

$$S_i^{\left(\alpha \right)}=\left\{\begin{array}{cc}-2\alpha w_i^{\left(\alpha \right)}-4(L_i^{\left(\alpha \right)}-y_i),\hfill & y_i<L_i^{\left(\alpha \right)},\hfill \\ -2\alpha w_i^{\left(\alpha \right)},\hfill & y_i\in [L_i^{\left(\alpha \right)},U_i^{\left(\alpha \right)}],\hfill \\ -2\alpha w_i^{\left(\alpha \right)}-4(y_i-U_i^{\left(\alpha \right)}),\hfill & y_i>U_i^{\left(\alpha \right)},\hfill \end{array}\right.$$

$$S^{\left(\alpha \right)}=\frac{1}{n}\sum _{i=1}^n{S}_i^{\left(\alpha \right)},$$

where $w_i^{\left(\alpha \right)}$, $S_i^{\left(\alpha \right)}$, and $S^{\left(\alpha \right)}$ represent the absolute width of the prediction interval at confidence level $\alpha$, with an interval score at each point, and the final result of the Winkler score. When the real value is not covered by the prediction interval, the interval score will punishes the part exceeding the upper or lower bounds. Therefore, larger the Winkler score, the better the interval quality.

In addition, MPICD is selected to measure the position of the real value in the prediction interval. The specific formula is as follows:

$$\mathrm{MPICD}=\frac{1}{n}\sum _{i=1}^n\left|y_i-\frac{L_i^{\left(\alpha \right)}+U_i^{\left(\alpha \right)}}{2}\right|.$$

This formula indicates that MPICD quantifies the average difference between the real value and the midpoint of the prediction interval. The smaller the MPICD is, closer the real value is to the middle point of the interval, and the better is the prediction interval quality.

4. Results and Discussion

In this section, we conducted empirical research on the real data with both point prediction and interval prediction methods introduced in Section 2, along with modeling, analysis, prediction and model evaluation. All programs are implemented in Python 3.8.5 or R language 4.1.1.

4.1. Results of Point Prediction

For point prediction, the prediction performance of six methods, namely Ridge, RF, GRU, NGB, GBRT-Mean, and GBRT-Med, are compared. For each prediction model, the choice of hyperparameters is crucial for the performance of the model. We use grid search and cross validation on the validation set to evaluate the performance of the model and determine the optimal hyperparameter of each model, which are demonstrated in

Table 1. Hyperparameters of different methods in point prediction.

Forecasting Method	Hyperparameter
RF	Estimators = {100, 125, 150, 175, 200}
	Max depth = {10, 30, 50, 70, 100}
	Max features = {5, 6, 7, 8, 9, 10, 11}
GRU	Learning rate = {0.005, 0.01, 0.015}
	Hidden nodes = {50, 60, 70, 80}
	Hidden layer = {2, 3, 4, 5}
	Regularization parameter = {0.00005, 0.0001, 0.00015, 0.0002}
GBRT	Estimators = {100, 200, 300, 400, 500}
	Learning rate = {0.05, 0.1, 0.15, 0.2}
	Min samples leaf = {5, 10, 15}
	Min samples split = {5, 10, 15, 20}
	Max depth = {5, 10, 15, 20}
NGB	Learning rate = {0.005, 0.01, 0.015}
	Minibatch frac = {0.3, 0.4, 0.5}
Ridge	Alphas = {0.01, 0.1, 1}

. The order of importance of the RF model variables is illustrated in Figure 4.

It can be observed from Figure 4 that the importance of global horizontal radiation ranks first among all independent variables, which constitutes for more than 50% of variable importance, followed by t-15 and t-30.

To ensure a faster convergence rate and learning effect, we standardize the data before establishing GRU model. As a mainstream optimization algorithm, Adam [46] is opted as our optimizer. The number of iterations is set to 1000, learning rate to 0.01, hidden layer to 4, hidden node to 60, and the regularization parameter to 0.0001.

In the NGB model, we choose the Gaussian distribution, and the learning rate is set to 0.01, the number of iterations is 532, and the percent subsample of rows to use in each boosting iteration is 0.4. The importance ranking of location parameter variables is depicted in the left panel of Figure 5. Like the RF results, the importance of global horizontal radiation ranks is first followed by t-15, and the importance of month_cos is higher than month_sin. However, the third one is slightly different, which is the diffuse horizontal radiation.

In the GBRT-Mean method, we set the maximum depth of the tree to 5, the number of boosting stages to perform is 400, the minimum number of samples required to split an internal node is 10, the minimum number of samples required to be at a leaf node is 15, and the learning rate is 0.05. For the median method, GBRT-Med, we set the maximum depth of the tree to 15, the number of boosting stages to perform is 400, the minimum number of samples required to split an internal node is 15, the minimum number of samples required to be at a leaf node is 10, and the learning rate is 0.15.

To measure the closeness between the predicted and real values, the relevant evaluation indexes of the prediction results of the six models mentioned earlier are calculated to evaluate the prediction performance of the models, which are listed in

Table 2. Performance of six point prediction methods.

Evaluating Indicator	Model
	GRU	NGB	RF	Ridge	GBRT-Mean	GBRT-Med
MAE	0.054	0.066	0.045	0.127	0.052	0.050
RMSE	0.081	0.099	0.079	0.180	0.081	0.080
${\mathrm{R}}^2$ (%)	99.75	99.62	99.76	98.74	99.74	99.75
MAPE	0.788	0.237	0.048	1.120	0.104	0.055
SMAPE	0.086	0.086	0.024	0.154	0.051	0.028

. The best results under each evaluation index are displayed in bold according to the description in Section 3.2.1.

Table 2 reveals that compared to other models, MAE, RMSE, MAPE, and SMAPE calculated from the prediction results of RF model are the minimum and ${\mathrm{R}}^2$ is the maximum, which indicates that the RF model has the best performance. However, the difference with RF in most of the indicators in GBRT-Mean and GBRT-Med are negligible, indicating that their performances are also similar, followed by GRU and NGB. The worst is Ridge regression, in which the accuracy is far from that of other methods.

4.2. Results of Interval Prediction

In this study, we have compared the performance and prediction interval quality of twelve relatively new interval prediction methods proposed recently. Then, we have used six ensemble methods for a combination of prediction intervals of the above-mentioned partial methods to obtain better prediction intervals.

Specifically, the following methods are chosen: KDE, including GRU-KDE, RF-KDE, Ridge-KDE, GBRT-Mean-KDE, and GBRT-Med-KDE based on their point prediction residuals, J+ab, including J+ab-Ridge, J+ab-MLP, and J+ab-RF, RF-OOB, QRF, and SC-RF based on random forest, and NGB.

For the KDE method, we train the point prediction models, including GRU, RF, Ridge, GBRT-Mean, and GBRT-Med (see Section 4.1 for the training process and hyperparameter adjustment). Then, the error between the predicted and actual values of the five models is calculated. Next, we estimate the kernel density bandwidth of the residual by cross validation, of which the values are from 0.005 to 0.15. The values obtained for the five models are 0.040, 0.016, 0.072, 0.034, and 0.026, respectively. Then, we calculate the cumulative distribution function and corresponding quantiles. When the confidence levels are 95%, 90%, 85%, and 80%, respectively, the upper and lower quantiles corresponding to the five methods are demonstrated in

Table 3. Upper and lower quantiles of five KDE methods with different confidence levels.

Method	Confidence Level
	95%	90%	85%	80%
GRU-KDE	(−0.176, 0.177)	(−0.131, 0.135)	(−0.108, 0.112)	(−0.094, 0.099)
RF-KDE	(−0.078, 0.075)	(−0.051, 0.052)	(−0.039, 0.040)	(−0.032, 0.035)
Ridge-KDE	(−0.440, 0.436)	(−0.355, 0.335)	(−0.312, 0.293)	(−0.280, 0.261)
GBRT-Mean-KDE	(−0.159, 0.157)	(−0.115, 0.117)	(−0.095, 0.097)	(−0.079, 0.081)
GBRT-Med-KDE	(−0.087, 0.095)	(−0.057, 0.064)	(−0.047, 0.049)	(−0.042, 0.044)

.

For the J+ab method, the multi-layer perceptron in J+ab-MLP chooses the Adam optimizer, maximum number of iterations is set to 8000, activation function selected is tanh, the sizes of the three hidden layers are 50, 40 and 30 in order, and the regularization parameter is 0.001. The parameter settings of J+ab-Ridge and J+ab-RF base learners are the same as those in Section 4.1.

For the NGB method, the specific modeling process and parameters are described in Section 4.1. For the interval prediction in NGB, its scale parameters are more important. The order of importance of the variables affecting scale parameters is depicted in the right panel of Figure 5. It can be observed that the three most important factors affecting the scale parameters are the global horizontal radiation, t-15, and diffuse horizontal radiation.

For the prediction intervals established by the above method, we draw the resulting graph of active power from 1 September 2015 to 4 September 2015, as illustrated in Figure 6. In addition, the corresponding weather conditions including wind speed, temperature (Celsius), and relative humidity are depicted in Figure 7. It can be observed from Figure 6 that, in general, the variation trend of the prediction intervals obtained by the twelve methods is similar to that of the actual value, and they include most of the actual values. Among them, the prediction interval obtained by J+ab-Ridge, J+ab-MLP, and J+ab-RF is relatively wide, followed by Ridge-KDE, while other methods are relatively narrow.

In addition, we calculate PICP, PINAW, Winkler score, CWC, and MPICD from the prediction interval of each method and the results are depicted in Figure 8. The specific values at the 95% confidence level are listed in

Table 4. Results of five interval quality indicators of prediction interval of twelve methods (95% confidence level).

Method	Evaluating Indicator
	PICP	PINAW	Winkler Score	CWC	MPICD
J+ab-Ridge	0.975	0.253	−2.301	0.253	0.223
J+ab-MLP	0.981	0.367	−3.316	0.367	0.350
J+ab-RF	0.953	0.402	−3.632	0.402	0.479
RF-OOB	0.968	0.108	−0.983	0.108	0.048
SC-RF	0.965	0.115	−1.046	0.115	0.052
QRF	0.944	0.106	−0.952	0.227	0.074
NGB	0.961	0.078	−0.714	0.078	0.066
GRU-KDE	0.964	0.075	−0.684	0.075	0.054
RF-KDE	0.838	0.032	−0.342	0.562	0.045
Ridge-KDE	0.974	0.185	−1.684	0.185	0.127
GBRT-Mean-KDE	0.951	0.067	−0.617	0.067	0.052
GBRT-Med-KDE	0.837	0.038	−0.391	0.687	0.051

(see

Table A1. Results of five interval quality indexes of prediction interval of twelve methods (90% confidence level).

Method	Evaluating Indicator
	PICP	PINAW	Winkler Score	CWC	MPICD
J+ab-Ridge	0.963	0.179	−1.563	0.179	0.178
J+ab-MLP	0.954	0.307	−2.661	0.307	0.317
J+ab-RF	0.900	0.352	−3.063	0.709	0.530
RF-OOB	0.927	0.065	−0.590	0.065	0.048
SC-RF	0.927	0.069	−0.630	0.069	0.052
QRF	0.924	0.077	−0.665	0.077	0.060
NGB	0.931	0.066	−0.577	0.066	0.066
GRU-KDE	0.925	0.056	−0.501	0.056	0.055
RF-KDE	0.736	0.022	−0.257	1.345	0.045
Ridge-KDE	0.954	0.146	−1.275	0.146	0.126
GBRT-Mean-KDE	0.891	0.049	−0.447	0.110	0.052
GBRT-Med-KDE	0.723	0.026	−0.289	2.159	0.051

,

Table A2. Results of five interval quality indexes of prediction interval of twelve methods (85% confidence level).

Method	Evaluating Indicator
	PICP	PINAW	Winkler Score	CWC	MPICD
J+ab-Ridge	0.945	0.164	−1.372	0.164	0.185
J+ab-MLP	0.834	0.284	−2.4	0.709	0.408
J+ab-RF	0.823	0.31	−2.581	0.918	0.499
RF-OOB	0.892	0.045	−0.413	0.045	0.048
SC-RF	0.893	0.049	−0.454	0.049	0.052
QRF	0.901	0.063	−0.514	0.063	0.056
NGB	0.899	0.057	−0.487	0.057	0.066
GRU-KDE	0.881	0.046	−0.405	0.046	0.055
RF-KDE	0.652	0.017	−0.221	2.371	0.045
Ridge-KDE	0.935	0.128	−1.071	0.128	0.126
GBRT-Mean-KDE	0.84	0.041	−0.366	0.093	0.052
GBRT-Med-KDE	0.641	0.02	−0.249	3.775	0.05

and

Table A3. Results of five interval quality indexes of prediction interval of twelve methods (80% confidence level).

Method	Evaluating Indicator
	PICP	PINAW	Winkler Score	CWC	MPICD
J+ab-Ridge	0.905	0.148	−1.203	0.148	0.206
J+ab-MLP	0.834	0.237	−1.912	0.237	0.34
J+ab-RF	0.808	0.275	−2.176	0.275	0.474
RF-OOB	0.842	0.033	−0.316	0.033	0.048
SC-RF	0.841	0.035	−0.341	0.035	0.052
QRF	0.876	0.053	−0.415	0.053	0.053
NGB	0.869	0.051	−0.42	0.051	0.066
GRU-KDE	0.845	0.041	−0.347	0.041	0.055
RF-KDE	0.588	0.014	−0.202	2.814	0.045
Ridge-KDE	0.906	0.114	−0.918	0.114	0.126
GBRT-Mean-KDE	0.786	0.034	−0.308	0.082	0.052
GBRT-Med-KDE	0.604	0.018	−0.231	2.488	0.05

for the results at other confidence levels).

According to the results in Figure 8, the models with PICP close to the nominal level are J+ab-Ridge, RF-OOB, SC-RF, NGB, GRU-KDE, and Ridge-KDE for different indexes under each confidence level. The models with narrow PINAW and high Winkler score are RF-OOB, SC-RF, QRF, NGB, GRU-KDE, RF-KDE, GBRT-Mean-KDE and GBRT-Med-KDE, which are less than 0.12 and greater than −1.1, respectively. The models with smaller CWC are RF-OOB, SC-RF, QRF, NGB, GRU-KDE, Ridge-KDE, and GBRT-Mean-KDE, all of which are less than 0.23. The models with smaller MPICD are RF-OOB, SC-RF, QRF, NGB, GRU-KDE, RF-KDE, Ridge-KDE, GBRT-Mean-KDE, and GBRT-Med-KDE, all of which are less than 0.13.

More specifically, for the performance of each method, although the PICP of the three J+ab methods are basically close to the confidence level, PINAW is too large to be accurate enough and other indexes are not ideal as well. The PICP of QRF and GBRT-Mean-KDE sometimes cannot reach the confidence level. However, the value is relatively close with other indexes, which are relatively ideal. The PICP of RF-KDE and GBRT-Med-KDE are far from the given confidence level. Compared with other methods, the CWC of RF-KDE and GBRT-Med-KDE are larger, while other indexes are ideal. The PICP of Ridge-KDE is achieved, but the PINAW is slightly higher resulting in a relatively wide interval. In contrast, NGB, GRU-KDE, RF-OOB, and SC-RF perform well with all indexes.

In addition, we also compare the total computational time of each method for obtaining the prediction interval under four confidence levels, as depicted in Figure 9. The training was completed by a personal computer with AMD R7-5800h CPU, 3.20 GHz processor and 16 GB memory. It can be observed from Figure 9 that the methods with the shortest time are RF-OOB, NGB, and SC-RF, which are all less than 40 s, followed by QRF, J+ab-Ridge, J+ab-MLP, and RF-KDE, all of which are less than 200 s. Meanwhile, the most time-consuming methods are J+ab-RF, GBRT-Mean-KDE, GBRT-Med-KDE, GRU-KDE, and Ridge-KDE, all of which have taken more than 200 s. In general, the KDE method takes a longer time than other methods, probably due to the time-consuming cross validation when selecting the bandwidth.

To ensure that the results are more stable, we have repeated the tests ten times. As illustrated in Figure 10, the results obtained by J+ab method are unstable, with values fluctuating by more than 0.1 for PICP and 0.05 for PINAW, while those obtained by other methods are overlapped and almost close to a straight line, indicating having more stable results, which are not shown in the graph for aesthetic reasons. Therefore, we only demonstrated the test corresponding to the fifth PICP of each method in the ten tests when the confidence level was 95% in the previous results.

Based on the above results, the J+ab method will not be considered in the further ensemble interval with the above-mentioned methods because it is already an ensemble method, performs poorly in various indicators, and has a number of outliers in the prediction interval.

As for the ensemble part, we have implemented six methods, including Ensemble-Mean, Ensemble-Med, Ensemble-En, Ensemble-TE, Ensemble-TI, and Ensemble-PM to combine the prediction intervals obtained by nine methods, namely RF-OOB, SC-RF, QRF, NGB, GRU-KDE, RF-KDE, Ridge-KDE, GBRT-Mean-KDE, and GBRT-Med-KDE to obtain the ensemble prediction intervals. Then, we compare the results from the ensemble methods with those of the previously implemented models with best performance, such as RF-OOB, SC-RF, NGB, and GRU-KDE, and calculate the five indexes, namely PICP, PINAW, Winkler score, CWC, and MPICD. The results are reflected in Figure 11, in which the method with the best performance on the basis of four indexes except PICP is marked in black, and the corresponding confidence levels are indicated by dotted lines. Moreover, we have listed the index comparison results under 90% confidence level in

Table 5. Results of five interval quality indicators measured by four interval prediction methods and six ensemble methods (90% confidence level, 15-min interval).

Method	Evaluating Indicator
	PICP	PINAW	Winkler Score	CWC	MPICD
RF-OOB	0.927	0.065	−0.590	0.065	0.048
SC-RF	0.927	0.069	−0.630	0.069	0.052
NGB	0.931	0.066	−0.577	0.066	0.066
GRU-KDE	0.925	0.056	−0.501	0.056	0.055
Ensemble-Med	0.944	0.057	−0.501	0.057	0.047
Ensemble-Mean	0.961	0.064	−0.559	0.064	0.049
Ensemble-En	0.996	0.165	−1.409	0.165	0.109
Ensemble-TE	0.920	0.045	−0.413	0.045	0.045
Ensemble-TI	0.978	0.077	−0.668	0.077	0.051
Ensemble-PM	0.980	0.082	−0.704	0.082	0.061

, in which the method with the best performance based on the four indexes except PICP is marked in bold (see Tables

Table A4. Results of five interval quality indexes measured by four interval prediction methods and six ensemble methods (95% confidence level, 15-min interval).

Method	Evaluating Indicator
	PICP	PINAW	Winkler Score	CWC	MPICD
RF-OOB	0.968	0.108	−0.983	0.108	0.048
SC-RF	0.965	0.115	−1.046	0.115	0.052
NGB	0.961	0.078	−0.714	0.078	0.066
GRU-KDE	0.964	0.075	−0.684	0.075	0.054
Ensemble-Med	0.976	0.079	−0.719	0.079	0.052
Ensemble-Mean	0.983	0.089	−0.809	0.089	0.049
Ensemble-En	0.998	0.209	−1.880	0.209	0.112
Ensemble-TE	0.960	0.066	−0.608	0.066	0.045
Ensemble-TI	0.992	0.107	−0.962	0.107	0.052
Ensemble-PM	0.993	0.125	−1.123	0.125	0.072

,

Table A5. Results of five interval quality indexes measured by four interval prediction methods and six ensemble methods (85% confidence level, 15-min interval).

Method	Evaluating Indicator
	PICP	PINAW	Winkler Score	CWC	MPICD
RF-OOB	0.892	0.045	−0.413	0.045	0.048
SC-RF	0.893	0.049	−0.454	0.049	0.052
NGB	0.899	0.057	−0.487	0.057	0.066
GRU-KDE	0.881	0.046	−0.405	0.046	0.055
Ensemble-Med	0.917	0.045	−0.383	0.045	0.045
Ensemble-Mean	0.941	0.052	−0.437	0.052	0.048
Ensemble-En	0.995	0.145	−1.17	0.145	0.108
Ensemble-TE	0.873	0.035	−0.319	0.035	0.044
Ensemble-TI	0.965	0.064	−0.523	0.064	0.051
Ensemble-PM	0.963	0.062	−0.508	0.062	0.055

and

Table A6. Results of five interval quality indexes measured by four interval prediction methods and six ensemble methods (80% confidence level, 15-min interval).

Method	Evaluating Indicator
	PICP	PINAW	Winkler Score	CWC	MPICD
RF-OOB	0.842	0.033	−0.316	0.033	0.048
SC-RF	0.841	0.035	−0.341	0.035	0.052
NGB	0.869	0.051	−0.42	0.051	0.066
GRU-KDE	0.845	0.041	−0.347	0.041	0.055
Ensemble-Med	0.883	0.036	−0.303	0.036	0.044
Ensemble-Mean	0.91	0.044	−0.358	0.044	0.048
Ensemble-En	0.992	0.13	−0.99	0.13	0.106
Ensemble-TE	0.824	0.028	−0.261	0.028	0.044
Ensemble-TI	0.95	0.054	−0.426	0.054	0.05
Ensemble-PM	0.937	0.049	−0.389	0.049	0.052

for the results at other confidence levels). In addition, we have also compared the data with an interval of 5 min according to the same method. The corresponding results are illustrated in Figure 12 and Table 5 (see

Table A7. Results of five interval quality indexes measured by four interval prediction methods and six ensemble methods (95% confidence level, 5-min interval).

Method	Evaluating Indicator
	PICP	PINAW	Winkler Score	CWC	MPICD
RF-OOB	0.968	0.103	−0.947	0.103	0.041
SC-RF	0.966	0.113	−1.044	0.113	0.044
NGB	0.97	0.074	−0.684	0.074	0.061
GRU-KDE	0.966	0.076	−0.704	0.076	0.049
Ensemble-Med	0.97	0.074	−0.684	0.074	0.047
Ensemble-Mean	0.978	0.085	−0.775	0.085	0.043
Ensemble-En	0.997	0.195	−1.755	0.195	0.102
Ensemble-TE	0.957	0.063	−0.591	0.063	0.041
Ensemble-TI	0.985	0.1	−0.912	0.1	0.046
Ensemble-PM	0.99	0.117	−1.056	0.117	0.064

,

Table A8. Results of five interval quality indexes measured by four interval prediction methods and six ensemble methods (85% confidence level, 5-min interval).

Method	Evaluating Indicator
	PICP	PINAW	Winkler Score	CWC	MPICD
RF-OOB	0.893	0.038	−0.365	0.038	0.041
SC-RF	0.892	0.04	−0.387	0.04	0.043
NGB	0.927	0.054	−0.467	0.054	0.061
GRU-KDE	0.894	0.042	−0.376	0.042	0.049
Ensemble-Med	0.91	0.035	−0.322	0.035	0.04
Ensemble-Mean	0.936	0.043	−0.377	0.043	0.042
Ensemble-En	0.991	0.123	−0.998	0.123	0.096
Ensemble-TE	0.884	0.028	−0.277	0.028	0.039
Ensemble-TI	0.956	0.052	−0.445	0.052	0.044
Ensemble-PM	0.956	0.052	−0.437	0.052	0.049

and

Table A9. Results of five interval quality indexes measured by four interval prediction methods and six ensemble methods (80% confidence level, 5-min interval).

Method	Evaluating Indicator
	PICP	PINAW	Winkler Score	CWC	MPICD
RF-OOB	0.85	0.026	−0.27	0.026	0.041
SC-RF	0.851	0.028	−0.288	0.028	0.043
NGB	0.901	0.048	−0.402	0.048	0.061
GRU-KDE	0.842	0.034	−0.308	0.034	0.049
Ensemble-Med	0.877	0.027	−0.251	0.027	0.039
Ensemble-Mean	0.914	0.035	−0.305	0.035	0.042
Ensemble-En	0.987	0.111	−0.847	0.111	0.094
Ensemble-TE	0.838	0.021	−0.222	0.021	0.039
Ensemble-TI	0.942	0.044	−0.359	0.044	0.044
Ensemble-PM	0.934	0.039	−0.328	0.039	0.046

for the results at other confidence levels).

It can be observed from Figure 11 and Figure 12, Table 5 and

Table 6. Results of five interval quality indicators measured by four interval prediction methods and six ensemble methods (90% confidence level, 5-min interval).

Method	Evaluating Indicator
	PICP	PINAW	Winkler Score	CWC	MPICD
RF-OOB	0.932	0.059	−0.544	0.059	0.041
SC-RF	0.933	0.066	−0.606	0.066	0.044
NGB	0.948	0.062	−0.553	0.062	0.061
GRU-KDE	0.935	0.052	−0.474	0.052	0.049
Ensemble-Med	0.942	0.048	−0.445	0.048	0.043
Ensemble-Mean	0.955	0.056	−0.502	0.056	0.043
Ensemble-En	0.994	0.143	−1.222	0.143	0.096
Ensemble-TE	0.924	0.039	−0.379	0.039	0.039
Ensemble-TI	0.971	0.067	−0.592	0.067	0.045
Ensemble-PM	0.973	0.072	−0.629	0.072	0.054

that the PICP of all ensemble methods reaches the confidence level at both the time intervals of fifteen and five minutes. It is demonstrated that Ensemble-TE is the optimal method at all confidence levels with the value of PINAW and CWC being significantly smaller than that of other methods, and the value of Winkler score is higher than others. Meanwhile, in most cases, Ensemble-TE method has the smallest MPICD. It only reflects that the optimal method with MPICD is Ensemble-mean at 80% confidence level with an interval of fifteen minutes. Similarly, the optimal method with MPICD is RF-OOB at 95% confidence level with an interval of five minutes. However, in these two cases, it can be found that a slight difference exists between the results of Ensemble-TE method and that of the optimal method within ${10}^{-3}$. Therefore, it is considered that Ensemble-TE is the best interval prediction method among all the other methods.

Finally, the numerical results of five interval quality indicators at different confidence levels are described in

Table 7. Results of five interval quality indicators of Ensemble-TE at different confidence levels (15-min interval).

Confidence Level	PICP	PINAW	Winkler Score	CWC	MPICD
0.95	0.960	0.066	−0.608	0.066	0.045
0.90	0.920	0.045	−0.413	0.045	0.045
0.85	0.873	0.035	−0.319	0.035	0.044
0.80	0.824	0.028	−0.261	0.028	0.044

and

Table 8. Results of five interval quality indicators of Ensemble-TE at different confidence levels (5-min interval).

Confidence Level	PICP	PINAW	Winkler Score	CWC	MPICD
0.95	0.957	0.063	−0.591	0.063	0.041
0.90	0.924	0.039	−0.379	0.039	0.039
0.85	0.884	0.028	−0.277	0.028	0.039
0.80	0.838	0.021	−0.222	0.021	0.039

, respectively.

In addition, the indexes calculated by the interval prediction results in this paper also perform better than those in other literatures. To be more specific, when using the A-GRU-KDE [15] method on the same data set with a fifteen-min and a five-min interval, the PINAW of the prediction interval obtained are 0.258 and 0.195, respectively, at a confidence level of 95%. In the same case, however, the results of PINAW in our method are 0.066 and 0.063, respectively, indicating that the relative average width of the interval is reduced by nearly three quarters with guaranteed coverage. Meanwhile, the Winkler scores in previously implemented methods are −2.39 and −1.88, respectively, at a confidence level of 95%. In contrast, the Winkler scores in our method are −0.608 and −0.591, and increased by about 70%, which indicates a high-quality prediction interval. Similar conclusions can be obtained at other confident levels, including 90%, 85%, and 80%. Thus, the proposed method exhibits increased accuracy and higher quality prediction interval on the same DKASC data set.

5. Conclusions

In this study, we implement six methods to obtain point prediction for solar photovoltaic power generation, namely Ridge regression, RF, GRU, NGB, GBRT-Mean, and GBRT-Med. On comparing the prediction results of these methods, we found that the results of RF method are the best under five point prediction evaluation indexes, including MAE, RMSE, ${\mathrm{R}}^2$, MAPE, and SMAPE.

We also compare twelve interval prediction methods in this paper, namely J+ab-Ridge, J+ab-MLP, J+ab-RF, RF-OOB, SC-RF, QRF, NGB, GRU-KDE, RF-KDE, Ridge-KDE, GBRT-Mean-KDE, and GBRT-Med-KDE. For the performance of each method, although the PICP of the three J+ab methods are basically close to the confidence level, PINAW is too large to be accurate enough and other indicators are not ideal as well. The PICP of QRF and GBRT-Mean-KDE cannot reach the confidence level in some cases, but they are relatively close, and the other indicators are relatively ideal. The PICP of RF-KDE and GBRT-Med-KDE are far from the given confidence level. In addition, the CWC of RF-KDE and GBRT-Med-KDE are larger, while other indicators are ideal. The PICP of Ridge-KDE is achieved; however, the PINAW of it is slightly higher, which leads to a relatively wide interval. In contrast, NGB, GRU-KDE, RF-OOB, and SC-RF perform well with all indicators. From the perspective of the computational time of each method, the methods with the shortest time are RF-OOB, NGB, and SC-RF, all of which are less than 40 s, followed by QRF, J+ab-Ridge, J+ab-MLP, and RF-KDE with a time within 200 s. The most time-consuming methods are J+ab-RF, GBRT-Mean-KDE, GBRT-Med-KDE, GRU-KDE, and Ridge-KDE, which all took more than 200 s.

Furthermore, we use six methods of ensemble interval prediction, namely Ensemble-Med, Ensemble-Mean, Ensemble-En, Ensemble-TE, Ensemble-TI, and Ensemble-PM to combine the prediction intervals obtained by nine methods, including RF-OOB, SC-RF, QRF, NGB, GRU-KDE, RF-KDE, Ridge-KDE, GBRT-Mean-KDE, and GBRT-Med-KDE, with an aim to obtain the ensemble prediction intervals. Then, we compare the ensemble results with the previously implemented models with the best performance, such as RF-OOB, SC-RF, NGB, and GRU-KDE. Finally, it is found that the prediction interval obtained by Ensemble-TE is basically optimal under five indicators, including PICP, PINAW, Winkler score, CWC, and MPICD, and at four confidence levels, including 95%, 90%, 85%, and 80%. Therefore, we propose an ensemble interval prediction for solar photovoltaic power generation, which is, a combination of the above nine interval prediction methods using Ensemble-TE method. The results demonstrate that, compared to other methods, the proposed method can obtain the interval with an coverage close to the nominal level, higher accuracy, and closer prediction between the interval and actual value, with an interval of fifteen and five minutes, at four confidence levels of 95%, 90%, 85%, and 80%.

In the future, we will focus on applying the ensemble or combination method of prediction for solar photovoltaic power generation power in other regions or countries and try to make long-term predictions, such as 24 h. In addition, we will further utilize other state-of-the-art interval prediction methods for ensemble prediction to obtain more reliable and accurate prediction intervals. Furthermore, we will further explore the impact of weather uncertainty on the model and find methods possessing good prediction performance with insensitivity to weather changes.