*3.2. Evaluation Metrics*

In order to compare HyGPR with the other benchmark prediction model, we defined four metrics to assess quantitatively its point and interval forecasting ability. There accuracy metrics for the point prediction including root mean square error (RMSE), mean absolute error (MAE), and mean absolute percentage error (MAPE) were formulated in Equations (24)–(26).

$$\text{RMSE} = \sqrt{\frac{\sum\_{i=1}^{N} \left(y\_i - \hat{y}\_i\right)^2}{N}} \tag{24}$$

$$\text{MAE} = \frac{1}{N} \sum\_{i=1}^{N} \left| y\_i - \mathcal{g}\_i \right| \tag{25}$$

$$\text{MAPE} = \frac{1}{N} \sum\_{i=1}^{N} \left| \frac{y\_i - \hat{y}\_i}{y\_i} \right| \times 100\% \tag{26}$$

where *yi* and *y*ˆ*<sup>i</sup>* denote respectively the observed and predicted oxygen consumption in the *i th* test sample; *N* is the size of the test sample. Note that the small values of these metrics indicate high prediction accuracy.

The proposed HyGPR model is able to provide not only the forecasting point *y*ˆ*<sup>i</sup>* but also the confidence interval *y*ˆ− *<sup>i</sup>* , *y*ˆ + *i* of future oxygen consumption. Therefore, we defined a coverage metric for interval prediction named hit ratio (HRI) in Equation (27) which is applied to calculate the number of test samples fallen into the 95% confidence interval.

$$\text{HRI} = \frac{1}{N} \sum\_{i=1}^{N} I\_A(\hat{y}\_i) \times 100\% \tag{27}$$

where *A* = , *y*ˆ*i y*ˆ− *<sup>i</sup>* ≤ *y*ˆ*<sup>i</sup>* ≤ *y*ˆ + *i* - , and *IA*(*y*ˆ*i*) = ! 1, *y*ˆ*<sup>i</sup>* ∈ *A* 0, *<sup>y</sup>*ˆ*<sup>i</sup> <sup>A</sup>* .

Additionally, we also used the CPU running time (seconds) to evaluate the learning speed of the tested models.

#### *3.3. Results and Analysis*

In this study, all proposed and benchmark models are implemented with the MATLAB 2017 software. Especially, we used the GPML (Gaussian processes for machine learning) toolbox [26] to construct the GPR model and the one in HyGPR, and other compared models were provided by the toolboxes installed in MATLAB. All programs ran on a personal computer with an Intel Core i7-8550U Processor (1.8550GHz) and 16.0GB Memory and installing a Windows 10 operating system.

In the proposed HyGPR model, the cluster count (*Q*) of K-means is a very important factor that may influence the final prediction performance. To select the most appropriate value of *Q*, we carried out five group experiments with different clusters. The results of the four accuracy metrics were listed in Table 2 and the forecasting plots were shown in Figure 4.

**Figure 4.** *Cont*.

**Figure 4.** Forecasting results with candidate clusters: (**a**) Q = 1; (**b**) Q = 2; (**c**) Q = 3; (**d**) Q = 4; (**e**) Q = 5.

According to results listed in Table 2, we found that the RMSE, MAE and MAPE of the HyGPR model with different clusters were approximate, but the CPU time was reduced greatly when *Q* > 1. Since the HyGPR with four clusters could run successfully within the shortest time, we set *Q* = 4 in following computational experiments. In addition, we also found that most of the actual data located in the 95% confidence interval was provided by HyGPR, which means the proposed model was able to make a probabilistic sense.


**Table 2.** Experimental results for candidate clusters (The best metrics are highlighted in bold).

In order to further evaluate the prediction accuracy of the HyGPR model, we compared it with three benchmark models including MLR, ANN [27], and the support vector machine (SVM) [28]. The MLR model was created by the function *fitlm* in MATLAB. The function *fitnet* in MATLAB was adopted to construct ANN with three layers and 10 nodes in the hidden layer, in which the network parameters were optimized by the Levenberg–Marquardt method. The function *fitrsvm* in MATLAB was used to construct the SVM model with the radial basis function (RBF) kernel, in which the hyperparameters were automatically optimized by minimizing the five-fold cross-validation loss function.

To quantitatively select the best one from the testing models, the computational results of the point evaluation metrics were listed in Table 3. It can be observed that the proposed HyGPR model obtained the smallest of RMSE, MAE and MAPE, while the proposed MLR got the worst results of RMSE, MAE and MAPE. The above results focused on the prediction accuracy of the single valued point predictions (as shown in Figure 5).

**Table 3.** Performance evaluation of compared models on the test set (The best metrics are highlighted in bold).


**Figure 5.** Point forecasting results of the oxygen consumption.

Usually, theMLR, ANN and SVM can only provide point estimations of future oxygen consumption. However, the HyGPR model can provide not only the forecasting point *y*ˆ*<sup>i</sup>* but also the confidence

interval *y*ˆ− *<sup>i</sup>* , *y*ˆ + *i* of future oxygen consumption. To test its HRI within 95% confidence interval of HyGPR, we compared it with the standard GPR with the *squared exponential* kernel function. Table 4 listed the evaluation results of GPR and HyGPR. The HyGPR obtained the same value of RMSE and slightly better value of MAE, MAPE and HRI, but its computing speed is more than five times as fast as the standard GPR. Figure 6 showed the point forecasts and the corresponding 95% confidence intervals. It can also be observed that nearly all of the actual observations fell in the confidence intervals.

**Table 4.** Accuracy and interval metrics of GPR and HyGPR (The best metrics are highlighted in bold).


**Figure 6.** Interval forecasting results of oxygen consumption: (**a**) HyGPR; (**b**) standard GPR.

#### **4. Conclusions**

With the increased concerns on the management and optimization of energy systems, it is necessary for modern integrated iron and steel works to develop an accurate and robust model to forecast oxygen consumption. However, it is challengeable to directly forecast oxygen consumption with a simple regression model due to its intermittent and uncertain features. In this study, we introduce a novel hybrid model named HyGPR integrating MLR and GPR. In the proposed prediction model, the MLR model is developed to figure the global trend of the oxygen consumption, and the GPR is applied to explore the local fluctuation caused by noise. Additionally, to overcome the shortcoming of GPR on training speed, a K-means clustering method is applied to decompose the training dataset into a number of subsets. The effectiveness of the HyGPR was verified using the actual process data collected from a

large integrated iron and steel works located in the north of China. Afterwards, HyGPR is compared with MLR, ANN, SVM and GPR. The results show that HyGPR can obtain the best point prediction metrics in terms of RMSE, MAE, and MAPE, and the better interval prediction performance in terms of HRI. Furthermore, it runs more than five times faster than the standard GPR. Therefore, it can be concluded that the proposed method is an effective tool to improve the forecasting accuracy and coverage. Moreover, HyGPR runs faster than the standard GPR model due to implementing the decomposition policy.

In future studies, we will investigate the following issues that may be meaningful for industrial application and scientific research:


**Author Contributions:** Methodology and writing, S.-L.J.; Data curation, X.S.; Supervision, Z.Z.

**Funding:** National Natural Science Foundation of China (No. 51734004, 61873042) and Fundamental Research Funds for the Central Universities (No. 2018CDXYCL0018).

**Acknowledgments:** We would like to thank the anonymous reviewers and the editors for their constructive and pertinent comments.

**Conflicts of Interest:** The authors declare no conflict of interest.
