*3.3. The Ensemble Strategy of Multiple Models*

The predictive ensemble model (PEM) in this study is an adaptation of the random forest algorithm. PEM integrates series of forecasting models to formulate a forecasting model. The integration process is in two stages: optimal weight estimation and stochastic forecast result generation. The optimal weight estimation method estimates the hyperparameters of the parametric predicting models using particle swarm optimization (PSO) at each given time, as defined in Equation (13).

$$f\_{c,t}(X\_{n,t}) = \sum\_{n=1}^{N} w\_{n,t} \cdot f\_{n,t}(X\_{n,t}) \tag{13}$$

The weights ω are estimated by solving the optimization problem using the heuristic function above and the algorithm in Equation (14), where ωˆ *<sup>T</sup>* and ω*n*,*<sup>t</sup>* is the estimated optimal weight vector for the period *T* and weight parameter of the *n*-th model at the time, *t*, respectively.

$$\phi\_T = \underset{\omega \tau}{\operatorname{argmin}} \sum\_{t \in T} L\_t \Big( \sum\_{n=1}^N \omega\_{n,t} \cdot f\_{n,t}(\mathbf{X}\_{n,t}) \, \_\prime \, \mathcal{Y}\_t \Big) \tag{14}$$

s.t.

$$\sum\_{n=1}^{N} \omega\_{n,t} = 1; \omega\_{n,t} \ge 0 \; \forall \; n \in \{1, \dots, N\} \tag{15}$$

The determination of the weights for each model per each forecast instance is initially chosen at random following the set constraints. The weights are subsequently adjusted dynamically as a result of the bias of each point forecast. With the loss function defined to estimate the error associated with each point forecast, the goal of the optimization process is to minimize the loss function, *Lt*, as described in Equation (16), where *Xn*,*t* and *yt* are the input parameter vector of the *n*-th parametric model and the actual load at time, *t*, respectively.

$$L\_t = \sum\_{n=1}^{N} (\omega\_{n,t} \cdot f\_{n,t}(X\_{n,t}) - y\_t); \forall \ t \in \{1, \dots, \dots, T\} \tag{16}$$

The weight vector with the minimum loss function becomes the optimal weight vector. Subsequently, the optimal weight estimation is the error estimation distribution with the optimal weight vector. The stochastic forecast results are, therefore, estimated based on the error distribution and the optimal weight vector, as described in Algorithm 1.

