*3.3. EGO*

EGO is closely linked with kriging metamodeling. The EGO approach is focused on solving optimization problems in a low number of function evaluations and the approach offers clear stopping criteria based on expected improvement (*EI*). The EI function is produced based on the Kriging model. To ge<sup>t</sup> a new sampling point, the EI function is maximized. Then this new data point is added to the initial set. This process is repeated until the EI function value does not change significantly.

The Kriging model can be simply defined as shown in Equation (4), where **x**(*i*) = *x* (*i*) 1 ,..., *x* (*n*) *k* and **y** (*i*) = *y* **x**(*i*) .

$$y\left(\mathbf{x}^{(i)}\right) = \mu + \epsilon \left(\mathbf{x}^{(i)}\right) \tag{4}$$

In this equation, *μ* is the mean of the stochastic process; **x**(*i*) is normally distributed independent error term with mean zero and variance *σ*2. Correlation between **x**(*i*) and **x**(*j*) could be defined as shown in Equation (5) [81].

$$\text{Corr}\left[\boldsymbol{\varepsilon}\left(\mathbf{x}^{(i)}\right),\,\boldsymbol{\varepsilon}\left(\mathbf{x}^{(j)}\right)\right] = \sum\_{h=1}^{k} \theta\_h \left|\mathbf{x}\_h^i - \mathbf{x}\_h^j\right|^{p\_h}, \,\theta\_h \ge 0, \, p\_h \in \left[1, 2\right], \, i, j = (1, \ldots, n) \tag{5}$$

*θh* is importance measuring for the variable *xh* and *ph* is the smoothness parameter of the correlation function. *μ* and *σ*<sup>2</sup> are unknown. They can be estimated by using the parameters of the correlation function which are *θh* and *ph*. For estimating the parameters, MLE is used. Likelihood function could be written as shown in Equation (6) [81]:

$$L = \frac{1}{(2\pi)^{\frac{\mu}{2}}(\sigma^2)^{\frac{\mu}{2}}|\mathbf{R}|^{\frac{1}{2}}} \exp\left[-\frac{(\mathbf{y} - 1\mu)'\mathbf{R}^{-1}(\mathbf{y} - 1\mu)}{2\sigma^2}\right] \tag{6}$$

where **y** = (*y*(*i*), ... , *y*(*n*)) is the *n*-vector for response values and 1 is a vector of ones. Since *μ* and *σ*<sup>2</sup> are unknown, estimations of *μ* and *σ*<sup>2</sup> could be calculated as shown in Equations (7) and (8).

$$
\hat{\mu} = \frac{\mathbf{1}'\mathbf{R}^{-1}\mathbf{y}}{\mathbf{1}'\mathbf{R}^{-1}\mathbf{1}} \tag{7}
$$

$$\mathfrak{d}^2 = \frac{(\mathbf{y} - 1\boldsymbol{\mu})^\prime \mathbf{R}^{-1} (\mathbf{y} - 1\boldsymbol{\mu})}{n} \tag{8}$$

By changing the Equations (7) and (8) with *μ*ˆ and *σ*ˆ 2 from the likelihood function, "concentrated likelihood function" is created. It depends only on *θh* and *ph*. Denote that **r** gives the correlation between the error terms for **<sup>x</sup>\***, which is not observed previously, and the error for **x**, which is observed previously. The correlation between those two could be written as shown in Equation (9).

$$\mathbf{r}(\mathbf{x}^\*) \equiv \text{Corr}[\boldsymbol{\epsilon}(\mathbf{x}^\*), \; \boldsymbol{\epsilon}(\mathbf{x})].\tag{9}$$

After having all the equations together, the Kriging model can be converted into the form shown in Equation (10).

$$\hat{y}(\mathbf{x}^\*) = \hat{\mu} + \mathbf{r}'\mathbf{R}^{-1}(\mathbf{y} - 1\hat{\mu}) \tag{10}$$

Following the process of creating the Kriging model, *EI* criteria is described as follows. Denote that the function *y* = *f*(*x*), the improvement (*I*) over *fmin*, which is the minimum response value of *f*(*x*). The improvement now can be defined as

$$I = \begin{cases} \ \ (f\_{\min} - y) \ \ y < f\_{\min} \\ \ \ 0, \ otherwise \end{cases} \tag{11}$$

When *y* has normal distribution with *y*ˆ mean and *s*2 variance, expected value of *I* can be calculated by following Equations (12) and (13).

$$E(I) = \int\_{-\infty}^{f\_{\rm min}} (f\_{\rm min} - y) \phi(y) dy \tag{12}$$

Expected Improvement (*EI*) function can be shown as follows:

$$EI = (f\_{min} - \mathfrak{H})\Phi\left(\frac{f\_{min} - \mathfrak{H}}{\mathfrak{s}}\right) + s\Phi\left(\frac{f\_{min} - \mathfrak{H}}{\mathfrak{s}}\right) \tag{13}$$

where Φ () is cumulative distribution function (CDF) and φ is PDF of a standard Normal distribution [81].
