A Non-Linear Trend Function for Kriging with External Drift Using Least Squares Support Vector Regression

Abstract

Spatial interpolation of meteorological data can have immense implications on risk management and climate change planning. Kriging with external drift (KED) is a spatial interpolation variant that uses auxiliary information in the estimation of target variables at unobserved locations. However, traditional KED methods with linear trend functions may not be able to capture the complex and non-linear interdependence between target and auxiliary variables, which can lead to an inaccurate estimation. In this work, a novel KED method using least squares support vector regression (LSSVR) is proposed. This machine learning algorithm is employed to construct trend functions regardless of the type of variable interrelations being considered. To evaluate the efficiency of the proposed method (KED with LSSVR) relative to the traditional method (KED with a linear trend function), a systematic simulation study for estimating the monthly mean temperature and pressure in Thailand in 2017 was conducted. The KED with LSSVR is shown to have superior performance over the KED with the linear trend function.

1. Introduction

Spatial interpolation is a fundamental technique employed in spatial data analysis to estimate the variable of interest at unobserved locations based on available data. Kriging is a geostatistical approach for spatial interpolation that provides the best linear unbiased prediction with the minimum estimation variance. An approximation of the kriging models relies on the assumption that a random process can be decomposed into a trend function and a random residual component. Ordinary kriging (OK) is a commonly used method with a constant trend, which is not suitable in the presence of a strong trend structure. On the other hand, kriging with external drift (KED) allows for the inclusion of auxiliary variables that have a strong spatial correlation with the target variable in order to increase precision in estimates [1,2]. In the KED, a trend model that is fitted to both target data points and significant auxiliary samples is first generated. The empirical variogram is thereafter derived from residuals computed from the difference between the trend estimates and measured values. A final prediction of the target variable is obtained as a weighted linear combination of observations, in which the weights are calculated through the Lagrange multiplier method. The KED method has been applied in various fields, including meteorology [3,4,5,6], geology [7,8,9,10], environmental modeling [11,12,13,14], agronomy [15,16,17], and hydrology [18].

The trend term for the KED is conventionally modeled by polynomial functions of degree one or two. However, in practice, a non-linear relationship often exists between influence factors and response variables where the application of such a polynomial is not adequate. Despite extensive research on the prediction using KED having been conducted, a study of non-linear trend functions for the KED remains scarce. Snepvangers et al. [19] developed a non-linear trend represented by a logarithmic function to interpolate soil water content using the KED technique with net precipitation as an auxiliary variable. Freier and Lieres [20] introduced a novel extension to universal kriging (UK), a specific instance of KED, aimed at handling non-linear trend patterns. They utilized a Taylor-based linearization approach in conjunction with an iterative parameter estimation procedure to construct a non-linear trend model. The method was applied to the Michaelis–Menten equation, which describes an enzymatic reaction. Freier et al. [21] subsequently employed this kriging technique to interpolate biocatalytic data with low and irregular density. Their method can be particularly of use in the presence of an explicit expression of the non-linear trend functions. Nevertheless, an interaction between the design factors and system response in real-world applications is naturally described by diverse and complex behavior, which is difficult to establish in an explicit form.

Machine learning (ML) has recently been gaining attention as a computationally efficient tool for identifying implicit relationships between variables. This allows one to generate and optimize the complex model based on the huge amount of data available for analysis. Support vector machine (SVM) is a kernel-based machine learning approach used for classification and regression. The particular use of SVMs for regression problems is called support vector regression (SVR), which was first introduced by Vapnik in 1992 [22]. The method adopts the structural risk minimization principle by minimizing the upper bound of the generalization errors. This leads to a linear decision function, which is essentially a convex quadratic programming (QP) problem. The core element of SVR is to search for the optimal hyperplane that fits the learning data while maximizing the distance between the hyperplane and the data points. In the case of non-linear problems, the SVR procedure starts by projecting input data into the high-dimensional feature space through some non-linear mapping, and the SVR subsequently performs linear regression to obtain the optimal hyperplane. Apart from producing high prediction accuracy for non-linear data, SVR is also suitable for applications characterized by small datasets [23,24,25]. Furthermore, this SVM regression-based algorithm has the generalization ability to reduce overfitting issues by introducing a regularization term into the loss function. Due to all these advantages, the technique has, therefore, been applied in diverse disciplines, including finance [26,27,28], economics [29,30,31], climate modeling [32,33], and healthcare [34,35]. However, SVR requires substantial computational time and significant memory usage to solve the QP problem. To overcome these limitations, Suykens and Vandewalle [36] proposed a variant of SVR known as least squares support vector regression (LSSVR). This method extends the traditional SVR by using a squared loss function rather than quadratic programming. The LSSVR results yield higher accuracy and require less computational resources compared to the reliability method relying on SVR [37].

There is a notable absence of research on the utilization of ML in geostatistical techniques. In this work, we will present a novel interpolation method in which the LSSVR method is used to compute non-linear trend functions within the context of KED. The proposed technique entails expressing the trend function in a structured form through explicit feature mapping. The purpose of our technique is to enhance the predictive capabilities of the KED model by incorporating the powerful capabilities of LSSVR for capturing non-linear relationships between variables.

The remainder of this paper is outlined as follows. Section 2 reviews the theory regarding the KED methodology and LSSVR technique. A detailed description of the KED using the LSSVR for modeling the non-linear trend functions is provided in Section 3. In Section 4, we conduct a comparative simulation study using the conventional KED model and the proposed method for temperature and pressure estimation in Thailand. Conclusions and discussion are drawn in Section 5.

2. Mathematical Background

2.1. Kriging with External Drift

Kriging is a spatial interpolation method that uses variogram analysis to predict the variable of interest at an unmeasured location based on the values of surrounding measured locations. It is the best linear unbiased estimator (BLUE) for the random function, $\{Z\left(s\right):s\in D\subset {\mathbb{R}}^d\}$, where D is a defined spatial domain and d is a positive integer representing the number of dimensions in the spatial domain. The value of $Z\left(s\right)$ can be obtained through

$$\begin{array}{c}\hfill Z\left(s\right)=\mu \left(s\right)+ϵ\left(s\right),\end{array}$$

(1)

where a deterministic component $\mu \left(s\right)$ indicates the underlying trend or drift and $ϵ\left(s\right)$ is a stochastic residual component with a mean of zero and a variogram, which is a function of lag vector [2].

In KED, the trend is modeled by a function of auxiliary variables which can be expressed as

$$\mu \left(s\right)=\sum _{l=0}^L{a}_l{f}_l\left(X\left(s\right)\right),$$

(2)

where $a_l\in \mathbb{R}\setminus \left\{0\right\}$ is the coefficient to be estimated, $X\left(s\right)={\left[X_1\left(s\right),\dots ,X_{\eta }\left(s\right)\right]}^T$ is the vector of $\eta$ auxiliary variables at location s, $f_l$ is the prescribed function of auxiliary variables, and $L+1$ is the number of terms used in the approximation. Additionally, $f_0$ is defined as the constant function with a value of 1 [2].

To determine the unknown coefficients in Equation (2), we can use the ordinary least squares (OLS) estimator or its extension, the generalized least squares (GLS) estimator, which accounts for the spatial correlation between individual observations [38].

Given n observed values, $Z\left(s_1\right),\dots ,Z\left(s_n\right)$, at sample points, $s_1,s_2,\dots ,s_n$. The attribute $Z\left(s_0\right)$ at an ungauged site $s_0$ is estimated as a linear combination of observed values so that

$$\begin{array}{c}\hfill Z^{*}\left(s_0\right)=\sum _{i=1}^n{\omega }_{i}Z\left(s_i\right),\end{array}$$

(3)

where ${\omega }_i$ is the kriging weight assigned to $Z\left(s_i\right)$. The weights are computed by minimizing the estimation error variance subject to the unbiased constraint. This results in the following optimization problem:

$$\begin{array}{cc}\hfill \mathrm{minimum}\mathrm{of}& \mathrm{Var}\left[Z^{*}\left(s_0\right)-Z\left(s_0\right)\right],\hfill \\ \hfill \mathrm{subject}\mathrm{to}& \mathbb{E}\left[Z^{*}\left(s_0\right)-Z\left(s_0\right)\right]=0.\hfill \end{array}$$

(4)

The optimal weights of the system (4) for the KED model can be solved by using the Lagrange multiplier method, which leads to the KED system

$$\begin{array}{c}\hfill \left\{\begin{array}{cc}\sum _{j=1}^n{\omega }_j{\gamma }_{ϵ}\left(s_i-s_j\right)+\sum _{l=0}^L{\lambda }_l{f}_l\left(X\left(s_i\right)\right)={\gamma }_{ϵ}\left(s_i-s_0\right),& \forall i=1,\dots ,n,\\ \sum _{j=1}^n{\omega }_j{f}_l\left(X\left(s_i\right)\right)=f_l\left(X\left(s_0\right)\right),& \forall l=0,1,\dots ,L,\end{array}\right.\end{array}$$

(5)

where ${\gamma }_{ϵ}(·)$ denotes the residual variogram function of $Z\left(s\right)$ and ${\lambda }_l\in \mathbb{R}$ is a Lagrange multiplier.

The variogram is a fundamental and important tool that quantifies the spatial correlation structure of the sample points. The variogram model is a smooth function that is reasonably well-fitted to the empirical variogram estimated from the data. In the present study, we use the empirical variogram estimator introduced by Matheron [39], and the parametric variogram is represented by an exponential model [38].

In general, both linear and quadratic functions are usually treated as a trend representation [9,18,40,41]. However, in certain scenarios, the relationship between the target and auxiliary variables is too complex to be captured by simple polynomial functions. In this work, least squares support vector regression (LSSVR) is used to model a non-linear trend function within the KED framework.

2.2. Least Squares Support Vector Regression

Given a dataset ${\{Y_i,Z_i\}}_{i=1}^n$, where $Y_i\in {\mathbb{R}}^{\eta }$ is a $\eta -$dimensional training data point and $Z_i\in \mathbb{R}$ represents a target output. The objective of least squares support vector regression (LSSVR) is to find a function that minimizes the square error between the predicted values and the actual values. In LSSVR, the input data $Y_i$ are mapped into a higher-dimensional feature space ${\mathbb{R}}^{{\eta }_h}$, in which a linear model is adopted, so that a model function $\mu$ is formulated as

$$\begin{array}{c}\hfill \mu \left(Y\right)={\mathbf{a}}^T\varphi \left(Y\right)+b,\end{array}$$

(6)

where $\varphi \left(Y\right)$ is an ${\eta }_h-$dimensional feature mapping, $\mathbf{a}$ is an ${\eta }_h\times 1$ weight vector, and $b\in \mathbb{R}$ indicates a bias term.

In Equation (6), the unknown vector $\mathbf{a}$ and parameter b can be calculated by solving the following optimization problem:

$$\begin{array}{cc}\hfill \mathrm{minimum}\mathrm{of}& \frac{1}{2}{\mathbf{a}}^T\mathbf{a}+\frac{\nu }{2}\sum _{i=1}^n{\zeta }_i^2,\hfill \\ \hfill \mathrm{subject}\mathrm{to}& Z_i={\mathbf{a}}^T\varphi \left(Y_i\right)+b+{\zeta }_i,i=1,\dots ,n,\hfill \end{array}$$

(7)

where $\nu$ is a regularization constant that constitutes a trade-off between the model complexity and the empirical error, and ${\zeta }_i$ is a regression error.

The problem (7) can be reformulated as an unconstrained problem through the Lagrange multiplier method [42]. A set of linear equations corresponding to optimality conditions is consequently obtained, which provides an expression of the weight vector $\mathbf{a}$:

$$\begin{array}{c}\hfill \mathbf{a}=\sum _{i=1}^n{\alpha }_i\varphi \left(Y_i\right),\end{array}$$

(8)

where ${\alpha }_i$ is a Lagrange multiplier. This system of equations can be reduced to the following form:

$$\begin{array}{c}\hfill \left[\begin{array}{cc}0& {\mathbf{1}}_{\mathit{n}}^{\mathit{T}}\\ {\mathbf{1}}_{\mathit{n}}& \mathbf{\Omega }+{\nu }^{-1}{\mathit{I}}_{\mathit{n}}\end{array}\right]\left[\begin{array}{c}b\\ \mathit{\alpha }\end{array}\right]=\left[\begin{array}{c}0\\ \mathit{Z}\end{array}\right],\end{array}$$

(9)

where $\mathbf{\Omega }$ is referred to as the kernel matrix whose element is ${\varphi }^T\left(Y_i\right)\varphi \left(Y_j\right)$ for $i,j=1,\dots ,n$ and ${\mathit{I}}_{\mathit{n}}$ is the identity matrix of size n. The matrix ${\mathbf{1}}_{\mathit{n}}$ is an $n\times 1$ unit matrix and $\mathit{Z}={\left[Z_1,\dots ,Z_n\right]}^T$ is the $n\times 1$ matrix of observed values together with the $n\times 1$ matrix of Lagrange multipliers, $\mathit{\alpha }={[{\alpha }_1,\dots ,{\alpha }_n]}^T$.

The solutions of Equation (9) are

$$\begin{array}{cc}\hfill b& =\frac{{\mathbf{1}}_{\mathit{n}}^{\mathit{T}}{\mathbf{A}}^{-1}\mathit{Z}}{{\mathbf{1}}_{\mathit{n}}^{\mathit{T}}{\mathbf{A}}^{-1}{\mathbf{1}}_{\mathit{n}}},\hfill \end{array}$$

(10)

$$\begin{array}{cc}\hfill \mathit{\alpha }& ={\mathbf{A}}^{-1}(\mathit{Z}-b{\mathbf{1}}_{\mathit{n}}),\hfill \end{array}$$

(11)

where $\mathbf{A}=\mathbf{\Omega }+{\nu }^{-1}{\mathit{I}}_{\mathit{n}}$, which is a symmetric and positive semi-definite matrix, thereby ascertaining the existence of its inverse, denoted as ${\mathbf{A}}^{-1}$.

By replacing Equation (8) with Equation (6), the model for the LSSVR function, therefore, becomes

$$\begin{array}{cc}\hfill \mu \left(Y\right)& =\sum _{i=1}^n{\alpha }_i{\varphi }^T\left(Y_i\right)\varphi \left(Y\right)+b,\hfill \\ \hfill & =\sum _{i=1}^n{\alpha }_{i}K\left(Y_i,Y\right)+b,\hfill \end{array}$$

(12)

where $K(,)$ is the kernel associated with the feature mapping $\varphi$, and it is defined as [43]

$$\begin{array}{c}\hfill K\left(Y_i,Y\right)={\varphi }^T\left(Y_i\right)\varphi \left(Y\right).\end{array}$$

(13)

Numerous kernel functions are available for the construction of various models, such as:

Linear kernel: $K\left(Y_i,Y\right)=Y_i^{T}Y$.

Polynomial kernel: $K\left(Y_i,Y\right)={(k+Y_i^{T}Y)}^p$, $k>0$ and $p\in \mathbb{N}$.

Radial basis function kernel: $K\left(Y_i,Y\right)=exp(-g\parallel Y-Y_i{\parallel }^2)$, $g>0$ and $\parallel ·\parallel$ means the Euclidean norm.

The LSSVR regarding the KED scheme is used to characterize the underlying trend. The notation for the dataset at the sample point $s_i\in {\mathbb{R}}^d$ is represented by ${\left\{X\left(s_i\right),Z\left(s_i\right)\right\}}_{i=1}^n$, where $X\left(s_i\right)={\left[X_1\left(s_i\right),\dots ,X_{\eta }\left(s_i\right)\right]}^T\in {\mathbb{R}}^{\eta }$ is the vector of $\eta$ auxiliary variables and $Z\left(s_i\right)$ denotes the observation value.

3. A Novel Trend Function of KED Based on LSSVR

This section introduces a method for constructing the trend function in KED using the LSSVR method. The approach involves identifying the fundamental functions of the trend through explicit feature mapping. Examples of explicit feature mappings derived from the corresponding kernel functions are also demonstrated.

3.1. Construction of the Trend Function

Let $\varphi \left(X\right(s\left)\right)$ be an $M-$dimensional feature mapping, such that

$$\begin{array}{c}\hfill \varphi \left(X\left(s\right)\right)={\left[{\varphi }_1\left(X\left(s\right)\right),\dots ,{\varphi }_M\left(X\left(s\right)\right)\right]}^T,\end{array}$$

(14)

where ${\varphi }_m\left(X\left(s\right)\right)$ is the mth component of the feature mapping.

According to Equation (13), the kernel function is in the following form:

$$\begin{array}{c}\hfill K\left(X\left(s_i\right),X\left(s\right)\right)=\left[{\varphi }_1\left(X\left(s_i\right)\right),\dots ,{\varphi }_M\left(X\left(s_i\right)\right)\right]\left[\begin{array}{c}{\varphi }_1\left(X\left(s\right)\right)\\ ⋮\\ {\varphi }_M\left(X\left(s\right)\right)\end{array}\right].\end{array}$$

(15)

By substituting Equation (15) into Equation (12), $\mu \left(s\right)$ can be rewritten as

$$\begin{array}{cc}\hfill \mu \left(s\right)=\mu \left(X\right(s\left)\right)& =\sum _{i=1}^n{\alpha }_i\left(\left[{\varphi }_1\left(X\left(s_i\right)\right),\dots ,{\varphi }_M\left(X\left(s_i\right)\right)\right]\left[\begin{array}{c}{\varphi }_1\left(X\left(s\right)\right)\\ ⋮\\ {\varphi }_M\left(X\left(s\right)\right)\end{array}\right]\right)+b,\hfill \\ \hfill & =\sum _{i=1}^n{\alpha }_i\left(\sum _{l=1}^M{\varphi }_l\left(X\left(s_i\right)\right){\varphi }_l\left(X\left(s\right)\right)\right)+b,\hfill \\ \hfill & =\sum _{l=1}^M\sum _{i=1}^n{\alpha }_i{\varphi }_l\left(X\left(s_i\right)\right){\varphi }_l\left(X\left(s\right)\right)+b,\hfill \\ \hfill & =\sum _{l=1}^M\left(\sum _{i=1}^n{\alpha }_i{\varphi }_l\left(X\left(s_i\right)\right)\right){\varphi }_l\left(X\left(s\right)\right)+b.\hfill \end{array}$$

(16)

The trend function can, hence, be written in the following form:

$$\begin{array}{c}\hfill \mu \left(s\right)=\sum _{l=0}^M{ã}_l{\varphi }_l\left(X\left(s\right)\right),\end{array}$$

(17)

where the coefficient ${ã}_l={\sum }_{i=1}^n{\alpha }_i{\varphi }_l\left(X\left(s_i\right)\right)$ and ${\varphi }_l\left(X\left(s\right)\right)$ is the known function for $l=1,\dots ,M$ in which ${ã}_0=b$ and ${\varphi }_0\left(X\left(s\right)\right)=1$.

By comparing Equation (2) to Equation (17), ${ã}_l$ and ${\varphi }_l$ can be treated as $a_l$ and $f_l$, respectively, and ${\varphi }_l$ in (17) is then employed in the KED system (5). This verifies the use of the kernel function as a non-linear trend model for the KED. The process of the KED based on the LSSVR method is provided according to the flowchart shown in Figure 1.

3.2. Examples of Explicitly Feature Mapping

There have been various kernels for the LSSVR method, namely linear, polynomial, and radial basis function kernels [44,45,46]. This section presents the last two kernels, as they are widely used and relatively easy to tune. They will also be applied in our model to formulate the trend component.

3.2.1. Polynomial Kernel

The polynomial kernel function is defined as

$$\begin{array}{c}\hfill K\left(X\left(s_i\right),X\left(s\right)\right)={(k+X^T\left(s_i\right)X\left(s\right))}^p,\end{array}$$

(18)

where $k>0$ and $p\in \mathbb{N}$ is the degree of polynomial.

The feature mapping for polynomial kernel degree$-p$ is given by [47]

$$\begin{array}{c}\hfill \varphi \left(X\left(s\right)\right)=\left[\sqrt{\frac{p!}{j_1!···j_{\eta +1}!}}{X}_1^{j_1}\left(s\right)···X_{\eta }^{j_{\eta }}\left(s\right){\sqrt{k}}^{j_{\eta +1}}|j_i\ge 0\mathrm{with}\sum _{i=1}^{\eta +1}{j}_i=p\right],\end{array}$$

(19)

where the dimensionality of $\varphi \left(X\left(s\right)\right)$ is $\frac{(\eta +p)(\eta +p-1)...(\eta +1)}{p!}$ [48]. For example, when the degree of the polynomial kernel and the number of auxiliary variables are both equal to 2, with k being 1, then

$$\begin{array}{c}\hfill \varphi \left(X\left(s\right)\right)={\left[1,\sqrt{2}{X}_1\left(s\right),\sqrt{2}{X}_2\left(s\right),X_1^2\left(s\right),X_2^2\left(s\right),\sqrt{2}{X}_1\left(s\right)X_2\left(s\right)\right]}^T.\end{array}$$

(20)

Compared with Equation (14), the components of the feature mapping are as follows: ${\varphi }_1\left(X\left(s\right)\right)=1$, ${\varphi }_2\left(X\left(s\right)\right)=\sqrt{2}{X}_1\left(s\right)$, ${\varphi }_3\left(X\left(s\right)\right)=\sqrt{2}{X}_2\left(s\right)$, ${\varphi }_4\left(X\left(s\right)\right)=X_1^2\left(s\right)$, ${\varphi }_5\left(X\left(s\right)\right)=X_2^2\left(s\right)$, ${\varphi }_6\left(X\left(s\right)\right)=\sqrt{2}{X}_1\left(s\right)X_2\left(s\right)$.

3.2.2. Radial Basis Function Kernel

The implicit kernel function, exemplified by the radial basis function (RBF) kernel, assumes the following form:

$$\begin{array}{c}\hfill K\left(X\left(s_i\right),X\left(s\right)\right)=exp(-g\parallel X\left(s\right)-X\left(s_i\right){\parallel }^2),\end{array}$$

(21)

where $g>0$ is an RBF kernel parameter.

The feature mapping for the RBF kernel function can be formulated as

$$\begin{array}{c}\hfill \varphi \left(X\left(s\right)\right)={exp(-g\parallel X\left(s\right)\parallel }^2)\left[\sqrt{\frac{{\left(2g\right)}^r}{r!}}{\mathsf{ø}}_r\left(X\left(s\right)\right)|r=0,\dots ,\infty \right],\end{array}$$

(22)

where

$$\begin{array}{c}\hfill {\mathsf{ø}}_r\left(X\left(s\right)\right)=\left[\sqrt{\frac{r!}{j_1!···j_{\eta }!}}{X}_1^{j_1}\left(s\right)···X_{\eta }^{j_{\eta }}\left(s\right)|j_i\ge 0\mathrm{with}\sum _{i=1}^{\eta }{j}_i=r\right],\end{array}$$

(23)

which is described in more detail in [49]. The RBF kernel function, which maps the auxiliary data to an infinite−dimensional space, can be approximated by Taylor polynomial-based monomial feature mapping (TPM feature mapping). In the work of [49], a finite-dimensional approximated feature mapping of the RBF function is obtained as follows:

$$\begin{array}{c}\hfill \varphi \left(X\left(s\right)\right)={exp(-g\parallel X\left(s\right)\parallel }^2)\left[\sqrt{\frac{{\left(2g\right)}^r}{r!}}{\mathsf{ø}}_r\left(X\left(s\right)\right)|r=0,\dots ,r_u\right],\end{array}$$

(24)

where $r_u$ is a selected approximation degree and the TPM feature mapping degree$-r_u$ has $\frac{(\eta +r_u)(\eta +r_u-1)···(\eta +1)}{r_u!}$ dimensions.

Although an increase in $r_u$ leads to improved estimation as $\varphi \left(X\left(s\right)\right)$ approaches the true function, it is sufficient enough to use the TPM feature mapping with low dimensions [49]. An example of TPM feature mapping with a degree of two and two auxiliary variables is

$$\begin{array}{c}\hfill \varphi \left(X\left(s\right)\right)={exp(-g\parallel X\left(s\right)\parallel }^2){\left[1,\sqrt{2g}{X}_1\left(s\right),\sqrt{2g}{X}_2\left(s\right),\sqrt{2}g{X}_1^2\left(s\right),\sqrt{2}g{X}_2^2\left(s\right),2g{X}_1\left(s\right)X_2\left(s\right)\right]}^T.\end{array}$$

(25)

By comparing Equation (25) with Equation (14), this results in ${\varphi }_1\left(X\left(s\right)\right)={exp(-g\parallel X\left(s\right)\parallel }^2)$, ${\varphi }_2\left(X\left(s\right)\right)={exp(-g\parallel X\left(s\right)\parallel }^2)\sqrt{2g}{X}_1\left(s\right)$, ${\varphi }_3\left(X\left(s\right)\right)={exp(-g\parallel X\left(s\right)\parallel }^2)\sqrt{2g}{X}_2\left(s\right)$, ${\varphi }_4\left(X\left(s\right)\right)={exp(-g\parallel X\left(s\right)\parallel }^2)\sqrt{2}g{X}_1^2\left(s\right)$, ${\varphi }_5\left(X\left(s\right)\right)={exp(-g\parallel X\left(s\right)\parallel }^2)\sqrt{2}g{X}_2^2\left(s\right)$, ${\varphi }_6\left(X\left(s\right)\right)={exp(-g\parallel X\left(s\right)\parallel }^2)2g{X}_1\left(s\right)X_2\left(s\right)$.

4. Case Study: Estimations of Temperature and Pressure in Thailand

4.1. Study Area

The evaluation of the efficiency and accuracy of the proposed techniques was carried out to interpolate temperature and pressure in Thailand. The country is located between $5^{°}{37}^{\prime }$ N and ${20}^{°}{27}^{\prime }$ N latitude and ${97}^{°}{22}^{\prime }$ E and ${105}^{°}{37}^{\prime }$ E longitude, with a total area of 513,115 km${}^2$ and a coastline of 3219 km [50,51]. The data used in this study consist of monthly averages of temperature, pressure, relative humidity, digital elevation model (DEM), and geographic locations (coordinates) spanning from January 2017 to December 2017. The summary statistics of the data, including the mean values and standard deviations, are presented in

Table 1. Descriptive statistics of the data.

Variables	Mean Values	Standard Deviations
Temperature	28.3146	1.2584
Pressure	989.5460	14.4254
Relative humidity	75.3258	4.4412
Latitude	15.1074	3.5582
Longitude	100.9769	1.8696
DEM	159.7653	148.1880

. These data were acquired from the National Hydroinformatics and Climate Data Center (NHC), developed by Hydro-Informatics Institute (HII) [52]. Figure 2 displays 213 meteorological stations after data preparation and cleaning.

4.2. Evaluation of Model Accuracy

In this study, we compare the accuracy of KED with three different trend functions: the linear trend function estimated using the GLS estimator (KED−GLS), the non-linear trend function based on LSSVR with polynomial feature mapping of degree one and two (KED−Poly1 and KED−Poly2), and the non-linear trend function based on LSSVR with TPM feature mapping of degree one and two (KED−TPM1 and KED−TPM2).

The $k-$fold cross-validation technique was applied to examine the performance of the models. The data were randomly divided into 10 folds. For each iteration, each fold was used as a testing dataset for the model built upon the remaining nine folds. After 10 iterations in which each fold was once selected as testing data, the overall estimation accuracy is an average of the accuracy scores calculated from each iteration [53]. The root-mean-square error (RMSE) [54] and the mean-absolute-percentage error (MAPE) [55] were the model performance indicators, which are formulated as follows:

$$\begin{array}{cc}\hfill \mathrm{RMSE}& =\sqrt{\frac{1}{N}{\displaystyle \sum _{i=1}^N}{\left(Z\left(s_i\right)-Z^{*}\left(s_i\right)\right)}^2},\hfill \end{array}$$

(26)

$$\begin{array}{cc}\hfill \mathrm{MAPE}& =\frac{1}{N}{\displaystyle \sum _{i=1}^N}\frac{|Z\left(s_i\right)-Z^{*}\left(s_i\right)|}{Z\left(s_i\right)}\times 100,\hfill \end{array}$$

(27)

where N is the number of observations, $Z\left(s_i\right)$ and $Z^{*}\left(s_i\right)$ denote the observed data and the estimated value at coordinate $s_i$, respectively.

4.3. Results

Before proceeding to the KED estimation, a selection of auxiliary factors is required.

Table 2. Correlation coefficients between the target and auxiliary variables.

Auxiliary Variables	Temperature		Pearson
	Pearson	Spearman	Pearson	Spearman
Temperature	1.0000	1.0000	0.5537	0.5092
Pressure	0.5537	0.5092	1.0000	1.0000
Relative humidity	−0.2474	−0.2389	0.1540	0.1869
Latitude	−0.1322	−0.2425	−0.5338	−0.5867
Longitude	0.0198	0.0345	0.0283	−0.0399
DEM	−0.4501	−0.4421	−0.7350	−0.7470

presents the statistical analysis of the interdependence between each selected variable and the target variables through the Pearson and Spearman correlation coefficients [56]. The results indicate a significant positive correlation between temperature and pressure (correlation coefficients greater than 0.5). The pressure is negatively correlated with both DEM and latitude, with correlation coefficients less than −0.5. This suggests that pressure can be chosen as an auxiliary variable for temperature estimation and vice versa. On the other hand, both DEM and latitude are additionally included as auxiliary factors for interpolating pressure. The scatter plots depicting the relationships between the target and auxiliary variables in March, July, and November are presented in Figure 3, Figure 4, and Figure 5, respectively.

The exponential variogram model employed in this study encompasses three parameters: nugget, sill, and range. The fitted residual variogram parameters for temperature data in fold 1 are presented in

Table 3. Theresidual variogram parameters for temperature data.

Months	GLS Trend Estimation			TPM2 Trend Estimation
	Nugget	Sill	Range	Nugget	Sill	Range
March	0.6314	0.8036	575.4646	0.6282	0.8526	522.9156
July	0.2279	0.3905	178.4360	0.2347	0.4214	161.7477
November	0.1587	0.3813	138.0978	0.2608	0.5346	201.1555

, while those for pressure data are shown in

Table 4. The residual variogram parameters for pressure data.

Months	GLS Trend Estimation			Poly2 Trend Estimation
	Nugget	Sill	Range	Nugget	Sill	Range
March	18.8195	68.2006	68.4513	15.9667	68.5886	55.4515
July	22.4502	72.1840	93.5990	19.8437	70.9177	76.0110
November	10.0162	32.9229	65.6270	10.2971	33.1893	71.1975

. Moreover, the empirical residual variograms and exponential variogram models of temperature and pressure in Thailand in March, July, and November 2017 are presented in Figure 6 and Figure 7, respectively.

Table 5. Prediction errors of kriging with external drift and three different trend functions for temperature and pressure data in 2017.

Target Variables	Auxiliary Variables	Errors	KED with Linear Trend	KED with Non-Linear Trend Based on LSSVR
				Polynomial Feature Mapping		TPM Feature Mapping
			KED-GLS	KED-Poly1	KED-Poly2	KED-TPM1	KED-TPM2
Temperature	Pressure	RMSE	0.8252	0.8275	0.8232	0.8512	0.8123
		MAPE	2.3397	2.3486	2.3439	2.4041	2.2888
Pressure	DEM	RMSE	8.6212	8.6111	7.7541	9.3854	8.7372
	Latitude Temperature	MAPE	0.6170	0.6181	0.5466	0.6698	0.6329

reports the estimation efficiency of the KED model with three different trend functions via MAPE and RMSE measures. According to the accuracy statistics, the KED with a non-linear trend function based on LSSVR has a superior estimation performance to that of the KED with a linear trend for both temperature and pressure. Specifically, the prediction errors for temperature generated by the KED−TPM2 are smaller than those of all other methods, with RMSE being 0.8123 and MAPE being 2.2888. These are, respectively, equivalent to 1.5633% and 2.1755% improvement compared to the KED−GLS method. The optimal pressure estimates are achieved by using KED−Poly2 with the RMSE and MAPE equal to 7.7541 and 0.5466, respectively. KED−Poly2 reduces both MAPE and RMSE values by over 10% with respect to the KED−GLS approach.

To further compare the estimation performance of all methods, visual–spatial distribution patterns of the monthly averages of temperature and pressure in Thailand in March, July, and November 2017 in fold 1 are presented. These maps were created using QGIS (Quantum Geographic Information System) software version 3.34.0 and the study area was partitioned into a grid of square cells of 0.05 degrees per side.

Figure 8 shows the spatial distribution patterns of the monthly mean temperature. The panels on the left column depict the results generated by KED−GLS, whereas the panels on the right column illustrate the results obtained from KED−TPM2. Both KED−TPM2 and KED−GLS produce a roughly similar distribution pattern for the average July temperature. This may be due to the fact that there is a small variation in temperature across the country during the rainy season (July–October). The discrepancy in temperature derived from these two models is therefore not significant. On the contrary, clear differences can be observed in March and November, in which the area of high temperature is more broadly distributed in the central part of the country for the KED−TPM2. The model also generates an overall lower temperature level concentrated in the northern region in November. Figure 9 displays spatial distribution maps of monthly mean pressure where the left column again corresponds to the estimates attained from the KED−GLS while those in the right column are derived from the KED−Poly2. The results show a distinct difference between these two methods. In particular, lower pressure values estimated by KED−Poly2 are clearly marked in the northern and western parts of the study area.

5. Conclusions and Discussion

This paper presents the novel KED method that applies the LSSVR technique to improve spatial interpolation accuracy in the presence of non-linear trends. The method involves determining the drift component through explicit feature mapping which is expressed in terms of kernel functions. A comparison between our proposed method and the KED with the linear trend is demonstrated in the case of the temperature and pressure estimation in Thailand in 2017. The results show that the KED with LSSVR outperforms the KED approach with a linear trend function regarding estimation accuracy.

The advantage of the KED with LSSVR can be attributed to its ability to extract implicit non-linear relationships between the target and auxiliary variables. This gives rise to more accurate interpolation results. Furthermore, the LSSVR is a powerful machine learning algorithm that has been proven effective in a variety of regression tasks. This allows our method to adapt to various data types. However, the choice of kernel function in LSSVR can have a significant impact on the estimation accuracy. Although a higher-degree polynomial kernel or a higher-degree TPM feature mapping can model more complex relationships in the data, it can also result in several equations in the kriging system. This can lead to more time-intensive computation and an increase in the likelihood of model overfitting issues.