An Expanded Spatial Durbin Model with Ordinary Kriging of Unobserved Big Climate Data

Abstract

Spatial models are essential in the prediction of climate phenomena because they can model the complex relationships between different locations. In this study, we discuss an expanded spatial Durbin model with ordinary kriging on unobserved locations (ESDMOK) to predict rainfall patterns in Java Island. The classical spatial Durbin model needed to be expanded to obtain a parameter estimation for each location. We combined this with ordinary kriging because the data were not available in some locations. The data were taken from the National Aeronautics and Space Administration Prediction of Worldwide Energy Resources (NASA POWER) website. Since climate data are big data, we implement a big data analytics approach, namely the data analytics life cycle method. As the exogenous variables, we used air temperature, humidity, solar irradiation, wind speed, and surface pressure. The authors developed an R-Shiny web applications to implement our proposed technique. Using our proposed technique, we obtained more accurate and reliable climate data prediction, indicated by the mean absolute percentage error (MAPE), which was equal to 1.956%. The greatest effect on rainfall was given by the surface pressure variable, and the smallest was wind speed.

1. Introduction

Climate is the long-term average weather and atmospheric conditions in a specific region, typically measured for 30 years or more, both regionally and globally [1]. These climate phenomena have significant effects on various aspects of human life, including agriculture, health, coastal ecosystems, and water quality [2]. One notable impact is La Nina, which increases rainfall in the Western Pacific region. Empirical data from BMKG indicates that La Nina can increase rainfall on Java Island by 20% and 70% [3]. Extreme rainfall causes landslides and floods, among other natural disasters [4,5]. The community suffers greatly from the financial and health effects of these natural disasters, and those who depend on rainfall conditions, like farmers and fishermen, truly need information related to rainfall prediction. The intensity of rainfall has a significant impact on every aspect of life, and exceptionally high rainfall has an impact on the occurrence of natural disasters and so should be warned of before the event. People who depend on rainfall conditions, such as fishermen and farmers, really need information related to rainfall prediction. Rainfall is closely linked to Sustainable Development Goal (SDG) 13, which focuses on “Climate Action”. This goal emphasizes urgent action to combat climate change and its impacts [4]. Understanding and managing rainfall patterns are crucial for achieving SDG 13. Effective climate action requires addressing the impacts of changing rainfall on various sectors, enhancing resilience to extreme weather events, and ensuring sustainable water and land management.

The spatial model is an effective tool for predicting rainfall and climate. Creating statistical or computational models to represent spatial relationships or patterns in data is a prevalent practice in spatial modelling [5]. Spatial models are widely used in various fields to analyse patterns, relationships, and phenomena related to the spatial dimension in data [6]. Rainfall is closely related to other climate elements, such as air temperature, humidity, solar irradiation, wind speed and surface pressure, which vary from region to region [7]. These climate variables are observed and modelled using spatial analysis. Falah et al. (2023) conducted hybrid modelling of the Spatial Autoregressive Exogenous (SAR-X) model using Casetti’s model approach for the prediction of rainfall in West Java, Indonesia [8]. The weakness of the SAR-X model is that it only facilitates spatial dependence on the response variable, not paying attention to spatial dependence on the exogenous variables. Discussing spatial aspects, there is a possibility that there is spatial dependency not only on the response variable but also on exogenous variables. Thus, a model approach is needed that can facilitate the existence of spatial dependencies between these variables. Therefore, to overcome these problems, the expanded spatial Durbin model (ESDM) was used to overcome spatial dependence on exogenous variables [9].

In various phenomena, there are often unknown values at observation locations. A linear interpolation model called the ordinary kriging (OK) model is utilized to predict these values at unobserved locations [10]. The OK method is a spatial interpolation method that uses the spatial variability of data to estimate values at unobserved locations [11]. The advantage of the OK method is that it produces an optimal estimate by taking into account the spatial information and covariance structure of the data [12]. This is to obtain the most accurate prediction at unobserved locations [13]. The predictions generated by OK often produce smooth maps without any sharp spikes or dips between observation points [14].

The ongoing expansion of data poses a challenge to the prediction of climate and rainfall patterns. We discovered a gap in rainfall prediction utilizing the ESDM with OK of climate data based on the summary above. These climate data were collected from 119 districts/cities in Java Island using the National Aeronautics and Space Administration Prediction of Worldwide Energy Resources (NASA POWER) website. In this research, the data analytics life cycle was the big data approach methodology, which was utilized to predict rainfall using big climate data. Rainfall description and prediction have a more in-depth stage in the data analytics life cycle. The application study in this research is supported by computing an integrated R script for the development of the ESDM using an R-Shiny web application to facilitate the prediction process.

2. Materials and Methods

2.1. Experimental Semivariogram

The OK method was used to analyse geostatistical data and interpolate values based on observed data. It is worth noting that D. G. Krige, a gold mining expert from South Africa, first introduced the model. Furthermore, OK relies on the assumption that there is a spatial correlation between observed data as determined by the distance between the entities [15]. To perform the interpolation, the model employs a semivariogram calculation representing the spatial differences and values between all pairs of observed data. The semivariogram also shows the weights used in the interpolation process, which is calculated based on a sample semivariogram with distance h, different z values, and n data samples. The experimental semivariogram at distance h can be expressed as follows [16]:

$$\widehat{\gamma }\left(h\right)={\displaystyle \frac{1}{2N\left(h\right)}}{\displaystyle \sum _{i=1}^{N\left(h\right)}{\left[Z\left(s_i+h\right)-Z\left(s_i\right)\right]}^2}$$

(1)

where:

• $\widehat{\gamma }\left(h\right)$: experimental semivariogram value with distance $h$;
• $Z\left(s_i\right)$: observation value in location $s_i$;
• $Z\left(s_i+h\right)$: observation value in location $s_i+h$;
• $N\left(h\right)$: many pairs of data with the same distance $h$;
• $h$: distance.

All possible pairs of distances were calculated using a Euclidean distance equation as follows:

$$\left|h\right|=\sqrt{{\left(s_i\left(u_i\right)-s_j\left(u_j\right)\right)}^2-{\left(s_i\left(v_i\right)-s_j\left(v_j\right)\right)}^2}$$

(2)

The results of the Euclidean distance calculation were converted to kilometres with $\left|d_{ij}\right|\times 111.319$. A value of 111.319 was obtained by converting 1 degree longitude to kilometres [17]. A location can be determined as $s_{ij}\left(u_{ij},v_{ij}\right)$, where $s_{ij}$ is the symbol of the location $i$ and $j$, with $i=1,2,3\dots ,N$ and $j=1,2,3\dots ,N$, while $u$ and $v$ indicate the latitude and longitude coordinates.

2.2. Theoretical Semivariogram Model

There are three commonly used theoretical semivariogram models in kriging: the spherical, the Gaussian, and the exponential [16]. Furthermore, theoretical semivariogram values can be divided into three using the equations in

Table 1. Theoretical semivariogram model.

Model	Function
Spherical	$\gamma \left(h\right)=\{\begin{array}{ll}c\left[\left({\displaystyle \frac{3h}{2a}}\right)-{\left({\displaystyle \frac{h}{2a}}\right)}^3\right],& h\le a\\ c,& h>a\end{array}$	(3)
Exponential	$\gamma \left(h\right)=\{\begin{array}{ll}c\left[1-exp\left({\displaystyle \frac{-h}{a}}\right)\right],& h\le a\\ c,& h>a\end{array}$	(4)
Gaussian	$\gamma \left(h\right)=\{\begin{array}{ll}c\left[1-exp{\left({\displaystyle \frac{-h}{a}}\right)}^2\right],& h\le a\\ c,& h>a\end{array}$	(5)

: h is the location distance between samples, c is the sill value, and a is the range [18]. Theoretical semivariogram models can be seen in Table 1, and a theoretical semivariogram plot can be seen in Figure 1.

2.3. Ordinary Kriging (OK) Method

The kriging method is a prediction method that provides a BLUE (best linear unbiased estimation) estimator of point values or averages for prediction at unobserved locations. This method uses a semivariogram calculation that represents the spatial and value differences between all pairs of data samples. According to [19], the kriging estimator $\widehat{Z}\left(x\right)$, where $s$ is the location at the unsampled point, is a linear combination of random variables; this can be seen in the kriging estimation formulated in the following equation:

$$\widehat{Z}\left(s\right)-m\left(s\right)={\displaystyle \sum _{i=1}^n{\lambda }_i\left[Z\left(s_i\right)-m\left(s_i\right)\right]}$$

(6)

with:

• $s$: predicted locations;
• $s_i$: i-th data location adjacent to the predicted location;
• $m\left(s\right)$: the expected or average value of $Z\left(s\right)$;
• $m(s_i)$: the expected or average value of $Z(s_i)$;
• n: the number of data used for prediction;
• ${\lambda }_i$: weight value at i-th location.

The objective of the kriging method is to determine the weight values ${\lambda }_i$ that result in minimum estimator variance and an unbiased estimator. The estimator variance can be expressed as follows [20]:

$${\sigma }_e^2\left(s\right)=Var\left[\widehat{Z}(s)-Z(s)\right]$$

(7)

while the requirement to produce an unbiased estimator is:

$$E\left[\widehat{Z}(s)-Z(s)\right]=0$$

(8)

The OK method is one of the kriging methods that assumes that the mean is unknown. If $Z\left(s\right)=m(s)$ and $Z(s_i)=m(s_i)$, then $m\left(s\right)=m(s_i)=m$, and then Equation (6) becomes:

$$\widehat{Z}(s)-m={\displaystyle \sum _{i=1}^n{\lambda }_i\left[Z(s_i)-m\right]}$$

$$\widehat{Z}(s)=m+{\displaystyle \sum _{i=1}^n{\lambda }_i\left[Z(s_i)-m\right]}$$

$$\widehat{Z}(s)={\displaystyle \sum _{i=1}^n{\lambda }_{i}Z(s_i)-m\left({\displaystyle \sum _{i=1}^n{\lambda }_i-1}\right)}$$

Since $m$ is assumed to be unknown, the OK estimator is obtained as $\widehat{Z}(s)={\displaystyle \sum _{i=1}^n{\lambda }_{i}Z(s_i)}$ with the condition ${\displaystyle \sum _{i=1}^n{\lambda }_i}=1$.

The BLUE (best linear unbiased estimation) properties of the OK method are shown as follows:

• Linear
  The OK estimator obtained from n observations of the data used forms a linear model, namely:

  $$\widehat{Z}(s)={\displaystyle \sum _{i=1}^n{\lambda }_{i}Z(s_i)}$$

  (9)
• Unbiased
  The OK estimator is unbiased if it satisfies Equation (8)

  $$\begin{array}{ll}\hfill E\left[\widehat{Z}\left(s_i\right)-Z\left(s\right)\right]& =E\left[{\displaystyle \sum _{i=1}^n{\lambda }_{i}Z\left(s_i\right)-Z\left(s\right)}\right]\hfill \\ & ={\displaystyle \sum _{i=1}^n{\lambda }_{i}E\left[Z\left(s_i\right)-Z\left(s\right)\right]}\hspace{1em}\hspace{1em}\left(\because E\left[Z\left(s_i\right)-Z\left(s\right)\right]=0\right)\hfill \\ & =0\hfill \end{array}$$

  since the mean is assumed to be unknown, $E\left[Z\left(s_i\right)-Z\left(s\right)\right]=0$, and the unbiased estimator property is satisfied by the OK method.
• Best

The best here means that the OK estimator has the minimum error variance. The variance of the estimator of OK is as follows:

$${\sigma }_{OK}^2=Var\left[\widehat{Z}\left(s\right)-Z\left(s\right)\right]$$

$${\sigma }_{OK}^2=Var\left[\widehat{Z}\left(s\right)\right]+Var\left[Z\left(s\right)\right]-2Cov\left[\widehat{Z}\left(s\right),Z\left(s\right)\right]$$

(10)

to describe $Var\left[\widehat{Z}(s)\right]$ in Equation (10), it is known that $\widehat{Z}(s)={\displaystyle \sum _{i=1}^n{\lambda }_{i}Z(s_i)}$, which can be expressed as follows:

$$Var\left[\widehat{Z}(s)\right]=Var\left[{\displaystyle \sum _{i=1}^n{\lambda }_{i}Z(s_i)}\right]$$

$$Var\left[\widehat{Z}(s)\right]={\displaystyle \sum _{i=1}^n{\displaystyle \sum _{j=1}^n{\lambda }_i}{\lambda }_{j}Cov\left[Z(s_i),Z(s_j)\right]}$$

(11)

to describe $Cov\left[\widehat{Z}(s),Z(s)\right]$ in Equation (11):

$$\begin{array}{l}Cov\left[\widehat{Z}(s),Z(s)\right]=E\left[\widehat{Z}(s)Z(s)\right]-E\left[\widehat{Z}(s)\right]E\left[Z(s)\right]\\ Cov\left[\widehat{Z}(s),Z(s)\right]=E\left[\left({\displaystyle \sum _{i=1}^n{\lambda }_{i}Z\left(s_i\right)}\right)Z(s)\right]-E\left[{\displaystyle \sum _{i=1}^n{\lambda }_{i}Z\left(s_i\right)}\right]E\left[Z(s)\right]\\ Cov\left[\widehat{Z}(s),Z(s)\right]={\displaystyle \sum _{i=1}^n{\lambda }_{i}E\left[Z(s_i)Z(s)\right]-{\displaystyle \sum _{i=1}^n{\lambda }_i}E\left[Z(s_i)\right]E\left[Z(s)\right]}\end{array}$$

$$Cov\left[\widehat{Z}(s),Z(s)\right]={\displaystyle \sum _{i=1}^n{\lambda }_{i}Cov\left[Z(s_i),Z(s)\right]}$$

(12)

Supposing that $Var\left[Z\left(s\right)\right]={\sigma }^2$, Equations (11) and (12) can be substituted into Equation (10), and the following equation can be obtained:

$${\sigma }_{OK}^2={\displaystyle \sum _{i=1}^n{\displaystyle \sum _{j=1}^n{\lambda }_i}{\lambda }_{j}Cov\left[Z(s_i),Z(s_j)\right]}+{\sigma }^2-2{\displaystyle \sum _{i=1}^n{\lambda }_{i}Cov\left[Z(s_i),Z(s)\right]}$$

(13)

with the condition ${\displaystyle \sum _{i=1}^n{\lambda }_{\mathrm{i}}=1}$.

Based on Equation (13), to obtain the minimum value of the estimator variance, the Lagrange multiplier (LM) method was used with the parameter $\mu$. The LM equation is expressed as follows:

$$F({\lambda }_i,\mu )={\displaystyle \sum _{i=1}^n{\displaystyle \sum _{j=1}^n{\lambda }_i}{\lambda }_{j}Cov\left[Z(s_i),Z(s_j)\right]}+{\sigma }^2-2{\displaystyle \sum _{i=1}^n{\lambda }_{i}Cov\left[Z(s_i),Z(s)\right]}+2\mu \left[{\displaystyle \sum _{i=1}^n{\lambda }_i-1}\right]$$

(14)

By deriving Equation (14) for the following variables ${\lambda }_i$:

$${\displaystyle \frac{\partial F({\lambda }_i,\mu )}{\partial {\lambda }_i}}=2{\displaystyle \sum _{j=1}^n{\lambda }_{j}Cov\left[Z(s_i),Z(s_j)\right]}-2Cov\left[Z(s_i),Z(s)\right]+2\mu =0$$

Because ${\displaystyle \frac{\partial F({\lambda }_i,\mu )}{\partial {\lambda }_i}}=0$, the following is obtained:

$${\displaystyle \sum _{j=1}^n{\lambda }_{j}Cov\left[Z(s_i),Z(s_j)\right]}=Cov\left[Z(s_i),Z(s)\right]-\mu$$

(15)

By deriving Equation (14) with respect to the parameter $\mu$, we obtain:

$${\displaystyle \frac{\partial F({\lambda }_i,\mu )}{\partial \mu }}=2\left[{\displaystyle \sum _{i=1}^n{\lambda }_i-1}\right]=0$$

Because ${\displaystyle \frac{\partial F({\lambda }_i,\mu )}{\partial \mu }}=0$, the following is obtained:

$$\begin{array}{l}2\left[{\displaystyle \sum _{i=1}^n{\lambda }_i-1}\right]=0\\ {\displaystyle \sum _{i=1}^n{\lambda }_i-1}=0\end{array}$$

$${\displaystyle \sum _{i=1}^n{\lambda }_i=1}$$

(16)

Equations (15) and (16) compose an OK system. By solving the equation, the following matrix can be formed:

$$\begin{array}{l}{\lambda }_1{C}_{11}+{\lambda }_2{C}_{12}+{\lambda }_3{C}_{13}+\dots +{\lambda }_n{C}_{1n}+\mu =C_{10}\\ {\lambda }_1{C}_{21}+{\lambda }_2{C}_{22}+{\lambda }_3{C}_{23}+\dots +{\lambda }_n{C}_{2n}+\mu =C_{20}\\ {\lambda }_1{C}_{31}+{\lambda }_2{C}_{32}+{\lambda }_3{C}_{33}+\dots +{\lambda }_n{C}_{3n}+\mu =C_{30}\\ ⋮\\ {\lambda }_1{C}_{n1}+{\lambda }_2{C}_{n2}+{\lambda }_3{C}_{n3}+\dots +{\lambda }_n{C}_{nn}+\mu =C_{n0}\\ {\lambda }_1+{\lambda }_2+{\lambda }_3+\dots +{\lambda }_n+0=1\end{array}$$

$$\left(\begin{array}{cccccc}{C}_{11}& C_{12}& C_{13}& \cdots & C_{1n}& 1\\ C_{21}& C_{22}& C_{23}& \cdots & C_{2n}& 1\\ C_{31}& C_{32}& C_{33}& \cdots & C_{3n}& 1\\ ⋮& ⋮& ⋮& \ddots & ⋮& ⋮\\ C_{n1}& C_{n2}& C_{n3}& \cdots & C_{nn}& 1\\ 1& 1& 1& 1& 1& 0\end{array}\right)\left(\begin{array}{c}{\lambda }_1\\ {\lambda }_2\\ {\lambda }_3\\ ⋮\\ {\lambda }_n\\ \mu \end{array}\right)=\left(\begin{array}{c}{C}_{10}\\ C_{20}\\ C_{30}\\ ⋮\\ C_{n0}\\ 1\end{array}\right)$$

(17)

Meanwhile, to determine the weight value of each observed point against the unobserved point, this can be expressed as follows:

$$\left(\begin{array}{c}{\lambda }_1\\ {\lambda }_2\\ {\lambda }_3\\ ⋮\\ {\lambda }_n\\ \mu \end{array}\right)={\left(\begin{array}{cccccc}{C}_{11}& C_{12}& C_{13}& \cdots & C_{1n}& 1\\ C_{21}& C_{22}& C_{23}& \cdots & C_{2n}& 1\\ C_{31}& C_{32}& C_{33}& \cdots & C_{3n}& 1\\ ⋮& ⋮& ⋮& \ddots & ⋮& ⋮\\ C_{n1}& C_{n2}& C_{n3}& \cdots & C_{nn}& 1\\ 1& 1& 1& 1& 1& 0\end{array}\right)}^{-1}\left(\begin{array}{c}{C}_{10}\\ C_{20}\\ C_{30}\\ ⋮\\ C_{n0}\\ 1\end{array}\right)$$

(18)

$$\lambda ={C_{nn}}^{-1}{C}_{n0}$$

where:

• $C_{nn}$: the variance covariance matrix between the variables sampled at the location $n$ and the sampled variable at location $n$;
• $C_{n0}$: the variance vector of the covariance between the sampled variable at the location $n$ and the predicted variable;
• $\mu$: the Lagrange multiplier parameter.

To obtain the variance equation of the OK estimator in Equation (13), Equation (15) can be substituted, and the following is obtained:

$$\begin{array}{l}{\sigma }_{OK}^2={\displaystyle \sum _{i=1}^n{\displaystyle \sum _{j=1}^n{\lambda }_i}{\lambda }_{j}Cov\left[Z(s_i),Z(s_j)\right]}+{\sigma }^2-2{\displaystyle \sum _{i=1}^n{\lambda }_{i}Cov\left[Z(s_i),Z(s)\right]}\\ {\sigma }_{OK}^2={\displaystyle \sum _{i=1}^n{\lambda }_i{\displaystyle \sum _{j=1}^n{\lambda }_{j}Cov\left[Z(s_i),Z(s_j)\right]}}+{\sigma }^2-2{\displaystyle \sum _{i=1}^n{\lambda }_{i}Cov\left[Z(s_i),Z(s)\right]}\\ {\sigma }_{OK}^2={\displaystyle \sum _{i=1}^n{\lambda }_i}Cov\left[Z(s_i),Z(s)\right]-\mu +{\sigma }^2-2{\displaystyle \sum _{i=1}^n{\lambda }_{i}Cov\left[Z(s_i),Z(s)\right]}\end{array}$$

$${\sigma }_e^2={\sigma }^2-{\displaystyle \sum _{i=1}^n{\lambda }_i}Cov\left[Z(s_i),Z(s)\right]-\mu$$

$${\sigma }_{OK}^2={\sigma }^2-\left({\lambda }_1{C}_{10}+{\lambda }_2{C}_{20}+{\lambda }_3{C}_{30}+\dots +{\lambda }_n{C}_{n0}\right)-\mu$$

$${\sigma }_{OK}^2={\sigma }^2-\left(\begin{array}{cccccc}{C}_{10}& C_{20}& C_{30}& \cdots & C_{n0}& 1\end{array}\right)\left(\begin{array}{c}{\lambda }_1\\ {\lambda }_2\\ {\lambda }_3\\ \cdots \\ {\lambda }_n\\ \mu \end{array}\right)$$

(19)

The minimum estimator variance is commonly referred to as the OK estimator variance; thus, the best estimator satisfied the OK method.

2.4. Expanded Spatial Durbin Model (ESDM)

The ESDM was used to overcome spatial dependence on the exogenous variables. The ESDM is formulated as follows [9]:

$$\mathit{y}=\rho \mathbf{W}\mathit{y}+\alpha {\mathsf{1}}_n+\mathbf{X}\mathbf{Z}\mathbf{J}{\mathit{\beta }}_0+\mathbf{W}\tilde{\mathbf{X}}\mathit{\theta }+\mathit{\epsilon }\mathrm{with}\mathit{\epsilon }\stackrel{iid}{~}N(0,{\sigma }^{2}I)$$

(20)

Letting $\mathbf{A}=\mathbf{X}\mathbf{Z}\mathbf{J}$, it follows that:

$$\mathit{y}=\rho \mathbf{W}\mathit{y}+\alpha {\mathsf{1}}_n+\mathbf{A}{\mathit{\beta }}_0+\mathbf{W}\tilde{\mathbf{X}}\mathit{\theta }+\mathit{\epsilon }\mathrm{with}\mathit{\epsilon }\stackrel{iid}{~}N(0,{\sigma }^{2}I)$$

(21)

$$\mathit{y}=\rho \mathbf{W}\mathit{y}+\mathit{U}\mathit{\delta }+\mathit{\epsilon }$$

(22)

where:

$$\mathit{U}=\left[\begin{array}{ccc}{\mathsf{1}}_n& \mathbf{A}& \mathbf{W}\tilde{\mathbf{X}}\end{array}\right],$$

$$\mathit{\delta }=\left[\begin{array}{c}\alpha \\ {\mathit{\beta }}_0\\ \mathit{\theta }\end{array}\right],$$

with:

• $y$: vector of dependent variables of size $\left(n\times 1\right)$;
• $\tilde{\mathbf{X}}$: matrix of independent variables of size $\left(n\times k\right)$;
• $X$: matrix of independent variables of size $\left(n\times nk\right)$;
• $\rho$: spatial lag coefficient of the dependent variable;
• $\alpha$: constant parameter;
• $\mathbf{W}$: spatial weight matrix of size $\left(n\times n\right)$;
• $Z$: location information that contains elements $Z_{xi},Z_{yi}\mathrm{with}i=1,\dots ,n$, representing the latitude and longitude of each observation, of size $\left(nk\times 2nk\right)$;
• $J$: expansion of the identity matrix of size $\left(2nk\times 2k\right)$;
• $\beta$: matrix of size $\left(nk\times 1\right)$, which contains parameter estimators for all explanatory $k$ variables at each observation;
• ${\beta }_0$: parameter expressed by ${\beta }_{latitude},{\beta }_{longitude}$ of size $\left(2k\times 1\right)$;
• $\mathit{\theta }$: spatial lag parameter vector of covariate variable of size $\left(k\times 1\right)$;
• $\otimes$: Kronecker product;
• $\epsilon$: error vector of size $\left(n\times 1\right)$;
• $s_i$: location matrix with $i=1,\dots ,n$.

2.5. Mean Absolute Percentage Error (MAPE)

To evaluate the model’s performance, the mean absolute percentage error (MAPE) is calculated as follows:

$$MAPE=\left({\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n\left|\frac{\mathit{y}\left(s_i\right)-\widehat{\mathit{y}}\left(s_i\right)}{\mathit{y}\left(s_i\right)}\right|}\right)\times 100\%$$

(23)

with

• $\mathit{y}\left(s_i\right)$: the values in the actual data at the location $s_i$;
• $\widehat{\mathit{y}}\left(s_i\right)$: the values in the prediction data at the location $s_i$;
• $n$: the number of observation locations.

According to Lawrence’s criteria (2009) [21], MAPE values can be categorized as follows (

Table 2. MAPE score scale.

Scale MAPE	Accuracy Score
$\le$10%	Very accurate prediction
10 < MAPE $\le$20%	Good prediction
20 < MAPE $\le$50%	Reasonable prediction
>50%	Inaccurate prediction

):

2.6. Data Analytics Life Cycle

Large data quantities, a variety of data architectures, and rapid growth in data were the challenges faced by big data and data science, which led to the creation of the data analytics life cycle. There are six stages in this life cycle, which might happen simultaneously in some circumstances. This analysis could typically proceed both forward and backward, enabling an iterative process that takes into account newly discovered information as it becomes available [22]. This makes it possible to solve problems and go through the procedure repeatedly, which also makes it easier to operationalize research objectives. Best practices for the analytical process, from discovery to research work completion, are defined by the data analysis life cycle. The data analytics life cycle consists of six stages that apply looping or backward/forward. The following is a summary of the six stages of the data analysis life cycle [23]:

• Discovery (Problem Formulation): At this point, a literature review was conducted to prepare for the research problem analysis phase. This phase required assembling resources, including data, technology, references, and time. Developing a problem framework as an analytical task to be tackled in the following stage and developing preliminary hypotheses to investigate and evaluate the data were crucial tasks in this phase.
• Data Preparation: Initial data analysis was part of the data pre-processing performed at this stage. A necessary step before building the model was to prepare the data for collection in the database repository, which included procedures such as data cleaning, extraction, transformation, and integration.
• Model Planning: This stage focuses on planning the model by determining the methods, techniques, and research flow to be followed during the model-building stage.
• Model Building: At this stage, this research focused on creating datasets for testing, training, and creating output models. The model’s efficiency in running on the current hardware, such as its quick hardware and parallel processing capabilities, was considered.
• Communicating Results: This step entailed testing the data model and any modifications with the user or in an experimental environment to ascertain whether the output complied with the development criteria. Should the model fail to satisfy the specifications, an assessment was carried out, and the procedure could revert to the earlier phase for further improvement.
• Operationalizing (Operationalization): At this point, the final report, instructions, codes, and technical documents had to be submitted. To guarantee a wider application, this stage can also include implementing the model as a pilot project.

If more enhancements are needed, the data analytics life cycle might be carried out again from phases 1 through 5. The evaluation of the modelling process, from steps 6 to 1, was indicated by dotted lines, highlighting the possibility of revisiting certain stages if the modelling results did not meet the desired criteria.

3. Results

3.1. Data Description

The objective of this study was to predict big climate data using the ESDMOK in Java Island, Indonesia. The model was applied to secondary data obtained from the National Aeronautics and Space Administration (NASA) Prediction of Worldwide Energy Resources (POWER). The POWER project provides solar and meteorological data generated by NASA to support renewable energy, building energy efficiency, and agricultural needs. The POWER project started in 2003 as an outgrowth of Surface meteorology and Solar Energy (SSE). NASA-generated satellite data are essential in supporting researchers and the public in studying Earth’s climate and climate processes [24]. The POWER project provides long-term climatological mean estimates of meteorological data and solar energy flux surface data. In addition to these long-term climatological averages, daily data in the form of time series are also available. Solar data are based on satellite observations, and meteorological data are derived from the MERRA-2 assimilation model. These satellite and model-based products have proven to be quite accurate in providing reliable solar and meteorological resource data in regions where surface measurements could be more sparse or non-existent. The uncertainty estimates of POWER data are based on comparisons with measurement data [25,26,27].

In addition, POWER also provides high-resolution precipitation data derived from NASA’s Global Precipitation Measurement (GPM) mission’s Integrated Multi-satellite Retrievals for GPM (IMERG) with a resolution of 0.5° × 0.625° latitude–longitude grid (approximately 50 km) [28]. NASA POWER data can be downloaded free via the web at https://power.larc.nasa.gov/ (accessed on 5 July 2023). The data retrieval process began with inputting the latitude and longitude coordinates of the location and determining the time interval. In this study, the daily data interval was taken from 1 January 1982 to 5 July 2023 and was recorded in a daily data format. The selected climate variables included rainfall, air temperature, humidity, wind speed, solar irradiation, and surface pressure, along with latitude and longitude coordinate information. The output data obtained were stored in files of comma-separated value (.csv) format.

3.2. Data Analytics Life Cycle for the ESDMOK

In this research, rainfall prediction with ESDMOK uses the data analytics life cycle methodology shown in Figure 2. The process begins with formulating research problems, including natural disasters caused by rainfall, problem identification, climate variables affecting rainfall levels, and initial hypotheses based on theories that support ESDMOK. Furthermore, the data preparation stage includes determining the source of climate data to be analyzed and data pre-processing. The data collection process begins with inputting information on the location coordinates of 119 districts/cities on the Java Island, determining the observation time interval in the form of daily data, and selecting climate variables. In the data collection process, this research utilizes the application programming interface (API) by running the “pynasapower” package in Python. The data pre-processing stage includes; removing missing values, aggregating daily data to monthly data, selecting the climate variables. The model planning stage integrates location data with climate variables to provide input data for ESDMOK, such as spatial modelling. Furthermore, at the model development stage, based on the NASA POWER grid resolution of latitude-longitude 0.5° × 0.625° (approximately 50 km), 55 locations with the same data were generated. Thus, the locations were split into 64 observed locations and 55 unobserved locations, and the prediction stage was performed using ESDMOK. The communication results stage includes the model evaluation stage, which calculates accuracy using the MAPE calculation, the post-processing stage, which visualizes spatial mapping for rainfall prediction, and the interpretation of results to gain knowledge that can be used as a recommendation. The last stage is operationalization, which involves documenting research results and disseminating scientific papers in journals.

3.3. Framework ESDMOK for Prediction

Based on the gap analysis, ESDMOK is a spatial model that can predict unobserved locations and considers spatial dependencies in exogenous variables. Figure 3 shows the framework diagram for ESDMOK planning as part of the model-building stage. As mentioned in the data preparation stage, the process starts with inputting climate data sourced from NASA POWER. Next, in the data pre-processing stage, removing missing values (−999), aggregating daily data into monthly data, and selecting duplicate data. In this study, climate data from NASA POWER went through a pre-processing data stage using an R-Shiny web application, available at the following link: https://annisanurfalah.shinyapps.io/Pre-ProcessingData/ (accessed on 26 September 2023). The resulting pre-processed data were split into observed and unobserved locations, which can be seen in Appendix A. The observed locations were used as an input in the OK method for predicting climate data at unobserved locations. In this study, predictions at unobserved locations were calculated with an OK method using an R-Shiny web application, available at the following link: https://annisanurfalah.shinyapps.io/Ordinary-Point-Kriging/ (accessed on 7 May 2024). The integration results of climate data at observed and unobserved locations were used to construct an inverse distance weight matrix and spatial autocorrelation was assessed using the Moran Index and Scatterplot. If spatial autocorrelation is detected, the process continues with ESDM, using the Maximum Likelihood Estimation (MLE) method. Parameter estimates for ESDM are then calculated, followed by evaluating prediction accuracy using Mean Absolute Percentage Error (MAPE).The prediction calculations used the ESDM via an R-Shiny web application, available at the following link: https://andriyanafalah.shinyapps.io/SDM-Expansion/ (accessed on 11 June 2024). The prediction results are then processed by visualizing the spatial mapping in the form of web application-based maps, choropleth maps and providing interpretation to gain valuable insights.

3.4. Prediction Result of OK Method at Unobserved Locations

Semivariogram values were calculated based on all possible distance pairs, where the distance function used was the Euclidean distance, a function of the distance $h$, which describes the difference between the main variable and the difference in the additional variable $h$. Equation (1) was used to determine the semivariogram value and the number of distance pairs. The research data that were observed consisted of 64 districts/cities for six climate data categories, including rainfall, air temperature, humidity, wind speed, solar irradiation, and surface pressure. The calculation results of the experimental semivariogram values are shown in

Table 3. Experimental semivariogram values.

No.	The Number of Data Points with the Same Distance	Distance	Experimental Semivariogram Values
			Rainfall	Air Temperature	Humidity	Wind Speed	Solar Irradiation	Surface Pressure
1	9	16,986.956	12.612	0.865	1.904	0.186	0.033	2.442
2	40	32,999.848	68.209	1.389	4.282	0.358	0.074	3.495
3	76	54,226.021	72.565	1.429	5.398	0.232	0.107	3.252
4	99	74,327.202	90.908	1.789	5.820	0.285	0.144	4.465
5	96	96,082.271	168.067	2.390	8.202	0.400	0.133	5.657
6	109	117,949.815	159.668	2.880	9.046	0.327	0.180	7.131
7	94	138,289.058	169.586	2.650	8.612	0.386	0.204	5.904
8	90	159,416.873	219.595	2.116	9.571	0.299	0.295	4.560
9	86	181,276.821	265.071	3.191	10.495	0.388	0.297	7.804
10	76	202,391.4	335.187	2.184	9.989	0.399	0.264	4.687
11	83	223,437.78	377.226	2.491	12.527	0.346	0.340	5.015
12	72	244,803.453	371.607	2.246	11.461	0.280	0.419	4.565
13	84	268,191.079	510.885	1.918	12.129	0.429	0.482	3.515
14	55	287,619.435	534.357	3.059	18.205	0.300	0.424	5.342
15	63	309,682.512	581.690	2.848	14.276	0.359	0.488	6.496

.

The experimental semivariogram is used for fitting the theoretical semivariogram model. Reference to Table 1, the theoretical semivariogram model with varying sill and range values in each climate data set was fitted to the experimental semivariogram values. The theoretical semivariogram was fit for climate data, and the results are plotted in Appendix B. The sum square error (SSE) number was used to determine which theoretical semivariogram model was the best. The SSE values for the spherical, exponential, and Gaussian models are shown in

Table 4. Theoretical semivariogram for OK method.

SSE	Spherical	Exponential	Gaussian
Rainfall	0.0001093	0.0001135	8.86 × 10−5
Air Temperature	6.96 × 10−9	8.56 × 10−9	8.93 × 10−9
Humidity	1.35 × 10−7	8.59 × 10−8	6.28 × 10−8
Wind Speed	1.92 × 10−8	5.35 × 10−10	4.47 × 10−10
Solar Irradiation	6.76 × 10−10	8.82 × 10−11	4.27 × 10−11
Surface Pressure	7.11 × 10−8	7.82 × 10−8	7.33 × 10−8

. Based on the lowest SSE values, indicated by bold numbers, the optimal models for the rainfall, air temperature, humidity, solar irradiation, wind speed, and surface pressure are chosen.

Based on Table 4, the Gaussian semivariogram model had the minimum SSE value for rainfall and air temperature. Then, the exponential semivariogram model had the minimum SSE value for humidity, wind speed, solar irradiation and surface pressure. Therefore, these models were selected as the input for the calculation prediction of the OK method at unobserved locations (55 districts/cities). The integration results at observed locations (64 districts/cities) and unobserved locations are presented in the form of spatial mapping visualization, which can be seen in Figure 4. Climate variables in this data visualization include rainfall, air temperature, humidity, wind speed, solar irradiation, and surface pressure, which are variables in 119 districts/cities in Java Island, Indonesia. The data of these climate variables are represented in colours and bars, namely in several colours or one colour with different levels of intensity, to provide information on the level of value of the data. This data visualization aims to convey the value of climate variables so that data visualization can be classified into the explanation category. Still, with the addition of several interactive features and for easy use, this data visualization was made in the form of a web application. Data visualization in the form of spatial mapping based on web applications was developed using the Javascript programming language, HyperText Markup Language (HTML), and Cascade Style Sheets (CSS) style sheet language. Javascript was used to modify the display to handle data processing and data structures, while HTML and CSS were used for the content of the webpage, namely the layout and structure of the display. Several libraries were used in this data visualization, namely the leaflet library as a base map, map features such as overlay, zoom in, zoom out, and pan, and another library including highchart for data representation in the form of bar charts.

3.5. Prediction Result of the ESDM

The estimation of prediction parameters in the ESDM was conducted using an R-Shiny web application. An estimated $\widehat{\rho }$ value of 0.999 was obtained, producing an optimum spatial lag with a positive value $\left(\widehat{\rho }>0\right)$ and indicating spatial lag dependence. It signified the influence of adjacent locations within the Java Island region on rainfall prediction data. It shows that the phenomenon of rainfall on Java Island has a positive spatial autocorrelation, meaning that if districts/cities on Java Island have high rainfall, then other districts/cities around it have high rainfall as well or are not much different from the other districts/cities. The results of the parameter estimate calculation ${\widehat{\mathit{\beta }}}_0$ and $\widehat{\mathit{\theta }}$ are shown in

Table 5. Parameter-estimated value of SDM.

Coefficient	Parameter-Estimated Value
		${\widehat{\mathit{\beta }}}_0$	$\widehat{\mathit{\theta }}$
${\mathit{X}}_{\mathsf{1}}$ (air temperature)	${\widehat{\beta }}_{latitude}$	1.602	−9.262
	${\widehat{\beta }}_{longitude}$	2.963
${\mathit{X}}_{\mathsf{2}}$ (humidity)	${\widehat{\beta }}_{latitude}$	0.197	−3.725
	${\widehat{\beta }}_{longitude}$	0.427
${\mathit{X}}_{\mathsf{3}}$ (wind speed)	${\widehat{\beta }}_{latitude}$	0.838	−9.497
	${\widehat{\beta }}_{longitude}$	0.143
${\mathit{X}}_{\mathsf{4}}$ (solar irradiation)	${\widehat{\beta }}_{latitude}$	6.598	−0.499
	${\widehat{\beta }}_{longitude}$	−0.416
${\mathit{X}}_{\mathsf{5}}$ (surface pressure)	${\widehat{\beta }}_{latitude}$	−5.551	3.015
	${\widehat{\beta }}_{longitude}$	10.844

.

Base on Table 5, the estimate ${\widehat{\mathit{\beta }}}_0$ measures the direct impact of the exogenous variables on rainfall level in the same region, and the estimate $\widehat{\mathit{\theta }}$ captures the spillover effects of the exogenous variables on rainfall level. Based on the estimate ${\widehat{\mathit{\beta }}}_0$, we can obtain the estimate $\widehat{\mathit{\beta }}$ from which different parameter estimates were obtained for each exogenous variable in 119 districts/cities. It explains that the ESDM produces different parameter estimates for each exogenous variable at each location due to the expansion of the exogenous variable matrix involving latitude and longitude information at each location. The highest effect on the rainfall is given by the surface pressure and the lowest is humidity. The ESDM equation for each location can be found in Appendix C. A visualization of rainfall prediction in 119 districts/cities of Java Island is shown in Figure 5.

Based on Figure 5, the highest monthly rainfall predictions, with values above 220 mm, were in West Java, such as in Ciamis, Tasikmalaya, and Pangandaran, while the lowest monthly rainfall predictions, with values below 140 mm, were in East Java, such as in Probolinggo, Probolinggo City, Situbondo, and Bondowoso. The proposed model resulted in a MAPE value of 1.956%, indicating very accurate prediction.

4. Discussion

Data analytics life cycle methodology consists of six stages and is used to analyze big climate data sourced from NASA POWER. The initial step taken is formulating research problems regarding the impact of rainfall on natural disasters. Problem identification includes spatial dependencies on climate variables that affect rainfall on the island of Java, Indonesia and initial hypotheses based on the theory that supports ESDMOK. Data preparation begins with collecting climate variable data containing location coordinate information, determining the observation time interval in the form of daily data, and selecting six climate variables: rainfall, air temperature, humidity, wind speed, solar irradiation, and surface pressure. Furthermore, the data pre-processing stage includes cleaning missing values, aggregating daily data to monthly data, and selecting duplicate data. Model planning integrates location data with climate variables as inputs in ESDMOK. Model development is one of the main objectives of our proposed technique for rainfall prediction. In the model-building stage, climate variables were predicted at 55 unobserved locations using the OK method based on 64 observed locations. The OK method uses the experimental semivariogram to fit the theoretical semivariogram model. The best theoretical semivariogram models for fitting are Gaussian and exponential models, with minimum SSE. Furthermore, based on the spatial autocorrelation test, spatial dependency exists in both rainfall and exogenous variables, contributing significantly to the accuracy of prediction results using ESDM. Communication of results gives very accurate prediction results, as shown by MAPE of 1.956%. The surface pressure variable has the largest influence on rainfall, and the smallest is wind speed. Furthermore, the post-processing stage is a visualization of spatial mapping for rainfall prediction and the interpretation of results to gain knowledge that can be used as a recommendation.

The ESDMOK was applied to predict rainfall in 119 districts/cities in Java Island, Indonesia, which is influenced by exogenous variables in the form of climate variables. Rainfall prediction is important in climate change in accordance with goal 13 of the Sustainable Development Goals (SDGs) concerns climate action. The summary of this research underscored that the level of rainfall in each region, based on the data from 119 districts/cities in Java Island, Indonesia, was significantly influenced by other climate-variable factors, such as air temperature, humidity, solar irradiation, wind speed and surface pressure [7]. The data complexity implies the need for a more effective technique. The incorporation of deep learning approaches and leveraging of big data should be considered to further enhance the prediction and analysis of rainfall in the study region.

5. Conclusions

In conclusion, this study proposes an ESDM and an interpolation technique with the OK method to calculate predictions at unobserved locations. ESDMOK can be used for rainfall prediction in spatial dependence on exogenous variables. Our proposed technique can identify spatial rainfall patterns, capture spatial dependence between observation units within the region and incorporate relevant exogenous variables; the model improves rainfall prediction accuracy. The surface pressure variable effects the most significant influence of exogenous variables on rainfall, and the smallest is wind speed.

The results of this model support disaster mitigation, water resources management, and infrastructure development that is resilient to natural disasters. The prediction results of the ESDMOK in all districts and cities in Java Island can be used as a recommendation by the Meteorology Climatology and Geophysics Agency (BMKG), Indonesia, agribusiness companies, and the general public in improving agricultural planning and planting seasons and providing climate information for the general public, especially related to rainfall in areas that have Monsoonal patterns.

6. Patents

Granted Copy Right: Copy Right for Computer Program, number 000484474.

Entitled “Application of RShiny Program for Ordinary Point Kriging Method on Rainfall Data in West Java”, Ministry of Law and Human Rights of the Republic of Indonesia (Falah, A. N., Ruchjana, B. N., Abdullah, A. S., Rejito, J.), 2023. https://annisanurfalah.shinyapps.io/Ordinary-Point-Kriging/ (accessed on 7 May 2024).