A Spacetime RBF-Based DNNs for Solving Unsaturated Flow Problems

Abstract

This study presents a novel approach for modeling unsaturated flow using deep neural networks (DNNs) integrated with spacetime radial basis functions (RBFs). Traditional methods for simulating unsaturated flow often face challenges in computational efficiency and accuracy, particularly when dealing with nonlinear soil properties and complex boundary conditions. Our proposed model emphasizes the capabilities of DNNs in identifying complex patterns and the accuracy of spacetime RBFs in modeling spatiotemporal data. The training data comprise the initial data, boundary data, and radial distances used to construct the spacetime RBFs. The innovation of this approach is that it introduces spacetime RBFs, eliminating the need to discretize the governing equation of unsaturated flow and directly providing the solution of unsaturated flow across the entire time and space domain. Various error evaluation metrics are thoroughly assessed to validate the proposed method. This study examines a case where, despite incomplete initial and boundary data and noise contamination in the available boundary data, the solution of unsaturated flow can still be accurately determined. The model achieves RMSE, MAE, and MRE values of 10⁻⁴, 10⁻³, and 10⁻⁴, respectively, demonstrating that the proposed method is robust for solving unsaturated flow in soils, providing insights beyond those obtainable with traditional methods.

1. Introduction

Unsaturated flow in soils is essential in a wide range of engineering applications, especially in civil, environmental, and agricultural engineering [1,2,3]. Comprehending these physical behaviors is vital for effective water management across multiple disciplines. The integration of unsaturated flow principles into engineering practices leads to more sustainable, efficient, and resilient water management systems [4]. Solving unsaturated flow, as described by the Richards equation, involves identifying how unsaturated flow is distributed in both space and time [5,6,7,8]. This process typically employs advanced numerical techniques such as the discrete element method, finite element method, finite difference method, finite volume method, or meshfree methods [9,10,11,12].

Among these, meshfree methods are distinguished by their proficiency in handling complex geometries and irregular domains without necessitating meshing [13,14]. Prominent meshfree approaches include the Trefftz method, method of particular solutions, method of fundamental solutions, boundary node method, and radial basis function (RBF) collocation method. The flexibility and adaptability of meshfree methods make them highly suitable for simulating unsaturated flow problems. In particular, the RBF in meshfree methods offers distinct advantages that enhance their utility for numerical simulations. RBFs are mathematical functions whose value is based exclusively on the distance from a central point. Common types include multiquadric (MQ), inverse multiquadric (IMQ), polyharmonic spline, thin plate spline, Gaussian, and compactly supported RBFs [15,16,17]. Of particular significance, the shape parameter in RBFs holds pivotal importance in defining the influence radius of individual RBFs. This parameter governs the extent and configuration of the basis functions, thereby influencing the precision and stability of numerical simulations [18,19,20,21].

In recent years, there has been significant advancement in the development of simplified RBFs. This approach treats center points as source points situated outside the computational domain, eliminating the need for determining shape parameters while maintaining high-precision results [22]. Referred to as simplified RBF, this method has proven successful in solving a variety of problems, including Laplace equations, diffusion, Poisson equations, and Helmholtz equations, consistently achieving higher accuracy compared to traditional RBF methods [23,24]. Moreover, the simplified RBF approach can be integrated with artificial neural networks (ANNs) because of their ability to capture complex relationships between input and output variables [25]. ANNs are proficient at capturing nonlinear relationships and intricate patterns in simulated data, thus facilitating more precise modeling compared to conventional methods. Deep neural networks (DNNs) are a sophisticated subset of ANNs designed to model complex patterns and relationships in data through several layers of interconnected neurons [26,27,28,29]. DNNs have the capability to automatically unveil and extract hierarchical features from raw data, leading to superior performance across various domains compared to traditional shallow neural networks. An alternative DNN approach has been investigated as a more efficient and less complex solution compared to traditional numerical methods for solving differential equations [30]. This DNN captures the dynamics of oscillators without the need for computationally expensive simulations. By incorporating a modified neural structure with an oscillating activation function, the DNN effectively simulates both linear and nonlinear harmonic oscillators, delivering more precise solutions than conventional methods. Applied to the Van der Pol and Mathieu equations, the approach outperforms methods like Livermore Solver for Ordinary Differential Equations (LSODA), Adams–Bashforth, and backward differentiation formula (BDF) in terms of both accuracy and computational efficiency. Additionally, a DNN-based approach was designed to simulate the Sel’kov glycolysis model, effectively managing nonlinearity and stiffness and surpassing traditional methods in reliability and insight into biochemical systems [31]. Another DNN method, incorporating advanced activation functions, addresses nonlinear oscillations in microelectromechanical systems. This approach introduces the Amplifying Sine Unit (ASU) for enhanced performance, validated against LSODA [32].

This study introduces a spacetime RBF-based DNN approach to solve unsaturated flow problems by utilizing the capabilities of DNNs to approximate the Richards equation. However, the Richards equation exhibits significant nonlinearity due to the dependence of unsaturated hydraulic conductivity on pressure head. To address the nonlinearity of the Richards equation, we introduced a linearization process using the Gardner exponential model. The technique could demonstrate the proposed method’s capability in producing reliable solutions across a range of nonlinear scenarios. The structure of this study includes several sections. Section 1 provides an introduction, while Section 2 delves into the unsaturated flow problem. In Section 3, the development of DNNs with spacetime RBF is outlined, encompassing elements such as data collection, architecture design, model training, and optimization and tuning. Section 4 focuses on validation, investigating the impact of spacetime RBFs, collocation points, and dilation parameters on accuracy, and hyperparameter tuning, along with comparison analysis. Finally, Section 5 explores applications.

2. Unsaturated Flow Problem

To model the flow behavior in unsaturated soil, the variably saturated groundwater flow equation is utilized. This equation can be represented in three different forms: pressure head-based, water content-based, and mixed form. This study adopts the pressure head-based form of the variably saturated groundwater flow equation as the governing equation, defined as follows:

$${\displaystyle \frac{\partial {\theta }_m}{\partial h}}{\displaystyle \frac{\partial h}{\partial t}}={\displaystyle \frac{\partial }{\partial x}}\left[k_x(h){\displaystyle \frac{\partial h}{\partial x}}\right]+{\displaystyle \frac{\partial }{\partial y}}\left[k_y(h){\displaystyle \frac{\partial h}{\partial y}}\right]+{\displaystyle \frac{\partial }{\partial z}}\left[k_z(h){\displaystyle \frac{\partial h}{\partial z}}\right]+{\displaystyle \frac{\partial k_z(h)}{\partial z}},$$

(1)

where x is oriented along the ground surface, y is aligned with the tangent of the topographic contour at the origin, z is the vertical coordinate perpendicular to the xy plane, t denotes time, h denotes the pressure head, ${\theta }_m$ denotes the moisture content, $k_x(h)$, $k_y(h)$, and $k_z(h)$ represent the unsaturated hydraulic conductivity functions in the horizontal and vertical directions, respectively.

Equation (1) exhibits significant nonlinearity because unsaturated hydraulic conductivity is dependent on the pressure head. To address the Richards equation, three essential functions are required: the soil-water characteristic curve, the unsaturated hydraulic conductivity function, and the specific moisture capacity function. Assuming the unsaturated soils are isotropic and homogeneous, the governing equation for two-dimensional flow in these soils is represented by the following:

$${\displaystyle \frac{\partial {\theta }_m}{\partial t}}={\displaystyle \frac{\partial }{\partial x}}\left(k{\displaystyle \frac{\partial h}{\partial x}}\right)+{\displaystyle \frac{\partial }{\partial z}}\left[k({\displaystyle \frac{\partial h}{\partial z}}+1)\right].$$

(2)

The hydraulic conductivity of unsaturated soil is typically normalized relative to its maximum value and expressed as the relative permeability coefficient, defined as $k_r=k/K_S$. Therefore, the aforementioned governing equation can be rewritten as follows:

$${\displaystyle \frac{1}{K_s}}{\displaystyle \frac{\partial {\theta }_m}{\partial t}}={\displaystyle \frac{\partial }{\partial x}}\left(k_r{\displaystyle \frac{\partial h}{\partial x}}\right)+{\displaystyle \frac{\partial }{\partial z}}\left(k_r{\displaystyle \frac{\partial h}{\partial z}}\right)+{\displaystyle \frac{\partial k_r}{\partial z}},$$

(3)

where $k_r$ and $K_s$ denote the relative and saturated hydraulic conductivity, respectively. The aforementioned equation represents the two-dimensional Richards equation. Gardner [33] introduced a straightforward one-parameter exponential model to describe the relationship between volumetric water content and matric suction. By integrating Gardner’s exponential model into Equation (3), the linearized Richards equation can be derived.

$${\displaystyle \frac{{\alpha }_g({\theta }_s-{\theta }_r)}{K_s}}{\displaystyle \frac{\partial \overline{h}}{\partial t}}={\displaystyle \frac{{\partial }^2\overline{h}}{{{\partial }_x}^2}}+{\displaystyle \frac{{\partial }^2\overline{h}}{{{\partial }_z}^2}}+{\alpha }_g{\displaystyle \frac{\partial \overline{h}}{\partial z}},$$

(4)

where ${\alpha }_g$ represents the parameter associated with the distribution of soil’s pore size, $\overline{h}$ represents the linearized pressure head or matric suction, defined as $\overline{h}=e^{{\alpha }_{g}h}-e^{{\alpha }_g{h}_d}$, $h_d$ represents the linearized pressure head in dry soil conditions, and ${\theta }_s$ and ${\theta }_r$ represent residual and saturated water content, respectively. Equation (4) represents the linearized form of the Richards equation, which serves as the primary core governing equation in this study to describe the behavior of unsaturated flow. Since unsaturated flow is a transient behavior, initial and boundary conditions must be specified, as expressed in the following formula:

$$\overline{h}(x,z,t=T_0)=P_I,$$

(5)

$$\overline{h}(x,z,t)=P_B,$$

(6)

where $T_0$ represents initial time, $P_I$ represents the initial linearized pressure head or initial matric suction, and $P_B$ denotes the boundary value of linearized pressure head or initial matric suction.

3. Spacetime RBF-Based DNNs

This study employs a combination of the spacetime RBF and DNNs method to characterize the dynamics of unsaturated flow. The proposed method encompasses several pivotal stages, including data collection, the architecture design of DNNs with spacetime RBF, model training, and optimization and tuning.

The proposed spacetime RBF-based DNNs for solving the linearized Richards equation offer several advantages, including the capacity to learn complex patterns and effectively manage intricate boundary conditions. Although the Richards equation has been linearized, the behavior of unsaturated flow can remain complex. The proposed spacetime RBF-based DNNs excel at capturing and learning this complexity by modeling nonlinear relationships between inputs and outputs, enhancing the accuracy of the solution even for linearized systems. Additionally, the proposed DNNs can efficiently manage complex or irregular boundary conditions, which often pose challenges for traditional numerical solvers. By learning boundary behavior from data, the proposed DNNs can accurately approximate solutions without relying on complicated meshing or interpolation methods.

Figure 1 illustrates the structure of the proposed method. During the initial phase of the proposed approach, a comprehensive dataset relevant to the problem is assembled. This dataset may be sourced from diverse origins such as databases, sensors, or online repositories. The subsequent step entails designing the architecture of the proposed method. This involves determining the suitable neural network architecture for the problem at hand. In this study, DNNs are selected as the primary architecture. Following the architecture design, model training is conducted. Model training iteratively adjusts the network’s parameters, including biases and weights, to reduce a given loss function. The final stage involves model optimization and tuning. This process entails adjusting various hyperparameters and regularization techniques in the DNNs to enhance model performance and mitigate overfitting. The subsequent sections elaborate on each step in detail.

3.1. Data Collection

This dataset is crucial for training the neural network to solve the linearized Richards equation. The process entails gathering and structuring input–output pairs essential for training. Within this study, three simplified types of RBFs serve as foundational elements. These simplified RBFs eliminate the need for determining shape parameters or establishing central points. Instead, external source points situated beyond the spacetime computational domain are utilized as center points. The three variations of simplified RBFs, $\varphi \left(r_i\right)$, comprising simplified MQ, simplified IMQ, and simplified Gaussian functions, are expressed as follows:

$$\varphi \left(r_i\right)=r_i,$$

(7)

$$\varphi \left(r_i\right)={\displaystyle \frac{1}{r_i}},$$

(8)

$$\varphi \left(r_i\right)=e^{-{(r_i)}^2},$$

(9)

where $r_i$ is the spacetime distance, defined as $r_i=\left|x-x_i^s\right|$, and $x_i^s$ is the ith source point, defined as $x_i^s=(x_i^s,z_i^s,t_i^s)$. The aforementioned RBF serves as the input dataset for the proposed DNNs incorporating spacetime RBFs. Our method requires the placement of boundary and source points. Importantly, the simplified RBF introduced in this study integrates the concept of spacetime by treating time as an additional virtual spatial dimension. As a result, the original two-dimensional spatial coordinates, when extended by incorporating the one-dimensional time component, are transformed into three-dimensional spacetime coordinates. The above simplified RBFs serve as the training data for the DNN. Through Equations (7)–(9), it is shown that the spacetime distance includes expressions for both space and time.

Traditional DNNs generally rely on large existing datasets to build their training databases. The main innovation of this study is the use of RBFs as training data for the DNN. The formulas for these RBFs are outlined in Equations (7)–(9). This approach involves calculating the spacetime distance between boundary points and source points, which serves as the training data for the DNN, with spacetime distance incorporating both spatial and temporal expressions.

In the framework of the proposed spacetime RBF-based DNNs, the training dataset consists of boundary data obtained from accurate solutions and radial distances from external fictitious sources to boundary points. These distances are crucial for constructing RBFs. A notable advantage of the proposed DNNs is their deliberate avoidance of discretizing the governing equation. This distinctive approach allows the proposed DNNs to provide a straightforward solution to unsaturated flow problems, requiring only boundary data and the selected RBFs.

The study domain, $\Omega$, in this case features an irregular shape, with the boundaries, $\partial \Omega$, defined by the following equations, characterized as follows:

$$\partial \Omega =\left\{(x=x,z,t)|x=\rho (\theta )\cos \theta ,z=\rho (\theta )\sin \theta \right\}.$$

(10)

Furthermore, for the configuration of source points, a dilation parameter is introduced to maintain a specific distance between source points and boundary points, ensuring that the source points are arranged outside the boundary. The parametric equation utilized to define the outer boundary of the source point is as follows:

$$\partial {\Omega }^s=\left\{(x_i^s=x_j^s,z_j^s,t)|x_j^s=\eta {\rho }_j^s({\theta }_j^s)\cos ({\theta }_j^s),z_j^s=\eta {\rho }_j^s({\theta }_j^s)\sin ({\theta }_j^s)\right\},$$

(11)

where ${\rho }_j^s$ and ${\theta }_j^s$ denotes the radius and angle of source point, and $\eta$ represents the dilation parameter. The configuration of boundary points and source points is illustrated in Figure 2. Figure 2a presents the collocation pattern in the two-dimensional spatial domain, while Figure 2b shows the pattern in the three-dimensional spacetime domain. The source points are placed unevenly outside the boundary, and the time domain is uniformly distributed across each time step. Overall, after determining the configuration of boundary points and source points using Equations (10) and (11), one of the three simplified types of RBFs (Equations (7), (8), or (9)) can be used to select the simplified RBF. Upon completing this step, the input data for data collection are established.

Regarding the output data for data collection, the output data can be obtained either by utilizing the exact solution of the Equation (4), or by using known values of linearized pressure head or initial matric suction, as shown in Equations (5) and (6). If the exact solution of Equation (4) is chosen as the output data, it involves computing the corresponding exact solution values at selected boundary points on the computational domain’s boundary. If known values of unsaturated flow data are used, it entails directly using the known distribution of unsaturated flow and corresponding data values at the boundary locations.

To combine the spacetime RBF and DNN, the most important aspect of this study is the introduction of the RBF as training data for the DNN, including both inputs and outputs. The information provided for the inputs and outputs is closely related to the collocation points in the study area, which includes the arrangement of boundary points and source points.

Using the simplified RBFs derived from Equations (7)–(9) as input data, along with their corresponding solution values as output data, facilitates the construction of training data for the DNNs. These three types of simplified RBFs employed in this investigation necessitate only boundary points and source points for acquiring the RBFs. Moreover, these simplified RBFs introduce innovative characteristics by amalgamating spacetime coordinates, treating time akin to a virtual spatial dimension. For instance, in the context of a two-dimensional unsaturated flow problem, encompassing two spatial dimensions and one temporal dimension, the introduction of spacetime coordinates can be conceptualized as a three-dimensional spacetime coordinate system. Consequently, both source and boundary points are situated within this three-dimensional spacetime coordinate framework.

Subsequently, the dataset is divided into training, validation, and test subsets. Typically, 70% of the data is used for training, while the remaining 30% is equally distributed between validation and testing.

3.2. Architecture Design of Proposed Method

The proposed model architecture design involves defining the structure, dimensions, and configuration of the neural network in the proposed method. This process encompasses various crucial elements, such as selecting the network type, arranging the layers, specifying the input and output layers, and optimizing hyperparameters. During network type selection, this study identifies the appropriate architecture by choosing the neural network model best suited for the problem, particularly focusing on DNNs. Subsequently, in constructing the architecture design of DNNs with spacetime RBF, the layer configuration phase entails defining the number of neurons in each layer, the number of layers, and the activation functions to be used. Additionally, hyperparameter selection entails initializing key parameters such as optimization algorithms, batch size, learning rate, and epoch count.

Initially, the DNNs architecture is constructed by integrating weights, which represent the linear combination of inputs and biases. This method is applied to compute the activation value for each neuron in the hidden layer of the neural network. The process involves calculating the weighted sum of the neuron’s inputs and then using an activation function to generate the output. This can be outlined as follows:

$$S_j={\beta }_j+{\displaystyle \sum _{i=1}^I[{\lambda }_{ij}\varphi (r_i\left)\right]},$$

(12)

where $S_j$ is the weighted sum, ${\beta }_j$ is the bias, I is the aggregate input elements originating from the input layer, ${\lambda }_{ij}$ is the weight in the hidden layer, and j is the output of neuron. As described in Equation (12), the RBF is used as the training data for the DNN. This data includes boundary values derived from exact solutions and the radial distances between external fictitious sources and boundary points, which are utilized to construct the RBFs. The architecture of the DNN in Equation (12) is first established by incorporating weights, representing the linear combination of inputs and biases. The proposed DNN is applied to calculate the activation value for each neuron in the hidden layer of the neural network. The process involves computing the weighted sum of the neuron’s inputs, followed by applying an activation function to produce the output.

To determine each neuron’s value in the hidden layer, the dot product between its weights and the associated input values is calculated, as detailed in Equation (12). This value is then passed through an activation function to obtain the final output. The estimated value of each neuron is then transmitted to the neurons in the succeeding layers, a process that can be illustrated as follows:

$$S_{ij}={\beta }_{ij}+{\displaystyle \sum _{j=1}^J{\displaystyle \sum _{i=1}^I{\mu }_{ij}f(S_j)}},$$

(13)

where $S_{ij}$ is the total weighted sum, ${\beta }_{ij}$ is the bias, ${\mu }_{ij}$ is the weight, J is the neuron count, and $f(S_j)$ is the activation function. The subsequent method employed in this study to obtain numerical solutions is to utilize the procedures outlined in the suggested architecture of the proposed method, resulting in the following:

$${\overline{h}}_{net}={\displaystyle \sum _{j=1}^J{\upsilon }_{j}f(S_{ij})},$$

(14)

where ${\overline{h}}_{net}$ represents the approximations of the proposed method, and ${\upsilon }_j$ represents the weights associated with the output layer. Overall, to address unsaturated flow problems, the process initiates with the generation of weights, which are then linearly combined with inputs and a bias term, as specified in Equations (12) to (14). For a hidden layer with several neurons, the computation for each neuron j in the hidden layer is represented by Equation (12). The weighted sum in Equation (12) acts as the input to the neuron’s activation function, and the output of this function represents the neuron’s final activation value. This weighted sum is obtained by combining the inputs with their corresponding weights and biases for a single neuron. In the hidden layer, multiple neurons perform their own individual weighted sum calculations and activations. Subsequently, the combination as described in Equation (12) is used to compute the activations in the hidden layer. The activations from the first hidden layer are then fed into neuron j in subsequent hidden layers, where further processing takes place to produce the output layer, as shown in Equation (13). Ultimately, the goal of this study is to obtain an approximate solution to the unsaturated flow problems, as denoted by Equation (14).

3.3. Model Training

Training a DNN involves repeatedly modifying the network’s parameters, such as weights and biases, in order to reduce a specified loss function. The process of training DNNs includes initialization, loss calculation, backpropagation, weight updates, validation, epochs, batch training, and model evaluation. By iteratively adjusting the model parameters based on observed errors, DNNs gradually learn to map input data to desired outputs, effectively capturing complex relationships and patterns in the data.

The first step in model training is to initialize the biases and weights of the DNNs with random values or predefined initializations. Next, the predicted output is compared with the actual output using a loss function. Subsequently, backpropagation is conducted in this study. This phase involves computing the gradients of the loss function concerning the network’s parameters by applying the chain rule of calculus. In this procedure, gradients are propagated backward through the network, and the weights and biases of each layer are iteratively adjusted using an optimization algorithm to minimize the loss function.

Common optimization algorithms include Levenberg–Marquardt (LM), gradient descent (GD), and Broyden–Fletcher–Goldfarb–Shanno (BFGS). The LM algorithm is a widely used optimization method, especially in curve fitting and neural network training. It is an iterative technique that merges the concepts of gradient descent and the Gauss–Newton method. The LM algorithm dynamically adjusts the step size, switching between gradient descent for large residuals and the Gauss–Newton method for smaller ones, providing a robust and efficient approach to finding the optimal solution. GD is a fundamental optimization algorithm utilized for minimizing functions by iteratively moving in the direction of the steepest descent, as determined by the negative gradient. The GD algorithm updates parameters in proportion to the negative gradient of the function at the current point. There are multiple variants of gradient descent, including batch gradient descent, mini-batch gradient descent, and stochastic gradient descent (SGD), each differing in the number of samples used to compute the gradient. Gradient descent is widely utilized in machine learning and deep learning for training models by minimizing the loss function. The BFGS algorithm is an advanced quasi-Newton approach employed for addressing unconstrained nonlinear optimization problems. In contrast to the standard Newton’s method, the BFGS algorithm does not necessitate the computation of second-order derivatives (Hessian matrix). Instead, it approximates the Hessian matrix iteratively using first-order gradient information, making it more efficient and suitable for large-scale optimization problems. BFGS is known for its rapid convergence and stability, making it a preferred choice in various scientific and engineering applications, including machine learning optimization.

Given that each of these algorithms has its strengths and is suitable for different applications, offering various approaches to effectively tackle optimization problems, this study employs a comparison of the three optimization algorithms—LM, GD, and BFGS—as the optimizers. The algorithm iteratively updates the network’s parameters, including biases and weights, using an optimization algorithm. The learning rate dictates the magnitude of each step taken during the optimization process.

The next phase is validation, where the model’s performance is periodically assessed using a validation dataset. This study uses four different metrics to evaluate the accuracy of the proposed method in the case studies: root mean square error (RMSE), Nash–Sutcliffe efficiency (NSE), maximum relative error (MRE), and maximum absolute error (MAE).

$$\mathrm{RMSE}=\sqrt{{\displaystyle \frac{{\displaystyle \sum _{i=1}^N{({\widehat{h}}_i-{\overline{h}}_{net,i})}^2}}{N}}},$$

(15)

$$\mathrm{NSE}=1-{\displaystyle \frac{{\displaystyle \sum _{i=1}^N{({\widehat{h}}_i-{\overline{h}}_{net,i})}^2}}{{\displaystyle \sum _{i=1}^N{({\widehat{h}}_i-{\tilde{h}}_i)}^2}}},$$

(16)

$$\mathrm{MRE}=\max \left({\displaystyle \frac{\left|{\widehat{h}}_i-{\overline{h}}_{net,i}\right|}{\left|{\widehat{h}}_i\right|}}\right),$$

(17)

$$\mathrm{MAE}=\max \left|{\widehat{h}}_i-{\overline{h}}_{net,i}\right|,$$

(18)

where N represents the data points number, ${\widehat{h}}_i$ represents the actual value or exact solution, ${\tilde{h}}_i$ represents the mean of the actual values, and ${\overline{h}}_{net,i}$ represents the predicted linearized pressure head using the proposed method.

Additionally, hyperparameters, including the learning rate, batch size, regularization, etc., are adjusted based on the validation performance. Throughout the research process, the forward pass, loss calculation, backpropagation, and weight update steps are repeated for a predefined number of epochs, representing complete passes through the entire training dataset. After training is completed, this study evaluates the trained model on a separate test dataset to assess its performance on unseen data.

3.4. Optimization and Tuning

Optimization and fine-tuning in DNNs involve modifying various hyperparameters and regularization techniques to improve model performance and prevent overfitting. In this study, the DNNs with spacetime RBFs, designed for solving the unsaturated flow problem, undergo experimentation with different hyperparameters and optimization algorithms to optimize training performance. Moreover, adjustments are made to the learning rate schedule (e.g., utilizing learning rate decay) to refine training dynamics. The process of optimization and tuning in DNNs is iterative, requiring experimentation and thoughtful consideration of various hyperparameters and techniques to develop robust and effective models. In the computation process, spacetime distances between source and boundary points, as well as their associated unsaturated flow conditions, are utilized to refine and optimize the training of the proposed method.

4. Validation Example

In the validation case, we initially investigate two-dimensional transient flow in unsaturated soil, utilizing Equation (4) as the governing equation. The study domain in this case features an irregular shape defined by the Equation (10), where $\rho (\theta )={\cos }^2(3\theta )e^{\sin (2\theta )}+{\sin }^2(3\theta )e^{\cos (2\theta )}$. Due to the forward nature of this study, initial and boundary conditions need to be specified, expressed as follows.

$$\overline{h}(x,z,t=T_0)=x{e}^{\frac{-{\alpha }_{g}z}{2}},$$

(19)

$$\overline{h}(x,z,t)=x{e}^{\frac{-{\alpha }_g^{2}t}{4c}}{e}^{\frac{-{\alpha }_{g}z}{2}}.$$

(20)

The above boundary conditions are defined as Dirichlet conditions, and the Dirichlet boundary data are implemented using the exact solution [34]. The exact solution, presented in Equation (20) and referenced from [34], is employed to validate the numerical results generated by the proposed DNN.

The parameters including ${\alpha }_g$, $K_s$, ${\theta }_s$, ${\theta }_r$, and $T_0$ are 2 × 10⁻⁵, 10⁻⁴, 0.35, 0.14, and 5 respectively. The total calculation time for this study is 5. This study utilizes Equation (11) for the configuration of the source points, where ${\rho }_j^s({\theta }_j^s)=2[1+{\cos }^2(4\theta \left)\right].$ The configuration of boundary points and source points in this case is illustrated in Figure 2. Figure 2a presents the collocation pattern in the two-dimensional spatial domain, while Figure 2b shows the pattern in the three-dimensional spacetime domain. The source points are placed unevenly outside the boundary, and the time domain is uniformly distributed across each time step. For this analysis, 90 source points and 1550 boundary points are employed.

In the proposed method, the dataset is divided into three categories: 70% allocated for training, with the remaining 30% split equally between testing and validation. Specifically, 70% of the dataset equates to 1084 data points, while 15% corresponds to 233 data points each for testing and validation. The network consists of two hidden layers, with the LM algorithm employed for optimization. The number of epochs is set to 10³, with a performance goal of 0 and a minimum performance gradient of 10⁻⁷. The initial damping parameter is 10⁻³. When updating the damping parameter, a decreasing factor of 0.1 and an increasing factor of 10 are used, with an upper limit for the damping parameter set at 10¹⁰. The number of neurons in each hidden layer in the architecture of DNNs is 13. The setting of training parameters for the proposed method is summarized in

Table 1. Setting of training parameters for the proposed method.

Unit	Initial Value	Stopping Criteria	Target Value
Epoch	0	103	103
Performance	11.7	1.15 × 10−13	0
Elapsed time	-	207 s	-
mu	10−3	10−11	1010
Gradient	38.8	1.52 × 10−7	10−15
Validation checks	0	0	2 × 103

. As listed in Table 1, mu can be defined as either the learning rate or the momentum parameter. The learning rate governs the extent to which the network weights are updated in response to the gradient of the loss function. It dictates the size of the steps taken toward minimizing the loss function during training. Momentum, on the other hand, enhances gradient descent by accelerating it in the appropriate direction and reducing oscillations. It achieves this by updating the weights based on a blend of the current gradient and the previous weight adjustments. As this model includes multiple parameter settings that significantly affect solution accuracy, the subsequent analysis investigates the effects of collocation points, the dilation parameter, the hidden layer number, and hyperparameter tuning on accuracy to determine the optimal parameter settings for the proposed method.

4.1. Evaluation of Spacetime RBFs on Accuracy

The DNNs architecture in this study can utilize three types of spacetime RBFs, including simplified MQ, simplified IMQ, and simplified Gaussian functions. This case first tests the impact of using these three types of spacetime RBFs on the solution accuracy. The parameters used for the three types of spacetime RBFs in this case are shown in Table 1, and the accuracy during the testing phase are presented in

Table 2. Accuracy during the testing phase for three types of spacetime RBFs over 20 runs.

Basis Function	Accuracy
	RMSE	NSE	MRE	MAE
Simplified MQ	4.87 × 10−6	1	3.52 × 10−6	3.41 × 10−5
Simplified IMQ	4.23 × 10−5	1	7.13 × 10−5	6.91 × 10−4
Simplified Gaussian	1.78 × 10−4	1	1.69 × 10−4	1.6 × 10−3

. According to Table 2, the RMSE, NSE, MRE, and MAE when using the simplified MQ can achieve accuracies of 10⁻⁶, 1, 10⁻⁶, and 10⁻⁵, respectively, which are significantly higher than those obtained with the simplified IMQ and simplified Gaussian functions.

The results of this study, which employs DNNs combined with three types of spacetime RBFs for training, validation, and testing datasets, are illustrated in Figure 3. The training performance of DNNs, combined with all three types of spacetime MQ RBFs, is commendable. However, in terms of validation data, the highest accuracy is achieved with DNNs paired with spacetime MQ RBF. This indicates that the proposed model’s predictions closely match the actual unsaturated flow analysis results. Consequently, the development of the DNN architecture in this study incorporates the simplified MQ RBF and proceeds with further hyperparameter tuning.

This study has further detailed the precise values used in conjunction with DNNs paired with spacetime MQ RBF, as shown in Figure 4. The figure clearly illustrates that the numerical solution derived from the proposed DNN with spacetime MQ RBF aligns closely with the exact solution, thereby validating the accuracy and reliability of the proposed approach.

4.2. Evaluation of Collocation Point Number on Accuracy

As the proposed method depends on integrating the collocation method to generate a dataset for model training through the arrangement of boundary and source points, efficient data collection is essential. The arrangement of these points is pivotal for the accuracy of model training. Thus, this study conducts a convergence analysis on the number of collocation points, encompassing both boundary points and source points.

In this convergence analysis, the boundary points are uniformly distributed along the three-dimensional spacetime coordinate system, while the source points are arranged irregularly and helically outside the boundary points in the three-dimensional spacetime coordinate system. This study first examines the impact of varying the number of boundary points from 320 to 2150, while keeping the number of source points constant at 80. The effect of boundary point quantity on computational results is analyzed, as depicted in Figure 5a. This figure illustrates the results between MAE and the number of boundary points, demonstrating that an MAE ranging from 10⁻⁴ to 10⁻⁵ is achievable when the number of boundary collocation points is between 334 and 2150.

This study also investigates how varying the number of source points from 6 to 122 affects the results, with the number of boundary points fixed at 950. Figure 5b depicts this analysis, showing the results between MAE and the number of source points. It demonstrates that an MAE of 10⁻⁵ can be achieved when the number of source points ranges from 14 to 122. When the MAE is close to 10⁻⁵, it means that the average magnitude of the errors between the predicted values and the actual values is on the order of 10⁻⁵. Practically, an MAE of 10⁻⁵ signifies that the prediction model is highly accurate, with errors being very minimal. This level of MAE means the predictions are very close to the actual values. Additionally, it suggests that the model’s error magnitude is on par with the accuracy of the calculation or measurement system used. Such an MAE level demonstrates that the model performs effectively within the precision limits of the system.to the accuracy of the system used for calculations or measurements. This level of MAE demonstrates that the model is performing well within the limits of the system’s precision. Based on the results of this convergence analysis, the number of boundary and source points is specified as 1550 and 80, respectively, for subsequent analyses.

4.3. Evaluation of Location of Collocation Point on Accuracy

This study utilizes collocation techniques by strategically positioning boundary and source points. The boundary points are set at a specific distance from the source points, known as the dilation parameter. Adjusting the dilation parameter modifies the distance between source and boundary points and influences solution accuracy. Consequently, the study examines the effect of the dilation parameter on the analysis results. Figure 6a illustrates the influence of the dilation parameter on solution accuracy, indicating that an MAE ranging from 10⁻⁵ to 10⁻⁶ is obtained when the dilation parameter ranges from 1 to 20. Consequently, a dilation parameter of 14 is chosen for following analysis.

4.4. Hyperparameter Tuning and Comparison Analysis

4.4.1. Influence of Hidden Layer Quantity on Accuracy

The number of hidden layers using the proposed method with spacetime MQ RBF can profoundly impact the outcome of the analysis. Increasing the count of hidden layers may potentially enable the network to grasp intricate patterns and correlations within the data, enhancing performance in specific scenarios. However, an excessive number of hidden layers could lead to overfitting, where the model memorizes the training data rather than generalizing effectively to new data. In contrast, having a limited number of hidden layers can lead to underfitting, where the model fails to sufficiently represent the underlying data patterns. Hence, determining the optimal number of hidden layer entails considering the problem’s complexity, available dataset size, and other variables. It often necessitates experimentation and validation to ascertain the most suitable configuration.

This study varied the number of hidden layers from 1 to 20 to investigate their effect on MAE. The analysis results, as shown in Figure 6b, indicate that with two or fewer hidden layers, only an MAE in the range of 10⁻³ to 10⁻⁴ can be achieved. However, with three to twenty hidden layers, an MAE of 10⁻⁵ is consistently attained. Therefore, based on the analysis results, the study proceeds with 13 hidden layers for subsequent analyses.

4.4.2. Influence of Optimization Function on Accuracy

The optimization function is crucial in determining the efficiency of training the proposed method, influencing its accuracy, convergence, and overall performance. Since the choice of optimization function can greatly affect the accuracy of the proposed method, this study compares various optimization algorithms, including LM, GD, and BFGS, as summarized in

Table 3. Accuracy during the testing phase for various optimization functions across 20 runs.

Optimization Function	Accuracy
	RMSE	NSE	MRE	MAE
LM	2.14 × 10−6	1	8.12 × 10−7	7.75 × 10−6
GD	4.36 × 10−2	0.99	1.31 × 10−2	1.29 × 10−1
BFGS	3.91 × 10−5	1	1.41 × 10−5	1.38 × 10−4

.

This study assesses the performance of three optimization functions using metrics such as RMSE, NSE, MRE, and MAE. The findings indicate that the LM optimization function delivers superior accuracy across all metrics compared to the other two functions. Specifically, with LM, the RMSE, NSE, MRE, and MAE are 10⁻⁶, 1, 10⁻⁷, and 10⁻⁶, respectively. The BFGS optimization function achieves RMSE, NSE, MRE, and MAE values of 10⁻⁵, 1, 10⁻⁵, and 10⁻⁴, respectively. In contrast, the GD optimization function yields less accurate results, with RMSE, NSE, MRE, and MAE values of 10⁻², 0.99, 10⁻², and 10⁻¹, respectively. Figure 7 demonstrates the performance of the proposed method with spacetime MQ RBF using various optimization functions. Findings reveal that LM delivers superior performance overall, with BFGS being the second most effective and GD performing the least favorably. Therefore, LM is chosen as the optimization algorithm for the proposed neural network.

4.4.3. Influence of Loss Function on Accuracy

The selection of a loss function greatly impacts the performance of DNNs, influencing training behavior, convergence speed, and the overall accuracy and generalization of the model. It is essential to choose a suitable loss function tailored to the specific problem and dataset to attain the best results. This study explores different loss functions, including mean squared error, cross-entropy, and mean absolute error, to evaluate their effects.

The mean squared error is a standard loss function for regression tasks, quantifying the average squared deviation between predicted and actual values. Mean squared error is particularly sensitive to outliers because errors are squared. Alternatively, the mean absolute error is also used for regression, measuring the average absolute difference between actual and predicted values. MAE is less affected by outliers compared to mean squared error. For multi-class classification problems, cross-entropy loss is applied. It evaluates the divergence between the true label distribution and the predicted probabilities and is particularly effective when the output layer uses a Softmax activation function.

Table 4. Accuracy during the testing phase for various loss functions across 20 runs.

Loss Function	Accuracy
	RMSE	NSE	MRE	MAE
Mean squared error	2.10 × 10−6	1	8.00 × 10−7	7.64 × 10−6
Mean absolute error	1.56 × 10−6	1	5.55 × 10−7	5.37 × 10−6
Cross-entropy	5.00 × 10−5	1	2.09 × 10−5	2.13 × 10−4

provides a summary of the performance results for three different loss functions across 20 runs during the testing phase. The results demonstrate that all three loss functions perform well across various metrics. Specifically, each loss function achieves RMSE, NSE, MRE, and MAE values of 10⁻⁶, 1, 10⁻⁷, and 10⁻⁶, respectively. Based on these results, the study uses mean squared error as the loss function for further analysis, as it is commonly used in the proposed method.

4.4.4. Influence of Activation Function on Accuracy

The choice of activation function can significantly impact the performance and efficiency of DNNs. Selecting the appropriate activation function for the specific problem and network architecture is essential for achieving optimal results. This study further investigates the impact of using different activation functions on accuracy. The activation functions examined in this study include sigmoid, rectified linear unit (ReLU), and hyperbolic tangent (Tanh) functions.

The sigmoid activation function features a smooth, S-shaped curve and can lead to the vanishing gradient problem, where gradients become very small during backpropagation, thus slowing down learning. The ReLU activation function is commonly used in the hidden layers of deep networks due to its simplicity and efficiency. It introduces non-linearity while maintaining computational efficiency. Conversely, the Tanh activation function is commonly used in the hidden layers of neural networks. Its zero-centered nature aids in mitigating the vanishing gradient issue more effectively compared to the sigmoid function. However, it may still encounter vanishing gradients in very deep networks.

Table 5. Accuracy during the testing phase for various activation functions across 20 runs.

Activation Function	Accuracy
	RMSE	NSE	MRE	MAE
Sigmoid	2.14 × 10−6	1	8.12 × 10−7	7.75 × 10−6
ReLU	2.17 × 10−6	1	8.01 × 10−7	7.67 × 10−6
Tanh	2.16 × 10−6	1	7.74 × 10−7	7.88 × 10−6

summarizes a comparison of the results from the testing phase for three activation functions over 20 runs. The results illustrate that all three activation functions—sigmoid, ReLU, and Tanh—perform well across various performance metrics. Specifically, all three functions yield RMSE, NSE, MRE, and MAE values of 10⁻⁶, 1, 10⁻⁷, and 10⁻⁶, respectively. Based on these findings, this study proceeds with using the sigmoid activation function, which is widely adopted for the proposed neural networks in further analysis.

5. Applications

5.1. Two-Dimensional Green–Ampt Problem

The application of the infiltration problem in unsaturated soil is examined. The governing equation used to describe water infiltration into unsaturated soils is the Green–Ampt equation. This equation relies on simplifying assumptions that make the complex infiltration process more manageable to describe.

This study focuses on the Green–Ampt problem, with the governing equation represented as Equation (4). Figure 8 depicts the arrangement of collocation points in unsaturated soil with specified dimensions of 1 m in length (a) and 1 m in height (L). The Green–Ampt scenario simulates the infiltration of rainwater into unsaturated soil. Initially, the unsaturated soil is dry, and as time progresses, rainwater gradually infiltrates the unsaturated soil. The analysis evaluates the distribution of unsaturated flow. To simulate this infiltration behavior, the boundary conditions are defined as follows: the top of the unsaturated soil is set as a constant head boundary with a pressure head of zero. The left, right, and bottom boundaries are set to a dry condition with a pressure head of −1. This setup models the infiltration process from the top into unsaturated soil, where the infiltration rate quickly diminishes to reach a dry state.

The exact solution applied to model the aforementioned infiltration scenario is expressed as follows [35,36]:

$${\overline{h}}_t(x,z,t)=(1-e^{{\alpha }_g{h}_d})\sin ({\displaystyle \frac{\pi x}{a}})e^{\frac{{\alpha }_g(L-z)}{2}}{\displaystyle \frac{2K_s}{L{\alpha }_g({\theta }_s-{\theta }_r)}}{\displaystyle \sum _{k=1}^m{(-1)}^k(\frac{{\lambda }_k}{{\gamma }_{ik}})}\sin ({\lambda }_{k}z)e^{-{\gamma }_{ik}t},$$

(21)

where ${\overline{h}}_t$ represents the transient linearized pressure head, L represents the height of the unsaturated soil, $h_d$ represents the dry pressure head, which is set to 1, a represents the length of the unsaturated soil, k is the non-negative integer, ${\lambda }_k$ is a known value defined as ${\lambda }_k=\pi k/L_k$, $L_k$ is the characteristic length, ${\gamma }_{ik}$ is a known value defined as ${\gamma }_{ik}=K_s({\beta }_i^2+{\lambda }_k^2)/{\alpha }_g({\theta }_s-{\theta }_r),$ ${\beta }_i$ is a known value defined as ${\beta }_i=\sqrt{{\alpha }_g^2/4+{\lambda }_i^2}$, ${\lambda }_i$ is a known value defined as ${\lambda }_i=\pi i/L_i$, i represents the positive integer, and $L_i$ represents the characteristic length. In this case, $m=10$, $L_i=1$, and $L_k=1$. The unsaturated soil parameters, including ${\alpha }_g$, $K_s$, ${\theta }_s$, and ${\theta }_r$, are 2 × 10⁻⁵, 10⁻⁴, 0.35, and 0.14, respectively. The total simulation time is one hour.

Finally, the linearized pressure head can be converted to the actual pressure head using the following equation:

$$h={\displaystyle \frac{1}{{\alpha }_g}}\ln \left[{\overline{h}}_t+(1-e^{{\alpha }_g{h}_d})\sin ({\displaystyle \frac{\pi x}{L_i}})e^{\frac{{\alpha }_g}{2}(L_k-z)}{\displaystyle \frac{\sinh ({\beta }_{i}z)}{\sinh ({\beta }_i{L}_k)}}+e^{{\alpha }_g{h}_d}\right].$$

(22)

The initial condition is set by placing boundary points along the edges and bottom of the three-dimensional spacetime coordinate system. Source points, located outside this cubic region, are arranged in a scattered, spiral manner. For solving the two-dimensional Green–Ampt problem using the proposed method, the configuration includes 119 to 137 source points and 1070 boundary points. Previous validation results showed that combining a DNN with spacetime MQ RBF provided the highest accuracy; hence, this application exclusively utilizes the proposed approach for analysis.

In this study, the proposed spacetime RBF-based DNNs architecture divides the dataset into three parts: 70% is allocated for training, while the remaining 30% is split equally between validation and testing, with each subset comprising 15% of the data for analysis. The training dataset consists of 193 samples, while both the validation and test datasets each contain 65 samples. To evaluate the distribution of unsaturated flow over time, this study employed 4056 uniformly spaced internal points within the cubic spacetime domain to represent pressure head distributions at different time intervals.

Figure 9 demonstrates the performance for the two-dimensional Green-Ampt equation. From the results of the training state of the proposed approach, as shown in Figure 9a, the iteration count was configured to a maximum of 1000, with termination occurring at 853 iterations. The optimal validation mean squared error was 5.96 × 10⁻¹⁴, the optimal test mean squared error was 7.53 × 10⁻¹⁴, and the optimal training mean squared error was 7.31 × 10⁻¹⁵.

Moreover, the results of the neural network training regression, depicted in Figure 9b, demonstrate that the pressure head distributions derived from the proposed method with spacetime MQ RBF align closely with the analytical solution provided by Equation (21). Error metrics reveal that the RMSE, NSE, MRE, and MAE values are on the order of 10⁻², 10⁻², 10⁻³, and 10⁻¹, respectively. These results highlight the high accuracy of the proposed method, showcasing its effectiveness in addressing the Green–Ampt problem. The findings confirm that the proposed spacetime RBF-based DNNs offer a highly accurate solution for the two-dimensional Green–Ampt problem.

5.2. Two-Dimensional Inverse Richards Equation

The inverse Richards equation involves determining the soil hydraulic properties or initial conditions from known measurements of pressure head or moisture content over time and space. In the context of unsaturated flow, this typically means inferring properties such as the soil’s hydraulic conductivity and moisture retention characteristics. Inverse problems are often ill-posed, meaning they can have multiple solutions or be sensitive to measurement errors, with small changes in input data leading to significant variations in the estimated parameters. Additionally, the accuracy of the inverse solution is highly dependent on the initial and boundary conditions used in the simulation. Incorrect assumptions or data can result in erroneous estimates.

The inverse Richards problem aims to reconstruct the historical distribution of unsaturated pressure head from current or final data. This problem is represented by Equation (4), and the computational domain is described by Equation (10), where $\rho (\theta )=\sqrt{\cos (4\theta )+\sqrt{3-\sin (8\theta )}}$. In this study, the parameters including ${\alpha }_g$, $K_s$, ${\theta }_s$, ${\theta }_r$, and $T_0$ are 2 × 10⁻⁵, 10⁻⁴, 0.35, 0.14, and 5 respectively. The entire simulation runs for 5 h. The spacetime boundary condition in this scenario is specified by the analytical solution [34] as follows:

$$\overline{h}(x,z,t)=x{e}^{\frac{-{\alpha }_g^{2}t}{4c}}{e}^{\frac{-{\alpha }_{g}z}{2}}.$$

(23)

This study considers four different cases. The differences between these cases lie in the range of given boundary conditions. In Case A, both the initial conditions and the full set of boundary conditions are specified. In Case B, only the boundary conditions over time are given, while the initial conditions are unknown. In Case C, only partial information is provided for both initial and boundary conditions. In Case D, partial boundary conditions are considered, and the initial conditions are unknown. Based on the scenarios described above, this study places boundary points according to the given boundary conditions, as shown in Figure 10.

For Case A, Case B, Case C, and Case D, the boundary points number 1582, 1410, 766, and 720, respectively, and these points are evenly distributed according to the positions shown in Figure 10. Additionally, the number of source points in all four cases ranges from 645 to 650. These source points are irregularly distributed outside the boundary points. This study employs a DNN architecture with spacetime RBF to simulate the inverse Richards equation. The proposed method divides the data into three segments: 70% for training, 15% for validation, and 15% for testing.

The network features two hidden layers optimized using the LM algorithm. The training process is set to run for 10³ epochs, targeting a performance goal of 0 and a minimum performance gradient of 10⁻⁷. The initial damping parameter is 10⁻³, with an update strategy involving an increase factor of 10 and a decrease factor of 10⁻¹, capped at a maximum damping parameter of 10¹⁰. Relevant parameters are detailed in Table 1.

Furthermore, hyperparameters such as the activation function, loss function, optimization function, and hidden layer quantity are selected based on the hyperparameter tuning and comparison analysis presented in Section 4. Moreover, evaluating the robustness of the proposed method for solving the inverse Richards equation requires examining its performance with data affected by random noise. This study performs such an assessment using noise-perturbed data.

The noise level refers to the degree of random variation or disturbance introduced to ideal or exact data. In machine learning, noise represents errors or uncertainties present in the input data or output results. It is often introduced to simulate real-world conditions where data is inherently imperfect. Moreover, noise level also refers to the intensity or power of the noise in a signal. It can be measured in terms of power or amplitude. For power, it is denoted as $P_n$, and for amplitude, it can be expressed as the standard deviation of the noise signal. Furthermore, noise is typically quantified using either the standard deviation or the signal-to-noise ratio (SNR). The SNR measures the ratio between signal power and noise power. Additionally, the SNR is inversely proportional to the noise power; an increase in noise power results in a decrease in SNR, signifying a higher relative noise level compared to the signal. This inverse relationship highlights how increasing noise levels degrade the clarity and quality of the signal.

To solve the inverse Richards equation, it is essential to assess the effectiveness and stability of the proposed numerical method. In this case, boundary and final time data affected by random noise are utilized. The noisy data applied to the accessible boundary and final time are as follows [37,38]:

$${\widehat{P}}_B=P_B\times \left[1+s\times (2\times rand-1)\right],$$

(24)

$${\widehat{P}}_F=P_F\times \left[1+s\times (2\times rand-1)\right].$$

(25)

where $P_B$ represents the exact boundary data in Equation (4), $P_F$ represents the exact final time data, ${\widehat{P}}_B$ represents the noisy data on the accessible boundary, ${\widehat{P}}_F$ represents the noisy data in the final time data, s represents the level of noise, and $rand$ represents a random number in the range [0, 1] generated by the uniform distribution. The noisy data represented in Equations (24) and (25) are used to calculate the inverse Richards equation.

Table 6. Accuracy comparison for the applications over 20 runs.

Case		Case A	Case B	Case C	Case D
Boundary points quantity		1582	1410	766	720
Without noisy data	RMSE	2.14 × 10−6	2.69 × 10−6	8.41 × 10−6	3.74 × 10−5
	MRE	9.47 × 10−7	1.02 × 10−6	7.55 × 10−6	3.46 × 10−4
	MAE	9.19 × 10−6	1.02 × 10−5	7.39 × 10−5	3.46 × 10−4
With noisy data	RMSE	5.77 × 10−4	5.76 × 10−4	5.76 × 10−4	5.78 × 10−4
	MRE	1.06 × 10−4	1.06 × 10−4	1.10 × 10−4	1.18 × 10−4
	MAE	1.00 × 10−3	1.00 × 10−3	1.00 × 10−3	1.20 × 10−3

presents the computed errors for the inverse Richards equation across 20 runs. The results show that Case A yields RMSE, MRE, and MAE values of 10⁻⁶, 10⁻⁷, and 10⁻⁶, respectively. Case B produces RMSE, MRE, and MAE values of 10⁻⁶, 10⁻⁶, and 10⁻⁵. Case C results in RMSE, MRE, and MAE values of 10⁻⁶, 10⁻⁶, and 10⁻⁵. Case D shows RMSE, MRE, and MAE values of 10⁻⁵, 10⁻⁴, and 10⁻⁴. The analysis of these cases suggests that the proposed method, developed in this study, can accurately determine the distribution of unsaturated flow in soils, even when boundary conditions are insufficient, as illustrated in Figure 11. Furthermore, with the effect of noise at a level of 0.1, all four cases achieve RMSE, MRE, and MAE values of 10⁻⁴, 10⁻⁴, and 10⁻³, respectively. This demonstrates that the proposed method developed in this study can effectively determine the distribution of unsaturated flow in soils, even when boundary conditions are inadequate and data is affected by random noise, as illustrated in Figure 12. Although the results in Table 6 show that when the boundary conditions are disturbed by noise, the error is lower compared to when there is no noise interference, the proposed DNN in this study still achieves reliable numerical accuracy. Furthermore, with noise at a level of 0.1, all four cases reach RMSE, MRE, and MAE values of 10⁻⁴, 10⁻⁴, and 10⁻³, respectively. This demonstrates that the proposed DNN in this study can effectively determine the distribution of unsaturated flow in soils, even when boundary conditions are inadequate and data is affected by random noise, as shown in Table 6.

6. Conclusions

This study develops a spacetime RBF-based DNN for solving unsaturated flow problems. Given that unsaturated flow problems are inherently nonlinear, this study focuses on solving these issues as governed by the Richards equation. This equation is nonlinear due to the dependence of unsaturated hydraulic conductivity on pressure head, resulting in significant nonlinearity. To address the nonlinearity of the Richards equation, we introduced a linearization process using the Gardner exponential model. The technique could demonstrate the proposed method’s capability in producing reliable solutions across a range of nonlinear scenarios.

The key contributions of this study include the introduction of an innovative approach for modeling unsaturated flow in soils by integrating DNNs with spacetime RBFs. The proposed DNNs eliminate the need for discretizing the governing equation, enabling a direct solution across the entire time and space domain. This approach leverages the strengths of DNNs in capturing complex patterns and the capabilities of spacetime RBFs in handling spatiotemporal data, providing more insights than traditional methods. The main findings are outlined as follows:

(1)

In the proposed method, we incorporate spacetime RBFs, such as simplified MQ, simplified IMQ, and simplified Gaussian, which eliminate the need to directly discretize the Richards equation while providing the distribution of unsaturated flow over the entire time and space domain. This model highlights the DNN’s ability to identify complex patterns and the effectiveness of spacetime RBFs in modeling spatio-temporal data.

(2)

Given that this model involves multiple parameter settings that significantly impact solution accuracy, we analyze how the number and location of collocation points and the number of hidden layers affect accuracy. Additionally, we conduct hyperparameter tuning and comparison analysis to determine the optimal settings for the model. The results indicate that the highest accuracy is achieved with the DNNs paired with spacetime MQ RBF, with RMSE, NSE, MRE, and MAE values of 10⁻⁶, 1, 10⁻⁷, and 10⁻⁶, respectively, under the optimal hyperparameter settings. This demonstrates that the model’s predictions closely align with actual unsaturated flow analysis results.

(3)

Finally, when accounting for noise at a level of 0.1, all four cases yield RMSE, MAE, and MRE values of 10⁻⁴, 10⁻³, and 10⁻⁴, respectively. This indicates that the method proposed in this study can effectively determine the distribution of unsaturated flow in soils, even under conditions of inadequate boundary data and random noise contamination.

(4)

However, the limitation of this study is that the performance of the proposed method heavily relies on the quality and quantity of training data. The proposed method’s sensitivity to the choice of hyperparameters and RBF shape parameters may require extensive tuning and expertise, limiting its practical applicability in some cases.

(5)

Future studies could explore the applicability of the proposed model to other types of flow problems, such as multiphase flow or flow in heterogeneous media. Additionally, the model could be applied to real-world scenarios, including predicting soil moisture content for agricultural applications or managing groundwater resources. To enhance reliability under varying uncertainty and noise levels, future studies could integrate Monte Carlo simulations or other probabilistic methods for generating multiple input realizations. Further work could also focus on developing a sensitivity analysis framework to evaluate the influence of different parameters and initial conditions on the model’s outputs.