Parameter Calibration and Verification of Elastoplastic Wet Sand Based on Attention-Retention Fusion Deep Learning Mechanism

Abstract

The discrete element method (DEM) is a vital numerical approach for analyzing the mechanical behavior of elastoplastic wet sand. However, parameter uncertainty persists within the mapping between constitutive relationships and inherent model parameters. We propose a Parameter calibration neural network based on Attention, Retention, and improved Transformer for Sequential data (PartsNet), which effectively captures the nonlinear mechanical behavior of wet sand and obtains the optimal parameter combination for the Edinburgh elasto-plastic adhesion constitutive model. Variational autoencoder-based principal component ordering is employed by PartsNet to reduce the high-dimensional dynamic response and extract critical parameters along with their weights. Gated recurrent units are combined with a novel sparse multi-head attention mechanism to process sequential data. The fusion information is delivered by residual multilayer perceptron, achieving the association between sequential response and model parameters. The errors in response data generated by calibrated parameters are quantified by PartsNet based on adaptive differentiation and Taylor expansion. Remarkable calibration capabilities are exhibited by PartsNet across six evaluation indicators, surpassing seven other deep learning approaches in the ablation test. The calibration accuracy of PartsNet reaches 91.29%, and MSE loss converges to 0.000934. The validation experiments and regression analysis confirmed the generalization capability of PartsNet in the calibration of wet sand. The improved sparse attention mechanism optimizes multi-head attention, resulting in a convergence speed of 21.25%. PartsNet contributes to modeling and simulating the precise mechanical properties of complex elastoplastic systems and offers valuable insights for diverse engineering applications.

1. Introduction

The nonlinear mechanical characteristics of elastoplastic wet sand are ubiquitous in nature and allows it to be used in industrial processing, such as in feed industries [1], agricultural engineering, and biosystems engineering [2,3,4,5,6]. Elastoplastic wet sand formed through accumulation and cementation deviates from the fundamental assumption of continuum mechanics, making the discrete element method (DEM) the primary method for scholars to comprehend its properties. One critical bottleneck that impedes the development and deployment of the DEM is establishing precise mapping between macroscopic mechanical constitutive relationships and microscopic model parameters [7]. Moreover, numerous internal parameters such as the contact plasticity ratio cannot be determined through experimental measurements, and some of these parameters lack direct physical significance in elastoplastic mechanics. The accuracy of model parameters determines the confidence level of the DEM simulation results. Addressing the parameter identification in the DEM constitutes a prerequisite to overcome the aforementioned bottleneck.

Parameter calibration [8,9,10,11,12,13] is an effective method for addressing unidentifiability in parameters and their combinations, with optimization-based techniques being one common approach. The challenge in parameter calibration is identifying meaningful parameter combinations in high-dimensional and highly coupled parameter spaces that can accurately perform numerical simulations. Schmidt and Lipson [14] define meaningful functionalities as those that strike a balance between accuracy and complexity. Model-driven and data-driven technologies [15] are two main research paradigms oriented to parameter calibration. The optimization efficiency of the former depends on the discrepancies between the predicted constitutive model and real systems. Due to the limited expressive ability of the models, the information from real systems is dimensionally reduced, resulting in the natural loss of information during the modeling process. Data-driven strategies use the inherent connections in the data to create more precise parameter combinations, which reduces the need for prior knowledge and enhances the interpretability of the models [16]. Data-driven technologies have advanced significantly during the past two decades along with the emergence of deep learning [17,18]. Horni et al. [19] demonstrated the generalization of deep learning by highlighting, with a sufficient number of neurons and layers, that deep learning can learn to accurately map any number of inputs to any number of outputs, enabling it to model nearly anything, including the parameter unidentifiability in the DEM.

To achieve parameter calibration correctly, researchers have undertaken exploratory works in three aspects, as outlined in Figure 1. The first aspect involves theory-based methods, which utilize sensitivity analysis or the Plackett–Burman analysis [5] to reduce parameter dimensions. Subsequently, iterative cross-analysis [20], response surface methodology [21], adaptive moment estimation [22], and the least squares method [23] are employed for parameter calibration. However, theory-based methods have limited modeling capabilities for nonlinear systems. The assumptions and restrictions on the parameter space may lead to convergence on a solution domain that cannot cover all parameter combinations, making it difficult to achieve parallel calibration for a large number of parameters. The second aspect partially addresses these challenges by applying statistical optimization algorithms. Do et al. [24] proposed a parameter calibration method using nondominated Sorting Genetic Algorithm II, achieving a Pareto-optimal balance between the calibration accuracy and speed. Wang et al. [25] pointed out the limitations of empirical methods in parameter calibration and proposed a new method based on an improved Particle Swarm Optimization (PSO) algorithm. Their work indicated that the empirical formula between micro and macro is not reliable. Boikov et al. [26] introduced a strategy based on random forests, which offers advantages in terms of time complexity and incorporates classical machine learning methods into this field. These methods start to possess global search capability and randomness indicators in the parameter space. However, these methods struggle with handling practical constraints such as parameter value ranges and parameter sensitivity, leading to higher computational costs and time consumption. Furthermore, improved heuristic algorithms tend to be dependent on specific problems. In the third aspect, data-driven techniques centered around deep learning showcase its superiority. Deep learning [17] successfully overcomes the prior difficulties by leveraging automated feature extraction and nonlinear modeling. Researchers have applied DNNs (Deep Neural Networks), CNNs (Convolutional Neural Networks), RNNs (Recurrent Neural Networks), and RL (reinforcement learning) to parameter calibration [8,27,28,29], and deep learning frameworks based on Bayesian inference have also been utilized [30].

The existing methods rely on static sampling of the response process to calibrate model parameters. Despite the improving robustness of these methods, the understanding of time-series data from the response process remains inadequate. By utilizing sequence data rather than static sampling points, it becomes possible to learn high-dimensional coupled patterns present in the response data [8]. The generalization of deep learning for sequential data, such as RNNs, is constrained by the long-term dependencies within the data. It is essential to combine the attention mechanism to achieve the ultimate generalization of deep learning in DEM calibration tasks.

This paper proposes an attention retention fusion with a deep learning mechanism, PartsNet, which addresses long-term dependencies by incorporating an improved sparse multi-head self-attention mechanism with a GRU (gated recurrent unit). PartsNet takes the sequence data generated in simulations as the input and the constitutive parameters that need to be calibrated as the output. We study the parameter calibration of the Edinburgh elasto-plastic adhesion (EEPA) model to describe the triaxial compression of wet sand. The calibrated model accurately captures wet sand’s constitutive relationship with exceptional precision. PartsNet obtains the relationship between the macroscopic behavior of wet sand and the microscopic parameters in the EEPA model.

The rest of this paper is organized as follows: The principles, network topology, and implementation details of PartsNet are elucidated in Section 2. In Section 3, we introduce the simulations and experiments conducted on elastoplastic wet sand, along with the calibration results and verification tests of PartsNet for these experiments. In Section 4, we present the parameter calibration results obtained by PartsNet, the ablation experiments, and the error analysis. Finally, some conclusions are given in Section 5.

2. Methodology: PartsNet

In this section, we discuss the three core architectures of PartsNet: GDR-Net, Encoder-Net, and Decoder-Net. These sub-modules collectively establish a systematic framework for parameter calibration.

2.1. General Data Dimensionality Reduction Network (GDR-Net)

The sequence data generated by numerical models are taken as input for PartsNet. Sequence data comprise multidimensional feature parameters, which exhibit non-orthogonality and varying weights. Characterizing a system can be accomplished by calibrating only a few parameters with the most substantial weights and richest information. Previous works [31,32] have mainly relied on principal component analysis (PCA) and its improved algorithms to perform data feature dimensionality reduction. PCA is an unsupervised machine learning approach that may fail to recognize nonlinear connections because of its strict assumptions about the data. This paper proposes a general data dimensionality reduction method based on cross-correlation functions to identify significant features in large datasets. We employ Variational Autoencoders (VAEs) [33] to enhance the feature extraction capability of PartsNet. The sequential data consist of a set of signals.

Definition 1.

For any square-integrable functions,$X\in {ℝ}^{N \times K}$and$\tilde{X}\in {ℝ}^{N \times K}$, $\gamma \left(m\right)$ represents the cross-correlation of $X$ and $\tilde{X}$ at m. $\gamma \left(m\right)$ is subjected to

$$\gamma \left(m\right)=sigmoid\left({\displaystyle \frac{1}{NK}}{\displaystyle {\int }_K{\displaystyle {\int }_{N}X\left(n\right)\cdot \tilde{X}\left(n+m\right)}} dndk\right)$$

(1)

We use the sigmoid function to normalize the values. For discrete signal functions under investigation, it can be further expressed in the following weak form:

$$\gamma \left(m\right)=sigmoid\left({\displaystyle \frac{1}{NK}}{\displaystyle \sum _{k=1}^K{\displaystyle \sum _{n=1}^{N}X\left(n\right)\cdot \tilde{X}\left(n+m\right)}}\right)$$

(2)

Specially, let K equal 2, and consider the cross-correlation of discrete sequential data at $m=0$. $\gamma \left(m\right)$ is subjected to:

$$\gamma \left(m\right)=sigmoid\left({\displaystyle \frac{1}{2N}}{\displaystyle \sum _{n=1}^N\left(X_1\left(n\right)\cdot \widetilde{X_1}\left(n\right)+X_2\left(n\right)\cdot \widetilde{X_2}\left(n\right)\right)}\right)$$

(3)

The significance of this approach and the weak form lies in establishing a mutual correlation between simulation-generated sequence data and experimentally obtained data, facilitating the labeling process. A higher value of $\gamma \left(m\right)$ indicates a higher quality of the simulation-generated data, while a lower value indicates the opposite.

The core of GDR-Net is the use of a VAE for the dimensionality reduction in labeled feature data [34], as shown in Figure 2. A VAE is a generative neural network based on variational Bayesian inference, which addresses the overfitting issue commonly found in traditional autoencoders [35]. The encoder of a generative model uses the input data to learn a compressed form, known as latent variables, while the decoder uses this compressed representation to reconstruct the input. The differences between the original data and the reconstructed data is termed as a reconstruction error, which serves as the optimization objective of autoencoders. The VAE introduces a set of random variables, denoted as z, to represent the latent distribution $p\left(z\right)$ of the input data and facilitate the generation of new data. The latent model follows a Gaussian distribution, with its mean and variance generated by a parameterized function output by the encoder. The objective of the VAE is to maximize the Evidence Lower Bound (ELBO), which is a lower bound on the log-likelihood of the generative model. The ELBO is defined as follows:

$$\begin{array}{cc}{ℒ}_1\left(\theta ,\phi ; x\right)& =E_{q_{\phi }\left(z\left|x\right.\right)}\left[\log p_{\theta }\left(x\left|z\right.\right)\right]-D_{KL}\left(q_{\phi }\left(z\left|x\right.\right)‖p\left(z\right)\right)\hfill \\ & =E_{q_{\phi }\left(z\left|x\right.\right)}\left[{\displaystyle \sum _{i=1}^n\log p_{\theta }\left(x_i\left|z\right.\right)}\right]-{\displaystyle \int q_{\phi }\left(z\left|x\right.\right)\log \frac{q_{\phi }\left(z\left|x\right.\right)}{p\left(z\right)}} dz\end{array}$$

(4)

Here, $q_{\phi }\left(z\left|x\right.\right)$ represents the variational posterior distribution generated by the encoder, $p_{\theta }\left(x\left|z\right.\right)$ represents the generative distribution generated by the decoder, and $p\left(z\right)$ represents the prior distribution. The first term on the right-hand side corresponds to the reconstruction term, which maximizes the log-likelihood function $\log p_{\theta }\left(x\left|z\right.\right)$ to measure the similarity between the reconstructed data and the original input. The second term is the Kullback–Leibler (KL) divergence term, which minimizes the KL divergence between distributions $q_{\phi }\left(z\left|x\right.\right)$ and $p\left(z\right)$ to regularize the distribution of latent variables, ensuring that the learned distribution becomes closer to the prior. During the training process, a multilayer perceptron structure is employed.

Binary Cross-Entropy Loss (BCELoss) is utilized to measure the discrepancy between the predicted data and the original data in terms of labels.

$${ℒ}_2=-\left(y_{label}\cdot \log \left(p\right)+\left(1-y_{label}\right)\cdot \log \left(1-p\right)\right)$$

(5)

Here, $y_{label}$ represents the actual quality of the sequence data, which is typically the output of a simulation process, and $\log \left(\cdot \right)$ represents the probability predicted by the model that the data are a positive sample. GDR-Net achieves the learning process by minimizing the negative ELBO and BCELoss.

$${\theta }^{\ast },{\phi }^{\ast }=\arg {\min }_{\theta ,\phi }-\left({ℒ}_1\left(\theta ,\phi ; x\right)+{ℒ}_2\right)$$

(6)

While a VAE can achieve data dimensionality reduction, it cannot explicitly express the weights of different dimensional features on the results. In this study, a principal component ordering model-based PCA is utilized to handle the features learned in the latent space. This combined approach ensures that GDR-Net can capture crucial features from high-dimensional data and further determine their weights.

2.2. Encoder-Net

The core functionality of PartsNet is integrated within Encoder-Net, as shown in Figure 3. Encoder-Net takes the dimensionality-reduced data processed by GDR-Net as input, combining the attention and retention information from the sequence data to produce the output. The GRU layer yields a sequence matrix H comprising the hidden layer states, which serves as the initial extraction of the feature information from the sequential data. By introducing an improved sparse multi-head attention mechanism, weights are assigned to each hidden layer state of the sequence matrix. This addresses the issue of long-term dependencies in sequence data and reduces the time complexity of the Transformer multi-head attention from $\mathsf{𝒪}(n^2 · d_{\mathit{model}})$ to $\mathsf{𝒪}(n^2 · \log \mathit{n})$.

2.2.1. Gated Recurrent Unit (GRU) Layers

An RNN encounters two drawbacks when dealing with sequential data. Firstly, the limited retention capacity leads to the issues of vanishing or exploding gradients. Secondly, the model’s dependence on sequential data results in low parallelism. The GRU proposed by Cho et al. [36] in 2014 demonstrated good performance in sequential data prediction [8].

The GRU handles sequence data and learns the underlying structures through the reset gate $R_i$ and the update gate $Z_i$, as shown in Figure 4. $X_i$ represents the input data with sequence value of i, while $H_{i-1}$ denotes the hidden state vector output from the previous timestep of the GRU. Upon receiving inputs $X_i$ and $H_{i-1}$, the current GRU computes the gate control states $R_i$ and $Z_i$ through a calculation process.

$$R_i=\sigma \left(X_i{W}_{X_i^R}+H_{i-1}{W}_{h_R}+b_R\right)$$

(7)

$$Z_i=\sigma \left(X_i{W}_{X_i^Z}+H_{i-1}{W}_{h_Z}+b_Z\right)$$

(8)

$R_i$ is responsible for controlling to what extent the previous feature information can be remembered and how it should be incorporated into the candidate hidden state $\widetilde{H_i}$. $Z_i$ filters out certain information from $H_{i-1}$, retaining only the relevant and useful portions, enabling the neural network to exhibit retention capability.

$$\widetilde{H_i}=\tanh \left(X_i{W}_{X_i^H}+\left(R_i\odot H_{i-1}\right)W_{HH}+b_H\right)$$

(9)

$\widetilde{H_i}$ reflects the retention capacity of the GRU, while $Z_i$ determines the strength of the retention. The GRU ultimately outputs the current hidden state vector, which is a weighted sum of $H_{i-1}$ and the current candidate hidden state $\widetilde{H_i}$.

$$H_i=Z_i\odot H_{i-1}+\left(1-Z_i\right)\odot \widetilde{H_i}$$

(10)

Here, $\sigma \left(\cdot \right)$ and $\tanh \left(\cdot \right)$ are activation functions, and their output ranges are $\left[0, 1\right]$ and $\left[-1, 1\right]$, respectively. $W_{X_i^R}$, $W_{X_i^Z}$, $W_{X_i^H}$, $W_{h_R}$, $W_{h_Z}$, and $W_{HH}$ represent weight matrices, while $b_R$, $b_Z$, and $b_H$ represent bias terms.

2.2.2. Improved Sparse Multi-Head Attention Mechanism

The Transformer [37] is a sequence-to-sequence model that utilizes the multi-head attention mechanism. The attention mechanism assigns different weights to the information from each timestep and compares it to the information from the current timestep in order to better capture implicit structures and long-range dependencies in sequence data. The self-attention mechanism can be represented by the following equation:

$$Attention\left(Q,K,V\right)=soft\max \left({\displaystyle \frac{Q{K}^T}{\sqrt{d_k}}}\right)V$$

(11)

where Q, K, and V are obtained by linear transformations applied to the input sequence. $d_k$ represents the dimensionality of each query vector. $Q{K}^T$ represents the similarity matrix between the query and key vectors, and it corresponds to the inner product projection of the tensors.

Multi-head attention strengthens the ability of the neural network to capture highly nonlinear relationships. It consists of multiple independent self-attention layers, referred to as “heads.” Each head has its own set of learnable weight matrices, allowing it to capture different features and semantic information from the input sequence. The splitting of matrices is performed as follows: $Q_i=X{W}_i^Q,K_i=X{W}_i^K,V_i=X{W}_i^V$.

The multi-head attention mechanism is calculated as follows:

$$\begin{array}{c}MultiHead\left(Q,K,V\right)=Concat\left(hea{d}_1,\cdots ,hea{d}_H\right)W^O\\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}where hea{d}_i=Attention\left(Q_i{W}_i^Q,K_i{W}_i^K,V_i{W}_i^V\right)\end{array}$$

(12)

These transformation matrices belong to the vector space as follows: $W_i^Q\in {ℝ}^{d_{model}\times d_k}$, $W_i^K\in {ℝ}^{d_{model}\times d_k}$, $W_i^V\in {ℝ}^{d_{model}\times d_v}$ and $W^O\in {ℝ}^{h{d}_v\times d_{model}}$. $W^O$ is a linear transformation matrix so that the dimensions of the input and output are the same. H represents the number of heads. The time complexity of the multi-head attention mechanism is $\mathsf{𝒪}(n^2 · d_{\mathit{model}})$, and n represents the length of the sequence.

This paper proposes a sparse multi-head attention mechanism based on a sparse mask, aiming to reduce the time complexity of the Transformer encoder while ensuring learning accuracy. A sparse mask matrix, which has the same shape as $Q{K}^T$, is designed in this study. The dimensions to be preserved in the original projection matrix are determined based on the hyperparameter sparsity, which yields a sparse threshold. Subsequently, information positions in the sparse mask that are less than or equal to the threshold are set to negative infinity and mapped to $Q{K}^T$, introducing sparsity into the attention scores.

$$Sparse mask\left(Q{K}^T\right)=\left\{\begin{array}{c}-\infty ,\hspace{1em}if ‖Q{K}^T‖\le threshold\\ 0,\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}otherwise\end{array}\right.$$

(13)

The sparse multi-head attention mechanism involves sorting the attention scores, mask processing, and conventional matrix multiplication. The overall complexity is denoted as $\mathsf{𝒪}(n^2 · \log \mathit{n})$, where n represents the length of the sequence. The performance of the proposed mechanism will be further demonstrated in the ablation experiments conducted in Section 4.2.

$$\begin{array}{l}Sparse Attention\left(Q,K,V\right)=soft\max \left({\displaystyle \frac{{\left(Q{K}^T\right)}_{Sparse}}{\sqrt{d_k}}}\right)V\\ Sparse MultiHead\left(Q,K,V\right)=Concat\left(hea{d}_1,\cdots ,hea{d}_H\right)W^O\\ \hspace{1em}\hspace{1em} where hea{d}_i=Sparse Attention\left(Q_i{W}_i^Q,K_i{W}_i^K,V_i{W}_i^V\right)\end{array}$$

(14)

The processed retention data from the GRU layer is first subjected to layer normalization, making sure that each feature dimension has a comparable mean and variance. After obtaining sparse attention weights, a residual connection [38] is used before the subsequent layer normalization for enhancing model robustness. Finally, a multilayer perceptron is used to further improve Encoder-Net’s nonlinear expressive capability.

2.3. Decoder-Net

Processing sequential data in the DEM differs logically from NLP tasks like machine translation. Sequential data do not require masks for information preservation due to their inherent continuity. The classic Transformer decoder is not suitable for the calibration task. Instead, the Decoder-Net of PartsNet incorporates a two-layer multilayer perceptron with residual connection. The size of the first layer is determined by the hidden size, while the second layer is contingent upon the number of calibrated parameters. Multilayer perceptron is capable of learning more complex nonlinear mapping relationships, while residual connection enhances gradient information stability. Through Decoder-Net, high-dimensional data are mapped to the low-dimensional space of the parameters to be calibrated. The decoder is calculated as follows:

$$Decoder\left(x\right)=\mathit{Re}LU\left(Linear\left(x\right)\right)+x$$

(15)

Here, $ReLU\left(\cdot \right)$ represents the activation function, $Linear\left(\cdot \right)$ represents the linear transformation based on multilayer perceptron, and x represents the input tensor in Decoder-Net. The output is a vector composed of parameters to be calibrated. The coordination of these neural networks realizes the “end-to-end” solution of parameter calibration.

2.4. Implementation Details

PartsNet was developed in this study using open-source frameworks such as PyTorch and NumPy. The training of PartsNet was conducted on a 12th Gen Intel(R) Core(TM) i5-12500H (Santa Clara, CA, USA). The hyperparameters used in the training process are summarized in

Table 1. List of PartsNet hyperparameters.

Hyperparameters	Triaxial Test
Batch Size	40
Hidden Size	72
Number of Heads	4
Number of layers	1
Sparsity	0.49
Initial Learning Rate	0.00056
Optimizer	Adam
Loss Function	MES loss
Q-Quantile Value	0.75
Training Epochs	600
N (Sequence Length)	99
GDR-Net Latent_dim	3
GDR-Net Epochs	1000

. Two sets of hyperparameters were selected for the following two scenarios, respectively.

3. Triaxial Compression Calibration Test and Verification

This section discusses the parameter calibration case of PartsNet, focusing on its application in calibrating the EEPA model and employing it to simulate the mechanical behavior of elastoplastic wet sand under triaxial compression.

3.1. Edinburgh Elasto-Plastic Adhesion Model (EEPA)

The interaction among a granular system like wet sand is highly complex, influenced by factors such as surface topography, chemical properties, and the characteristics of the interstitial medium [39]. The EEPA model is an extension of the linear viscoelastic model [40]. This model can effectively simulate the nonlinear force-displacement behavior during compression and incorporates both viscous damping and rolling resistance mechanisms, similar to a linear rolling resistance model. In this study, EEPA (Figure 5) is employed to describe the elastoplastic wet sand, and PartsNet calibrates the key parameters of EEPA.

EEPA is influenced by seven parameters: the initial loading stiffness $k_1$, unloading and reloading stiffness $k_2$, contact plasticity ratio ${\lambda }_p$, constant pull-off force $f_0$, adhesion energy parameter $\Delta \gamma$, stiffness exponent n, and adhesiveness exponent $\chi$. The stiffness exponent n takes two values, 1 and 1.5, with $n=1$ indicating whether the model is equivalent to a linear elastic model. In order to define the nonlinear behavior during loading and unloading, this study adopts a value of 1.5 for n.

The overall normal force $F_N$ is composed of two parts, the hysteresis spring force $f_{hys}$ and the normal damping force $f_n^d$:

$$F_N=\left(f_{hys}+f_n^d\right)u$$

(16)

Here, $u$ is the unit normal vector pointing from the contact point towards the center of the particle. The mathematical expression for the normal hysteresis spring force $f_{hys}$ can be represented as follows:

$$f_{hys}=\left\{\begin{array}{cc}{f}_0+k_1{\delta }^n& if \left({\delta }^n-{\delta }_p^n\right)\ge k_1{\delta }^n\hfill \\ f_0+k_2\left({\delta }^n-{\delta }_p^n\right)& if k_1{\delta }^n>k_2\left({\delta }^n-{\delta }_p^n\right)>-k_{adh}{\delta }^n\hfill \\ f_0-k_{adh}{\delta }^n& f-k_{adh}{\delta }^n>k_2\left({\delta }^n-{\delta }_p^n\right)\hfill \end{array}\right.$$

(17)

The normal damping force $f_n^d$ can be expressed as follows:

$$f_n^d=-2\sqrt{{\displaystyle \frac{5}{6}}}{\beta }_{NL}\sqrt{K_n{m}^{\ast }}{v}_n^{\overrightarrow{rel}}$$

(18)

Here, ${\delta }^n$ represents the normal overlap, ${\delta }_p^n$ represents the normal plastic overlap, $k_{adh}$ represents the adhesive stiffness, and $v_n^{\overrightarrow{rel}}$ represents the relative normal velocity generated during particle contact. $k_1$ is defined by the Hertzian contact theory, and its magnitude depends on the equivalent radius of particles, $R^{\ast }$, and the equivalent Young’s modulus, $E^{\ast }$, in the particle system.

$$k_1={\displaystyle \frac{4}{3}}\sqrt{R^{\ast }}{E}^{\ast }$$

(19)

${\lambda }_p$ is defined based on the ratio of two-stage stiffness, and a value of 0 indicates the elastic contact of particles.

$${\lambda }_p=1-{\displaystyle \frac{k_1}{k_2}}$$

(20)

$K_n$ represents the Hertz normal stiffness, which depends on the equivalent radius, Young’s modulus, and the normal overlap quantity ${\delta }_n$.

$$K_n=2E^{\ast }\sqrt{R^{\ast }{\delta }_n}$$

(21)

${\beta }_{NL}$ represents the normal damping coefficient of the nonlinear model, and its calculation formula is provided by Morrissey [31].

$${\beta }_{NL}={\displaystyle \frac{\ln e}{\sqrt{{\ln }^{2}e+{\pi }^2}}}$$

(22)

Similarly, the total tangential force $F_T$ is also the sum of the tangential spring force $f_{ts}$ and the tangential damping force $f_{td}$:

$$F_T=\left(f_{ts}+f_{td}\right)v$$

(23)

where $v$ is the unit vector pointing tangentially from the contact point to the center of the particle, and $f_{ts}$ is defined in incremental form (for the nth time step):

$$f_{ts}=f_{ts\left(n-1\right)}+\Delta f_{ts}$$

(24)

Here, $\Delta f_{ts}=k_t{\delta }_t$, $k_t$ is the tangential stiffness, and ${\delta }_t$ is the tangential overlap.

$$k_t=8{\zeta }_{tm}{G}^{\ast }\sqrt{R^{\ast }{\delta }_n}$$

(25)

${\zeta }_{tm}$ is the tangential stiffness multiplier, and in this paper, ${\zeta }_{tm}={\displaystyle \frac{2}{3}}$. $G^{\ast }$ represents the equivalent shear modulus. The critical force for tangential force coupling with normal force can be determined based on the Coulomb friction criterion:

$$F_T\le F_{C-limit}=\mu ‖\left(f_{hys}+k_{adh}{\delta }^n-f_0\right)‖$$

(26)

Here, $F_{C-limit}$ represents the maximum tangential friction force, and $\mu$ is the friction coefficient. The modeling theory of EEPA can also refer to [8,13,39,41].

3.2. Test and Numerical Simulation of Triaxial Compression in Wet Sand

3.2.1. Triaxial Compression Test in Wet Sand

A triaxial test was conducted using the electronic dynamic–static fatigue testing machine at the Huazhong University of Science and Technology, as illustrated in Figure 6. The experimental material chosen consisted of wet sand characterized by particle sizes ranging from 1 mm to 2 mm with an aggregate weight of 80 g. The moisture content of the sand was measured to be 6.98%, with a pH value of 7.2. The moisture content is used to characterize the aggregation of the system. The density of the sand was measured before the experiment and found to be 2 g/cm³. The samples were loaded into acrylic cylinders with an outer diameter of 45 mm, an inner diameter of 35 mm, and a height of 85 mm, with a loading height of 60 mm. The loading and unloading speeds during the experiment were set to 0.5 mm/s and 0.2 mm/s, respectively. The data were sampled at a frequency of 200 Hz, and the load threshold of the testing machine was set to 600 N. Based on the experimental results, it was observed that the sand exhibited significant differences in stiffness between loading and unloading, with the nonlinear behavior being more pronounced during the loading phase.

3.2.2. Simulation of Triaxial Compression by DEM

In discrete element studies, using spherical particles to represent sand is common [42,43]. A total of 8238 particle units were generated, and a simulation model was established based on the experimental conditions. To ensure uniform force distribution in the granular system during simulation, a small load was applied for pre-compression and allowed to settle until the longitudinal force reached a stable value of 1.82 N. The process of loading and unloading through the piston is the critical stage in the DEM simulation. Due to the computational complexity of the DEM, it would be time-consuming to perform the simulation directly using the experimental loading and unloading velocities. To address this, a sensitivity analysis based on varying velocity gradients was performed to assess the dynamic response of wet sand to loading and unloading velocities. The sensitivity analysis results, depicted in Figure 7, include three experimental setups. The first setup utilized a loading speed of 50 mm/s for 0.04 s and an unloading speed of 20 mm/s from 0.04 to 0.06 s. The second setup featured a loading speed of 5 mm/s for 0.4 s and an unloading speed of 2 mm/s from 0.4 to 0.6 s. The third setup mirrored the actual experimental conditions with a loading speed of 0.5 mm/s for 4 s and an unloading speed of 0.2 mm/s from 4 to 6 s. The simulation times for these setups were 0.1714 h, 1.2985 h, and 6.467 h, respectively, with a notable increase in the computational time as the loading rate decreased. Despite the significant differences in the loading and unloading velocities, the load–displacement curves across all three setups exhibited similar nonlinear behavior with minimal variation. Consequently, to optimize simulation efficiency and enable the generation of a dataset through multiple simulation iterations, this study employed the loading and unloading velocities from the first setup for subsequent investigations. The other simulation conditions were consistent with the triaxial compression test. To accurately capture the data characteristics, we chose the sampling frequency as 2000 Hz, and each sequence of data was sampled 99 times. The entire simulation process is illustrated in Figure 8.

The DEM dynamics simulations in this research employ particles shaped as a dual sphere in EDEM 2022, where both spheres have a physical radius of 1 mm and are connected together by a contact zone. In the world coordinate system, the coordinates of these two spheres’ centers are, respectively, (−0.405 mm, 0.000 mm, 0.000 mm) and (0.405 mm, 0.000 mm, 0.000 mm). The smoothing value of the particles was set to 5. In order to accurately simulate the triaxial compression experiment, a container made of Perspex was used to simulate the acrylic tube in the experiment, and the upper surface of the wet sand was loaded and unloaded using a steel piston.

The experiment was simulated in EDEM 2022, and the input parameters used are shown in

Table 2. Simulation parameters of discrete element triaxial compression test.

Properties	Bulk Material (Sand)	Acrylic Tube	Steel (Piston)
Poisson’s ratio	0.3	0.4	0.3
Density (g/cm3)	2	1.2	8
Young’s modulus (Pa)	1.229 × 109	5.6 × 109	2.6 × 1011
Coefficient of restitution	$0.25 (r_{s-s}$)	$0.3 (r_{s-a}$)	0.3
Coefficient of rolling friction	0.23	0.22	0
Coefficient of static friction	$(0.51, 0.55) f_{s-s}$	$(0.38, 0.56) f_{s-a}$	0.5
Tangential stiff multiplier	0.667	\	\
Tensile Exp	3	\	\
Contact plasticity ratio	(0.59, 0.79)	\	\
Surface energy (J/m2)	1.64	\	\
Constant pull-off force (N)	−0.002	\	\
Particle size (mm)	1	\	\
Particle distribution	Dual sphere	\	\

.

In the dimensionality reduction experiment in Section 3.3, we found that $f_{s-s}$, $f_{s-a}$, and ${\lambda }_p$ are significant parameters for EEPA. Accurately capturing these information-rich parameters enables neural networks to analyze the mechanical behaviors of macroscopic systems at a faster pace and with greater simplification, underscoring the significance of parameter calibration. Therefore, this paper expands the range of these three parameters and generates 200 sets of random numbers. These numbers are then used in EDEM 2022 to obtain 200 sets of simulation “load-displacement” sequence data for PartsNet training.

3.3. Training and Results of PartsNet

This section presents the calibration results and performance of PartsNet in the aforementioned case study. In the data preprocessing of triaxial compression, GDR-Net examined the significant preset parameters of EEPA and the main contact parameters of wet sand (${\lambda }_p, f_{s-s}, f_{s-a}, \Delta \gamma ,f_0, r_{s-s}, r_{s-a}$). Through GDR-Net, feature dimension reduction and extraction were performed, resulting in three key parameters ${\lambda }_p, f_{s-s}, f_{s-a}$ in the latent space. Figure 9 displays the loss results of GDR-Net. Reconstruction loss and KL divergence loss on the test set converged to 4.34 × 10⁻³ and 2.723 × 10⁻², respectively, while the classification loss converged to 1.235 × 10⁻². According to the principal component ordering, the relative weights of ${\lambda }_p, f_{s-s}, f_{s-a}$ were found to be 0.378, 0.407, and 0.215, respectively. Therefore, $f_{s-s}$ had the most significant impact on the experimental results, followed by ${\lambda }_p$ and $f_{s-a}$.

We conducted 200 sets of EDEM 2022 simulations, generating “load–displacement” curve data, which we subsequently utilized as an input dataset for PartsNet. Using the predicted parameters from PartsNet, simulations were performed, and the calibrated curve showed a good fit with the experimentally determined curve. Figure 10 illustrates the convergence of the MSE loss and coefficient of determination (R²) during the training process. Parameter calibration can be seen as a regression prediction task, and using accuracy as an evaluation metric, as in classification problems, is not appropriate. Therefore, R² was used as the criterion for accuracy. In the DEM dataset used in this study, PartsNet achieved MSE convergence at 9.34 × 10⁻⁴ and R² convergence at 0.913.

PartsNet incorporates additional, more thorough regression analysis evaluation metrics in addition to the fundamental evaluation metrics (loss functions and coefficient of determination) to describe the calibration performance. The specific calculation formulas can be found in Appendix A. Figure 11 illustrates the convergence of additional evaluation metrics, including the Mean Absolute Error (MAE), Root Mean Square Error (RMSE), Median Absolute Error (MedAE), and the 0.75-quantile loss (0.75-QLoss) when Q = 0.75 during the training process of PartsNet. They converge at 0.019, 0.031, 0.011, and 0.008, respectively. Furthermore, it is worth noting that the training time of PartsNet is only 159.586 s, which will be further analyzed for its significant advantage in the ablation experiment in Section 4.2.

The neural network trained using the aforementioned process can be used to predict the key simulation parameters of the triaxial compression calibration test:$\left[{\lambda }_p, f_{s-s}, f_{s-a}\right]=\left[0.575, 0.520, 0.347\right]$. Figure 12 illustrates the comparison between the simulated sequence data generated using the calibrated parameters and the experimental data. The nonlinear behaviors are well captured, and the calibrated curve reflects the visco-plastic coupling behavior of EEPA. During the initial loading phase, the changing rate in the slope of the calibration curve closely mirrors that of the experimental data, and it accurately predicts both the trends in load and displacement peaks. Moreover, it provides a highly precise description of the unloading phase and residual stress.

PartsNet enhances the fidelity of DEM simulations pertaining to the triaxial compression of elastoplastic wet sand by incorporating time series response data. Through the calibration of key parameters of the EEPA constitutive model, the calibrated model accurately captures the nonlinear sequential mechanical behavior within elastoplastic wet sand. Consequently, PartsNet elevates the precision of DEM simulations concerning loading conditions in elastoplastic wet sand. This method demonstrates excellent capabilities in elastoplastic mechanics calibration tasks. With this method, the issue of unidentifiability in constitutive parameters and their combinations in DEM can be addressed. PartsNet can also be generalized to other scenarios involving multiple processes and coupled physical fields. By directly associating the sequential input data with the key parameters, PartsNet provides a unified solution for parameter calibration that integrates attention and retention.

3.4. Verification Tests for PartsNet

After the confirmation of PartsNet for its accuracy, more intricate cases will be calibrated to demonstrate its general applicability in calibrating the EEPA constitutive relationship for elastoplastic wet sand. In practice, these force–displacement sequential data can be obtained from the above experimental instruments, while here, for simplicity, this paper uses the DEM simulated data generated by the DEM as ground truth. Nine sets of parameter combinations ${\lambda }_p, f_{s-s}, f_{s-a}$, were sampled based on the Gaussian distribution, deliberately including design points beyond the scope of the original training dataset of PartsNet. Numerical experiments were conducted using EDEM 2022. Experimental data were then predicted using the pre-trained PartsNet to calibrate the corresponding parameters. The relative errors between the calibrated parameters and the true parameters are displayed in

Table 3. The actual parameters and calibration parameters of 9 verification cases.

Case	${\mathit{\lambda }}_{\mathit{p}}$	${\mathit{f}}_{\mathit{s}-\mathit{s}}$	${\mathit{f}}_{\mathit{s}-\mathit{a}}$	$\mathbf{Calibrated} {\mathit{\lambda }}_{\mathit{p}}$	$\mathbf{Calibrated} {\mathit{f}}_{\mathit{s}-\mathit{s}}$	$\mathbf{Calibrated} {\mathit{f}}_{\mathit{s}-\mathit{a}}$	Error 1	Error 2	Error 3
1	0.660	0.471	0.501	0.672	0.495	0.427	1.87%	5.11%	14.86%
2	0.730	0.544	0.385	0.719	0.549	0.366	1.50%	0.93%	4.97%
3	0.658	0.515	0.405	0.627	0.513	0.427	4.70%	0.38%	5.39%
4	0.614	0.463	0.378	0.6423	0.462	0.330	4.67%	0.15%	12.82%
5	0.647	0.542	0.416	0.595	0.534	0.438	8.06%	1.39%	5.35%
6	0.592	0.523	0.396	0.599	0.525	0.378	1.35%	0.46%	4.49%
7	0.585	0.483	0.397	0.623	0.487	0.381	6.52%	0.84%	4.15%
8	0.675	0.491	0.325	0.633	0.492	0.384	6.22%	0.17%	18.08%
9	0.680	0.509	0.457	0.692	0.502	0.448	1.79%	1.44%	2.07%

.

Excellent accuracy is achieved by PartsNet in the prediction of parameters $f_{s-s}$ and ${\lambda }_p$, with the majority of relative errors for $f_{s-s}$, which is the most weighted, being within 1%. Parameter $f_{s-a}$, with a smaller weight and sharing some out-of-training-set design points with ${\lambda }_p$, has relative errors exceeding 10% in three cases.

Figure 13 illustrates the calibration results of PartsNet for the nine validation experimental datasets and compares them with the experimental force–displacement curves. The parameters calibrated by PartsNet accurately capture the EEPA constitutive relationship during the loading and unloading processes of the elastoplastic wet sand. The dynamic response obtained from EDEM 2022 exhibits good consistency with the experimental curves, reflecting the dynamic trends, peak load values, and post-unloading displacement values.

The peak load values from the nine validation experiments are discretely distributed between 405.138 N and 678.339 N. In Figure 14, 5% of the experimental peak load is used as error bars to assess the calibration of peak load values. It was found that the results from PartsNet accurately match the experimental data, all falling within the confidence interval. This indicates that PartsNet maintains accuracy even under extreme conditions.

The most precise peak fit and the least amount of error are found in the ninth dataset. These parameters, ${\lambda }_p$, f_s−s, f_s−a, determine the micromechanical behavior and macroscopic response of the sand. The simulation stress–strain curve is faithfully reproduced when the contact plasticity ratio ${\lambda }_p$ is chosen accurately, capturing the subtleties of the material’s elastic–plastic transition during contact. The accuracy of the static friction coefficient between particles f_s−s directly affects the prediction of the shear strength and the formation and evolution of the shear zone, which realistically reproduces the sliding and rolling behaviors between particles. The distribution of wall shear stress and particle flow resistance may be accurately simulated by the static friction coefficient between the sand and the wall f_s−a. This, in turn, influences the sand’s total deformation and energy dissipation.

Figure 15 depicts the correlation assessment between the dynamic responses obtained using calibrated parameters and test data across nine validation experiments. This assessment includes R² values, Pearson correlation coefficients, and linear fitting results. All cases exhibit R² values and Pearson correlation coefficients exceeding 0.99. The fitting coefficients range between 0.968 and 1.032, highlighting remarkable calibration accuracy. Through these nine numerical experiments, the generalization capacity of the PartsNet capacity is validated, providing reasonable inferences for assumption space distributions located at the periphery of the pre-trained dataset.

4. Discussions

4.1. Ablation Test and Calibration Performance Analysis

To demonstrate the outstanding calibration performance of PartsNet in addressing the issue of parameter unidentifiability, we designed the following four sets of ablation experiments: pure RNN and GRU, RNN and GRU with dot-product attention mechanism, RNN and GRU with Transformer encoder (TransE), and RNN and GRU with our proposed sparse mask-based sparse attention mechanism Transformer encoder (STE). The comparison between the first set and the other sets highlights the necessity of introducing the attention mechanism, while the comparisons within the latter three sets reveal the differences among the different types of attention mechanisms in the DEM parameter calibration task. PartsNet demonstrates state-of-the-art performance in these experiments based on multiple indexes.

Table 4. Ablation experimental data of PartsNet in triaxial test.

	MSE	MAE	RMSE	MedAE	Qloss	R2	Time
RNN	0.024	0.113	0.155	0.071	0.059	0.548	57.010
GRU	0.006	0.052	0.080	0.037	0.026	0.881	213.409
RNN+A	0.011	0.067	0.106	0.039	0.030	0.791	65.679
GRU+A	0.006	0.049	0.076	0.031	0.023	0.892	232.825
RNN+TransE	0.020	0.100	0.141	0.062	0.044	0.629	102.360
GRU+TransE	0.005	0.050	0.072	0.035	0.022	0.902	202.654
RNN+STE	0.019	0.098	0.138	0.068	0.043	0.643	43.205
PartsNet	0.0009	0.019	0.031	0.011	0.008	0.913	159.586

presents the ablation experiments conducted on PartsNet and other neural networks. In these experiments, the eight networks shared the hyperparameters specified in Section 2.4. Across all evaluation metrics, RNNs without attention mechanisms performed the worst, with a nearly 40% lower calibration accuracy compared to PartsNet. The introduction of dot-product attention led to some improvements. It is crucial to emphasize that PartsNet achieves the highest training accuracy (0.913) while simultaneously attaining a decent convergence speed. Specifically, PartsNet exhibits an improvement with a speed advantage of 21.25% over the Transformer encoder–GRU-based network. Additionally, the GRU performs significantly better than the RNN both with and without the addition of the attention mechanism. Overall, PartsNet demonstrated the highest calibration precision in these tasks and exhibited a significant advantage in terms of time compared to the multi-head attention mechanism used in Transformer.

4.2. Error Quantization Format of PartsNet

Error analysis is a crucial component in quantitatively describing the accuracy of numerical computations. In this study, we proposed the error quantification format for parameter calibration. Errors can be broadly categorized into systematic errors and random errors. Systematic errors can be understood as the limited expressive power of numerical models and the constrained optimization ability of algorithms, including deep learning. Random errors arise from the varying degrees of numerical discrepancies observed in experiments and calculations.

$$E=E_{systematic}+E_{stochastic}$$

(27)

Here,

$$\left\{\begin{array}{c}{E}_{systematic}=e_{\mathit{modeling}}+e_{optimizing}\\ E_{stochastic}=e_{\mathit{experiment}}\end{array}\right.$$

Indeed, the analysis of random errors in mechanical engineering can be complex. Common sources of such errors include limitations in sensor accuracy and unavoidable environmental noise, leading to measurement errors. In this study, we currently do not focus on quantifying these errors stemming from inherent priors. Instead, our main emphasis is on quantifying systematic errors.

Systematic error $E_{systematic}$ is primarily composed of $e_{\mathit{modeling}}$, which originates from the constitutive relationship of the model. The quantitative constitutive relationship serves as a dimensional reduction representation of the real system, and the information lost during this process contributes to the error. Systematic error represents the upper limit of all objective limitations of the parameter calibration algorithm. Without loss of generality, this study employs a third-order Taylor expansion to calculate $E_{systematic}$ and utilizes data-oriented numerical differentiation methods to compute gradients. The novelty of this approach lies in circumventing the issue of dimension reduction caused by deterministic constitutive relationships.

Let $X_N$ be an m-dimensional orthogonal sequence with N samples. We input this sequence into the trained PartsNet and use the predicted parameters to perform a simulation, resulting in another set of sequence data $Y_N$ with the same shape.

$$E_{systematic}=\left|{\displaystyle \frac{1}{m}}\left[{\displaystyle \sum _{i=1}^m\left(x_i-y_i\right)}-{\displaystyle \sum _{i=1}^m{\displaystyle \sum _{j=1}^{order}\frac{{\epsilon }^j\cdot \left(adaptive\_difference\left(x_i, y_i\right)\right)}{j!}}}\right]-e_{optimizing}\right|$$

(28)

The “order” parameter was set to 3 in this paper, and $\epsilon$ represents the expansion position, which we set to 10⁻⁷. This study introduces an adaptive difference algorithm to calculate the numerical derivatives of the discrete sequence. At each internal data point, the algorithm selects the smallest spacing as the step size based on the distances between neighboring data points and uses the central difference scheme to compute the derivative value. For boundary points, forward and backward difference schemes are still used. This method adjusts the step size based on the local characteristics of the data points, improving the accuracy and stability of numerical differentiation.

The optimization of the system error $e_{optimizing}$ reflects the accuracy of the optimization algorithm in parameter calibration, which is also a demonstration of the superiority of deep learning. In PartsNet, the MSE is employed to quantify the system error.

$$e_{optimizing}=MSE={\displaystyle \frac{1}{N}}{{\displaystyle \sum _{i=1}^N\left(y_i-h_{\theta }\left(x_i\right)\right)}}^2$$

(29)

Here, $h_{\theta }\left(x_i\right)$ represents the mapping relationship learned by PartsNet, and $y_i$ represent the corresponding label values.

We fully defined and quantified the systematic error. The results of the ablation experiments demonstrate that PartsNet and the GRU architecture based on the Transformer encoder outperform other networks in terms of calibration accuracy. Therefore, we focus on analyzing the errors of these two network architectures in

Table 5. Error quantization data for PartsNet and GRU with Transformer encoder.

Neural Networks	Task	Standard Error
PartsNet	Triaxial test	6.863
GRU+TransE	Triaxial test	26.622

. In the triaxial test, the standard error of PartsNet is only one-fourth of that of the other method, indicating its significantly superior accuracy. This comparative experiment demonstrates the outstanding generalization of PartsNet from the perspective of error analysis.

5. Conclusions

This paper focuses on parameter calibration in the DEM and explores a universal parameter calibration method based on deep learning. We improve upon the multi-head attention mechanism by proposing a sparse multi-head attention mechanism implemented using a sparse mask matrix. This paper also establishes an error quantification format for parameter calibration. By utilizing the Taylor expansion form of the difference function, an adaptive differencing method is introduced to establish a quantitative analysis of errors in discrete sequences. PartsNet is an innovative end-to-end solution, which efficiently calibrates constitutive parameters by integrating attention and retention mechanisms. The conclusions of this study mainly reveal two aspects:

• The attention retention fusion with the deep learning mechanism, PartsNet, addresses long-term dependencies in calibrating parameters using high-dimension sequential dynamical information on wet sand. The microscale parameters calibrated through PartsNet excellently describe the macroscopic mechanical behavior and nonlinear mechanical properties of elastoplastic wet sand, surpassing the results achieved by existing methods. The performance of PartsNet in wet sand within the triaxial compression test and its verification tests highlight its robust generalization capacity.
• PartsNet demonstrates the capability to accurately obtain the EEPA model’s constitutive parameters of elastoplastic wet sand, surpassing seven other deep learning approaches in the ablation test. In the triaxial compression calibration test, PartsNet demonstrates the best predictive accuracy (91.29%) and surpasses the Transformer encoder–GRU architecture by 1.12% while achieving a 21.25% faster convergence. Moreover, the standard error of PartsNet is only one-fourth of that of the other method.