A Deep Learning Approach for Stochastic Structural Plane Generation Based on Denoising Diffusion Probabilistic Models

Abstract

The stochastic structural plane of a rock mass is the key factor controlling the stability of rock mass. Obtaining the distribution of stochastic structural planes within a rock mass is crucial for analyzing rock mass stability and supporting rock slopes effectively. The conventional Monte Carlo method generates each parameter of stochastic structural planes separately without considering the correlation between the parameters. To address the above problem, this study novelly uses the denoising diffusion probabilistic model (DDPM) to generate stochastic structural planes. DDPM belongs to the deep generative model, which can generate stochastic structural planes without assuming the probability distribution of stochastic structural planes in advance. It takes structural plane parameters as an integral input into the model and can automatically capture the correlations between structural plane parameters during generation. This idea has been used for stochastic structural plane generation of the Oernlia slope in the eastern part of Straumsvatnet Lake, Nordland County, north-central Norway. The accuracy was verified by descriptive statistics (i.e., histogram, box plot, cumulative distribution curve), similarity measures (i.e., mean square error, KL divergence, JS divergence, Wasserstein distance, Euclidean distance), error analysis, and the linear regression plot. Moreover, the linear regression plots between the dip direction and the dip angle verified that DDPM can effectively and automatically capture the correlation between parameters.

1. Introduction

Rock mass stability is closely linked to the distribution of structural planes [1]. These structural planes intersect the rock mass, dividing it into blocks of varying shapes and sizes. When subjected to external forces, certain blocks on the free surface may slip along the structural planes, leading to instability of the rock mass; these blocks are referred to as “key blocks” [2]. Accurately determining the distribution of structural planes within the rock mass is crucial for the timely identification of key blocks and the subsequent analysis of rock mass stability [3]. While large-scale deterministic structural planes are relatively straightforward to measure, contemporary techniques such as terrestrial laser scanning facilitate the geometric identification of these planes. Conversely, the number of stochastic structural planes is large, and the scale is small and can only be generated stochastically by analyzing the statistical characteristics of the planes exposed on rock mass surfaces.

Since the 1980s, the Monte Carlo method has been widely employed for generating stochastic structural planes. This method operates by analyzing the probability distribution of each parameter and then generating stochastic numbers according to the corresponding probability distributions [4,5,6,7,8]. Rafiee and Vinches [9] combined geological statistical analysis and conventional methods to model the three-dimensional structure of rock masses; Wang et al. [10] used the Monte Carlo method to generate stochastic structural planes, based on which movable blocks were analyzed using the block theory and limit equilibrium. Sun et al. [11] used the Monte Carlo method to develop a 3D stochastic network model of the structural plane with a Baecher disk model based on the SfM photogrammetric method.

However, some studies has shown that there are correlations between parameters of structural planes, such as the correlation between the dip direction and the dip angle [6,12,13,14]. The traditional Monte Carlo method establishes probability density functions of various geometric parameters of structural planes based on measured factual structural planes and samples them according to these known probability density functions to obtain stochastic variables approximating the factual probability distribution function. This method, which independently samples each geometric parameter, ignores the correlations. Therefore, it is challenging to generate structural planes that are accurately consistent with the internal structural planes of the rock mass, and it is crucial to propose a new method for generating structural planes that considers the correlation between multiple parameters.

In recent years, deep learning has been extensively applied in content generation, exemplified by models such as ChatGPT. Deep generative models are highly generalizable, relying on generic probabilistic modelling and feature learning, which can be applied across various domains and data types. DGMs utilize deep neural networks to learn the patterns and structures of samples. Through multiple nonlinear transformations, they can automatically capture the high-dimensional features and the complex relationships within the data. In addition, they introduce latent variables in the latent space to generate new data that are similar but not identical to the samples. DGMs mainly include generative adversarial networks (GAN) [15], denoising diffusion probabilistic models (DDPM) [16], and variational autoencoder (VAE) [17]. These models have been widely used in various fields, such as image generation [18], text generation [19], natural language processing [20], and other fields [21,22,23].

DDPMs have shown significant advantages in generative tasks and have gradually become the most popular generative model [24,25]. For example, Xu et al. [26] utilized the DDPMs to predict the LOS mass-weighted number density of GMCs from column density maps in astronomy. Li et al. [27] successfully generated point–label pair generation based on a DDPM generative model. Nair and Patel [28] proposed an effective solution for V2T face translation using DDPMs. Their approach proved to be an ideal solution to generating samples from the conditional distribution of visible images given thermal images. The noise prediction module of DDPM is a deep neural network that excels at universal function approximation within numerical paradigms due to its self-learning abilities, adaptivity, fault tolerance, nonlinearity, and advanced input-to-output mapping capabilities [29]. DDPM is capable of generating data without assuming a prior probability distribution. During training, DDPM can automatically capture all associations, dependencies, and structures in the data, including higher-order features and interactions in the data, and is able to maintain the consistency of these features when generating new samples.

Other deep generative models, such as GANs and VAEs, can also be used to generate stochastic structural planes. However, DDPMs provide a more stable training process compared to the adversarial training between the generator and discriminator in GANs, and they exhibit lower sensitivity to hyperparameters. The theoretical foundation of DDPMs is based on a relatively simple diffusion process, making them easier to understand and implement compared to the variational inference framework of VAEs. In addition, DDPMs offer a wider variety of generation modes and possess an interpretable generation process [30]. Therefore, DDPMs can be considered as a novel method for generating contents such as stochastic structural planes.

To address the limitation of the traditional Monte Carlo method, which does not consider parameter correlations in the generation process, we propose a deep learning approach based on DDPM to generate stochastic structural planes. The essential idea is to utilize the characteristics of neural networks to input different parameters of structural planes into the DDPM as a whole, taking into account the correlation between the parameters while generating. The traditional Monte Carlo method ignores the correlation between the structural plane parameters when generating the structural plane. The proposed DDPM-based method considers the correlation between the parameters during generation, and the generated stochastic structural planes can be more consistent with the measured factual structured planes. This allows for more accurate identification of key blocks and analysis of rock mass stability.

The remainder of this paper is summarized as follows. Section 2 provides a detailed description of the DDPM. Section 3 introduces the overall process and verification method of this study. Section 4 explains the experimental setting, results, and validation in this paper. Section 5 discusses the advantages and shortcomings of this study and prospects for future work. Section 6 concludes the paper.

2. Background: Denoising Diffusion Probabilistic Model (DDPM)

The idea of the diffusion probabilistic models was initially introduced by Sohl-Dickstein in 2015 [31] and was further improved by Jonathan Ho in 2020 with the denoising diffusion probabilistic models [16]. The noise prediction module of DDPM employs a deep neural network, which is highly effective in universal function approximation within numerical paradigms. This effectiveness is attributed to the network’s self-learning capabilities, adaptability, fault tolerance, nonlinearity, and sophisticated input-to-output mapping [29]. DDPM is capable of generating data without presuming any prior probability distribution.

DDPM inputs and generates data as a whole, allowing for the capture of the correlation between parameters [32]. Additionally, DDPM is easier to train than other deep generation models, offers a wider variety of generation modes, and has an interpretable generation process [30]. Li et al. [27] successfully generated annotated point clouds based on the DDPM. Utilizing the DDPM and aggregating the intermediate features of the generator, a feature interpreter is proposed to convert intermediate features into semantic labels. An uncertainty metric is introduced to enhance the quality of the generated point cloud dataset, further showcasing the effectiveness and efficiency of the DDPM for sparse supervised labelling examples. Nair and Patel [28] utilized the DDPM to generate point clouds by annotating point clouds from a long-wave infrared thermal image to a corresponding visible image transformation problem. During the training, the model learns the conditional distribution of visible facial images given their corresponding thermal images through the diffusion process. In the reverse process, the visible domain image is obtained by starting from Gaussian noise and iteratively performing denoising.

Figure 1 shows the principle of DDPM; the model is composed of two main processes: (1) the diffusion process and (2) the reverse process.

2.1. The Diffusion Process

This process samples original data ($X_0$) and adds noise to it. The noise addition process is divided into T steps, with each step adding a small amount of noise to obtain a series of samples with noise ($X_0$, $X_1$...to $X_T$). As the number of steps increases, the final result, $X_T$, can be approximated as an isotropic Gaussian distribution. The process obeys the following formula: $x_t=\sqrt{1-{\beta }_t}{x}_{t-1}+\sqrt{{\beta }_t}z$; z represents the added noise. The noise added at each step is independent and follows a Gaussian distribution, with its intensity increasing with T. If the original data ($X_0$) are known, $X_T$ can be obtained at any time. The formula above can also be expressed as Equation (1), which represents the distribution of latent variables in the forward process.

$$q\left(x_t\mid x_{t-1}\right)=N\left(x_t;\sqrt{1-{\beta }_t}{x}_{t-1},{\beta }_{t}I\right),$$

(1)

where ${\beta }_t$ is the variance, ranging from 0 to 1; t is the number of steps.

2.2. The Reverse Process

In contrast to the forward diffusion process, the reverse process is the process of removing noise. In the reverse process, $X_T$ is used to gradually remove Gaussian noise at each step and produce a series of samples ($X_T$, $X_{T-1}$, $X_1$), culminating in a completely noise-free $X_0$. If the conditional probability distribution $q_{⌀}\left(x_{t-1}\mid x_t\right)$ is known, i.e., the data distribution of the overall sample is known, it is possible to iterate from $X_T$ to $X_0$, step by step, with t in the reverse process. But we do not know the data distribution of the sample; in this case, a neural network model $p_{\theta }\left(x_{t-1}\mid x_t\right)$ is needed to approximate the probability distribution. This model takes $X_0$ and t as inputs, and the outputs are of the same dimension as $X_0$. $\theta$ represents the parameters of the neural network. Since the Gaussian distribution is determined by the mean and variance, $p_{\theta }\left(x_{t-1}\mid x_t\right)$ can be expressed in the form of Equation (2), where ${\mu }_{\theta }$ represents the mean and ${\sigma }_{\theta }$ represents the variance.

$$p_{\theta }\left(x_{t-1}\mid x_t\right)=N\left(x_{t-1};{\mu }_{\theta }\left(x_t,t\right),{\sigma }_{\theta }{\left(x_t,t\right)}^{2}I\right)$$

(2)

Although $q\left(x_{t-1}\mid x_t\right)$ is not known, $q\left(x_{t-1}\mid x_t,x_0\right)$ can be derived from $q\left(x_t\mid x_{t-1}\right)$ and $q\left(x_t\mid x_0\right)$, as shown in Equation (3). Then, $q\left(x_{t-1}\mid x_t,x_0\right)$ can be used to supervise $p_{\theta }\left(x_{t-1}\mid x_t\right)$ conduct training.

$$q\left(x_{t-1}\mid x_t,x_0\right)=q\left(x_t\mid x_{t-1}\right)\ast \frac{q\left(x_{t-1}\mid x_0\right)}{q\left(x_t\mid x_0\right)}$$

(3)

It can be deduced that

$$q\left(x_{t-1}\mid x_t,x_0\right)=N\left(x_{t-1};\frac{1}{\sqrt{{\alpha }_t}}{x}_t-\frac{{\beta }_t}{\sqrt{{\alpha }_t\left(1-\sqrt{{\overline{\alpha }}_t}\right)}}z,\frac{1-{\overline{\alpha }}_{t-1}}{1-{\overline{\alpha }}_t}\right),$$

(4)

where ${\overline{\alpha }}_t=1-{\beta }_t$, $1-\sqrt{{\overline{\alpha }}_t}$ and $\sqrt{{\overline{\alpha }}_t}$ are combination coefficients whose sum of squares is 1, z represents the added noise, and t is the number of steps.

2.3. Loss Function

The aim of DDPM is to train $p_{\theta }\left(x_{t-1}\mid x_t\right)$ using the variational bound to optimize its negative logarithmic likelihood function. The formula is as follows:

$$L=E_q\left[-\log \frac{p_{\theta }\left(x_{0:T}\right)}{\mathrm{q}\left(x_{1:T}\mid x_0\right)}\right].$$

(5)

It can be derived as

$$L=D\left(KL\left(\mathrm{q}\left(x_T\mid x_0\right)\right)\parallel p_{\theta }\left(x_t\right)\right)+\sum _{t>1}D\left(KL\left(\mathrm{q}\left(x_{t-1}\mid x_t,x_0\right)\right)\parallel p_{\theta }\left(x_{t-1}\mid x_t\right)\right)-\log p_{\theta }\left(x_0\mid x_1\right).$$

(6)

From the derived formula (Equation (6)), it can be seen that the training $p_{\theta }\left(x_{t-1}\mid x_t\right)$ is intended to minimize the KL divergence of $p_{\theta }\left(x_{t-1}\mid x_t\right)$ and $q\left(x_{t-1}\mid x_t,x_0\right)$. Here, q represents the known Gaussian distribution, and $p_{\theta }$ is the distribution to be fitted. The variance of $p_{\theta }$ is constant, and we only need to approximate the mean of q and $p_{\theta }$, which is equivalent to minimizing Equation (7):

$$L_t=E_q\left(\frac{1}{2{\sigma }_t^2}{∥{\overline{\mu }}_t\left(x_t,x_0\right)-{\mu }_{\theta }\left(x_t,t\right)∥}^2\right)+C,$$

(7)

where C is a constant, and the derived Equation (7) becomes

$$E_{x_{0,ϵ}}\left[\frac{1}{2{\sigma }_t^2}{∥\frac{1}{\sqrt{{\alpha }_t}}\left(x_t\left(x_0,ϵ\right)-\frac{{\beta }_t}{\sqrt{{\alpha }_t\left(1-\sqrt{{\alpha }_t}\right)}}ϵ\right)-{\mu }_{\theta }\left(x_t\left(x_0,ϵ\right),t\right)∥}^2\right].$$

(8)

The further simplified Equation (8) can become

$$E_{x_{0,ϵ}}\left[\frac{{\beta }_t^2}{2{\sigma }_t^2{\alpha }_t\left(1-{\overline{\alpha }}_t\right)}{∥ϵ-{ϵ}_{\theta }\left(\sqrt{{\overline{\alpha }}_t}{x}_0+\sqrt{1-{\overline{\alpha }}_t}ϵ,t\right)∥}^2\right],$$

(9)

where $ϵ$ is the added Gaussian noise, ${ϵ}_{\theta }$ is a neural network for predicting the noise added from moment $x_0$ to $x_t$. The author [16] found that removing the coefficients in front of the equation is beneficial in that it improves sample quality (and simpler to implement). The optimization objective of DDPM can be expressed as (Equation (10)):

$$L_{\mathrm{simple}}={∥ϵ-{ϵ}_{\theta }\left(\sqrt{{\overline{\alpha }}_t}{x}_0+\sqrt{1-{\overline{\alpha }}_t}ϵ,t\right)∥}^2.$$

(10)

The key procedure of DDPM is to train this model for estimating noise, and the loss function is an expression of the difference between the estimation and the actual result.

3. Methods

3.1. Overview

This paper aims to generate stochastic structural planes while considering parameter correlations. The process is divided into three steps, as illustrated in Figure 2. First, observed or measured stochastic structural planes are acquired and subsequently cleaned to remove incomplete or unreasonable elements, followed by normalization of the data. Second, utilizing the processed stochastic structural plane data, we developed a deep generative model known as the DDPM to generate stochastic structural planes. This model performs joint sampling of high-dimensional data and learns the correlations between parameters. Third, various evaluation methods are employed to assess the advantages in terms of accuracy and effectiveness.

3.2. Step 1: Data Collection

To begin with, the information on stochastic structural planes exposed on the surface of a rock mass needs to be obtained using various methods. Traditional approaches involve using compasses and tape measures for on-site geological measurements, including one-dimensional scanline surveys and two-dimensional window surveys [33,34,35]. However, these traditional methods are often inefficient, requiring significant labor and time [36]. Moreover, external factors such as weather, terrain, and safety conditions can further impact the accuracy and feasibility of these measurements [34,37].

In recent years, non-contact measurement techniques have gradually become the preferred methods for measuring structural planes. These techniques include 3D laser scanning [38], close-range terrestrial digital photogrammetry [39], unmanned aerial vehicle (UAV) digital photogrammetry [40], remote sensing [41], and other techniques. They enable the collection of rock mass information without direct contact, utilizing machines to extend the measurement range. Furthermore, these techniques address limitations such as weather restrictions, making them valuable tools for rock mass assessment.

After measuring the rock slope, it is necessary to extract and calculate the occurrence of structural planes. Numerous studies have focused on extracting geometric features of structural planes. For example, Wang et al. [10] employed a three-dimensional image reconstruction technique to conduct topological identification of joints and obtained information on deterministic structural planes. Guo et al. [42] proposed a semi-automatically extraction method of rock mass structural plane information based on three-dimensional point clouds. Due to differences in the development time and genesis mechanisms of structural planes, their occurrences exhibit certain regular patterns. Therefore, structural planes should be categorized into dominant groups based on their occurrences.

Subsequently, data cleaning is required after obtaining the structural plane data. Common data errors include missing values, incorrect information resulting from input, measurement, or processing errors, and duplicates in datasets [43]. Low-quality data inevitably affect the reliability of the extracted information. The primary goal of data cleaning is to develop comprehensive and accurate methods to detect and correct errors within the data [44]. Common data cleaning tasks include checking for zero values, duplicate values, outliers, and other issues, followed by clearing and supplementing the data based on the identified outlier information.

3.3. Step 2: Model Construction

In this section, we provide a detailed description of the model underlying the proposed DDPM-based method for stochastic structural plane generation. The model is divided into six components. The first component involves importing data and using the MinMaxScaler model to normalize the structural planes to a range of 0∼1. The second component is intended to set the values of hyperparameters (T, ${\alpha }_t$, learning rate, batch_size, epoch). The third component constructs the forward diffusion function, which determines the sampling value at any given moment. The fourth part determines the model for fitting the Gaussian distribution of the reverse process, known as the noise prediction model ${ϵ}_{\theta }$. The most commonly used noise prediction models include fully connected layers and convolutional neural networks, among others. The samples for this study are structural plane data, including three parameters of dip direction, dip angle, and trace length; the input to the neural network has three features. We choose the relatively simple fully connected layer network as the noise prediction model. The fifth component is intended to determine the reverse diffusion sampling function, while the final component starts the training process.

The forward process of DDPM is a Markov process that adds noise to the data of stochastic structural planes [45,46]. The structural plane data are used as the $X_0$ input model, providing the initial data distribution. Based on the parameters set in the second part, $X_0$ was diffused for T iterations to obtain a series of $X_1$, $X_2$, $X_3$ ... $X_T$ with noise. The final $X_T$ follows a homogeneous Gaussian distribution. The reverse process gradually removes noise by running a learnable Markov chain, relying on the noise prediction model established in the fourth part.The model parameters are adjusted through network training so that the noise addition at any moment and the original samples can be predicted. The training runs for a total of n_epochs, after which the generation of structural plane data can be performed iteratively, step by step, starting from the Gaussian noise samples.

3.4. Step 3: Effectiveness Evaluation

In this section, the accuracy of the generated structural planes needs to be evaluated. The methods of evaluation include (1) descriptive statistics, (2) error analysis, and (3) similarity metric.

3.4.1. Descriptive Statistics

Descriptive statistics involve visually assessing the similarity between two sets of data using methods such as box plot comparisons, histogram comparisons, and cumulative distribution curve comparisons. Traditional methods generate stochastic numbers based on the probability distribution of the structural plane. Histogram comparisons intuitively reflect whether the probability distributions of the measured factual structural planes and the generated stochastic structural planes are identical. Additionally, cumulative distribution curves indicate the probability of data occurring within specific intervals. Box plots effectively describe the discrete distribution of data in a stable manner, minimizing the influence of outliers and facilitating comparisons of distribution characteristics across multiple data sets.

3.4.2. Error Analysis

In addition to comparing the graphs, quantitative testing is also employed to compare the generated stochastic structural planes with the measured factual structural planes. The difference ratio, which epresents the relative error percentage between the generated data and the sample data, is typically used to verify differences in means. The generated data are considered acceptable if the error is less than 30%. The formula for calculating the difference ratio is given by Equation (11).

$$\sigma =\frac{|\mu -\theta |}{\theta },$$

(11)

where $\sigma$ is the difference ratio, $\mu$ represents the mean value of a parameter of the generated stochastic structural planes, and $\theta$ represents the mean value of a parameter of the measured factual structural planes.

3.4.3. Similarity Metrics

Various similarity metrics are used to measure the difference between two datasets in machine learning. The metrics mainly include Kullback–Leibler divergence (KL divergence), Jensen–Shannon divergence (JS divergence), Wasserstein distance (W distance), Euclidean distance, and mean absolute error (MAE).

KL divergence, also known as relative entropy, measures the difference between two probability distributions and is inherently asymmetric. JS divergence is a symmetric and smooth version of KL divergence, providing a more balanced assessment of distribution similarity. Euclidean distance calculates the straight-line distance between two points in Euclidean space, offering a straightforward measure of spatial separation. W distance, also known as the Earth Mover’s Distance, measures the distance between two probability distributions in a specified metric space. MAE is the average of the absolute differences between predicted values and observed values, commonly used in regression analysis to assess the accuracy of predictive models. It is robust to outliers and provides an easily interpretable error metric. These measurement methods evaluate similarity from different perspectives; for each method, a smaller calculated value indicates a closer similarity between data distributions.

4. Results and Analysis

In this section, we apply the proposed DDPM-based generation method to a real-world rock slope and evaluate the results using various metrics.

4.1. Experimental Data and Environment

4.1.1. Experimental Data

The experimental data used in this paper are obtained from the rock slope data provided by Larissa Elisabeth Darvell [47]. The study site, Oernlia slope, is situated in north-central Norway at coordinates 15$°$43$\prime$ E, 67$°$19$\prime$ N. It is positioned to the east of Lake Straumsvatnet in Nordland County, above the mountain road which serves as an access road to the local power plant. The study area, which measures 230 m in length and lies between elevations of 280 m and 425 m above sea level, consists of a gently dipping rock slope and a talus [47]. The rock slope faces westward and rises to a height of approximately 125 m, and is predominantly composed of granitic and gneiss formations [47]. Figure 3 shows the location of the case slope and the 3D point cloud model.

In the Oernlia study area, three sets of joints were identified on west-facing rock slopes. The rocks consist of massive, high-quality granite/granodiorite with relatively few structural planes. Oernlia exhibits characteristics such as a gentle slope dip angle, specific orientations of joints relative to the slope, and the presence of curved surfaces. The dominant set of structural planes ($J_1$) has an average dip direction of 35/257, parallel to the slope [47]. $J_1$ defines the base plane for rock detachment. $J_2$ strikes northwestward, while $J_3$ dips southeastward at an angle of 66 degrees [47]. With only 68 mapped planes, joint set $J_3$ is the least prominent among the three sets. $J_2$ and $J_3$ form lateral and rear release planes for rock blocks. Their orientations are inclined with the slope, defining a wedge with an angle ranging from 60 to 120 degrees [47].

In our study, we selected three parameters—dip direction, dip angle, and trace length—to exemplify the generation of structural planes. Before conducting our experiments, we performed an extensive literature review, which revealed that scholars commonly employ the Baecher model to define the shape of structural planes. The Baecher model comprises three main components: fracture occurrence probability distribution, fracture size distribution, and fracture location probability distribution. Therefore, we adopted the Baecher model in our paper, focusing on six parameters, including dip direction, dip angle, trace length, and the center coordinates of the disc, to generate stochastic structural planes. Notably, the center point of the disc is typically stochastically and uniformly distributed. Consequently, we selected the remaining parameters—dip direction, dip angle, and trace length—for experimental generation.

For our experiment, we selected the first set of structural planes as our experimental data. The structural planes in the research area include a total of 767 pieces. We used two methods for outlier elimination. One method involved using the $duplicate\left(\right)$ function for logical evaluation, which effectively identified any duplicate values in the dataset. Also, we used the $isnull\left(\right)$ function to carefully check for any missing or empty values. There are 766 pieces of structural planes remaining after eliminating the outliers (1 piece of structural plane). To determine their respective probability distributions, we drew histograms, fitting curves, and quantile–quantile (Q-Q) plots, as shown in Figure 4. The occurrence of the first group of structural planes is listed in

Table 1. Probability distribution and value of parameters of structural planes of the first set.

Dip Direction ($°$)			Dip Angle ($°$)			Trace Length (m)
Distribution Type	Mean Value	Standard Deviation	Distribution Type	Mean Value	Standard Deviation	Distribution Type	Mean Value	Standard Deviation
Normal	36.030	9.018	Normal	258.777	12.462	Log-normal	3.95	2.803

.

4.1.2. Experimental Environment

The experimental tests were carried out on a Windows 10 system with 16 GB of memory, an AMD Ryzen 7 5800H processor with Radeon Graphics 3.20 GHz, and an NVIDIA GeForce RTX 3060 graphics card. The programming language is Python 3.8.8, and the deep learning framework is PyTorch1.9.0.

In the experiments, we did not differentiate between the training set and the test set due to the limited number of structural planes. We employed various methods to enhance the efficiency of DDPM during the experiment. Given the small number of samples and features, we reduced the model size to simplify it without significantly impacting performance. For example, we continuously adjusted hyperparameters (e.g., batch_size, learning rate, number of epochs) during training to achieve faster convergence and improved performance. In addition, we implemented early stopping based on the calculated loss to prevent overfitting.

The noise prediction model ${ϵ}_{\theta }$ utilized fully connected layers, comprising five hidden layers. The $ReLU\left(\right)$ function was applied to each layer, with 256 nodes configured. The training process starts with the forward diffusion process, where Gaussian noise is added to the sample data progressively at a time step of T. This method iteratively produces noisy samples, following the guidelines set forth in Equation (7) in Section 2.1. Subsequently, the noise prediction model is trained to forecast the noise introduced at each time step. This training is accomplished by optimizing the loss function, which measures the discrepancy between the actual noise and the predicted noise. The underlying rationale for this process is based on the formulae presented in Section 2.3. The reverse process then denoised the Gaussian noise to generate data, utilizing the predicted noise distribution as described by the formulas in Section 2.2.

Parametric trial-and-error methods are employed to iteratively fine-tune hyperparameters throughout the training process to achieve optimal performance. These hyperparameters typically include the learning rate, the number of diffusion steps, batch size, and model architecture parameters such as the number of layers and the number of hidden units. Various combinations of these parameters (e.g., learning rate = 0.00001, step size = 100, batch_size = 128) were systematically tested, and the outcomes were visualized using histograms to elucidate the impact of each hyperparameter. This iterative adjustment process continued until optimal performance was attained. Ultimately, the total number of steps T was set to 1000, with a batch_size of 16. The Adam optimizer was employed with a learning rate of 0.0001, and the training process comprised 4000 epochs.

4.2. Results Generated by the Conventional Monte Carlo Method

Based on the probability distribution of structural planes analyzed in Section 4.1.1, we generated the dip direction, the dip angle, and the trace length separately. A total of 766 stochastic structural planes were generated to facilitate comparison with the measured factual structural planes. As shown in Figure 5, the histograms of the measured structural planes are compared with the histograms of the generated stochastic structural planes using the conventional Monte Carlo method.

Figure 6 displays box plots comparing the parameters of the structural plane generated by the conventional Monte Carlo method with those of the measured factual structural planes. Upon observing the three sets of box plots, it is evident that the overall distribution of each parameter is relatively similar to the distribution of the corresponding parameter of the measured factual structural planes. It can be considered that the generated stochastic structural planes are well-fitted with the measured factual structural planes.

Figure 7 presents a comparison of the cumulative distribution curves for each parameter between the generated structural planes using the conventional Monte Carlo method and the measured factual structural planes. The horizontal axis denotes the parameter value, while the vertical axis represents the probability.

4.3. Results Generated by the Proposed DDPM-Based Method

In this section, we present the results of comparing the stochastic structural planes generated using the proposed DDPM-based method with the measured factual structural planes.

DDPM takes three parameters—dip direction, dip angle, and trace length—as inputs and generates them simultaneously as a whole. Using the proposed DDPM-based method, 766 pieces of stochastic structural planes were generated, and the generated results are also consistent with the measured factual structural planes. Figure 8 illustrates a comparison of the histograms between the measured factual structural planes and those generated using the proposed DDPM-based method.

Figure 9 illustrates the box plots comparing the parameters of the structural plane generated by the proposed DDPM-based method with those of the measured factual structural planes. Upon observing the three sets of box plots, it is evident that the overall distribution of each parameter is relatively similar, and it can be considered that the generated structural planes are well-fitted with the measured factual structural planes.

Figure 10 illustrates the comparison of the cumulative distribution curves of each parameter of the generated structural planes using the proposed DDPM-based method and the measured factual structural planes. The horizontal axis represents the parameter value, and the vertical axis represents the probability.

4.4. Comparison of Results and Analysis

Through a statistical comparison of the measured factual structural planes and generated stochastic structural planes, we also conducted a quantitative analysis. Initially, similarity measurements were employed to evaluate the resemblance between the two sets of structural planes.

Table 2. Similarity measurement (KL divergence, JS divergence, and Euclidean distance) for measured factual structural planes and structural planes generated using the conventional Monte Carlo method and the proposed DDPM-based method.

Method	KL Divergence			JS Divergence			Euclidean Distance
	Dip Direction	Dip Angle	Trace Length	Dip Direction	Dip Angle	Trace Length
Monte Carlo	0.092	0.003	0.62	0.02	0.0006	0.095	614
DDPM-based	0.094	0.003	0.65	0.02	0.0006	0.099	616

and

Table 3. Similarity measurement (Wasserstein distance and MAE) for measured factual structural planes and structural planes generated using the conventional Monte Carlo method and the proposed DDPM-based method.

Method	Wasserstein Distance			MAE
	Dip Direction	Dip Angle	Trace Length	Dip Direction	Dip Angle	Trace Length
Monte Carlo	0.95	0.46	0.32	10.03	14.34	2.83
DDPM-based	0.56	0.55	0.33	10.28	14.04	2.96

list the measurement results of comparing the stochastic structural planes generated by the conventional Monte Carlo method and the proposed DDPM-based method with the measured factual structural planes.

In addition to employing similarity measures, the difference ratio was also utilized as a metric by which to evaluate the generated results.

Table 4. Difference ratio of the mean value and standard deviation between the generated stochastic structural planes and the measured factual structural planes.

	Dip Direction (°)			Dip Angle (°)			Trace Length (m)
	Mean Value	Error	Standard Deviation	Mean Value	Error	Standard Deviation	Mean Value	Error	Standard Deviation
Measured	36.030	-	9.018	258.777	-	12.462	3.95	-	2.803
Generated by Monte Carlo method	36.347	0.88%	9.183	258.805	0.01%	12.599	4.014	1.62%	2.830
Generated by DDPM-based method	36.210	0.50%	9.146	258.490	0.11%	12.465	4.201	6.35%	2.924

presents a comprehensive comparison between the mean and variance of the stochastic structural planes generated using the Monte Carlo method and the proposed DDPM-based method with the mean and variance of the measured factual structural planes. Relative errors are calculated accordingly, revealing a maximum relative error of 6.35%. Specifically, the maximum absolute error for dip direction and dip angle does not exceed 1°, while the maximum absolute error for trace length does not exceed 0.3m. These error values fall within an acceptable range of tolerance.

The presence of errors can be attributed to factors such as inaccurate measurements, missing data, or other sources encountered during the measurement of structural planes [48,49]. These factors compromise data accuracy, consequently affecting the quality of generation. Moreover, the Monte Carlo method simplifies the process by using a single probability distribution formula to describe structural plane distributions, relying on underlying assumptions [48]. The estimation of parameters for these probability formulas based on measured structural planes, which inherently possess a certain degree of error, can result in the generation of stochastic structural planes accompanied by errors. Similarly, as for the proposed DDPM-based method, the generation process is influenced by sample data and noise [50], and a certain degree of error may exist even if the model consists of a highly complex network.

In addition, Monte Carlo is a sophisticated method with extensive applications in the field of structural plane generation. For this method, its principle is a simple single-function fitting, which effectively captures the distribution patterns of parameters in data with clear regular distributions. Notably, this method is relatively independent of sample size, with the degree of fit remaining largely unaffected by increases in sample size. In contrast, DDPM is a deep learning model primarily composed of deep neural networks, which consist of hundreds or thousands of nonlinear transformation functions. This model facilitates the acquisition of more abstract representations, exhibiting robust expressive capabilities compared to conventional methods. However, this powerful representation learning ability requires sufficient samples to avoid overfitting and ensure training accuracy.

In our case study, the limited number of structural planes in the rock slope impacts the quality of the stochastic structural planes generated by the proposed DDPM-based method. Therefore, compared to the structural planes generated by the Monte Carlo method, the structural planes generated by DDPM perform worse in some aspects. However, overall, the structural planes generated by the proposed DDPM-based method are accurate. The structural planes presented in Table 4 further demonstrate that the generated structural planes are consistent with the measured factual structural planes.

The essential idea behind this paper is to generate stochastic structural planes and automatically capture the correlation between dip direction and dip angle using neural networks. In addition to the use of various verification methods to prove the accuracy of the stochastic structural planes generated by the proposed DDPM-based method, we also draw linear regression curves of the dip direction and the dip angle in this section to demonstrate the advantages of the proposed DDPM-based method.

Figure 11 shows the linear regression plots plotted for dip direction and dip angle for each set of structural planes. In the linear regression plots, the symbol “r” represents the correlation coefficient, which serves as a metric to quantify the strength of the linear relationship between two variables [51,52]. The correlation coefficient, ranging from −1 to 1, serves as an indicator of the direction and magnitude of the correlation. Negative values indicate a negative correlation, positive values indicate a positive correlation, and a larger absolute value of “r” represents a stronger correlation.

In Figure 11, (a) shows the linear regression curve of the dip direction and the dip angle of the measured structural surface, with a correlation coefficient of −0.685, indicating a significant negative correlation. In contrast, (b) is the linear regression curve of the dip direction and the dip angle of stochastic structural planes generated by the conventional Monte Carlo method, with a correlation coefficient of −0.046, suggesting a near absence of correlation. Moreover, (c) describes the linear regression curves of the dip direction and dip angles of the stochastic structural planes generated by the proposed DDPM-based method with a correlation coefficient of −0.599, indicating a strong negative correlation. This analysis leads to the conclusion that the proposed DDPM-based method successfully captures the correlation among the structural plane parameters. This phenomenon can be attributed to the inherent mechanisms of deep learning models. During the training process, deep generative models are typically trained on a specific dataset with the objective of minimizing an optimization function. By iteratively optimizing this function, the model acquires the ability to encode and decode data while preserving the interconnectedness of various features. Consequently, the statistical patterns and correlations inherent in the training data are effectively captured when generating real samples [32].

5. Discussion

The validation results demonstrate the effectiveness of the proposed DDPM-based method for generating stochastic structural planes. This section discusses the advantages and shortcomings of this method and suggests potential improvements for future research.

5.1. Advantages of the Stochastic Structural Planes Generation Using the Proposed DDPM-Based Method

One significant advantage of using the proposed DDPM-based method for generating stochastic structural planes is its ability to learn data distribution without requiring prior probability distributions. Traditional methods rely on pre-assumed probability distributions, such as normal, uniform, or exponential distributions, to generate stochastic structural planes based on probability statistics. However, the distribution of structural planes is complex and cannot be accurately described by a single probability distribution. DDPM, as a type of deep generative model, employs multiple layers of neural networks containing hundreds of non-linear functions. This architecture enables the model to approximate complex high-dimensional probability distributions without needing prior probability distributions [53,54]. The validation results presented in Section 4 demonstrate that DDPM effectively learns the probability distribution of the measured factual structural planes.

The second advantage of using the proposed DDPM-based method for generating stochastic structural planes is its ability to learn data distribution while accounting for parameter correlations. Traditional structural plane generation methods often fail to effectively capture the correlations between structural plane parameters. In contrast, deep learning models can utilize feature unions as an overall input, enabling the model to automatically extract features from high-dimensional data. In this study, the three parameters—dip direction, dip angle, and trace length—are collectively input into the model for training. As shown in Figure 11, the correlations between the dip direction and the dip angle are successfully captured by DDPM.

5.2. Shortcomings of the Stochastic Structural Planes Generation Using the Proposed DDPM-Based Method

One issue with the proposed DDPM-based method is its inefficiency. This is due to the fact that DDPM is more computationally complex compared to the Monte Carlo method. As detailed in Section 2, the generation process consists of two parts: the diffusion process and the reverse process. The diffusion process requires T-steps and constant iterations to add noise to the sample data, while the recovery of samples from Gaussian noise also necessitates continuous iterations. In addition, the noise prediction network of DDPM consists of a deep neural network, involving a large number of node parameters that must be optimized. Therefore, the hyperparameters of the model need to be constantly adjusted to achieve optimal performance. In contrast, the traditional Monte Carlo method establishes probability density functions of various geometric parameters of structural planes based on measured factual structural planes and samples them according to these known probability density functions to obtain stochastic variables approximating the factual probability distribution function. The Monte Carlo method is comparatively less complex to comprehend and implement than the DDPM. For the generation of structural planes using the proposed DDPM-based method in this research, 766 pieces of structural plane samples were used, with the final training time amounting to approximately 880 s. This generation speed of DDPM is considerably slower than that of the Monte Carlo method.

In addition, as can be seen from Section 4, the DDPM-based method performs worse in some aspects compared to traditional Monte Carlo method. Monte Carlo is a sophisticated method with extensive applications in the field of structural plane generation. For this method, its principle is a simple single-function fitting, which effectively captures the distribution patterns of parameters in data with clear regular distributions. This method is relatively independent of sample size, with the degree of fit remaining largely unaffected by increases in sample size. DDPM is a deep learning model primarily composed of deep neural networks, which consist of hundreds or thousands of nonlinear transformation functions. This model facilitates the acquisition of more abstract representations, exhibiting robust expressive capabilities compared to conventional methods. However, this powerful representation learning ability requires sufficient samples to avoid overfitting and ensure the accuracy of training. In our case study, the limited number of structural planes in the rock slope impacts the quality of the stochastic structural planes generated by the DDPM-based method. Therefore, compared to the structural planes generated by the Monte Carlo method, the structural planes generated by DDPM perform worse in some aspects. In the future, we will explore deep generative models designed for scenarios with small sample sizes to further improve the accuracy of the generated results.

5.3. Future Work and Outlook

Our future work aims to enhance the convergence speed of the proposed DDPM-based method without compromising the quality of generation. This will enable us to maintain the high efficiency of our approach, even when applied to large and complex slopes with numerous structural planes.

The focus of this paper is to address the limitation of traditional method in generating structural planes, i.e., the inability of traditional methods to take into account parameter correlations. The traditional method regards the structured plane as the Baecher model and focuses on six parameters of the structural plane, including dip direction, dip, trace length, and center coordinates of the disc when generating the structural plane [10,47]. These structural planes are distributed stochastically in a certain size of 3D space. Before the generated structural planes are visualized, the size of the 3D cube containing the structural planes is determined based on the specific characteristics of the rock slope. The center coordinates of the disk are generated through a random generation method, where random values are produced along the X, Y, and Z axes within the boundaries of the cube’s side lengths and then combined to form the center coordinates of the disk. The number of generated coordinate points depends on the density of the structure planes. Therefore, the proposed DDPM-based method is employed to generate three parameters: dip direction, dip angle, and trace length of the structural plane. In future work, we intend to explore additional parameters such as the relative position of the structural plane, the shape of the structural plane, and the strength of the structural plane. By doing so, we aim to enhance the comprehensiveness of our study and facilitate the generation of structured planes more effectively.

In addition, we consider using DDPM to analyze complex, abstract, and comprehensive spatial patterns in rock formations based on geological data, guiding the 3D reconstruction process of complex geological models and further enhancing the geological interpretability of the reconstruction results.

6. Conclusions

The distribution of structural planes plays a vital role in the stability of the rock mass. Stochastic structural planes, in particular, are challenging to observe due to their small size, large number, and random distribution. Therefore, accurately and promptly identifying key blocks and analyzing the stability of the rock mass requires obtaining the internal stochastic structural plane distribution. Currently, the Monte Carlo method is the most widely used approach for generating stochastic structural planes. This method establishes probability density functions of various geometric parameters of structural planes based on measured factual structural planes and samples them according to known probability density functions to obtain stochastic variables that are approximate to the factual probability distribution function. This method, which independently samples each geometric parameter, cannot capture the correlations between the parameters.

To address this limitation, in this paper, we propose a deep learning approach based on DDPM for generating stochastic structural planes. The advantage of using a neural network for generation is its ability to learn high-dimensional and complex sample distribution features without requiring a priori probability distributions. Moreover, by inputting each parameter into the neural network simultaneously, the method can automatically capture the correlations between structural plane parameters.

The study applies both the DDPM-based method and the conventional Monte Carlo method to generate stochastic structural planes on the Oernlia slope in Nordland County, Norway. Section 4 presents the results of the structural plane generation, using various validation methods, including histograms, box plots, and cumulative distribution curves, to visualize data distribution, as well as quantitative analysis methods such as mean square error, KL divergence, JS divergence, Wasserstein distance, and Euclidean distance. Additionally, error analyses of mean and variance and linear regression plots illustrating the correlation between dip direction and dip angle were performed. The verification results indicate that (1) the proposed DDPM-based method is able to generate stochastic structural planes that are well with the measured factual structural planes, which verifies the accuracy of this method; (2) the proposed DDPM-based method can automatically capture the correlation between the dip direction and the dip angle; (3) the stochastic structural planes generated by the proposed DDPM-based method are consistent with the original measured structural planes. The proposed DDPM-based method for generating stochastic structural planes is reliable.