1. Introduction
As a key method in underground mining, cemented paste backfill involves mixing tailings, water, and cement into a slurry, which is then transported through pipelines to fill mined-out tunnels, providing essential ground support [
1]. A retaining wall is often constructed to contain the backfill, allowing the slurry to harden in a controlled area, thereby reinforcing the surrounding rock and ensuring safe operations [
2]. A nonstop backfill process is crucial, as overfilling, rapid filling, or poor curing can lead to accidents, property damage, production losses, or operational shutdowns [
3].
In this regard, various research studies have been implemented to monitor the filling condition and guarantee a better filling effect. A comprehensive monitoring system, including total earth pressure cells, water pressure/suction cells, electromagnetic transducers, etc., was utilized to study poorly understood phenomena such as stress arching within the backfill mass, temperature-induced stresses, and the retaining wall pressure reduction with increasing setback from the stope [
4]. In regard to the retaining wall, the in-stope pressure monitoring instrument for the retaining wall has been deployed to determine the relationship between the rise rate of cemented paste backfill and the moderating effect of cement hydration on in situ pressures [
5]. Inside the backfill material, roof subsidence sensors and stress sensors were installed during coal mining and backfilling to monitor the variations in subsidence and stress, thus the stress distribution in the coal mine gob was analyzed [
6]. To investigate the variation law of the stope backfill strength along the flow direction of the slurry, the VS150 three-dimensional laser scanner was used to scan the final flowing track of slurry 28 days after the completion of the backfill process [
7]. In the wider area, Zhang et al. [
8] conducted field monitoring of cemented paste backfill in an underground coal mine, including measurements of temperature, humidity, stress conditions inside the backfill body, and displacement. This provides guidance for backfilling design and predictions of surface subsidence in underground mines. Backfill grouting is also generally used to limit the ground surface settlement induced by shield tunneling, which also provides valuable guidance for backfill monitors in underground mines. Ground-penetrating radar (GPR) was employed in a shield tunneling machine to monitor the thickness and volume of the grouting layer, successfully detecting several defects [
9,
10]. In recent years, advanced technologies such as intelligent sensors and artificial intelligence have been used to detect subsidence and assess the stability of backfill bodies, making backfill monitoring safer and more efficient [
11,
12,
13,
14].
However, former studies on backfill monitors, as highlighted earlier, have primarily focused on the physical and mechanical parameters during the backfill process detected using different types of sensors, with the aim of ensuring a stable backfill system; nevertheless, methods for monitoring backfill progress are still insufficient. In general mining operations, the methods for monitoring the backfill progress are often rudimentary. These typically involve using indirect indicators like the rate of material usage and pumping pressure. Without advanced sensors or real-time data monitoring, miners rely on experience, occasional sampling, or manual gauges to estimate when the backfill process is nearing completion. If there were a way to digitize and visualize the backfill progress, we could monitor the process more intuitively, identify issues in a timely manner, prevent the waste of backfill materials, and reduce costs.
Among the methods for digitizing and visualizing three-dimensional scenes, LiDAR sensors and related technologies have rapidly developed in recent years, driven by the surge in autonomous driving applications [
15]. LiDAR emits laser beams into the surrounding environment and receives the reflected laser signals. By measuring the round-trip time of the laser, it can determine the position and shape of objects in three-dimensional space. These data are compiled into a point cloud, which is a collection of points distributed in space. Through a series of point cloud processing algorithms, the scene can be modeled with high precision [
16,
17]. In underground structures, where dust concentration is high and light conditions are poor, traditional visual monitoring schemes are not suitable. LiDAR is not sensitive to ambient light brightness and has strong anti-interference capabilities, making it widely used in underground environments. Methods for processing chronological LiDAR data obtained in underground tunnels typically involve comparing the latest scan with previous ones to identify and calculate tunnel deformations [
18,
19,
20,
21,
22,
23]. LiDAR can be used to estimate the volume of blasted rock [
24] and load volume in trucks [
25,
26] by adding the top surface of materials scanned by LiDAR to the container boundaries.
In this work, LiDAR technology was applied to obtain the three-dimensional spatial form of the backfill process. To digitize and visualize backfill progress, a simulation environment of the paste backfill process was constructed using CARLA, a flexible and open-source simulator designed for autonomous driving research, allowing for the customization of sensor suites and environmental conditions [
27]. Algorithms to compute the backfill progress were tested within this simulation environment, and a LiDAR-based backfill monitoring system was designed and assembled to perform the aforementioned functions.
3. Methods
The backfill progress is expressed as the volume of the backfill material divided by the volume of the initial tunnel (Equation (
3)). Thus, the key to calculate the filling progress is to find the slurry surface, then reconstruct the mesh model of the intersection body, and estimate the volume of the filling body at every moment. The approach used in calculating the backfill progress is shown in
Figure 3. As long as the backfill monitor remains open, these three steps will continue to cycle:
Here, P represents the backfill progress, is the volume of the backfill material, and denotes the initial tunnel volume.
The following steps are used in this process:
(1) Slurry surface fitting involves applying the ICP (Iterative Closest Point, which aligns two point clouds by iteratively minimizing the distance between corresponding points) registration algorithm [
30] to align the initial tunnel point cloud with the backfilling point cloud, and capturing the point cloud data representing the top surface of the backfill material. After completing step 1, the top surface is located and the point cloud of the surface boundary is obtained (see
Section 3.1);
(2) Generate points of slurry surface, and determine which points belong to the slurry body. After completing step 2, the entire point cloud model of backfill body is obtained (see
Section 3.2);
(3) Reconstruct the point cloud model into a 3D mesh, and compute the volumes of both the slurry body and the original tunnel in order to generate the backfill progress output (see
Section 3.3).
3.1. Slurry Surface Fitting
Similar to methods that combine the top surface of the load and the truck bucket to estimate load volume, the backfill monitoring system estimates the volume by combining the backfill material with the space below its boundary [
25,
26]. Previous methods deployed LiDAR to scan the top surface of the trucks and directly used these data to reconstruct the load model. However, the slurry top surface remains nearly level over a large area according to the rheological properties of paste [
31]. Instead of directly using the point cloud of the top surface, this method locates the entire slurry surface by using RANSAC (Random Sample Consensus, which fits models to data with outliers by randomly sampling subsets to find the best fit) algorithm to obtain the coefficients of the plane equation from data within the LiDAR range [
32]. The coefficients indicate the location of the top surface in geometric space. This is equivalent to assuming that at locations far from the LiDAR, the head height remains the same as the rising slurry surface. The estimated volume would then represent the entire tunnel volume beneath the slurry top surface.
If the plane-fitting algorithm is directly used for the point cloud model, either the slurry plane needs to be screened out among a large numbers of fitted planes, or the fitting results are possibly not correct. Since the results of each LiDAR scan are stored in chronological order, which is closely related to the process of the rising backfill slurry, as long as the subsequent scans are compared with the initial scans, the rising slurry surface with a few noise points can be screened out, as follows:
The first ten scans are merged and named cloud_base. The subsequent point cloud models are merged by every ten scans and the merged model is named cloud_process. Each point in cloud_process is iterated and placed into the three-dimensional space of cloud_base. For each point in cloud_process, if there are fewer than three points from cloud_base within the surrounding spherical space defined by a radius threshold, this point in cloud_process should be classified as an outlier, so that the backfill slurry surface can be roughly screened out.
On the basis of the preliminary screening, the RANSAC algorithm can be used to accurately find the backfill slurry surface, and the coefficient of the slurry plane equation can be obtained. The intersection boundary of the backfill slurry surface and the tunnel can be obtained by substituting the coefficients of the plane equation back to the tunnel.
3.2. Slurry Points Determination and Generation
This step aims to generate the complete point cloud model of the backfill body, consisting of two main components.
3.2.1. Slurry Points Determination
Assume the plane equation of slurry surface is ax + by + cz + d = 0 and a point P (A, B, C) is not on the surface. The signed distance from point P to the surface is the result of substituting the coordinate of P to the plane equation. A positive value indicates that point P is on one side of the plane, while a negative value signifies it is on the opposite side. Therefore, points under the backfill slurry surface can be determined as follows:
In the two three-dimensional spaces divided by the slurry surface, it is crucial to select a point that consistently lies on the opposite side of the slurry points. The product of the signed distance from this point to the slurry surface and the signed distance from slurry points to the slurry surface must be negative. Since the Livox Mid-40 LiDAR always maintains itself as the origin, and the scanning space is a conical space in the positive direction of x-, y-, and z-axes, thus the point (−1, 100, 100) on the upper space above the slurry surface is selected. The procedure is illustrated in
Figure 4.
3.2.2. Surface Point Generation Using the Moving Sampling Method
The process of 3D surface reconstruction creates a continuous 3D surface from a set of scattered 3D point data. It is necessary to ensure that the point cloud model is watertight, otherwise the void will cause the normal vector to be miscalculated, and the generated 3D grid model may not be watertight. However, according to the work in the previous section, substituting the coefficient of the plane equation into the tunnel point cloud model can only obtain the intersection boundary between the backfill slurry surface and the tunnel. However, there is no point on the slurry plane. Therefore, it is necessary to generate points on the slurry surface.
Since the front of the LiDAR scanning space is the positive direction of the x-axis, which is also the orientation of the tunnel, the moving sampling method can be used to generate plane point clouds according to this feature. First, the points with the maximum and minimum x values in the intersection boundary between the slurry surface and the tunnel are obtained, and named as x_max and x_min, respectively. Sampling points are arranged every certain distance from x_min to x_max. If there are more than two points on the intersection boundary equal to the x-coordinate of the sampling point, a line point cloud is uniformly generated on the line between the two farthest points between these points. By moving the sampling point from x_min to x_max according to the above method, a uniform point cloud covering the plane of the intersection boundary can be generated, as illustrated in
Figure 5.
3.3. Backfill Progress Computation
Poisson surface reconstruction is to generate high-quality 3D surface models from a given set of disordered point cloud data [
33]. By calculating the normal vector of the point set and applying the Poisson surface reconstruction algorithm, meshes of the tunnel and the backfill body are obtained, which can facilitate the subsequent backfill volume calculation and visual monitoring of the backfill progress.
The mesh is regarded as a tetrahedron composed of multiple triangular faces and the coordinate origin. The volume of each tetrahedron can be obtained through the following vector operations [
34]:
where
are the coordinate vectors of the three vertices that define the tetrahedron.
When encoding all vertices of a mesh into triangles, the computer stores all the vertices sequentially in a list. Triangles are defined by vertex indices, which point to specific vertices in the list. For example, a triangle might be defined by the indices [0, 2, 5], indicating that it is composed of the 1st, 3rd, and 6th vertices in the vertex list. The order of the triangle vertices (clockwise or counterclockwise) determines the direction of the normal vector, which affects the orientation of the face. According to the right-hand rule, if the vertices of a triangle face are arranged clockwise, the face is oriented toward the origin, representing the front. The signed volume of tetrahedrons formed by the fronts and the origin are positive, and tetrahedrons formed by the backs and the origin are negative (
Figure 6). By calculating the sum of the signed volumes of all tetrahedrons, the result is the volume of the entire mesh.
4. Experiment
A simulation environment is constructed based on the real-world workflow’s construction steps, device arrangement, and communication methods to replicate real-world conditions of the cemented paste backfill process in underground mines. The proposed methods for digitizing and visualizing the backfill progress are tested within this environment, and the results of each step are presented in this section.
4.1. The Operational System Workflow in Mines
The operational workflow of the proposed system applied in underground mines is illustrated in
Figure 7. The initial information of the tunnel before backfill should be obtained by a technician before starting filling the tunnel with the backfill material. The LiDAR is used to scan the target tunnel from the top to the end, point cloud data are recorded and save as an ROS bag (a file format used for storing message streams in Robot Operating System [
35]). The SLAM (Simultaneous Localization And Mapping) [
36,
37] technology is used to build the three-dimensional point cloud map of the tunnel. This initial map is detailed and precise, showcasing physical characteristics and arrangement of rocks. When the retaining wall is under construction, it is necessary to consider grooving the top of the wall to fix the backfill monitor. As the backfill material slowly enters the tunnel, the LiDAR begins operating.
The backfill 3D geometric model generated in the NUC can be stored in various digital formats, which can be directly imported and visualized in most 3D model processing software. After the three-dimensional model of the backfill body is obtained, the tunnel model and the slurry model are visualized with different marks and colors to visually display the backfilling situation. Based on the communication networks used in the mine where the backfill monitor is deployed, the calculation results are transmitted to the mining control center via LAN (Local Area Network), 5G, or WiFi [
38,
39,
40]. Therefore, the control center can monitor the backfill progress and make corresponding control measures according to all kinds of emergencies.
4.2. Simulation Environment of Backfill
For tunnel paste backfill simulation, CARLA is not only necessary to edit the dynamic properties of the backfill body to simulate the backfill process, but also to build different lengths and shapes of tunnels to improve the generalization ability of the backfill monitoring model. The flexible environment editing capability in CARLA can meet the above requirements. The simulation environment of backfill is designed correspond to the on-site backfill scenario described in
Section 4.1.
In CARLA, three different shapes (straight, slightly curvy, and curvy) of tunnels are chosen from the 3D model of an underground mine, the dip angle of each tunnel is 5°, and the selected filling length is about 170 m (
Figure 8). The simulation represents an idealized scenario where the tunnel is not supported by any structures such as rock bolts or reinforcing mesh, and there are no backfill dams controlling the backfill process. First, retaining walls are placed at the bottom of the tunnels, and the tunnels are mapped by SLAM technology to obtain the initial point cloud data. Secondly, a new retaining wall is built at the entrance of each tunnel (
Figure 9a), and a space on the top of the retaining wall is reserved for placing the LiDAR sensor(
Figure 9b). A LiDAR sensor is set up that meets the basic parameters of the Livox Mid-40 LiDAR, and the measuring ranges of 100 m and 200 m are designed, respectively, to simulate application scenarios in various lengths of tunnels. Finally, three cuboids are placed beneath tunnels and edited to rise continuously (
Figure 9a), simulating the process of filling the tunnel with slurry. Two cubes are placed in the surrounding rocks of each tunnel to construct correspondences for the registration algorithm. The diagram of LiDAR scans at three different times and the point cloud of the unfilled tunnel are shown in
Figure 10. Obviously, the only change in the scanning area is the red slurry top surface, due to the fixed position of the LiDAR placement. By processing each LiDAR scan, the backfill progress of different periods can be calculated.
To improve the generalization ability of the filling monitoring model, different simulation environments are designed. Parameters of simulation environments are presented in
Table 2. According to the mechanism of backfill progress computation and the experiment results, results are not affected by different dip angles. Therefore, only tunnels with a 5° dip angle are adopted.
4.3. Results of Each Step
According to the backfill progress computation workflow described in
Section 3, the processing of a set of LiDAR scans includes three steps, representing one complete cycle of computing the current backfill progress. Algorithms were tested in the simulation environments of backfill (see
Table 2), and results of LiDAR with ranges of 100 m were selected to illustrate the results of each step, as shown in
Figure 11.
In step 1, the intersection boundary of the tunnel and the backfill slurry surface was extracted. For the purpose of improving the computing speed, the initial tunnel point cloud has been downsampled, making the intersection boundary sparser. As a result, the surface point cloud generated by the moving sampling method in step 2 is sparser, but this does not affect the subsequent surface reconstruction and volume calculation results. In step 3, the mesh of the filling body generated by Poisson surface reconstruction is illustrated above the point cloud result, which shows the filling condition inside the tunnel. Dragging the left mouse button allows for monitoring the filling progress from different angles, but the computation results are always oriented towards the user.
The computation results of step 3 are presented using the visualization tool Rviz [
41]. The visualization results include the volume of the slurry and the initial tunnel and the division of slurry volume and tunnel volume. For each set of LiDAR scans processed, the slurry volume and backfill progress are updated.
4.4. Accuracy
An accurate method to measure the progress of the backfill process is not available, so the precision can be evaluated by comparing it with other approaches, such as Monte Carlo-based volume estimation [
42] and Hexahedrization [
43].
The Monte Carlo method can be used to estimate the volume of a point cloud by randomly sampling points in a defined bounding box and determining the proportion of points that fall within the point cloud. Here is a step-by-step outline of how it works:
(1) Define the Bounding Box Volume: Enclose the point cloud within a simple geometric shape with a known volume.
(2) Generate Random Points: Randomly generate a large number of points within the bounding volume.
(3) Count Points Inside: Check how many of the random points fall within the point cloud.
(4) Estimate Volume: Calculate the point cloud volume using Equation (
5)
The Hexahedrization method involves filling the model with a finite number of small hexahedra and then summing the volumes of all the hexahedra to obtain the volume of the model (Equation (
6)). The steps are as follows:
(1) Define the bounding box, then divide the entire bounding box into hexahedral cells according to the provided hexahedral element dimensions.
(2) Perform planar cuts through the center of the bounding box along any axis and perpendicular to that axis, generating a series of cross-sections for the surface model.
(3) In each cross-section, calculate the volume of the hexahedra within the contour line of the model and sum them up.
where
is the volume of the
i-th hexahedron, and
n is the total number of hexahedra.
Given that the development of volume computation algorithms is relatively mature and the tunnel model is not particularly complex, the results from several methods are quite similar. Both of the results obtained using the methods described above show less than 1% difference compared to the result computed by the method in
Section 3.3. On the premise of ignoring the errors generated by other steps, such as the low scanning accuracy of LiDAR and the low Poisson surface reconstruction accuracy, it can be seen that the accuracy of backfill monitor is high. By further upgrading each step in the backfill progress workflow, such as applying LiDAR sensors with a higher range and point rate, and optimizing the RANSAC and surface reconstruction algorithms, the overall accuracy could be improved.
4.5. Processing Speed
The time to process each set of scans in different LiDAR ranges is shown in
Figure 12. Inside the slightly curvy tunnel, the top space of the retaining wall is larger than that of the other tunnels, causing the progress to end at around 80%. Additionally, the curvature of the tunnel causes the laser beam to be blocked, delaying the start of progress monitoring in such tunnels. While increasing the LiDAR range (e.g., to 200 m) can allow earlier detection of the slurry top surface, it remains constrained by obstructions at corners.
Overall, as the size of the backfill body increases, the time taken to process each set of scans becomes longer. The time is not affected by the curvature of tunnels, but the number of points in each LiDAR scan takes a toll of the processing speed. Nonetheless, the maximum time to process a set of scans is 20 s. For the straight tunnel with a volume of 8990 m
3, there is up to a 0.006% increase in backfill progress within 20 s, based on a flow rate of 100 m
3/h. Determining the critical flow velocity of the backfill mixture is crucial for accurately estimating the backfilling time [
44]. In conclusion, the backfill monitor is sensitive, and is capable of capturing the rising slurry top surface and effectively monitoring the backfill progress.
5. Conclusions
A LiDAR-based backfill monitoring system was designed to monitor the process of slurry entering and filling the mined-out tunnel. LiDAR technology was used to obtain the constantly changing spacial distribution shape of the filling process. An NUC was connected to the LiDAR to compute the backfill progress and return the real-time processed results to the mining control center. The proposed system enables timely monitoring and precise visualization of backfill progress, demonstrating its feasibility and practicality.
A simulation environment of backfill was constructed. Different types of filling conditions were designed and recorded into ROS bags using a simulated LiDAR sensor. Because of the fixed LiDAR position, the only change in all of the LiDAR scans is the ever-rising top surface of the slurry. Therefore, the key to compute the backfill progress is to compute the volume of the growing backfill body.
To exploit this feature, outlier searching and the RANSAC plane-fitting algorithm were used to locate and screen out the top surface, which is recognized as the intersection boundary of the slurry top surface and the tunnel. After applying slurry points determination and the moving sampling method, the whole point cloud of backfill body was generated. By using Poisson surface reconstruction on the point cloud of the backfill body, the triangular mesh was obtained. The backfill progress was determined by computing the percentage of the slurry to the volume of the tunnel.
If the accuracy of LiDAR scanning and Poisson reconstruction algorithm are not considered, the error of volume calculation results compared with other methods is less than 1%. In the late stage of backfill, the large amount of point cloud data affects the processing speed of Poisson surface reconstruction and volume computation, resulting in the time to process one set of LiDAR scans to rise to around 45 s. According to the designed flow rate of the slurry, the maximum time to update the progress is less than the time needed for the progress to increase by 0.5%.
The backfill system in underground mining includes both tunnel and mined-out area backfilling. A key limitation of the proposed system is that a single LiDAR cannot capture the vast mined-out areas, even with a wide field of view. Multiple LiDAR sensors are required, and challenges in data transmission, fusion, and processing must be addressed.
Additionally, measurements for filling thick deposits with layered divisions can be a valuable consideration for future work. Layered backfilling allows for better control over material distribution and settling behavior, which can influence the accuracy of volume calculations and the overall progress monitoring. This approach could improve the robustness of backfill monitoring systems in complex mining environments.