*4.1. Full-Scale Rock Excavation Experiments*

Since the pile identification method was not fully optimized at the time of the onsite tests, and time for executing the experiments in the mine was limited, the development team divided the loading procedure in two stages in order to maximize the total number of full-scale rock excavation experiments and evaluate each stage separately. One stage included rock pile identification and positioning, and the other included the charging, excavation, pull back, and weighting processes. To perform this, loading experiments required a human operator to position the machine five to ten meters in front of the draw point, effectively bypassing the rock pile identification and positioning stages, and starting the system's state machine in "positioning assistance", as described in Section 3.

A total of 30 complete loading experiments were carried out. Similar to in manual operation, multiple loading attempts were required in order to fill the bucket during each experiment. Manual operation of LHD in sublevel stoping mines usually needs two or three attempts to achieve a proper bucket fill factor, since the procedure is performed using line-of-sight radio control with limited visibility behind the LHD (safety regulations forbid operators to enter the stope area). In some attempts, the tip of the bucket was lowered too much, causing high resistive forces from the ground during charging, and thus triggering the collision detection too early. A total of 81 loading attempts were performed in order to fulfill the 30 complete loads. Of these, 7 failed (early collision trigger) and 74 were successful, resulting in the 91% success rate of the charging and collision detection method.

For each experiment, a fixed value of 0.5 was used for the pedal command. An operator monitoring the system's behavior during the complete loading process from a remote-control station decided when the bucket was full enough, and hence if a new attempt was needed. The operator relied only on the visual input of the machine's forward camera to make this decision.

Figure 23 shows an example of the relevant variables in an autonomous loading attempt, which has some resemblance to the operator-controlled loading shown in Figure 5.

Table 3 shows the number of experiments carried out, classified by the number of attempts to achieve a full bucket, and the average fill factor for each category. It can be seen that most experiments required three loading attempts, and that the operator continued to reattempt loading until the fill factor was about 90%. This can be seen more easily on Table 4, where the average fill factor and average number of attempts for all experiments are shown. It must be noted that while the fill factor at the end of each experiment met the criteria for manual operation (since it was the same operator deciding when the bucket was full), the average number of attempts was found to be higher than that in manual operation, which, according to manual operators was between two and three attempts.

Table 5 and Figures 24 and 25 show more detail about progressive filling of the bucket through loading attempts for the 30 loading experiments. Table 5 shows the average durations of the excavation step and the resulting fill factor, classified according to the progressive sequence of attempts, as well as the total number of experiments in each case. On average, the first attempt managed to fill 62% of the bucket capacity; then, the second attempt achieved 77%; the third attempt achieved 82%; and for the six experiments that needed a fourth attempt, the average that resulted was a 90% fill factor. The average excavation duration was about 10–12 s, which comprised the time between the collision of the LHD with the rock pile and the end of the excavation algorithm. The required time for positioning, charging, and weighing was not taken into account in this measurement. It can be seen that the average duration of the excavation in the first attempt is slightly longer than the others.

Figure 24 shows a box plot of the excavation step duration for all loading attempts. The time taken for the excavation ranges from 6 to almost 20 s, depending on the conditions of the rock pile and the specific interaction between the bucket and the rock pile. Figure 25 shows a box plot of the fill factor that was obtained progressively through loading attempts. It can be seen that the biggest jump in performance happened in the second loading attempt.

Some outliers with performance above 100% are also depicted for the second and third attempts and appear as a consequence of loading a large boulder.

**Figure 23.** Relevant variables in an autonomous loading attempt.

**Table 3.** Loading experiments and average fill factor per number of loading attempts. N-Attempts: number of attempts required to fill bucket. N-Exp: number of experiments. Total-Attempts: total number of performed attempts. %-Experiments: % of the experiments. Fill-Factor: average fill factor.


**Table 4.** Total average number of attempts and fill factors.



**Table 5.** Duration and fill factor in consecutive attempts. P-Attempts: progressive sequence of attempts. N-Exp: number of experiments. Duration: average duration. Fill-Factor: average fill factor.

**Figure 24.** Excavation step duration per loading attempt. Red "+" signs represent outliers.

**Figure 25.** Progressive fill factor across loading attempts. Red "+" signs represent outliers.

During experimentation, the value of accurately placing the bucket of the machine against the ground, prior to the charging against the rock pile, was recognized. A bucket tilted too far down would cause an early trigger of a collision detection, while a bucket tilted just slightly upwards has a relevant impact on how far the machine can penetrate into the rock pile. An even more relevant factor that affects the performance of the loading maneuver is the status of the rock pile. Newly blasted ore is far easier to load than draw points that have had more time to settle and compress due to the weight of the rock. Draw points with large amounts of ore are also easier to load than those that have little material left. Despite these relevant factors, the proposed method was able to extract full buckets each time, at the expense of the operator having to perform more than the usual number of reattempts required under manual operation.

It is important to note that in a real mining operation, the efficiency of an LHD is measured in the amount of ore that it is able to haul from the extraction point to the dumping point (usually in tons/hour). Therefore, the percentage of bucket filling is not the only relevant factor when evaluating the system, but also the amount of time it uses to load the bucket. This is especially important when deciding if the system should make another attempt at the excavation procedure to achieve a fuller bucket. This criterion varies among different operations, as the dumping and haulage time is different.

#### *4.2. Offline Results Using Field Data: 2.5D Modeling of the Extraction Point*

Personnel cannot be inside a stope as it is forbidden for security reasons (i.e., rock can fall from the ceiling of the stope), so it was not possible to obtain accurate measurements of the ore pile characteristics. However, an accurate modeling of the ore pile is not really necessary, as it only needs to be detected, as well as having its width and inclination roughly estimated in order for the excavation algorithm to work.

To obtain the datasets required to characterize our 2.5D modeling algorithm, the LHD was driven from a fixed distance of about 50 m from the loading point to the entrance of the stope, while all sensor and machine data was recorded. These datasets were captured at different times so the drawing point was not located in the same place, nor did the ore pile have the same shape. First, in each case the point cloud was computed and then the pile's width and inclination estimated. Figure 26 shows an example of the point cloud obtained while the LHD is approaching an extraction point, after registering the position of the points using the LIDAR-based odometry and the inclinometer data (see details in Section 3.2). The color of the points represents the distance to the LHD (in the x-axis) at the time of registration. In the case of the rock pile at the end of the tunnel, red indicates that the machine was further away when that reading was taken.

**Figure 26.** Example of the scanning obtained while the LHD is approaching an extraction point. (**a**) Top view of the pile reconstruction. (**b**) Isometric view of the pile reconstruction.

The results of several modeling attempts where the pile's width and inclination were estimated are summarized in Table 6. They consider ground truth width, predicted width, ground truth inclination, and predicted inclination. Both ground truth width and ground truth inclination were determined manually from the integrated point clouds. The average absolute error in the predicted width is 0.58 m, while the average absolute error in the inclination is 1.7 degrees. The precision achieved by the characterization procedure is sufficient for the task of autonomous loading.


**Table 6.** Results of modeling extraction points.

#### **5. Conclusions**

A complete autonomous loading system for LHD machines for underground mining was presented. The loading system considers identification of the rock pile, positioning of the machine in front of it, charging against the rock pile, excavating, moving away from the draw point, and estimating the bucket weight or fill factor. Despite that the proposed system was fully implemented (including the necessity for a human operator to be involved in the process in order to complete the task when the system fails), it could not be tested as a whole. The proposed system which implements the whole loading process, the excavation algorithm, and the tests in a real production environment are the most important contributions of this work.

The experimentation phase was divided between onsite execution and validation of the excavation algorithm, and offline data processing for the rock pile identification method. In the onsite experiments, identification and positioning were bypassed in favor of teleoperation by a human operator. It was also the operator's choice whether or not to perform multiple loading attempts in order to fill the bucket. A total number of 30 excavation experiments were carried out, most of them requiring multiple attempts to achieve a full bucket. An average of 3.6 attempts per experiment was needed in order to obtain a bucket fill factor of 90%. By comparison, manual operation usually needs two or three attempts to achieve this bucket fill factor. This performance may seem inferior to previous published work: however, there are other factors involved that prevent direct comparison of the results. Equipment, type of ore, and mining method should be considered. Thus, comparison to the human operators of the specific mining site should be preferred. It is important to mention that the main driver for automating the loading process is to increase the safety of workers rather than obtain higher efficiency. In this regard, the proposed excavation algorithm fulfills our expectations.

Since these experiments are time-consuming in an industry where time is an expensive resource, only enough experiments needed to validate the excavation algorithm were able to be executed. Part of our future work will be to carry out new experiments in an underground mine in order to validate the complete autonomous loading system, and to measure its performance more accurately. Despite this, the problem remains relevant and mining companies are looking forward to integrating autonomous loading to LHDs in their operations, as it would enable them to close the loop for a fully autonomous production cycle.

A video showing the operator's graphic interface while the system is autonomously performing an excavation procedure can be found in https://youtu.be/Oa11kTBJf2Y (accessed on 7 September 2021).

The system is now being installed and tested in a room and pillar mine in Germany, where it will be tested as a whole system.

**Author Contributions:** Conceptualization, C.T., M.M. and J.R.-d.-S.; methodology, M.M., C.T. and J.R.-d.-S.; software, C.T.; validation, M.M. and C.T.; resources, J.R.-d.-S.; data curation, C.T.; writing—original draft preparation, C.T., M.M. and J.R.-d.-S.; writing—review and editing, C.T., M.M. and J.R.-d.-S.; funding acquisition, J.R.-d.-S. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research was funded by the Chilean National Research Agency ANID under project grant Basal AFB180004 and FONDECYT 1201170.

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Data Availability Statement:** The data presented in this study is contained in the article itself.

**Acknowledgments:** We thank Paul Vallejos for the valuable discussions and support for onsite execution of the experiments; we thank David Leottau for his valuable work in the early implementations of the 2.5D modeling of the ore pile. We also thank Patricio Loncomilla and Martin Calvo for their valuable work enhancing the 2.5D modeling algorithm and its associated results. We also acknowledge Compañía Minera San Gerónimo for providing the mine infrastructure for testing the system, and GHH Chile for supplying the LHD machine needed for this work.

**Conflicts of Interest:** The authors declare no conflict of interest.
