*4.3. Validation in Test-Fields*

The LHD automation navigation was installed in a GHH's LF11H LHD (GHH Fahrzeuge). The installation required mechanical and electrical modifications of the equipment to place the system's sensors, processing units, and wireless communication equipment. All the automation software runs in an industrial fan-less computer equipped with an Intel i7 processor and with 4 logical cores. The interface between the automation and the machine was implemented on the machine controller (IFM mobile mini controller), based on GHH's factory program (implemented in Codesys).

After the system was installed and all basic control, communications, and safety functions were thoroughly tested, the equipment was moved to a test-field nearby the OEM (Original Equipment Manufacturer) facilities in Santiago, Chile. The test-field emulated an underground tunnel by using light material to mimic the walls of the mine (Figure 13), which was enough to trick the system. It also had a dummy loading point, a dumping point, and a truck loading point. These tests were used to calibrate the system controller's parameters and the kinematic characteristics of the vehicle's model, such as the acceleration, steering, and breaking response to different operation inputs.

**Figure 13.** Test-field close to GHH facilities in Santiago de Chile.

Because the test-field was located just outside the city where our development team is based (Santiago, Chile), it was possible to make a short trip to test new versions of the software, which contained bug fixes or improvements to the system. Usually a team of developers would go to the test-field two or three times a week to try out different modifications in the algorithms of the automation system.

The last milestone of this stage was to validate the reactive navigation algorithm. In order to do this in a safe manner, a hybrid operation mode was used, in which the speed of the LHD was remotely controlled by an operator while the autonomous navigation system handled the steering of the vehicle in an assisted tele-operation mode. To ensure safety precautions, an onboard operator who could shut down or override the automation system commands was on the vehicle for every test.

#### *4.4. Validation in a Real Mining Operation*

In order to carry out the final stages of development, the equipment had to be tested in its real operation environment, where the last design assumptions and algorithms needed to be validated, and the presence of personnel from the operation site were required.

In the case of the automation system described here, after test site validation, the LHD was transported to a real sublevel stopping mine in the north of Chile, where the development team, the OEM, and the mine personnel coordinated the final system validation and tests.

#### **5. Results and Discussion**

The validation in the test-field was executed from March to June of 2017, requiring approximately 300 h of work. On-site tests were carried out in a medium-scale sublevel stoping mining operation called the *Mina 21 de Mayo* (21st of May Mine), the property of *Compañía Minera San Gerónimo*, located in the north of Chile. The tests were comprised of two phases:


Between the first and second phases, several upgrades were made to the system in order to improve its robustness, consistency, and performance. The most important improvement was on the self-localization system, because the first batch of tests proved that the initial method (not described here) could not maintain the self-localization estimation along the test tunnel. Assisted tele-operation tests during the first phase were mainly used to tune the parameters of the *Guidance* module for the tunnel and intersection navigation modes (Equations (13), (14), (16), and (18)). Once the autonomous navigation was operating properly, further adjustments were made to all system's parameters, including the parameters for the *Command Executor* module (Equations (20) and (21)). Parameters of the map were tuned, such as the maximum speed for certain segments of the tunnel, 2D poses of APs/WPs, and navigation modes for different parts of the tunnel.

On-site, at the mine, two validations were carried out: surface level tests, and underground tests. Surface level tests were done to test all the modules before entering the mine, and to visualize any problem that the LHD or the implemented automation system could have. After the arrival of the machine at the mine site, all sensors, antennas, and communication modules were re-installed and tested. The first teleoperation tests were carried out on the surface, on one of the dump sites of the mine, to verify that the operation of the LHD was correct.

The second validation was done inside the mine in a production tunnel. The system was tested incrementally from teleoperation to full autonomous operation. A network infrastructure was installed inside the test tunnel, and an operating station, consisting of a computer, screens, and controls, was installed inside the mine. Communication tests were carried out between the LHD inside the mine and the computer in the operation center. Teleoperation and assisted teleoperation modes were the first functionalities tested. In the first mode, the operator drives the equipment just as would be done aboard, and in the second mode the operator mainly indicates the direction of movement and the system keeps the LHD away from the walls keeping it from colliding with them. The system was successful in avoiding collisions between the equipment and the inner walls of the tunnel, and the general performance of the operation was similar to manual navigation.

The autonomous navigation tests showed that the system allowed tramming along a 180 [m] tunnel from its entrance to the loading point. The LHD took approximately 2 [min] to go from one point to another, which is comparable to the performance achieved by an experienced human operator. Some of the difficulties that were found included the tunnel being too narrow for the LHD (sized according to the manufacturer's specifications), and the floor having a large number of irregularities, pot holes, and varying inclinations. Of these factors, only the narrowness of the tunnel was included in the simulated environment. A view of the operator's control interface is shown in Figure 14.

**Figure 14.** Operator's graphical interface of the navigation system.

Because of time constraints in the mine, testing, development and parameter tuning were done simultaneously. Because of this, most datasets of the tests in the mine are from a work-in-progress version of the navigation system. Results presented in this section are from 14 datasets (labeled 1, 2, 3, etc.), taken on a single afternoon two weeks before the end of on-site tests. 4 manual operation datasets (labeled M1, M2, M3, and M4) are also presented to have a reference for the performance of an experienced human LHD operator. These manual operation datasets were compiled a week later than the autonomous navigation datasets.

An important problem during tests was roaming between different Wi-Fi access points inside the tunnel. For safety reasons, the system stops accelerating the LHD if communication with the operation station becomes unstable, generating an emergency stop if the loss of communications is longer than a few seconds. Because of this, and a wireless network that did not have fast roaming capabilities, the system often stopped when switching from one access point to another. This can be seen in Figure 15, where stops produced by roaming, and by unstable communications, are shown. The Figure 15 also shows the instant speed of the vehicle (in km/h), the operation mode (with a value of 10 for autonomous navigation, 0 for idle, and −10 for tele-operation), distance traveled (in decameters), the Wi-Fi channel of the access point, at which the LHD is connected (different channels are used for faster roaming), and, finally, the RSSI and Noise values reported by

the wireless modem of the vehicle. All the scales have been selected to fit in a single figure, to show the relation better between these variables.

**Figure 15.** Instant speed, operation mode, distance traveled, wireless communication channel, RSSI, and noise for dataset 1. It can be seen in the selected areas that the LHD comes to a stop when switching between different access points, or when the communication network becomes unstable.

Another consideration for these datasets, since the navigation map was still being tuned, is the intervention of the operator through tele-operation (or assisted teleoperation) to help the LHD go through some narrow passages, or to get back and try again to pass autonomously through a given part of the tunnel. This is shown in Figure 16, as the vehicle needed to stop, then go back a couple of meters (with teleoperation assistance), to later reengage in the autonomous navigation mode, this time getting to the desired destination without further intervention.

Taking these factors (stops because of communication problems and tele-operation) into account, a series of performance indicators were computed for all the datasets. The mean and max speed of the LHD are presented in Table 3. When analyzing the results, it is important to consider that some datasets were compiled with the LHD having a fully loaded bucket, and others with an empty bucket. In some of these datasets, the LHD is moving forward, towards the draw point of the tunnel, and in others, it is moving backward, towards the dump point of the tunnel. To better understand the performance of the system, and the effects of roaming and tele-operation, other indicators are presented, such as the length of the dataset, the total distance of the movement, and the total distance that the LHD was driven by the autonomous system. With an empty bucket, the seasoned operator drove through the tunnel at an average speed of 6.4 [km/h], while the autonomous system did the same at 5.8 [km/h], thus slightly underperforming. The maximum speed achieved by the autonomous system was 11.3 [km/h] with a loaded bucket, while the seasoned operator achieved a maximum speed of 10.6 [km/h] with an empty bucket.

**Figure 16.** Instant speed, operation mode, distance traveled, and wireless communication channel for dataset 5.

**Table 3.** Mean Speed, Maximum Speed, Navigation Time, and Navigation Distance for autonomous navigation and manual operation datasets. ID: Dataset Identifier. T.Op Time: Tele-operation Time. TD: Total Distance. TAD: Total Autonomous Distance. E/L B: Empty or Loaded Bucket. H F/B: Heading Forward or Backwards.


The Navigation time is either an autonomous navigation time or a manual operation time, depending on the dataset. Stop time is the time the machine was stopped, which includes the time at the start and the end of each dataset. Tele-operation time is the amount of time spent tele-operating the LHD so that it is able to resume autonomous tele-operation, usually because the autonomous navigation system didn't approach a curve appropriately, and reached a point where it didn't know how to proceed. The LHD was able to go through the tunnel without remote assistance in only 6 datasets, but it is important to remember

that these were done during development, and small tweaks and adjustments, some of them that worked and some of them did not, were made in between.

To further assess the driving abilities of the autonomous system, the smoothness of its operation is considered. For this, two different metrics are used. The first measures the change between two consecutive command inputs (propel and steering), as is shown in equation (22). The second, in a similar way, measures the difference between two consecutive measures of the dynamic state of the LHD, namely, its speed and the angle of its articulation, as is shown in Equation (23).

$$E\_{\rm \mu}(k) = \left(\mu\_{\rm \upsilon}(k+1) - \mu\_{\rm \upsilon}(k)\right)^2 \Delta t + \left(\mu\_{\rm \omega}(k+1) - \mu\_{\rm \omega}(k)\right)^2 \Delta t \tag{22}$$

$$E\_{LHD}(k) = \left(\upsilon(k+1) - \upsilon(k)\right)^2 \Delta t + \left(\gamma(k+1) - \gamma(k)\right)^2 \Delta t \tag{23}$$

The average and maximum values of both metrics for all datasets are presented in Table 4. Again, the human operator shows a better performance than the autonomous system. The consistency of the human operator is quite remarkable, and it shows its expertise and knowledge of the machine and the tunnel. The autonomous system is also quite consistent on these metrics, but that is usually expected of an automation system. In order to have a better idea of the difference between them, Figures 17–20 show the machine inputs (propel and steering) as well as the instant speed and steering angle of the LHD. Figures 17 and 19 show dataset 14, while Figures 18 and 20 show dataset M1. For clarity, Steering command and steering angle have been plotted separately from and propel command and LHD's speed. Both were made with the LHD having a fully loaded bucket, and with the vehicle moving backward, towards the dump point of the tunnel. Straight lines can be seen in Figure 16 on the propel command line, showing a constant output by the autonomous system. Looking at both figures, it can be seen that the human operator uses fewer steering commands, perhaps showing a better understanding of the LHD kinematics, and, therefore, greater abilities to predict the behavior of the vehicle.

**Table 4.** LHD input difference, state difference and distance to the walls for autonomous navigation and manual operation. ID: Dataset Identifier. ACD: Average Command Difference (*Eu*). MCD: Max Command Difference (*Eu*). ASD: Average State Difference (*ELHD*). MSD: Max State Difference (*ELHD*). H F/B: Heading Forward or Backwards. ADLW: Average Distance to Left Wall. MDLW: Minimum Distance to Left Wall. ADRW: Average Distance to Right Wall. MDRW: Minimum Distance to Right Wall.


**Figure 17.** Autonomous System steering commands and LHD steering angle on dataset 14.

**Figure 18.** Human operator steering commands and LHD steering angle on dataset M1.

**Figure 19.** Autonomous System propel commands and LHD speed on dataset 14.

**Figure 20.** Human operator propel commands and LHD speed on dataset M1.

The average and minimum distances to both tunnel walls are also shown on Table 4. In this regard, the system and the human operator have similar performance, with the human operator preferring to be slightly closer to the left wall (since the cabin is on that side, therefore the operator has better visibility on that side), while the autonomous system is usually closer to the right side of the tunnel.
