**1. Introduction**

As the most common mobile construction machinery, wheel loaders are widely used in the construction and mining industry, which are the important economic sectors across the world [1], due to their flexibility and adaptability. The main task of wheel loaders is to transport materials, including soil and rock, from a site to nearby dumpsite or trucks in a complex and changing working environment [2]. Control of the throttle is critical to the operation of wheel loaders. Accurately predicting the throttle action of a wheel loader expert operator can better achieve autonomous operation. The predicted throttle action can be used to directly the machine to imitate the expert operator's operations, to help achieve autonomous operation. In addition, predictions on the state of wheel loaders can be applied to model predictive control and energy managemen<sup>t</sup> to achieve a good performance in terms of efficiency and fuel consumption.

The automation of construction machinery can reduce the cost and improve the safety of construction sites. Based on this, the last three decades have seen a growing trend towards the automation of construction machinery [3–5]. Many researchers have discussed the division method from manual operation to fully autonomous operation [6,7]. Dadhich et al. [8] proposed five steps to achieve the full automation of wheel loaders: manual operation, in-sight tele-operation, tele-remote operation, assisted tele-remote operation, fully autonomous. Despite the extensive research on automating construction machinery [9–11], a commercial system with autonomous construction machinery is still being explored [12].

**Citation:** Huang, J.; Cheng, X.; Shen, Y.; Kong, D.; Wang, J. Deep Learning-Based Prediction of Throttle Value and State for Wheel Loaders. *Energies* **2021**, *14*, 7202. https:// doi.org/10.3390/en14217202

Academic Editors: Arno Eichberger, Zsolt Szalay, Martin Fellendorf, Henry Liu and Fernando Sánchez Lasheras

Received: 21 July 2021 Accepted: 20 October 2021 Published: 2 November 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

Remote-operated construction machinery is being tried for commercial purposes [13,14]. However, this has led to a greater reduction in productivity and fuel efficiency [15] than manual operation because there are not enough sensory inputs to the remote operators. Therefore, to increase the fuel efficiency and productivity of construction machinery, it is necessary to improve the degree of automation of the loader to reduce the operator's remote intervention.

Most previous works related to the automation of construction machinery are based on a physical model that requires accurate mathematical representations [16–18]. Meng et al. [10] presented a way of optimizing the bucket trajectory for the autonomous loading of load-hauldump (LHD) machines by solving the optimal trajectory through optimizing the minimum energy consumption calculated by Coulomb's passive earth pressure theory. Filla et al. [11] analyzed different autonomous scooping trajectories for wheel loaders by developing a simulation model of the uniform gravel pile. Shen and Frank [12,14] introduced dynamic programming into the solvution of the optimal control variable trajectory based on a mathematical model of the machine. However, these physical-model-based approaches have some common limitations. The method requires a dynamic model of construction machinery to be built, but the dynamic model simplifies machinery in the real world, and the dynamic model of machinery may change under the condition of wear during the operation. Additionally, modeling the interaction force between the tool and material is challenging, as the working environment is unpredictable and variable, and the properties of the different media to be excavated or moved are diverse.

The data-driven approach makes it possible to deal with the complex machinery dynamics [19–22] et al. used the data collected from tests to construct a nonlinear, nonparametric statistical model to predict the behavior of soil excavated by an excavator bucket. Heteroscedastic Gaussian process regression is used as the prediction framework. Machine learning, as a significant means of analyzing complicated data, can adjust its weight parameters by learning from data. In recent years, machine learning has made remarkable progress in solving pattern classification or prediction problems, such as image recognition [23], pattern recognition [24,25], and fault diagnosis [26]. Deep learning has been widely used in construction machinery [12,27,28]. Kim et al. [29] proposed a visionbased action recognition framework that considers the sequential working patterns of earthmoving machinery to recognize the operation types. The earthmoving machinery's sequential patterns are modeled and trained with convolutional neural networks and double-layer long–short-term memory (LSTM).

Due to its powerful ability to characterize complicated systems, process big data, and automatically extract features, deep learning has feasibility and superiority in the prediction task. The deep-learning-based prediction has received grea<sup>t</sup> attention for the automated operation of machinery. Yao et al. [30] designed a two-stage Convolutional Neural Networks model, including a classifier and some regressors, to automatically extract image features to obtain the piled-up status and payload distribution of the current state. The final prediction result is output via a backward-propagation neural network. Luo et al. [31] proposed a framework to predict the pose of construction machinery based on historical motions and activity attributes. The Gated Recurrent Unit is used to predict future machine poses, considering working patterns and interaction characteristics. Shi et al. [32] constructed a deep long–short-term memory network to predict the brake pedal aperture for different braking types by combining the driving data of experienced drivers in different driving environments with deep learning. Xing et al. [33] proposed an energy-aware personalized joint time-series modeling approach based on a recurrent neural network and LSTM to accurately predict the trajectory and velocity of the vehicle. Dai et al. [34] employed two groups of LSTM networks to predict the trajectory of the target vehicle. One LSTM is used to model the target vehicle and the individual trajectory of the surrounding vehicle, and the other is used to model the interaction between the target vehicle and each of the surrounding vehicles.

In this study, based on driving data of the experienced operator, a deep-learningbased method is proposed to accurately predict the throttle value and states (including lift cylinder, tilt cylinder, engine speed, vehicle velocity) of wheel loaders to help achieve autonomous operation and make predictive control algorithms and energy managemen<sup>t</sup> strategies work with an acceptable performance. The sensor signals of wheel loaders under different working conditions are used instead of images as an important basis for predicting the throttle value and states of wheel loaders, as images will be inevitably affected by occlusions, deviations in viewpoint and scale, ambient illumination, and other factors [35,36]. Considering the time series characteristics of the working process of wheel loaders, LSTM networks are used to extract features. To reduce the computation load, the prediction of throttle value and state share the same LSTM network structures and weights. Two backward-propagation neural networks (BPNNs) are introduced to output the prediction results, as the throttle is controlled by the driver and the state of the wheel loaders is randomly influenced by the environment. Each working cycle of wheel loaders consists of several working phases, which possess their own unique characteristics, so the prediction results at different stages are output by neural networks with different weights to improve the prediction accuracy. Two different materials are used to study the adaptability of the prediction model. The relationship between the prediction performance and signal sampling frequency is also studied. Compared with the existing works, the method proposed in this study does not require a physical model and can be applied to different working conditions. The method proposed in this study can provide technical support for the autonomous operation of construction machinery and contribute to the intelligent process of the mining and construction industry.
