Next Article in Journal
Identification and Diagnosis of Transboundary River Basin Water Management in China and Neighboring Countries
Previous Article in Journal
Assessing the Drivers of Carbon Intensity Change in China: A Dynamic Spatial–Temporal Production-Theoretical Decomposition Analysis Approach
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Dangerous Driving Behavior Recognition Based on Hand Trajectory

1
Transportation College, Jilin University, Changchun 130022, China
2
China FAW Group Corporation Co., Ltd., No. 1, Honaqi Street, Changchun 130013, China
*
Author to whom correspondence should be addressed.
Sustainability 2022, 14(19), 12355; https://doi.org/10.3390/su141912355
Submission received: 1 September 2022 / Revised: 19 September 2022 / Accepted: 26 September 2022 / Published: 28 September 2022

Abstract

:
Dangerous driving behaviors in the process of driving will produce road traffic safety hazards, and even cause traffic accidents. Common dangerous driving behavior includes: eating, smoking, fetching items, using a handheld phone, and touching a control monitor. In order to accurately identify the dangerous driving behaviors, this study first uses the hand trajectory data to construct the dangerous driving behavior recognition model based on the dynamic time warping algorithm (DTW) and the longest common sub-sequence algorithm (LCS). Secondly, 45 subjects’ hand trajectory data were obtained by driving simulation test, and 30 subjects’ hand trajectory data were used to determine the dangerous driving behavior label. The matching degree of hand trajectory data of 15 subjects was calculated based on the dangerous driving behavior recognition model, and the threshold of dangerous driving behavior recognition was determined according to the calculation results. Finally, the dangerous driving behavior recognition algorithm and neural network algorithm are compared and analyzed. The dangerous driving behavior recognition algorithm has a fast calculation speed, small memory consumption, and simple program structure. The research results can be applied to dangerous driving behavior recognition and driving distraction warning based on wrist wearable devices.

1. Introduction

During the driving process, the driver may perform dangerous driving behavior with one hand. When an emergency occurs, the driver cannot respond in time [1]. Common driving errands include eating, smoking, and using a phone. Statistics show that traffic accidents caused by distracted dangerous driving behaviors account for 30% of total accidents [2]. According to the definition of the International Organization for Standardization, driving distraction refers to the phenomenon that attention is directed to activities that are not related to normal driving, resulting in a decrease in driving ability [3]. The study of driving distraction is mainly to analyze the influence of driving distraction on driving stability and vehicle control ability, and to characterize the change of driver state index, which in turn can become the standard to identify driving distraction. The cognitive distraction criteria were: right prefrontal cortex activation, heart rate, and skin conductive response level [4,5]. Visual distraction index: lead to saccade time, fixation times, fixation duration, pupil diameter, etc. [6]. The operation distraction indexes are: reaction time, throttle opening, steering wheel angle, etc. [7]. Studies have shown that giving drivers warning information when there is a potential danger during driving or providing reminders when drivers are sleepy can reduce the incidence of accidents [8,9]. Therefore, it is necessary to give the driver an appropriate reminder after identifying the dangerous driving behavior. This study identifies dangerous driving behaviors based on the driver’s hand motion trajectory data.
Most scholars use machine vision and machine learning algorithms to identify hand motion trajectories. Wang Bing proposed a dynamic gesture recognition algorithm using Kinect equipment, and used a hidden Markov model to train and identify hand motion trajectories [10]. Chengfeng proposed a hand trajectory recognition method based on an improved long short-term memory (LSTM) model and proposed a new loss function in the LSTM training process to reduce the accuracy loss caused by the increase in the number of iterations [11]. V. Kav detected the hand motion trajectory through optical flow and recognized the gesture based on deep learning. This algorithm solves the problem of hand motion trajectory recognition in different environments [12]. The steering wheel gripping position controlled by the driver’s hands can reflect the driver’s physiological load. Dick and other scholars use machine vision detection devices to obtain the position of the driver’s hand and determine whether the hand is willing to leave the steering wheel [13]. Esed calculated the driver’s hand posture based on machine vision and decision tree algorithms to complete the recognition of driving behavior [14]. Based on the machine vision form of hand trajectory recognition, the data processing process is complex, and the light environment requirements of the collected samples are high, which ultimately leads to the low accuracy of hand trajectory data acquisition.
Some scholars identify the motion trajectory of the hand based on the sensor device. Wang Xuemei measured the motion state of the human arm based on an MEMS gyroscope, accelerometer, and magnetometer, and used a neural network and support vector machine algorithm to identify the arm state [15]. Based on CNN’s detection network and Gaussian heat map transformation, Liu Tangbo identified drivers’ phone calls and smoking status [16]. Li Hongtao proposed a steering wheel angle detection system based on multi-axis inertial sensors, which determines the driver’s fatigue state according to the steering wheel deflection angle of hand operation [17]. Zhaojie Ju made a multi-channel sensor to identify the hand motion posture by collecting hand force signals and electromyography signals and using a fuzzy Gaussian mixture model [18]. Kieran and other scholars have made a multi-modal wearable system, obtained the elbow motion curve through the system, and compared and analyzed 10 machine learning algorithms. The comparison results show that the neural network algorithm has the best feature extraction effect compared with other machine learning algorithms [19].
The hand motion trajectory recognition algorithm based on machine learning requires a large amount of data as the training set, and consumes computer memory resources [20]. There are some disadvantages in driving behavior recognition based on image or video. In the environment of variable cockpit illumination, the driving feature extraction method based on unsupervised learning to extract the driver’s contour is easily affected by illumination changes and has the problem of inaccurate segmentation of the driver’s contour. The method based on target detection has great limitations in detecting the types of driving distractions. The method of extracting driving characteristics based on the Kinect structured light camera relies too much on its own human posture recognition algorithm, and the algorithm has a poor detection effect in crowded car cockpits [21]. In order to solve the problem that the machine learning algorithm has a slow calculation speed and occupies a large amount of computer memory, and the image or video recognition is susceptible to light and has low calculation accuracy, the application of wearable devices to identify the driving behavior of drivers is considered. The hand motion trajectory recognition algorithm is mostly applied to wearable devices, so it needs a recognition algorithm with a low sample size, high recognition efficiency, and small memory consumption. In order to avoid the problems of long training time and large computer memory occupied by machine learning algorithms, this study selected a trajectory recognition algorithm based on time series data.
The purpose of our study is to identify the driver’s dangerous driving behavior through the driver’s hand motion trajectory data. Firstly, we analyzed the advantages and disadvantages of DTW algorithm and LCS algorithm according to the characteristics of hand motion trajectory, and constructed a driving secondary task recognition model. Secondly, the UC-win/Road driving simulator was used for the experiment, and the hand motion trajectories of 45 subjects in five dangerous driving behaviors were obtained through the self-developed hand motion trajectory acquisition device, including eating, smoking, taking items, touching a control display, and using a handheld phone. Each dangerous driving behavior used 150,000 hand trajectory data of 30 subjects to determine the label and calculated the matching degree based on the hand trajectory data of 30 subjects to determine the dangerous driving behavior recognition threshold. The recognition accuracy of the model was verified by 75,000 hand motion trajectory data of 15 subjects. Finally, the differences in recognition accuracy, calculation speed, and memory consumption between the neural network algorithm, support vector machine algorithm, and this model algorithm are compared and analyzed. The technology and the model can be applied to the dangerous driving behavior recognition and driving distraction warning based on wrist wearable devices.

2. Materials and Methods

2.1. Construction of Driving Secondary Task Recognition Model

Because the hand trajectory data are time series data, this study uses the DTW algorithm and LCS algorithm and identifies the dangerous driving behavior based on the driver’s hand trajectory. DTW algorithm mainly identifies two time series with the same length. If the data lengths of the two time series are inconsistent, the DTW algorithm fails [22].
In order to make up for the defects of the DTW algorithm, the LCS algorithm is further introduced in the process of model construction. The LCS algorithm can identify the length curve of any time series, and its calculation concept is an exhaustive comparison. The LCS algorithm compares all the time series data of the hand trajectory until the longest common factor time series of the two curves is found. The disadvantages of the LCS algorithm are: (1) For the long trajectory curve of time series, the number of segmentation and comparison of time series is significantly high. (2) Two time series data for recognition calculation. In order to achieve a high-precision recognition rate, it is necessary to eliminate the interference caused by data error [23].
In summary, when the hand trajectory data acquisition is delayed, the length of the detected hand trajectory time series is longer than the label length. At this time, the LCS algorithm is used for recognition, and the DTW algorithm is used for recognition under normal data acquisition. Combining the advantages of the DTW algorithm and LCS algorithm, the dangerous driving behavior recognition model is constructed. The algorithm process is shown in Figure 1.
Step 1: The characteristics of the driver’s hand movement in the driving process can be characterized by the change of the six-axis data (triaxial acceleration and triaxial angular velocity) of the hand movement, and the use of the six-dimensional data has too many calculations in the comparative calculation. In order to reduce the number of model calculations, the six-axis data are reduced to one-dimensional data by Formula (1).
d i s ( i ) = ( a x ( i ) ) 2 + a y ( i ) 2 + a z ( i ) 2 + r x ( i ) 2 + r y ( i ) 2 + r z ( i ) 2
where d i s ( i ) represents the hand trajectory six-axis data integration value; a x ( i ) represents the acceleration in the x-axis direction; a y ( i ) represents the acceleration in the y-axis direction; a z ( i ) represents the acceleration in the z-axis direction; r x ( i ) represents angular velocity in the x-axis direction; r y ( i ) represents the angular velocity of the y-axis; r z ( i ) represents angular velocity in the z-axis direction.
The time series data d i s ( d ( 1 ) , d ( 2 ) , , d ( n ) ) 1 × n of hand trajectory is obtained by Formula (1).
Step 2: Take the mean value of multiple hand trajectory time series data obtained in the experiment, and use the smooth function in Matlab to denoise the trajectory data, and obtain the label label(l(1),l(2),…,l(n))m.
Step 3: When the driver controls the steering wheel, the change range of the six-axis data integration value of the hand trajectory is [0, 30.5]. When the six-axis data integration value of hand trajectory is greater than 30.5, it indicates that the driver’s driving behavior changes. At the same time, the test vector T e x t ( t ( 1 ) , t ( 2 ) , , t ( n ) ) 1 × d is determined according to the data characteristics of label vector l a b e l 1 × m , and the dangerous driving behavior is identified.
Step 4: When m = d, the data acquisition is continuous; we use the DTW algorithm to calculate. When identifying the hand trajectory, the time series data of the label trajectory are directly compared with the time series data of the sample trajectory. Due to the great randomness of the hand trajectory data, the identification calculation process is complex. Therefore, the DTW algorithm is selected to regularize the trajectory time series data and complete the hand trajectory recognition combined with distance measurement. In the following, the hand trajectory and DTW algorithm are combined for modeling.
(1)
The cumulative distance matrix is constructed according to the label vector and test vector to calculate the shortest distance.
The leftmost vector and the uppermost side vector of the matrix in the distance matrix of the DTW algorithm are the time series vectors, and the matrix composed of their respective parts is the shortest distance matrix. The calculation process of the shortest distance is as follows.
The formula for calculating the shortest distance on the left side of the matrix is
D ( i , 0 ) = d i s ( L a b e l ( i ) , T e x t ( 0 ) ) + D ( i 1 , 0 )
In: D represents the shortest distance matrix; i, j represents the sequence of time series.
The calculation formula of the shortest distance on the uppermost side of the matrix is:
D ( 0 , j ) = d i s ( L a b e l ( 0 ) , T e x t ( j ) ) + D ( 0 , j 1 )
Other calculation formulas of the shortest distance are
D ( i , j ) = d i s ( L a b e l ( i ) , T e x t ( j ) ) +   min ( D ( i 1 , j ) , D ( i , j 1 ) , D ( i 1 , j 1 ) )
(2)
The calculation path of the DTW algorithm is regularized to obtain the time series corresponding to the shortest path. If the label vector is consistent with the test vector, the two vectors after DTW algorithm normalization are consistent with the two vectors before normalization. If the label vector is inconsistent with the test vector, the length of the two vectors after regularization is larger than that before regularization.
(3)
Hand trajectory matching degree calculation. According to the actual situation, the model identification effect is calculated from two aspects.
According to the shortest distance calculated by the model, the matching degree of the curves of the two time series can be determined. The smaller the shortest distance value is, the closer the driving behavior representing the test is to the driving behavior corresponding to the label.
According to the DTW algorithm, the time series coordinates corresponding to the shortest trajectory can also be used to calculate the matching degree of the model. The model identification results show that the better the curve matching degree, the closer the shortest path to the diagonal of the shortest path matrix. The path curve of the shortest path matrix is a monotonic increase and decrease curve, and the slope of the curve is less than zero. After the slope of the adjacent two points in the path is added, the mean is calculated. If the absolute value of the mean is closer to 1, the driving behavior of the test is closer to that of the label.
k ( i ) = { w ( i ) i = 1 w ( i ) w ( i 1 ) 1 < i m
where: k ( i ) denotes the slope of the trajectory path of the adjacent time series, and w ( i ) is the time series of the trajectory.
T 1 = i = 1 m k ( i ) / m
where: T 1 indicates the matching degree between the driving behavior of the test and the label.
Step 5: When m ≠ d, it indicates that the data acquisition process is delayed, and the LCS algorithm should be used for calculation. The concept of the LCS algorithm can also be used for trajectory recognition. For two similar hand trajectories, the time series data should be equivalent. The longer the longest common factor time series, the higher the matching degree of the two trajectories; therefore, this paper uses the LCS algorithm to identify the hand trajectory. The hand trajectory and LCS algorithm are combined for modeling below.
(1)
The input two gesture trajectory data are calculated according to the LCS algorithm, and the formula is:
C [ i , j ] = { 0 if   i = 0   or   j = 0 C [ i 1 , j 1 ] + 1 if   i , j > 0   and   L a b e l ( l ( i ) ) = T e x t ( t ( i ) ) max ( C [ i , j 1 ] , C [ i 1 , j ] ) if   i , j > 0   and   L a b e l ( l ( i ) ) T e x t ( t ( i ) )
where C [ i , j ] represents the longest common factor sequence traversed.
(2)
The test vector is denoised, and then the longest common factor sequence length is calculated.
(3)
Hand trajectory matching calculation.
After the calculation of the LCS algorithm, the common part of the label vector and the test vector can be determined. The longer the length of the longest common factor sequence, the closer the driving behavior representing the test to the dangerous driving behavior corresponding to the label. The threshold calculation formula is:
T 2 = L LCS L Text
where T 2 denotes the matching degree between the dangerous driving behavior of the test and the dangerous driving behavior corresponding to the label; L LCS represents the length of cumulative longest common factor sequence; L Text represents the length of the time series of the tag’s hand trajectory.
Step 6: Determination of the recognition threshold of dangerous driving behaviors.
Repeat the above process, the hand trajectory corresponding to multiple groups of dangerous driving behaviors is obtained by experiments, and the matching degree of hand trajectory is calculated. The minimum matching degree is used as the recognition threshold of dangerous driving behaviors. When the matching threshold is greater than or equal to the minimum threshold, the corresponding dangerous driving behavior is determined.

2.2. Platform Construction and Test Scheme Design

2.2.1. Platform Construction

Due to the execution of driving behaviors in the driving process, there will be road traffic safety hazards, so we carried out in the form of driving simulation. Using Up-Coming’s driving simulator and its UC-win/Road simulation software, the test platform was built, as shown in Figure 2. The road type in the test scene is a two-way six-lane urban road with a total length of 10 km. The vehicle runs in the middle lane. The traffic flow on both sides of the target vehicle is set to 260 pcu·h−1. The traffic flow in each lane obeys uniform distribution. The average speed of traffic flow is 50 km·h−1 [24,25].
Collect the driver’s hand trajectory data in the driving simulation scene. The hand motion trajectory acquisition system is composed of two parts: the data acquisition software independently developed based on Lab-view software and the six-axis sensor. The six-axis sensor is a GY-95T sensor (China Guangdong Shenzhen Vankesheng Technology Co., Ltd., Shenzhen, China), as shown in Figure 3, which can collect the six-axis data of hand motion, and the sampling frequency is 100 Hz. The hand trajectory synthesized by six-axis data can accurately describe the hand motion process, and some of the collected data are shown in Figure 4.

2.2.2. Experimental Design

In our study, 45 participants were recruited, aged between 20 and 30 years old (mean = 24.2, SD = 3.4). In the real driving scene, if the driver performs other tasks outside the driving place, it is impossible for the driver to leave the steering wheel with both hands. In most cases, the right hand will be used for execution. Therefore, the hand motion trajectory acquisition device was fixed at the driver’s right wrist for the driving simulation test before the test. In the process of driving simulation, under the premise of ensuring safe driving, the driver completed the dangerous driving behaviors of eating, smoking (All drivers only imitate smoking action, not real smoking), taking items, touch control, and handheld phone in turn. The test staff needs to record the hand motion data under five dangerous driving behaviors during the driving process.

3. Results

3.1. Identification Results and Characteristics Analysis of Dangerous Driving Behaviors

In order to improve the efficiency of dangerous driving behaviors recognition, we selected 5 s hand trajectory data to make labels. For each dangerous driving behavior, we selected 15,000 hand trajectory data from 30 subjects for label training, and tested the label through 75,000 hand trajectory data of 15 subjects. The training set was used to train the trajectory labels of eating, smoking, picking up items, using handheld phones, and touching control monitors, respectively, and to determine the threshold of dangerous driving behavior recognition. The test set was used to test the accuracy of model recognition.
In total, 5 kinds of hand trajectories of 30 subjects are shown in Figure 5, Figure 6, Figure 7, Figure 8 and Figure 9. The mean values of five kinds of hand trajectories of 30 subjects are calculated, and the data are denoised by the Matlab smoothing function. The five kinds of dangerous driving behavior recognition tags are obtained and shown in Figure 10.
Taking the handheld phone behavior as an example, the model established in the first section is analyzed. Figure 11 is the time series curve for the reference group (n = 30 subjects) and also for the test group (n = 15 subjects) for the behavior of holding a phone. The time series data of the label curve and the time series data of the test curve are substituted into the DTW algorithm for calculation. The calculation results are shown in Figure 12 and Figure 13, respectively.
Figure 12 is the shortest distance visual graph of the DTW algorithm, in which the blue curve is the label curve and the red curve is the test curve. In the gray image corresponding to the two curves, the white trajectory line is the DTW algorithm to regularize the shortest path time series curve. If the white trajectory line is diagonal, the higher the matching degree of the label curve and the test curve. The color on both sides of the white trajectory represents the shortest distance between the label curve and the test curve corresponding to the time series. The color is a gradual change from black to white, black represents the shortest distance of 0, and white represents the shortest distance greater than 10,000. The shortest path time series curve of the DTW algorithm in Figure 12 is close to the diagonal line, which shows that the label curve and the test curve match well.
Figure 13 is the shortest path curve of the DTW algorithm for the label curve and test curve. The blue curve is the label curve, and the red curve is the test curve. If the time series data of the same curve are used as label and test curve, the label and test curve after DTW regularization are consistent with the original curve, then the length of the time series after regularization is equal to that before regularization. The time series length of the curve before normalization is 500, and the time series length of the curve after normalization is 618. The label and test curve of the handheld phone is calculated by Formulas (2)–(4), and the shortest distance of the curve path is 4239.75. The matching degree of the curve is 85.43% by Formulas (5) and (6).
In order to analyze the calculation effect of the LCS algorithm when the data acquisition delay is delayed, the device is set to randomly delay 2 s to collect data during the data acquisition of handheld phone driving behavior. Figure 14 is the handset phone label curve and delay data curve, calculated by the LCS algorithm. In order to improve the accuracy of the LCS algorithm, the LCS algorithm is optimized according to the characteristics of the six-axis data integration value of the hand trajectory. According to the hand trajectory of the 30 subjects, which determines the label curve, the average standard deviation of 30 hand trajectory data is 51.40; before the calculation of the LCS algorithm, the smooth function of Matlab is used to denoise the delayed data curve, and the following processing is carried out:
T e x t ( t ( i ) ) = { L a b e l ( d ( i ) ) | L a b e l ( l ( i ) ) - T e x t ( t ( i ) ) | 52 T e x t ( t ( i ) ) | L a b e l ( l ( i ) ) - T e x t ( t ( i ) ) | > 52
where L a b e l ( l ( i ) ) represents the i time series data of the label curve; T e x t ( t ( i ) ) represents the i time series data of the test curve.
The time series data of the test curve are calculated by Formula (9), and the optimized test curve is calculated by the LCS algorithm. The longest common factor sequence is calculated, and the matching degree calculated by the LCS model is 97.2%.
Since there are very few delays in the data acquisition device, the threshold determination and algorithm comparison in this study is only for normal data acquisition. In order to determine the recognition threshold of different dangerous driving behaviors, the hand trajectory and tag trajectory of 30 subjects were matched and calculated, and the box diagram of the matching degree value was obtained, as shown in Figure 15. Taking the minimum matching degree of five dangerous driving behaviors as the recognition threshold, the recognition thresholds of dangerous driving behaviors are as follows: the recognition threshold for eating is 73%, the recognition threshold for taking is 74%, the recognition threshold for smoking is 75%, the recognition threshold for handheld phone use is 75%, and the recognition threshold for using a touch control is 74%.

3.2. Verification of Recognition Accuracy

In order to verify the performance of the system, the hand trajectory of the common dangerous driving behaviors is selected, which are eating, smoking, picking up items, using a handheld phone, and touching controls. Each dangerous driving behavior has a standard template, and each dynamic gesture carries out 15 recognition tests. The test results of all dangerous driving behaviors are shown in Table 1.
The recognition results of Table 1 show that the proposed algorithm can accurately identify the dangerous driving behavior out of five possible matches. In recognition, it is found that the handheld phone is easy to identify errors, because the driver’s hand trajectory fluctuates during the handheld phone, and it is prone to identify errors. In this case, increasing the distance between the starting point and the ending point of the hand movement can effectively identify the handheld phone driving behavior.

4. Discussion

The advantages of the dangerous driving behavior recognition algorithm are that the algorithm has a simple structure and strong mobility, which can be applied to the wrist-worn device; the shortcomings of the algorithm are that the requirements for the label curve are high, and it is necessary to collect the hand trajectory data of multiple drivers to create the dangerous driving behavior label. With the advancement of science and technology, many wrist wearable devices have built-in six-axis gyroscopes, which can realize the acquisition of six-axis hand trajectory data. Thus, a wrist wearable device could identify risky driving behavior based on the dangerous driving behavior recognition algorithm and remind the driver to control the steering wheel with both hands to reduce the driving safety hazard; therefore, the wrist wearable device can consider configuring the dangerous driving behavior recognition algorithm.
At present, the commonly used hand trajectory recognition algorithms are the neural network algorithm and support vector machine algorithm. However, a large number of training calculations are needed in recognition of these two algorithms, and there is a problem of long calculation time or memory consumption [26]. In order to analyze the difference in the calculation of the algorithm, the neural network algorithm, the support vector machine algorithm, and the algorithm of this model are compared with the same computer. Five dangerous driving behaviors were performed by 45 drivers, of which 30 individuals and 150,000 hand trajectory data were randomly selected for each dangerous driving behavior, and 15 individuals and 75,000 hand trajectory data were selected for testing. The calculation results of the three algorithms are shown in Table 2.
The statistical results in Table 2 show that the average recognition rate of this model algorithm is 98.6%, the average recognition time is 9.5 s, and the average running memory consumption is 1968.0 KB, with the fastest recognition speed. The average recognition rate of the neural network algorithm is 82.64%, the average recognition time is 21.8 s, the average running memory consumption is 1.6 × 105 KB, the recognition speed is medium, and the calculation memory consumption is the largest. The average recognition rate of the support vector machine algorithm is 100%, the average recognition time is 102.5 s, the average running memory consumption is 196.0 KB, and the memory consumption is the smallest, but the calculation time is the longest.
In order to further analyze the algorithm performance, the profile viewer function of Matlab is used to view the structure of this model algorithm, neural network algorithm, and support vector machine algorithm. The results are shown in Figure 16, Figure 17 and Figure 18.
Given Table 2 and the algorithm structure diagram, it can be seen that the neural network algorithm is complex, and it is clear that many functions need to be monitored in the calculation process of the algorithm, so the computer memory required is very great. The support vector machine algorithm has a simple structure, so it consumes less computer memory, but the training speed of the support vector machine is very slow, resulting in 102.5 s to complete the recognition calculation. In summary, the recognition accuracy and efficiency of the model algorithm are very high, and the calculation does not consume too much computer memory, so it is suitable for the wrist-worn device recognition algorithm.

5. Conclusions

We identified dangerous driving behaviors according to the characteristics of hand trajectory. Firstly, the advantages and disadvantages of the DTW algorithm and LCS algorithm are analyzed according to the characteristics of hand motion trajectory, and the dangerous driving behavior recognition model is constructed. Secondly, 45 subjects were selected for the driving simulation test, and 150,000 hand trajectory data of 30 subjects were used to train eating, smoking, picking up items, touching the control display, and using handheld phone trajectory labels. The threshold of dangerous driving behavior recognition was determined by calculating the matching value of the hand trajectory of 30 subjects. The recognition accuracy of the algorithm was tested by using the hand trajectory of the dangerous driving behavior of 15 participants. Finally, from the perspective of the application, comparative analysis of the model algorithm and neural network algorithm and support vector algorithm differences. The following conclusions were reached:
(1)
There are differences in the matching degree of different dangerous driving behaviors. The matching degree of dangerous driving behaviors with a large hand fluctuation range is 77–84%, and the matching degree of dangerous driving behaviors with a small hand fluctuation range is 75–89%.
(2)
Through the calculation of curve matching degree, it is determined that the recognition thresholds of different dangerous driving behaviors are greater than 70%, and the algorithm recognition calculation value is greater than this threshold, which can be determined as the corresponding dangerous driving behavior.
(3)
Compared with the neural network algorithm (100.00%) and the support vector machine algorithm (82.64%), the average recognition accuracy of the dangerous driving behavior recognition algorithm in the present study is much greater at 98.6%, the average recognition speed is 9.5 s, and the average consumption of computer memory is 1968.0 KB. The program recognition rate is high, the structure is simple, and the portability is strong. In future research, the wrist wearable device can identify the driver’s hand trajectory and warn the driver of distraction based on the dangerous driving behavior recognition algorithm.

Author Contributions

Conceptualization, W.L. and H.Z.; methodology, W.L. and H.Z.; software, H.L.; validation, H.Z.; formal analysis, H.Z.; investigation, H.L.; resources, H.L.; data curation, H.Z.; writing—original draft preparation, W.L.; writing—review and editing, W.L.; visualization, H.L.; supervision, H.L.; project administration, H.Z. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by grants from the National Natural Science Foundation of China, under Grant Number U1564214, and the National Natural Science Foundation of China, under Grant Number U1664263.

Institutional Review Board Statement

This study did not involve humans or animals.

Informed Consent Statement

Informed consent was obtained from all participants involved in the study.

Data Availability Statement

The data presented in this study are available in the paper.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Seymour-Hart, I.I. Road Traffic Accident Reconstruction: Vision, Alertness and Reaction Relating to Driving. Exp. Mol. Med. 2000, 40, 11–18. [Google Scholar]
  2. Ma, Y.; Cao, Y.; Shi, H. In-vehicle distraction intervention strategies considering dangerous driving behavior requirements. J. Harbin Inst. Technol. 2016, 48, 5. [Google Scholar]
  3. Liu, T.; Yang, Y.; Huang, G.-B.; Yeo, Y.K.; Lin, Z. Driver Distraction Detection Using Semi-Supervised Machine Learning. IEEE Trans. Intell. Transp. Syst. 2016, 17, 1108–1120. [Google Scholar] [CrossRef]
  4. Ebnali, M.; Ahmadnezhad, P.; Shateri, A.; Mazloumi, A.; Heidari, M.E.; Nazeri, A.R. The effects of cognitively demanding dual-task driving condition on elderly people’s driving performance; real driving monitoring. Accid. Anal. Prev. 2016, 94, 198–206. [Google Scholar] [CrossRef] [PubMed]
  5. Tarabay, R.; Abou-Zeid, M. Assessing the effects of auditory-vocal distraction on driving performance and physiological measures using a driving simulator. Transp. Res. Part F Traffic Psychol. Behav. 2018, 58, 351–364. [Google Scholar] [CrossRef]
  6. Zhang, H.; Smith, M.; Witt, G.J. Identification of real-time diagnostic measures of visual distraction with an automatic eye-tracking system. Hum. Factors 2006, 48, 805–821. [Google Scholar] [CrossRef] [PubMed]
  7. Wang, L.; Li, H.; Guo, M.; Chen, Y. The Effects of Dynamic Complexity on Drivers’ Secondary Task Scanning Behavior under a Car-Following Scenario. Int. J. Environ. Res. Public Health 2022, 19, 1881. [Google Scholar] [CrossRef] [PubMed]
  8. Song, X.; Yin, Y.; Cao, H.; Zhao, S.; Li, M.; Yia, M.B. The mediating effect of driver characteristics on risky driving behaviors moderated by gender, and the classification model of driver’s driving risk. Accid. Anal. Prev. 2021, 153, 106038. [Google Scholar] [CrossRef] [PubMed]
  9. Ma, J.; Gong, Z.; Tan, J.; Zhang, Q.; Zuo, Y. Assessing the driving distraction effect of vehicle HMI displays using data mining techniques. Transp. Res. Part F Traffic Psychol. Behav. 2020, 69, 235–250. [Google Scholar] [CrossRef]
  10. Wang, B.; Dong, H.-W.; Zhang, M.-M.; Pan, Z.-G. Dynamic hand gesture recognition based on Kinect. Transducer Microsyst. Technol. 2018, 37, 143–146. [Google Scholar]
  11. Jian, C.; Yang, M.; Zhang, M. Mobile Terminal Gesture Trajectory Recognition Based on Improved LSTM Model. IET Image Process. 2019, 13, 991–997. [Google Scholar] [CrossRef]
  12. Kavyasree, V.; Sarma, D.; Gupta, P.; Bhuyan, M.K. Deep Network-based Hand Gesture Recognition using Optical Flow guided Trajectory Images. In Proceedings of the 2020 IEEE Applied Signal Processing Conference (ASPCON), Kolkata, India, 7–9 October 2020. [Google Scholar]
  13. De Waard, D.; Van den Bold, T.G.; Lewis-Evans, B. Driver hand position on the steering wheel while merging into motorway traffic. Transp. Res. Part F Psychol. Behav. 2010, 13, 129–140. [Google Scholar] [CrossRef]
  14. Ohn-Bar, E.; Trivedi, M.M. Beyond just keeping hands on the wheel: Towards visual interpretation of driver hand motion patterns. In Proceedings of the IEEE International Conference on Intelligent Transportation Systems, Qingdao, China, 8–11 October 2014. [Google Scholar]
  15. Wang, X.-M.; Liu, Z.; Ni, W.-B. Motion measurement and recognition method of human arm based on MEMS sensors. J. Chin. Inert. Technol. 2017, 6, 701–707. [Google Scholar] [CrossRef]
  16. Liu, T.; Yang, R.; Wang, W.; He, C. Driver’s Hand Motion Detection Based on Pose Estimation. J. Signal Process. 2019, 35, 2062–2069. [Google Scholar]
  17. Quan, W.; Li, H. Steering Wheel Angle Detection System Based on Multiple-Axis Inertial Sensor. J. Chongqing Jiaotong Univ. Nat. Sci. 2020, 39, 116–122. [Google Scholar]
  18. Ju, Z.; Liu, H. Human Hand Motion Analysis with Multisensory Information. IEEE/ASME Trans. Mechatron. 2014, 19, 456–466. [Google Scholar] [CrossRef]
  19. Little, K.; Pappachan, K.B.; Yang, S.; Noronha, B.; Campolo, D.; Accoto, D. Elbow Motion Trajectory Prediction Using a Multi-Modal Wearable System: A Comparative Analysis of Machine Learning Techniques. Sensors 2021, 21, 498. [Google Scholar] [CrossRef]
  20. Sarma, D.; Bhuyan, M.K. Hand Detection by Two-Level Segmentation with Double-Tracking and Gesture Recognition Using Deep-Features. Sens. Imaging 2022, 23, 9. [Google Scholar] [CrossRef]
  21. Wang, H.-Y.; Zhao, M.-M.; Beurier, G.; Wang, X.-G. Automobile Driver Posture Monitoring Systems: A Review. China J. Highw. Transp. 2019, 32, 1–18. [Google Scholar]
  22. Hg, A.; Bx, B.; Hong, Y.C.; Bl, D.; Yy, D.; Shan, Z.B. Cuda-based parallelization of time-weighted dynamic time warping algorithm for time series analysis of remote sensing data. Comput. Geosci. 2020, 164, 105122. [Google Scholar]
  23. Wang, X.; Gao, J.-H. Automatic test for XML injections based on improved genetic algorithm. Comput. Eng. Des. 2021, 8, 2203–2210. [Google Scholar]
  24. Liu, W.; Chen, Y.; Li, H.; Zhang, H. Quantitative Study on Road Traffic Environment Complexity under Car-Following Condition. Sustainability 2022, 14, 6251. [Google Scholar] [CrossRef]
  25. Wang, L.; Li, H.; Li, S.; Bie, Y. Gradient illumination scheme design at the highway intersection entrance considering driver’s light adaption. Traffic Inj. Prev. 2022, 23, 266–270. [Google Scholar] [CrossRef] [PubMed]
  26. Yuan, G.; Bing, R.; Liu, X.; Dai, W.; Zhang, Y.-M.; Cai, Z. Spatial-Temporal Graph Neural Network based Hand Gesture Recognition. Acta Electron. Sin. 2022, 50, 921–931. [Google Scholar]
Figure 1. Flow chart of dangerous driving behavior recognition.
Figure 1. Flow chart of dangerous driving behavior recognition.
Sustainability 14 12355 g001
Figure 2. Driving simulator platform.
Figure 2. Driving simulator platform.
Sustainability 14 12355 g002
Figure 3. GY-95T sensor.
Figure 3. GY-95T sensor.
Sustainability 14 12355 g003
Figure 4. Six-axis data of hand trajectory. If there are multiple panels, they should be listed as: (a) Triaxial acceleration curve of hand track; (b) three-axis angular velocity curve of hand trajectory.
Figure 4. Six-axis data of hand trajectory. If there are multiple panels, they should be listed as: (a) Triaxial acceleration curve of hand track; (b) three-axis angular velocity curve of hand trajectory.
Sustainability 14 12355 g004
Figure 5. Hand movement trajectory curve for eating in 30 drivers and 15,000 trials.
Figure 5. Hand movement trajectory curve for eating in 30 drivers and 15,000 trials.
Sustainability 14 12355 g005
Figure 6. Hand movement trajectory curve for taking items in 30 drivers and 15,000 trials.
Figure 6. Hand movement trajectory curve for taking items in 30 drivers and 15,000 trials.
Sustainability 14 12355 g006
Figure 7. Hand movement trajectory curve for smoking items in 30 drivers and 15,000 trials.
Figure 7. Hand movement trajectory curve for smoking items in 30 drivers and 15,000 trials.
Sustainability 14 12355 g007
Figure 8. Hand movement trajectory curve for phone holding in 30 drivers and 15,000 trials.
Figure 8. Hand movement trajectory curve for phone holding in 30 drivers and 15,000 trials.
Sustainability 14 12355 g008
Figure 9. Hand movement trajectory curve for using a touch control monitor in 30 subjects and 15,000 trials.
Figure 9. Hand movement trajectory curve for using a touch control monitor in 30 subjects and 15,000 trials.
Sustainability 14 12355 g009
Figure 10. Five dangerous driving behavior labels.
Figure 10. Five dangerous driving behavior labels.
Sustainability 14 12355 g010
Figure 11. Handheld phone label curve and test curve.
Figure 11. Handheld phone label curve and test curve.
Sustainability 14 12355 g011
Figure 12. DTW algorithm regularized shortest distance visualization.
Figure 12. DTW algorithm regularized shortest distance visualization.
Sustainability 14 12355 g012
Figure 13. DTW algorithm for regularizing shortest distance curves.
Figure 13. DTW algorithm for regularizing shortest distance curves.
Sustainability 14 12355 g013
Figure 14. Handheld phone label curve and test curve.
Figure 14. Handheld phone label curve and test curve.
Sustainability 14 12355 g014
Figure 15. Box figure of matching degree of dangerous driving behavior (The circular sign is the mean value, and the diamond sign is the abnormal value).
Figure 15. Box figure of matching degree of dangerous driving behavior (The circular sign is the mean value, and the diamond sign is the abnormal value).
Sustainability 14 12355 g015
Figure 16. Structural flame diagram of this model algorithm.
Figure 16. Structural flame diagram of this model algorithm.
Sustainability 14 12355 g016
Figure 17. Support vector machine algorithm structure flame graph.
Figure 17. Support vector machine algorithm structure flame graph.
Sustainability 14 12355 g017
Figure 18. Structural flame diagram of neural network algorithm.
Figure 18. Structural flame diagram of neural network algorithm.
Sustainability 14 12355 g018
Table 1. The recognition rate table of dangerous driving behaviors.
Table 1. The recognition rate table of dangerous driving behaviors.
Dangerous Driving BehaviorsTest TimesTesting SetMean Matching DegreeRecognition Rate
Feeding451580.31%100%
Take goods451581.25%100%
Smoking451581.28%100%
Handy-phone451579.39%93.33%
Touch Control451580.00%100%
Table 2. Comparison of calculation effects of different algorithms.
Table 2. Comparison of calculation effects of different algorithms.
Dangerous Driving BehaviorsRecognition Rate/%Program Running Time/sProgram Running Consumes Memory/KB
ABCABCABC
Feeding100.093.3100.08.033.797.21968.01.6 × 105760
Take goods100.086.6100.09.917.099.91968.01.6 × 105100
Smoking100.093.3100.09.917.588.91968.01.6 × 10552
Handy-phone93.380.0100.09.915.089.81965.01.6 × 10532
Touch Control100.060.0100.09.825.7136.61971.01.6 × 10536
Feeding98.682.64100.09.521.8102.51968.01.6 × 105196
Average98.682.64100.09.521.8102.51968.01.6 × 105196
A is the model algorithm, B is the neural network algorithm, C is the support vector machine algorithm.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Liu, W.; Li, H.; Zhang, H. Dangerous Driving Behavior Recognition Based on Hand Trajectory. Sustainability 2022, 14, 12355. https://doi.org/10.3390/su141912355

AMA Style

Liu W, Li H, Zhang H. Dangerous Driving Behavior Recognition Based on Hand Trajectory. Sustainability. 2022; 14(19):12355. https://doi.org/10.3390/su141912355

Chicago/Turabian Style

Liu, Wenlong, Hongtao Li, and Hui Zhang. 2022. "Dangerous Driving Behavior Recognition Based on Hand Trajectory" Sustainability 14, no. 19: 12355. https://doi.org/10.3390/su141912355

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop