Vehicle Auto-Classification Using Machine Learning Algorithms Based on Seismic Fingerprinting

Abstract

Most vehicle classification systems now use data from images or videos. However, these approaches violate drivers’ privacy and reveal their identities. Due to various disruptions, detecting automobiles using seismic ambient noise signals is challenging. This study uses seismic surface waves to compare time series data between different vehicle types. We applied various artificial intelligence approaches using raw data from three different vehicle sizes (Bus/Truck, Car, and Motorcycle) and background noise. By using vertical component seismic data, this study compares the decoding abilities of Logistic Regression, Support Vector Machine, and Naïve Bayes (NB) approaches to determine the class of automobiles. The Multiclass classifiers were trained on 4185 samples and tested on 1395 randomly chosen from actual and synthetic data sets. Additionally, we used the convolutional neural network approach as a baseline to assess the effectiveness of machine learning (ML) methods. The NB method showed relatively high classification accuracy during training for the three multiclass classification situations. Overall, we investigate an ML-based decoding technique that can be used for security and traffic analysis across large geographic areas without endangering driver privacy and is more effective and economical than conventional methods.

1. Introduction

Many nations have made significant investments in traffic monitoring systems [1], which gather and analyze traffic data to produce statistics concerning the number of vehicles on the road and their usage trends over time. Governments can use these figures to plan road upkeep and estimate future transportation demands. A crucial aspect in predicting noise levels and road damage is determining the size of vehicles—according to the Traffic Monitoring Guide report released by the Federal Highway Administration in the United States [2], a roadway’s geometry is designed based on the typical mix of vehicle types that utilize that route.

Vehicle detection systems use UpToDate sensing technology supported by machine learning (ML) approaches [3]. While modern systems achieve increasingly accurate vehicle classification, their features and requirements vary, including the types of sensors used, parameter settings, operational environments, and cost. Many traffic monitoring systems include methods for classifying vehicles based on eyesight, often cameras; these systems achieve high classification accuracy between 90 and 99% [4].

Although camera-based systems have a high degree of classification accuracy, other factors, such as weather and lighting, can impact their effectiveness. For example, an automobile can be fully or partially missed when a larger vehicle covers the camera’s view.

Additionally, such systems require significant infrastructure expenditure to cover the road network fully. Since many individuals do not feel comfortable being monitored by cameras, the associated privacy issues for vehicle occupants are also a significant issue. An inductive loop detector based on vehicles’ magnetic properties is one of the most commonly used traffic monitoring systems for vehicle recognition and categorization [5]. This loop detector technology is built on a coil of wire buried beneath the road that records changes in the amplitude, phase, and frequency of the magnetic profile signal when a vehicle passes over it [6]. The loop detector technique has been widely studied, with previous works demonstrating its high accuracy (99%) for classifying large vehicles, such as cars, trucks, and vans [7,8,9,10]. The accuracy of loop detector systems has been shown to be independent of vehicle speed [11].

Different sensor types have been utilized in other proposed privacy-preserving systems. When combining several sensor networks, magnetic sensors positioned in the middle of or adjacent roads can achieve vehicle classification accuracy levels of up to 96.4% [12,13,14]. In comparison, a combination of infrared and ultrasonic sensors can provide an accuracy of up to 99% [15]. In addition to the current methods, new approaches have been suggested for monitoring traffic congestion in metropolitan areas based on GPS, social media, and network data collected directly from vehicles [16,17,18,19,20]. Intelligent transportation systems have been developed using ML techniques, with detailed information on metropolitan area traffic patterns and travel patterns now available. However, compared to inductive loops and camera-based systems, most of the proposed methods are less accurate at categorization; in addition, these approaches may require specialized installations, such as loop detectors for public roadways [21].

A range of vibration-based vehicle classification techniques has been proposed to address this issue. Vehicles’ engine systems and the interaction of their tires with the road are the principal causes of vehicle vibrations [22,23,24], and the vehicle’s size significantly impacts these signals. Due to the impact of the underlying geology on the propagation of the seismic wave, it can be challenging to locate these signals. Due to the similar vibrations of most cars at frequencies around 20 Hz, identifying seismic signals from moving objects in real life remains challenging. However, these signals are less susceptible to wind noise due to passing through the ground, a factor that is advantageous for vehicle detection [25]. This study explores using artificial intelligence (AI) approaches to identify automobiles from seismic waves. AI has significantly improved voice recognition technologies, such as voice analysis, in the past ten years [26].

Additionally, AI has been applied to seismic event categorization [27,28,29,30]; thus, we consider this a promising approach to vehicle identification. In addition to its benefits for vehicle occupant privacy, AI seismic data-based traffic monitoring is also expected to achieve low power consumption and cost. Furthermore, seismic-based traffic monitoring can be implemented in remote and limited-access areas, and it can operate for extended periods (months) with minimum maintenance and power supply. Most seismic sensor systems are prepared for extreme environments, unlike camera-based systems. Moreover, the seismic system does not have a blind spot; it can cover a relatively large area and be hidden easily, making it suitable for security and border control purposes. A neural network was employed in a 2010 study to categorize automobiles based on seismic data [31], while another study published in 2019 only used seismic signatures [25]. In the latter work, a log-scaled frequency cepstral coefficient matrix was proposed to extract spectral information from vehicle seismic signals; this approach was able to classify objects with an accuracy of up to 91.39%. Both studies focused on large military vehicles; thus, these approaches cannot be directly applied to civilian cars. Additionally, neither technique can utilize seismic data without preprocessing or without the inclusion of additional data. Ahmad and Tsuji (2021) developed a seismometer-based traffic monitoring system using a convolutional neural network (CNN) [32]. However, this approach investigated only one specific branch of ML (the neural networks).

In this study, we used three distinct ML algorithms to classify the size of cars using seismic waves—Logistic Regression (LR), Support Vector Machine (SVM), and Naïve Bayes (NB). Previous studies have captured and categorized waveform data from various sources to investigate how effectively ML approaches can extract information from seismic waves; thus, we also contrast the findings of this investigation with the published CNN-based approach. This study will extend the investigation of using other ML algorithms to classify vehicles of different sizes based on their seismic signals. Although this study shares the same goal and data set as the previous study of Ahmad and Tsuji (2021) [32], it discusses the usage of more straightforward and linear methods such as LR to conclude if those methods are beneficial. Another difference between these methods (SVM, LR, and NB) and neural networks is that the idea of neural networks highly depends on optimized variables (weights/filters), making it relatively computationally consuming. The previous study investigated only a specific branch of ML, neural networks, and focused on the well-known architectures for voice recognition, such as CNN and RNN. To our knowledge, this is the first study to use Logistic Regression, Support Vector Machine, and Naïve Bayes to predict and classify vehicles using their ground motions.

This study highlights the ability of seismic surface ambient noise to carry more information than has been recognized, and we hope that the work presented here will inspire researchers to investigate more uses for seismic surface noise. This understanding can improve earthquake early warning systems, civil engineering, plant operations optimization, crime prevention, and even enhance hydrocarbon and water exploration by extracting hidden information.

2. Materials and Methods

2.1. Data Set

For this investigation, we collected seismic vertical motion data for several cars using geophones at Kyushu University in July 2020. The geophones were set up at three sites, each 15 m apart and 0.5 m from the road. Vibrations in the vertical direction were captured at a 250 Hz sampling rate. We categorize vehicles as large (e.g., buses and trucks), medium (e.g., passenger cars), and small (e.g., motorcycles and scooters) size. Figure 1 shows the configuration of the seismic sensors used in the survey.

We used video records as a visual guide to preparing the events/vehicles catalog. The video camera and geophones were synchronized to a GPS clock to ensure accuracy in data preparation. However, the video/images were not used as data input for the algorithms. When an automobile was near the geophone, each event (i.e., the passage of a vehicle) lasted 2–3 s.

We calculated the automobiles’ speeds based on the signals from three stations. Most of the cars measured in this experiment traveled between 25 and 35 km/h, with a top speed of 45 km/h. To prevent overfitting the models, we only included clear vehicle signals during the training phase, excluding any signals that had background noise or overlapped with other cars. The chosen occurrences were sliced from the record as 5-sec windows containing 1251 data points. The following mathematical formulations assume vibration data:

W = S(t)₁, S(t)₂, S(t)₃, S(t)₄, ……., S(t)_n,

S is the geophone reading at time (t), which is 0.004 s (250 Hz) in this case, and n is the number of samples (1251) in the study case. This period was selected to ensure that the entire seismic waveform was fully included. We retrieved an average of 68 waveform windows per geophone station per vehicle class with a total of 612 events. To cover the noise as the fourth class in our data, we chose 318 representative waveform windows; strong winds, bicycles, pedestrians, trolley-pushing pedestrians, road maintenance, and general background noise represent several examples of noise sources. Thus, the total original data set comprised a 930 windows. Figure 2 shows an example of each vehicle class and noise.

2.2. Training Data Augmentation

Extensive training data set is required in ML for precise prediction and to avoid overfitting [33]. For this reason, our initial dataset of 930 samples was inadequate, and therefore we created extra synthetic data from it for training. In order to alter the signal-to-noise ratio (SNR) of waveforms, random noise was applied, as shown in Figure 3. We varied the SNR [34] from 1 to 5, as determined by the following expression:

SNR = P_signal/P_noise = (A_signal/A_noise)²,

(1)

where A is the root mean square amplitude and P is the average power. ~4650 artificial samples were employed in the enhanced dataset created and used for training.

2.3. Machine Learning Schemes

2.3.1. Logistic Regression (LR) for ML

The probability of a target variable is predicted using the supervised learning classification technique known as logistic regression. In LR, we take the output of the linear function and compress the value to the range of [0, 1] using the sigmoid function (logistic function). Any real-valued integer may be mapped to a value between 0 and 1 using the sigmoid function, which is an S-shaped curve but never precisely at those values [35]. For building our classifier model, we used generalized linear models with LR (Figure 4).

A logistic regression model makes mathematical predictions about P(Y = 1) as a function of X. Several category problems, including spam detection and diabetes prediction, may be solved using one of the most fundamental machine learning methods.

2.3.2. Support Vector Machine (SVM)

SVM algorithm was implemented in Matlab to train SVM classifiers for model-building and then use the optimal classifier for new data classification. We used SVM with a non-linear kernel, e.g., a radial basis function. The traditional C-SVM model was used as a classification model and can be described as follows:

$$\frac{1}{2} \Vert w{\Vert }^2+C{{\displaystyle \sum }}_{i=1}^l{\zeta }_i,$$

(2)

$$y_i\left[w·x_i+b\right]\ge 1-{\zeta }_i, \left({\zeta }_i\ge 0\right), i=1,2,3\dots l,$$

(3)

where w is the average vector, C is the penalty factor, and ζ_i is the margin of error. x_i is the input, y_i is the class label, b is the bias to the separation hyperplane, and l is the number of samples of the input x_i.

Additionally, the radial basis kernel function, which was applied to address the non-linear characteristics of the geophysical data, can be described as:

$$K\left(x_i,x_j\right)=exp\left(-g\Vert x_i-x_j{\Vert }^2\right), g>0,$$

(4)

where g is the kernel parameter that denotes the transformed data’s gamma distribution, and the kernel parameter g, and penalty factor C are adjusted to search for optimal separation hyperplane. We used ten-fold cross-validation for training classifiers to avoid overfitting. In contrast, the SVM approach involves adopting a non-linear kernel function to transform the input data into a higher-dimensional feature space, making it easier to separate the data. The iterative learning process in SVM identifies the optimal hyperplanes with the maximum margin between each class in a higher-dimensional feature space.

2.3.3. Naïve Bayes (NB)

NB is a probabilistic classifier that applies Bayes’ theorem with strong (naïve) independence assumptions between the features. The following was used to calculate a posterior probability of A happening given that B happened:

$$P\left(A\mid B\right)=P\left(A{\displaystyle \cap }B\right)/P\left(B\right)=P\left(A\right)\times P\left(B\mid A\right)/P\left(B\right),$$

(5)

where A and B are events or classes, P(A) and P(B) are the probabilities of A occurring and B occurring independently of each other. P(B) should be greater than zero, and P(B∣A) is the probability of B occurring, given that A is true. Bayes’ Rule was applied to our classification problem; we classified our data into the population that maximizes the posterior probability for the decision rule.

2.3.4. Convolutional Neural Networks (CNNs)

Neural Networks form the main backbone for deep learning. CNN is an enhanced neural network design. This AI technique was inspired by biological processes between neurons of the animal visual cortex [36]. In this study, we used CNNs to compare the performance of ML, and deep learning (DL) approaches. CNNs use filters before the neural networks called the convolutional layer. In our CNN, we used multiple convolutional layers with multi-channel filters in each layer and maximum pooling. The convolutional layer extracts the unique features and passes them to the neural networks. CNN breaks problems into smaller-scale tasks, making it an effective method for solving complex problems. Every neuron in each layer holds values depending on the previous input layer, and the weight connecting the neurons is given by:

$$Y={{\displaystyle \sum }}_1^n\left(X*W\right)+b,$$

(6)

where n represents the number of neurons in a hidden layer, X is the value that the neuron holds, W represents the weight that connects Y with X, and b is the bias for the equation.

We used published CNN architecture shown in Figure 5. The architecture contains five convolutional layers, each containing 50 1D mathematical filters. After the mathematical filter for downsampling, we used the MaxPool layer with dimensions (1 × 3). The CNN contains four hidden layers with redactional size, and the final layer is four output classes normalized with the softmax equation to 0 and 1 [32]. We used the pre-trained CNN model for comparison with the other three ML methods.

3. Results and Discussion

We calculated precision, recall, f1-score, and accuracy to evaluate the three proposed methods using the same training and test data set [37]. The formula for each evaluation technique is described as the following equation:

$$Precision=\frac{True Positive}{True Postive+FalsePostive},$$

(7)

$$Recall=\frac{True Positive}{True Positive+FalseNegative },$$

(8)

$$f1-score=\frac{2\times Precision\times Recall}{Precision+Recall },$$

(9)

$$Accuracy=\frac{True Positive+True Negative }{all predictions },$$

(10)

We randomly split the dataset into training and testing data sets in a 75–25 ratio. The testing data set includes (259 buses, 439 cars, 228 motorcycles, and 469 noise) data samples, including actual and synthetic data.

We used a confusion matrix heat map to visualize the performance of LR, SVM, and NB shown in Figure 6. Apart from the decision accuracy evaluation, we measured the computational time for the three methods and compared those results with the state-of-the-art CNN architecture.

The confusion matrix of LR detection is presented in Figure 6a.

Table 1. The evaluation parameters for LR model prediction for 1395 data samples, including precision, recall, and f1-score calculated per class, averaged, and weighted average on the number of samples for each class.

	Precision	Recall	F1-Score	Number of Predictions
Bus/Truck	0.53	1.00	0.69	136
Passenger car	0.39	0.68	0.49	252
Motorcycle	0.00	0.00	0.00	11
Noise	0.99	0.47	0.64	966
Average	0.48	0.54	0.45	1395
Weighted average	0.83	0.55	0.61	1395

shows the precision, recall, and f1-score for each class of LR calculated based on the result of Figure 6a. Based on Figure 6a, LR has failed to detect any motorcycle signals, while it has overpredicted the noise, especially for car and motorcycle classes.

LR has made 11 predictions for a motorcycle, but all were false positive predictions toward the bus class, as shown in Figure 6a. Table 1 shows that LR scores 99% precision for the noise class and 47% recall only for the same class. On the other hand, LR scores 100% recall and 53% precision for the bus class. These scores indicate that the LR classifier is unsuitable for the proposed task and will probably miss or incorrectly detect the right vehicle’s class. Figure 6b shows the decisions made by the SVM model. We can see a slight improvement compared to LR model predictions. Based on Figure 6b and

Table 2. The evaluation parameters for SVM model prediction for 1395 data samples, including precision, recall, and f1-score calculated per class, averaged, and weighted average on the number of samples for each class.

	Precision	Recall	F1-Score	Number of Predictions
Bus/Truck	0.43	0.97	0.60	116
Passenger car	0.50	0.67	0.57	330
Motorcycle	0.06	0.25	0.10	57
Noise	0.97	0.51	0.67	892
Average	0.49	0.60	0.49	1395
Weighted average	0.78	0.58	0.62	1395

, SVM has 14 true positive motorcycle predictions. However, the SVM’s scores are relatively too low to be a reliable method for the given task.

The third algorithm we tested in this study (NB) has significantly improved prediction scores. NB could successfully predict most motorcycle data samples, which LR and SVM failed to do. NB has also sourced 98% precision and 97% f1-score for the bus class, as shown in

Table 3. The evaluation parameters for NB model prediction for 1395 data samples, including precision, recall, and f1-score calculated per class, averaged, and weighted average on the number of samples for each class.

	Precision	Recall	F1-Score	Number of Predictions
Bus/Truck	0.98	0.96	0.97	265
Passenger car	0.53	0.86	0.66	272
Motorcycle	0.62	0.39	0.48	363
Noise	0.90	0.85	0.88	495
Average	0.76	0.77	0.75	1395
Weighted average	0.77	0.75	0.75	1395

. NB scored a 47% f1-score for the motorcycle class, considered the highest among the three methods. NB has scored relatively high precision, recall, and f1-scores which might make it a potential competitor for the CNN algorithm in this task.

In addition to the previous evaluation parameters, we have calculated the accuracy using equation 10, measured the running time for all three methods, and compared it with the accuracy and running time for CNN, as shown in

Table 4. Accuracy and running time of LR, SVM, NB, and CNN. The running time includes the training and testing on data set with a size of 5580 samples.

	LR	SVM	NB	CNN
Accuracy	55%	58%	75%	94%
Running time (Seconds)	1.664	12.49	0.150	112.3 *

* CNN has trained for 100 iterations.

. We used a work frame consisting of Python code based on ObsPy, NumPy, and scikit-learn libraries for this task. We used a cloud computing platform containing dual Tesla K80 GPUs with 12 GB RAM to run all algorithms. While CNN still scores the highest accuracy along all proposed methods, NB has 750 times faster computation time. While CNN required 112 s to finalize the training and test, NB needed 0.15 s to do the same task using an identical data set.

4. Conclusions

We have tested three ML algorithms, Logistic Regression, Support Vector Machine, and Naïve Bayes, using the same data set to investigate which is suitable for traffic monitoring based on the ground motion generated by vehicles. We have observed four factors in training and testing to evaluate the efficiency of the methods and have compared them to the state-of-the-art CNN using 5580 data samples. After testing and observation, we find that both Logistic Regression and Support Vector Machine are not suitable for the mentioned task. Logistic Regression and Support Vector Machine scored low in all evaluations. LR and SVM failed to recognize any motorcycle seismic signals, and both of them have low precision for vehicles and high precision for noise, indicating that these methods frequently mispredicted vehicles as noise. SVM was computationally expensive relative to the other ML algorithm. Therefore, we do not recommend considering Logistic Regression nor Support Vector Machine for the given task.

On the other hand, Naïve Bayes has shown promising results. NB has average F-1 scores of 75%, up to 97% in some cases. NB did not have difficulty correctly predicting motorcycles and scored 62% precision. Although CNN is the superior method in accuracy and precision, Naïve Bayes has shown modest computational cost. NB was 750 times faster than CNN under the same conditions. We recommend developing and enhancing models for traffic monitoring using NB with frequency domain data. We will consider this factor in future research. Therefore, we recommend NB for vehicle classification application that has real-time processing. Otherwise, CNN can be adopted for offline applications based on the significant compromise on accuracy.

However, this approach faces some challenges as the current approach is limited to a single-vehicle pass; this problem may be overcome by installing two or more geophones within a known distance near the roadway. Another consideration that should be studied in the future is real-time monitoring. So far, all studies have used seismic data records but not real-time data. We will consider this purpose in future research. The proposed method can be further enhanced by forecasting various aspects of traffic flow, such as speed, direction, and whether a road user is driving safely. This approach may also be adapted and applied to other modes of transportation, including ships, bicycles, foot traffic, and airplanes.