1. Introduction
The issue of traffic monitoring and management has arisen due to a growing number of personal vehicles, trucks, and other types of vehicles. Due to existing road capacities being based on historic designs, the condition of these road communications deteriorates with a lack of growing financial investment to maintain and expand the road network. With these requirements, vehicle visual identification is not sufficient for traffic management and the prediction of the future state of traffic and road conditions. For this purpose, existing monitoring areas are being innovated with new sensor platforms, not only for the statistical purpose of monitoring areas. Additional information such as traffic density, vehicle weight distribution, overweight vehicles, and trucks could be included in automatic warning systems for the prediction of possible critical traffic situations. There are several technological approaches based on different principles. Each of them has various advantages and disadvantages, such as operating duration, traffic density, meteorological condition limits, resistance to chemical and mechanical damage from maintenance vehicles, etc.
All motor vehicles are classified into 11 base classes by current legislation in the states of the central European Union. Meanwhile, according to the Federal Highway Administration under the United States Department of Transportation, there are even 13 classes. These classes consist of personal vehicles, trucks, technical vehicles, public transport vehicles, and their subclasses. For decades, the only sufficient method to classify vehicles was by visual recognition. This method was strongly limited by meteorological conditions. In the last two decades, several different technical designs for classifying vehicles without a visual part of classifications have been proposed. At first, based on metallic vehicle chassis and axle parts, there were designs to measure magnetic field parameters of crossing vehicles. This included inductive loops or anisotropic magneto-resistive sensors built into the road pavement [
1,
2,
3,
4]. These technological designs achieved accurate results for specific vehicle classes with magnetic signatures. A different approach was by vehicle weight signature. Technological solutions based on piezoelectric sensors [
5] and bending plate sensors are widely used in road traffic monitoring and vehicle measurements [
6]. There were also experimental solutions such as the usage of hydro-electric sensors [
7] with a bending metal plate at the top of the vessel filled with a specific liquid. Weigh-in-motion technologies measuring specific parameters such as the weight signature could be used, as well as other technologies including fiber optic sensors [
8], wireless vibration sensors [
9], or using embedded strain gauge sensors [
10]. As an additional capability, this could be measured by smart pavements based on conductive cementitious materials [
11]. Optic sensors based on Fiber Bragg Grating (FBG) were also successfully tested on different types of transport, such as railways. It was in Naples in Italy where this type of sensor was used for speed and weight measurements with the detection of train wheel abrasions as additional information for transport safety [
12].
Vehicle classification and the measurement of vehicle parameters, such as weigh-in-motion, were the aim of several international research projects. The weighing-in-motion of road vehicles was a research aim in the European research project COST323 over two decades ago [
13]. In the last decade, research ideas relating to infrastructure monitoring including road traffic have been studied, e.g., by COST projects TU1402 for structural health monitoring and TU1406 for roadway bridges [
14,
15,
16].
Optical fiber sensors are becoming a very important part of smart Internet of Things (IoT) infrastructures, also on roads and highways. They can additionally perform different functions in critical infrastructure protection and monitoring. There is a broad spectrum of technological solutions of fiber optic sensors and optical sensors systems. For our investigation, we used the FBG sensor network built into the entry road into the campus of our university. Fiber Bragg Grating (FBG) sensors are classified as passive optical fiber components that are compatible with existing types of telecommunication fiber systems and can operate directly with incident light (most commonly in the 1550 nm range). Thus, they can be directly incorporated into the optical transmission chain. The fundamental principle on which the FBG work is based is Fresnel diffraction and interference. The propagating optical field may be refracted or reflected at the interface in the transmission medium with different refractive indices. The FBG operates as a light reflector for a specific (desired) spectrum of wavelengths, to ensure that the phase-matching condition is met. Other (undesirable) wavelengths are only slightly influenced by the Bragg grating [
17,
18,
19].
In recent years, the different Convolutional Neural Network (CNN) architectures [
20,
21,
22] applied to image processing constitute the current dominant computer vision theory, especially in tasks such as image classification (vehicle classification). The main goal of these networks is to transform the input image layer-by-layer from the input image to the final class scores. The input image is processed by a series of convolution layers with filters (kernels), max pooling layers, and Fully Connected (FC) layers. Finally, the activation function, such as softmax or sigmoid to classify the outputs (small cars, sedans, crossovers, family vans, or trucks), is used. In our case, the AlexNet [
20] and GoogLeNet [
23] convolutional neural networks were chosen. The basic architecture of the AlexNet consists of some convolutional layers (five layers), followed by max pooling layers, FC layers (three layers), and a softmax layer [
20,
24,
25]. On the other hand, the architecture of GoogLeNet consists of 22 layers (nine inception modules). The main motivation for the inception modules’ (layers’) creation is to make a deeper CNN network so that highly accurate results could be achieved [
23,
26,
27]. For vehicle classification, several works using deep learning and convolutional neural networks were described in [
20].
The aim of this article is vehicle classification with FBG sensor arrays using artificial intelligence from partial records. The proposed neural network was trained using a dataset with a lack of information on the vehicle’s speed, which we created by visual recognition of the vehicle passing through our testing platform. The majority of recorded vehicles were detected only through their left wheels, which reduces records from a 3D vehicle surface to one line of deformation. These records simulated situations where the vehicle’s driver tried to avoid detection with a changed trajectory through the roadside or an emergency line without visual recognition.
3. Experimental Results
For all CNNs, the training had the same option setup as training for 200 epochs with the batch size set to twenty. On the main diagonal in the confusion matrix, correctly classified vehicles of all tested vehicles are shown in
Table 4. For the first class (hatchback class), forty-nine-point-six percent of vehicles were correctly classified (valid column), as shown in
Table 5. For the second class (combo/sedan class), twelve-point-point-eight percent of vehicles were correctly classified. For the third class (SUV class), fifty-six-point-three percent of vehicles were correctly classified. For the fourth class (MPV/minivan class), twenty-six-point-eight percent of vehicles were correctly classified. For the last class (van/truck class), sixty-two percent of vehicles were correctly classified. An overall accuracy of 28.9% for all tested vehicles using the proposed CNN was achieved. Due to the classification into five classes and their similarities, a validation accuracy of only 28.9% was achieved.
The proposed CNN was modified to three classes for better spatial separation of classes. On the main diagonal in the confusion matrix, correctly classified vehicles of all tested vehicles are shown in
Table 5. For the first class (hatchback class), seventy-four-point-three percent of vehicles were correctly classified (valid column), as shown in
Table 5. For the second class (SUV class), thirty-seven-point-eight percent of vehicles were correctly classified. For the third class (van/truck class), seventy-eight-point-nine percent of vehicles were correctly classified. An overall accuracy of 60.0% of all tested vehicles using the proposed CNN was achieved.
The proposed CNN was modified for classification between two classes (hatchback class to van/truck class). An overall accuracy of 92.7% for both tested vehicle classes, as shown in
Table 6, using the proposed CNN was achieved.
Proposed CNN for Binary Vehicle Classification
For the improvement of the achieved validation for three classes in the combination of binary classification, we used the designed CNN for classification of three variations of the prepared dataset. Three classes were compared as binary, with one to the rest. Continuing, data from the first class were classified in opposition to the combined data of the other two and the second class in opposition to the first and third class. Finally, data from the third class were classified into the combined data from the first and second classes, as shown in
Figure 15.
In the first part, the proposed CNN for binary vehicle classification (small vehicles to the rest of the vehicles) using the training dataset was trained. This training dataset was modified to a ratio of 1:1 (800 records for each class). The results from the test dataset are shown in
Table 7.
In the second part, the proposed CNN for binary vehicle classification (SUV vehicles to the rest of the vehicles) using the training dataset was trained. This training dataset was modified to a ratio of 1:1 (800 records for each class). The results from the test dataset are shown in
Table 8.
In the third part, the proposed CNN for binary vehicle classification (truck vehicles to the rest of the vehicles) using the training dataset was trained. This training dataset was modified to a ratio of 1:2 (400 records for the van/truck class, 800 images for the rest of the vehicles). The results from the test dataset are shown in
Table 9.
An improved process by binary predicting three classes as one to the rest achieved valid classification on the test dataset of 70.8%, as shown in
Table 10. Each row in the confusion matrix represents a test group for the class, and the columns represent the category. Highlighted values in the diagonal show properly classified vehicles.
Due to there being no information about vehicle speed, the wheelbase with axle configurations, and wheel sizes, the valid classification of 70.8% achieved is acceptable. These results are from one line of vertical FBG sensors with partially overpassing vehicles by one line of wheels. The speed and wheelbase similarities between the first and second classes created significant incorrect classifications, which can be reduced with a larger part of the dataset for the training of the designed CNN.
4. Discussion
The obtained results from the experimental platform, which consists of vertically oriented FBG sensor arrays, are presented. Due to the location of the testing platform on the two way access road into the university campus, because the sensor arrays are installed in the middle of this road, there was some limitation. More than 80% of recorded vehicles passed over the inbuilt sensors. Those vehicles were recorded only with one line of wheels without measurement by the horizontally oriented FBG sensors. A minimum of two lines of sensors is necessary for wheelbase distance determination and vehicle speed measuring. We focused on the classification of passing vehicles only from one line of vertical FBG sensors. The proposed neural network was capable of separating trucks from other vehicles with an accuracy of 94.9%. To classify three different classes, an accuracy of 70.8% was achieved. Based on experimental results, extending the sensor arrays is recommended.
The approach that solved the width of the vehicle’s wheels is based on horizontal fiber optic sensors with a 45° orientation in the vehicle’s direction over the test platform for the left and right side of the vehicle, separated as shown in
Figure 16. This solution is limited by vehicle speed due to the double bridge construction over the road. Another technological approach, which is already widely used, is based on bending plates installed in a concrete block of the road. These sensors could be separated for the left and right side of the vehicle or be combined for weighing the whole vehicle’s axle. With these realizations, there is no need to know the wheel width, because there is a whole contact area of the wheel with road sensors.
In order to gain significant improvements of these results, it would be necessary to extend the sensor arrays to the full width of the road. An alternative solution will be a change from two way road management to one way.