1. Introduction
Bridges are critical infrastructure structures that support transportation networks. Bridges daily degrade due to varied external factors, such as deterioration caused by chemical reactions with oxygen and chloride ions in the air, high cycle fatigue caused by traffic vibrations, loss of members due to impact loads of traffic accidents, and large-scale destruction caused by natural disasters. The economic impact of bridge malfunction is significant and sometimes life-threatening. Therefore, early detection of structural performance deterioration of bridges is essential, and appropriate monitoring and maintenance are crucial. One method of vibration monitoring in bridges is installing accelerometers directly on the bridge. While this method can evaluate bridge performance with high accuracy, there are challenges in applying it to multiple bridges. The need to install multiple sensors on a single bridge is time-consuming, costly, problematic in securing a power source, requires traffic control during installation, and is sometimes hazardous.
Yang et al. (2004) [
1] proposed mounting sensors on traveling vehicles instead of the target bridge. Since the vehicle vibration contains the bridge’s mechanical information, it is expected to estimate the bridge condition indirectly from the vehicle vibration data. For example, Lin and Yang (2005) [
2] experimentally extracted the bridge’s natural frequency from the Fourier’s power spectrum of the measured vehicle vibration. Yang and Chang (2009) [
3,
4] tried to estimate the bridge’s natural frequencies accurately for considering its application to bridge damage detection.
Inspired by Yang’s indirect approach, many researchers have vigorously developed varied drive-by monitoring technologies. Xiang et al. (2010) [
5] applied STFT (Short-Time Fourier Transform) to vibration data of a traveling vehicle with a shaker to evaluate the bridge condition. Their numerical examination shows that the power of vehicle vibration in the time-frequency domain reacts sensitively to the bridge’s local damage. Nguyen et al. (2010) [
6] also showed a similar result by applying wavelet transform. According to their results, it is efficient to use the spatial indices of the bridge vibration to detect damage. Thus, Oshima et al. (2014) [
7] proposed a method to estimate the bridge’s mode shape from vehicle vibrations. However, these drive-by monitoring technologies based on spatial indices assume that the vehicle position can be measured accurately. Takahashi et al. (2019) [
8] used SSMA (Spatial Singular-Mode Angle) for damage detection and also confirmed its high sensitivity, while the SSMA-based method also requires accurate vehicle position data temporally synchronized with the vehicle vibration data.
The data measured in the actual vehicles passing through the actual bridges contain various noises caused by varied factors—for example, the effects of road surface roughness, vehicle speed, and temperature. Malekjafarian et al. (2019, 2020) [
9,
10] used artificial neural networks and Gaussian processes to analyze the relationship between these factors and vehicle response analysis. Corbally and Malekjafarian (2021) [
11] showed that the model above can be used to detect changes in boundary conditions due to cracks in the bridge deck and bearing seizures. Locke et al. (2020) [
12] showed that bridge damage discrimination using vehicle vibration can be estimated with high accuracy even when environmental and operational noise, such as road traffic, road surface roughness, and temperature, are included. Sarwar and Cantero (2021) [
13] suggested a method to estimate the location and extent of bridge damage using deep autoencoders from vehicle vibration. As environmental factors, the number of vehicles, vehicle speed, measurement noise, and variation in vehicle characteristics are considered, and robust estimation results can be obtained for each of them. Although these studies are based on vehicle vibrations simulated numerically, they suggest the possibility of bridge damage detection by drive-by monitoring.
Although many papers explicitly state that they measure vehicle vibration, many also assume the use of vehicle position in their signal processing [
7,
8]. Satellite positioning, navigation, and timing systems, the so-called GPS (Global Positioning System), can measure a vehicle’s position. However, the readily available GPS devices are also subject to significant errors, so a vehicle’s travel position is not reliable. Bridge locations can be obtained from GIS (Geographic Information System) data, but the resolution is too coarse. Accurate information on the location of each bridge is usually unknown or not digitized. Furthermore, drive-by monitoring primarily uses data only while a vehicle passes over a bridge. Because the time that a vehicle travels on a bridge is so short, the location information measured by GPS devices is limited to a rough extraction of interest data. Therefore, when driving data on bridges are extracted from GPS-only location data, it is likely to include data from sections of the bridge that are not bridges, and this will likely harm the results of the analysis. In addition, multiple runs are a promising approach for drive-by monitoring to overcome the problems of road surface irregularities, environmental effects, and limited measurement time [
14]. Therefore, for the social implementation of drive-by monitoring, it is essential to have a technology that can automatically extract only the target data from considerable driving data.
As a possible idea about signal processing to extract the “on a bridge” vehicle response, the method proposed by Wang et al. (2018) [
15] can be considered. This technique is for estimating the natural bridge frequencies from vehicle vibration. A particle filter (sequential Monte Carlo method) is used to estimate the input profile from the vehicle vibration. The input profile is expressed as the sum of the road surface roughness and bridge vibration at the axle position. Assuming that the vehicle is moving in a straight line, the road profiles of the front and rear wheels are the same. The bridge vibration component can be obtained as the difference between the input profiles of the front and rear wheels. However, this method assumes that the vehicle characteristics are given in advance. Murai et al. (2019) [
16] succeeded in extracting “on a bridge” signals by using the difference of the unsprung mass vibration of the vehicle. This method is verified by numerical simulation. Shin et al. (2021) [
17], who examined this method in a field test, pointed out that it is difficult to visually determine the “on a bridge” section due to the influence of environmental uncertainties and measurement noise. However, Shin et al. also showed that C-LSTM networks, which learn from temporal and spatial features, can estimate entry and exit timing. However, utilizing “on a bridge” labels remains a challenge. Because of the low reliability of the GPS devices, it is not easy to eliminate the influence of the road profile. Therefore, this study investigates automatic labeling of the vehicle response to a bridge. The data are measured on the actual vehicle. The models are constructed separately by using the data obtained at each axle.
Neural networks are used in various fields, most notably in speech recognition and natural language processing, where they have shown excellent performance (Donahue et al., 2015 [
18]). CNN (Convolutional Neural Network) has the advantage of learning local responses from spatial-domain–time-domain data, but they are not suitable for learning sequential correlations. On the other hand, RNN (Recurrent Neural Network) can be characterized for sequential modeling but is not suitable for parallel processing (Zhou et al., 2015 [
19]). In addition, for the gradient exploding or vanishing problem caused by general RNNs, a LSTM unit can be introduced to analyze the data, including past information (Hochreiter and Schmidhuber [
20]). The combination of CNN and LSTM can be used for visual recognition and description (Donahue et al., 2015 [
18]), text classification (Zhou et al., 2015 [
19]), web traffic anomaly detection (Kim and Cho, 2018 [
21]), and residential energy consumption prediction (Kim and Cho, 2019 [
22]). Acceleration data measured by vehicles traveling on bridges are time-series data that contain both temporal and spatial features. In recent years, it has been applied to direct bridge damage detection (Yang et al., 2020 [
23], Yang et al., 2021 [
24]), and we thought that it would be possible to create a model that discriminates only the portion of the vehicle acceleration data traveling on the bridge by using C-LSTM.
Wang et al. (2019) [
25] proposed estimating tire ground forces with relatively high accuracy even at vehicle speeds of about 30 km/h using a Kalman filter based on vehicle vibrations measured with an iPod installed on the vehicle. Yang et al. (2020) [
26] compared the accuracy of estimating the natural frequencies of bridges using data measured by sensors installed on special vehicles and directly measured bridge vibration. When traveling at relatively low speeds, it is possible to estimate up to the second-order natural frequencies of bridges without being affected by the frequency of the vehicle by determining the ground point force from the vehicle vibration. Yang et al. (2022) [
27] proposed a comprehensive indirect method for estimating girder bridges’ fundamental mode dynamic properties with two identical trailer-mounted passing tractors and evaluating the element stiffness. They were validated both numerically and in field experiments. Locke et al. (2022) [
28] compared the system identification capabilities of different OMA (Operational Modal Analysis) techniques on bridges less than 18.28 m (60 ft). The experimental results demonstrate that OMA techniques can be utilized to correctly identify the frequencies of short-span bridges in the dynamic response of a passenger vehicle even in the presence of time-varying and nonlinear system characteristics. In prior research, as pointed out in the above paper, when the roughness of the road surface on which the left and right tires’ travel paths are different, the estimation accuracy of dynamic tire force and bridge frequency is reduced. However, no onboard sensors have been proposed to verify this, and no measurements have been conducted on heavy vehicles capable of exciting bridges. Therefore, this study will develop a new in-vehicle measurement sensor system to measure 3D behavior.
This study focuses on the automatic determination of driving sections on bridges included in vehicle vibrations for the social implementation of drive-by monitoring. A sensor system capable of measuring the three-dimensional behavior of vehicles was proposed and field-tested. By comparing three types of information, namely vehicle vibration, location information measured by GPS sensors installed on the bridge, and location information obtained from Google Maps, the correct label for the segment traveling on the bridge can be obtained. A machine learning method is used to determine whether the vehicle is traveling on the bridge section or not. The discrimination model is a supervised binary classification problem using C-LSTM networks.
4. Result and Discussion
For the data measured in the field test, the detection of the traveling section on the bridge by C-LSTM networks showed high accuracy in both the front and rear models. The results obtained are shown in
Table 3. In particular, high accuracy was obtained for the front wheel models when no filter was applied, and the highest accuracy was obtained for the rear wheel models when a low-pass filter was applied. The vehicle used in this experiment was designed to transport crushed stones and other materials, and the tires and suspension of the front wheels were designed to enhance riding comfort. Therefore, the data measured at the front wheels are expected to have reduced low-frequency and high-frequency noise effects. Therefore, the filtering performed as preprocessing had little effect. On the other hand, vibration contamination by high-frequency components was observed, especially in the rear wheels, and it is considered that the low-pass filter made the steep peaks more pronounced when running at joints.
Only bridges with joints were included in this study and were given the label “on a bridge” based on the peaks in the vehicle vibration. However, since some bridges do not have joints, this technique has only limited applicability and needs to be improved to acquire practicality. For example, non-jointed bridges are integral bridge abutments (IABs). While IABs are easy to run and construct, soil settlement occurs as the bridge expands and contracts due to temperature. To mitigate this, Zadehmohamad et al., 2021 [
29], considered a backfill mixture of tire material and soil, which is being developed with high interest. To automatically detect non-jointed bridges, it is necessary to study a method to extract only the bridge vibration component from the vehicle vibration and utilize it for bridge labels. However, non-jointed bridges have only short spans and are often destroyed by scour. Therefore, it can be assumed that many non-joint bridges will be monitored based on water level data rather than vehicle vibration data, and different approaches, such as vehicle–bridge simultaneous measurement, can be expected.
In addition, the model considered in this study may respond to peaks in vehicle vibration when traveling outside of joints. Therefore, it is necessary to investigate a method to extract only the bridge vibration component from the vehicle vibration and utilize it for bridge labels. Methods to extract only the bridge vibration component have been proposed by Wang et al. (2018) [
15] and Murai et al. (2019) [
16]. However, it is necessary to synchronize the positions of the front and rear wheels to remove the road surface irregularity component in the vehicle vibration. This method is challenging to apply when there are measurement errors in position information or high vehicle speed because even minor errors in position synchronization can affect the method. Therefore, assuming that the measured vehicle vibration includes bridge vibration, it may be possible to solve this problem by increasing the complexity of the discriminant model. It is expected to develop a multi-class classification model that considers multiple labeling, such as when driving on bridges, when driving on non-bridges, and when driving on seams. Unsupervised learning methodologies would also be effective. In advance, a model is trained to estimate vehicle vibration during non-bridge driving. When the model is applied to the measured vehicle vibration, if the model does not fit well, the vehicle may be traveling on a bridge.
5. Conclusions
In this study, an in-vehicle sensor system that can measure the three-dimensional response was developed, and vehicle vibration data were measured while driving on a bridge in Ibaraki Prefecture, Japan. This is the first paper to address the identification of the traveled sections on bridges in vehicle vibration. The accuracy of bridge labeling of segments traveling on bridges was compared based on vehicle vibration data, location information obtained from Google Maps, and location information measured by a GPS receiver grounded on the bridge. In this study, the peaks generated at the joints before and after the bridge in the vehicle vibration were used; supervised learning with C-LSTM networks was used to perform binary classification, and it was possible to estimate the driving section on the bridge with high accuracy for each model of the front and rear wheels. However, the trained model has a problem in that it responds to vehicle vibration peaks other than joints. Future work includes extracting bridge vibration components from vehicle vibration and labeling them accordingly. This is possible if the axle positions of the front and rear wheels can be synchronized, and the effects of road surface irregularities can be eliminated, but this is challenging because of the effects of GPS receiver measurement errors and vehicle speed. Therefore, it is considered realistic to extend the model to multi-class classification and build a model with multiple labels, such as when driving on and off bridges, when entering bridges, and when exiting bridges. Unsupervised learning is also expected to be used to build models that do not fit well only on sections on the bridge.
Additionally, this study did not attempt to compare the accuracy of the prediction models. Since various C-LSTM models and related models have been proposed in previous studies, comparing them is also a future task. The mechanical relationship of the method to the joints and bridge structures has not been clarified. Numerical simulation is useful to verify this. The proposed model was constructed for the front and rear wheel sections. However, heavy vehicles often have more than just front and rear wheels. Therefore, it is necessary to examine whether a model with the same accuracy can be constructed using a vehicle model other than the one considered in this paper.