BOPVis: Bridge Monitoring Data Visualization for Operational Performance Mining

Abstract

Bridges are fundamental facilities in the transportation system, and their operational performance is crucial for economic and social development. Many large bridges are now equipped with structural health monitoring (SHM) systems that collect various types of real-time data. However, our user study found that despite the accumulation of massive amounts of monitoring data, current analysis methods cannot efficiently process large-scale, high-dimensional data. To address this, we have developed BOPVis, a visualization system for bridge monitoring data. BOPVis allows users to intuitively locate sensors and extract corresponding data from a 3D digital model of a bridge. It also provides convenient and flexible interactions for examining trends over time and correlations across hundreds of monitoring channels. A real-world long-span suspension bridge in China is used as a case study to demonstrate the advantages of the BOPVis system for operational performance mining. Through BOPVis, the global temperature deformation behaviors of the bridge are explored and found to align with the physical mechanism documented in the SHM literature. The BOPVis system, with its interactive visualization analysis capabilities, offers a new method for analyzing bridge monitoring data.

1. Introduction

The safe and reliable operation of bridges is crucial for the development of the economy and society. The routine environmental actions and operational loads as well as the extreme events such as typhoons and earthquakes may cause unfavourable effects on bridges and shorten their service life [1]. Over the past three decades, an increasing number of bridges have been equipped with structural health monitoring (SHM) systems to assess their condition. These systems use advanced sensing technologies to continuously monitor the loads acting on the bridges, as well as their dynamic and static responses [2,3,4,5,6,7]. SHM systems help bridge engineers promptly detect structural damage or degradation, accurately predict the remaining lifespan of the structure, and make informed decisions about inspections and maintenance [8].

Since SHM systems collect data continuously and in real time, they accumulate a massive amount of monitoring data. However, by observing the workflows of bridge engineers and interviewing them, we found that traditional methods of analyzing monitoring data are cumbersome, monotonous, and inefficient. These methods struggle to effectively extract valuable information from the vast, high-dimensional data, resulting in much of the monitoring data remaining underutilized.

In this paper, a bridge monitoring data visualization system for operational performance mining, called “BOPVis”, is proposed. This system intuitively locates sensors and extracts corresponding data from a 3D digital model of the bridge. It allows users to easily view trends and correlations over time across hundreds of monitoring channels through flexible interactions. By incorporating human-computer interaction technologies, BOPVis integrates bridge engineering expertise into the data mining process. This enables engineers to analyze data more intuitively and efficiently, and to better understand the bridge’s behavior during operation [9]. The advantages of BOPVis are demonstrated using a real-world example of a suspension bridge in China. The main contributions include:

• A complete interactive visualization system called BOPVis is developed based on user studies, which greatly improves the efficiency of bridge engineers in analyzing bridge monitoring data.
• A real-world long-span suspension bridge in China is used as a case study to demonstrate the advantages of the BOPVis system for operational performance mining. Using BOPVis, we explore the global deformation patterns and dominant factors, which conform to the physical mechanisms of suspension bridges documented in the SHM literature.

The rest of the paper is organized as follows. Section 2 reviews related work while Section 3 covers the user studies. Section 4 describes the bridge monitoring data visualization system. Section 5 presents two case studies illustrating the operational performance mining results using BOPVis. Finally, Section 6 concludes the paper.

2. Related Work

2.1. Structural Health Monitoring System

The structural condition of large-span bridges can be significantly affected by various internal and external factors over time, such as material degradation, temperature fluctuations, wind speeds, and vehicle loads. Ensuring the healthy operation of bridges has been a hot and challenging issue in civil engineering [10,11,12,13]. With advancements in data acquisition, signal processing, data management and analysis techniques, a large number of systems have been deployed on bridges around the world, such as the Tsing Ma Bridge in Hong Kong, China [14], the Wuhan Yangtze River Bridge, China [15], the Z-24 Bridge in Switzerland [16] and the Seohae Bridge in Korea [17]. These systems generate large-scale, heterogeneous SHM data that exhibit clear big data characteristics [18].

After long-term operation, these monitoring systems have accumulated vast amounts of data, encompassing environmental factors, operational loads, and structural responses. For example, the SHM system on the Jiubao Bridge in Hangzhou can monitor the bridge’s displacement and external environmental data in real time. Preliminary results from this system reveal the relationship between structural temperature and the deformation of the main girder [19]. However, compared to the hardware, the software component of these monitoring systems has received insufficient attention. Most current SHM systems have relatively simple data analysis functions, making it difficult to efficiently process large volumes of high-dimensional data. This limitation has become a bottleneck restricting the practical application of SHM technology.

This paper employs visual analytics of big data technology to develop the BOPVis system, which integrates human cognitive abilities into the data analysis process and improves the efficiency of traditional data analysis methods.

2.2. Visualization Systems for Knowledge Discovery

There are various visualization systems for knowledge discovery in many fields, like business, sports, medicine, agriculture, industry and finance [20]. According to knowledge categories, knowledge discovery methods can be divided into generalization, classification and clustering, association, prediction and deviation. Based on the characteristics of bridge monitoring data and bridge engineering analysts’ behavior, our visualization systems for knowledge discovery focus on user studies and user behavior, time-series data analysis and correlation analysis.

In terms of user studies and user behavior, Xie et al. collaborate with domain experts and characterize requirements to analyze the dynamic changes in a team’s passing tactics, especially propose a glyph-based design to reveal the multi-variate information of passing tactics within different phases of attacks [21]. In this paper, we also invite domain experts for user studies to identify user needs and propose a visualization system to boost efficiency based on user behaviors.

In terms of time-series data analysis, Leite et al. propose a visual analytics approach combining different temporal aspects of COVID-19 data with the output of a predictive model [22]. Zhang et al. propose a visualization tool for temporal event sequences with multidimensional, interrelated data [23]. In terms of correlation analysis, for industrial alarm data, Yang et al. use a correlation color map of the transformed or pseudo data to show clusters of correlated variables [24]. A longstanding hypothesis that people use visual features in a chart as a proxy for statistical measures like correlation is investigated [25]. In this paper, we use the above conclusions and experiences for the correlation analysis of bridge monitoring data.

3. User Study

We conducted a user study to identify the bridge engineering analysts’ pain points. The bridge engineering analysts were invited to the field of bridge structure monitoring for the interview. The user interview outline mainly included three parts: (1) workflow, (2) bridge monitoring data processing and analysis tasks, and (3) user needs and expectations for the BOPVis system. Specifically, the questions include:

• Workflow:

  -

  What is your current workflow?

  -

  What tools or software do you use now?

  -

  What specific functions can this tool or software achieve?

• Bridge monitoring data processing and analysis tasks:

  -

  How do you get the data and what is the data format?

  -

  How is the data processing carried out?

  -

  What is the task of data analysis and how is it accomplished?

• User needs and expectations for the BOPVis system:

  -

  What do you think are the advantages of the current workflow?

  -

  What challenges do you see in the workflow?

  -

  What tasks does the BOPVis system need to help you accomplish?

In order to obtain the authentic and original working behavior of the bridge engineering analysts, the workflow was obtained by the on-the-spot observation of their working process. After sorting out the workflow, a semi-structured in-depth interview [26,27] was carried out in this user study. Since the questions are open and subjective, semi-structured interviews mean that the interviewer can adjust the interview content according to the specific situation of the interviewees’ answers during the interview.

We conducted interviews with three experts in bridge monitoring data analysis at CCCC Highway Consultants Co., Ltd., which is headquartered in Beijing, China. By setting open questions to a certain extent, the interviewer can dig deeply into the real thoughts of the interviewees. Combined with the workflow observation and the interviews, we conclude the bridge engineering analysts’ pain points presented as a storyboard shown in Figure 1.

The SHM system collects data from every sensor channel and exports it into binary or text files. To extract operational performance insights from this vast amount of data, the pain points of bridge engineering analysts are summarized below:

• Inconvenient sensor localization. Bridge engineering analysts need to select specific channels for analysis. This process requires repeatedly searching through numerous Excel tables, bridge drawings, and channel instruction documents to locate the relevant data. This is particularly time-consuming and inconvenient, especially for less experienced analysts.
• Repetitive work in knowledge discovery process. Once the data to be analyzed are located, analysts plot graphs to observe trends over time and channel correlations. Although there are a limited number of commonly used charts, such as time-series plots and correlation matrix graphs, these must be regenerated frequently due to the hundreds of channels. Creating these static charts in Excel or MATLAB is time-consuming and lacks flexible interaction.
• Cumbersome data processing. The graphs generated may not always present useful results, often due to abnormal data. Consequently, analysts must process the data and repeat the knowledge discovery process. However, data processing in Excel is cumbersome and difficult, adding to the challenges faced by bridge engineering analysts.

  -

  Manual data cleaning. The original data often contains many obvious abnormal data points caused by sensor damage and other factors. Marking and removing these abnormal points cannot be carried out on the static charts and must be performed manually.

  -

  Data preprocessing. Preparing data for analysis requires engineers to write their own code to build models in MATLAB. This process is labor-intensive and adds to the workload. Developing simpler scripts specifically designed for bridge operational performance management could greatly reduce the repetitive effort involved in data preprocessing.

4. Bridge Monitoring Data Visualization System

Based on the bridge engineering analysts’ pain points obtained by the above user study, we developed a bridge monitoring data visualization system. The technical framework of the system is shown in Figure 2. The system inputs include the sensor location information and the data collected from each channel. The data cleaning and preprocessing are performed on the server side, encompassing tasks such as reading, sorting, identifying abnormalities, validating data, and selecting relevant data. On the client side, users can select monitoring data channels and observe various types of visualization charts for knowledge mining. These visualization charts are used to discover abnormal points, present time-series data, show correlations between different channels, etc.

In this bridge monitoring data visualization system, the bridge engineering analysts can mine the knowledge of the operational performance by flexible user interaction. The interface of the BOPVis system, shown in Figure 3, includes three functional areas: selection of monitoring spots, monitoring data details, and the correlation panel.

Selection of monitoring data channels. Users can select monitoring data channels directly from the 3D bridge digital model, eliminating the need to search through Excel tables, bridge drawings, and channel instruction documents. The top left panel displays sensors with channel information on a 3D bridge model to facilitate sensor localization. Users can view the bridge from various angles by moving the mouse and zooming in and out by scrolling. In the upper left corner of the 3D model panel, users can select a specific time. Clicking a monitoring point on the bridge pops up a detail panel displaying the position, type of sensor, and detailed data at that moment.

Monitoring data details. The bottom left panel shows monitoring data details in time-series plots, including four types of data: GPS, ambient temperature (TMP), relative humidity (RH), and wind speeds (WS). Users can switch data types by clicking the corresponding TAB button and choose time intervals using the time bar at the bottom of the chart. A plus sign in the upper left corner of the panel allows users to enlarge it to the whole page for a clearer view. Selecting curves in the time-series plot automatically highlights the corresponding measurement points in the bridge digital model.

Correlation. The correlation panel includes a correlation matrix graph and a parallel coordinate plot to present channel correlations. Users can enlarge this panel for further interactions by clicking the plus sign in the upper left corner.

Data cleaning on the server. Obvious abnormal data points, which differ significantly from normal data, are identified through data observation. These can be classified as abnormal using the k-means clustering algorithm and set to null values. The empty values are then filled using linear interpolation of adjacent non-missing values.

Data preprocessing on the server. Users need to observe trends and correlations of bridge data over long periods, such as a quarter, a year, or more. The data collected from sensors is extensive; for instance, temperature data from one location can reach 36,000 points in one hour. To avoid redundancy, the system averages the daily data when users look at yearly data. It also provides scripts for users to process data at different granularities. Users analyze monitoring data through automatically generated graphs in the monitoring data details and correlation panels. These easy-to-use scripts provided by BOPVis help process raw data on the server side, significantly reducing manual effort and improving efficiency.

5. Case Studies

5.1. Dataset

In the case studies, a real-world suspension bridge in China is used. A suspension bridge usually consists of five components: the stiffening girders, main cables, main towers, anchorage blocks, and suspenders, as shown in Figure 4. The self-weight of the girder and the traffic load on it are transmitted through the vertical suspenders to the main cables, which are suspended between the two upright towers and are finally anchored to the anchorage blocks at each end of the bridge. Although suspension bridges have the largest spanning ability among all types of bridges, they are quite flexible systems and thus vulnerable to excessive deformation under various loadings and actions during the operational stages. Therefore, the global deformation of suspension bridges is one of the major concerns of bridge owners.

In the following two cases, the global deformation pattern of suspension bridges and the dominant factor of operational deformation are explored through our BOPVis system. The considered bridge is a two-tower suspension bridge with the main span, i.e., the horizontal distance between both towers, being over 1.5 km. There are more than 200 sensors deployed on the bridge, including GPS rovers to measure the deformation, temperature sensors, humidity sensors, and anemometers to measure wind speeds at different points on the bridge.

The bridge’s global deformation is measured by 13 GPS rovers with a reference station. Each GPS rover records three translational movements in three orthogonal directions, i.e., X-, Y-, and Z-axis. The X-axis is along the bridge axis from west to east (longitudinal direction), the Y-axis is perpendicular to the bridge axis from south to north (lateral direction), and the Z-axis is along the vertical direction from bottom to top. There are a total of 39 data channels related to structural displacements.

5.2. Global Deformation Pattern of Suspension Bridges

Users may use the scripts to obtain the 1-h averages of structural displacements over one year, as illustrated in the bottom left panel of Figure 3. Clearly, the yearly variations in various displacements are of different ranges. For example, the mid-span vertical displacements of the main cables (GPS 2-Z and GPS 3-Z) are approximately one order of magnitude larger than the other displacements.

To examine the trend of GPS5-X, GPS1-X, GPS6-X, GPS7-X, GPS4-X and GPS8-Y, we remove the GPS 2-Z and GPS 3-Z curves from the time-series plot to narrow the vertical axis range to between −0.15 and 0.15 m. This adjustment highlights the tower-top horizontal displacements along the bridge axis (GPS 1-X and GPS 4-X). As shown in Figure 5, the GPS 1-X and GPS 4-X curves stand out. Both curves evolve at the same pace yet out of phase, which means the two towers deflect in the opposite directions under normal operational conditions.

Further, the scatter plots in the top right panel, as shown in Figure 3, are used to explore the correlation between different variables in the time-series plot. Despite the great difference in variation ranges, the mid-span vertical displacements (GPS2-Z and GPS3-Z) exhibit a significant linear correlation with the tower-top horizontal displacements (GPS1-X and GPS4-X). The absolute value of the Pearson correlation coefficient for GPS2(3)-Z versus GPS1(4)-X is quite close to 1.0. Similar to scatter plots, the parallel coordinate plot in the bottom right corner can also be used to investigate the relationships between various variables.

When a data point in any scatter plot is selected, the corresponding data points at the same time instant in the other plots are highlighted automatically. For example, in Figure 6, the green dots represent a deformation state of the bridge (designated as State A), while the red dots represent another state (State B). This feature allows for a clear interpretation of the bridge’s transition between different operational states.

The physical explanation for the observed phenomenon is attributed to the overall deformation compatibility of suspension bridges, as illustrated in Figure 4. The two tower tops moving close to each other (i.e., transitioning from State A to State B) is always accompanied by a decrease in the mid-span elevation of the main girder, and vice versa. This deformation pattern revealed in BOPVis, aligns with the deformation mechanisms of real suspension bridges [28], thereby verifying the reliability of the monitoring datasets.

5.3. Cause of the Operational Deformation

The dominant factors driving the operational deformation of the background bridge are further investigated. For a long-span highway bridges, the traffic load is much smaller than the dead load. Additionally, the hourly averaging can filter out the traffic-related high-frequency components in the measured displacements. Consequently, traffic measurements are not considered in interpreting the bridge’s deformation. Instead, structural temperature, ambient temperature, relative humidity, and wind speeds are incorporated into the analysis as explanatory variables.

The bridge has 32 temperature sensors to measure the steel girder’s temperature distribution. Four top-plate temperature sensors at two cross-sections of the girder, namely TMP1, TMP2, TMP3, and TMP4, are selected. By clicking the plus sign in the upper left corner of the correlation panel, the scatter plot matrix is used to display all pairwise relationships between these four temperature variables, as shown in Figure 7. The correlation coefficients are all greater than 0.97, indicating a high level of uniformity in structural temperatures due to the steel’s excellent heat transfer performance. This uniformity is also evident in the time-series plot Figure 8.

As a result, the average of all 32 temperature measurements is selected as the representative structural temperature, denoted as TS. Similarly, the representative variables of the ambient temperature, wind speeds, and relative humidity are determined as TA, WS, and RH, respectively. To explore the cause of the operational deformation, the vertical- and horizontal-axis variables in the scatter plot matrix correspond to the structural displacements (GPS 1-X, GPS 2-Z, GPS 3-Z, and GPS 4-X) and environmental actions (TS, TA, WS, and RH), respectively, as shown in Figure 9. When the “fitted lines” option is enabled, the best-fitted lines overlay the scatter plot where the absolute values of correlation coefficients are greater than 0.8. These lines serve as a guide to the tendency of variations in structural displacements with environmental actions. Based on the distribution patterns of scattered points in Figure 9, the correlations for each pairwise variable combination can be intuitively compared. Moreover, the correlation coefficients in each plot are mapped to colors in the colormap, determining the scattered points’ color. Obviously, the correlations of the structural displacements with TS and TA are much higher than those with WS and RH, suggesting that temperature changes primarily govern the concerned structural deformations.

When the temperature increases, the midpoint of the main girder can be observed to move downward and simultaneously, both tower tops move toward the central span. A linear regression analysis shows that the sensitivity coefficients of GPS1-X, GPS2-Z, GPS3-Z and GPS4-X with regard to TS are, respectively, 5.30 mm/°C, −41.1 mm/°C, −44.2 mm/°C, and −4.2 mm/°C, indicating that the girder vertical displacement is much larger than the tower-top horizontal displacement for the same temperature changes. The above findings in BOPVis are a good agreement with the previous results in the bridge engineering community. According to [29], the temperature-induced deformation of a suspension bridge can be estimated by the formula $L_E{\theta }_C\times \delta T_C$, where $L_E$, ${\theta }_C$ and $T_C$ are the equivalent length, linear expansion coefficient and temperature variation. The equivalent length $L_E$ varies for different displacements. For the background bridge, the calculated displacements per unit temperature increase ($\delta T_C$ = 1 °C) are listed in

Table 1. Sensitivity coefficients of structural displacements regarding temperature (mm/°C).

Items	Formula in [29]	BOPVis
Horizontal displacement of the west tower top (GPS 1-X)	6.15	5.30
Vertical displacement of the main cable at mid-span (upstream side, GPS 2-Z)	−58.1	−41.1
Vertical displacement of the main cable at mid-span (downstream side, GPS 3-Z)	−58.1	−44.2
Horizontal displacement of the east tower top (GPS 4-X)	−6.26	−4.20

. The temperature sensitivity coefficients of structural displacements fitted by BOPVis have the same sign and order of magnitude as the theoretical solutions, indicating that BOPVis has successfully identified the patterns within the data.

In addition to the vertical displacement variables, the variation patterns of the lateral displacement of suspension bridges can also be explored in BOPVis. Figure 10 replaces the vertical-axis variables in Figure 9 with GPS5-Y, GPS6-Y, GPS7-Y, and GPS8-Y. The resulting scatter plot matrix indicates that the temperature change is not a dominant factor influencing the bridge lateral movements. In short, BOPVis can provide bridge engineers with valuable insights into the operational deformation of suspension bridges through interactive visualization technology.

6. Conclusions

This paper proposes BOPVis, a bridge monitoring data visualization system designed for operational performance mining. By integrating human cognitive abilities, which computers are not proficient at, into the data analysis process, BOPVis streamlines the analysis process and enhances the efficiency of analyzing large-scale, high-dimensional monitoring data. BOPVis enables users to intuitively locate sensors and extract corresponding data from a 3D digital model of the bridge. Users can examine trends over time using time-series plots and analyze correlations across hundreds of monitoring channels through flexible interactions. To demonstrate the advantages of BOPVis, a real-world suspension bridge in China is used as an example. Using BOPVis, the global temperature deformation behaviors of the bridge are explored and found to align with theoretical solutions.

In future work, BOPVis will be enhanced with advanced features, including more sophisticated time-series forecasting models, such as Long Short-Term Memory (LSTM) networks, and the integration of weigh-in-motion (WIM) data for multi-source data fusion and predictive analytics. Additionally, BOPVis will be applied to other types of bridges, addressing compatibility, data transmission protocols, and interoperability to ensure seamless integration into existing SHM systems.