Drone-Based Vibration Monitoring and Assessment of Structures

Abstract

This paper presents a novel method of procuring and processing data for the assessment of civil structures via vibration monitoring. This includes the development of a custom sensor package designed to minimize the size/weight while being fully self-sufficient (i.e., not relying on external power). The developed package is delivered to the structure utilizing a customized Unmanned Aircraft System (UAS), otherwise known as a drone. The sensor package features an electropermanent magnet for securing it to the civil structure while a second magnet is used to secure the package to the drone during flight. The novel B-Spline Impulse Response Function (BIRF) technique was utilized to extract the Dynamic Signature Response (DSR) from the data collected by the sensor package. Experimental results are presented to validate this method and show the feasibility of deploying the sensor package on structures and collecting data valuable for Structural Health Monitoring (SHM) data processing. The advantages and limitations of the proposed techniques are discussed, and recommendations for further developments are made.

1. Introduction

This work presents an innovative process by which the health status of difficult-to-access structures (e.g., bridges) can be determined, utilizing a wireless sensor package for data acquisition, deployed by an Unmanned Aircraft System (UAS) (commonly known as a drone), which provides data for Dynamic Signature Response (DSR) techniques for damage detection. This sensor technology is a compact and low-cost sensor package with an onboard power supply that allows wireless acceleration data collection, eliminating the need for wired connection to the power grid. The sensor package is equipped with an electropermanent magnet that facilitates the rapid installation to various points on the underside of a metal structure or any structure outfitted with metal plates. The sensor package is deployed and retrieved using a drone designed to operate under structures and deliver the packages safely to regions that may not otherwise be readily accessible (such as the underside of bridges that cross rivers or busy highways). The data collected by the sensor package is then analyzed with damage detection algorithms that process the vibration data and extract the DSR, which depends only on the structural condition. Based on the correlation between data collected at a prior, known, and healthy, state and the current state, Level 1 damage detection (the presence of damage in the structure) can be determined [1].

This paper is organized as follows: Section 2 provides the relevant background on structural health assessment using drones and vibration-based sensing algorithms. In Section 3, we present the development of the novel sensor package, the UAS for sensor deployment, and the data processing algorithm for the structural vibration data collected. The experimental validation of the proposed system along with experimental results are presented and discussed in Section 4. Finally, in Section 5, we conclude the paper and discuss future work.

2. Background

2.1. Accelerometer-Based Sensing

Accelerometers have countless uses, from flipping the screen orientation on smartphones to keeping planes on their trajectory in flight. Accelerometers also play a crucial role in Structural Health Monitoring (SHM). Researchers have created many methods of determining the condition of a structure using the vibration signature [2]. However, hardware implementations of such methods are limited due to multiple factors including size, cost, and power requirements. The high-sensitivity, low-noise electronics required to make such measurements are relatively large, expensive, and require experienced personnel to operate [2]. Attempts to make more compact systems and increase the resolution and sensitivity are included in the references of this work [3,4]. However, these compact systems can be labor-intensive and dangerous to deploy in remote areas. Such areas also make it impossible to connect these systems to the power grid, creating a need for wireless and power-independent sensor packages.

Accelerometers are electromechanical devices that detect the change in acceleration in a given direction. Piezoelectric (PE) and Integrated Electronics Piezoelectric (IEPE) accelerometers use internal crystals and the piezoelectric effect to generate a charge related to the acceleration introduced to the system. There are also piezoresistive accelerometers that vary their resistance in relation to an input acceleration. The third common type of accelerometers are the Capacitive Microelectromechanical Systems (MEMSs). These accelerometers have a suspended microstructure, embedded in the Integrated Circuit (IC), with a small mass that is free to vibrate and measure the change of capacitance of the microstructure due to acceleration [5]. DC coupling of capacitive MEMS accelerometers allows for the detection of low-frequency signals, which is critical when measuring the vibration signature of large structures, which typically resonate at frequencies below 1 Hz. Furthermore, MEMSs are capable of detecting steady-state acceleration [6,7] in addition to allowing the integration and double integration of their outputs to determine velocity and displacement, which is important for quantifying the stresses and loads on structures. Figure 1 compares performance versus qualitative cost estimates for the three main types of accelerometers. For this work, a MEMS accelerometer was chosen for its reliability, affordability, and ease-of-use.

2.2. Unmanned Aircraft Systems in SHM

One of the major challenges in SHM is accessing remote areas or difficult-to-access segments of structures. UAS (drones) are increasingly used as a solution to this problem. Thus far, much of the research considering drones in SHM applications has focused on the use of onboard cameras for image acquisition [8,9]. The assessment is then performed by a qualified engineer through visual inspection of the images with or without the use of image processing software. Another approach is to use a drone to hold a reflector prism against the bridge for total station measurements instead of holding the prism manually [10]. In this case, the prism is affixed directly to the UAS in a configuration described as a helping hand [11]. Moreover, Unmanned Aerial Manipulators (UAMs) are a new type of drone that achieve higher precision by utilizing a gripper or multifingered end effector to provide additional degrees of freedom over helping hands [11].

Jimenez-Cano et al. [10] presented an application of drones for SHM in which a dynamic model and controller were developed to achieve contact between the manipulator arm and the underside of a structure. They proposed to use an ultrasound sensor, attached to the end effector, to assess the depth of cracks in the structure. During the experiments, the proposed aerial manipulator was able to maintain continuous contact with the structure, which is necessary for the assessment; however, the experiments were performed without the actual ultrasound sensor. In another study by Bodie et al. [12], a contact-based UAS measured the electrical potential difference between materials in a sample of concrete as a method of nondestructive testing.

In addition to these contact-based methods, there is ongoing research into the deployment and retrieval of sensors by drones. For SHM applications, these sensors can be any of the variety of options considered in the previous subsection. For example in [13], a drone was developed with a hooked gripper able to attach a dummy package with a preprepared adhesive to a target by using the arm to apply pressure to the package against the desired surface. The package was then removed using a loop attached to the package, with which the hook can engage, and then, reverse thrust was used to remove it from the surface. Closer to this work, drones were attached directly to civil structures using electropermanent magnets for the collection of vibration data from civil structures [14]. In this case, an electropermanent magnet was utilized to maintain contact between the drone and the underside of the structure until the onboard systems have completed the data collection. However, the sensor package was integrated with the UAS and not treated as a separate system for deployment.

2.3. Vibration-Based SHM

SHM refers to activities used to diagnose structures over time. Structural damage can be detected by investigating the dynamic properties of the structure. The change in the modal parameters (e.g., frequencies, mode shapes, and modal damping) indicates the change of the physical properties of the structure (e.g., mass, damping, stiffness) [15]. Methods for SHM can be divided into local and global methods; local methods are used in locations where damage is expected, while global methods are used to detect damage through monitoring changes in the structure’s dynamic fingerprint [16]. Vibration-based damage detection methods are classified into four levels based on what the method is able to do [17]:

• Level 1: indicates that damage is present;
• Level 2: identifies the location of the damage;
• Level 3: quantifies the severity of the damage;
• Level 4: finds the remaining lifetime of the structure.

Research developments in early damage detection focus on model-based and data-based methods. Model-based methods assume that a detailed numerical model of the structure is available, and the model parameters are updated based on measurements obtained from the physical structure [15,18]. Data-driven methods use in situ monitoring to identify changes in the structure’s response characteristics and correlate those changes to the location (Level 2) and severity (Level 3) of damage [18,19,20]. Contactless techniques are also being developed to capture the pure dynamic response of the structure without added masses or stiffening due to the attached sensors [21].

In this work, a data-driven method is investigated to detect Level 1 damage using time-domain responses. The B-Spline Response Function (BIRF) technique [22,23,24,25] was implemented to extract the DSR of the structure from experimental vibration measurements. The DSR is a unique characteristic of the structure and will remain the same if the geometry, materials, and boundary conditions of the structure are not altered. The DSR is independent of the actual loads that caused the vibration. Therefore, the DSR represents the current state and condition of a structure. Comparing the DSR before and after damage can be used for damage detection.

3. System Development

The goal of this work was to develop a system that can identify structural damage using vibration data collected by wireless sensor packages deployed with drones. This section presents the three main elements of this system: sensor package, drone, and method for damage identification.

3.1. Sensor Package

The sensor package consisted of three components: (i) electronics for data gathering, storage, and wireless control, (ii) a docking mechanism based on an electropermanent magnet, and (iii) the power supply (battery). These components are presented in Figure 2 and Figure 3, while

Table 1. Summary of the sensor package features.

Sample Size	Rate of Data Collection	Resolution	Sensitivity	Steady-State Power	Package Weight
74,000 Samples	28 kHz	16-bit	5400 LSB/g	0.296 W	380 g

summarizes the main features of the sensor package. A compact 3D-printed frame was built to house these components, forming the standalone sensor package shown in Figure 4 (black box).

The sensor package features the ARM Cortex-M7 processor onboard the Teensy 4.0 high-performance microcontroller. The system utilizes the 600 MHz clock speed of the Teensy 4.0 and the Serial Communication Interface (SPI) to communicate between the modules and achieve a data collection rate up to 28 kHz. The sensor package also includes three control options (Figure 2): a PWM port used for wireless control, a digital 5 V as a hard-wired trigger, and manual controls for testing and calibration purposes.

Power is supplied using a DC-DC buck converter (MP1584EN manufactured by Monolithic Power Systems) and a lithium polymer battery (Lumenier 7.4V 1000 mAh), as shown in Figure 3. This battery currently runs the sensor package for about 11 h on a single charge. The deployment period can be prolonged by utilizing standby mode where unused modules are turned off to conserve energy. In cold weather though, testing has shown that batteries tend to discharge relatively faster, in turn requiring a more restrictive power conservation schedule, limiting the number of tests conducted per deployment.

The docking mechanism consists of an electropermanent magnet (NicaDrone EPM V3 R5C) able to mount the sensor package onto metal surfaces (Electropermanent Magnet #1 in Figure 4). The package collects acceleration data using a MEMS accelerometer (Murata SCA3300-d01) mounted to the metal frame of the electropermanent magnet to ensure contact with the structure being examined. The system can collect up to 74,000 samples as one continuous dataset, before having to transfer the data onto the onboard nonvolatile memory (SD card), a process that takes approximately 3.288 s.

3.2. UAS Platform

The standalone sensor package does not require connection to the power grid, so it can be deployed to any structure. Traditional (human operator) deployment and retrieval though may be challenging in remote locations. For example, in remote bridges, a lift from the ground is required to reach the underside (which may be impractical if the bridge crosses a body of water or a busy highway) or a crane on the topside of the bridge, which can position the operator underneath the structure. Both methods are risky and require costly equipment and roads to be shut down to make the underside of the structure accessible. A drone platform has been developed to overcome these limitations and securely deploy and retrieve sensor packages in difficult-to-access areas.

The drone platform was based on the DJI F550, a hexacopter with six standard 2312E motors, shown in Figure 5 [26]. Pixhawk 4 was used as the flight controller, and the system was operated via the ArduPilot Mission Planner. Onboard sensors included an IMU, accelerometers, gyroscope, magnetometer, and barometer to provide data for a variety of parameters including drone orientation and altitude. There was also an external GPS used for global positioning when the signal was available. The FrSky X8R radio was used as the onboard receiver to communicate with the FrSky 2.4 GHz Taranis X9D radio controller for manual flight commands. The sensor package, as previously described, was attached to the UAS via a second electropermanent magnet (#2 in Figure 4), which was controlled using one of the RC servo ports available on the power management board of the Pixhawk 4.

The system was designed to work as follows: On the ArduPilot Mission Planner, two channels from the FrSky Taranis were set to send high, low, and neutral commands to the two magnets. During the high command, the magnet begins the charging sequence, which can only be completed once it is in contact with a ferromagnetic surface. Once the magnet is mounted on the metallic surface, this command can be shifted to neutral as the electropermanent magnet will remain magnetized until the low command is sent. When the low command is sent, it will disengage the magnet.

During operations, the sensor package is delivered onboard the drone (Magnet #2 engaged) to the desired point on the structure where Magnet #1 arms and connects the sensor package and drone to the structure. Once the package is secured in place, Magnet #2 disengages, and the drone detaches and flies away. The sensor package then periodically collects and stores acceleration data for DSR processing. When this operation is complete, the drone conducts a retrieval sequence by reversing the docking procedure. A summary of the status of both magnets in each phase of operations is given in

Table 2. Operation status for Electropermanent Magnets #1 and #2 for each sensor package and UAS position.

Electropermanent Magnet #1	Electropermanent Magnet #2	Sensor Package	UAS
Unmagnetized	Magnetized	Attached to UAS	At delivery
Magnetized	Unmagnetized	Attached to structure	Package deployed

.

To assist with the retrieval of the sensor package, the UAS was outfitted with a 3D printed alignment tool (docking platform in Figure 4). The alignment tool features angled walls, which help to center the sensor package over the electropermanent magnet and prevent the sensor package from coming into contact with the propellers.

3.3. Method of Damage Identification

The Dynamic Signature Response (DSR) of a system is a characteristic response of the system and was first introduced by Rizos et al. [23] as the B-Spline Impulse Response Function (BIRF) for soil–structure interaction analysis. The BIRF is a time-varying function that is defined between any two points in a mechanical system. It represents the time history of the response of one of the two points due to a unit amplitude B-spline excitation applied to the other point [23,24,27,28]. The B-spline functions are piecewise smooth polynomials of order k and belong to a family of base functions used in data interpolation and approximation. They are defined with respect to a time knot sequence $t_n,n=1,2,\dots$, by the recursive formula,

$$B_n^k\left(t\right)=\left\{\begin{array}{cc}1,\hfill & \mathrm{if}{t}_n<t<t_{n+1}\hfill \\ 0,\hfill & \mathrm{elsewhere}\hfill \end{array}\right.k=1,$$

(1)

$$\begin{array}{c}\hfill B_n^k\left(t\right)=\left(\frac{t-t_n}{t_{n+k-1}-t_n}\right)B_n^{k-1}\left(t\right)+\left(\frac{t_{n+k}-t}{t_{n+k}-t_{n+1}}\right)B_{n+1}^{k-1}\left(t\right),k>1.\end{array}$$

(2)

The $B_n^k$-spline polynomials are of degree $k-1$ and have $k-2$ continuous derivatives. While the BIRFs are computed efficiently in numerical models of physical systems [23,28,29], it is almost impossible to measure the BIRFs directly in physical systems due to the difficulty of accurately reproducing the B-spline loading on the physical system. The BIRF between two points of a mechanical system, however, can be extracted from vibration tests that acquire the time history record of a response, R, and excitation, f, at the two points. For example, the fourth-order-based BIRF is computed in a discrete form at a time step N as:

$$BIR{F}_N=\left(R_N-\sum _{n=1}^{N-2}BIR{F}_n{f}_{N-n+2}+BIR{F}_{N-1}\left(f_3-f_1\right)\right)/\left(f_2+2f_1\right),$$

(3)

where subscripts indicate the time step at which the corresponding functions are evaluated. Details of the method along with verification and validation studies were presented in Alkharousi et al. [30,31].

Structural assessment procedures through dynamic testing focus on identifying changes in the dynamic properties of the system through modal analysis and/or model updating techniques. In Equation (3), the BIRFs are independent of any external excitations and are considered the DSRs that capture the current state of the dynamic system. Any changes in the current state of the structure are captured as changes in its DSRs. Therefore, if the response time history, R, and the corresponding excitation, f, are known through vibration test measurements conducted before and after a damage-inducing event, the difference in the corresponding DSR is an indication of damage [30,31].

The computation of the BIRF shown in Equation (3) represents in essence a discrete deconvolution of the response signal, and the BIRF can be perceived as an equivalent time domain transfer function of the system that captures all frequencies in the range defined by the sampling time interval. It is noted and had been demonstrated that the analysis of only a short duration segment of the recorded response is adequate for change detection through the DSR algorithm, compared to mainstream methods that require, in most cases, the entire response record from multiple sensors. However, the noise inherent to the time history response records may have to be removed through filtering before the DSR is extracted. Nevertheless, the computing demand of the proposed method is very low, minimizing the postprocessing times, leading potentially to damage detection in real time, and facilitating the integration to the damage detection algorithms in the overall proposed system.

4. Experimental Validation and Results

In order to validate the developed system (drone, sensor package, damage detection algorithm), we created a laboratory experimental testbed and designed experiments to collect and analyze data. The goal was to show the feasibility of the drone-based deployment and retrieval of our sensor package, as well as the extraction of damage information from the collected vibration data.

4.1. Test Apparatus

A test apparatus was constructed in the lab to replicate a bridge with a metallic surface, as shown in Figure 6. The structure was 6.5 ft tall and 6.5 ft wide (2 m × 2 m) with a 2 ft by 4 ft (60 cm × 120 cm) steel plate mounted in the center to serve as a docking platform. Figure 7 shows a schematic of this plate. An electromagnetic shaker (Ling Dynamic Systems V408) was permanently mounted on top of the plate and used to excite the structure during our experiments. The test apparatus was placed inside a drone cage where the OptiTrack Motion Capture system was used for rigid body tracking. This system can provide accurate pose data (position and orientation) for our drone flights.

4.2. Drone-Based Deployment Validation

In this experiment, we aimed to validate the use of the drone for the sensor package deployment. The experiment was designed as follows: The drone approaches the structure and conducts the docking sequence described in Section 3.2, then mounts the sensor package to the structure. After sufficient time for data collection has passed, the UAS approaches the structure again and reverses the docking sequence to retrieve the package. When the package has been retrieved, the entire system lands, and the test is complete. During this experiment, the electromagnetic shaker was set to excite the structure at 11.3 Hz while we collected acceleration data from the sensor package and displacement data from the OptiTrack Motion Capture system. Due to the underside of the structure being outside of the OptiTrack field of view, only the UAS position and orientation were tracked using the OptiTrack system.

The results are presented in Figure 8. The first 25 s of the test were used to coordinate the data collection and ensure that the electropermanent magnets were properly set before takeoff. Adjustments were performed to properly align the UAS with the structure before making contact at 38 s, at which point, the UAS maintained contact until the 48 s mark (area II) to ensure the engagement of the electropermanent magnet on the sensor package and the disengaging of the electropermanent magnet on the UAS. Once the magnets were set, the UAS left the sensor package on the structure from the 38 s mark until the 75 s mark (area III). During this time, the UAS hovered near the structure to allow the sensor package to perform data collection without interference from the hanging mass of the UAS.

After allowing sufficient time for data collection, the UAS began the retrieval process by making contact with the package at 75 s. Contact was again maintained for 10 s to confirm the change in the electropermanent magnet state (area IV). For retrieval, the magnet on the UAS was engaged to maintain hold of the sensor package during landing prior to the magnet on the sensor package disengaging, allowing the system to begin its descent. This descent was performed immediately after securing the sensor package, and the drone landed at 94 s, completing the flight test.

The acceleration of the sensor package presented in Figure 8 substantiated these movements showing intense vibration when the package was onboard the UAS and significantly smaller acceleration while in contact with a structure, indicating successful deployment. Experimental results showed the feasibility of the method, which can now be further developed [26].

4.3. Sensor Package Validation

This experiment aimed at validating the accuracy of acceleration data collected by the sensor package. For this purpose, the sensor package was manually mounted along with a PCB accelerometer onto the steel plate, which was excited by the electromagnetic shaker (as shown in Figure 6). Using a data acquisition module, the excitation signal was swept from 1–20 Hz over a test period of 50 s. Due to the limited sample size of the sensor package (74,000 samples), a sampling frequency of 1480 Hz was used as a result of the limited buffer size onboard the sensor package. The validation was conducted utilizing a frequency response function of the sensor package data over the reference PCB accelerometer, shown in Figure 9. The results indicated an exponentially increasing error as the frequency approached zero. At lower frequencies (<5 Hz), the acceleration signal was below the sensitivity of the sensor package (5400 LSB/g), resulting in the large error shown. This could be mitigated in future designs by utilizing a higher accuracy accelerometer and decreasing the damping caused by the sensor package frame.

4.4. Damage Detection Validation

This experiment was designed to validate the ability of the sensor package to collect meaningful data for SHM applications. Since the system was developed to detect Level 1 damage, the sensor package needed to collect data before and after the structure had sustained damage. To simulate damage, three bolts supporting the steel plate were removed, as shown in Figure 7. This experiment also utilized the electromagnetic shaker to generate harmonic excitation forces. Additionally, a load cell (Ling Dynamic Systems V408) was placed between the shaker and the plate to record the force transmitted to the structure to provide ground truth measurements. The sensor package was then manually mounted directly underneath the electromagnetic shaker and load cell, in the position a0 shown in Figure 7. The shaker was first actuated at 7 Hz for 4 s and then stopped to allow the structure to vibrate freely. This was repeated at 14 Hz for both the undamaged and damaged structure.

The collected data underwent preprocessing in which the data from each experiment were smoothed through a 50-point rolling average and resampled to a total of 200 points to be considered as the response, $R_N$, in Equation (3). The associated forcing function, $f_N$, in Equation (3) can be either the load cell record or the voltage signal that drives the linear actuator [31]. In this paper, the load cell record was utilized. Equation (3) can now be implemented on each record to extract the DSR of the structure. It has been shown that damage can be detected by processing even a short segment of the record [31], provided that the time step of the acquisition is adequate to capture the maximum desired frequency affecting the response.

Figure 10 shows a 400-step segment of the DSR of the undamaged structure. Each DSR was extracted from the acceleration time history sampled in each of the three tests with the 14 Hz excitation force. As expected, the three DSR signals were almost identical and showed a high correlation ($R^2$ = 0.989). This indicated that the state of the structure had not changed between acquisitions, and therefore, no damage was present. Figure 11 compares the DSR of the damaged and undamaged structure to the 14 Hz excitation force. These two BIRFs showed no correlation, indicating a change in the structural condition; thus, the presence of damage is detected.

In Figure 10 and Figure 11, the damage detection algorithm operated on a 0.8 s window of the free vibration structural response with an acquisition step of 2 × 10⁻³ s. This allowed for approximately 10 sampling points within a response cycle, and thus, this was the minimum acquisition step required to capture frequencies up to approximately 300 Hz. It is noted that the structural response record was resampled to 1/4th of the original sampling frequency. It is also noted that an even shorter segment of the structural response could potentially be used in the damage detection algorithm.

Table 3. Correlation coefficients between BIRFs of the undamaged and damaged structure for the 7 Hz and 14 Hz excitation frequencies.

			Before Damage						After Damage
			7 Hz			14 Hz			7 Hz			14 Hz
		Test	1	2	3	1	2	3	1	2	3	1	2	3
Before Damage	7 Hz	1	1.000	0.883	0.890	0.900	0.888	0.895	0.092	0.080	0.086	0.084	0.087	0.079
		2		1.000	0.891	0.909	0.898	0.905	0.098	0.086	0.090	0.083	0.087	0.086
		3			1.000	0.865	0.864	0.865	0.086	0.078	0.079	0.083	0.096	0.078
	14 Hz	1				1.000	0.989	0.991	0.097	0.095	0.094	0.087	0.091	0.087
		2					1.000	0.990	0.095	0.096	0.093	0.088	0.088	0.085
		3						1.000	0.099	0.098	0.096	0.092	0.088	0.081
After Damage	7 Hz	1							1.000	0.879	0.892	0.813	0.798	0.815
		2								1.000	0.906	0.812	0.809	0.839
		3									1.000	0.805	0.787	0.813
	14 Hz	1										1.000	0.921	0.927
		2											1.000	0.910
		3												1.000

summarizes the $R^2$ correlation coefficients of the BIRFs extracted from all tests for the two excitation frequencies on the undamaged and damaged structure. It was observed that all BIRFs correlated well ($R^2$ > 0.8) when extracted from responses of either the undamaged or damaged structure, for both frequencies. However, the correlation was completely lost ($R^2$ ≈ 0) when the BIRFs of the undamaged and damaged structures were compared, regardless of the excitation.

5. Conclusions

This paper presented an initial prototype and the associated methodologies for a sensor package that can be deployed via a drone onto civil structures for structural health monitoring. Alongside this demonstration, Level 1 damage detection (the presence of damage in the structure) was performed using a novel BIRF technique to extract the DSR from data collected by reference accelerometers. Through manual flights, the feasibility of delivering and retrieving a sensor package from the underside of a structure was proven. Similarly, experimental validation showed that the BIRF method is an effective method for determining damage based on data collected from any sensor package.

In future work, the landing mechanism of the UAS system will be fully automated, instead of relying on manual flight controls. The sensitivity of the sensor package will be improved, and wireless communication and data transfer will be added. The use of elastomers will be considered in an effort to reduce the effect of drone vibrations on the measurements. As for the damage detection algorithm, future work will focus on expanding the methodology to include damage quantification and damage localization, which relate to Level 2 and 3 damage detection, respectively.