Full-Field Modal Analysis Using Video Measurements and a Blind Source Separation Methodology ^†

Abstract

The adoption of wireless sensor networks has brought a significant breakthrough in structural health monitoring, providing an effective alternative to the challenges associated with traditional cable-based sensors. In recent years, a growing interest in developing contactless, vision-based vibration sensors like video cameras has led to advancements, potentially alleviating the previously mentioned drawbacks. In this study, a video of a vibrating frame is converted into a set of frames, so that local phase information can be extracted. The motion matrix is then derived from the phase information; since the number of measuring points is usually greater than the number of the excited modes of the system, the problem can become over-determined. Therefore, by applying dimensionality reduction techniques, the dimension of the motion matrix is significantly reduced. Finally, by exploiting an output-only identification technique, modal parameters are computed. The proposed approach is proven to accurately identify the structural frequencies and mode shapes.

1. Introduction

Modal analysis can be used to extract parameters that characterize the dynamic response of structures to external excitations [1]. This process can rely upon both input and output data or output data only. As the modal parameters depend on the inherent properties of the structure and are supposed to be unaffected by the external loads, they can be used for the assessment of the health of the monitored structures [2,3].

Typically, experimental and operational modal analyses involve the use of wired or wireless sensor networks, featuring optimized placement and/or cost [4,5,6], to observe the structural response [7,8]. Such sensors can provide measurements at sparse locations, often resulting in poor spatial resolution. In recent years, the capabilities of high-speed video measurements have significantly increased so that vision-based measurements can be effectively utilized in a wide range of applications in civil engineering, see, e.g., [9]. Estimating the optical flow by assuming a constant intensity between frames is a widely used technique; the equation imposing the invariant condition can be solved using different methods [10]. For instance, Chen et al. [11] identified the modes of cantilever beams and pipes by using phase-based motion magnification. Yang et al. [12,13] extracted the displacement of a three-story structure by combining phase-based optical flow and deep learning [14], while Dasari et al. [15,16] addressed the extraction of mode shapes and frequencies in the case of non-ideal rigid body motion. Martinez et al. [17] investigated the combination of compressed sensing sampling with system identification through images.

In the process of identification through video-based methods, due to the substantial dimensions of data, dimension reduction is necessary to reduce the dimensionality of the motion matrix to the number of excited modes. This issue represents one of the main challenges within the field of vision-based identification.

In this study, we attack the problem of dimensionality reduction in two ways: first, full-field identification is achieved by handling a few pixels in the images; second, the dimensionality of said matrix is reduced to match the number of excited modes and, simultaneously, denoise the information. After having extracted the phase and amplitude from all of the frames of a video by applying the Gabor filter, pixels are selected by means of two criteria: a customarily adopted one, based on the amplitude, and a newly proposed one, based on the amplitude coupled with a Canny edge detection method. A motion matrix representing the time history of the pixel phase is then extracted and, via non-negative matrix factorization (NMF), the dimension of the data is reduced. Finally, the dimension of the phase matrix is reduced, and by means of complexity pursuit (CP), the full-field mode shapes and the vibration frequencies are obtained. The accuracy of the results is assessed with an example related to a three-story building.

2. Phase Extraction with the Gabor Filter

Phase information has been proven to be more correlated with structural motion than the amplitude and intensity value of the image [11]. In Fourier analysis, time delay in a signal corresponds to phase variations in the frequency domain; similarly, in two-dimensional signals (images), the spatial motion results in a phase change. In this context, we refer to the local phase corresponding to a pixel in a specific coordinate system, which can be extracted using the Gabor filter. This filter type, which is a complex quadrature filter, is the product of the Gaussian kernel and a complex sinusoidal function; due to article length constraints, additional details are omitted herein and readers are referred to [18]. The real and imaginary part of this filter in two directions is represented in Figure 1.

Assuming that $\mathrm{I}(\mathrm{x},\mathrm{y},{\mathrm{t}}_0)$ is the intensity of the image at time ${\mathrm{t}}_0$ and at the location $(\mathrm{x},\mathrm{y})$ in the frame, the local phase (${\mathsf{\Phi }}_{\mathsf{\theta }}$) and local amplitude (${\mathrm{A}}_{\mathsf{\theta }}$) in the θ direction are obtained by convolving the filter $({\mathrm{G}}_{\mathsf{\theta }}+\mathrm{i}{\mathrm{H}}_{\mathsf{\theta }})$ with the image as:

$${\mathrm{A}}_{\mathsf{\theta }}(\mathrm{x},\mathrm{y},{\mathrm{t}}_0){\mathrm{e}}^{\mathrm{i}{\mathsf{\Phi }}_{\mathsf{\theta }}\left(\mathrm{x},\mathrm{y},{\mathrm{t}}_0\right)}=({\mathrm{G}}_{\mathsf{\theta }}+\mathrm{i}{\mathrm{H}}_{\mathsf{\theta }})\otimes \mathrm{I}(\mathrm{x},\mathrm{y},{\mathrm{t}}_0)$$

(1)

In the phase-based method, it is assumed that the local phase contour remains constant. The mathematical representation of this concept is given by:

$${\mathsf{\Phi }}_{\mathsf{\theta }}(\mathrm{x},\mathrm{y},{\mathrm{t}}_0)=\mathrm{C}$$

(2)

This equation can be used to determine the displacement field.

3. Full-Field Identification

By applying the Gabor filter to the frames of a video of a vibrating structure, the local phase $\mathsf{\Phi }(\mathrm{x},\mathrm{y},\mathrm{t})$ is obtained to capture the structural vibrations $\mathrm{d}(\mathrm{x},\mathrm{y},\mathrm{t})$ provided as a linear combination of modal responses:

$$\mathrm{d}\left(\mathrm{x},\mathrm{y},\mathrm{t}\right)=\sum _{\mathrm{i}=1}^{\mathrm{n}}{\mathsf{\phi }}_{\mathrm{i}}{\left(\mathrm{x},\mathrm{y}\right)\mathrm{q}}_{\mathrm{i}}\left(\mathrm{t}\right)$$

(3)

where n is the number of excited modes; $\mathsf{\varphi }\in {\mathrm{R}}^{\mathrm{N}\times \mathrm{n}}$ is the matrix of the mode shapes, with ${\mathsf{\phi }}_{\mathrm{i}}$ being the i-th vibration mode; N is the number of pixels in one frame; and $\mathrm{q}\left(\mathrm{t}\right)\in {\mathrm{R}}^{\mathrm{n}\times \mathrm{T}}$ is the vector of modal coordinates, with T being the number of handled frames.

Within blind source separation (BSS) methodologies aimed at modal identification, modal responses and vibration frequencies are extracted from the measured output without prior knowledge of the mixing matrix that represents the mode shapes. This is accomplished via the measured output by extracting phase information from the frames. Given that the number of image pixels is significantly higher than the number of typically excited modes, namely given that N >> n, the problem becomes characterized by a large amount of data leading to high computational costs. Consequently, the direct application of BSS methods is not feasible for modal identification; dimensionality reduction techniques must be employed to reduce the dimensionality of the time-series matrix.

3.1. Non-Negative Matrix Factorization

Non-negative matrix factorization was introduced in [15]. Assuming the matrix $\mathrm{V}\in {\mathrm{R}}^{\mathrm{n}\times \mathrm{m}}$ to comprise the phase time history of pixels (motion matrix), reduced to rank r, its non-negative factorization involves the two matrices $\mathrm{W}\in {\mathrm{R}}^{\mathrm{n}\times \mathrm{r}}$ and $\mathrm{H}\in {\mathrm{R}}^{\mathrm{r}\times \mathrm{m}}$ such that $\mathrm{V}\approx \mathrm{W}\mathrm{H}$. This is achieved by minimizing the Euclidean distance between V and WH, according to:

$$\min ‖\mathrm{V}-\mathrm{W}\mathrm{H}‖ \mathrm{s}\mathrm{u}\mathrm{b}\mathrm{j}\mathrm{e}\mathrm{c}\mathrm{t}\mathrm{e}\mathrm{d} \mathrm{t}\mathrm{o} \mathrm{W}\ge 0,\mathrm{H}\ge 0$$

(4)

NMF relies upon non-convex optimization during its iterative process, potentially resulting in different solutions based on the initialization values of $\mathrm{W}$ and $\mathrm{H}$. The initialization procedure adopted in this paper is as follows: the eigenvectors corresponding to the largest eigenvalues (larger than 1% of the maximum one), as a result of PCA on the pseudo inverse of the motion matrix, are retained in the analysis; $\mathrm{W}$ is obtained by applying ICA on the whitened matrix of step 1; $\mathrm{H}$ is obtained by means of the motion matrix. Not only can different initialization procedures to set $\mathrm{W}$ and $\mathrm{H}$ affect the accuracy of the identified modes but also the reduced dimension of the problem can be modified; this is a rather common challenge in vision-based identification problems. A comparison between the adopted method and others available in the literature is beyond the scope of this conference paper.

3.2. Complexity Pursuit

By applying NMF on the extracted phase matrix, the reduced one $\mathrm{W}$ can be obtained. Assuming that $\mathrm{W}\prime$, where the prime stands for transpose, can be decoupled into modal coordinate q according to:

$$\mathrm{q}=\mathrm{L}\mathrm{W}\prime$$

(5)

The de-mixing matrix ${\mathrm{L}}^{r\times r}$ has to be acquired through the use of CP [19]. The mode shape matrix, see Equation (3), is then computed with:

$$\mathsf{\varphi }=\mathrm{H}\prime {\mathrm{L}}^{-1}$$

(6)

4. Numerical Example

In this study, we assess the capability of the procedure outlined in Section 3, by means of an on-purpose generated video of a three-story structure with mass $m=[12 10 5]\times 10^3$ kg, stiffness $k=[45 40 23]\times 10^6$ N/m, and damping proportional to the mass, subject to free vibrations. The phase matrix is obtained by applying the Gabor filter to all of the frames. To effectively handle the data volume and to prove the feasibility of obtaining mode shapes using a reduced number of pixels, the procedure reads as follows: pixels are selected on the basis of the amplitude in the initial frame; Canny edge detection is employed in conjunction with the amplitude information; and NMF and CP are applied to the phase matrix. The entire procedure is sketched out in Figure 2.

As previously mentioned, one of the inherent challenges within vision-based identification methodology pertains to dimension reduction. At this stage, obtaining a matrix with dimensions matching the number of excited modes of a vibrating structure is not straightforward. To illustrate this issue, we refer to the approach employed in [20] for a three-story structure, whose motion to catch the vibrations can be affected by noise, see Figure 3a. A singular value decomposition (SVD) of the covariance matrix derived from the phase matrix is adopted. The dimension reduction is based on the selection of the eigenvalues exceeding 1% of the largest one, as depicted in Figure 3b. If noise is present, the number of eigenvalues exceeding this threshold turns out to be three.

Upon the adoption of CP for the phase matrix, the number of identified modes may be larger than the actual number of excited modes, thereby needing a post-processing step. While it is relatively straightforward to visually select the correct modes in structures with a simple geometry, it may become prohibitively difficult for structures characterized by a complex shape.

By applying the procedure illustrated in Figure 2 to the noisy video, the phase matrix is obtained using two distinct phase selection methods. The dimension of the phase matrix obtained by employing the Canny edge detection in combination with the amplitude information results in it being smaller than the one obtained by solely using the amplitude data. Figure 4 shows that the identified mode shapes are identical in the two cases.

In Figure 5, a comparison between the identified mode shapes extracted from the video and the mode shapes used to generate the video is provided. It is evident that the first and second modes closely align with the real mode shapes of the structure at story height. However, the third mode shows some disparities at the first story. Upon comparison with the actual values obtained using the modal assurance criterion, it can be observed that the identified vibration frequencies exhibit a high degree of similarity with the real ones, amounting to at least 98.2%.

5. Conclusions

This study has focused on the identification of the mode shapes and frequencies of vibrating structures. Given the substantial dimensionality of the data to handle such aspects in the analysis, we implemented a two-step pixel selection procedure by applying a threshold to the pixel amplitude obtained by way of the Canny edge detection method. Next, we applied NMF to reduce the dimensionality of the phase matrix and to also denoise the phase time-series. In fact, it has been demonstrated that noisy data can significantly impact the eigenvalue selection phase, potentially leading to the identification of incorrect mode shapes. Subsequently, CP was employed for the identification process.

The results show that this approach can be an accurate and reliable method to identify vibration frequencies and mode shapes, at least for problems related to the vibrations of shear-type buildings. The denoising procedure in fact resulted in the accurate identification of mode shapes. In future research, we aim to investigate the impact of the initialization method of NMF and to refine the process of selecting the rank for NMF dimension reduction.