Next Article in Journal
A Machine-Learning Approach to Measure the Anterior Cruciate Ligament Injury Risk in Female Basketball Players
Previous Article in Journal
A Review of RFID Sensors, the New Frontier of Internet of Things
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Subpixel Matching Using Double-Precision Gradient-Based Method for Digital Image Correlation

School of Civil Engineering, Chongqing University, No. 83 Shabei Street, Chongqing 400045, China
*
Author to whom correspondence should be addressed.
Sensors 2021, 21(9), 3140; https://doi.org/10.3390/s21093140
Submission received: 15 March 2021 / Revised: 4 April 2021 / Accepted: 28 April 2021 / Published: 30 April 2021
(This article belongs to the Section Intelligent Sensors)

Abstract

:
Digital image correlation (DIC) for displacement and strain measurement has flourished in recent years. There are integer pixel and subpixel matching steps to extract displacement from a series of images in the DIC approach, and identification accuracy mainly depends on the latter step. A subpixel displacement matching method, named the double-precision gradient-based algorithm (DPG), is proposed in this study. After, the integer pixel displacement is identified using the coarse-fine search algorithm. In order to improve the accuracy and anti-noise capability in the subpixel extraction step, the traditional gradient-based method is used to analyze the data on the speckle patterns using the computer, and the influence of noise is considered. These two nearest integer pixels in one direction are both utilized as an interpolation center. Then, two subpixel displacements are extracted by the five-point bicubic spline interpolation algorithm using these two interpolation centers. A novel combination coefficient considering contaminated noises is presented to merge these two subpixel displacements to obtain the final identification displacement. Results from a simulated speckle pattern and a painted beam bending test show that the accuracy of the proposed method can be improved by four times that of the traditional gradient-based method that reaches the same high accuracy as the Newton–Raphson method. The accuracy of the proposed method efficiently reaches at 92.67%, higher than the Newton-Raphon method, and it has better anti-noise performance and stability.

1. Introduction

Using displacement, strain, and vibration-based measurements, structural damage detection and integrity assessment have become popular in the last two decades [1]. The displacement response of a structure exhibits a potential to more accurately detect damage since it directly reflects the structural overall stiffness [2]. However, displacement measurement is very difficult to apply for realistic engineering structures due to the expensive cost such as the global navigation satellite system, or incapable requirements such as stationary reference points. Fortunately, a camera-based vision measurement, namely digital image correlation (DIC), has emerged in recent years as a promising alternative to these aforementioned displacement measurement technologies [3]. This approach can provide a relatively continuous measurement on the patterned area of a structure and can be called a full-field measurement technique, essentially within the entire line of sight of the cameras, and DIC has been widely used in different fields [4,5].
Digital image correlation (DIC) is an optical measurement technique that works based on gray-scale variations of continuous patterns. There are two main steps to implement DIC for displacement extraction: integral pixel and subpixel level matching. A multitude of integral pixel algorithms, such as the coarse-fine search and diamond search, have been proposed [6], and the performance of these methods has been satisfactorily verified by tests [7]. Chen et al. proposed an adaptive point cloud correction algorithm designed to optimize the point cloud structure after image stitching of a multi-camera and improve the accuracy of surface reconstruction [8]. Tang et al. applied the algorithm to the dynamic real-time detection of surface deformation and full-field strain in recycled aggregate concrete-filled steel tubular columns [9,10]. Rizo-Patron used DIC to accurately identify the modal parameters of rotating helicopter rotor blades [11]. Since digital images record discrete grayscale information, no matter how the defined correlation function is utilized for the correlation search, the translation of the subset can only be carried out in the unit of integer pixel; therefore, displacement resolution is limited and cannot meet the requirement for detecting damage in many cases.
Subpixel displacement extraction, which is performed based on the results of the integral pixel level identification, is considered a key technique to improve DIC-based measurement resolution. A sea of subpixel matching algorithms based on correlation coefficients has been presented, such as curved fitting [12], cross search [13], Newton–Raphson (N–R) [14], and gradient-base (GB) methods [15]. Among these approaches, the frequently utilized algorithms are the N–R and GB methods [16]. The N–R algorithm implements an iterative procedure through optimizing a non-linear correlation function to acquire accurate subpixel extraction. Nevertheless, one inherent limitation or computational burden is that the Hessian matrix must be re-evaluated and inverted in every iterative procedure [17,18]. The GB method utilizes the gray gradient or the statistical properties of micro-regions to obtain the subpixel displacement [19,20], hence, the calculation efficiency is superior to the N–R method. The accuracy of the GB method, however, is lower than the N–R algorithm [21].
Although the GB method has the advantage of fast calculation, its accuracy gradually decreases with the increase of image noise. This paper is motivated by this feature of the GB method to try to find a subpixel algorithm that can inherit the high efficiency of the GB method while having the characteristics of high precision. Furthermore, the accuracy of the GB method is greatly affected by the gray gradient, and the gray gradient is usually acquired from the bicubic spline interpolation algorithm using five-point pixels, whose accuracy is determined by the center selection of interpolation. While feasible points as the interpolation center have been given in the integral pixel matching step, there is still a paradox of which point is the optimal center. To address this problem, the double precision gradient-based (DPG) method, which is a subpixel matching algorithm with high efficiency and precision, is proposed in this study. Two interpolation functions that take the nearest neighborhood integer points as the center of the interpolation are implemented to acquire the corresponding subpixel displacement. A weight coefficient is introduced and then the identified displacement is linearly combined to obtain the final subpixel matching result. One advantage of the proposed DPG method is that it not only maintains high computational efficiency but also improves the accuracy of the conventional GB method. Furthermore, the DPG method shows a good anti-noise capability because the influence of noise is considered during interpolation.
This paper starts with a brief description of traditional integral pixel and subpixel matching methods in Section 2. The proposed DPG method for subpixel matching is then presented in Section 3. In Section 4, a numerical simulation and an experimental test are conducted to verify the proposed method. Finally, the conclusions are given in Section 5.

2. Matching Algorithm for DIC

2.1. Integral Pixel Level Matching

The basic principle of the standard GB method is to track the same subimage (or subset) located in the reference image and deformed images to retrieve the full-field displacement. Typically, a unified Cadier coordinate system is established for the reference and deformed images as shown in Figure 1. For a pixel in the reference image, marked as the point P, a subset with (2M + 1) × (2M + 1) pixels centered at point P is chosen, where M is radius of the subset. Correspondingly, a stochastic point is picked as the initial search point P′ in the deformed image, and a target subset with (2M + 1) × (2M + 1) pixels centered at P′ is selected as the target subset. A pre-defined correlation ZNSSD criterion is then performed to assess the similarity between these two subsets. For its good accuracy and anti-noise capability, the standardized covariance cross-correlation coefficient C(•) is mostly used and can be defined as [22],
C ( u , v ) = x = M M y = M M [ f ( x , y ) - f m x = M M y = M M [ f ( x , y ) f m ] 2 g ( x + u , y + v ) g m x = M M y = M M [ g ( x + u , y + v ) g m ] 2 ] 2
where x,y are position of the point P along x-y direction in the coordinate system, respectively; u and v are displacement between point P and P′ along x-y direction, respectively; f(x, y) and g(x + u, y + v) are the gray values of the subset in the reference and deformed image, respectively; and gm and fm are the ensemble averages of the selected subset in reference and target subset, respectively.
The (u,v) with maximum C(u,v) is considered the true displacement for point P. Thus, the process of integral pixel level matching is actually to search an optimal matching subset in the deformed image. Several integral pixel search algorithms are proposed to determine the parameter M and the search procedure for efficient matching, such as coarse-fine, three steps, and diamond search methods [23]. The coarse-fine algorithm was adopted in this paper because it is the most commonly used integral pixel search algorithm and has been successfully verified in many fields [24]. The procedure of this algorithm includes: firstly, according to the pre-estimated displacement, determine the search area with a random P’ point in the deformed image as the center, and use Equation (1) to find the translation solution upt and vpt with the global maximum correlation coefficient in the search area. Secondly, take upt and vpt t as the initial value, assuming that the shear deformation parameters ∂upt/∂y and ∂vpt/∂x are zero, estimate the tensile or compression deformation parameters ∂upt/∂x and ∂vpt/∂y, and use the same search strategy as the first step until the optimal solution ∂upt/∂x and ∂vpt/∂y are found. Finally, use the upt, vpt, ∂upt/∂x and ∂vpt/∂y obtained in the first two steps as initial values, use the previous search method to find the optimal solutions ∂upt/∂y and ∂vpt/∂x. The first-order mapping function is used to combine upt, vpt, ∂upt/∂x, ∂vpt/∂y, ∂upt/∂y, and ∂vpt/∂x to get the integral pixels up and vp. The detailed procedure of the coarse-fine algorithm can be found in reference [23].
It should be noted that the minimal unit in a digital image is one pixel; the displacement identified from integral pixel level matching is an integer multiple of one pixel, that is, the acquired u and v are integer numbers. To further improve the registration resolution, the subpixel matching algorithm has to be implemented.

2.2. Subpixel Level Matching Based on Gradient-Based (GB) Method

The gradient-based (GB) method was first developed by Davis and Freeman as an optical flow method [25]. Based on the basic assumption of the GB method, the subset rigid body translation exists when the subset is small enough, that is,
f ( x , y ) = g ( x + u + Δ x , y + v + Δ y )
where Δx and Δy are the subpixel displacements along the x-y direction, respectively. After neglecting the high-order terms, the first-order Taylor’s expansion of function g(•) yields
g ( x + u + Δ x , y + v + Δ y ) = g ( x + u , y + v ) + Δ x g x ( x + u , y + v ) + Δ y g y ( x + u , y + v )
where gx, gy are the first-order derivatives of intensities at the center of the best matching subset in the target image. According to reference [26], Equation (3) can be solved using the least squares technique with the following closed form:
[ Δ x Δ y ] = [ ( g x ) 2 ( g x g y ) ( g x g y ) ( g y ) 2 ] 1 [ [ ( f g ) g x ] [ ( f g ) g y ] ]
where gx(x + u,y + v) and gy(x + u,y + v) are abbreviated as gx and gy, respectively for simple expressions. f(x,y) and g(x + u,y + v) are substituted by simple description f and g, respectively. Equation (4) indicates that the key step to acquire the subpixel displacement depends on the gradient operators gx and gy.
Since the value of grayscale intensity function g(•) is only available at the integer pixel point, the interpolation scheme is usually adopted to acquire gradient value for gx and gy. The Not-a-Knot interpolation algorithm, one of the bicubic spline interpolation algorithms using five-point pixels, is used to compute the gradient operators in this paper as shown in Figure 2. Details about this algorithm can be found in reference [27]. After Δx and Δy are obtained using Equation (4), the total displacement for point P along x and y direction are u + Δx and v + Δy, respectively.

3. Double Precision Gradient-Based Method for Subpixel Matching

3.1. Gradient-Based Subpixel Matching Method

Although the gradient-based method has the advantage of high efficiency for subpixel matching, Pan pointed out that the accuracy should be improved using additional steps [26], especially when the true subpixel displacement is far away from the interpolation center. This phenomenon is related to accuracy reduction using an extrapolation technique since the accurate gx and gy in Equation (4) can only be acquired around the interpolation center. To solve this problem, the best matching integral pixel point is selected as the center for the bicubic spline interpolation. However, the accuracy of the calculated subpixel displacement will decrease unacceptably when the subpixel distance is farther than 0.5 pixel. This phenomenon can be explained using the following numerical simulation.
An 8-bit speckle pattern with a resolution of 256 × 256 pixels was obtained using the iterative spatial-gradient based algorithm [20], as shown in Figure 3a. Ten translated images were generated with an equal distance in x direction, corresponding to a shift of 0.1 pixels between two successive speckle images. Thus, only subpixel displacement extraction in x direction was considered here. A total of 576 (24 × 24) points located in the intersection of grid of 10 pixels were chosen to calculate subpixel displacement, that is, the distance between each point was 10 pixels, as shown in Figure 3a. Moreover, two adjacent integral pixels (here 0 represents the left pixel and 1 denotes right pixel) to each point were employed as the interpolation center, and then subpixel displacements corresponding to all 576 points were extracted using the traditional gradient-based (GB) method. The mean error and standard variance of subpixel displacement calculated using 576 points at each deformed image were plotted in Figure 4. The dots represent the mean error, and the upper and lower bars represent the 90% deviation lines. It was shown that mean error would increase significantly when the distance between the actual subpixel displacement and the interpolation center exceeded 0.5 pixel. This trend is directly related to the first-order derivatives used in Equation (4), which are accurate only in the vicinity of the interpolation center. Meanwhile, the standard deviation error has the same trend, which will increase from 0.003 to 0.007 when the distance between the actual subpixel position and interpolation center is near or beyond 0.5 pixel.
Furthermore, images may be contaminated by a variety of noise sources, such as photon noise, thermal noise, readout noise, and shot noise [28]. A high noise level will lead to unacceptable errors using the gradient-based method. To practically simulate random noise occurring in real situations, Gaussian noises with zero mean and five different standard deviations (varying from 1 to 5) [26] were added into the first image set (noiseless image) to generate five other image sets. The image with standard deviation (SD) 5 is depicted in Figure 3b. For all five sets of images with various noise levels, the same operation procedure in the noise-free case was implemented and results are shown in Figure 5.
As can be seen from Figure 5, mean errors and standard deviations increased with the augment of noise variance levels. It can be concluded that the noise in the image must be considered to improve the subpixel matching accuracy.

3.2. Double Precision Gradient-Based (DPG) Method

To calculate the subpixel displacement, the GB method uses the nearest integral pixel as the center for the five-point bicubic spline interpolation, and reasonable accuracy will be acquired when the appropriate integer pixel is chosen. Unfortunately, it is impossible to take the appropriate integral pixel as the center every time. Therefore, these two near neighborhood integer pixels along one direction are used to calculate subpixel displacement associated with white noise, and then the identified displacement is linearly combined via a weight coefficient to obtain the final identification results. Only the details in the x direction are described in the following part since the subpixel matching is recognized in x and y directions under 2D measurement conditions.
Firstly, the subpixel displacements Δx0 and Δx1 were extracted with the traditional GB method using the left and right pixel as an interpolation center, respectively. Then, the final subpixel displacement Δx was obtained by
Δ x = ( 1 a S D , d ) Δ x 0 + a S D , d Δ x 1
where aSD,d represents the weight coefficient, subscript d denotes the subpixel position and SD corresponds to the noise variance.
From Figure 4a, the mean error is very small when the true subpixel displacement is closer to the interpolation center, that is, Δx should be equal to Δx0 if the true subpixel displacement is 0 and the left pixel is used as interpolation center. Correspondingly, Δx should be equal to Δx1 when the true subpixel displacement is 1 and the right pixel is utilized as interpolation center. At the same time, the mean errors calculated from either interpolation center were almost equal when the true subpixel displacement is round 0.5 pixel, so the weight for Δx0 and Δx1 should be equal. Therefore, it is easy to draw the conclusion that aSD,0 = 0, aSD,1 = 1 and aSD,0.5 = 0.5.
Then, the discrete point aSD,d was obtained by substituting the data in Figure 5 into Equation (5), and results from the first two images, where the true subpixel displacement is 0.1 and 0.2 pixel respectively, are plotted in Figure 6. According to the distribution rule of discrete points a:,d (denotes the aSD,d under different variance of noise) in Figure 6, it is known that the weight coefficient a:,d first increases with the variance and then tends to be stable. Consequently, a formula in exponential form was adopted to fit the parameter a:,d for a fixed subpixel displacement:
a : , d = c 1 e c 2 · S D + c 3
where c1, c2, and c3 are fitting parameters and SD represents the variance of noise. It should be noted that we assumed that the noise level was the same for the reference image and a series of deformed images. For example, these factors, such as lighting and noise, remained unchanged during the measurement procedure. Furthermore, parameters in Equation (6) for other a:,d were obtained and are listed in Table 1 using the same operation.
Therefore, the parameter aSD,d in Equation (5) was further obtained by fitting using the spline curve as,
a S D , d = s p l i n e [ a : , d ]
Figure 7 shows the curved surface of the weight coefficient aSD,d under different SD and the pre-assigned displacement at decile points between 0 and 1. Extracting parameter aSD,d from this surface, the subpixel displacement Δx could then be calculated using Equation (5).
The detailed procedure of the proposed DPG method is as follows:
Step 1: Calculate the noise variance SD. Before the test, two consecutive images under the test condition are taken and the noise variance SD is obtained by calculating the mean square error of the gray-scale intensity of these two consecutive images.
Step 2: Determine Δx0 and Δx1. For the subpixel matching, as a point P’ in the deformed image is selected, the subpixel displacement Δxt is computed using the traditional GB method. If Δxt is less than 0.5 pixels, replace Δxt with Δx0. Then, taking the integral pixel at the right of the point P’ as the interpolation center, the subpixel displacement Δx1 is further extracted using the GB method; if Δxt is greater than 0.5 pixels, replace Δxt with Δx1. Taking the integral pixel at the left of the point P′ as the interpolation center, the subpixel displacement Δx0 is then computed using the GB method.
Step 3: Determine the d. Calculate d using d = min(|Δx0|, |1 − Δx1|).
Step 4: Calculate the weight coefficient aSD,d. The a:,d was obtained by Equation (6) according to the SD and parameters cj (j =1,2,3) in Table 1. Substitute the a:,d into Equation (7) and determine the weight coefficient aSD,d using the subscript SD and d.
Step 5: Calculate the subpixel displacement Δx. Substitute aSD,d, Δx0, and Δx1 into Equation (5) to get the identification subpixel displacement Δx.
Subpixel displacement in the y direction can be calculated using the same procedure, so it will not be described in detail here.

4. Experimental Verification

4.1. Numerical Simulation Using Speckle Pattern

To verify the proposed DPG method, a speckle pattern with size 256 × 256 pixels and 256 gray levels was simulated with the iterative spatial gradient-based algorithm, as shown in Figure 8a. The speckle granule was set as 2000, and the radius of each speckle granule was 3 pixels with 0.7 peak intensity. A unified coordinate system x-y was established for the speckle pattern and the section inside the red wireframe with 250 × 250 pixels was set as the region of interest. The simulated speckle pattern was employed as the reference image. Deformed images were copied from the reference one with random shift from 0.1 to 1 pixel in the x direction since the purpose was only to verify the subpixel matching. It should be noted that only in-plane translation was performed for deformed images in the numerical example, and the maximum translation was 1 pixel, so that the integral pixel search algorithm dose was not used in the numerical simulation. Moreover, to simulate the photography noise from camera, additive random Gaussian noise with zero mean and random variance from 1 to 5 was added to each shifted image. Here, randomly selected speckle images with different noise variances are expressed using PSNR, as shown in Figure 8b.
The proposed DPG, GB, and N–R method were performed on a desk computer (4 cores of Inter(R) Core (TM) i5 CPU750 with 2.67GHz main frequency) and the computational tool was MATLAB on the computer. The subset size was set as 41 × 41 pixels, and the mean errors with random standard deviation for those three algorithms are given in Figure 9.
As seen in Figure 9, the maximum error of the GB method was close to 0.01 pixel, while the error of the improved DPG was less than 0.0025 pixel, which is equivalent to that of the N–R method. This is because the DPG method not only adopts weight coefficient to consider the two nearest integral pixels to improve accuracy, but also covers the influence of noise on the recognition results.
Computation times for the three different algorithms are listed in Table 2. It should be noted that computation efficiency depends on a number of factors, such as programming efficiency, usage of program language, the performance of hardware, etc. The relative performance, however, can be demonstrated.
Table 2 shows that the GB method was the most efficient way where a reliability-guided displacement scanning strategy was employed to avoid the time-consuming displacement searching. Moreover, a pre-computed global interpolation coefficient look-up table using bicubic spline interpolations was employed to eliminate repetitive interpolation calculation. Furthermore, the computational efficiency of the DPG method was approximately twice longer than that of the GB method. This is because the proposed method uses one more interpolation calculation than the GB method. However, the computation cost of the N–R method is 13.7 times that of the proposed DPG method, that is to say, the calculation efficiency of the proposed method was 92.67% higher than that of the Newton–Raphson method, indicating that the N–R method was prohibited when dynamic displacement needed to be identified from an ocean of images.

4.2. Painted Beams Experiment

4.2.1. Experiment Setup

To further verify the proposed method, a simply supported beam with Chinese ancient painting on the flank was implemented, as shown in Figure 10 and Figure 11. The length, height, and width of the beam were 1400, 100, and 50 mm, respectively. The concentrated load was applied using a two-point distribution beam under a third-point bending of the tested beam. Three micrometers were used to measure deflection on quarter positions of the beam and the acquired data were taken as the reference displacement.
A DIC measurement system with charge-couple devices (CCD) camera (model GZL-CL-41C6M-C with 2048 × 2048 pixels resolution) was placed on a tripod about 4 m in front of the painted beam. Its optical axis was aligned to the shape center of the beam and perpendicular to the painted pattern surface. It was assumed that the optical axis of the camera was perpendicular to the plane of the measured object. Images before and after the loading procedure with a size of 2048 × 2048 pixels at 8-bit gray levels recorded through the CCD camera. In order to confirm the loading equipment and instrument, preloading was performed before formally loading, and the noise variance could be obtained using the CCD camera after setting up the experimental environment.

4.2.2. Experiment Results

To compare with test values obtained by these micrometers, only the area marked with a red dot line near the micrometers is considered in this subsection, as shown in Figure 11b. Coordinates of these selected pixel points from top to bottom of the beam were set as 1–168 pixel, and the corresponding distance measured with the micrometer was 100 mm. Therefore, the calibration factor was 100/168 = 0.5952 mm/pixel for the experiment system. It should be noted that the coarse-fine search method was adopted in the integer pixel search step both for the GB and DPG methods. When using the algorithmic process, directly using the captured images for processing may be affected by uncertainty and/or inaccuracy. At this time, you can choose to use techniques based on fuzzy logic methods [29,30]. These methods are very suitable for the preprocessing of images affected by performance and/or imprecision. Although this technology was not used in this study, these technologies can be considered in any future development. Taking the mid-span displacement of the beams as a reference, 5 loading cases were implemented, as shown in Table 3.
Taking the displacement from the micrometer as true displacement, the GB and DPG method were used to identify the displacement under different test conditions. The identification results are shown in Table 4, Table 5 and Table 6. The mean error of the deflection identification results of the DPG method was generally less than 0.1 mm, and the maximum absolute error was 0.083 mm, while the mean error of the deflection identification results of the GB method was generally above 0.1mm, and the maximum absolute error was 0.554 mm. It can be concluded that the accuracy of the DPG method was superior to that of the GB method. Moreover, the standard deviation obtained from the GB method was generally larger than the DPG method, which shows that the DPG method has better noise stability.
Identification results along the height of the wooden beam under case 5 are depicted in Figure 12. It can be clearly seen that displacement identified by GB method differed greatly from those identified by the other two methods, and the result fluctuates up and down all the time, indicating that the accuracy of the GB method is not high and the stability is poor. The DPG method not only achieved high accuracy as that of the N–R method, but it also has good stability. At the mid-span monitoring position near the top of the beam, the identified deflection may produce a small error. This is because the image recognition will be affected by the background image when the boundary of pixel subsets contacts the edge of the wood beam or exceeds the edge of the wood beam, so recognition results may fluctuate. Therefore, when the displacement is monitored by the DIC method, some special algorithms should be utilized to address the boundary registration, which is beyond the scope of this study.

4.2.3. Light Condition Analysis

Some studies have reported that light variation has a great influence on the accuracy and stability of identification results for DIC method [31]. In order to exploit the influence of light condition on the accuracy of the proposed method, the light conditions were set as light and dark groups, and their grayscale image is shown in Figure 13. The experimental setup in Section 4.2.1 was also used here. Because the experiment was carried out in a laboratory environment with lights, the experiment at that time was defined as the light group and the experiment done with the light in the laboratory turned off was defined as the dark group.
Since the trend was similar for all cases, only the results at micrometer 2 under case 5 are depicted in Figure 14, and Figure 15 shows the mean error of the recognition results under two groups of light and dark using different algorithm. Combining Figure 14 and Figure 15, it can be seen that in the “light” group with better lighting conditions, the mean error of the recognition results using GB, DPG, and N–R methods was almost the same. Meanwhile, in the “dark” group with poor lighting conditions, the deflection acquired by the DPG and N–R method fluctuated slightly along the identification area, but still maintained relatively high precision, while identification results by the GB method fluctuated significantly, indicating that the accuracy of the GB method is greatly dependent on the light condition.

5. Conclusions

The DPG algorithm is proposed in this study for fast and accurate displacement extraction for DIC subpixel matching. The grayscale of the two nearest integral pixels was taken as the center point for the five-point bicubic spline interpolation. Then, subpixel displacement was obtained from these two integral pixels using the traditional gradient-based method. In order to improve accuracy and anti-noise capability, a method using linear combination strategy was presented and the procedure to calculate a weight coefficient was introduced in detail. Consequently, the final subpixel displacement was achieved by linear merging these identification results from the traditional GB method. A simulation speckle pattern and a painted beam experiment were implemented to verify the practicability of the proposed method. Results show that the accuracy of the proposed method for recognizing subpixel displacement was improved by 4 times compared with the GB method. Meanwhile, the computational efficiency of the proposed DPG method was at least 13.7 times higher than that of the traditional N–R method, which provides the possibility for real-time dynamic displacement measurement for an ocean of images. Furthermore, the proposed method had better anti-noise performance and stability than the GB method under different light conditions. This merit will greatly reduce the impact of the environment on identification accuracy in the actual experiment. This DPG algorithm can be considered a complement to target tracking and displacement measurement.
The proposed DPG method also has certain limitations, such as calculation speed and stability, under certain noise conditions. Further study will be focused on exploiting other types of noise, such as salt and pepper noise, and large deformation cases where the non-translation subset should be considered.

Author Contributions

Conceptualization, G.L., M.L. and W.Z.; methodology, M.L. and W.Z.; software, M.L. and W.Z.; validation, M.L. and J.G.; formal analysis, M.L. and J.G.; writing—original draft preparation, G.L. and M.L.; writing—review and editing, G.L. and M.L.; supervision, G.L. and M.L. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by the Fundamental Research Funds for the Central Universities under grant number 2020CDCGTM002, the Natural Science Foundation of China (52078084, 51578095), the 111 Project of Ministry of Education of China and the Bureau of Foreign Expert of China (B18062), and the graduate scientific research and innovation foundation of Chongqing (CYB20027).

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Torres, B.; Varona, F.B.; Baeza, F.J.; Bru, D.; Ivorra, S. Study on Retrofitted Masonry Elements under Shear Using Digital Image Correlation. Sensors 2020, 20, 2122. [Google Scholar] [CrossRef] [Green Version]
  2. Wang, N.; Ri, K.; Liu, H.; Zhao, X. Structural Displacement Monitoring Using Smartphone Camera and Digital Image Correlation. IEEE Sens. J. 2018, 18, 4664–4672. [Google Scholar] [CrossRef]
  3. Silvani, C.; Réthoré, J.; Nicaise, S. Measuring coarse grain deformation by digital image correlation. Strain 2021. [Google Scholar] [CrossRef]
  4. Baqersad, J.; Poozesh, P.; Niezrecki, C.; Avitabile, P. Photogrammetry and optical methods in structural dynamics–A review. Mech. Syst. Signal Process. 2017, 86, 17–34. [Google Scholar] [CrossRef]
  5. Feng, D.; Feng, M.Q. Computer vision for SHM of civil infrastructure: From dynamic response measurement to damage detection—A review. Eng. Struct. 2018, 156, 105–117. [Google Scholar] [CrossRef]
  6. Hammoudi, Z.S.; Alazawi, D.A.; Mohammed, M.N. Assessment of Stress and Strain Distribution at Crack Tip Using Digital Image Correlation. IOP Conf. Ser. Mater. Sci. Eng. 2021, 1076. [Google Scholar] [CrossRef]
  7. Lee, J.; Kim, E.J.; Gwon, S.; Cho, S.; Sim, S.-H. Uniaxial Static Stress Estimation for Concrete Structures Using Digital Image Correlation. Sensors 2019, 19, 319. [Google Scholar] [CrossRef] [Green Version]
  8. Chen, M.; Tang, Y.; Zou, X.; Huang, K.; Li, L.; He, Y. High-Accuracy multi-camera reconstruction enhanced by adaptive point cloud correction algorithm. Opt. Lasers Eng. 2019, 122, 170–183. [Google Scholar] [CrossRef]
  9. Tang, Y.; Li, L.; Wang, C.; Chen, M.; Feng, W.; Zou, X.; Huang, K. Real-time detection of surface deformation and strain in recycled aggregate concrete-filled steel tubular columns via four-ocular vision. Robot. Comput.-Integr. Manuf. 2019, 59, 36–46. [Google Scholar] [CrossRef]
  10. Tang, Y.; Chen, M.; Lin, Y.; Huang, X.; Huang, K.; He, Y.; Li, L. Vision-Based Three-Dimensional Reconstruction and Monitoring of Large-Scale Steel Tubular Structures. Adv. Civ. Eng. 2020, 2020, 1–17, (Prepublish). [Google Scholar] [CrossRef]
  11. Rizo-Patron, S.; Sirohi, J. Operational Modal Analysis of a Helicopter Rotor Blade Using Digital Image Correlation. Exp. Mech. 2017, 57, 367–375. [Google Scholar] [CrossRef]
  12. Guizar-Sicairos, M.; Thurman, S.T.; Fienup, J.R. Efficient subpixel image registration algorithms. Opt. Lett. 2008, 33, 156–158. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  13. Rui, J.B.; Jin, G.C. A new digital speckle correlation method and its application. Chin. J. Theor. Appl. Mech. 1994, 26, 599–607. [Google Scholar] [CrossRef]
  14. Bruck, H.A.; McNeill, S.R.; Sutton, M.A.; Peters, W.H. Digital image correlation using Newton-Raphson method of partial differential correction. Exp. Mech. 1989, 29, 261–267. [Google Scholar] [CrossRef]
  15. Zhang, J.; Jin, G.; Ma, S.; Meng, L. Application of an improved subpixel registration algorithm on digital speckle correlation measurement. Opt. Laser Technol. 2003, 35, 533–542. [Google Scholar] [CrossRef]
  16. Xu, X. Research progress of DSCM in sub-pixel level measurement. In IOP Conference Series: Materials Science and Engineering; IOP Publishing: Bristol, UK, 2019; Volume 490. [Google Scholar] [CrossRef]
  17. Lava, P.; Cooreman, S.; Debruyne, D. Study of systematic errors in strain fields obtained via DIC using heterogeneous deformation generated by plastic FEA. Opt. Lasers Eng. 2010, 48, 457–468. [Google Scholar] [CrossRef]
  18. Wang, Y.Q.; Sutton, M.A.; Bruck, H.A.; Schreier, H.W. Quantitative error assessment in pattern matching: Effects of Intensity Pattern Noise, Interpolation. Strain Image Contrast Motion Meas. 2009, 45, 160–178. [Google Scholar] [CrossRef]
  19. Feng, D.; Feng, M.Q.; Ozer, E.; Fukuda, Y. A Vision-Based Sensor for Noncontact Structural Displacement Measurement. Sensors 2015, 15, 16557–16575. [Google Scholar] [CrossRef]
  20. Zhou, P.; Goodson, K.E. Subpixel displacement and deformation gradient measurement using digital image/ speckle correlation. Opt. Eng. 2001, 40, 1613–1620. [Google Scholar] [CrossRef]
  21. Khoo, S.W.; Karuppanan, S.; Tan, C.S. A review of surface deformation and strain measurement using two-dimensional digital image correlation. Metrol. Meas. Syst. Metrol. I Syst. Pomiar. 2016, 23, 461–480. [Google Scholar] [CrossRef]
  22. Pan, B.; Xie, H.; Wang, Z. Equivalence of digital image correlation criteria for pattern matching. Appl. Opt. 2010, 49, 5501–5509. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  23. Simončič, S.; Kompolšek, M.; Podržaj, P. An advanced coarse-fine search approach for digital image correction applications. Facta Univ. Ser. Mech. Eng. 2016, 14, 63–73. [Google Scholar] [CrossRef] [Green Version]
  24. Tong, W. Formulation of Lucas-Kanade Digital Image Correlation Algorithms for Non-contact Deformation Measurements: A Review. Strain 2013, 49, 313–334. [Google Scholar] [CrossRef]
  25. Davis, C.Q.; Freeman, D.M. Statistics of subpixel registration algorithms based on spatiotemporal gradients or block matching. Opt. Eng. 1998, 37, 1290. [Google Scholar] [CrossRef]
  26. Pan, B.; Xu, B.Q.; Li, K.J. Performance of gradient operators in algorithm of gradient-based subpixel registration. Opt. Tech. 2006, 31, 26–31. [Google Scholar] [CrossRef]
  27. Guo, R.X.; Cai, G.C.; Kun, M.A.; Li, J.C. A comparative study on the subpixel displacement calculation methods used in digital speckle measurement. Laser J. 2005, 5, 65–67. [Google Scholar] [CrossRef]
  28. Wang, Z.Y.; Li, H.Q.; Tong, J.W. Statistical Analysis of the Effect of Intensity Pattern Noise on the Displacement Measurement Precision of Digital Image Correlation Using Self-correlated Images. Exp. Mech. 2007, 47, 701–707. [Google Scholar] [CrossRef]
  29. Orujov, F.; Maskeliūnas, R.; Damaševičius, R.; Wei, W. Fuzzy based image edge detection algorithm for blood vessel detection in retinal images. Appl. Soft Comput. 2020, 94, 106452. [Google Scholar] [CrossRef]
  30. Versaci, M.; Morabito, F.C. Image Edge Detection: A New Approach Based on Fuzzy Entropy and Fuzzy Divergence. Int. J. Fuzzy Syst. 2021, 1–19, (Prepublish). [Google Scholar] [CrossRef]
  31. Gu, G. and Wang, Y. Uneven intensity changes correction of speckle images using morphological Top-Hat transform in digital image correlation. Imaging Sci. J. 2015, 63, 488–494. [Google Scholar] [CrossRef]
Figure 1. Displacement identification principle of two-dimensional DIC during integral pixel step.
Figure 1. Displacement identification principle of two-dimensional DIC during integral pixel step.
Sensors 21 03140 g001
Figure 2. Bicubic spline interpolation diagram: (a) the discrete grayscale points of integral pixel; (b) the bicubic spline interpolated graph. Notes: The negative pixel in this figure does not mean that it is absolutely negative, but only represents relative position.
Figure 2. Bicubic spline interpolation diagram: (a) the discrete grayscale points of integral pixel; (b) the bicubic spline interpolated graph. Notes: The negative pixel in this figure does not mean that it is absolutely negative, but only represents relative position.
Sensors 21 03140 g002
Figure 3. Speckle pattern: (a) without noise; (b) Gaussian white noise with zero mean and variance 5.
Figure 3. Speckle pattern: (a) without noise; (b) Gaussian white noise with zero mean and variance 5.
Sensors 21 03140 g003
Figure 4. Matching error without noise: (a) mean error; (b) standard deviation error.
Figure 4. Matching error without noise: (a) mean error; (b) standard deviation error.
Sensors 21 03140 g004
Figure 5. Matching error with noise and the interpolation centered at 0 pixel: (a) mean error; (b) standard deviation error.
Figure 5. Matching error with noise and the interpolation centered at 0 pixel: (a) mean error; (b) standard deviation error.
Sensors 21 03140 g005
Figure 6. Fitting diagram of noise SD and weight coefficient a:,d: (a) a:,0.1; (b) a:,0.2.
Figure 6. Fitting diagram of noise SD and weight coefficient a:,d: (a) a:,0.1; (b) a:,0.2.
Sensors 21 03140 g006
Figure 7. The curved surface of weight coefficient aSD,d.
Figure 7. The curved surface of weight coefficient aSD,d.
Sensors 21 03140 g007
Figure 8. (a) Speckle pattern of the reference image used in numerical simulation; (b) PSNR under randomly selected noise.
Figure 8. (a) Speckle pattern of the reference image used in numerical simulation; (b) PSNR under randomly selected noise.
Sensors 21 03140 g008
Figure 9. Mean errors of displacements measured with different noise level by (a) GB, (b) DPG, and (c) N–R method.
Figure 9. Mean errors of displacements measured with different noise level by (a) GB, (b) DPG, and (c) N–R method.
Sensors 21 03140 g009
Figure 10. The schematic diagram of the experiment (units: mm).
Figure 10. The schematic diagram of the experiment (units: mm).
Sensors 21 03140 g010
Figure 11. The testing site with panorama: (a) experiment setup; (b) the test beam.
Figure 11. The testing site with panorama: (a) experiment setup; (b) the test beam.
Sensors 21 03140 g011
Figure 12. Identification results along the height of beam: (a) Micrometer 1; (b) Micrometer 2; (c) Micrometer 3.
Figure 12. Identification results along the height of beam: (a) Micrometer 1; (b) Micrometer 2; (c) Micrometer 3.
Sensors 21 03140 g012
Figure 13. Grayscale images of the painting beam: (a) light group; (b) dark group.
Figure 13. Grayscale images of the painting beam: (a) light group; (b) dark group.
Sensors 21 03140 g013
Figure 14. Identification results with different lighting conditions at micrometer 2: (a) GB; (b) DPG; (c) N–R.
Figure 14. Identification results with different lighting conditions at micrometer 2: (a) GB; (b) DPG; (c) N–R.
Sensors 21 03140 g014
Figure 15. Mean error of recognition results under different algorithms.
Figure 15. Mean error of recognition results under different algorithms.
Sensors 21 03140 g015
Table 1. Coefficient for a:,d.
Table 1. Coefficient for a:,d.
d0.00.10.20.30.40.50.60.70.80.91.0
c10.000−0.143−0.287−0.403−0.4320.0000.3750.4110.2820.1330.000
c20.000−0.132−0.222−0.271−0.3620.500−0.370−0.307−0.240−0.1700.000
c30.0000.1100.2050.2980.4030.5000.6080.7040.8030.9011.000
Table 2. Computation efficiency comparison.
Table 2. Computation efficiency comparison.
MethodGBDPGN–R
Compution time (s)0.4040.82711.287
Table 3. Beam deflection under different cases (units: mm).
Table 3. Beam deflection under different cases (units: mm).
Loading Case12345
Deflection at mid-span2.44.87.29.612
Table 4. Deflection identification at the micrometer 1 (units: mm).
Table 4. Deflection identification at the micrometer 1 (units: mm).
CaseMicro-MeterGBDPG
IdentificationMean ErrorStandard DeviationIdentificationMean ErrorStandard Deviation
11.9791.916−0.0631.0501.971−0.0080.010
23.7353.691−0.0441.9843.724−0.0110.015
35.4675.302−0.1652.5085.457−0.0100.024
47.1816.962−0.2190.8947.178−0.0030.028
58.9368.719−0.2170.3848.930−0.0060.040
Table 5. Deflection identification at the micrometer 2 (units: mm).
Table 5. Deflection identification at the micrometer 2 (units: mm).
CaseMicro-MeterGBDPG
IdentificationMean ErrorStandard DeviationIdentificationMean ErrorStandard Deviation
12.4142.5730.1590.0162.4750.0610.012
24.8284.717−0.1112.0384.9110.0830.015
37.2146.797−0.4172.9157.2890.0750.018
49.6029.048−0.5543.0999.6260.0240.037
512.01211.465−0.5470.11611.969−0.0430.042
Table 6. Deflection identification at the micrometer 3 (units: mm).
Table 6. Deflection identification at the micrometer 3 (units: mm).
CaseMicro-MeterGBDPG
IdentificationMean ErrorStandard DeviationIdentificationMean ErrorStandard Deviation
11.9722.0470.0750.4411.922−0.0500.021
23.7023.7740.0722.4183.634−0.0680.030
35.4205.6170.1970.9825.359−0.0620.041
47.1326.871−0.2610.4477.112−0.0200.050
58.8608.602−0.2580.2988.825−0.0350.059
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Liu, G.; Li, M.; Zhang, W.; Gu, J. Subpixel Matching Using Double-Precision Gradient-Based Method for Digital Image Correlation. Sensors 2021, 21, 3140. https://doi.org/10.3390/s21093140

AMA Style

Liu G, Li M, Zhang W, Gu J. Subpixel Matching Using Double-Precision Gradient-Based Method for Digital Image Correlation. Sensors. 2021; 21(9):3140. https://doi.org/10.3390/s21093140

Chicago/Turabian Style

Liu, Gang, Mengzhu Li, Weiqing Zhang, and Jiawei Gu. 2021. "Subpixel Matching Using Double-Precision Gradient-Based Method for Digital Image Correlation" Sensors 21, no. 9: 3140. https://doi.org/10.3390/s21093140

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop