Significance of Parallel Computing on the Performance of Digital Image Correlation Algorithms in MATLAB
Round 1
Reviewer 1 Report
Dear authors,
Thank you for submitting this manuscript. The manuscript deals with investigating computational performance of various algorithms on finding displacements in digital image correlation framework as implemented in MATLAB. The main finding of the paper is that parallelisation of DIC process leads to increased computational efficiency, which is well known to the point that commercial platforms (e.g. VIC, DaVis, MatchID) have already parallel capabilities. In my opinion the research has significant flaws and cannot be published in the current form.
General comments:
-DIC is already a mature technology with number of commercial platforms offering holistic approach to DIC measurements. As such performance of in-house Matlab codes is no longer an important issue as compiled codes run orders of magnitude faster and the identified issues with Matlab implementations might not be necessarily relevant to compiled codes.
-Treatment of DIC settings/data is lacklustre. I recommend familiarizing with “A Good Practices Guide for DIC“ published by iDICs (https://idics.org/guide/) – the manuscript I received tackled DIC in very superficial way. The data processing parameters were not properly discussed (or even mentioned) – throughout the paper I was coming back to the questions “How many data points are they processing? What is the subset size? What is the step size (sampling)?”. Authors used linear interpolation functions which seem to be completely out of data with majority of commercial codes utilizing higher order polynomials/splines/N-taps. Shape function was not even mentioned – from the text it seems rigid movement is assumed with maybe IC-GN algorithm allowing for affine deformation?
-Results reporting is poor. While all the necessary data is available in Tables 4-6 it could be presented in graphical chart somehow to have a quick comparison between the three scenarios. From the presenting data point of view I think it would be much better if time was normalised by number of data points processed (subsets in an image*number of images) – that would make Sets 1 and 2 more comparable. I think there is also a missed opportunity of not using detailed code analyser in Matlab. My immediate questions were: Does the difference between sets and scenarios (series/parallel) come from algorithm efficiency, overheads due to setting up the algorithm, overheads taken by Matlab implementation of code, data transfer etc? This could be all quantified using Matlab and would add a lot of weight to the points being made in the paper.
-In terms of parallelisation by splitting the image pairs, so-called incremental correlation is primarily used with large deformation situations as it leads to increased error levels. In the case of applying the more efficient parallelisation however this could be overcame – I imagine that even when all images are correlated with respect to the first one (reference) they could be split and parallelised as a copy of the reference image could be sent to every worker. Just a small suggestion ?
Minor comments to the manuscript:
-Section 2, first paragraph – f(x,y) and g(x,y) could be written with a different font to stand out from the text
-Section 2.1, second paragraph – Bars over f and g are misplaced
-Section 2.3, Citations [22] and [23] in the text seem not to match what is in the reference section
-Section 2.3, third paragraph – sentence “Due to the vastness and complexity of the parallel computing procedures, this study has limited itself to a discussion about the same in general terms […]” grammatically incorrect.
-Section 2.3, fourth paragraph – “worker” stands out in terms of formatting
-Section 3, fourth paragraph – So called-incremental correlation is primarly when deformation between the reference and the deformed images is too large to obtain correlation, then the sequential correlation might be employed to reduce the deformation between the pairs of images. This process is commonly used with e.g. rubbers, however should not be used in every situation as it leads to increased displacement noise (as noise pairs sum up over deformation history), see further “The good practices guide”.
-Section 3, fifth paragraph, line 4: Missing ‘.’ Before “All”?
-Table 3, Data transfer rate for computer 2, should use ‘.’ Instead of ‘,’ (consistent with CPU 2”.”4 GHz)
--Section 3, last paragraph – I don’t understand why plotting is included in the run, it should be similar between (nominally the same) across all the algorithms so it just artificially adds time to the results which skews then relative differences between algorithms!
-Section 4, figures 3 and 4 – Could you set the same color scale in both pictures (in-house code and Ncorr) so that they are easily comparable? This seems like minimal effort leading to much better presentation. While the fields look similar to the human eye they are not the same and when differentiated to get strains would lead to significant differences. Could you plot the difference between the two to appreciate the extent of the difference and comment where it comes from? Is it DIC settings that are different? Is it convergence of sub-pixel identification?
-Section 4.2, as covered in major comments section I have number of issues how the data is presented here. The two datasets used are extremely different (one is 200+ images, the other 11) and so will scale differently in terms of data read/overheads etc. This should be quantified if the main point of the paper is to investigate computational efficiency! Moreover, the number of data points in the region of interest of each data set is never mentioned – it might be the case that set 2 has twice the number of data points and we would never know. Again, I’d recommend scaling the execution time by the number of points: t_new = t_old/(N_pts*N_frames) as it allows to compare the two data sets more meaningfully.
-Figure 1 – Could you demonstrate the histogram for this data set? Images look extremely oversaturated – this will have tremendous effect on the accuracy of DIC (not that important in the context of the paper), however might also significantly affect convergence rate and correlation level.
-Section 5, paragraph 1: “baud” was typed instead of “band”
-Section 5.5: “Besides, the GPU accelerated system uses a random characteristic microstructure sample surface for correlation without need for markers, hence no surface preparations are required even on polished surfaces [22]”. – Surely that is not intrinsic property of GPU accelerated system!!
-Section 5.5: “According to Conrad et al. [33], GPU-based DIC system is capable of measuring total strain at the speed up to 850 Hz and the strain fields overcome parallel computing limitations. “ – This is clearly dependent on the image size/data density of the problem. I’m not sure you could process 25 MPIx high resolution image with subset size 21 and step 1 at that speed, however a standard 3 MPix image with subset size 35 and step 17 – quite possibly. In the reference [33] the reported computation times correspond to the stereo reprojection between pairs of stereo-images and I cannot find the total correlation time.
-Section 6.: 0.1 pix is indeed low accuracy. Typically DIC application require 0.01 px accuracy – this could be reduced even further when the experiment is carried out very carefully. In standard applications anything between 0.02-0.04 px is a reasonable guess.
-Reference list is inconsistent with some citations including full first name of authors and some only the first letter.
Author Response
Please find the Reply to the Review Report attached.
Author Response File: 
Author Response.pdf
Reviewer 2 Report
The manuscript is devoted to the analysis of time resolution on the performance of Digital Image Correlation. The manuscript requires revision since there is no clear understanding of the applied use of the calculated data. The direct relation is not well explained. It is necessary to present the results in such a way that can be better to understand by the reader. Figure 2 is missing in the text. Maybe it should be joined with figure 3?. Also, figures and tables are far away from the paragraphs were their explanation is presented. In the majority of the cases, the figures are another way to present the data presented in the tables. This must be reviewed and try to present the data in table or a figure. Join the data. Table 4 and figure 5 are far from the explanation, table 6 appear in the text before the others. The order is missing. At the end of page 13, "Comparing table 4 to table...." the number is missing. Table 5 and figure 6 represent the same. One should be chosen and maintained through the text. Reference17 and 19 are missing the year. Reference 23 does not much the text, maybe?. Reference 24 should show the consultation date. Introduction section, first paragraph, "DIC method has been continuously improved by different researchers" and no references are provided. The introduction and the theoretical background is lacking in references. The first paragraphs of point 2 show no references, where the method is explained. Point 2.1 "the algorithms available vary in many ways" Which ways? references? explanation? A nomenclature table would be necessary cause some of the variables are missing, like M. The authors estate that "DIC is usually limited to 2D analysis due to its long processing time" That depends on what the method is compared with. The 3D implementation is also usually applied. More references and explanation is needed. Authors decided to focus on MATLAB but the advantages are poorly explained or presented. Point 2.3 "Nonetheless, some studies have been published recently..." One of the 2 references is from 1985. That is not recently. The same happens with reference 25 that is presented as recent and is from 2004. Point 3. "Some modifications of those algorithms..." which modifications? There is no option to present these modifications and discuss them? The first set consists of 204 imagens vs the second that is only 11. Is no there a big difference between the number of images? Table 1 and 2 can be combined. The images are huge. I do not understand why the computer different is explained in such detail. This is something that can be assumed. A better hardware improves the analysis (discussion, first paragraph). The statement "therefore, in most cases, using a modified version will not result in a significant disadvantage, but may be beneficial" need more explanation cause, after the reeding, it can be understood that the two versions are the same. Conclusions section. Something is missing in the first line. Maybe "shows"?. "The correct parallelization type..." is open to interpretation by the reader. Finally, it is not clear what are the advantages of the changes introduced.
Author Response
Please find the Reply to the Review Report attached. Thanks
Author Response File: Author Response.pdf
Round 2
Reviewer 2 Report
All the suggestions have been taken into account. Some references are missing in the text and an error message is shown. Also, a blank page appears in the middle of the manuscript.
Author Response
Please see the attachment. Thanks.
Author Response File: Author Response.pdf
This manuscript is a resubmission of an earlier submission. The following is a list of the peer review reports and author responses from that submission.
Round 1
Reviewer 1 Report
The author presented detailed performance analysis of Digital Image Correlation (DIC) algorithms widely used in various fields.
The author analyzed introduced DIC algorithms using MATLAB in the manuscript and showed Modified Particle Swarm Optimization (PSO) with combination of Newton-Raphson algorithm on parallel performs best in Table 6.
However, a parallel computation used in this manuscript is limited to CPU parallel computation, whereas recent studies focus on GPU parallel computation.
Therefore, performance improvement of parallel computing using CPU presented in this manuscript is not impressive. The manuscript also lack of reference to recent researches.
Therefore, I recommend reject the manuscript.
Reviewer 2 Report
The aim of the authors is a rather narrow problem. One is the comparative study of serial and parallel computational implementations in MATLAB of the most common DIC methods and evaluation their performance to determine whether the real-time analysis is possible with these methods.
However, the decision to focus on matlab is in conflict with the purpose of the study of real-time possibilities.
This is largely due to the lack of a sufficient overview of the current state of research in the DIC domain. Relevant references after 2016 are not presented in the article. Current research in DIC focuses on high resolution imaging (HRDIC – High Resolution DIC) and optimizing parallel implementations.
As an example, the following works can be indicated.
- Lunt, et al. Comparison of sub-grain scale digital image correlation calculated using commercial and open-source software packages, Materials Characterization 163 (2020) 110271.
- Shuai, et al. Accelerate multi-thread path-dependent digital image correlation by minimizing thread competition for real-time deformation measurement, Optics and Lasers in Engineering 111 (2018) 98–107.
The work Lunt et al. (2020) clearly shows that for real-time analysis it is necessary to use more powerful packages that do not use MATLAB. The results of this article show that Ncorr using MATLAB is significantly (up to ~10–20 times) slower than the other software packages.
Optimization of parallel implementations is discussed in more detail in Shuai et al. (2018). And this is more crucial issues then the simple comparison of parallel implementations with different ways of image partition.
The authors have implemented the basic well-known DIC methods in the Matlab environment, three in the group of integer-pixel search algorithms (Brute Force Search Algorithm, Particle Swarm Optimization (PSO), Modified PSO with integrated Star Search algorithm) and three in the group of sub-pixel search algorithms (Newton Raphson Method, Inverse-Compositional Gauss-Newton (IC-GN) Method self-implemented and IN-GN Method by Baker and Matthews [25]). Only the Modified PSO method is relatively new, but the details of its modification are not described in the article. It is not clear why it was not used the open-source Ncorr realization of IC-GN method by Blaber et al. [26].
Several pairwise combinations of these methods in combination with two different parallelization approaches were tested on two computers. The used computers have very average characteristics in term of parallelization. The timing evaluationincluded the launch of the program, loading images and displaying the results, so it is not possible to identify advantages of parallelizing a particular processing stage. Possibly the parallelization of the sub-pixel search stage is not the optimal choice for the Sub-Image partition, but the presented results cannot clarify this.
In general, the article is enough obvious work, the results of which are weakly applicable for progress in the real-time DIC analysis and HRDIC.
Reviewer 3 Report
The reviewed work concerns the importance of parallel computing on the performance of digital image correlation algorithms. The authors used the Matlab program. Unfortunately, not everything has been explained.
1. Why did the authors not use the Parallel Computing Toolbox?
2. The work mentions the GPU, and I am not sure if such tests have been performed?
If so, why is Table 3 missing GPU information?
If not, I think it would be necessary to test on the GPU as well.
3 Computer 1 and computer 2 are very different. I think it would be enough to test on a better version.
4. Why do pictures 2 and 3 differ in the size presented? Image 2 has an additional frame. This is the result of an error or is there such a frame in the image on purpose? The same is the case with figure 4.
5. In Table 4, the time unit [s] for PC1 has been shifted to the line below. I propose to expand the column. Numeric results are easier to read if they are right-aligned.
