Geodesic Length Measurement in Medical Images: Effect of the Discretization by the Camera Chip and Quantitative Assessment of Error Reduction Methods

Round 1

Reviewer 1 Report

The authors seem to have nearly complete addressed a study to figure out possible sources of error in the field of measuring vessels (and derived parameters like blood flow) using remote sensing. Therefore, the authors investigated the error in geodesic length measurement caused by discretization on the camera chip. They not only developed a computational vessel segmentation model based on mathematical functions, but also a small physical model using silicone tubes filled with a solution containing Indocyanine Green. I really like the idea of the study, especially because it is not limited to bypass surgeries on patients suffering from cerebrovascular diseases (this is the typical field of application of the methods described). The work can rather bring benefits in areas of sensor-based diagnostic imaging (remote sensing). One example is the vessel analysis of the fundus of the eye, which has similar problems of sensing and measuring. However, there are some minor problems and questions and one major point of criticism.

 

Minor:

Introduction:

In the introduction the authors focus on remote sensing in the area of surgical interventions in patients suffering from cerebrovascular diseases. To increase the relevance of the paper, further literature from other scientific fields should be added. I see a direct connection with similar questions in ophthalmology in the field of retinal vessel analysis (“Rieger S, Klee S and Baumgarten D (2018). Experimental Characterization and Correlation of Mayer Waves in Retinal Vessel Diameter and Arterial Blood Pressure. Frontiers in Physiology.”) and fundus-controlled electrophysiology (“Klee S, Link D, Bessler P, & Haueisen J. (2011”). Optoelectrophysiological stimulation of the human eye using fundus-controlled silent substitution technique. Journal of Biomedical Optics.). Please add at least these scientific areas to the introduction.

 

1. Methods:

“The order and structure of the methods used in this paper are sketched in Figure 1.” - Please check the numbers in Figure 1, these refer to the results chapter.  

 

1.1. In-silico model:

The authors explain here the mathematical design of their vessel modelling. Table 1 shows the necessary mathematical formulas and the variables. I really like the idea behind this approach. However, the structure of the segments modelled in this way, influences the errors of the length measurement methods after centerline extraction. It is therefore extremely important to validate the model or at least to discuss possible errors. The given literature (“Naber, A.; Berwanger, D.; Nahm, W. In Silico Modelling of Blood Vessel Segmentations for Estimation of Discretization Error in Spatial Measurement and its Impact on Quantitative Fluorescence Angiography. 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). IEEE, 2019, pp. 4787–4790. doi:10.1109/EMBC.2019.8857146.”) is only a conference proceeding, in which the authors point out, that limits were chosen empirically.

It is clear to me that an independent biologically driven validation is extremely complex, but the reader must be informed about this limitation. Please discuss this critically and estimate the influence of the empirically chosen limits on the error.

 

1.4. Physical length measurement:

The authors explain that a total of 56 images were recorded. Please give more information about the test procedure. Were the positions always approached in the same order? What is the positioning accuracy of the rotational plate (discuss the resulting error for evaluation)?

 

1.5. Evaluation

The authors employed multiple measurements and calculated the mean in case the values differed from each other. This is a weakly standardized approach. Please describe exactly how to proceed. how often were measurements taken? Was the caliper zeroed and reapplied? Please discuss the measurement error.

 

Major:

2.1. In silico results

Based on the computational model the authors were able to perform a large number of error calculations for the different methods, this represents a great strength of the work! The more surprising it is, that the authors do not apply any further statistics. One sentence in the discussion (“This enables statistical analysis on a large and diverse data set and training of self-learning systems.”) does not release the authors to do so. One example: in table 4, does the relative error in the polynomial approach for "strait lines" differs significantly from "parabolas"? The lack of statistics is in my opinion the biggest problem of the work.

Please create a concept for the statistical analysis of the data. Please observe the principles of multiple testing and the necessary corrections (e.g. Bonferroni). Consider how the different number of function types (Table 3) influences the safety of the tests. Especially with small effect sizes a slight "trimming" of the data can be useful - maybe this helps to bring out significant effects. Please consider the basic statements in the discussion according to your statistical certainty.

Author Response

Please see the attachment

Author Response File: Author Response.pdf

Reviewer 2 Report

After reading carefully the manuscript I have the following comments and questions, as well as some recommendations for changes:

1-Not clear why this work was submitted to a photonics journal as it does not propose any method or algorithm operating at the light level, dealing rather with the effect of different interpolation methods on the error of vessel length estimation from digital (binary) images. In my view an image processing or biomedical image processing journal would have been a better choice.

2-The work extends previous results from reference 12 by the same authors. This is OK, but in my view due to its depth and extension the new material does not deserve publication as a journal paper.

3-As repeatedly stated by the authors, this study does not take into consideration other important source of measurement errors, like those due to projection from 3D to 2D. Without at least an estimate of the magnitude of those errors one cannot fully appreciate if the error reduction announced (roughly 7% to 3%) is significant and will make a difference in the overall sum of errors. Same can be said about binarization errors.

4-Some parts of the work need to be better described. Examples are the choice of the interpolation polynomials (why order 10 ?  wouldn't e.g. 6 be ok ?) and the analysis of the conditions under which the error is negative. The text in section 3 (article page 11, near the bottom) seems to imply that some thorough analysis was performed, but regretably the paper does not provide much details about it.  

5-The quality of the English should be improved as some sentences sound unnatural.

Author Response

Please see the attachment

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

The authors have extensively improved their paper “Geodesic Length Measurement in Medical Images: Effect of the Discretization by the Camera Chip and Quantitative Assessment of Error Reduction Methods”. The comments on the model parameters in the discussion are also helpful. Statistical analysis based on a non-parametric test is sufficient.

A little remark: the use of standard distribution tests is discussed again and again. Many statisticians recommend the subjective evaluation of standard plots (e.g. Q-Q diagram), which are very unusual in engineering journals. Nevertheless, it helps to evaluate your own data internally. In parallel to the p-value-based tests, confidence interval analyzes are also very helpful for checking uncertain tests.

From a content perspective, the paper is now ready for publication.

Reviewer 2 Report

After reviewing the changes made by the authors addressing the comments to the previous version, it is my opinion that the paper quality is now higher and the description of the methods proposed/studied and results obtained are now much more thorough.

In my opinion the paper is now fit for publication, after  minor spelling errors (like "accorance" on page 4, line 91) are corrected.

This manuscript is a resubmission of an earlier submission. The following is a list of the peer review reports and author responses from that submission.