Automatic Classification of A-Lines in Intravascular OCT Images Using Deep Learning and Estimation of Attenuation Coefficients

Round 1

Reviewer 1 Report

General

The study approaches OCT for atherosclerosis. Plaque detection and characterization is carried out in vivo, and processing of images is made with deep learning techniques. The topic is of interest, but the study has to be completed-a lot, and the manuscript must be better written and structured, therefore the paper must be improved a lot to be considered for publication, as pointed out bellow.

Specific

1) The English and style must be corrected and polished substantially throughout the manuscript. Numerous edits exist, the template must be considered, the entire manuscript must be carefully revised from this point of view. An English native should see the manuscript.

2) Please shorten the Abstract and remove from it titles of subsection.

3) All figures must be revised, to have them clear, to include notations (a,b,c, etc), to include in the legends the meaning of each fig a, b, c, etc, to have all notations with letters of the same size, etc.

4) Please refer in the Intro to (and complete the Refs list with) relevant titles on the OCT technology that are missing now:

- the first paper that introduced OCT:

1. Huang, E. A. Swanson, C. P. Lin, J. S. Schuman, W. G. Stinson, W. Chang, M. R. Hee, T. Flotte, K. Gregory, C. A. Puliafito, and J. G. Fujimoto, “Optical coherence tomography,” Science 254(5035), 1178-1181 (1991).

- at least a relevant review on the OCT technique, to point out advantages of FD OCT as the one used in the study:

1. A. Choma, M. V. Sarunic, C. Yang, and J. A. Izatt, “Sensitivity advantage of swept-source and Fourier-domain optical coherence tomography,” Opt. Express 11, 2183-2189 (2003).

- a state-of-the-art OCT technique for high (i.e. 2 microns) axial and lateral resolution

Cogliati A., Canavesi C., Hayes A., Tankam P., Duma V.-F., Santhanam A., Thompson K. P., and Rolland J. P., MEMS-based handheld scanning probe with pre-shaped input signals for distortion-free images in Gabor-Domain Optical Coherence Microscopy, Optics Express 24(12), 13365-13374 (2016)

etc.

5) Roughly 1/3 of the Intro should go into the Discussion section. Do not include M&M aspects, Results or Conclusions in the Intro. Clearly define the aim of the study and work hypotheses. Present if they were confirmed or not in the Conclusions.

6) Fig. 1 and related content is presented too abruptly. Please define the problem to be solved first, clearly for readers, eventually using information now placed in the Intro.

7) Define all elements prior to be sued, for example A-scans, etc. Such elements, including A-scans, should be pointed out in all figs (which are very empty and difficult to follow now).

8) Please provide refs for all equations that are not original.

9) Explanations for Fig. 4??

10) “The architecture in Fig. 4 slightly surpasses in most cases the performance of other architectures.” Where is this proved? Where are the other architectures described? Refs?

11) Fig. 5: why some areas are abnormal? Please give detailed explanations.

12) the first two paragraphs of Results must go to M&M.

13) Table 1 is trivial, please redo it.

14)  How are test accuracy, sensitivity, etc evaluated – in Tables 1, 2, etc.

15) “In test runs that correspond to Tables 2 and 3 the difference between training, validation and accuracy is not significant as a mean value”. Statistics must be provided, level of p, please complete everywhere in the manuscript. Cannot be accepted like this.

16) Fig. 6: all figs must be explained, with their relevant features. For now, statements are not demonstrated.

17) Fig. 7: what plaque types – in the legend? Explanations? What do readers see there??

18) Discussion: what about 3D OCT images, when there are none in the study. Please complete the study with 3D images or redo the Discussion.

19) Originality of study must be clearly demonstrated in Discussion against refs, impact must be pointed out.

In conclusion, this paper is actually closer to rejection with encouragement to be resubmitted, it requires a lot of work to be considered for publication.

Author Response

Response to the reviewers: Thank you for your time and effort, your reviews and your comments. We tried to address them the best way  we could. With your help, we have improved our paper considerably. More specifically:

Reviewer 1

Comment 1: The English and style must be corrected and polished substantially throughout the manuscript. Numerous edits exist, the template must be considered, the entire manuscript must be carefully revised from this point of view. An English native should see the manuscript.

Answer: Almost the entire manuscript has been revised (in terms of the structure and the content of the various sections of the paper)

Comment 2: Please shorten the Abstract and remove from it titles of subsection.

Answer: We agree and thus we removed some details and reformatted it.

Comment 3: All figures must be revised, to have them clear, to include notations (a,b,c, etc), to include in the legends the meaning of each fig a, b, c, etc, to have all notations with letters of the same size, etc.

Answer: All figures were revised.

Comment 4: Please refer in the Intro to (and complete the Refs list with) relevant titles on the OCT technology that are missing now:

- the first paper that introduced OCT:

1. Huang, E. A. Swanson, C. P. Lin, J. S. Schuman, W. G. Stinson, W. Chang, M. R. Hee, T. Flotte, K. Gregory, C. A. Puliafito, and J. G. Fujimoto, “Optical coherence tomography,” Science 254(5035), 1178-1181 (1991).

- at least a relevant review on the OCT technique, to point out advantages of FD OCT as the one used in the study:

1. A. Choma, M. V. Sarunic, C. Yang, and J. A. Izatt, “Sensitivity advantage of swept-source and Fourier-domain optical coherence tomography,” Opt. Express 11, 2183-2189 (2003).

- a state-of-the-art OCT technique for high (i.e. 2 microns) axial and lateral resolution

Cogliati A., Canavesi C., Hayes A., Tankam P., Duma V.-F., Santhanam A., Thompson K. P., and Rolland J. P., MEMS-based handheld scanning probe with pre-shaped input signals for distortion-free images in Gabor-Domain Optical Coherence Microscopy, Optics Express 24(12), 13365-13374 (2016)   etc.

Answer: We added an Intro segment and the references proposed.

Comment 5: Roughly 1/3 of the Intro should go into the Discussion section. Do not include M&M aspects, Results or Conclusions in the Intro. Clearly define the aim of the study and work hypotheses. Present if they were confirmed or not in the Conclusions.

Answer: We moved Intro parts to the Discussion section and removed other aspects. We defined the aim of the study in the Intro with the sentences: “The aim of this work is the development of a new deep learning method to detect the atherosclerotic tissues in IVOCT images .” and “The main idea is to configure the input to a CNN architecture and to choose CNN properties and training options that offer better classification results.”

Comment 6: Fig. 1 and related content is presented too abruptly. Please define the problem to be solved first, clearly for readers, eventually using information now placed in the Intro.

Answer: We updated the Fig.1 and modified the text according to reviewer’s suggestion and we updated the figure’s legend. The text before the figures is now as follows: “The method which classifies A-lines to normal and plaque, consists of the following analysis steps: (1) preprocessing - image preparation, (2) arterial wall segmentation, (3) OCT-specific transformation based on the attenuation coefficient estimation, (outcome the A-lines that are input to the CNN), (4) CNN training – testing procedure (for the detection of the pathological tissue and classification of the different tissue types) and (5) post-processing based on the majority of the classifications. The employed image processing and CNN classification procedure workflow illustrated in Figure 1 and described below. “

Comment 7: Define all elements prior to be used, for example A-scans, etc. Such elements, including A-scans, should be pointed out in all figs (which are very empty and difficult to follow now).

Answer: We added the explanation for A-line at the beginning of Methodology Overview: “A-line is the propagation line of the light beam that starts from the catheter and reaches to the depth of the biological tissue where it is completely attenuated. Terms, like true negative and false positive, etc. were also clarified.

Comment 8: Please provide refs for all equations that are not original.

Answer: We provided refs for the equations.

Comment 9: Explanations for Fig. 4??

Answer: Fig. 4 is explained in the two previous paragraphs. We updated the Fig and the legend to better explain it.

Comment 10: “The architecture in Fig. 4 slightly surpasses in most cases the performance of other architectures.” Where is this proved? Where are the other architectures described? Refs?

Answer: We don’t refer to published architectures but to convolutional neural networks that were tested by our team where the size of layers, the size of filters, the dropout properties were different than in the proposed architecture. We changed the sentence to “However, the architecture in Fig. 4 slightly surpasses in most cases the performance of other deeper CNNs. “to link it to the previous sentences.

Comment 11: Fig. 5: why some areas are abnormal? Please give detailed explanations.

Answer: By abnormal A-lines, we were referring to A-lines that contain plaque. The word “abnormal” was changed to “plaque” in the caption. In this example lipid plaque is present.

Comment 12: the first two paragraphs of Results must go to M&M.

Answer: We moved the paragraphs in a section with title “Experimental setup”

Comment 13: Table 1 is trivial, please redo it.

Answer: We consider that it should be better to remove entirely from the manuscript the inadequate table 1. The relevant paragraph (our argument for selecting the depth of 90 pixels) was rewritten in 3.5.

Comment 14:  How are test accuracy, sensitivity, etc evaluated – in Tables 1, 2, etc.

Answer: We added the sentence to clarify: “For each case, accuracy, sensitivity and specificity was calculated based on the true positive, false positive, true negative and false negative A-line classifications.”

Comment 15: “In test runs that correspond to Tables 2 and 3 the difference between training, validation and accuracy is not significant as a mean value”. Statistics must be provided, level of p, please complete everywhere in the manuscript. Cannot be accepted like this.

Answer: The initial expression was not accurate. We change it to:” In test runs that correspond to Tables 2 and 3 the difference between training, and accuracy is not large as a mean value. We added the sentence: ”For a specific test run, the mean training accuracy was 75.87% while the mean test accuracy was 74.73%. A confidence interval -4.4712 to 5.9712 of the difference is reported.”

Comment16: Fig. 6: all figs must be explained, with their relevant features. For now, statements are not demonstrated.

Answer: We updated the figs and added a detailed legend text to explain them.

Comment 17: Fig. 7: what plaque types – in the legend? Explanations? What do readers see there??

Answer: Fig.7 legend was updated to include more information.

Comment 18: Discussion: what about 3D OCT images, when there are none in the study. Please complete the study with 3D images or redo the Discussion.

Answer: We removed the reference to 3D OCT images from the Discussion.

Comment 19: Originality of study must be clearly demonstrated in Discussion against refs, impact must be pointed out.

Answer: Discussion was updated to demonstrate the originality of the study.

 

Reviewer 2 Report

The authors have presented an automatic method for A-line classification in IVOCT images using CNN. However, instead of presenting the details of the proposed work, there are lots of part for describing already well-known general things throughout the manuscript including introduction as well as Ch. 2. Furthermore, image segmentation and/or classification based on A-line scans are not new. Some other additional comments are:

- Resolution for Figure 4 should be improved.

- The result images should be changed to original image form (circular form before transformation)

- The results in the Tables should include values for standard deviation to be more meaningful.

-  The reasons and/or explanation of the results should be provided. Currently, the results in Table or Figure forms are simply rephrased without describing any reasons for the obtained results.

- The results in the figure images should be carefully compared with the original images before segmentation.

- Some typos should be carefully corrected throughout the manuscript.

- The details of each step in the method should be accompanied with appropriate supporting results.

- The numeric results should be compared with other existing methods to be fairly evaluated.

- How was the line tracking obtained? It is able to observe some errors in the lines drawn upon the images.

Author Response

Thank you very much for your helpgul comments. Following are our answers to your comments.

Reviewer 2

 

Comment1: Resolution for Figure 4 should be improved.

Answer: We updated the figure

Comment 2: The result images should be changed to original image form (circular form before transformation)

Answer: We agree that original form is more suitable for presentation. However, we could not reproduce the images to original form in this time frame.

Comment 3: The results in the Tables should include values for standard deviation to be more meaningful.

Answer: We added standard deviation values in Table 1.

Comment 4: The reasons and/or explanation of the results should be provided. Currently, the results in Table or Figure forms are simply rephrased without describing any reasons for the obtained results.

Answer: We consider that in results section and throughout the document, there are explanations for the results of the proposed method.

Comment 5: The results in the figure images should be carefully compared with the original images before segmentation.

Answer: We could not address this issue during this reviewing process.

Comment 6: Some typos should be carefully corrected throughout the manuscript.

Answer: We tried to correct  some typos throughout the manuscript.

Comment 7: The details of each step in the method should be accompanied with appropriate supporting results.

Answer: The details of each step are justified by supporting results in the section 5.

Comment 8: The numeric results should be compared with other existing methods to be fairly evaluated.

Answer: To the best of our knowledge, the specific dataset and task that are not presented elsewhere to make it possible to compare with other methods.

Comment 9: How was the line tracking obtained? It is able to observe some errors in the lines drawn upon the images.

 Answer:  The line tracking was obtained by the method presented in “ G.-A. Cheimariotis et al.  ARC-OCT: Automatic detection of lumen border in intravascular OCT images, Comput. Methods Programs Biomed. 151 (2017). https://doi.org/10.1016/j.cmpb.2017.08.007. “However, the line tracking appears different to obtain A-lines and some errors are possible. 

   

 

Reviewer 3 Report

Dear authors

your article fits the publication objectives of the journal but some aspects must be improved. I list most of them below.

• the introduction is too long (3 pages): this is far above what is necessary.
• line 161-162 is "contradictory" with line 87.
•  line 176-178: already said.
• line s 75 and 200: the fiirst announce 100-300 images per patient and the second announce 183 images only. which is confusing for me. could you please make databse organisation clearer ?
• line 226: the expression "calibration markers" is ambiguous. a calibration object is usually importantin engineering  for the status of a device. And you propose to cancel it. Can you explain.
• 254: you attribute a tag  to plaques even with a single pixel which means a small DICE. BUt i didn't see any consequence of this choice on you final result.
• 235: your equation if not numbered. please correct this.
• none of your finures are   centered. please correct this.
• 271:transformation  equation if not understandable. make it fully comprehensive in your text.
• line 317:"keep small the receptive field":  could you explain why your objjetive is to keep this receptive field small? in my experience, it is usually the opposite: to enlarge it...
• Figure 4 is confusing because it mixes a figure and a table. the etxt in the table is and deserve to be detailled in the caption.
• line 345: "It is assumed that...": who decides this ? any reference?
• line 346: "A-line which is not classified as belonging to the
  same class with most of a group of 40 adjacent A-lines, is probably misclassified".  "probably" : on what basis ?
• the postprocessing section is not explained. please give some mathematical detaiils.
• in table 1 page 11: define mathématically  what is "test accuracy" :   is itaccuracy= (TP+TN)/(TP + TN + FP + FN) ? what is the size database ?
• in table 2, the mean test accuracy is not good as mentionned inthe paragraoh starting line 395.
• line 478: "Very few studies which attempt automatic classification...": very few or none? just atfer you explain"there is no study for A-line classification that distinguishes four plaques." .
• the number in Fig 8 are too small.
• line 527:"On the other hand, data augmentation techniques are not a favorable option because of the specific nature of OCT A-lines." why , could you explain what is the specificity  of theOCT-A lines which prevent to use GAN ?

I dont understand sentence (lines 110-111).

in summary,I enjoy the reading of the  the paper but as we said by us, "devil is in the details", and  you skip some explanations of the algorithm,  in particular the postprocessing step which is not explained. the final impression is your pipeline is functionning better than others, but not so good and  I congratulate you on having had had the honesty of the deal.to give them. 

Author Response

The authors would like to thank the reviewer for his helpful comments. We hope we addressed them satisfactorily. Please find them below.

 

Reviewer 3

 

Comment1: the introduction is too long (3 pages): this is far above what is necessary.

Answer: We shortened the Introduction section at the indicated points and we moved related work to a separate section 

 

Comment 2: line 161-162 is "contradictory" with line 87.

• 87: It is important to clarify that plaques are grouped into more categories but this work deals with the type of plaques that are present in our data and are more interesting for medical experts.
• 161-162: More often, there is no distinction between fibrous tissue and fibrous plaque.

Answer: Concerning line 87: In medical journals 7 plaque types are referred. However, other methods attempt to classify less than 4. More often, other methods do not consider fibrous plaque.  We change lines 161-162 with the “More often, these methods do not distinguish fibrous plaque. “

 

Comment 3: line 176-178: already said.

Answer: We removed this sentence.

 

Comment 4: lines 75 and 200: the fiirst announce 100-300 images per patient and the second announce 183 images only. which is confusing for me. could you please make database organisation clearer ?

Answer: Line 75 refers to routine acquisitions. However, only images with presence of plaque were annotated and included in this study. Non-annotated images could be included in the study with the assumption that every A-line is normal. This could lead to imbalanced dataset. Besides, the main task of this study is to dinstiguish plaque and normal A-lines within one image.  We change the following sentences: “Expert cardiologist annotated 183 images with tissue and plaque types according to the standards of published consent of experts [5]. Plaques which appeared in the annotated images were: 84 lipid, 80 calcified, 70 fibrous, και 42 mixed”

Comment 5: line 226: the expression "calibration markers" is ambiguous. a calibration object is usually important in engineering  for the status of a device. And you propose to cancel it. Can you explain.

Answer: By calibration markers, in this context, we mean lines of white pixel which are superimposed in the raw image by the acquisition software (see top image Figure 1.)

 

Comment 6: 254: you attribute a tag  to plaques even with a single pixel which means a small DICE. BUt i didn't see any consequence of this choice on your final result.

Answer: We don’t use DICE. In each setup,  each A-line is classified to 2 ,3 or 5 classes, Evaluation metrics (accuracy, sensitivity, specificity) were calculated by TP, TN,FP, FN of these classifications.

 

Comment 7: 235: your equation is not numbered. please correct this.

Answer: We numbered the equation.

 

Comment 8: none of your figures are   centered. please correct this.

Answer: We centered the figures.

 

Commnet 9: 271:transformation  equation is not understandable. make it fully comprehensive in your text. 

Answer:We changed the sentence explaining the equation to: “The transformation formula is: μ[i,j=I[i,j]/(2*Δ* I[i,n])  where I[i,j] is the intensity in the original image , Δ is the physical size of a pixel, i indicates the A-line number and j indicates the depth. “

 

Comment 10: line 317:"keep small the receptive field":  could you explain why your objjetive is to keep this receptive field small? in my experience, it is usually the opposite: to enlarge it...

Answer: Since A-lines are dealt as images of 1x90, we are interested in capturing the fine details. However, the coarse details are also captured by the deeper convolutional layers.

We changed the sentence to: “The size of the convolutional kernels was small to keep small the receptive field, so that high spatial frequencies stimulate the kernels.  “

 

Comment 11: Figure 4 is confusing because it mixes a figure and a table. the text in the table is and deserve to be detailed in the caption.

Answer: We renew Figure 4 and complete its legend (in order to clarified and better explain it).

 

Comment 12: line 345: "It is assumed that...": who decides this ? any reference?

Answer: It is an assumption that we made based on the minimum extent of plaques in our dataset. No plaque consisted of less than 40 adjacent A-lines. Besides, there cannot be a single normal A-line inside a plaque region. We changed the sentence to : Based on the minimum extent of plaques which is 40 A-lines  and to the fact that a single normal A-line cannot be inside a plaque region, we assumed that an individual A-line which is not classified as belonging to the same class with most of a group of 20 adjacent A-lines, might be misclassified.

 

Comment 13: line 346: "A-line which is not classified as belonging to the same class with most of a group of 40 adjacent A-lines, is probably misclassified".  "probably" : on what basis ?

Answer: We agree that “probably” is ambiguous. Based on the explanation of the  previous comment, we consider that in a group of A-lines if there is a small fraction with different classifications than the majority, the small fraction is misclassified and we update the outcome with the majority classification outcome. However, the small fraction may be  correctly classified. That is why we used the word “probably”. We change “is probably misclassified” with “might be misclassified. “

 

Comment 14: the postprocessing section is not explained. please give some mathematical detaiils.

Answer: We agree that mathematical details should be added and we added the part: “Let the notation of classification be: C(i)=0(normal) or 1(plaque) where i is the A-line number. For i=1,21,42,…,M (where M is the total number of A-lines in an image): C(i:i+20) values are updated to 0 if sum(C(i:i+20)=<10 and C(i:i+20) are updated to 1 if sum( C(i:i+20)>10.”

 

Comment 15: in table 1 page 11: define mathématically  what is "test accuracy" : is itaccuracy= (TP+TN)/(TP + TN + FP + FN) ? what is the size database ?

Answer: We agree that it is not clear how test accuracy is evaluated. We added the sentence to clarify: For each case, accuracy, sensitivity and specificity was calculated based on the true positive, false positive, true negative and false negative A-line classifications.

The size of database is presented in the last sentence of material and methods.

 

Comment 16: in table 2, the mean test accuracy is not good as mentioned in the paragraph starting line 395.

Answer: There is no claim that accuracy is so good in the paragraph but we considered it good, taking into account the small data set available.

 

Comment 17: line 478: "Very few studies which attempt automatic classification...": very few or none? just atfer you explain"there is no study for A-line classification that distinguishes four plaques."

Answer: In the first sentence, we refer to studies that classify tissue. In the next sentence we refer to studies that classify A-lines specifically.

 

Comment 18: the number in Fig 8 are too small.

Answer: We enlarged the image to make numbers more visible.

 

Comment 19: line 527:"On the other hand, data augmentation techniques are not a favorable option because of the specific nature of OCT A-lines." why, could you explain what is the specificity  of the OCT-A lines which prevent to use GAN ?

Answer: The use of GAN is not prevented and it is a good idea to test in OCT data. We modified the sentence about data augmentation and added another one. On the other hand, various data augmentation techniques are not a favorable option because of the specific nature of OCT A-lines. However, generative adversarial network (GAN) for data augmentation may be a suitable option and it will be applied in future work.

Comment 20: I don’t understand sentence (lines 110-111).

Answer: We agree that the sentence is not clear. “Being an image classification method, it does not classify image regions into the above categories”. We changed this sentence: “This method does not classify image regions but whole images.”

Round 2

Reviewer 1 Report

The paper was revised according to all comments made. It is in much better shape now, but a certain revision must still be made to have it published - in the opinion of this reader.

Figs. 1 and 4 are still unclear, they must be replaced with better resolution figs. Also, Fig. 4 is put together with a tab. This is not satisfactory,  please split them apart and renumber tabs. 

Please remove from the title the notation IVOCT, as the readers may not be familiar with it. Instead, use 'Intravascular Optical Coherence Tomography' or, eventually, 'Intravascular OCT'. 

English must be revised (e.g. 'in comparison with' should be 'in comparison to,' etc.). Please also revise the text of the manuscript, to better comply with the journal's style and template. References for the two equations must still be provided. 

Author Response

ANSWERS TO THE REVIEWER

The paper was revised according to all comments made. It is in much better shape now, but a certain revision must still be made to have it published - in the opinion of this reader.

Answer : The authors would like to thank the reviewer for his thorough comments. We have addressed all the specific comments as follows.

Figs. 1 and 4 are still unclear, they must be replaced with better resolution figs. Also, Fig. 4 is put together with a tab. This is not satisfactory,  please split them apart and renumber tabs. 

Answer : Done. Thank you for pointing this out.

Please remove from the title the notation IVOCT, as the readers may not be familiar with it. Instead, use 'Intravascular Optical Coherence Tomography' or, eventually, 'Intravascular OCT'. 

Answer : Done. Thank you for pointing this out.

English must be revised (e.g. 'in comparison with' should be 'in comparison to,' etc.). Please also revise the text of the manuscript, to better comply with the journal's style and template. References for the two equations must still be provided. 

Answer : Extensive editing for English was done. The references for the two equations are provided as suggested by the reviewer.

Reviewer 2 Report

After carefully checking the responses to each comment and changes accordingly made, I've found that the authors did not address all the previous comments appropriately. Even though some changes were satisfactorily made, I'm afraid to say I could not recommend for publication at this time with the current revised form.

Author Response

We would like to thank the reviewer for the thorough review. We think that our paper has merit, both in introducing an annotated intravascular OCT dataset in our community, and in developing and testing new CNN based techniques for classifying A-Lines. 

There are a number of further interventions in the discussion toning up the contributions of our paper. We hope that they add to the novelty of the paper and address up to a point the reviewer's comments.

Round 3

Reviewer 2 Report

No other further comments to the authors.