Ultrasound Intima-Media Complex (IMC) Segmentation Using Deep Learning Models

Round 1

Reviewer 1 Report

In the article, the authors mention several networks, such as the DeepCrack network, PidiNet, and CCTrans, but they don't provide a clear explanation of how these networks are combined. It's unclear whether the approach is end-to-end learning or an ensemble learning approach. The methodology overall is sparse and lacks consistency, making it difficult for readers to follow along. To improve the article, the authors should revise their methodology section to provide more details on how the networks are integrated and to ensure consistency throughout the article.

Author Response

We'd like to thank the editor and referees for your helpful comments on this manuscript.  We greatly appreciate the input, and we've enclosed a revised version that will hopefully address the concerns in the reviews, to which we detail our responses below. The corresponding changes and refinements made in the revised paper are summarized in our response below. The reviewers’ comments are in black and the authors' responses are the red font while the previous response are in blue font.

Reviewer 1:

 C1.In the article, the authors mention several networks, such as the DeepCrack network, PidiNet, and CCTrans, but they don't provide a clear explanation of how these networks are combined. It's unclear whether the approach is end-to-end learning or an ensemble learning approach. The methodology overall is sparse and lacks consistency, making it difficult for readers to follow along. To improve the article, the authors should revise their methodology section to provide more details on how the networks are integrated and to ensure consistency throughout the article.

R1. Thank you for reviewing our paper as well as for your helpful comments on this manuscript. We did not combine the networks, we tested each network separately to see how well it performs in our data, so each network is trained as an end-to-end approach. We report the result of each network separately as shown in Table 1. We improved the section considering your notice.

Reviewer 2 Report

1.       Need detailed explanation of the preprocessing and postprocessing steps.

2.       More motivation/context regarding the application side of it, particularly on the aspects that make this technique particularly suited for medical application scenarios, and how it would be applied in real scenarios. These aspects could additionally be supported with some related work context.

3.       Authors should explain the reason why they choose these methods. What are the limitations of this work? How can the rigor of this work be demonstrated?

4.       Results need more explanations. Additional analysis is required at each experiment to show the its main purpose.

5.       The author needs to check the formatting and writing specifications of the manuscript more carefully, which contains some relatively simple errors, such as, Figure 3. “(b) Erosion result” should be changed to “(c) Erosion result”.

6.       The title of the manuscript is "Deep learning-based Ultrasound IMC Segmentation and cIMT Measurement", in which the authors focus more on "Ultrasound IMC Segmentation" and less on "cIMT Measurement". Ultrasound IMC Segmentation", with little mention of "cIMT Measurement". The title of the manuscript does not exactly match the content of the study.

Author Response

We'd like to thank the editor and referees for your helpful comments on this manuscript.  We greatly appreciate the input, and we've enclosed a revised version that will hopefully address the concerns in the reviews, to which we detail our responses below. The corresponding changes and refinements made in the revised paper are summarized in our response below. The reviewers’ comments are in black and the authors' responses are the red font while the previous response are in blue font.

 

Reviewer 2:

C1.  Need a detailed explanation of the preprocessing and postprocessing steps.

R1. Thank you for the comments. Our experiment does not include preprocessing step. The postprocessing step which is a morphological erosion is further explained in the postprocessing section on page 6.

C2.More motivation/context regarding the application side of it, particularly on the aspects that make this technique particularly suited for medical application scenarios, and how it would be applied in real scenarios. These aspects could additionally be supported with some related work context.

R2. Thank you for the comment. One of the effects of carotid artery stenosis, the accumulation of plaque on the carotid artery, is ischemic stroke. If the stenosis is detected early and the amount of plaque is determined, the problem can be addressed immediately. The proposed method can help to simplify the detection by segmenting the carotid artery region and then computing the Common Carotid Intima-Media Thickness (CIMT) to diagnose carotid artery stenosis. The proposed method used Carotid Intima-Media Thickness (CIMT) method using machine learning like in [1]. But using deep learning techniques by segmenting the carotid artery region from Ultrasound data, the detection and the classification of carotid artery stenosis can be more simplified for experts to do the diagnosis.

[1]. Meiburger, K. M., Marzola, F., Zahnd, G., Faita, F., Loizou, C. P., Lainé, N., ... & Molinari, F. (2022). Carotid Ultrasound Boundary Study (CUBS): Technical considerations on an open multi-center analysis of computerized measurement systems for intima-media thickness measurement on common carotid artery longitudinal B-mode ultrasound scans. Computers in Biology and Medicine, 144, 105333.

 

C3. Authors should explain the reason why they choose these methods. What are the limitations of this work? How can the rigor of this work be demonstrated?

R3. Thank you for the comments. Due to the complexity of Ultrasound images, we trained many deep learning architectures that are based on multi-scale representation on CUBS dataset. The most accurate models are PidiNet, and DeepCrack which are used for edge detection which is a close subject. Also, we adopted CCtrans model used for crowd counting and using it for segmenting carotid artery regions because it's a multiscale model using transformers. These three models are the most accurate models (after being trained and tested on CUBS dataset). We exclude UNet SegNet and two other architectures. Also, We attempted to apply Self-ONN technique to DeepCrack model which improve the segmentation results. The implemented models and the experiments can be uploaded as supplementary files.

C4. Results need more explanations. Additional analysis is required at each experiment to show its main purpose.

R4.  Thank you for the comments. We attempted to add more explanation in all the sections of this paper. The added analysis is in blue font in the manuscript.

C5.  The author needs to check the formatting and writing specifications of the manuscript more carefully, which contains some relatively simple errors, such as, Figure 3. “(b) Erosion result” should be changed to “(c) Erosion result”.

R5. Thank you for the comments. We revised the manuscript and corrected all the typos.

C6.  The title of the manuscript is "Deep learning-based Ultrasound IMC Segmentation and cIMT Measurement", in which the authors focus more on "Ultrasound IMC Segmentation" and less on "cIMT Measurement". Ultrasound IMC Segmentation", with little mention of "cIMT Measurement". The title of the manuscript does not exactly match the content of the study

R6. Thank you for the comments. As suggested, the title of the manuscript has been changed to “ Ultrasound Intima-Media Complex (IMC) Segmentation Using Deep Learning Models”

Reviewer 3 Report

This manuscript compares the intima-media complex (IMC) segmentation performance of four deep learning models. However, the author's description and result analysis of these algorithms are relatively thin, and it is recommended to publish after major revisions.

1. It is suggested to improve English writing. There are lots of writing errors such as capitalization errors, singular and plural errors, spelling errors, punctuation errors, grammatical errors, etc.(For example:line 110,111,115-116,129,183,233,etc)

2. The division of data should be given. What is the proportion of the training set, validation set, and test set?

3. It is suggested to modify Figure 4 since the Precision-Recall curve in the third graph is incomplete.

4. What are the number of parameters and training times of each deep learning model? It is important because models of different sizes are difficult to compare.

5. Line 198, why the segmented thickness is generally fatter than the ground truth, can you try to explain that?

6. Line 241, are you sure that the reason for the better performance of transformer-based models is lacking enough data? Because it is often the case that the transformer needs large data to be fed if you want to get good performance. 

7. Please refer to the format of the bibliography according to the requirements of the journal and use the correct and consistent format.

 

Author Response

We'd like to thank the editor and referees for your helpful comments on this manuscript.  We greatly appreciate the input, and we've enclosed a revised version that will hopefully address the concerns in the reviews, to which we detail our responses below. The corresponding changes and refinements made in the revised paper are summarized in our response below. The reviewers’ comments are in black and the authors' responses are the red font while the previous response are in blue font.

 

Reviewer 3:

This manuscript compares the intima-media complex (IMC) segmentation performance of four deep learning models. However, the author's description and result analysis of these algorithms are relatively thin, and it is recommended to publish after major revisions.

Thank you for giving us a chance to modify the paper. In the sequel and in the revised manuscript, we address the concerns and the issues raised.

C1. It is suggested to improve English writing. There are lots of writing errors such as capitalization errors, singular and plural errors, spelling errors, punctuation errors, grammatical errors, etc.(For example:line 110,111,115-116,129,183,233,etc)

R1. The paper has gone through the MDPI English editing tool.

C2. The division of data should be given. What is the proportion of the training set, validation set, and test set?

R2. We used 80% of the data for training and 20% for testing. We added this to the Dataset and evaluation metrics section.

C3. It is suggested to modify Figure 4 since the Precision-Recall curve in the third graph is incomplete.

R3. We removed the third graph as the information is also explained in Table 1.

C4. What are the number of parameters and training times of each deep learning model? It is important because models of different sizes are difficult to compare.

R4. Thank you for the comment. The training parameters have been added for each model. While the training time is added also for each model using frame per second (FLOPs).

C5. Line 198, why the segmented thickness is generally fatter than the ground truth, can you try to explain that?

R5. Image segmentation algorithms typically rely on edge detection and thresholding techniques to separate regions of interest from the background. However, these techniques can be affected by image noise, leading to the detection of false edges and the inclusion of noise as part of the segmented object. Additionally, image segmentation algorithms may also introduce some level of smoothing or blurring of the image, which can further contribute to the fattening effect. This smoothing operation can cause the boundaries of the segmented object to become slightly blurred and more diffuse, resulting in a larger area being assigned to the object than is present in the ground truth. We added this explanation in the last paragraph of the Evaluation section.

C6. Line 241, are you sure that the reason for the better performance of transformer-based models is lacking enough data? Because it is often the case that the transformer needs large data to be fed if you want to get good performance.

R6. Thank you for the comment. Sure all the deep-learning-based models including transformers-based models need large-scale datasets for better performance.

C7. Please refer to the format of the bibliography according to the requirements of the journal and use the correct and consistent format.

R7. Thank you for the comments. We followed the same bibliography format for the revised manuscript based on the published papers in the journal.

Round 2

Reviewer 1 Report

N/A

Reviewer 2 Report

 

The manuscript is sound, well written, and has a much more sense compared to previous version. My opinion is that it would be interesting to the Applied Sciences readership and thus suggest to consider it for publication.

 

Reviewer 3 Report

The revised manuscript can be accepted in present form.