Noise Resilience in Dermoscopic Image Segmentation: Comparing Deep Learning Architectures for Enhanced Accuracy

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

The manuscript introduces no novel methods or significant improvements over existing techniques. The comparison of U-Net, SegAN, and MultiResUNet has been extensively studied in prior literature, offering little new insight into their performance under Gaussian noise.

 

The study's methodology, a cornerstone of rigorous research, must be fortified. The authors' explanation of the preprocessing steps and noise addition process needs to be revised, raising concerns about the results' reproducibility. A more robust methodology is essential to uphold the high research standards in this field.

 

The manuscript uses the ISBI 2017 dataset but does not justify why this dataset is appropriate for evaluating the robustness of segmentation frameworks against noise. More diverse datasets should have been included to validate the generalizability of the findings.

 

The literature review needs to be completed to cover recent profound advancements in learning-based segmentation techniques. Essential works that could provide context and depth to the study need to be included, making the review section inadequate.

 

The manuscript overemphasizes well-known techniques without contributing any novel insights. The discussion merely reiterates existing knowledge about U-Net, SegAN, and MultiResUNet without offering a fresh perspective or unique findings.

 

The discussion section needs more depth and critical analysis. The authors must adequately compare their results with previous studies or explore the potential reasons for the observed performance differences among the models.

 

The manuscript needs formatting issues and typographical errors that distract from the content. The figures and tables are not well-integrated into the text, making it difficult to follow the results and understand their significance.

 

It looks wrong "The detailed results are given in the Table 5.1, 5.2 and 5.3 ….."

 

The resolution of the figures could be better, and figures should be drawn more professionally.

 

 

Author Response

Dear Reviewer, thank you for your valuable comments on increasing the impact of our study.

Comment 1: The manuscript introduces no novel methods or significant improvements over existing techniques. The comparison of U-Net, SegAN, and MultiResUNet has been extensively studied in prior literature, offering little new insight into their performance under Gaussian noise.

Response 1: We have revised the literature review section by presenting the milestones of dermoscopic segmentation in the last decade. Even if these methods have been studied in lesion classification problems, no prior study evaluated their segmentation performance under Gaussian noise for dermoscopic images.

Comment 2: The study's methodology, a cornerstone of rigorous research, must be fortified. The authors' explanation of the preprocessing steps and noise addition process needs to be revised, raising concerns about the results' reproducibility. A more robust methodology is essential to uphold the high research standards in this field.

Response 2: We have revised the methodology section. As we used default parameters of U-Net, SegAN, and MultiResUNet (as given in Figure 1) we tried to minimize the theoretical aspects of DNN backbones in the methodology section.

Comment 3: The manuscript uses the ISBI 2017 dataset but does not justify why this dataset is appropriate for evaluating the robustness of segmentation frameworks against noise. More diverse datasets should have been included to validate the generalizability of the findings.

Response 3: In dermoscopic segmentation studies, both ISIC 2017 (also known as ISBI 2017) and  ISBI 2019 were used for different purposes. ISIC 2019 generally provides more advanced and detailed segmentation masks, specifically benefiting from higher resolution and greater precision in annotations. ISIC 2017 includes melanoma data but also covers a broader range of lesions, which might dilute the focus on melanoma-specific segmentation. In skin lesion segmentation, there are no studies with additive noise to ensure the quality of segmentation for melanomas using these datasets. We preferred ISBI Challenge 2017  - Skin Lesion Analysis Towards Melanoma Detection : Lesion Segmentation dataset in this study as it covers a broader range of lesions, which might dilute the focus on melanoma-specific segmentation. We focused on the noise resilience in melanoma segmentation. While there may not be direct references asserting ISIC 2017 as superior to ISIC 2019, ISIC 2019 offers more advanced features in image resolution. On the other hand, ISIC 2017 emphasized melanoma detection, providing valuable data for developing algorithms specifically aimed at identifying melanoma. Moreover, ISIC 2017 featured a wide range of skin lesions, which can be beneficial for developing models that generalize across different types of skin conditions ISIC 2019 reduced emphasis on a broader range of lesions. In future steps, we will address the problem by creating our database where we will locate the melanoma features through different color features such as texture and contour with a follow-up paradigm. We will compare the segmentation perfor-mance on ISIC 2019 and ISIC 2020 by generalize segmentation aspects of melanoma and their features. Therefore, melanoma prediction would serve to explore the spa-tial characteristics of skin lesions.

Comment 4: The literature review needs to be completed to cover recent profound advancements in learning-based segmentation techniques. Essential works that could provide context and depth to the study need to be included, making the review section inadequate.

Response 4: We have revised the literature reviews by adding new DNN architectures in dermoscopic segmentation

Comment 5: The manuscript overemphasizes well-known techniques without contributing any novel insights. The discussion merely reiterates existing knowledge about U-Net, SegAN, and MultiResUNet without offering a fresh perspective or unique findings.

Response 5: We have revised the discussion and conclusion sections to highlight our motivation for noise resilience in the image segmentation task which is not a common field in dermoscopic imaging.

Comment 6: The discussion section needs more depth and critical analysis. The authors must adequately compare their results with previous studies or explore the potential reasons for the observed performance differences among the models.

Response 6: As ISBI 2017 dataset offers a broad aspect of image features in dermoscopic segmentation we have revised our findings by reviewing the pros & cons of the dataset and DNNs

Comment 7: The manuscript needs formatting issues and typographical errors that distract from the content. The figures and tables are not well-integrated into the text, making it difficult to follow the results and understand their significance.

Response 7: We checked and corrected the errors

Comment 8: It looks wrong "The detailed results are given in the Table 5.1, 5.2 and 5.3 ….."

Response 8: We corrected this error

Comment 9: The resolution of the figures could be better, and figures should be drawn more professionally.

Response 9: We added the high-resolution version of Figure 1 attached to our manuscript

 

Reviewer 2 Report

Comments and Suggestions for Authors

The followings are my observations and suggestions while reviewing the paper.

1) The topic says about Assessment of Gaussian Noise, which seems unclear how they used it for segmentation, many advanced Gaussian Noise filter are already existing, why they just put this name not justified in this paper.

2) Not Novel. The paper should clearly explain what are the innovative approach they apply and if something they achieve which outperform the existing approaches, but in this paper, there is no proper analysis of the current approaches and their comparison details are missing.

3) Experimental methodology needs more clear and consistent integrity in terms of presenting them in a paper and clear functional flow. But many details are missing and inconsistent throughout the paper. 

4) Many self citation is observed, like no 48 etc.  Delete all the self citations from the paper.

Comments on the Quality of English Language

The followings are my observations and suggestions while reviewing the paper.

1) The topic says about Assessment of Gaussian Noise, which seems unclear how they used it for segmentation, many advanced Gaussian Noise filter are already existing, why they just put this name not justified in this paper.

2) Not Novel. The paper should clearly explain what are the innovative approach they apply and if something they achieve which outperform the existing approaches, but in this paper, there is no proper analysis of the current approaches and their comparison details are missing.

3) Experimental methodology needs more clear and consistent integrity in terms of presenting them in a paper and clear functional flow. But many details are missing and inconsistent throughout the paper. 

4) Many self citation is observed, like no 48 etc.  Delete all the self citations from the paper.

Author Response

Dear Reviewer, thank you for your valuable comments on increasing the impact of our study.

Comment 1: The topic says about Assessment of Gaussian Noise, which seems unclear how they used it for segmentation, many advanced Gaussian Noise filter are already existing, why they just put this name not justified in this paper.

Response 1: We have changed our title to reflect this.

Comment 2: Not Novel. The paper should clearly explain what are the innovative approach they apply and if something they achieve which outperform the existing approaches, but in this paper, there is no proper analysis of the current approaches and their comparison details are missing.

Response 2: We have revised the literature review section by presenting the milestones of dermoscopic segmentation in the last decade. Even if different approaches have been studied in lesion classification problems, there is no prior study that evaluated their segmentation performance under Gaussian noise for dermoscopic images.

In the literature, we noticed that researchers focused on segmentation tasks to generate high scores in challenges. However, there are no studies in the field of melanoma segmentation to evaluate the performance of these backbones.

Additive noise is a common technique used in data augmentation and robustness testing in machine learning, including in medical image segmentation tasks. It’s worth noting that specific studies applying additive noise directly to ISIC segmentation datasets are less common. Researchers often apply noise in a general context of data augmentation or robustness testing and may not always detail specific datasets in the context of noise. Data augmentation is preferred in image classification problems due to imbalance challenges.

Comment 3: Experimental methodology needs more clear and consistent integrity in terms of presenting them in a paper and clear functional flow. But many details are missing and inconsistent throughout the paper.

Response 3: We have revised the methodology, discussion, and conclusion sections.  

Comment 4: Many self citation is observed, like no 48 etc.  Delete all the self citations from the paper.

Response 4: We removed the self-citation

Reviewer 3 Report

Comments and Suggestions for Authors

The motivation of this study is to evaluate the effectiveness of deep learning methods for early detection and monitoring of skin lesions. Early detection of skin cancer is critical for patient health, and automatic segmentation methods reduce the workload of dermatologists and accelerate the diagnostic process. This study compares three DL architectures (U-Net, SegAN, and MultiResUNet) for segmentation of skin lesions. These architectures were used to segment lesions in dermoscopic images and their performances were evaluated at various noise levels. The study shows that SegAN and MultiResUNet are more robust to noise and provide high classification accuracy. In this sense, the study includes features that will contribute to the literature. References are sufficient. The discussion section of the study is sufficient, and the results are satisfactory. The mathematical background and model are well implemented. However, I kindly recommend that the following gentle suggestions be taken into consideration.

 

1- Tests should be conducted on more data sets and different types of noise to increase the generalization ability of the model.

2- Studies on skin lesion segmentation of new DL architectures can be discussed further in the discussion section.

3- Figure 1 should be more readable.

4- Figures 6, 8 and Table1 should be explained in more detail.

Author Response

Dear Reviewer, thank you for your valuable comments on increasing the impact of our study.

Comment 1: Tests should be conducted on more data sets and different types of noise to increase the generalization ability of the model.

Response 1: In dermoscopic segmentation studies, both ISIC 2017 (also known as ISBI 2017) and  ISBI 2019 were used for different purposes. ISIC 2019 generally provides more advanced and detailed segmentation masks, specifically benefiting from higher resolution and greater precision in annotations. ISIC 2017 includes melanoma data but also covers a broader range of lesions, which might dilute the focus on melanoma-specific segmentation. In skin lesion segmentation, there are no studies with additive noise to ensure the quality of segmentation for melanomas using these datasets. We preferred ISBI Challenge 2017  - Skin Lesion Analysis Towards Melanoma Detection: Lesion Segmentation dataset in this study as it covers a broader range of lesions, which might dilute the focus on melanoma-specific segmentation. We focused on noise resilience in melanoma segmentation. While there may not be direct references asserting ISIC 2017 as superior to ISIC 2019, ISIC 2019 offers more advanced features in image resolution. On the other hand, ISIC 2017 emphasized melanoma detection, providing valuable data for developing algorithms specifically aimed at identifying melanoma. Moreover, ISIC 2017 featured a wide range of skin lesions, which can be beneficial for developing models that generalize across different types of skin conditions ISIC 2019 reduced emphasis on a broader range of lesions. In future steps, we will address the problem by creating our database where we will locate the melanoma features through different color features such as texture and contour with a follow-up paradigm. We will compare the segmentation performance on ISIC 2019 and ISIC 2020 by generalize segmentation aspects of melanoma and their features. Therefore, melanoma prediction would serve to explore the spatial characteristics of skin lesions.

In the literature, we noticed that researchers focused on segmentation tasks to generate high scores in challenges. However, there are no studies in the field of melanoma segmentation to evaluate the performance of these backbones.

Additive noise is a common technique used in data augmentation and robustness testing in machine learning, including in medical image segmentation tasks. It’s worth noting that specific studies applying additive noise directly to ISIC segmentation datasets are less common. Researchers often apply noise in a general context of data augmentation or robustness testing and may not always detail specific datasets in the context of noise. Data augmentation is preferred in image classification problems due to imbalance challenges.

Comment 2: Studies on skin lesion segmentation of new DL architectures can be discussed further in the discussion section.

Response 2: We have revised the literature review, methodology discussion, and conclusion sections to highlight the pros & cons of U-Net, SegAN, and MultiResUNet over other techniques in dermoscopic  segmentation

Comment 3: Figure 1 should be more readable.

Response 3: We added the high-resolution version of Figure 1 attached to our manuscript

Comment 4: Figures 6, 8 and Table 1 should be explained in more detail.

Response 4: We have revised the results section and detailed Figures and Tables.

Round 2

Reviewer 2 Report

Comments and Suggestions for Authors

The paper improves a lot due to the review process. The authors addresses most of the review remarks very well. I do not have any further objections. This paper could be accepted.