Deep Learning Automation of Kidney, Liver, and Spleen Segmentation for Organ Volume Measurements in Autosomal Dominant Polycystic Kidney Disease

Round 1

Reviewer 1 Report

Very  interesting manuscript

The subject of the manuscript is interesting, but its overall quality must be improved. I have made a few comments and corrections/suggestions in the pdf file that I hope can be considered as a positive contribution to the manuscript. Several points need to be revised before the manuscript is resubmitted:

Formatting style

1-There are several inaccuracies in the manuscript format. Authors should carefully check the entire text of the manuscript, as some spaces between letters / numbers and commas or periods are lost. In other cases, some spaces must be removed;

2-Avoid the use of possessive pronouns like "We"(Page4, page 6….) and try to replace them with passive form

3-Check the numbering of sections and sub-sections and professional editing is required (especially section methods and materials)

5-Check the quality of figures: resolution should be 300 dpi at least

Technical points

1-In the introduction section, expand your state of art based on related work which used supervised or not supervised learning for the segmentation approach?

2- specify which type of algorithm used (in the results section for Deep learning automation of kidney, liver, and spleen segmentation for organ volume measurements in Autosomal Dominant Polycystic Kidney Disease?

3-In the section methods and materials: specify the protocol of MR Imaging exam before and after segmentation and avoid citing the DICOM tags in the section of MRI imaging

4-(Table 2. A. Model accuracy on External validation (n=30)) and (Table 4. Standard Deviations of organ volumes): add some research results to compare your accuracy model with other works  

5- in the section Cropping Artifact:

“Replacing the cropping with patch-based training eliminated the artifact but also resulted in decreased overall performance.   After experimenting with using less cropping we found a robust outcome

with excellent model performance over a wide spectrum of cases ….”: it’s not clear how you reduce the artifact and obtain a high performance of the model? Have you used the patch to generate 3D model after the segmentation?

6- You did not discuss the limits of your approach: do you have an over fitting or under fitting problem? the implication of your approach within the clinical routine and how you propose the storage within the Pacs system?

Author Response

Very interesting manuscript.  Thank you for these helpful comments.

The subject of the manuscript is interesting, but its overall quality must be improved. I have made a few comments and corrections/suggestions in the pdf file that I hope can be considered as a positive contribution to the manuscript. Several points need to be revised before the manuscript is resubmitted:

Formatting style

1-There are several inaccuracies in the manuscript format. Authors should carefully check the entire text of the manuscript, as some spaces between letters / numbers and commas or periods are lost. In other cases, some spaces must be removed;  done

2-Avoid the use of possessive pronouns like "We"(Page4, page 6….) and try to replace them with passive form  done

3-Check the numbering of sections and sub-sections and professional editing is required (especially section methods and materials)  Numbering of each section has been corrected.

5-Check the quality of figures: resolution should be 300 dpi at least   Figures are all now 300 dpi or greater.

Technical points

1-In the introduction section, expand your state of art based on related work which used supervised or not supervised learning for the segmentation approach?  We have expanded the introduction to include related work on supervised learning for this approach including the addition of Table 1 summarizing the recent literature on deep learning methods for organ segmentaion in ADPKD. The unsupervised methods are not yet being investigated. 

2- specify which type of algorithm used (in the results section for Deep learning automation of kidney, liver, and spleen segmentation for organ volume measurements in Autosomal Dominant Polycystic Kidney Disease?   The algorithm is now specified, see Results, 3.1. Model Accuracy, first 2 sentences. 

3-In the section methods and materials: specify the protocol of MR Imaging exam before and after segmentation and avoid citing the DICOM tags in the section of MRI imaging  Dicom tags have been moved to Supplemental Table S1 and the extaneous DICOM tags have been eliminated.  It is now clarified in 2.2. MR Imaging that the MR Imaging data were acquired with the routine clinical protocol without any special preparation and the pulse sequences used in the routine clinical protocol are listed : Axial T2, Coronal T2, Axial 3D spoiled gradient echo with fat suppression or Dixon fat-water separation, axial steady state free precession (SSFP), coronal SSFP and axial diffusion weighted imaging. 

For external and prospective cases, the procedures following model inference are listed in 2.9. External and Prospective Validation.  These include saving the corrected labels on the deep learning server within the PACS firewall and including the organ volume measurements in the MRI reports.

4-(Table 2. A. Model accuracy on External validation (n=30)) and (Table 4. Standard Deviations of organ volumes): add some research results to compare your accuracy model with other works  Comparisons to other published results are now added, see Table 1 and 3.2 Model Accuracy, last 3 sentences of the 1st paragraph.  Table 4 showing improved reproducibility with model-assisted annotation is a benefit of deep learning which has not been demonstrated by any of the prior papers on this topic.

5- in the section Cropping Artifact:  

“Replacing the cropping with patch-based training eliminated the artifact but also resulted in decreased overall performance.   After experimenting with using less cropping we found a robust outcome with excellent model performance over a wide spectrum of cases ….”: it’s not clear how you reduce the artifact and obtain a high performance of the model? Have you used the patch to generate 3D model after the segmentation?  We realize that our discussion of the cropping artifact was confusing because it was partly in the methods, 2.8. and partly in the results 3.1.  It has now been consolidated into the methods section 2.8.  Our use of the patch terminology may also have been confusing and this has now been deleted. The inclusion of cropping for training and removal of cropping for validation and running the model inference are now described in greater detail.  We now mention 3D modeling as an opportunity for further improvement in the discussion.

6- You did not discuss the limits of your approach: do you have an over fitting or under fitting problem? the implication of your approach within the clinical routine and how you propose the storage within the Pacs system?  We now discuss the limitations of our approach, see discussion, 8th paragraph. We now discuss that under fitting and overfitting are not an issue because of the excellent performance on training, prospective and external data. Storage on a dedicated deep learning sever behind the PACS firewall is also discussed, see discussion, end of 2^nd paragraph..

Author Response File: Author Response.pdf

Reviewer 2 Report

This article described by Dr. Sharbatdaran provides a deep learning method for assessment of ADPKD.  Organ Volume measurement of enlarged bilateral kidneys and liver is very important factor in the medical care of ADPKD patients. The authors present an interesting and well-written article.
I have only few comments:
1. I think volume measurement of spleen is not important in the medical care of ADPKD patients. Please mention the importance of spleen volume measurement.
2. The same sentence is duplicated on lines 380 to 389.
3. Somatostatin has not yet been proven to be effective against liver complications and should be avoid.

Author Response

This article described by Dr. Sharbatdaran provides a deep learning method for assessment of ADPKD.  Organ Volume measurement of enlarged bilateral kidneys and liver is very important factor in the medical care of ADPKD patients. The authors present an interesting and well-written article.
I have only few comments:  The authors thank reviewer 2 for this positive comment and the other helpful comments.

1. I think volume measurement of spleen is not important in the medical care of ADPKD patients. Please mention the importance of spleen volume measurement. It is now indicated in the introduction that splenomegaly is a feature of ADPKD, but the significance of splenomegaly in ADPKD is not fully established, it can contribute to mass effect in the left upper abdomen potentially causing early satiety and the challenge of differentiating among other causes of splenomegaly.
2. 
   The same sentence is duplicated on lines 380 to 389. Thank you. This is now fixed.
3. 
   Somatostatin has not yet been proven to be effective against liver complications and should be avoid. The comment about somatostatin has been deleted.

Reviewer 3 Report

The authors take up the issue of assessing the volume of abdominal organs in patients with ADPKD. This issue is of unequivocally significant clinical importance. The project has been planned carefully and the results are verified reliably and comprehensively.

However, the lack of work is a comparison with other programs proposed in the literature to assess the volume of internal organs, especially kidneys. In the last year, there have been more than 20 reports in this area - only in the Pubmed database.

Why would a radiologist decide to choose the method proposed by the authors? There are no comments in the Discussion section, nor in the entire article.

Author Response

The authors take up the issue of assessing the volume of abdominal organs in patients with ADPKD. This issue is of unequivocally significant clinical importance. The project has been planned carefully and the results are verified reliably and comprehensively. The authors thank reviewer 2 for this positive comment and the other helpful comments.

1. However, the lack of work is a comparison with other programs proposed in the literature to assess the volume of internal organs, especially kidneys. In the last year, there have been more than 20 reports in this area - only in the Pubmed database. We now provide a more detailed discussion of how our work compares to others and including a new Table 1 showing results from all other deep learning papers measuring organ volumes in ADPKD patients that we could find.

 

2. Why would a radiologist decide to choose the method proposed by the authors? There are no comments in the Discussion section, nor in the entire article. There is now a discussion of how this is working in clinical practice as well as the benefits to the radiologist, see discussion paragraph 3.