Radiomic Machine-Learning Analysis of Multiparametric Magnetic Resonance Imaging in the Diagnosis of Clinically Significant Prostate Cancer: New Combination of Textural and Clinical Features

Round 1

Reviewer 1 Report

Overall, the paper is well put together, highly innovative, with a thorough research protocol.

The introduction could potentially benefit from the following suggestions:

-        I would advise the authors to include the definition of clinically significant and insignificant prostate cancer.

-        I suggest that the authors include the percentages of csPCa cited for each PI-RADS category lesion (3, 4 and 5), detected through transrectal prostate biopsy.

            The Materials and methods section could be improved by:

-        Explaining what ‘definite boundaries lesions’ mean.

-        Explaining which location of the lesion was considered to preclude the segmentation.

-        Explaining more accurately if the region of interest was contoured by a circle or the margins were refined according to the lesion’s shape.

            The Results section elaborates on valuable findings, supported by thorough statistical analysis. However, it is not entirely clear the prostatic region of each biopsied lesion (peripheral zone, transitional zone, anterior fibromuscular stroma). Moreover, adding the mean lesion diameter, average millimeters of PCa for the total length of the fragments and the number of cases diagnosed through systematic and targeted sampling alone.

            Finally, the Discussion section could be improved by adding a table summarizing the previously published papers targeting a similar topic. Additionally, discussing previous studies that have elaborated radiomics-based nomograms would be of great benefit for this paper.        

Author Response

Overall, the paper is well put together, highly innovative, with a thorough research protocol.

We thank the reviewer for agreeing to review our paper.

 

The introduction could potentially benefit from the following suggestions:

 

• I would advise the authors to include the definition of clinically significant and insignificant prostate cancer.

We thank the reviewer for the suggestion.

The definition of clinically significant prostate cancer has no general consensus and is a dynamic process initiated many years ago before the screening era, when a great proportion of prostate cancers were diagnosed at the time of autopsy as indolent tumors. Nonetheless, the actual most shared definition of clinically significant prostate cancer is a histopathology ISUP grade ≥ 2 and/or volume ≥ 0.5 cc and/or have extra prostatic extension. The introduction section was improved accordingly.

 

• I suggest that the authors include the percentages of csPCa cited for each PI-RADS category lesion (3, 4 and 5), detected through transrectal prostate biopsy.

We thank the reviewer for the suggestion.

The percentages of CS PCa for each PI-RADS category detected through transrectal prostate biopsy were reported into table 1.

 

   The Materials and methods section could be improved by:

-        Explaining what ‘definite boundaries lesions’ mean.

We thank the reviewer for the comment.

Definite boundaries lesions means that the radiologist could easily define the margins and shape of the lesions and contouring all the lesion in the region of interest (ROI) without including a high proportion of non-suspicious prostate tissue, otherwise this would compromise the accuracy of the radiomic analysis. This section was improved accordingly.

 

-        Explaining which location of the lesion was considered to preclude the segmentation.

We thank the reviewer for the comment.

One of the key features of MRI is tissue contrast that allows to distinguish different tissues and potential pathological lesions. In manual and automatic segmentation of MRI, largest segmentation inaccuracies are typically located in the most basal and apical slices due to low tissue contrast ratios. The manuscript was improved accordingly.

 

-        Explaining more accurately if the region of interest was contoured by a circle or the margins were refined according to the lesion’s shape.

We thank the reviewer for the comment.

OsiriX DICOM viewer software allowed the radiologist to delineate ROIs using either a circle of refining margins according to lesion’s shape if necessary. The manuscript was improved accordingly.

 

 

 The Results section elaborates on valuable findings, supported by thorough statistical analysis. However, it is not entirely clear the prostatic region of each biopsied lesion (peripheral zone, transitional zone, anterior fibromuscular stroma).

We thank the reviewer for the comment.

As described in the Methods section, prostate biopsies were performed as MRI targeted TRUS fusion biopsy in addition to standard 12-core systematic sampling (6 cores for each lobe, from the base to the apex, including peripheral lateral and para-median zone). Moreover, 2 to 4 cores from the MRI-targeted lesion were collected in addition to standard 12-core systematic sampling. Biopsy cores from the transitional zone were collected if the MRI identified a suspicious lesion, otherwise systematic random sampling included the peripheral lateral and para-median zone. As stated in the limitations of the study, for targeted lesion, we did not distinguish between peripheral and transitional PCa, instead we focused on peripheral PCa assuming that transitional PCa, having low malignant potential, would be included in the non-clinically significant PCa.

 

Moreover, adding the mean lesion diameter, average millimeters of PCa for the total length of the fragments and the number of cases diagnosed through systematic and targeted sampling alone.

We thank the reviewer for the interesting observation.

Even if adding the mean lesion diameter, average millimetres of PCa for the total length of the fragments and the number of cases diagnosed through systematic and targeted sampling alone could add significant information to the study, these information may be out of the endpoints of our study (to develop a non-invasive radiomic tool for the prediction of CS PCa). In future perspective studies, this could be a secondary endpoint as to externally validate our radiomic tool.

 

Finally, the Discussion section could be improved by adding a table summarizing the previously published papers targeting a similar topic. Additionally, discussing previous studies that have elaborated radiomics-based nomograms would be of great benefit for this paper. 

We thank the reviewer for the comment.

As stated by the reviewer, adding a table summarizing the previously published papers targeting a similar topic will for sure improve the quality of the discussion section by the way we think that this could be out of the scope of our manuscript. In case of a systematic or narrative review, summarizing previous works targeting the same topic could add value to the paper, but being this an original article we think that the discussion section should focus more the differences (as pros and cons) between our work and those previously published.

Regarding the discussion about previous studies that elaborated radiomic-based nomograms, this section was improved accordingly.

Reviewer 2 Report

This is original research article which address important aspect of the diagnostic pathway of prostatic acncer. Despite recent advances in mpMRI in order to improve detecttion of clinically significant prostate cancer (CS PCa), its interpretation remais controversial due to high inter-observe variability. Artificial itelligence (AI) assisted tools are expected to improve inter-observer variations. Therefore, this study is one of the first investigations building radiomics based computational model to improve subjective interpretation of MRI images detecting significant PCa. The authors confidently demonstrate that that proposed AI assisted feature analysis was able tp predict Gleason Score >7 with an accuracy of 83.5% and resulting AUC of 80.4%.    

The authors may be congratulated with impressive results of the study, though they are based on relatively small sample size and lack external validations. High detection accuracy  may be also attributed to selected features. DRE-positive PCa patients usually have huge cancer lesions; they do not need multiple/targeted biopsies and therefore may distort results of radiomic analysis 

Author Response

This is original research article which address important aspect of the diagnostic pathway of prostatic acncer. Despite recent advances in mpMRI in order to improve detecttion of clinically significant prostate cancer (CS PCa), its interpretation remais controversial due to high inter-observe variability. Artificial itelligence (AI) assisted tools are expected to improve inter-observer variations. Therefore, this study is one of the first investigations building radiomics based computational model to improve subjective interpretation of MRI images detecting significant PCa. The authors confidently demonstrate that that proposed AI assisted feature analysis was able tp predict Gleason Score >7 with an accuracy of 83.5% and resulting AUC of 80.4%.   

We thank the reviewer for agreeing to review our paper and for his/her comments.

 

The authors may be congratulated with impressive results of the study, though they are based on relatively small sample size and lack external validations. High detection accuracy  may be also attributed to selected features. DRE-positive PCa patients usually have huge cancer lesions; they do not need multiple/targeted biopsies and therefore may distort results of radiomic analysis 

We thank the reviewer for the comment.

We acknowledge the relatively small sample size and the lack of an external validation, in fact in the limitations (page 9, line 311-312) we stated that this “was a single-center data radiomic analysis and future works will need to be performed to get an external validation”.

Even if positive DRE patients usually have huge cancer lesions, DRE remains of subjective interpretation. Nonetheless, to select the most effective features in the most objective way and avoiding selection/information biases, all the artificial intelligence features and clinical features were considered and a Wrapper feature selection was adopted, using a Random Forest algorithm with a Best First search (page 5, line 202-205).

Prostate biopsy cores were collected as for standard clinical practice (2-4 for the targeted lesions and 12-core standard sampling for the two prostatic lobes). That was not a study need, instead it was evidence- and guidelines-based clinical practice. Furthermore, multiple sampling consented us to provide with higher accuracy prostatic tissue from lesions and to compare with peripheral benign tissue. Thus, our radiomic tool can be considered more generalizable. Nonetheless, this is a very good point that the reviewer suggested and further studies could focus only on targeted lesions in order to compare the accuracy with this tool.

Reviewer 3 Report

I reviewed in detail the study titled “Radiomic machine-learning analysis of multiparametric Magnetic Resonance Imaging in the diagnosis of clinically significant prostate cancer: new combination of textural and clinical features”.

The authors aimed to develop a radiomic tool for the prediction of clinically significant prostate cancer in their study. 91 patients were selected for the study. 487 features were used for the prediction using the random forest. A feature selection algorithm was used to identify the most predictable features. However, the researchers stated that they selected 9 features with 10-fold cross-validation. RF has no feature selection. I think this is a typo. The relevant part of the summary needs to be corrected. Authors should add a paragraph about the organization of the article at the end of the Introduction section. The authors should also mention the novelty of the study and its contribution to the literature in this section. Presenting the performance evaluation metrics in a tabular form in the study will increase the readability of the article. Similar studies on the subject were not discussed in the study. A subtitle in the form of a literature review should be added to the study. The study is remarkable in general, if the mentioned deficiencies are addressed, the quality of the study will increase.

Author Response

I reviewed in detail the study titled “Radiomic machine-learning analysis of multiparametric Magnetic Resonance Imaging in the diagnosis of clinically significant prostate cancer: new combination of textural and clinical features”.

 

The authors aimed to develop a radiomic tool for the prediction of clinically significant prostate cancer in their study. 91 patients were selected for the study. 487 features were used for the prediction using the random forest. A feature selection algorithm was used to identify the most predictable features.

We thank the reviewer for agreeing to review our paper.

 

 

However, the researchers stated that they selected 9 features with 10-fold cross-validation. RF has no feature selection. I think this is a typo. The relevant part of the summary needs to be corrected. 

We thank the reviewer for the comment.

As stated in the methods section, in order to select the most effective features in the most objective way and avoiding selection/information biases, all the artificial intelligence features and clinical features were considered and a Wrapper feature selection was adopted, using a Random Forest algorithm with a Best First search (page 5, line 202-205). The relevant part of the summary has been corrected accordingly.

 

Authors should add a paragraph about the organization of the article at the end of the Introduction section. The authors should also mention the novelty of the study and its contribution to the literature in this section.

We thank the reviewer for the comment.

The article is already organized in paragraphs and sub-paragraphs. We think that adding a dedicated paragraph about the organization of the article may confuse readers.

The novelty of the study and its contribution to the literature were already displayed in the discussion section.

“By extracting the same number of features from both T2w and ADC acquisitions, almost 500 descriptors were analysed. Hopefully this provided robustness to the results and made the classifier able to build a more generalised interpreting model of the data. Some author recently tried to develop and validate nomograms for the prediction of CS PCa. Nevertheless, these nomograms focused on a specific population, as gray-zone PSA level (4-10 ng/ml) or PI-RADS 3 lesions. Instead of targeting our work only to a pre-specified category, we avoided to super select our population in order to obtain data from a real-life cohort and a radiomic tool that could be more generalizable.” (page 9, lines 290-297). Moreover, in this section the AUC obtained were compared to those of previous similar studies that evaluated T2 and ADC feature too showing that both AUC and accuracy of our radiomic tool were higher.

 

Presenting the performance evaluation metrics in a tabular form in the study will increase the readability of the article. 

We thank the reviewer for the suggestion.

The performance evaluation of our radiomic tool was already presented in a tabular form for the univariate AUC of the features selected, while the resulting AUCs obtained from T2, ADC, clinical features and multivariable analysis were reported in figure 3 in order to improve the visualization of ROC curves and to show how the multivariate AUC was the most relevant one. 

 

Similar studies on the subject were not discussed in the study. 

We thank the reviewer for the comment.

Similar studies have tried to develop and validate nomograms for the prediction of CS PCa. Nevertheless, these studies focused on a specific population, as gray-zone PSA level (4-10 ng/ml) or PI-RADS 3 lesions. Instead of targeting our work only to a pre-specified category, we avoided to super select our population in order to obtain data from a real-life cohort and a radiomic tool that could be more generalizable. This discussion was added to the manuscript (page 9, lines 291-295).

 

A subtitle in the form of a literature review should be added to the study. 

We thank the reviewer for the remarkable suggestion.

A running title was added to the manuscript accordingly.

 

The study is remarkable in general, if the mentioned deficiencies are addressed, the quality of the study will increase.

We thank the reviewer for agreeing to review our paper and for his/her comments.

Round 2

Reviewer 1 Report

I would like to congratulate the authors for the revised manuscript and thank them for taking into consideration my suggestions.

I propose the current form for publication.

Author Response

We thank the reviewer for the suggestions and the opportunity to improve the manuscript.

Reviewer 3 Report

The study titled "Radiomic machine-learning analysis of multiparametric Magnetic Resonance Imaging in the diagnosis of clinically significant prostate cancer: new combination of textural and clinical features" reached a higher quality level after the revision. There are still some minor deficiencies in the study. A literature review on the subject should be done. There are similar studies on the subject in the literature. Example "https://doi.org/10.1016/j.compeleceng.2022.108275" . In addition, the environment in which the application results were obtained should be explained. Data from an average of 135 patients were used in the study. Over-fitting in models is a common situation in machine learning. The measures taken to prevent this situation should be expressed. It is clear that the article will come to a better point if the deficiencies I have mentioned are eliminated.

Author Response

We thank the reviewer for the comments and the suggestions.

The study titled "Radiomic machine-learning analysis of multiparametric Magnetic Resonance Imaging in the diagnosis of clinically significant prostate cancer: new combination of textural and clinical features" reached a higher quality level after the revision. There are still some minor deficiencies in the study. A literature review on the subject should be done. There are similar studies on the subject in the literature. Example "https://doi.org/10.1016/j.compeleceng.2022.108275" .

We thank the reviewer for this comment. A literature review was done and, compared to the similar study suggeste, our radiomic analysis has the advantage of considering in the radiomic tool also clinical (semantic) features as PSA and DRE. Thus, our study may be considered more adherent to current clinical practice and reflect the real patients' local status.

 

In addition, the environment in which the application results were obtained should be explained.

We thank the reviewer for raising this issue. All experiments were designed and executed within the Matlab 2017a developing environment and with the auxiliary help of the Weka 3.8.1 software. The graphs of the results were also produced with Matlab 2017.

 

Data from an average of 135 patients were used in the study. Over-fitting in models is a common situation in machine learning. The measures taken to prevent this situation should be expressed. It is clear that the article will come to a better point if the deficiencies I have mentioned are eliminated.

We thank the Reviewer for the opportunity to clarify this point. Both the Overfitting and Curse of Dimensionality problems, typical of Machine Learning approaches, were tackled through two main choices: first, by using a Random Forest classifier, which is accredited as being little prone to Overfitting and robust to the problems associated with an excessive number of features compared to the samples in the dataset; second, the feature selection phase was a strategy aimed at eliminating the least relevant measures which, in turn, reduces the risk of Overfitting the data.