Anomaly Analysis of Alzheimer’s Disease in PET Images Using an Unsupervised Adversarial Deep Learning Model

Round 1

Reviewer 1 Report

This works proposes an unsupervised, three-elements (two different convolutional networks and a decoder) adversarial deep learning model for detecting anomalies from PET images for the Alzheimer’s disease (AD). The proposed method would present high construction loss for AD patients, with respect to the healthy, training ones. The results are encouraging despite being mild. In particular, the classification is compared to benchmark models, resulting in a reasonable performance. As remarkable metrics and figures of merit, the AUC is about 75% and the accuracy is circa 96%.

The paper somehow flows, but there is room for improvement. The referencing is wrong from the very first page, reflecting a scarce proofreading. The referencing is scarce, especially about the biological part and the SVM classification methods, as well as the other imaging techniques such as MRI and fMRI. Section 2 must be improved. Some methodological details are missing in Sect. 4. The Results and Conclusion Sections deserve to be revised. See the attached comments below.

Specific Comments and Suggestions for Authors Throughout the Paper:
• Please correct the reference numbering.
• Sentences from line 26-37 without reference. Please provide a suitable reference. I suggest the author to read:

Munoz, D.G.; Feldman, H. Causes of Alzheimer’s disease. CMAJ 2000, 162, 65–72. [Google Scholar]

Wolk, D.A.; Price, J.C.; Saxton, J.A.; Snitz, B.E.; James, J.A.; Lopez, O.L.; Aizenstein, H.J.; Cohen, A.D.; Weissfeld, L.A.; Mathis, C.A.; et al. Amyloid imaging in mild cognitive impairment subtypes. Ann. Neurol. 2009, 65, 557–568.

• Lines 38-41: Missing references for such a bald statement. Also MRI and fMRI are relevant imaging techniques used for AD diagnosis. Please, discuss this point coherently and comment by considering the following references:

Machulda, M.M.; Ward, H.; Borowski, B.; Gunter, J.; Cha, R.; O’brien, P.; Petersen, R.C.; Boeve, B.F.; Knopman, D.; Tang-Wai, D.; et al. Comparison of memory fMRI response among normal, MCI, and Alzheimer’s patients. Neurology 2003, 61, 500–506.

Frisoni, G.B.; Fox, N.C.; Jack, C.R., Jr.; Scheltens, P.; Thompson, P.M. The clinical use of structural MRI in Alzheimer disease. Nat. Rev. Neurol. 2010, 6, 67.

Bron, E.E.; Smits, M.; Van Der Flier, W.M.; Vrenken, H.; Barkhof, F.; Scheltens, P.; Papma, J.M.; Steketee, R.M.; Orellana, C.M.; Meijboom, R.; et al. Standardized evaluation of algorithms for computer-aided diagnosis of dementia based on structural MRI: The CADDementia challenge. NeuroImage 2015, 111, 562–579.

Dachena, Chiara, et al. "Combined Use of MRI, fMRIand Cognitive Data for Alzheimer’s Disease: Preliminary Results." Applied Sciences 9.15 (2019): 3156.

• Missing references from 41-46. The bibliography must be improved.
• Lines 49-53: The referencing to the state of the art is not enough. Especially about SVM system and AD. For instance, I strongly suggest the author to read and cite:

Dachena, Chiara, et al. "Combined Use of MRI, fMRIand Cognitive Data for Alzheimer’s Disease: Preliminary Results." Applied Sciences 9.15 (2019): 3156.

for further details and fill this gap in their article. Indeed, in this paper, SVM is used for classifying AD subjects relying on features derived from fMRI and MRI images, coupled to MCI indexes, using the same dataset. Your paper can surely benefit from the comparison to this very recent paper (also published in this journal).

• Line 102, again poor and insufficient referencing about works which make use of fMRI (see Dachena et al. 2019).
• Line 131: Please correct the acronym (capital letters).
• Section 4.1: The authors are not reporting gender, age and other potentially relevant information. The bias or the relevance of these data is crucial to clinicians. The weight must be judged. Please increase the level of detail of Tab. 1.
• Section 4.2: No references for the formulas, please provide some, e.g.:

Singh, Vivek Kumar, et al. "Breast tumor segmentation in ultrasound images using contextual-information-aware deep adversarial learning framework." Expert Systems with Applications 162 (2020): 113870.

• The quality of Fig. 5 must be improved. Furthermore, it is the mixture of a table and a graph. It is strange. Please improve the quality of results presentation and scientific soundness.
• Line 211: Missing reference for Eq. (6).
• Conclusion must be improved by providing more comparative details.

Additional Comments:
The authors did not comment or justified properly the choice of PET as imaging modality. The PET images have a low resolution, the noise can be an overwhelming problem. Why the authors did not consider to couple this method to other imaging modalities?

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report

The articles proposes an anomaly detection approach to detect Alzheimer from PET images. The approach seems to be working extremely well. However, the experiments and comparisons are not described well enough, making it hard to judge if the proposed approach really as novel and high performing as claimed.   General comments: - Authors: an initiative cannot be an author.   - Although some readers (including myself) might know the advantages of an unsupervised anomaly detection approach, it does not become clear from the text why an unsupervised technique was used for labeled data. It should be described and explicitly explained why an unsupervised method is used for a supervised task.   - The creation of train and test set and the performance evaluation is not well described. For example, in Section 4.3 suddenly other approaches appear that were not discussed before. It appears that the author's approach significantly outperforms all other approaches. But they should point explicitly what is different in their approach and how the test were performed. Was the same data and same division into test and train used for all approaches? A table with exact description would be easier to read. The figures contain lots of mistakes (see more detailed comments below).   Precise comments to particular parts in the manuscript:   page 2, line 36: It's -> It has page 2, line 80: anomaly detection is not a novel method. this should be rephrased.   Section 2: Is there work predicting AD from ADNI data? Is it supervised? Please refer to that and state the performance. It would be a nice finding if the unsupervised would show better performance.     page 4, line 145 why not simply say that all y_i are 0 for train? Section 4.1 Please describe the train and test set in more detail. Previously, you mentioned that the train data contained only normal samples. So, there was a different split than a random 80-20 as described in Section 4.1.   Figure 1 (in Section 3) refers to Chapter 3 for more detailed information. Perhaps it was just copied from a book. Anyway, not understandable.   Figure 3: the confusion matrix is labeled wrong (hopefully), predicted label should start with AD. If not, it actually means that the proposed approach does not work at all.   page 9: the works mentioned here and compared with the proposed approach (GANomaly etc.) should be described in the related work section. Also, state how your approach differs from the others. Moreover, correct capitalization should be used (e.g., GANomaly and not Ganomaly).   Figure 5: the headline should be "AUC Scores for ADNI dataset" and not "AOC.."   There are no references in 4.2. Please cite the original works of the loss functions and also provide references for general statements (such as the first sentence referring to recent trends).   Figure 6. I like the pointer that the "proposed model produces more realistic images than its separate sub-networks." However, this is not obvious. Please explicitly explain what makes the image more realistic.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

The Authors responded adequately to all my comments/suggestions .

Author Response

The authors thank the reviewers’ efforts of quick, honest, and constructive criticism.

Reviewer 2 Report

The manuscript improved but it is still not ready for publication.
Mathematical definition: It does not make any sense that the training set is comprised of N normal images where N = 0.
Essentially this would mean that you have zero data for training.

Fig. 1 still refers to a Chapter 3 that is not part of the article. I already pointed that out in my previous comments but it was not addressed. 

Some added sentences must be rewritten 
E.g. line 149-152 ("comes from Schlegl et al. in that in the work, the authors..)

Some sentences are just incomplete and miss the full stop at the end.
E.g. line 303: "This unique feature As it is shown.. "

Abbreviations:
there are some abbreviations in the manuscript that are not included in the list

Author Response

Please see the attachment.

Author Response File: Author Response.pdf