Deep Spatial-Spectral Subspace Clustering for Hyperspectral Images Based on Contrastive Learning

Round 1

Reviewer 1 Report

Comments.

1. High percentage of plagiarism (26% without references), especially in Section 1 (2^nd, 3^rd, and 4^th paragraphs), Sections 4.4, 2.1, and 2.2. Please reduce up to 20%.

 

2. In Section 3, clarification of contributions is recommended.

 

3. In Section 4, experiments are not enough to validate the proposed approach. The use of another dataset from other satellites would add value to the experimental results section. Moreover, it is recommended to provide all the needed data, software, parameter configuration, ..., to reproduce the results. Finally, please comment on the complexity of the proposed method.

 

4. In Section 5, please comment on the future work in more details.

 

5. The references are old. Recent 2 years, many high-quality methods are proposed, the authors should read them. Moreover, I strongly suggest the authors compare their method to the current state-of-the-art methods, i.e., works proposed in 2019, 2020, and 2021.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report

The authors have proposed an interesting Deep-Learning (DL) scheme for tackling the Hyper-Spectral (HSI) clustering problem. The study is exhaustive and the proposed approach quite detailed and explained.

However, there are some critical points/information the authors should provide, concerning the designed system:

• In order to train the CNN component, data augmentation techniques are adopted in the form of patch-extraction. Although the mechanism of extracting them is explained, the selection of specific spatial and spectral size of them is not provided. As long as the size of the patches plays an important role in the training process, the authors should provide further insight concerning this specific choice (i.e. spatial & spectral patch-size selection).
• Concerning the CNN component, the authors state in Figs. 2 & 3 that the convolution operation is 3D. However, in Table 1 where they describe the specific hyper-parameters used, the two branches of the model seem performing 1D (i.e. spectral branch) and 2D (i.e. spatial branch) convolutions-by setting the respective kernel-sizes to 1. Although this choice is justified in terms of the proposed model architecture in order to derive spatial and spectral features separately, the authors should provide further explanation-intuition about the reasoning of not performing direct 3D convolution (i.e. in order to derive spatial and spectral features simultaneously).
• The authors should provide further information about the hyper-parameters’ selection in Table 1 (i.e. kernel-sizes of spectral and spatial CNNs).
• The authors should provide the experimental platform’s requirements needed (e.g. GPU-RAM memory).
• As long as the authors train various sub-systems/components, as well as a whole one, they should provide:
  1. Training/computational time (i.e. required time needed for training each component/stage), for each one of them (CNN component, spectral clustering, whole system).
• Please correct the following grammatical errors/typos:
  1. Page 1, Line 2-Abstract: clustering methoD (instead of methoDS)
  2. Page 3 Line 126: further increasES (instead of increasE)
  3. Page 11, Line 251: proposed methoD (instead of methoDS)

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 3 Report

The paper describes a hyperspectral data clustering model based on both Deep Learning (DL) and Contrastive Learning techniques. The authors propose a contrastive algorithm to augment the data set to make sure the proposed deep architecture can be properly trained. Then, the trained deep model is used to project the hyperspectral data set to a feature space where a classical spectral clustering technique is applied.

 

The topic is perfectly in line with mainstream community working on DL-based spectral clustering algorithms for remote sensing. Results seem to be promising. I recommend it for publication after major improvements.

 

Comments:

 

1. The state-of-the art revision doesn’t mention some important classical methods, like DBSCAN, SLIC or more recent works on spatial-spectral clustering, like semantic segmentation or HySSIC. I suggest reviewing these works:
   10. Barthakur and K. K. Sarma, "Semantic Segmentation using K-means Clustering and Deep Learning in Satellite Image," 2019 2nd International Conference on Innovations in Electronics, Signal Processing and Communication (IESC), 2019, pp. 192-196, doi: 10.1109/IESPC.2019.8902391.
   11. Sovi Guillaume Sodjinou, Vahid Mohammadi, Amadou Tidjani Sanda Mahama, Pierre Gouton, A deep semantic segmentation-based algorithm to segment crops and weeds in agronomic color images, Information Processing in Agriculture, 2021, ISSN 2214-3173, https://doi.org/10.1016/j.inpa.2021.08.003.
   12. Egaña, Á.F.; Santibáñez-Leal, F.A.; Vidal, C.; Díaz, G.; Liberman, S.; Ehrenfeld, A. A Robust Stochastic Approach to Mineral Hyperspectral Analysis for Geometallurgy. Minerals 2020, 10, 1139. https://doi.org/10.3390/min10121139
   13. Xu, H. Li, P. Liu and L. Xiao, "A Novel Hyperspectral Image Clustering Method With Context-Aware Unsupervised Discriminative Extreme Learning Machine," in IEEE Access, vol. 6, pp. 16176-16188, 2018, doi: 10.1109/ACCESS.2018.2813988.
   14. Ester, H. P. Kriegel, J. Sander, and X. Xu, ‘‘A density-based algorithm for discovering clusters in large spatial databases with noise,’’ in Proc. Int. Conf. Knowl. Discovery Data Mining (AAAI), 1996, pp. 226–231.
   15. Roy and D. K. Bhattacharyya, ‘‘An approach to find embedded clusters using density based techniques,’’ in Proc. Int. Conf. Distrib. Comput. Internet Technol., 2005, pp. 523–535.
2. In general, even though the provided results seem to be promising, there is an important lack of clarity on the three main model design decisions. For example,
   3. In the data augmentation algorithm explained in Section 3.1, why do the authors are using matrix flipping as a measure of contrastiveness? Are they expecting rotation invariance? If so, why not using more matrix rotations than just using horizontal and vertical flipping? On the other hand, how does rectangular area erasure and random point erasure ensure constrativeness?
   4. The CNN model has a very rich description regarding to diagrams and tables, but one (for example) must infer that the spatial and the spectral information are combined in the 3D convolution component. What is the expected behavior of this spatial-spectral integration? This is important because the proposed model should fall into the spatial-spectral clustering family, as the authors describe in the introduction.
   5. Step 13 in Algorithm 3 mentions a “spectral algorithm to get the clustering result” but that classical spectral algorithm is not specified anywhere in the manuscript – the authors just mention they used a scikit-learn implementation. How does the choice of this algorithm affect the overall result? I suggest specifying which algorithm is used and compare results among several choices of it.

I’m pretty sure that the authors trust their own work contribution but if the rationale behind these decisions is not well explained, it is difficult to see if the proposed method is novel or just a reconfiguration of some of the models mentioned in the related works review.

3. The database choices and the experimental set-up seem to be appropriate. I would suggest mentioning how the method would handle more challenging conditions, like luminosity, atmospheric conditions, spatial data sparsity or noisy spectral data. Regarding the latter, for example, I tend to believe that this should be absorbed by the random point erasure, but (again) this is not clear from the manuscript reading. Classical methods, like DBSCAN, are very robust to noisy data while more recent ones, like HySSIC, are more robust to data spatial sparsity. If the authors think their work is on a different track, they should explicitly mention it within a new section defining the specific application domain of their model.
4. Related to the previous comment, in the conclusions I’d suggest them being a bit more critic with their own work. What drawbacks do the authors find in their proposed method? Claiming that their method is superior based on some metrics on standard databases doesn’t talk much about it on real more challenging applications.
5. Finally, the authors mentioned that they implemented the model using Pythorch. Will that source code be available somewhere? This is important to ensure research reproducibility.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 4 Report

The authors adapt a Machine Learning technique used previously in other fields like image classification, or natural language processing to HSI classification. The adaptation process is not trivial, since the initial step of the proposed technique involves using some data augmentation techniques that must be designed for each set of data.

The authors have carried out several experiments, and have demonstrated the great potential of the proposed approach.

However, since one of the main contributions is adapting the data augmentation techniques for HSI I believe that that part should be analyzed in more detail. For example, you can only use data augmentation for the spatial information, only for the spectral information, and finally the combination of both of them. After that, you can analyze which is the best option. My intuition is that the spatial technique will be useful, but it is not clear to me that eliminating spectral bands will be. If the spectral augmentation technique proves useful. Then you should again try different options adjusting the average number of removed bands. With this analysis, we will have a better understanding of what can be done in the data augmentation step to improve the overall clustering results, and how critical is this step for the final results.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

The authors addressed most of my previous comments. I am satisfied with the authors' edits. 

Author Response

Please see the attachment

Author Response File: Author Response.pdf

Reviewer 2 Report

The authors have addressed the proposed suggestions, by providing further experimental results on specific parts that influence your study. Moreover, you have described in details the hardware used for your experiments, as well as the complexity of your method. Finally, you have corrected the typos mentioned, including other parts in the manuscript.

Consequently, on my behalf the required changes are met, and therefore I believe that your updated manuscript warrants publication in Remote Sensing. 

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 3 Report

From my point of view, all suggestions where properly addresses. Now the method novelty and its potential is clear from the manuscript reading. I still suggest doing some minor extra work on the overall presentation, specially regarding the organisation of the introduction and the related work sections.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 4 Report

Thank you for your answers. I recommend this manuscript for publication.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf