A Novel Knowledge Distillation Method for Self-Supervised Hyperspectral Image Classification

Round 1

Reviewer 1 Report

A Knowledge distillation method based self-supervised approach is proposed for hyperspectral image classification. The paper conducts experiments on publicly available hyperspectral image datasets, and the idea is clear. However, there still exist some issues to address.

1. In introduction, in addition to PCA, it will be better to introduce novel representation based feature extraction techniques that can consider spatial-spectral features. For instance;

i) Huang, H.; Chen, M.; Duan, Y. Dimensionality Reduction of Hyperspectral Image Using Spatial-Spectral Regularized Sparse Hypergraph Embedding. Remote Sens. 2019, 11, 1039. https://doi.org/10.3390/rs11091039

ii) Shah, C.; Du, Q. Spatial-Aware Collaboration-Competition Preserving Graph Embedding for hyperspectral image classification. IEEE Geosci. Remote Sens. Lett. 2021, 19, 1–5. doi: 10.1109/LGRS.2021.3074328

2. In experiments section, it will be better to show separate results of spatial and spectral transformation on knowledge distillation to illustrate the importance of proposed spatial-spectral transformation based knowledge distillation approach.

3. A new work on hyperspectral image classification can implement knowledge distillation based on class label samples, such as: Xu, M.; Zhao, Y.; Liang, Y.; Ma, X. Hyperspectral Image Classification Based on Class-Incremental Learning with Knowledge Distillation. Remote Sens. 2022, 14, 2556. https://doi.org/10.3390/rs14112556

4. In the classification maps shown, regions with less noise can be highlighted for emphasizing the importance of proposed method.

5. Feature map visualization can be shown to demonstrate the importance of proposed approach that incorporates adaptive soft label generation method.

Author Response

Thanks a lot to the reviewers for their valuable comments. Please see the uploaded files for our detailed responses and changes made to the manuscript.

Author Response File: Author Response.pdf

Reviewer 2 Report

Dear Authors,

   Thank you so much for the updated manuscript submission to MDPI Journal of Remote Sensing. After careful review, I believe that while this paper presents good set of work, the comprehensive quality still needs to be significantly improved. To ensure high quality of techinical paper coming into acceptance, I summarized some possible issues (not limited to), which are listed as below. Please prepare an updated version along with revision report to address the required edits. Appreciate that. 

   Major problematic issues suggested for your improvement:

   a) Abstract session: it contains almost 300 words, which seems too long. Consider shorten this session into 180~200 words in the final version. The verbose narrations in Lines 1-9 can be condensed. The final conclusion remarks should be updated with keynote quantitative results. Thanks a lot!

   b) Introduction: The paragraph at Lines 60-92 needs some edits. Besides, when emphasizing your keynote contributions right in the Introduction part, be sure to supplement more specific details. The last paragraph is very good, preferable with organization on the remainder of this paper. Thanks a lot!

   c) Section 2, Related Work: the two paragraphs are a bit too generic. While this section is quite concise, the main shortcomings on specification at this part is also explicit. The pros and cons on each of the subtopics on study, are not very clear. Please consider rearranging the statements in this section, one suggested option is to enrich the tabulated crucial state-of-the arts in one Table, also by supplementing more specific details. Thanks very much!.

   d) Section 3, Methodology: The current version looks fine, and the derivations are also good. Two suggestions: make the font size and style of legends, annotations and markers in each figure uniform. Fix some obvious typos in the context, i.e., in Line 193, entopy --> entropy. 

   e) Figures and Tables: In addition to the image size and resolutions (which should be further enhanced), the characters on the right of each legend at Figs. 4-6 and Fig. 10, should be replaced with font style of Palatino Linotype or Times New Roman. In Tables 5-7, each of the statistical values of OA, AA and Kappa, had better to be arranged in a single row. Please update. 

   f) Experiments and Results: It looks that your approach outperforms most of the other state-of-the-arts in each dataset, I think for some poorest results, the authors need to explain that more explicitly. Besides, if you have any ablation study or sensitivity analysis in your tests, please supplement. 

   g) Conclusion Section: consider making four sets of keynote statements in discussion more concise, and add some more specific details on limitations of study and suggested future work by adding a second paragraph. The current future work looks too generic. The title of this section can be named as conclusions and future work after applying these edits.

   h) References: Quite a few obvious problems must be fixed. (i) Did not use abbreviated citation style. Please apply the required changes after the title of cited journals (MDPI does not require citing the issue number); (ii) Some latest publications in Years 2020-2022 parallel to your study area, are expected to be added, including the current state-of-the arts on object detection and classification using wide-area datasets, both hybrid models and recent deep learning based approaches. (iii) When citing conference proceedings, do please supplement the missed period, location and other information, i.e., Refs. [34], [38], [40], [41], [48]-[51] need your rework.

   Minor issues recommended for updates in your revised version: 

   a) Re-arrange the workflow of related Figures, which are a bit rough at the current view. Apply the required enhancement on image resolution. Be sure that the size and position of each image comply with the MDPI template. 

   b) The literal quality of English should improved. There are minor language and grammatical issues persisting in the current version, consider inviting a native speak to proofreading the whole set of manuscript then re-submit the polished article to Remote Sensing. 

   c) Meanwhile, Fix the related linespacing, font-type and size issues. Be consistently uniform with the required styles as specified by MDPI. 

   In sum, I recommend this manuscript with a minor revison in condition of addressing all the related review comments, then consider coming into acceptance.  Once again, thank you very much, and good luck with your further edits!

Best wishes,

Yours sincerely,

Author Response

Thanks a lot to the reviewers for their valuable comments. Please see the uploaded files for our detailed responses and changes made to the manuscript.

Author Response File: Author Response.pdf

Reviewer 3 Report

The authors proposed a method for HSI classification with few samples. My comments are listed below: 

1. The literature review of KD is somewhat insufficient. From the manuscript the authors proposed a method that creates soft labels for unlabelled samples. The proposed method method is a little bit like the self-distillation, but there is no obvious relationship between the teacher/student at current state (or say, epoch) and the model at the previous state. More detail should be described in order to relate the proposed method to KD.

2. In section 3.1, line 175: Some descriptions of the model configuration in this paragraph should be also shown in fig. 2.

3. The left-hand side of eq. 7 is not specified.

4. eq. 10 is confusing. why R denotes the number of layers (what layer)? Besides, the definition of Lq is not clear. It should be the SSL loss but why the authors use the bce loss? 

5. Do labeled samples also have to calculate Ls? From fig.1 it seems like Lh is for labeled samples while Ls is for unlabeled samples. 

6. In the SSL training, the authors used both spatial and spectral transformation. The authors could show the classification results for the ablation study using only spatial or spectral transformation .

7. The authors mentioned SSAD, a major comparison target, and its shortcomings. However, by explaning SSAD so briefly, it is hard to understrand why the performance is unsatisfied.

8. In table 6 and 7, The description of the first column is missing.

Author Response

Thanks a lot to the reviewers for their valuable comments. Please see the uploaded files for our detailed responses and changes made to the manuscript.

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

The authors have addressed all of my comments. It looks ready for publication.

Reviewer 3 Report

The author has carefully revised the manuscript and clarified my questions. I have no further questions.