EL-NAS: Efficient Lightweight Attention Cross-Domain Architecture Search for Hyperspectral Image Classification

Round 1

Reviewer 1 Report

It is a quite interesting paper to read. Its main sections are well written. Equations are well presented. There are some modifications I'd like to see before this paper is accepted. Please find below comments:

1. Make Introduction section shorter and more concise about NAS. At this moment, it occpies more than three pages which I feel quite lengthy. Is that possible to include section 2.1 into introduction, and then move some context from introduction to Section 2? 

2. Check all figure labels to make sure references in the paper are pointing to correction figures. Line 271 and 322 are examples that wrong labels were used. 

3. Re-plot Figure. 3 "The effect of edge decision in the search process" and please consider using more distinguishable legends for better illustration?

4. I didn't get how the 3D convolution is decomposed from 5x3x3 to a 7x1x1 and 1x3x3? Figure 4 didn't help me.

5. The sentence from line 325 to line 328 needs to be re-written.

6. Make descriptions of datasets consistent for their details in Section 4.1. 

7. Re-write the sentense between Line 449 and 451.

Overall quality is quite alright. There are some minor typos and grammar issues in the paper. A proof reading is recommended after it is modified. 

 

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Reviewer 2 Report

The paper develops t the Efficient Lightweight Attention Network Architecture Search (EL-NAS) algorithm to automate the development of efficient DL architectures to enhance HSI classification accuracy. EL-NAS employs a differentiable network architecture search framework, speeding up architecture exploration using gradient descent. A tailored search space for lightweight attention modules is introduced, maintaining accuracy in compact architectures. The edge decision strategy minimizes the trade-off between validation accuracy and classification performance by using entropy-based distribution estimation. Experimental validation on real-world HSI demonstrates the effectiveness of EL-NAS's in generating smaller architectures while maintaining good accuracy.  

I have the following comments that need to be addressed for the next round of reviews:

1. A major challenge to adopt DL for HSI domains is data annotation. This challenge has been addressed by transfer learning:

a. Liu, B., Yu, X., Yu, A. and Wan, G., 2018. Deep convolutional recurrent neural network with transfer learning for hyperspectral image classification. Journal of Applied Remote Sensing, 12(2), pp.026028-026028.

b. Huang, Z., Pan, Z. and Lei, B., 2017. Transfer learning with deep convolutional neural network for SAR target classification with limited labeled data. Remote sensing, 9(9), p.907.

c. He, X., Chen, Y. and Ghamisi, P., 2019. Heterogeneous transfer learning for hyperspectral image classification based on convolutional neural network. IEEE Transactions on Geoscience and Remote Sensing, 58(5), pp.3246-3263.

d. Rostami, M., Kolouri, S., Eaton, E. and Kim, K., 2019. Deep transfer learning for few-shot SAR image classification. Remote Sensing, 11(11), p.1374.

e. Liu, X., Hu, Q., Cai, Y. and Cai, Z., 2020. Extreme learning machine-based ensemble transfer learning for hyperspectral image classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 13, pp.3892-3902.

I think a discussion on this challenge along with the above works should be added to Section 2.1

 

2. In Tables 2-4, please add the number of learnable parameters for each method so the reader can compare them in this aspect more straightforwardly.

 

3. I was wondering what was the procedure to come up with hyperparameter values in Section 4.4? The explanations are clear and my question is about the reason behind these explanations. 

 

4. While the comparison in terms of performance is certainly informative, I think adding the running time for architecture search is informative, too. Please add the running time for at least a subset of your experiments along with the exact specs of the hardware you use to run your code on.

 

5. Please provide the implementation of your work on a public domain such as GitHub to make reproducing your results by other researchers straightforward.

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Reviewer 3 Report

This paper proposes an efficient Lightweight Attention Architecture Search (EL-NAS), which uses lightweight attention network architecture search and edge decision strategies to optimize the design of deep learning structures. It aims to solve the difficulty of deep learning structure design and improve the performance of hyperspectral image classification. Experiments show that EL-NAS is a more efficient search process with smaller parameter sizes and higher hyperspectral image classification accuracy. However, there are some problems that must be solved before it is considered for publication.

(1) As one of the important comparison methods, the authors should explain clearly the difference between LMAFN (NAS-Guided Lightweight Multiscale Attention Fusion Network for Hyperspectral Image Classification) and the proposed method.

(2) The title this paper is “EL-NAS: Efficient Lightweight Attention Architecture Search for Cross-Domain Hyperspectral Image Classification”. However, in the section 4.7. (Cross Scenario Experiment), the experiments are limited. Several other Cross-Domain based methods should be compared with the proposed method.

(3) Many figures in the paper are quoted with incorrect numbers, such as Figure.3. in line 238 of page 6 should be Figure.2., ‘Figure.3.2.1’ in line 271 of page 7 should be Figure.4. The paper should be proofread carefully.

(4) In line 96 of page 3, please check the problem of “1014×”.

(5) The verb tenses in many sentences are not consistent.

(6) There are some problems with Table captions using mixed case, such as Table1. (Sample Setup OF THE IN, UP AND HU DATA SET), Table 5. (‘FoR’ should be changed to ‘For’), and Table 6, 7, 8 ’s table captions are all uppercase, which are different from others.

(7) In page 17, the ‘(l) EL-NAS’ described in the caption of Figure 12 should be ‘(k) EL-NAS’.

(8) In line 409 of page 15, the text is introduced as ‘Four types of candidate operations’, but the following list consists of five operations, which is ambiguous.

(9) In line 417 of page 15, please check the problem of ‘a 2 normal cell’.

(10) The authors should check the format of the references and the citations in the paper. For example, in line 28, the reference [8-10] should be changed to [9-10]; The information of reference [67] is not complete.

(11) The description of the experiments in this paper is too simple, and most of the experimental results are simply described. The superiority of the model can be better demonstrated by some more explanation of the results.

Overall, this paper is well written and easy to follow. There are some problems about verb tenses in some sentences.

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Round 2

Reviewer 2 Report

The authors have addressed the raised concerns.

Author Response

Thank you for taking the time to review our manuscript and for acknowledging that we have addressed the concerns raised. We appreciate your valuable feedback, which has helped improve the quality of our work.

Reviewer 3 Report

In the revised version, the authors have tried hard to improve the manuscript. However, I am not satisfied with the responses to my first two comments, which are important in my view.

 

My comment 1 is “As one of the important comparison methods, the authors should explain clearly the difference between LMAFN (NAS-Guided Lightweight Multiscale Attention Fusion Network for Hyperspectral Image Classification) and the proposed method”.

The corresponding revision in the manuscript is “LMAFN incorporates lightweight modules sourced from NAS on different tasks to construct its network. It’s important to note that the entire network is not derived solely from architectural searches”.

It is still not clear how the network of LMAFN is constructed. Is it constructed manually, or partially manually?

 

My comments 2 is “The title of this paper is “EL-NAS: Efficient Lightweight Attention Architecture Search for Cross-Domain Hyperspectral Image Classification”. However, in the section 4.7. (Cross Scenario Experiment), the experiments are limited. Several other Cross-Domain based methods should be compared with the proposed method”.

The response is “Thanks for the comments. The primary innovation and contribution of our research paper mainly keens on how to explore the application of cross-domain architectural search methods for hyperspectral image classification. Our research aims to leverage the inherent architectural knowledge from other domains to guide the generation of architectures, rather than training architectural models using data from those domains. This constitutes a fundamental difference from other cross-domain classification methods, therefore making direct comparisons is infeasible”

 

I am not satisfied with this response. The title of the manuscript indicates that it focuses on cross-domain hyperspectral image classification. However, neither the text nor the experiments talk about the cross-domain fully. What kind of knowledge is transferred from the source domain to the target domain? It is not described clearly. Furthermore, no comparison, how do we know whether the proposed method is effective? Maybe any other network that works well on the source domain can also works well on the target domain.

 

In addition, it is suggested that the authors copy the corresponding revised content of the manuscript under their responses to show what have been changed.

Minor editing of English language required.

Author Response

Comments 1: It is still not clear how the network of LMAFN is constructed. Is it constructed manually, or partially manually?

Response 1: Thank you for your insightful comment. To clarify, LMAFN is our previous research work, it is constructed based on architectural guidelines provided by NAS. It's important to note that LMAFN is a manually-designed neural network. It incorporates the architectural principles from NAS to guide its feature fusion and network architecture, but it does not employ automated architectural searches. In this sense, the entire network is manually constructed, closely aligning with the guiding rules established by NAS.

We have revised the manuscript to incorporate the following changes, in particular lines 59 to 67:

"The Lightweight SS Attention Feature Fusion Framework (LMAFN) [24] is constructed based on architectural guidelines provided by NAS [25], the proposed LMAFN achieves commendable classification accuracy and performance with a reduced parameter quantity. Specifically, LMAFN is a manually-designed neural network that incorporates the architectural principles from NAS to guide its feature fusion and network architecture. Therefore the entire network of LMAFN is manually constructed by the guiding rules established by NAS but does not utilize automated searches for its architecture."

 

Comments 2: I am not satisfied with this response. The title of the manuscript indicates that it focuses on cross-domain hyperspectral image classification. However, neither the text nor the experiments talk about the cross-domain fully. What kind of knowledge is transferred from the source domain to the target domain? It is not described clearly. Furthermore, no comparison, how do we know whether the proposed method is effective? Maybe any other network that works well on the source domain can also works well on the target domain.

Response 2: We sincerely apologize for not adequately addressing your concerns in our previous revisions and are grateful for your invaluable feedback.

To respond to your question, "What kind of knowledge is transferred from the source domain to the target domain? It is not described clearly. Furthermore, no comparison, how do we know whether the proposed method is effective? Maybe any other network that works well on the source domain can also work well on the target domain,"

We have corrected the paper title to "EL-NAS: Efficient Lightweight Attention Cross-Domain Architecture Search for Hyperspectral Image Classification" and refined our experimental procedure. Specifically, we now include a comparative analysis between scenarios with and without cross-domain architecture search. This modification aims to demonstrate that EL-NAS can indeed identify a more versatile and effective model when provided with additional data from other domains. These comprehensive changes are elaborated in Section 4.7 on page 20 in the revised manuscript and are further supported by Tables 9-10.