Auroral Image Classification Based on Second-Order Convolutional Network and Channel Attention Awareness

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

Manuscript “Auroral Image Classification Based on Second-Order Convolutional Network and Channel Attention Awareness”

The present research utilized a novel method named learning representative channel attention information from second-order statistics (LRCAISS), that highlighted with two innovative techniques: a second-order convolutional network and a novel second-order channel attention block. LRCAISS has been extensively tested on two public datasets of ground-based auroral images and shows competitive performance when compared to existing methods. This suggests indicates its potential as a valuable tool for researchers in the field of auroral studies, providing an effective means to analyze complex visual data that may contain subtle distinctions critical to scientific inquiry and discovery. The finding of the study offers valuable insights and demonstrate that LRCAISS achieves competitive performance compared to existing methods. The paper is well-written and deserving of publication after minor revisions. There are a few queries that need to be addressed before it can be accepted.

 

Recommendations/suggestions/queries/modifications

However, there are some suggestions/queries/modifications, and lastly recommendations that I would like to provide authors that will enhance the value of this work:

1.     Add a paragraph about recent literature in the introduction section.

2.     Mention specific performance metrics are used to evaluate LRCAISS, and how does its performance compare to existing methods? Are there any particular areas where LRCAISS significantly outperforms other approaches?

3.     What are the limitations of the LRCAISS method as discussed in the paper? Are there potential improvements to enhance its accuracy and applicability?

4.     Write future research directions and suggest for further developing and refining the LRCAISS method? Are there upcoming technologies or techniques that could be incorporated?

5.     Conclusion section should focus solely on significant findings, method advantages/disadvantages, and future research recommendations, omitting generic/introductory information and methodology.

6.     Recheck the references and correct them according to the journal format.

 

 

 

 

 

Comments on the Quality of English Language

The quality of english can be improved. 

Author Response

1. Add a paragraph about recent literature in the introduction section.

Response: Many thanks for the Reviewer’s suggestion. We reorganized the Introduction section to divide recent literature into four parts: traditional aurora classification methods, deep learning-based aurora image classification methods, methods with channel attention mechanisms, and methods based on second-order tensor features. We have also omitted descriptions of less relevant methods to enhance the coherence and brevity of the section.

2. Mention specific performance metrics are used to evaluate LRCAISS, and how does its performance compare to existing methods? Are there any particular areas where LRCAISS significantly outperforms other approaches.

Response: We gratefully appreciate for the valuable suggestion. In accordance with the Reviewer’s guidance, we have provided a more detailed comparison of LRCAISS with existing methods in Section 4.2, ‘Main Results’. As demonstrated in Table 7, LRCAISS surpasses other methods in terms of accuracy.

LRCAISS achieves a significant advancement over existing methods due to the effective feature extraction. Unlike most current auroral image classification approaches, which merely fine-tune deep neural network parameters without considering the inherent characteristics of auroral images, LRCAISS enhances the network's architecture, which recognizes the variance within feature maps across channels and revising the convolutional neural network. This revision is accomplished by learning attention scores that rescale the feature maps, thereby capturing the channel-wise dependencies more effectively. Furthermore, LRCAISS derives attention scores from second-order statistics instead of first-order statistics. This innovative approach allows for the extraction of more representative features, which in turn leads to a higher accuracy in classification tasks. We add a discussion section to detail the difference between LRCAISS and existing methods in manuscript.

3. What are the limitations of the LRCAISS method as discussed in the paper? Are there potential improvements to enhance its accuracy and applicability.

Response: Many thanks for the suggestions of the reviewer. The main limitation of LRCAISS is that current implementation of LRCAISS runs with long computation time, which could be a bottleneck when processing large image datasets. In the case of training LRCAISS on Dataset 1, which is divided into a training set and testing set in the ratio of 3000:846, the running time is approximately 155 seconds per epoch, while the Resnet50 requires only 27 seconds per epoch. We have supplemented the limitation of LRCAISS in Conclusion section.

The potential improvements can be included in two aspects: 1) Model architecture alternatives: We can explore the integration of alternative deep learning models, such as the Vision Transformer (VIT), which may offer enhanced performance while maintaining or improving upon the accuracy of LRCAISS. 2) Optimization of SCA block integration: The placement of the SCA block within the network is critical and our current experiments have not fully explored the optimal integration strategy. We believe that a more strategic insertion of the SCA blocks could potentially enhance both the accuracy and applicability of the model.

4. Write future research directions and suggest for further developing and refining the LRCAISS method? Are there upcoming technologies or techniques that could be incorporated.

Response: Future research will focus on optimizing the integration of SCA blocks, revising the backbone network, and reducing runtime. Additionally, LRCAISS could incorporate multiple attention mechanisms to expand its application scope. Recent advancements in computer vision, such as Vision Transformer (VIT) and ConvNeXt, could serve as alternative backbones for LRCAISS. Moreover, the concept of integrating multiple attention mechanisms, as demonstrated by the Visual Attention Network (VAN), could be utilized in LRCAISS to enhance its performance. We have supplemented the future research direction and suggestion in Conclusion section.

5. Conclusion section should focus solely on significant findings, method advantages/disadvantages, and future research recommendations, omitting generic/introductory information and methodology.

Response: We thank the reviewer for pointing this out. We have revised the conclusion in accordance with the guidance, carefully excluding introductory information and methodological descriptions, and have meticulously refined its structure to ensure clarity and coherence. Here is the revised conclusion:

This paper introduces a second-order channel attention network named LRCAISS for auroral image classification. Experimental results on two auroral image datasets demonstrate the generalization ability of the proposed method, underscoring the effectiveness of LRCAISS. It is superior to the traditional channel attention methods based on first-order statistics, showing the advantage of learning attention scores from second-order statistics over first-order counterparts. Moreover, LRCAISS outperforms methods relying solely on second-order convolutional networks, indicating that integrating an attention mechanism improves auroral representation learning. Thus, the dual mechanism approach enhances performance, facilitating more effective auroral analysis. However, current implementation of LRCAISS runs with long computation time, which could be a limitation when processing large image datasets. Future research should concentrate on refining the computational efficiency and incorporate recent deep models into LRCAISS for better effectiveness. Additionally, diverse attention mechanisms are proven effective in the domain of image classification. Therefore, exploring their relationships and integrating their strengths holds promise for advancing auroral image processing.

6. Recheck the references and correct them according to the journal format.

Response: We are sorry for bothering you with this kind of mistakes that we could have avoided. We have meticulously implemented the following revisions:1) We replaced the period before the DOI with a comma, and added a period after the DOI. 2) We amend the 'pp:' to 'pp.' within conference citations. 3) We remove extra spaces to ensure typographical precision.

Reviewer 2 Report

Comments and Suggestions for Authors

 

1) I recommend that you include literature in the introduction of your study.

 

2) Create a Discussions section and compare your results with the results of studies in the literature in this section.

 

 

3) Provide more detailed information about your study in the Results section.

Author Response

1. I recommend that you include literature in the introduction of your study.

Response: Thanks for the Reviewer’s suggestion. We reorganized the Introduction section to divide recent literature into four parts: traditional aurora classification methods, deep learning-based aurora image classification methods, methods with channel attention mechanisms, and methods based on second-order tensor features. We have also omitted descriptions of less relevant methods to enhance the coherence and brevity of the section.

2. Create a Discussions section and compare your results with the results of studies in the literature in this section.

Response: We gratefully appreciate for the valuable suggestion. we have complemented a discussion section to show the differences between LRACISS with exiting methods and analyze the reason for accuracy gain acquired by LRCAISS.

3. Provide more detailed information about your study in the Results section.

Response: Many thanks for the suggestions of the reviewer. We have added the detailed component of LRCAISS in Line 380 to Line 384.

Reviewer 3 Report

Comments and Suggestions for Authors

The paper needs more modifications.

Comments for author File: Comments.pdf

Comments on the Quality of English Language

Extensive editing of English language required.

Author Response

1. The Highlights, objectives and methodology of this paper should be included in the abstract, introduction, and conclusion of the paper. Check this in the paper.

Response: Thanks for the Reviewer’s suggestion. In accordance with Reviewer’s guidance, we have confirmed that the Abstract encompasses the key highlights, objectives, and methodology of this study. Additionally, the Introduction includes the objectives and methodology, while the Conclusion summarizes the highlights.

2. The section of "1. Introduction" is very long. This section is not perfect. Develop the introduction into three paragraphs: the first one for your subject, the second one for the literature review, and the last paragraph for the objectives and methodology of the paper. The introduction is not written well. The statements of paragraph LR is not connected by the common transition words and phrases.

Response: We appreciate the reviewer’s insightful suggestion.

Thank you for your helpful feedback. Based on your suggestions, we have rewritten our introduction to make it clearer and more concise. Here's how we've revised it:

The first paragraph now simply introduces the topic of auroral image classification. As for LR, we've shortened the literature review (LR) by focusing on a clear summary of the main methodological approaches, including four domains of methods: traditional aurora classification methods, deep learning-based aurora image classification methods, methods with channel attention mechanisms, and methods based on second-order tensor features. We've removed some methodological details that weren't as relevant to make the presentation smoother. We've also included a more comprehensive discussion that weighs the advantages and disadvantages of each methodology to ensure a balanced view. The last paragraph outlines the objective and methodology of our paper. To make the text flow better and be easier to read, we've used different connecting words.

3. Equations (1) and (2) should be not mentioned in the introduction section.

Response: Many thanks for the suggestions of the reviewer. In revised manuscript, equation (1) and equation (2) are removed.

4. The title "2. Methods" of section 2 is not suitable, not clear. I suggest to be "Proposed Methods".

Response: Many thanks for the suggestions of the reviewer. We revised “2. Methods” to “2. Proposed Methods”.

5. You didn't mention Table7 in Discussion or Conclusions.

Response: We are sorry for bothering you with this kind of mistakes that we could have avoided. We supplement a reference to the Table 7 in the second paragraph of “4.2 Main Results”.

6. You have to minimize some figures in the paper.

Response: Many thanks for the suggestions of the reviewer. We have minimized the size of Figure 5 and Figure 8.

7. You have to distinguish between your results and other results in LR.

Response: Many thanks for the suggestions of the reviewer. Followed by the Reviever2, we add “5. Discussion” section to detail the differences between our methods with others.

8. Put algorithms for the all methods in subsections.

Response: Many thanks for the suggestions of the reviewer. The training stage of LRCAISS is added as Algorithm 1 at the end of Section “2. Proposed Methods”.

9. You have use more than 25 times for the word "we". Avoid the use of "we" and "our" in your paper.

Response: Many thanks for the suggestions of the reviewer. “We” and “our” are excluded in the revised manuscript.

10. You used some concepts like " singular value decomposition (SVD) " and your introducing is not perfect. Put all the related concepts and classical methods in section 2 of " The section two should be titled " Preliminary and Related Works"

Response: Many thanks for the suggestions of the reviewer. We integrate these concepts within the “2. Proposed Method” section, that would create a more cohesive presentation and enhance the reader's comprehension.

11. Equation (10) is not clear. It should be having more details.

Response: We thank the reviewer for pointing this out. In response to your valuable feedback, we have expanded upon the framework to include a thorough explanation of the computational process for each step of the formula, as well as the feature sizes obtained subsequent to the calculations. To facilitate a clearer understanding, we respectfully request your attention to the description that now accompanies the formula's presentation. We trust that this additional detail will illuminate the methodology and findings of our study

12. Rewrite the subsection " 3.2.1. Implementation Details" to be more perfect.

Response: Many thanks for the suggestions of the reviewer. This section has been revised for enhanced coherence and readability, ensuring that the presentation of information flows logically and is accessible to the reader.

13. Every equation should be ended by "." Or ",". Check the equations (12)- (15).

Response: Many thanks for the precious comments. We have checked all equations and added “,”.

14. No section for statistical analysis or mathematical modeling.

Response: Many thanks for the helpful comments. Regarding statistical analysis, the classification accuracy of our method is detailed in Table 7. Figure 7 presents the confusion matrices for the two datasets, offering a clear visualization of performance metrics. Tables 8 and 9 provide a detailed breakdown of precision, recall, and F1 scores, further substantiating the effectiveness of our approach. As for mathematical modeling, the equations underlying each step of LRCAISS are clearly articulated and have supplemented our submission with Algorithm 1, which outlines the training stage of the proposed method, enhancing the transparency and reproducibility of our research.

15. Focus on your results. Put a section for "Main Results".

Response: Many thanks for the suggestions of the reviewer. We have restructured the "4. Experiments" section to enhance its clarity and coherence. We have streamlined the presentation by removing the "4.3 Results discussion" subsection. The section is now divided into two clear parts: the first part details our ablation experiments, which is conducted to identify the most effective configuration for the LRCAISS model, clarifying the impact of each component on the overall performance. The second part is dedicated to the main results of LRCAISS, where we present a comparative analysis with other state-of-the-art methods, demonstrating the robustness and superiority of our approach. Additionally, we include a thorough statistical analysis of the LRCAISS results in precision, recall, and F1_score, offering a detailed account of the model's performance through a variety of metrics, thereby providing a comprehensive overview of the outcomes and their implications in the context of our research.

16. The paper is written not perfectly. Put a new section "2. Background" or 2. Preliminary" to illustrate the ideas significant concepts, and preliminary.

Response: Many thanks for the helpful comments. In the revised manuscript, the background of our study is now outlined in the Abstract and expanded upon in the first paragraph of the 1. Introduction. This section sets the stage for our research, providing readers with essential context. Then we have illustrated significant concepts and preliminary work throughout the remainder of the 1. Introduction. We believe that this integration strengthens the coherence and readability of the paper. Furthermore, in the 2. Proposed Method, we have elaborated on specific concepts such as SVD, presenting them alongside the corresponding formulas and rationale for each step of the LRCAISS. This approach is intended to deepen the reader's understanding of our methodology, offering a more integrated and comprehensive perspective on our approach.

17. You said in your conclusion " Our method is superior to the traditional channel attention methods based on first-order statistics, showing the advantage of learning attention scores from second-order statistics over first order counterparts" is a general conclusion. Please give more details and prove it.

Response: We totally understand the reviewer’s concern. Empirical evidence from various studies, including references [35, 36], has demonstrated that attention mechanisms informed by second-order statistics outperform those based on first-order statistics. Consistent with these findings, our manuscript presents experimental results in Table 6 to prove this conclusion. Specifically, Table 6 illustrates the impact of using first-order versus second-order statistics in learning attention scores. The term click on “Attention” indicates the use of first-order statistics to derive attention scores, while the term click on both “Attention” and “Second-order” signifies that these scores are informed by second-order statistics. Table 6 clearly shows that incorporating second-order statistics leads accuracy gains of 4.97% on dataset 1 and 5.79% on dataset 2, thus demonstrating this conclusion.

18. In the abstract, you said that " Based on Resnet50, the LRCAISS constructs a second-order convolutional network to exploit richer statistic information encoded in convolutional neural network-based features.". You have to reformatted this formulation.

Response: Many thanks for the suggestions of the reviewer. This sentence is rewritten as “The LRCAISS extends from Resnet50 architecture by incorporating a second-order convolutional network, which effectively captures a more detailed statistical representation from convolutional neural networks”.

Reviewer 4 Report

Comments and Suggestions for Authors

Referee report on the paper of Y.Hu et al. ‘Auroral Image Classification Based on Second-Order Convolutional Network and Channel Attention Awareness’.

The work dial with classification of ground-based auroral images. To excavate more discriminative information from ground-based auroral images, authors propose a novel method named learning representative channel attention information from second-order statistics (LRCAISS). The LRCAISS is highlighted with two innovative techniques: a second-order convolutional network and a novel second-order channel attention block. Based on Resnet50, the LRCAISS constructs a second-order convolutional network to exploit richer statistic information encoded in convolutional neural network-based features. The covariance normalization is utilized and the novel second-order channel attention block effectively recalibrates these features. The proposed algorithms extensively evaluated on two public ground-based auroral image datasets.

The work is written well. All parts of the algorithm are described detail and understandable that is not often for papers of this subject.

As usual the results are depended on database. The auroral morphology in literature is not defined strictly. But in these conditions the experimental results demonstrate that LRCAISS achieves competitive performance compared to existing methods.

Comments on auroral image classification in datasets: 1) the auroral structures are very strongly distorted by projection. For example, ‘discrete’, ‘colored’, ‘edge’,’arcs’,’breakups’ classes depends on position of the structure relative point of view; 2) ‘colored’ structures usually is rays, lower energy electrons precipitations.

So, line 305 is more correct to write ‘….so colored are related to altitude and excited atmosphere component.’

Line 424: misprint in subtitle ‘Rerults Discussion’ → ‘Results and Discussion’

The paper may be published after these small corrections.

Author Response

1. The auroral structures are very strongly distorted by projection. For example, ‘discrete’, ‘colored’, ‘edge’,’arcs’,’breakups’ classes depends on position of the structure relative point of view; 2) ‘colored’ structures usually is rays, lower energy electrons precipitations.

So, line 305 is more correct to write ‘….so colored are related to altitude and excited atmosphere component.’

Line 424: misprint in subtitle ‘Rerults Discussion’ → ‘Results and Discussion’

Response: Thanks for the Reviewer’s suggestion. We have corrected Line 305 (in revised manuscript is Line 273) as so colored are related to altitude and excited atmosphere component. As for the subtitle ‘Rerults Discussion’, we are sorry for bothering you with this kind of mistakes that we could have avoided. And followed by Reviewer 2 and Reviewer 3, this subtitle is removed for clarity, and a new section 6.Discussion is supplemented for detailed analysis.

Round 2

Reviewer 3 Report

Comments and Suggestions for Authors

Figure 6 is big. You have to minimize.

 

 

Author Response

1. Figure 6 is big. You have to minimize.

Response：Many thanks for your valuable comments. We have revised Figure 6 by adjusting the text size within the figure to ensure that the textual content remains legible even when the figure is reduced in size.