Knowledge-Guided Heterogeneous Graph Convolutional Network for Aspect-Based Sentiment Analysis
Round 1
Reviewer 1 Report
Comments and Suggestions for AuthorsThe following comments will improve the quality of the paper:
-We propose a new knowledge-guided heterogeneous graph convolutional network 90 for aspect-based sentiment analysis. What's so new about the proposed method?
- I think more descriptive details should be provided on the datasets used. With so little or virtually no information, it's not possible to make any concrete judgments about the performance achieved.
- You evaluated your model using accuracy and F1 score. What are these metrics? What formula is used to calculate them? Why these choices? All these details should be provided before analyzing the results.
- How complex is the proposed method compared with other methods?
-Discuss the impact and implications of the results obtained.
Comments on the Quality of English LanguageNo comments.
Author Response
- Response to comment: “We propose a new knowledge-guided heterogeneous graph convolutional network 90 for aspect-based sentiment analysis. What's so new about the proposed method?”
The author’s answer: Thank you very much for your comment, and our response to your question will hopefully answer your question. The novelty of the model proposed in this paper mainly lies in the use of dynamic weighting mechanism to obtain word-level embeddings of BERT. Through the utilization of BiLSTM, HGCN, and external knowledge, the model incorporates multifaceted features of semantics, syntax and additional knowledge. We have highlighted this point in the revised manuscript.
- Response to comment: “I think more descriptive details should be provided on the datasets used. With so little or virtually no information, it's not possible to make any concrete judgments about the performance achieved.”
The author’s answer: According to your suggestion, in our revised manuscript, the sentence length statistics of the dataset used are added at Section 4.1 as shown in Table 2, moreover, in order to more intuitively show the sentence length information of the dataset used, we plotted a histogram of the distribution of the sentence lengths as shown in Figure 4.
- Response to comment: “You evaluated your model using accuracy and F1 score. What are these metrics? What formula is used to calculate them? Why these choices? All these details should be provided before analyzing the results.”
The author’s answer: Based on your professional comments, we have added "Section 4.2 Evaluation metrics" to the revised manuscript, which explains in detail how accuracy and Macro-F1 are calculated and why these evaluation metrics are used.
- Response to comment: “How complex is the proposed method compared with other methods?”
The author’s answer: We have added "Section 4.7 complexity analysis" in the revised manuscript. In this section we have selected AHGCN, Sentic-GCN, AIEN+BERT and KHGCN to discuss the time and space complexity, the final results are shown in Table 6.
- Response to comment:” Discuss the impact and implications of the results obtained.”
The author’s answer: Based on your helpful sugguestion, we have added "Section 4.8 Discussion" to the revised manuscript. We discuss the impact and implications, and describe possible directions for future research.
Author Response File: 
Author Response.pdf
Reviewer 2 Report
Comments and Suggestions for AuthorsThe aspect-based sentiment analysis (ABSA) is a crucial task in natural language processing, and this paper addresses some of the limitations present in previous approaches. The authors propose a knowledge-guided heterogeneous graph convolutional network (KHGCN) to enhance ABSA. One significant improvement is the dynamic weight mechanism used to merge subword vectors in the BERT embedding layer, which optimizes the representation of words.
Moreover, the utilization of heterogeneous graphs to fuse various feature associations between words is a commendable approach. This enables the identification of syntactic features specific to the context, contributing to a more comprehensive understanding of sentiment analysis. The incorporation of a knowledge graph into the model is a noteworthy step, as it allows the model to glean additional features from external sources, enriching its knowledge representation.
One of the standout features of this approach is the attention mechanism, which leverages knowledge to enhance aspect-specific representations. Finally, the dynamic combination of semantics features, syntactic features, and knowledge through feature fusion showcases a holistic approach to ABSA.
The experimental results on three benchmark datasets validate the effectiveness of the proposed model, making it a promising advancement in aspect-based sentiment analysis. Overall, this paper offers valuable insights and innovations for researchers and practitioners in the NLP community.
Weaknesses: 1. Complexity: The proposed KHGCN model introduces a high level of complexity due to the dynamic weight mechanism, heterogeneous graphs, and knowledge graph embedding. While these additions enhance the model's performance, they may also make it harder to implement and understand for researchers and practitioners who are new to the field. 2. Computational Resources: The paper doesn't provide detailed information on the computational resources required to train and run the KHGCN model. Complex models with dynamic mechanisms and graph structures may demand significant computational power, which could be a limitation for smaller research teams or organizations with limited resources. 3. Interpretability: The paper discusses various techniques and mechanisms, but it may lack a comprehensive explanation of how decisions are made within the model. This could be a drawback for those who prioritize interpretability in sentiment analysis models. 4. Dataset Specificity: The paper mentions that the model's effectiveness is demonstrated on three benchmark datasets. It would be beneficial to investigate how well the model generalizes to other datasets or domains, as performance on benchmark datasets doesn't always translate to real-world applications. 5. Training Data Size: While the paper claims improvements with a knowledge graph and attention mechanism, it is essential to consider the availability of large knowledge graphs and the volume of data required for training, which may not be feasible for all applications. 6. Evaluation Metrics: The paper mentions the effectiveness of the model based on experiments but doesn't provide detailed insights into the choice of evaluation metrics used or their potential limitations. Addressing these weaknesses could enhance the comprehensibility and practicality of the proposed KHGCN model for broader adoption in the field of aspect-based sentiment analysis.
Strengths:
1. Innovative Approach: The proposed knowledge-guided heterogeneous graph convolutional network (KHGCN) represents an innovative approach to aspect-based sentiment analysis (ABSA). It addresses the limitations of previous methods and introduces novel techniques for improving sentiment analysis.
2. Dynamic Weight Mechanism: The dynamic weight mechanism used to merge subword vectors in the BERT embedding layer is a noteworthy strength. This mechanism optimizes word representations and can lead to more accurate sentiment analysis by capturing nuanced word-level features.
3. Heterogeneous Graphs: The use of heterogeneous graphs to fuse different feature associations between words is a powerful concept. It allows the model to capture complex relationships between words and contextual information, enhancing the model's ability to understand sentiment in context.
4. Knowledge Graph Embedding: Incorporating a knowledge graph into the model is a significant strength. This enables the model to learn additional features from external sources, which can lead to a richer knowledge representation for sentiment analysis.
5. Attention Mechanism: The attention mechanism is a valuable addition, as it allows the model to focus on relevant information and improve aspect-specific representations. This attention mechanism can enhance the model's ability to extract sentiment information from text.
6. Feature Fusion: The dynamic combination of semantics features, syntactic features, and knowledge through feature fusion is a holistic approach. It leverages multiple sources of information to provide a comprehensive understanding of sentiment in text.
7. Empirical Validation: The paper provides experimental results on three benchmark datasets, demonstrating the effectiveness of the KHGCN model. Empirical validation is essential in showcasing the practical benefits of the proposed approach.
Overall, the strengths of the KHGCN model lie in its innovative techniques, dynamic mechanisms, and the ability to leverage external knowledge sources to improve aspect-based sentiment analysis. These strengths make it a promising contribution to natural language processing and sentiment analysis.
My evaluation is positive for your work; however, I recommend reviewing the background and considering significant contributions such as 'Knowing knowledge: epistemological study of knowledge in transformers.'"
Author Response
- Response to comment:” Complexity: The proposed KHGCN model introduces a high level of complexity due to the dynamic weight mechanism, heterogeneous graphs, and knowledge graph embedding. While these additions enhance the model's performance, they may also make it harder to implement and understand for researchers and practitioners who are new to the field.”
The author’s answer: Thank you very much for your comment. The embedding of heterogeneous graphs and knowledge graphs is a work that does not require repetition. The results of embedding can be saved, and can be directly used in the model training phase. We have added "Section 4.7 complexity analysis" in the revised manuscript, which discusses the time and space complexity of several models, and we hope it can answer your questions.
- Response to comment:” Computational Resources: The paper doesn't provide detailed information on the computational resources required to train and run the KHGCN model. Complex models with dynamic mechanisms and graph structures may demand significant computational power, which could be a limitation for smaller research teams or organizations with limited resources.”
The author’s answer: Thank you very much for your comments, computational resources are certainly very important in the ABSA task, we have added detailed information on the computational resources used: “The KHGCN model is trained on RTX 3080Ti, 12 vCPU Intel(R) Xeon(R) Silver 4214R CPU @ 2.40GHz, and the model is built using PyTorch 2.0.0”. With the help of the pre-trained model BERT, the computational resources required have been significantly reduced.
- Response to comment:” Interpretability: The paper discusses various techniques and mechanisms, but it may lack a comprehensive explanation of how decisions are made within the model. This could be a drawback for those who prioritize interpretability in sentiment analysis models.”
The author’s answer: Thank you very much for your comment, which we take very seriously. Model interpretability is a very important research direction in the field of deep learning. In the revised manuscript, we have added "Section 4.9. case study", which is an attention visualisation of how semantic, syntactic and external knowledge focuses on the internal vocabulary of the text, as shown in Fig. 5, and we hope that this can be a complementary approach to model interpretability, thank you again for your professional comment.
- Response to comment:”Dataset Specificity: The paper mentions that the model's effectiveness is demonstrated on three benchmark datasets. It would be beneficial to investigate how well the model generalizes to other datasets or domains, as performance on benchmark datasets doesn't always translate to real-world applications.”
The author’s answer: Thank you very much for your comment, it is indeed important for the model to be generalizable across different domains. In the revised manuscript, we statistically analyzed the used datasets, as shown in Table 2 and Figure 4. Most of the sentence text lengths are distributed ranging from 0-50, and the datasets used are the review data of Laptop and Restaurant, whose data itself contains different statistical features and rich diversity. We think they can represent the characteristics of the data in different domains to a certain extent.
the dataset used in this paper mainly covers Laptop and Restaurant, we will apply the model to other domains in our subsequent research work, thank you again for your comment.
- Response to comment:” Training Data Size: While the paper claims improvements with a knowledge graph and attention mechanism, it is essential to consider the availability of large knowledge graphs and the volume of data required for training, which may not be feasible for all applications.”
The author’s answer: Thank you very much for your comment, the aspects you have considered are very meaningful, a large amount of training data is necessary for large knowledge graphs. However, the knowledge graphs used in this paper are publicly available. It is unfortunate that we have not yet obtained a larger dataset.
- Response to comment:” Evaluation Metrics: The paper mentions the effectiveness of the model based on experiments but doesn't provide detailed insights into the choice of evaluation metrics used or their potential limitations. Addressing these weaknesses could enhance the comprehensibility and practicality of the proposed KHGCN model for broader adoption in the field of aspect-based sentiment analysis.”
The author’s answer: Thank you very much for your comment, according to your suggestion, we have added "Section 4.2. Evaluation metrics" in the revised manuscript to explain in detail the evaluation metrics used in this paper, we hope that this can answer your question, thank you very much for your professional suggestions to us.
Author Response File: Author Response.pdf
Reviewer 3 Report
Comments and Suggestions for AuthorsThe paper presents a neural architecture to solve the problem of aspect-based sentiment analysis. The proposed model is justified and explained in its functions and structures, and its performance is validated by comparing it with a pool of state-of-the-art models. Overall, the paper is well organized and the contribution is innovative.
I provide several comments that may improve the presentation of the contribution:
- The abstract can also contain that the proposed approach has been compared with many competitors in the literature and can report summary results to make the reader immediately understand the impact of such a neural architecture.
- In line 18, the authors write ABSA but since it is a new text, different from the abstract, this Reviewer suggests that the extended version of the acronym be expressed again.
- This Reviewer suggests a better description of the application contexts in which the proposed approach may be used. One area that I think could benefit from sentiment analysis is rehabilitation robotics. Integrating Physical and Cognitive Interaction Capabilities in a Robot-Aided Rehabilitation Platform could indeed improve patient performance and having a sentiment understanding of what patients say further improves the outcome of the interaction.
- In lines 67-99, the proposed approach is already greatly anticipated. I suggest the authors summarise this part and detail the functionality and methods of the approach in the appropriate section. In this part, the authors should only clearly and concisely define the objective of the work.
- This Reviewer appreciated the presence of a "Related Works" section to better narrate the existing literature. I suggest the authors add a small summary section summarising what limitations of the literature this contribution aims to address.
- In lines 173-174, is there an error in the references?
- Table 1, how was the validation done? Hold-out?
- Section 4.2, are all baseline models for ABSA? If yes, I suggest the authors recall how the proposed approach differs from these literature models.
- Table 2, write down what the bold and underlined mean. First and second best?
Comments on the Quality of English LanguageMinor editing of English language required
Author Response
- Response to comment: “The abstract can also contain that the proposed approach has been compared with many competitors in the literature and can report summary results to make the reader immediately understand the impact of such a neural architecture.”
The author’s answer: Thank you very much for your suggestion, we have added the following to the Abstract section: “ Experiments on three public datasets demonstrate that our model performs 80.87%, 85.42% and 91.07% on the accuracy metric, which is an improvement of more than 2% compared to other benchmark models based on HGCN and BERT.”
- Response to comment: “In line 18, the authors write ABSA but since it is a new text, different from the abstract, this Reviewer suggests that the extended version of the acronym be expressed again.”
The author’s answer: Thank you for your careful scrutiny, we are sorry for our carelessness and we have corrected the content.
- Response to comment: “This Reviewer suggests a better description of the application contexts in which the proposed approach may be used. One area that I think could benefit from sentiment analysis is rehabilitation robotics. Integrating Physical and Cognitive Interaction Capabilities in a Robot-Aided Rehabilitation Platform could indeed improve patient performance and having a sentiment understanding of what patients say further improves the outcome of the interaction.”
The author’s answer: We sincerely thank you for your valuable comments. We have added the following to the revised manuscript on lines 458-463: “The model proposed in this paper can be beneficial in domains that require sentiment analysis support, such as online shopping, where sellers can analyze the likes and dislikes of various types of products through customer reviews and better adjust the shelving strategy of the products. Another area is social media monitoring, which can help companies track user sentiment on social platforms and understand the public's views on specific issues, products or events.”.
- Response to comment: “In lines 67-99, the proposed approach is already greatly anticipated. I suggest the authors summarise this part and detail the functionality and methods of the approach in the appropriate section. In this part, the authors should only clearly and concisely define the objective of the work.”
The author’s answer: We appreciate your professional comments on our paper, and based on your suggestions, we have summarized and streamlined this section. You can view the streamlined content at lines 67-81 in the latest manuscript submission.
- Response to comment: “This Reviewer appreciated the presence of a "Related Works" section to better narrate the existing literature. I suggest the authors add a small summary section summarising what limitations of the literature this contribution aims to address.”
The author’s answer: We are very thankful for your valuable comments, and we agree that this is an excellent suggestion, and in the “Related Works” section, we have added “Subsection 2.4. Limitations “to summarize the limitations of the existing literature.
- Response to comment: “In lines 173-174, is there an error in the references?”
The author’s answer: Thank you for your careful scrutiny, we are sorry for our carelessness. In our resubmitted manuscript, the reference is correctly cited. Thanks for your correction.
- Response to comment: “Table 1, how was the validation done? Hold-out?”
The author’s answer: The dataset used in this paper is a public dataset, which has been divided into a training set and a testing set at the time of acquisition as shown in Table 1, which is used in the process of training the model through the training set, and the performance of the model is verified through the testing set. In the original public dataset, the Lap14 dataset has a total data volume of 2966 text sentences, the ratio between its training set and testing set is closer to 8:2, while in the other two datasets Rest15, Rest16, the ratio is closer to 7:3.
- Response to comment: “Section 4.2, are all baseline models for ABSA? If yes, I suggest the authors recall how the proposed approach differs from these literature models.”
The author’s answer: Thank you very much for your comment. In response to your comments, we have elaborated the proposed model in Section 4.3, which tells the differences between the proposed model and these literature models.”
- Response to comment: “Table 2, write down what the bold and underlined mean. First and second best?”
The author’s answer: We sincerely thanks for your comments. In Table 2, bold represents the best result, and underlined expressions represents the second best result. We have clarified this in our resubmitted manuscript: "Performance comparison (The best result for each model is shown in bold, while the second-best result is underlined) "
Author Response File: Author Response.pdf
Round 2
Reviewer 1 Report
Comments and Suggestions for AuthorsI think that the authors have made the necessary efforts to improve the quality of the paper according to the comments made before. I don't have any more suggestions for further revisions.
Comments on the Quality of English LanguageNo comments.
