Fuzzy Rough Nearest Neighbour Methods for Aspect-Based Sentiment Analysis
Round 1
Reviewer 1 Report
The authors proposed a manuscript that explores machine learning approaches based on fuzzy rough sets, which, in their opinion, are more interpretable than complicated state-of-the-art models. Their research examines a pipeline that consists of the three previously described processes and applies FRNN-OWA classification approaches using text embeddings based on transformers. However, some issues needs to be addressed as follows
1. Provide comparisons with non fuzzy approaches to better display the interpretability proposed by the authors
2. Results and performance evaluation of the proposed fuzzy rough set approach should be compared to other relevant methods.
3. The abstract has no discussion of results: The abstract mentions an "error analysis," but does not provide any information on the actual results of the study
Author Response
Dear reviewer,
We are grateful for your feedback and valuable comments. We did our best to address these comments and incorporate them in our work within one week. Below, we provide comments for each question:
- “Provide comparisons with non fuzzy approaches to better display the interpretability proposed by the authors”
We have included a more thorough discussion regarding the interpretability issue. Specifically, in Section 2.3, “Overview of interpretability for text classification methods” (page 3), we described several ways of models’ interpretability classification techniques, and positioned our approach w.r.t. the state-of-the-art approaches, highlighting their differences.
- “Results and performance evaluation of the proposed fuzzy rough set approach should be compared to other relevant methods.”
Here we would like to mention our own previous works, where we compared FRNN-OWA-based methods with neural networks or transformers-based solutions. In these experiments, we received comparable high results and the same way of explainability that is described in the current study. We provided additional information about these previous approaches in the Introduction (page 2), and in Section 2.4, “2.4. Description of our previous work”.
- “The abstract has no discussion of results: The abstract mentions an “error analysis,” but does not provide any information on the actual results of the study”
We revised our abstract significantly, describing our solutions and obtained results.
Best regards,
Olha Kaminska
The corresponding co-author
Reviewer 2 Report
Overall, the work is good and is nicely written.
Presentation can be improved. Section 6- Error Analysis, can be represented in better way, may be in tabular form or through some graphics etc. Its plain text, which makes the manuscript less presentable.
Author Response
Dear reviewer,
We are grateful for your feedback and valuable comments. We did our best to address these comments and incorporate them in our work within one week. Below, we provide comments for each question:
“Presentation can be improved. Section 6- Error Analysis, can be represented in better way, may be in tabular form or through some graphics etc. Its plain text, which makes the manuscript less presentable”
We re-formatted Section 6, “Error analysis”, so now every test sample we consider is presented in the “Example” environment. We replaced all quotation marks around classes with the bold font (“positive” became positive) and highlighted the terms inside examples instead of using the bold font (In fact, I will likely buy a second became In fact, I will likely buy a second). We also gathered all neighbouring training samples for each test instance in a separate table with corresponding training reviews’ text and labels (Table 3,4,5).
Besides Section 6, we also modified some other parts of our paper to provide a more accessible presentation; for example, we modified Appendix A.2, “Emotion Cost Matrix”, so it will be horizontal and easier to read.
Best regards,
Olha Kaminska
The corresponding co-author
Reviewer 3 Report
In the manuscript entitled “Fuzzy Rough Nearest Neighbour Methods For Aspect-Based Sentiment Analysis” the authors used machine learning based on fuzzy rough sets for Aspect-Based Sentiment Analysis with three level of categorization. The manuscript is interesting but the authors needs to consider the following
1. Abstract: Abstract too short and generic and abbreviation used without providing full text i.e. FRNN-OWA. The authors needs to significantly improve abstract and provide details of the results they have obtained for the three sub tasks.
2. Introduction: Some typos i.e. “In our paper, considering the fuzzy nature of textual data and human emotion, we we investigated fuzzy-rough-based classification models [6] for each subtask of the ABSA challenge”. Lot of text from this section can be moved to “literature review” section. This section must be improved.
3. Literature review: Literature review is limited. The authors need to move related work from other sections into this section. The authors needs to add more references in this section. It is also recommended to add a sub section at the end of this section to conclude the literature review and highlight research gap.
4. Conclusion: Can be improved to clearly highlight the results and overall contribution.
5. Overall: Some references are old i.e. 2004, 2010, 2013 etc. Sort out formatting issues i.e. appendix A.2 is between references. It is recommended to improve the flow of the manuscript by moving all related work in literature review section. The authors presented the results in percentage on the text but in table 2 the results are presented as a decimal number. The authors must use a consistent approach. Provide full text when an abbreviation is used for the first time.
Author Response
Dear reviewer,
We are grateful for your feedback and valuable comments. We did our best to address these comments and incorporate them in our work within one week. Below, we provide comments for each question:
- “Abstract: Abstract too short and generic and abbreviation used without providing full text i.e. FRNN-OWA. The authors needs to significantly improve Abstract and provide details of the results they have obtained for the three sub tasks.”
We updated the Abstract significantly, describing our solutions and obtained results. We added explanations for the mentioned term, highlighted our motivation and novelty, and described our solution for three classification tasks hidden in the ABSA challenge and our three pipelines, together with provided results. In the end, we also mentioned other authors working with the Dutch version of this dataset and that we obtained comparable results.
- “Introduction: Some typos i.e. “In our paper, considering the fuzzy nature of textual data and human emotion, we we investigated fuzzy-rough-based classification models [6] for each subtask of the ABSA challenge”. Lot of text from this section can be moved to “literature review” section. This section must be improved.”
We carefully examined the whole text to fix typos. We also moved a part of the Introduction section to Related Works. We added to the Introduction an overview of the comparison of our solution with the state-of-the-art approaches performed in our previous works (page 2).
- “Literature review: Literature review is limited. The authors need to move related work from other sections into this section. The authors needs to add more references in this section. It is also recommended to add a sub section at the end of this section to conclude the literature review and highlight research gap.”
Besides the part of the text from the Introduction section, we added information about the interpretability of models for text analysis in Section 2.3, “Overview of interpretability for text classification methods”, on page 3. We also split this section into subsections, where the last one is a summary of all considered works (page 4).
- “Conclusion: Can be improved to clearly highlight the results and overall contribution.”
We added more explanations to the Conclusion section to highlight our contribution.
- “Overall: Some references are old i.e. 2004, 2010, 2013 etc.”
After examining all our references, we cannot delete the old ones since they are important to describe the development of the ABSA task or establish fuzzy-rough-based methods. However, we added more recent references since we expanded our Related Work section.
- “Sort out formatting issues i.e. appendix A.2 is between references.”
We re-formatted appendix A2, so now it has a horizontal shape and is located before references (page 14). We also improved the presence of examples in Section 6, “Error Analysis”.
- “The authors presented the results in percentage on the text but in table 2 the results are presented as a decimal number.”
We kept percentages in the Abstract to give readers an instant impression of our results, but we replaced them with decimal numbers in the Conclusion; however, they have the same meaning as percentages since this is accuracy.
Best regards,
Olha Kaminska
The corresponding co-author
Reviewer 4 Report
Fuzzy Rough Nearest Neighbour Methods For Aspect-Based Sentiment Analysis by Olha Kaminska, Chris Cornelis, and Veronique Hoste.
An interesting research topic is discussed in this paper. Thus, the paper submitted is a valuable contribution. It is, however, necessary to prepare the article for publication in a journal. In my opinion, the following points need to be addressed:
1. Figure 1 is of inferior quality. Please upload a better-quality image, or redraw it in latex.
2. Increase the font size of the labels in the second figure.
3. Figure captions should better explain what is being illustrated.
4. Tables would be a better format for presenting cost matrices in appendices.
5. The abbreviation FRNN-OWA is given in the abstract, but the explanation is only on page 3.
6. In the introduction, it should be clearly stated what contributions the article makes to science.
7. The first line space in the paragraphs starting on lines 361, 372, 381, 395, 404, 418, and 431 is not needed. Please double-check.
8. For the examples in section 6, you could use bullets or the Latex "Example environment".
Author Response
Dear reviewer,
We are grateful for your feedback and valuable comments. We did our best to address these comments and incorporate them in our work within one week. Below, we provide comments for each question:
- “Figure 1 is of inferior quality. Please upload a better-quality image, or redraw it in latex.”
We generated a higher-quality version of this picture and replaced it in the paper (page 5).
- “Increase the font size of the labels in the second figure.”
We did it for each subfigure of Picture 2 (page 5).
- “Figure captions should better explain what is being illustrated.”
We expanded captions for the majority of pictures and tables in our paper.
- “Tables would be a better format for presenting cost matrices in appendices.”
We changed the appendixes to a table format and updated appendix A2, so now it has a horizontal design and is easier to read.
- “The abbreviation FRNN-OWA is given in the abstract, but the explanation is only on page 3.”
We added an explanation for this term and expanded the Abstract to clarify what was done in this paper and what results we obtained.
- “In the introduction, it should be clearly stated what contributions the article makes to science.”
We highlighted our motivation and novelty in the Abstract. Meanwhile, in the Introduction, we moved part of the content to the Related Work section and added an overview of the comparison of our solution with the state-of-the-art approaches performed in our previous works (page 2).
- “The first line space in the paragraphs starting on lines 361, 372, 381, 395, 404, 418, and 431 is not needed. Please double-check.”
It was an issue with our previous formatting of Section 6, but now we changed it, and those extra first lines disappeared.
- “For the examples in section 6, you could use bullets or the Latex “Example environment”.”
We re-formatted Section 6, “Error analysis”, so now every test sample we consider is presented in the “Example environment”. We replaced all quotation marks around classes with the bold font (“positive” became positive) and highlighted the terms inside examples instead of using the bold font (In fact, I will likely buy a second became In fact, I will likely buy a second). We also gathered all neighbouring training samples for each test instance in a separate table with corresponding training reviews’ text and labels (Table 3,4,5).
Best regards,
Olha Kaminska
The corresponding co-author
Author Response File: 
Author Response.pdf
Round 2
Reviewer 3 Report
The authors considered the points raised and improved their manuscript. I am happy with the current version of the the manuscript and after final English language review (which is already part of the final process), the paper can be published.
