Aggression Detection in Social Media from Textual Data Using Deep Learning Models

Round 1

Reviewer 1 Report

The work is promising and of interest to the research community and industry involved in automated detection of cyber-bullying on social media. The idea appears to be novel and the ambition is high. However, there are several key issues with the paper that need to be addressed for it to be published as indicated below:

 

1) Method: The study lacks sufficient transparency in scientific method, particularly in terms of the dataset used. There is no reference to to the source and there is no mention of the language of the dataset, which are critically important to assess the reliability of the method. To address this, the authors could point to the dataset, how it was chosen, how reliable is the manual coding (of emotion labels) of the training dataset, what content it represents exactly, etc. Additionally, what was the open-source code that was used (which programming language(s), which libraries (Python and NLTK, R's neuralnet, etc.), and which data banks were used. One key quality factor nowadays in any data-driven research is replicability and the article ranks rather poorly in that respect. Additionally, there is no mention of methodological limitations of the study. It is expected that some key aspects of language in terms of the used training dataset would limit the applicability of this approach. This was not mentioned. 

 

2) Literature: Much of the literature is relevant and useful, but a key set of literature on the use of emoticons in social media seems to be missing. Why are those of value? What earlier research has shown in terms of their significance in emotions in Twitter and social media at large and their connection to the overall degree of negativity of the textual message. The difference between emoticons, emojis, GIFs, etc.,particularly in an increasingly fast-paced video-based social media landscape. Since the study's dataset language is not defined, it is difficult to know what literature is missing, although it may well be that an Arabic dataset was used since there has been some emphasis on Arabic in the literature review.

3) Results/conclusions: The representation of the data with the various graphs and tables appear to be quite good. However, the size of the sample is relatively small to derive conclusive results, which is something that could have been noted in the conclusion since applying the method on bigger and more diverse datasets in different languages wold be necessary. This brings me back to the point of limitations, which could have been emphasized again in the conclusion while also pointing to the path forward to try to expand the dataset in terms of size and different languages and types of social media contents.

4) Language: there are many linguistic errors as if the paper was not proof-read. This made reading some parts particularly difficult to understand, which unfortunately weakened the scientific quality of the study.

If those above aspects are properly addressed, I do believe that the paper could be worth publishing.

Author Response

Hello Respected Editor,

 

Please find our revised manuscript with highlighted changes and answers to reviewer’s comments for your consideration for publication in Applied Sciences Journal. We have thoroughly revised and modified the manuscript according to the comments of the reviewers. Detailed responses to the comments are listed below point by point.

 

Thanks and Regards

Author Response File: Author Response.doc

Reviewer 2 Report

This paper addresses timely topic on aggression detection on textual data set. While the proposed method seems sound with proper combination of existing techniques, there are some ambiguities in the paper.

 

1. Eq 5) is not clear.

         I think eq 5) is one of the key aspects of the paper. However, the equation is not readable. For instance, what is Fxi? Also what is Xi? I think the authors did not  use right way to express subscripts and so on which prevents clear understanding of how one can extract emotional features from the raw text data. Please clarify the equation.

2. Feature selection via MI is not clear.

Finding a subset (25) of features that best expresses the input data via MI seems a nice way for such feature selection. But how can one solve 7)? If total features were, say, 100, does one need to do combinatorial search for 100 C 25 times? How did the authors solve eq 7? And would be nice to include total number of features after eq 7).

3. Eq 6) is confusing.

It is not common to put '%' like eq 6). It even seems that it is outside the logarithm. Please make it clear by using some standard notation of MI.

 

4). Reference for eq 6) is missing.

Since usage of MI for deep learning is quite popular, so I think there should be some prior works that considered MI for feature selection. I'm not familiar with feature selection with MI but there are many other works that does similar things such as information bottleneck or InfoMax. Please add some references for the feature selection to make eq 6 and 7 be more concrete. It would be nice if there's some reference that particularly do 7 as in this paper.

 

 

 

Author Response

Hello Respected Reviewer,

Please find our revised manuscript with highlighted changes and answers to your comments . Detailed responses to the comments are listed in attached file.

Thanks and Regards

Author Response File: Author Response.docx

Reviewer 3 Report

The paper needs major revision in English grammar and sentence formation.

Under section 3 (Methodology), 3.1, FN and FP have been incorrectly defined. FP are 'incorrectly classified as cyber aggression', but it was defined otherwise. same as FN. They need to be corrected.

In section 3.3.1 (Discrete Emotions), explain formula 5.

In section 3.4 (Feature selection), explain what features were selected. On page 8, authors mentioned that "one feature set is obtained as a result of Word2Vec embedding model..." - Explain what features were obtained that were combined with emotional features.

Author Response

Hello Respected Reviewer,

Please find our revised manuscript with highlighted changes and answers to your comments . Detailed responses to the comments are listed in attached file.

Thanks and Regards

Author Response File: Author Response.doc

Reviewer 4 Report

This investigation addresses one of the most important problems of democratic societies. It could be improved in the problematization of concepts such as "Hate Speech" and Cyberbullying, exploring their specific linguistic dimension, namely the performativity of language.

Regarding the methodology, it is based on an extensive literature review, it is solid and clear.

The conclusions are well supported.

It contributes to the development of this area of study, specifically with regard to the emotional dimension, highlighting the possibility of application to languages other than English.

Author Response

Hello Respected Reviewer,

Please find our revised manuscript with highlighted changes and answers to your comments . Detailed responses to the comments are listed in attached file.

Thanks and Regards

Author Response File: Author Response.docx