Automated Detection of Persuasive Content in Electronic News

Round 1

Reviewer 1 Report

In this paper, the authors propose a strategy for detecting persuasive content in electronic news articles. However, there are several aspects that should be improved.
Firstly, the authors should provide insights into the robustness of their results, particularly given the relatively small dataset consisting of only 1700 data points. Are the results statistically significant? Additionally, the paper could benefit from a more comprehensive review of related works in the field. For example, exploring works related to networked representation of texts could offer valuable insights into language nuances relevant to the problem at hand. Some examples to consider are the following papers "Authorship attribution based on life-like network automata" and "Research on topic detection and tracking for online news texts."
The authors should justify their choice of evaluation metrics.
Finally, the motivations for future work in this area are not clear in the current paper. Extending the conclusion section to outline potential avenues for future research.

Moderate editing of English language required.

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Reviewer 2 Report

Your paper should be revised as following.

- Extractive Text summarization that you should use Sentence Transformer [1] for.

- BERT-CNN is also a effective method for your task [2].

- You examine the lack of methods for summarization, especially for monolingual and multilingual BERT [3].

References

• [1] REIMERS, Nils; GUREVYCH, Iryna. Sentence-bert: Sentence embeddings using siamese bert-networks. arXiv preprint arXiv:1908.10084, 2019.
• [2] QUOC TRAN, Khanh, et al. Vietnamese hate and offensive detection using PhoBERT-CNN and social media streaming data. Neural Computing and Applications, 2023, 35.1: 573-594.
• [3] Huy Quoc To, Kiet Van Nguyen, Ngan Luu-Thuy Nguyen, and Anh Gia-Tuan Nguyen. 2021. Monolingual versus multilingual BERTology for Vietnamese extractive multi-document summarization. In Proceedings of the 35th Pacific Asia Conference on Language, Information and Computation, pages 692–699, Shanghai, China. Association for Computational Lingustics.

You should use Grammarly to check your paper.

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Reviewer 3 Report

This paper proposes a new method for detecting persuasive content in Indonesian language news articles by combining text summarization, word embedding, and deep classifier technology. The training data was collected and annotated by the author from various Indonesian websites, resulting in a dataset of 1708 samples, including 854 positive samples and 854 negative samples.

This work involved a substantial amount of effort, with three annotators labelling 1.7k news articles out of a total of 12k articles. All the data used in this paper is original. Additionally, this paper explored multiple algorithm combinations and identified a state-of-the-art model for detecting persuasive content, which achieved an accuracy of 0.95, significantly higher than other existing methods.

Below are some comments for the authors to consider:

1. As shown in the performance comparison results, TextRank outperforms LSA, it appears that the text summarization component does not contribute significantly to the model's performance. This may require further explanation.
2. Your proposed model demonstrates better performance than other methods. It is important to note that the tasks for which these models were designed are different. Evaluating other models with your unique data and task is not fair.

N/A

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Reviewer 4 Report

This manuscript proposes a method for detecting persuasive documents from news and is well-described and -organized, but the followings should be improved;

- Related Work:

1) LSA, TextRank, BiLSTM, and CNN should be described for readers.

2) When written in deductive style (the topic sentence is put in the head of the paragraph), the body should be written in the order presented. For example, the body should be written in the following order; DNN, DNN-based summarization, fake news detection.  

- Models

1) Figure 1 is almost same as some figure in the page of https://www.analyticsvidhya.com/blog/2018/11/introduction-text-summarization-textrank-python/, so must be modified due to copyrights.

2) Figure 1 and 3 is inconsistent. Boxes in Figure 1 represent data or output of some process, but those in Figure 3 represent processes. In usual, Boxes in flowcharts represent processes.

3) Section 3.1 and 3.2 are very abstract, that is, Section 3.1 and 3.2 describe general text summarization and document classification like a literature review, respectively. In general, the body is written in detail and your architecture is described in detail.

4) In Figure 2 (b), it is unclear that the relation between BiLSTM and MaxPooling.  Which one do you use CLS or all bidirectional hidden vectors?

- Experiments

1) In Figure 4, which one is the distribution of persuasive news: left or right?

- Please refer the following references published in this journal:

1. A Systematic Literature Review on Fake News in the COVID-19 Pandemic: Can AI Propose a Solution?

2. Fake Sentence Detection Based on Transfer Learning: Applying to Korean COVID-19 Fake News

3. The Detection of Fake News in Arabic Tweets Using Deep Learning

4. Efficient Fake News Detection Mechanism Using Enhanced Deep Learning Model

- Please modify the references according to the format regulation of this journal.

Please check some spelling: For example, dan -> and on Line 200. Omitted below.

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Reviewer 5 Report

This paper designed a system of text summarization to shorten sentences without persuasive content and used classifiers model to detect those persuasive indication. Then, the paper compared the performance of Latent Semantic Analysis and TextRank in text summarization methods, and used CNN and BiLSTM to detect persuasive news. The method is practical, and the experiments are adequate. However, this paper does not present any new method, and does not demonstrate the advance of the system compared with current systems. The novelty is limited.

Need be improved

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Round 2

Reviewer 4 Report

All suggestions are reflected in revised version.

Author Response

REVIEWER-4

Comment#1

1. All suggestions are reflected in revised version.

Response:

Thank you for the insightful suggestions. We would like to express our gratitude for the provided revisions.

in text classification tasks.

Reviewer 5 Report

The authors improve their work; however, the method is still not new and the experiments are far from enough. More state-of-the-art comparison methods should be considered, and their is no comparison result about persuasive news.  

NA

Author Response

REVIEWER-5

Comment#1

1. The authors improve their work; however, the method is still not new and the experiments are far from enough.

Response:

Thank you for the comment. Our novelty lies in both the methodology and the problem domain. While we utilized existing architectural components, our innovation stems from the integration of text summarization with deep learning, a previously unexplored approach within the field of news. Furthermore, our dataset specifically addresses persuasive news, and it is relatively novel within this domain. We have explained our novelty (page 3, line 100-104 and line 128-131) as follows.

This work has an objective to propose an architecture for the detection of persuasive news using text summarization methods and pre-trained deep learning models to make readers aware whether or not a news article contains persuasive elements. Our comparison studies in these experiments have addressed how well different combinations of text summarizing techniques and deep learning models perform.

Due to the lack of a computational method for detecting persuasive news, we conducted a literature review on studies that compare deep learning models, compare studies that combine text summarization methods with deep learning models, and studies that are closely related to the detection of fake news

Comment#2

2. More state-of-the-art comparison methods should be considered, and their is no comparison result about persuasive news.

Response:

Thank you for the insightful suggestions. We have added state-of-the-art text classification methods that are currently trending and applied them to the persuasive news dataset. (page 14, line 458-470) as follows.

In the years following 2010, the development of deep learning methods has brought about significant advancements in the field of text classification [31]. Various model architectures have been proposed and applied to text classification tasks. In this context, methods such as Machine Learning Perceptron (MLP), Convolutional Neural Networks (CNN), Attention, and Transformers are among the commonly used models. One relevant study was conducted by Kumar [32], in which he detected fake news using various deep learning-based approaches. This research also incorporated ensemble deep learning techniques and models with attention mechanisms. We attempted several text classification model architectures using a persuasive news dataset. Table 7 presents the evaluation ma-trix of various text classification model architectures. Our proposed method yielded significant performance that surpasses various other methods. These results indicate that our proposed approach has strong potential to outperform various methods in text classification tasks.