TrendFlow: A Machine Learning Framework for Research Trend Analysis
Round 1
Reviewer 1 Report
The paper is well organized and arranged in my opinion needs no edits. I recommend accepting the paper in its current form.
--
Best Regards,
Author Response
Thank you very much for the evaluation of this manuscript.
Reviewer 2 Report
This paper proposes a backbone network VoVNetv4 with a staged single aggregation and a regional single aggregation module ROSA. The AdaptiveStage module is also built. Divide the framework into two parts, the first-level node and the second-level node. Taking the age of the human body as the gradient information, the elderly are used as the senior resource for priority detection. The real-time detection of falls is realized to solve the problems of poor real-time performance and non-obvious detection gradient in the traditional fall detection framework.
1. When introducing the research background, there is a lack of a part that draws out your own research based on the shortcomings of existing research.
2. In addition, there is a problem of misspelling words in the subtitle introduced by the Rosa module.
3. The algorithm analysis title of this paper is not comprehensive.
4. The display part of the experimental results is not clear enough together, it is recommended to explain it in points.
Minor editing of English language required
Author Response
Thanks for your comment. It may be that the system is wrong. This review comment has nothing to do with the subject of this article, so I cannot reply.
Reviewer 3 Report
The conclusion is that it does not provide a clear summary of the main findings or results of the study. While it mentions that the evaluation demonstrates that TrendFlow performs well in identifying research trends and generating keyphrases, it does not provide specific details about the evaluation metrics used or the performance comparisons with existing methods. Without this information, it is difficult to assess the significance and reliability of the results.
Additionally, the conclusion briefly mentions limitations of the TrendFlow framework, such as constrained summarization, potential domain-specific adaptations, and scalability concerns. However, it does not elaborate on these limitations or provide examples or evidence to support these claims. The conclusion could be strengthened by providing more specific explanations of these limitations and discussing their potential impact on the effectiveness and applicability of the framework.
Author Response
Response: Thank you sincerely for your insightful feedback regarding the conclusion of our study. We genuinely appreciate your time and efforts to improve our manuscript.
In response to your points, I would like to clarify that our evaluation metrics have been exhaustively discussed in Section 4 of the paper. However, we understand from your comments that these details might not have been adequately reflected in the conclusion section. We will endeavor to address this concern by providing a more comprehensive summary of our main findings and their implications, including the specific evaluation metrics used. The details in Section 4.2 are as follows：
-----------------------------------------------------------------------------------------
we use standard clustering evaluation metrics, such as the Adjusted Rand Index (ARI) [49], Silhouette Score [50], and Calinski-Harabasz Index (CHI) [51]. We compute these metrics for each clustering algorithm and compare their performance.
-----------------------------------------------------------------------------------------
In addition, you rightly mentioned the brief discussion of the limitations of our TrendFlow framework in the conclusion. We would like to emphasize that a more in-depth exploration of these limitations, including their potential impacts and examples, is present in Section 6 of the paper. Nevertheless, we acknowledge that the summary of these aspects in the conclusion could be improved. The details in Section 4.2 are as follows：
-----------------------------------------------------------------------------------------
We adopt the same evaluation metric as used in the original KeyBART paper [4] to select the best configuration on validation set and for evaluation on the test set. The chosen metric is the F-score@M, which is a harmonic mean of precision and recall, considering the top-M generated keywords. By employing the F-score@M, we can compare our KeyBART-adapter with the KeyBART, ensuring a fair evaluation and allowing us to identify potential improvements or drawbacks in our approach.
-----------------------------------------------------------------------------------------
As for comparing with existing methods, as research trend analysis is a novel task, and in the experiments. We only compare different configurations of clustering and keyphrase generation module, we have added following sentences in the Conclusion:
We have done both quantitative and qualitative evaluations on different configurations of TrendFlow. Our evaluation demonstrates that TrendFlow with KeyBart-adapter as research trend generator performs well in identifying research trends and generating keyphrases that effectively represent these trends.
Author Response File: 
Author Response.pdf
Reviewer 4 Report
This paper presents a framework TrendFlow for analyzing research trends that utilizes machine learning and deep learning techniques. I think it is very interesting. Researchers in NLP or other areas can benefit from this study. Some comments are:
1) Section 2 should be improved to focus on topics that most related to this study;
2) Can you give some case studies?
3) English in this study should be improved.
English in this study should be improved.
Author Response
General comments:
This paper presents a framework TrendFlow for analyzing research trends that utilizes machine learning and deep learning techniques. I think it is very interesting. Researchers in NLP or other areas can benefit from this study. Some comments are:
We thank the reviewer for the positive evaluation. We have carefully modified the paper following your suggestions.
- Section 2 should be improved to focus on topics that most related to this study;
Response: In response to your first comment about improving Section 2, we understand your suggestion to focus more specifically on topics most related to our study. However, we would like to highlight that our TrendFlow framework comprises of submodules that tackle various tasks, including Text Encoding, Clustering, and Keyphrase Generation. Thus, we have discussed relevant literature for these areas in Section 2.
- Can you give some case studies?
Response: Additionally, we would like to point out that research trend analysis is a relatively novel focus in the field. Historically, the emphasis has been primarily on keyword extraction methodologies such as Topic Modeling, rather than on the generation of research trends. Therefore, we chose to discuss traditional topic modeling methods rather than specific case studies, which may not have the same direct relevance to our work.
That being said, we truly value your input and will consider revisiting Section 2, ensuring the content is as focused and relevant as possible to our study, without losing necessary context for the submodules of our framework.
- English in this study should be improved.
Response: We sincerely appreciate your feedback regarding the language used in our study. Your point is well taken, and we recognize the importance of clear and precise language in effectively communicating our research.
In response to your suggestion, we have thoroughly reviewed the entire manuscript and have made significant improvements to the quality of English. We have meticulously addressed issues related to grammatical accuracy, clarity of expression, and coherence of arguments.
Thank you once again for highlighting this issue. Your constructive feedback is crucial to us as we strive to enhance the readability and overall quality of our paper.
Author Response File: Author Response.pdf
Reviewer 5 Report
The authors effectively address a topic of great interest in their study. They propose a framework for analyzing research trends. The results obtained in the article have remarkable practical implications for research.
The authors employ a suitable methodology to investigate the subject matter, ensuring the validity and reliability of their findings.
Moreover, the article is very well founded, the authors extensively utilize relevant and up-to-date literature, demonstrating a comprehensive understanding of the current state of research.
Suggested Revisions
- In point 3, before the presentation of figure 1, the authors must briefly describe the figure 1.
- Repeatedly throughout the text appears the expression "Error! Reference source not found." For example: in lines 261, 347, 369, 450, 464, 493, 498, 548, 566, 570. Substitute by the corresponding figure or table.
- In figure 3, remove the figure and describe only the sequence.
- Following the title should be the table and not the text. The same in table 2. The same in figures 4 and 5. For example, put the following text “The best score is made bold from each metric. DBSCAN does not have any results since it fails to detect the number of clusters correctly” after table;
- In the section 5.1.2., just put “Table 2. Clustering evaluation results for the clustering algorithms used in TrendFlow framework with dimensionality reduction.” Put the text “The best score is made bold for each metric. DBSCAN still does not have any results due to its failure in detecting the correct number of clusters.”, after table 2;
- In the title of figure 4, just put “Figure 4. Training and validation results for KeyBART-adapter, with running average smoothing applied for better visualization.” Put in text “The best configuration (lr=7.3e-05, hidden = 32) is highlighted in bold blue, and we only display a subset of five configurations for clarity. These selected configurations have the same learning rate and only vary from hidden dimensions. The complete set of results is available on wandb.”
- In the title of Table 3, just put “Table 3. Keyphrase generation evaluation results for KeyBART-adapter and KeyBART.” Place below the table “Note: bigger values are made bold”.
- A final revision of the text is recommended.
Author Response
General comments:
The authors effectively address a topic of great interest in their study. They propose a framework for analyzing research trends. The results obtained in the article have remarkable practical implications for research.
The authors employ a suitable methodology to investigate the subject matter, ensuring the validity and reliability of their findings.
Moreover, the article is very well founded, the authors extensively utilize relevant and up-to-date literature, demonstrating a comprehensive understanding of the current state of research.
Suggested Revisions
We thank the reviewer for the positive evaluation. We have carefully modified the paper following your suggestions.
- In point 3, before the presentation of figure 1, the authors must briefly describe the figure 1.
Response: Thank you for your attention to detail and for suggesting a brief description of Figure 1 prior to its presentation. We understand the importance of a succinct explanation to help readers comprehend the visual representation more effectively.
We would like to kindly bring your attention to the last paragraph of Section 3.1, where we have provided a detailed description of the TrendFlow framework, including the three distinct modules: 1) Text Encoder, 2) Clustering, and 3) Keyphrase Generator. In this paragraph, we have explained how the Text Encoder transforms each literature abstract into a vector representation, how these vectors are then clustered based on their relative distances, and finally, how the Keyphrase Generator produces research trends for each cluster.
-----------------------------------------------------------------------------------------
TrendFlow utilizes three distinct modules to address the RTA task: 1) Text Encoder, 2) Clustering, and 3) Keyphrase Generator. The Text Encoder transforms each literature abstract into a vector representation. Subsequently, these vectors are clustered based on their relative distances. After the clusters have been established, the Keyphrase Generator produces research trends for each cluster. A comprehensive structure of the TrendFlow framework is available in Figure1.
-----------------------------------------------------------------------------------------
This detailed structure of the TrendFlow framework correlates directly with Figure 1. However, based on your feedback, we realize that we could make this connection clearer. We will, therefore, revise the manuscript to more explicitly link the description in the text to Figure 1.
- Repeatedly throughout the text appears the expression "Error! Reference source not found." For example: in lines 261, 347, 369, 450, 464, 493, 498, 548, 566, 570. Substitute by the corresponding figure or table.
Response: Thank you for pointing out this question. We have correct it in the manuscript.
- In figure 3, remove the figure and describe only the sequence.
Response: We appreciate your suggestion regarding Figure 3. However, we firmly believe in the adage that a picture is worth a thousand words. Visualizations often help to convey complex information in an easy-to-understand manner. Besides, the purpose to show a figure is to give a better visualization about how the labels in the dataset look like, which is very common in NLP papers. I would like to give you one example:
[1] Wang, Weizhi, et al. “Task-Oriented Dialogue System as Natural Language Generation.” Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2022, https://doi.org/10.1145/3477495.3531920.
In this example figure, there are some example representations of the input data.
In this specific case, we consider Figure 3 to be a valuable component of our paper as it succinctly depicts the sequence we're discussing, aiding in a more intuitive understanding for the reader. While we understand your point about describing the sequence in text, we believe that maintaining the figure in combination with the text description offers a richer comprehension experience.
Nevertheless, we will certainly revisit our text description to ensure it sufficiently elucidates the sequence even without direct reference to the figure.
Thank you once again for your feedback. Your suggestions are vital for us to improve our manuscript.
- Following the title should be the table and not the text. The same in table 2. The same in figures 4 and 5. For example, put the following text “The best score is made bold from each metric. DBSCAN does not have any results since it fails to detect the number of clusters correctly” after table;
Response: Thank you for pointing out this question. According to the reviewer’s comment, we have corrected it in the manuscript.
- In the section 5.1.2., just put “Table 2. Clustering evaluation results for the clustering algorithms used in TrendFlow framework with dimensionality reduction.” Put the text “The best score is made bold for each metric. DBSCAN still does not have any results due to its failure in detecting the correct number of clusters.”, after table 2;
Response: Thank you for pointing out this question. According to the reviewer’s comment, we have corrected it in the manuscript.
- In the title of figure 4, just put “Figure 4. Training and validation results for KeyBART-adapter, with running average smoothing applied for better visualization.” Put in text “The best configuration (lr=7.3e-05, hidden = 32) is highlighted in bold blue, and we only display a subset of five configurations for clarity. These selected configurations have the same learning rate and only vary from hidden dimensions. The complete set of results is available on wandb.”
Response: Thank you for pointing out this question. According to the reviewer’s comment, we have corrected it in the manuscript.
- In the title of Table 3, just put “Table 3. Keyphrase generation evaluation results for KeyBART-adapter and KeyBART.” Place below the table “Note: bigger values are made bold”.
Response: Thank you for pointing out this question. According to the reviewer’s comment, we have corrected it in the manuscript.
- A final revision of the text is recommended.
Response: We thank the reviewer for the positive evaluation. We have carefully modified the paper following your suggestions.
Special thanks to you for your kind comments.
Author Response File: Author Response.pdf
Round 2
Reviewer 3 Report
The revised paper is better than the first submission.
