Category Mapping of Emergency Supplies Classification Standard Based on BERT-TextCNN
, Huanhuan Wang 1 and Junhe Chen 1Round 1
Reviewer 1 Report
Comments and Suggestions for AuthorsThis manuscript proposes the application of a BERT Pre-Trained language model with Text CNN to map two separate supply categorization standards, highlighting their major contributions: (i) a category description dataset for the two standards; (ii) the application of a fine-tuned BERT model for the global semantic representations; and (iii) Applying the best BERT-TextCNN model to the inference dataset.
The references are updated with more than 60% from the last five years (2019 to 2024), and the topics in section 2 of the literature review cover the theoretical background for the research well, in my opinion.
The methodology presented in section 3 is comprehensive about the whole process and objective, avoiding overextending what is being described. The entire process is described adequately enough for an article, and from what I have seen, I have not detected any points of attention that need to be improved.
The results were also presented in a very representative way, using the necessary elements between tables and figures with graphs.
Below I point out some elements to be improved in the manuscript:
1. Please improve Figure 5, Part B, about GPC. I recommend placing the figures in sequence and stacking them to ensure more room to enlarge them.
2. Please translate what is being represented in the two parts of Figure 6, to ensure that readers who do not speak the authors' native language have access to the information.
3. Regarding the analysis of the experiment results, in the case of a classification problem, if possible, present the confusion matrix related to the proposed model (this is enough since it is the focus). If possible, I also recommend presenting the ROC-AUC analysis with the classes considered in the model to ensure an understanding of the correctness of the classifications.
4. The article does not have a section dedicated to the theoretical and practical implications of the study. I recommend creating a dedicated section, building on and expanding on what was already presented in the introduction, where the authors show the main contributions.
a. The three contributions listed can be expanded to clarify the study's contributions concerning the pre-existing knowledge on ways to solve the classification problem of emergency supply.
b. From the point of view of the proposed model's practical use, the authors failed to demonstrate their ideas on applications, how governments or private companies that may be involved can use what was developed, and the model's scalability for practical purposes.
5. In the final comments, within the conclusion, the authors state that the model is not without flaws, stating:
"Firstly, it is not very interpretable and can only make predictions, not inferences. Second, when adjusting the model, it is difficult to adjust specific features based on the training results because there is no concept of feature importance in the TextCNN layer."
This is a final comment on the limitations and challenges of the proposed model, which is relevant for other interested researchers to understand what they will face if they choose to use it. I strongly suggest that the authors develop this part of the text a little more, leaving it in a dedicated subsection right after the theoretical and practical implications of the study. It is interesting to highlight the challenges and problems faced by the authors throughout the study until reaching the current state of what they are presenting to ensure transparency of the process, but also considering that perhaps some related components cannot be strictly described for reasons of intellectual property protection.
Author Response
Please see the attachment.
Author Response File: 
Author Response.pdf
Reviewer 2 Report
Comments and Suggestions for AuthorsThe paper introduces an approach called "Mapping of Emergency Supplies Classification Standard Based on BERT-TextCNN." It proposes training the model using an existing text similarity corpus and applying it to other supplies classification standard category matching tasks through transfer learning. This approach aims to overcome the labor-intensive and time-consuming process of manually annotating the category-matching corpus. This contribution addresses the problem of communication and alignment between suppliers and supply taxonomies.
One aspect that requires improvement is the availability of the dataset and source code. For reproducibility purposes, the authors should make the dataset and source code easily accessible to others who may want to repeat the experiments,
Additionally, Figure 5 in the paper is too small, making the details difficult to discern. The authors should consider enlarging the figure to improve its readability and understanding.
Overall, the proposed approach and contribution are valuable, but it is essential to address the availability of the dataset and source code for reproducibility.
Comments on the Quality of English LanguageMinor editing of English is required.
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
Reviewer 3 Report
Comments and Suggestions for AuthorsThis paper presents a hybrid method that combines the BERT and CNN models for emergency supplies classification. The problem addressed by the authors is interesting and relevant. The authors conducted experiments using real-world annotated data (GB/T 38565 and GPC) and compared their proposed method with other similar models. The proposed model achieved an impressive accuracy of 98.58%.
The paper is well written and interesting. This paper focuses on a real-world and innovative application with challenges - a strong point of the work. Nevertheless, there are some aspects that should be improved before reaching the decision:
1) While the paper effectively presents the proposed BERT-TextCNN method, it falls short in providing a comprehensive experimental methodology and results analysis. Section 4 appears to amalgamate the methodology, results, and analysis. To enhance the clarity and depth of the manuscript, I recommend creating distinct sections dedicated to 'Methodology', 'Results', and 'Discussion'. The Methodology section should detail all the experimental steps, the Results section should present the outcomes, and the Discussion section should analyze these results in depth.
2) Figure 5 is too small and does not display well. Please correct this issue.
3) The paper constructs a dataset. I recommend that the authors provide a public link to the dataset to facilitate further research by others.
4) Regarding the results in Table 8, the authors present the performance of the models and the hybrid method only on the training dataset. Why did they not show the results for the testing dataset? I believe the objective evaluation should focus on the results based on the testing dataset rather than the training dataset. Please provide a strong justification for this approach.
5) While the paper discusses some limitations but they should provide more details for these.
6) The authors could also suggest some future approaches that might mitigate the limitations identified in this study.
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
Round 2
Reviewer 1 Report
Comments and Suggestions for AuthorsThe authors have met all the improvement demands I raised in the previous round.
Regarding the analysis using ROC-AUC, they provided the necessary clarifications in the response letter, determining that it would be necessary to extract the predicted probabilities, which would require more time to configure and run the associated experiments. However, the authors left a comment about the application of ROC-AUC in the second paragraph of the Future Work section.
I consider that the current state, with the adjustments made, is satisfactory.
I wish the authors great continuation of their research.
Reviewer 3 Report
Comments and Suggestions for AuthorsThank you to the authors for their responses and the revised version of the paper. Most of my concerns have been adequately addressed.
