Methodology of Labeling According to 9 Criteria of DSM-5
Round 1
Reviewer 1 Report
The research goal is interesting, there is a methodology applied as well. My main concern has to do with the results. There is a lack of robustness, or these are not clearly presented.
It is a must to translate all Korean text in the figures to English, otherwise is not possible to make an interpretation or appropriate understanding from the figures. Are the authors expecting that every reader makes a translation?
The example refereed in line 124 is not understandable in Figure 1, first because it is written in Korean. Second because is not described with details in the text, and besides nothing is said in its caption. What to interpret from Fig 1?
What this example referred to in line 136 of Word2Vec consists of is not described in the text. And Figure 2 is not clearly appreciated what the graph consists of, it seems unrelated to the referring paragraph.
It is highly recommended that all the figures that have the Korean language be translated into English or, if applicable, present both Korean and English. No examples can be understood otherwise. The English language is the standard for scientific publications. In particular, Figure 7 make up the dictionary of words for the diagnosis of each of the nine criteria, which it is necessary to understand. This is to facilitate the understanding of data labeling.
It is also necessary to provide other characteristics of the data dictionary, for example, how many words are made up of each of the diagnostic criteria. All the details for a clear and precise understanding must be given.
Figures 3, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15 have contents in Korean, it is asked to translate into English.
Names of the axes of most of the graphs are missing, it is necessary to name the axes.
Line 254 refers to section 5.4 which does not exist in the manuscript.
It is very important that the values used for the parameters of each of the applications or tools used be described in a table. Lines 253 – 250 try to describe some parameter values, however, all of them must be clearly and formally provided to the reader along with the explanation of why those values were chosen. This is required for the reproducibility principle of science.
Lines 260-261 say “Figure 14 shows the results of searching for 260 words similar to “lonely,” “terrible,” and other terms. However, what do you refer for “other terms” since Figure 14 is related only to “lonely” and “terrible”?
Figures 14 and 15 are not legible and therefore not understandable.
Describe in detail the process of word augmentation. This is an important process in your data analysis. Even a diagram would be more helpful.
Regarding Section 7 Results. To talk about a significant difference, it is necessary to carry out a statistical analysis, observing only the graph does not allow us to express significant differences. No statistical analysis was found in your manuscript. How do you justify your claimed “significant difference”? Your conclusions should clearly derive from your experimental studies.
Finally, double-check the format of your references, those referring to a website are incorrect. Need to fix them all.
Author Response
Please see the attachment
Author Response File: 
Author Response.docx
Reviewer 2 Report
It is an interesting work for an important and sensitive topic. However, the methods used by the authors are of moderate novelty. Please be careful on the following issues:
- The journal is addressed to an English-speaking audience. Tables and figures in Korean cannot be read by the majority of the reader, so either they should be translated or removed.
- In p.1, l.28-31 the source of these stats is not provided.
- There is a typo on p.8, l.253
- Figures 9, 12, 14, and 15 are of low quality and difficult to read.
- Figure 2 needs further explanation. What are the units of the parenthesis stand for?
- Why you did not use the four additional criteria of DSM-5
- The fifth criterion should be rephrased.
- The first criterion is practically the disorder itself. It should be rephrased to "depressed mood".
- In section 4.2 please provide further justification for deleting the sentences contending “or” and “/”.
- How did you evaluate your results? There is no information provided about the effectiveness of the proposed process
Some minor improvements should be made.
Author Response
Please see the attachment.
Author Response File: Author Response.docx
Reviewer 3 Report
Manuscript ID: applsci-2533722
Title: Labeling and Classification Techniques of Empathy Datasets 2 According to DSM-5
This manuscript proposed a methodology for classifying and labeling datasets based on DSM-5 diagnostic criteria for depressive disorder. It is quite unclear from the ‘Abstract’ and ‘7. Result’ section that what are the outcome of this research?
Following are comments for the major revision:
1. Authors should explain the specific application or some potential application of the work. Also, compare with previous research to justify their better results.
2. Authors should follow the journal guidelines for creating high resolution figures and use standard English for labeling and nomenclature. If local language/ script is required, then provide the appropriate translation as well.
3. Improve the ‘1. Introduction’ section and cite more previous research works, discuss previous research outcomes, their limitations, and the research gap.
4. Manuscript fails to convey the research outcome, its usability and significance.
Authors are advised to consider the major revision issues and improve accordingly.
Comments for author File: 
Comments.pdf
Please follow journal guidelines for standard English.
Author Response
Please see the attachment.
Author Response File: Author Response.docx
Reviewer 4 Report
Thank you for the opportunity to review your paper. It is very interesting.
The aim and objectives of the research are clear and a good justification of the work is made. The development of the dataset has a lot of potential to be used by other researchers. the contribution to the field is clear and novel. Some minor suggestions to improve. these are mainly grammar issues and presentation concerns.
The paper may be strengthened with a little more explanation of what models were used and how they are used. More detail in the methodology. Specifically the Hidden Markov model.
Could you clarify if Figure 5 and Figure 6 are translations of each other? I think it is important if you are publishing in English then all non-English text in the paper should have its translation in the paper as well (and make it clear where the translation is). I believe this will make the paper more accessible. Please check all figures with non-English text and include translations for each in the paper.
Figures 12, 13 14,15, 16, 17, and 18 are very low-quality and difficult to read. Please make sure all text in all the figures is readable. The y and x-axis text appears to be low resolution.
Just to clarify: "Are the results clearly presented?" I set this to must be improved as I believe some of the figures need to be better quality and higher resolution. Also making it clear the translation of non-English language in the paper will help improve this and the overall presentation.
Thank you again for allowing me to review the paper. I wish the team all the best.
The quality of the English language is good. But Please double-check the grammar for errors. For example line 53 "AI Hub Hub".
Author Response
Please see the attachment.
Author Response File: Author Response.docx
Round 2
Reviewer 1 Report
This is my second round of revision. I find your manuscript improved, and still the following observations are pending.
Figs. 4, 5 are not legible, font too small
Some figs mixed Korean and English, and others like Fig 11 have only English, why is this?
The setup of the t-test referred to in line 299 must be described, not only mentioned. It only says “t-test using data before and after the augmentation…”
Describe in detail the process of word augmentation. This is an important process in your data analysis, so even a diagram would be more helpful. Word2Vec is only described, the reader needs to find in your manuscript how it interacts with your input data to obtain the desired output data.
Author Response
Please see the attachment.
Author Response File: Author Response.docx
Reviewer 2 Report
Authors made all the necessary modifications. I believe that their work can be published at the current form.
Author Response
Thank you for your careful review.
Reviewer 3 Report
Manuscript ID: applsci-2533722
Title: Labeling and Classification Techniques of Empathy Datasets 2 According to DSM-5
I have been through the revised manuscript, and I could not find any significant improvement and changes made against the previous version comments.
Following are my comments for this revised manuscript:
1. The major fault/ limitation of this study: original dataset used was in English, then English dataset translated into Korean, processed in Korean, and the results translated back into English. In my opinion, it will highly compromise the accuracy of results!
In the defense of this, authors would have given accuracy of their results in percentage/ numbers/ figure, so that one can easily understand what exactly slightly mean on Page 12, Line 326–“……accuracy of the analysis may have been slightly reduced.”
2. Authors failed to clearly present the results in the ‘Abstract’ and in the ‘7. Result’ section. Figure 16~18 are not very helpful to understand the results, all 3 figures have ‘Number of data’ and ‘Label’ axis labeling and no other information in figures or text. It is cumbersome to understand and draw any meaningful information from these figures.
3. Figure 7 (DSM-5 diagnosis criteria for depressive disorder), in the previous version of manuscript it has 9 criteria, and now in the revised version it is 8?
4. Most of the figures in this manuscript, e.g., Figure 10, can be listed as a table for better understanding and clarity. It will also solve the issue of poor-quality figures.
5. Still lacks in ‘1. Introduction’ section, not enough previous research has been cited and did not established background before describing own method.
Comments for author File: Comments.pdf
Author Response
Please see the attachment.
Author Response File: Author Response.docx
Round 3
Reviewer 3 Report
Manuscript ID: applsci-2533722
Title: Labeling and Classification Techniques of Empathy Datasets 2 According to DSM-5
I have been through the third revision of the manuscript, and it has been significantly improved and satisfactory changes have been made against the previous version comments. Authors are suggested to further improve the figure quality, such as in ‘Figure 11. Tag table of KKMA.’ Also, authors should carefully check the manuscript for any typo, Korean-to-English translation mismatch/ error, and journal format guidelines related issues.
Comments for author File: Comments.pdf
Authors should carefully check the manuscript for any spelling typo, Korean-to-English translation mismatch/ error.
