Challenges, Techniques, and Trends of Simple Knowledge Graph Question Answering: A Survey
Round 1
Reviewer 1 Report
I find the article as an interesting read, a review of 14 different KGQA systems tested using the SimpleQuestions dataset containing only simple questions. This study included articles published between 1 January 2010 and 1 January 2020, but the authors chose 14 articles from 2015 - 2019 considered by them relevant to simple KGQA. Here I would like the authors to better emphasize why did they choose only the last period of study (5 years). Are there are no articles for the field studied between 2010 and 2015? What about 2020, given that we are in the middle of year 2021.
Even though the SimpleQuestions consists of more than 100000 questions, there could be a problem with this dataset due to the fact that answer triples are given by Freebase, which unfortunately has been closed since 2016 (meaning that the infrastructure for querying, browsing etc. is no longer there). I would like to know the authors' opinion on this problem, how it may affect their study (and perhaps their future studies), and also if they could consider another database (apart from the ones they mention, WebQuestions, LC-QuAD which address complex questions in addition to simple questions).
I appreciate the working method as interesting, the three steps solution (search, inclusion and exclusive criteria, and excluding step). I also appreciate section 3.1 Terminology, which resembles a dictionary of terms used in the paper, and section 4.1. which reviews the Existing Techniques used in the field studied ( but I highlighted some of the problems for 4.1. below).
The paper seems like a good synthesis studio, but I would like to see more of the authors' contribution, apart from section 4.3 Recommendation, which includes only one page, in which other 6 articles are also cited and the authors' recommendations are very little presented.
The figure on page 7, Classification of articles by challenge (2015-2019), misses its caption (Figure 2…), according to the template of this journal.
The equations/formulas should be properly numbered (1), (2) etc. according to the template of this journal.
I would recommend to the authors a further reading by a native English speaker, or at least a very careful reading, because there are some errors to be corrected, for example, in Conclusions the word "aThe" (aThe re-use of a pre-training NLP model ... ”)
Based on the Ithenticate software (see the attached iThenticate_review.pdf file), some pieces of the theoretical parts appear to have been copy-pasted (without proper quotation marks or modification) from the cited references. I would suggest the authors (so as not to have problems in their future career) to try to reproduce using their own words these elements of theory, and where it is not possible to put them in quotes. There are paragraphs that I strongly suggest should be at least slightly modified or put in quotation marks so to avoid them to appear so obvious in iThenticate or other plagiarism software: page 10 section 4.1.3, page 11 section 4.1.4, page 12 section 4.15, section 4.1.6, page 13 section 4.1.6, 4.1.7, page 14-15 section 4.1.8, 4.1.9 . Please be so kind to see the attached iThenticate_review.pdf file, I am trying to be of assistance.
I appreciate the article as an interesting working study, with a few things to improve for publication in this journal.
Comments for author File: 
Comments.pdf
Author Response
Dear reviewer,
Here, we enclose the response to the reviewer's comment.
Best regards,
Mohammad Yani
Author Response File: Author Response.pdf
Reviewer 2 Report
This paper present a comprehensive survey of answering simple questions to classify available techniques and compare their advantages and drawbacks, in order to have better insights of existing issues and recommendations to direct the future works.
Fig 1 is blurry, I would recommend to redraw the Figure.
Why only the 2010 to 2020 published research paper used? Why not before 2010?
What databases used to search for the papers?
In Table 2, advnatages and disavantages can be added. I am not sure, is it useful to add year?
Author Response
Dear reviewer,
Here, we enclose the response to the reviewer's comment.
Best regards,
Mohammad Yani
Author Response File: Author Response.pdf
Reviewer 3 Report
The authors present a review of simple knowledge graph question answering. There are several significant issues with the manuscript, of which I will highlight the two that are most severe:
--First, the manuscript seems to largely cover the same territory as
Höffner, K., Walter, S., Marx, E., Usbeck, R., Lehmann, J., & Ngonga Ngomo, A. C. (2017). Survey on challenges of question answering in the semantic web. Semantic Web, 8(6), 895-920.
This work was also published only in the 3-4 years, which is recent for a survey, if there have not been many significant advances in that time. While the authors of that review do not call it 'knowledge graph' QA they are, in essence, referring to the same thing. Many of the challenges they point out are similar to those of the authors. Therefore, the authors are not novel enough with respect to this work. Perhaps I am mistaken, but if so, the authors need to explicitly cite this work (which they have not) and describe why their work is significantly different from this one. For a survey, I don't think the only contribution can be that they are covering additional material in the last 3-4 years.
--A second concern is that the manuscript is more of a description, than a synthesis. It lacks structure and needs to be much better organized. Simply introducing a motley set of techniques and recent papers is not adequate for a survey. Some material belongs clearly in appendices rather than in the main text (such as the long description of question-answering benchmark datasets, or introductions to sub-topics such as BERT and knowledge graph embeddings like TransE and TransR)
Author Response
Dear reviewer,
Here, we enclose the response to the reviewer's comment.
Best regards,
Mohammad Yani
Author Response File: Author Response.pdf
Round 2
Reviewer 3 Report
I believe the authors have significantly re-organized the manuscript and the article could be accepted at this point.
