Question Classification for Intelligent Question Answering: A Comprehensive Survey

Round 1

Reviewer 1 Report

The paper does not align with the scope of the journal. Furthermore, the presentation and quality of paper writing is not at international level. 

The paper does not align with the scope of the journal. Furthermore, the presentation and quality of paper writing is not at international level. 

Author Response

The paper does not align with the scope of the journal. Furthermore, the presentation and quality of paper writing is not at international level.

Response:

Thanks for your question. The goal of the presented research is to propose a new comprehensive IQA_QC framework. The main achievement is developing a hierarchy-based framework with three levels, covering different types of questions and evaluation metrics.

We have revised the paper from content and grammar.

On the one hand, we not only propose the hierarchy-based IQA_QC framework, but also illustrate this framework with geographical questions examples, which presents the unify of theory and practice. What’s more, we add several contents to illustrate that we focus on geographical questions to construct the IQA_QC framework. Please see Line16-19, 26-27, Line 235- 240 and Line 242-248 for the detail. Compared with the existing frameworks, our hierarchy-based classification covers more types of questions, which is reflected in Figure 5. Compared with common metrics, evaluation metrics of broader dimensions are proposed, which could measure geographical domain questions.

On the other hand, we checked the grammar to make paper read more clearly.

The details are shown in the clean version.

Author Response File: Author Response.pdf

Reviewer 2 Report

The goal of the presented research is to propose a new comprehensive question classification framework for intelligent question answering systems. The main achievement is developing a 3-level framework covering different types of questions and evaluation metrics.
The methodology seems sound overall. One issue is that some of the question type examples are a bit unclear without more context. Providing more details on the example questions and dataset could help.
Some general issues:
The introduction could be condensed to highlight the key gaps motivating the work.
More discussion on how the framework was developed systematically would strengthen the methodology section.
The conclusion could highlight more directions for future work.
The technical content seems solid but tightening up the writing/presentation could improve the quality for this high quality journal.

There are some minor grammatical issues:
Some article use before nouns (e.g. "an/the IQA system")
"return the final answer" sounds better as "determine the final answer"
There are some unclear sentences, e.g.:
"With this view on individual, we can reason that the areas with rainfall more than 300mm can apply Ecological Planting Pattern for development."
"For example, with a general premise that Hangzhou can apply Ecotourism Ecological Civilization Pattern for development, and we can lead to a specific statement that can apply Ecotourism Ecological Civilization Pattern for development based on the condition that Xihu District belongs to Hangzhou."
So carefully proofreading grammar/phrasing could help improve clarity.

Author Response

The introduction could be condensed to highlight the key gaps motivating the work.

Response 1:

Thanks for your question and suggestions. Our description of Section 1.1 (Motivations) in introduction is not clear and comprehensive in the previous version. On the one hand, compared with the existing frameworks, our hierarchy-based classification covers more types of questions, which is reflected in Figure 5. On the other hand, compared with common metrics, evaluation metrics of broader dimensions are proposed, which could measure geographical domain questions. The details are shown in Line 88-97.

 

More discussion on how the framework was developed systematically would strengthen the methodology section.

Response 2:

Thanks for your commons. Section 3 contains the most important content in our paper. In order to strengthen this section and make it develop systematically, we revised the basic idea section of our hierarchy-based classification, which consist of three levels. The details are shown in Line 235- 240 and Line 242-248.

 

The conclusion could highlight more directions for future work.

Response 3:

Thank you for your comments. The previous description related to future work was not detailed enough. Thus, we supplement the above content with three different points, including the improvement of IQA_QC framework, the promotion of the IQA system to deal with geographical fquestions and the optimization IQA evaluation metrics. We added more details point by point in conclusion to highlight the direction for future work in Line 616-628 in clean version.

 

The technical content seems solid but tightening up the writing/presentation could improve the quality for this high quality journal.

Response 4:

Thanks for your question and suggestions. We have improved the quality in two ways. On the one hand, we added some content to emphasize the importance of our work in introduction to make the paper more logical; we also added some content to make the structure of the paper clearer; we added future directions in conclusion to make the paper more complete. On the other hand, we checked the grammar of our paper such as articles and revised several unclear sentences to make the paper read more fluently.

 

There are some minor grammatical issues:

Some article use before nouns (e.g. "an/the IQA system")

"return the final answer" sounds better as "determine the final answer"

There are some unclear sentences, e.g.:

"With this view on individual, we can reason that the areas with rainfall more than 300mm can apply Ecological Planting Pattern for development."

"For example, with a general premise that Hangzhou can apply Ecotourism Ecological Civilization Pattern for development, and we can lead to a specific statement that can apply Ecotourism Ecological Civilization Pattern for development based on the condition that Xihu District belongs to Hangzhou."

So carefully proofreading grammar/phrasing could help improve clarity.

Response 5:

Thanks for your suggestion. We have checked and corrected the grammatical issues. For example, we have added article before "IQA system" in whole paper; we have replaced "return" with "determine" to make paper read more clearly; we have revised several unclear sentences in 414-417 and Line 426-432 in the clean version.

 

Comments: More discussion on how the framework was developed systematically would strengthen the methodology section.

Response 3:

Thanks for your commons. Section 3 contains the most importance content in our paper. In order to strengthen this section and make it develop systematically, we added short introduction of the whole methodology section to make the structure clearer, including the basic idea of constructing the framework and the overall classification in Line 237- 243.

 

Comments: The conclusion could highlight more directions for future work.

Response 3:

Thank you for your comments. The previous description related to future work was not detailed enough. Thus, we supplement the above content with three different points, including the improvement of IQA_QC framework, better guidance of IQA performance and the optimization IQA evaluation metrics. We added more details point by point in conclusion to highlight the direction for future work in Line 617-628 in clean version.

 

Comments: The technical content seems solid but tightening up the writing/presentation could improve the quality for this high quality journal.

Response 4:

Thanks for your question and suggestions. We have improved the quality in two ways. On the one hand, we added some content to emphasize the importance of our work in introduction to make the paper more logical; we also added some content to make the structure of the paper clearer; we added future directions in conclusion to make the paper more complete. On the other hand, we checked the grammar of our paper such as articles and revised several unclear sentences to make the paper read more fluently.

 

Comments: There are some minor grammatical issues:

Some article use before nouns (e.g. "an/the IQA system")

"return the final answer" sounds better as "determine the final answer"

There are some unclear sentences, e.g.:

"With this view on individual, we can reason that the areas with rainfall more than 300mm can apply Ecological Planting Pattern for development."

"For example, with a general premise that Hangzhou can apply Ecotourism Ecological Civilization Pattern for development, and we can lead to a specific statement that can apply Ecotourism Ecological Civilization Pattern for development based on the condition that Xihu District belongs to Hangzhou."

So carefully proofreading grammar/phrasing could help improve clarity.

Response 5:

Thanks for your suggestion. We have checked and corrected the grammatical issues. For example, we have added article before "IQA system" in whole paper; we have replaced "return" with "determine" to make paper read more clearly; we have revised several unclear sentences in 416-418 and Line 428-434 in the clean version.

Author Response File: Author Response.pdf

Reviewer 3 Report

Comments for author File: Comments.pdf

Author Response

Why suddenly the topic of motivations appears is given an introduction first.

Why suddenly the topic of organization appears is given an introduction first.

Response 1:

Thanks for your question. Before the motivations and organization sections, we have added shortly content to briefly introduce the two sections in Line 78-80.

 

What are the Question Classification indicators for Intelligent Question Answering, so that they are explained first in the background

Response 2:

Thanks for your question. We reviewed the existing four classification framework, content-based, template-based, calculation-based and method-based classification, and the corresponding literature (3,9-16) is given. On this basis, we have summarized the indicators (hierarchy-based classification, which consists of three levels) to our IQA_QC framework, which indicate that our framework is more comprehensive than the above existing frameworks. The details are shown in Line 242-248.       

 

The relationships between variables are also discussed in detail

Response 3:

Thanks for your commons. We have added more details to describe the comparison of our IQA_QC framework and the existing IQA_QC frameworks. That is, there is a deficiency in the existing IQA_QC frameworks which indicates that they are unable to well guide the IQA system. We supplemented the above content to describe the relationship between variables concretely and adjusted the article structure of section 5.1 in Line 536-560.

 

where is section 1?

Response 4:

Thanks for your question. I’m sorry that the caption to Figure 5 is not clear. Figure 5 shows the comparison of IQA_QC framework proposed in this paper and the existing framework mentioned in Section 2. I have revised the caption in Line 562-563.

Author Response File: Author Response.pdf

Reviewer 4 Report

1. How were the formulas validated before inclusion in the study?

2. How can the limitations identified be addressed in future research?

Author Response

How were the formulas validated before inclusion in the study?

Response 1:

Thanks for your question. We listed the formulas of general metrics to evaluate the performance of the IQA system in Section 4. The formulas refer to the literature 5,15,38,39,40 which are common evaluation metrics in the IQA system.

 

How can the limitations identified be addressed in future research?

Response 2:

Thanks for your question. The previous statement is not enough. Therefore, we added the content to supplement how we address the limitations in future research in Line 628-633. On the one hand, we need to refine the IQA_QC framework to more comprehensive coverage of questions and well guide the IQA system. On the other hand, we need to evaluate the IQA system from a broader dimension instead of focusing on the general metrics.

 

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

Accepted