Exploring the Utility of Dutch Question Answering Datasets for Human Resource Contact Centres

Round 1

Reviewer 1 Report

This paper created a Dutch HR QA dataset with 300 questions in the format of the Squad 2.0 dataset. I think it is valuable to build such a dataset for the community of natural language processing or other related research area. My questions are as follows:

It’s better to compare with other datasets related to the proposed QA dataset. For example, the motivation for building such a dataset, the compared experimental results for algorithms on the compared dataset and the proposed dataset. 

Author Response

Dear reviewer,

Thank you for your time and your solid review. In this letter, your feedback will be checked point by point.

Language feedback
1. English language and style are fine/minor spell check required 
 - We improved the language and removed the language spelling mistakes

General feedback:
1. Does the introduction provide sufficient background and include all relevant references? Can be improved
 - We improved the language in the introduction
2. Are all the cited references relevant to the research? Can be improved
 - The references are extended with a few more citations to Bert and some surveys.
3. Is the research design appropriate? Can be improved
 - We added a figure about the different experiments in our method section to give a clear view of which datasets and models are used.
4. Are the methods adequately described? Can be improved
 - We noticed that the BERT language model part was important in our method section. We added an extra paragraph about the reasons to use RobBERT.
5. Are the results clearly presented? Can be improved
 - We improved the language in the results section
6. Are the conclusions supported by the results? Can be improved
 - We improved the language in the conclusions section

Comments:
1. It’s better to compare with other datasets related to the proposed QA dataset. For example, the motivation for building such a dataset, the compared experimental results for algorithms on the compared dataset and the proposed dataset. 
 - This paper compares the proposed QA dataset with Squad 2.0. In the results section, at 4.1, we addressed the differences between a general translated question and answering dataset and the P-Direkt QA dataset.

Once again, thank you for your feedback; in our view we improved our paper.

With kind regards,

Chaïm van Toledo

Reviewer 2 Report

minor comments:

* proof-read the document, language needs improvement

* the labels on the figures are not clearly readable

* the references are duplicated 

Author Response

Dear reviewer,

Thank you for your time and your solid review. In this lettre your feedback will be checked point by point.

Language feedback
1. English language and style are fine/minor spell check required 
 - We improved the language and removed the language spelling mistakes

General feedback
1. Does the introduction provide sufficient background and include all relevant references? Can be improved
 - We improved the language in the introduction
2. Are all the cited references relevant to the research? Must be improved
 - The references are extended with a few more cites to Bert and some surveys. Double references are removed.
3. Is the research design appropriate? Can be improved
 - We added a figure about the different experiments in our method section to give a clear view about which datasets and models is used.
4. Are the methods adequately described? Can be improved
 - We noticed that the BERT language model part was important in our method section. We added an extra paragraph about the reasons to use RobBERT.
5. Are the results clearly presented? Can be improved
 - We improved the language in the results section
6. Are the conclusions supported by the results? Can be improved
 - We improved the language in the conclusions section

Comments:
1. proof-read the document, language needs improvement
 - We improved the language in our paper. We changed the spelling mistakes. Also we changed some American English to British English (for example tokenization-> tokenisation).
2. the labels on the figures are not clearly readable
 - Indeed, some labels on the figures were difficult to read. We increased the font and we used to whole width for these figures
4. the references are duplicated 
 - References are checked on duplicates. There are no duplicates in the newer version. Although some papers have some similarity, for example Rajpurkar 2018 and Rajpurkar 2016.

Once again, thank you for your feedback, in our view we improved our paper.

With kind regards,

Chaïm van Toledo

Reviewer 3 Report

There is currently no question and answer dataset in Dutch. In this paper,  the authors have collected a small dataset in the HR domain. The entity linking methods are used on the emails of the in a CC at the Dutch government, and then a Dutch QA dataset for personnel recruitment  is constructed.  From this point of view, the article is of significance, but the paper has a few issues that need to be addressed.

1.The method of this paper can be organized in a more orderly manner. It is recommended to provide the overall pseudo-code of the given methods to increase the credibility of the methods.

2. In page 3, the author should give more details about how to find a link between answers and documentation using the entity linking method.

3. The experimental part is slightly simple. The author mainly uses cross-language Bert for verification and fine-tuning, and does not highlight the advantages of the method.

Author Response

Dear reviewer,

Thank you for your time and your solid review. In this lettre your feedback will be checked point by point.

Language feedback
1. English language and style are fine/minor spell check required 
 - We improved the language and removed the language spelling mistakes

General feedback
1. Does the introduction provide sufficient background and include all relevant references? Yes
2. Are all the cited references relevant to the research? Must be improved
 - The references are extended with a few more cites to Bert and some surveys. Double references are removed.
3. Is the research design appropriate? Yes
 - We added a figure about the different experiments in our method section to give a clear view about which datasets and models is used.
4. Are the methods adequately described? Can be improved
 - We noticed that the BERT language model part was important in our method section. We added an extra paragraph about the reasons to use RobBERT.
5. Are the results clearly presented? Can be improved
 - We improved the language in the results section
6. Are the conclusions supported by the results? Yes

Comments:
1. The method of this paper can be organized in a more orderly manner. It is recommended to provide the overall pseudo-code of the given methods to increase the credibility of the methods.
 - We provide now a figure to organise our experiments to order the methods also visually
2. In page 3, the author should give more details about how to find a link between answers and documentation using the entity linking method.
 - We extended figure 1 and we provide a better step by step description.
3. The experimental part is slightly simple. The author mainly uses cross-language Bert for verification and fine-tuning, and does not highlight the advantages of the method.
 - We provide now extra explanation about the BERT language models. 

Once again, thank you for your feedback, in our view we improved our paper.

With kind regards,

Chaïm van Toledo

Round 2

Reviewer 1 Report

I have no questions.

Reviewer 3 Report

The authors addressed all my issues in an appropriate manner. I would like to accept the paper in current form.