FedQAS: Privacy-Aware Machine Reading Comprehension with Federated Learning

Round 1

Reviewer 1 Report

I reviewed the paper entitled “FedQAS: Privacy-aware Machine Reading Comprehension with Federated Learning”. In this paper, the authors present an idea of a privacy-preserving machine reading system that can leverage large-scale private data without the need to pool datasets in a central location. The method proposed by the authors combines the transformer model with federated learning methods.

The paper is well written and sufficient information is provided about the proposed method. However, I have some comments and suggestions which will further improve the paper. These comments are;

1. Please remove typo errors and grammatical errors. It would be better to check the paper with some professional English services. For example, see line 163, word For
2. Abbreviations should be mentioned first, for example, NLP in the abstract. Please make corrections.
3. Very few details are available regarding the proposed deep learning-based framework. Please add some more details so it may be justified that your proposed method is addressing the problem in a better way.
4. What is Nbr in Table 2? It represents numbers.
5. Regarding the computational cost of the proposed research work; can you please add some details. Also, add details of the machine you used for experiments
6. The authors use 20% data for validation purposes. Do not you think some standard procedure will be suitable. Such as 5-fold or 10-fold cross-validation experiments.
7. Data regarding the experiment is made available by the authors which is a plus point. It makes the readers easy accessibility to the data. I appreciate this.

Author Response

We appreciate the detailed,s valuable comments of the referees on our manuscript (applsci-1591393) entitled: "FedQAS: Privacy-aware Machine Reading Comprehension with Federated Learning". Larger edits include adding additional references to the relevant literature and restructuring some parts of the paper to further improve the quality. As suggested, the changes are highlighted in blue color. Below is a point-by-point response to the reviewer comments As suggested, the changes are highlighted in blue color.

Please see the attachment

Author Response File: Author Response.pdf

Reviewer 2 Report

In the manuscript, the authors proposed FedQAS, a privacy-preserving machine reading comprehension system, based on federated learning. It seems the proposed idea possesses reasonable contributions to be considered for a publication in Applied Science. However, the following comments should be fully addressed before the final decision:

• I think, it’s better to combine Related section with Introduction section. The current logic of these both sections is not smooth.
• The authors are recommended to provide more details for the proposed method section. The current manuscript lacks technical details.
• The accuracy performance of the proposed method is compared with only a baseline method, which is not enough. Include more methods or algorithms in the performance benchmarking.
• Section 4.1 contains lots of common and unnecessary knowledges such as accuracy, precision, recall and f1 score. Those descriptions can easily be found from the literature.

Author Response

We appreciate the detailed,s valuable comments of the referees on our manuscript (applsci-1591393) entitled: "FedQAS: Privacy-aware Machine Reading Comprehension with Federated Learning". Larger edits include adding additional references to the relevant literature and restructuring some parts of the paper to further improve the quality. As suggested, the changes are highlighted in blue color. Below is a point-by-point response to the reviewer comments As suggested, the changes are highlighted in blue color.

Please see the attachment

Author Response File: Author Response.pdf

Reviewer 3 Report

This manuscript is dedicated to the development of a framework suited for privacy-preserving machine reading system. The presented approach is shown to be effective and reliable in handling large-scale private data. Authors provided a thorough investigation and the subject can be a matter of interest for the community. Authors need to consider the followings in their revision.

1- Sections 1 and 2 lack deep evaluation of recently developed frameworks, elaborating advantages and disadvantages of the existing approaches. The need for extending existing MRC models requires strong justification in these sections.

2- The examples provided by the authors are well elaborated. However, I did not find any justifications regarding the choice of the dataset in the experiment. It will be promising if the authors consider adding explanations in this regard.

3- Conclusion section needs more discussions about the potential future research. It will be helpful if the authors share their thoughts regarding the expansion of the proposed framework and other possible enhancements for future works.

Author Response

We appreciate the detailed,s valuable comments of the referees on our manuscript (applsci-1591393) entitled: "FedQAS: Privacy-aware Machine Reading Comprehension with Federated Learning". Larger edits include adding additional references to the relevant literature and restructuring some parts of the paper to further improve the quality. As suggested, the changes are highlighted in blue color. Below is a point-by-point response to the reviewer comments As suggested, the changes are highlighted in blue color.

Please see the attachment

Author Response File: Author Response.pdf

Round 2

Reviewer 2 Report

 Though I am not fully satisfied with the authors' revision, the current manuscript contains reasonable contributions.