Iterative Learning for K-Approval Votes in Crowdsourcing Systems

Round 1

Reviewer 1 Report

In this paper, we propose a novel and efficient iterative algorithm to infer correct answers for a K-approval voting, which can be directly applied to real world crowdsourcing systems. We analyze the performance of our algorithm, and prove the theoretical error bound of our scheme that decays exponentially in terms of the quality of workers and the number of queries.

I would recommend to highlight the main contribution of the paper.

In addition, the paper should be compared with state-of-the-art approaches such as: 

Yin, X., Wang, H., Wang, W. and Zhu, K., 2020. Task recommendation in crowdsourcing systems: A bibliometric analysis. Technology in Society, 63, p.101337.

Zhou, C., Tham, C.K. and Motani, M., 2020. Online auction for scheduling concurrent delay tolerant tasks in crowdsourcing systems. Computer Networks, 169, p.107045.

The related work section should be updated with most recent rsearch paper such as:

Dashtipour, K., Gogate, M., Li, J., Jiang, F., Kong, B. and Hussain, A., 2020. A hybrid Persian sentiment analysis framework: Integrating dependency grammar based rules and deep neural networks. Neurocomputing, 380, pp.1-10.

Author Response

Thanks to your valuable feedback

 

Please check the attached file for the detailed response.

Author Response File: Author Response.docx

Reviewer 2 Report

Authors present an iterative algorithm based on K-approval voting in order to collect high quality data from crowdsourcing systems. The paper it is almost well structured and contains adequate references. My overall impression is that the paper needs some minor improvements. As authors mentioned their algorithm needs only a dozen of iterations, it will be very interesting a comparative study between their algorithm and at least a similar one.

Author Response

Thanks to your valuable feedback

Author Response File: Author Response.docx