AI Fairness in Data Management and Analytics: A Review on Challenges, Methodologies and Applications
Round 1
Reviewer 1 Report
- The article explores the complexities, methodologies, and practical applications of AI Fairness in the context of data management. While the paper is well-presented, certain revisions are warranted.
- As a review article, it should ideally offer insights into the review protocol itself, including details such as the selection process for the covered papers, the number of papers considered, and the timeframe within which the review was conducted. Incorporating these specifics would provide readers with a clearer understanding of the scope and methodology of the review.
- In the first line of the abstract, instead of simply writing artificial intelligence, this must be written in proper terminology: Artificial Intelligence (AI)
- While the introduction effectively outlines the concept of AI Fairness, it would be beneficial for the author to explicitly outline their scientific contribution in the article. This could be achieved by presenting key contributions in bullet points, ensuring that readers can readily grasp the significance of the article's findings.
- The section dedicated to problem formulation raises some confusion, as it appears unsuited for a review paper, which typically does not focus on problem formulation. This discrepancy should be addressed.
- On page 3, lines 3 to 116, several facts are presented that require proper referencing to establish their credibility and validity.
- Similarly, references should be provided to substantiate the content within Table 1, particularly concerning the definition of fairness.
- Figures 1, 2, and 3 are introduced in the paper, yet they are neither cited nor adequately explained within the text. This omission should be rectified to ensure the reader comprehends their relevance.
- Throughout the paper, theoretical concepts and philosophies are discussed; however, these discussions lack proper references to reputable sources. It is recommended that these discussions be supported by relevant citations to enhance the scholarly rigor of the paper.
Minor correction are needed.
Author Response
Please see the attachment.
Author Response File: 
Author Response.pdf
Reviewer 2 Report
First of all, thank you for contributing to the issue of fairness in data management and analytics. This topic is interesting for further discussion, especially in the upcoming digital transformation era. Although the paper provides interesting remarks, I have suggestions and questions for improving the manuscript. Namely, I have not noticed the figure referenced in the manuscript. For instance, what is the meaning of Figure 1, Figure 2, etc., if they are not referenced and elaborated in the body of the text?
Also, you've stated there are 5 main types of biases and more than double the amount stated. Not for this reason alone, but I suggest you construct a table of items consisting of each bias and reference accordingly, similar to what you have done in the paper (Table 3 of algorithmic bias) but more comprehensive. What I want to suggest is to try developing a "Systematic Literature Review" considering these types of references. After all, aren't you committing selection bias by pinpointing the references at face value? Try explaining why you selected these individual studies or rephrasing the narrative to make it more transparent and objective.
In the section "Fair Training", you should reduce the number of paragraphs and make it more cohesive. Please don't forget the full stops and punctuations at the end of the sentence (e.g., line number 290). The manuscript lacks data visualisation to represent arguments more understandably and interestingly. The text is exhaustive to read and should be reduced. Table 10 is not referenced in the body of the text. Row number 431 capital letter "Regularization-based...". Add a reference to line 485 "Individual fairness is a concept introduced by Dwork et al.". Also, in line 496, "Equal Opportunity Fairness, proposed by Hardt et al.", and every other paragraph later where you reference an author(s) without citation.
I completely do not understand sentence line number 543-544. Also, line numbers 545 to 548 it is not understandable. Social administration is not commented nor called for a table 13. The whole section of practical implications of bias and fairness should be restructured, redesigned and reduced to a subsection.
The conclusion states, "...detailing the background and definition of the concept" - I see the overview and contextual settings of the use of fairness in different backgrounds, but I do not see the details and discussion on the definition of the concept. Also, "The article delves into measures to reduce bias and improve fairness in AI systems, reviews relevant problems and solutions in the practical application of AI fairness, and discusses possible future research" - I do not see how this is done in the paper. Also, "The organization of the article ensures a comprehensive exploration..." - I could not disagree more with this sentence. The overall readability of the paper is hard to track since you are going from one perspective to another without actually evoking or suggesting your opinion on the matter. It is like you are expressing an overview and narratively describing the article without challenging or proposing new concepts, ideas or perspectives. Finally, I would strongly recommend reducing the length of the manuscript, reworking the methodological representation of thoughts and possibly using a "Systematic Literature Review" approach in delineating definitions and aspects of fairness and bias in reporting.
The English require major improvements.
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
Reviewer 3 Report
The article provides a thorough analysis of the key research directions in AI for addressing the fairness challenge.
Specify about the information sources and provide a summary of the limitations of the evidence included in the review, in the abstract.
Specify the inclusion and exclusion criteria of sources of information for the review and how studies were grouped for the syntheses.
Specify all databases, registers, websites, organisations, reference lists, and other sources searched or consulted to identify studies. Also, include the PRISMA flow diagram.
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
Round 2
Reviewer 2 Report
Dear authors, I have added several comments and most of them were addressed only partly. Namely, I would still suggest reworking of the article. Also, please add in the methodological part the proces of PRISMA and transparent description of the findings, considering the review process so it can be transparent and replicable. You have added a comment and figure in your response to me, which is unnecesary. I am asking that because the readers need to understand your process and align their comments and thoughts with your findings. Otherwise, the sole description of the findings does not seem to rationalise your outcomes. You've tackled a lot of the problems in a single paper, consider maybe adding visualisations and charts that can further reduce the length of the article. In my opinion this is "too much to be read in one single breath", if you understand the point that I am making. Anyway, I would suggest adding a PRISMA flowchart and trim the paper further on by replacing it with charts and visualisations. Either way, I would leave the decision to the EDITOR, since the paper does have soundness and is beneficial to the ongoing issues regarding the bias in ml, but study lacks specificity, clarity and cohesiveness.
Kind regards,
the reviewer.
Author Response
Please see the attachment.
Author Response File: 
Author Response.docx
Reviewer 3 Report
The author needs to include the suggested modifications in the manuscript too, along with the author response sheet. Here a few of the suggestions, I can see in the covering letter. But, I could'nt find it in the manuscript. The author can refer few recent review articles in MDPI, for understanding the concept of including methodology to a review article.
Author Response
Please see the attachment.
Author Response File: Author Response.docx
