Fuzzy Artificial Intelligence—Based Model Proposal to Forecast Student Performance and Retention Risk in Engineering Education: An Alternative for Handling with Small Data

Round 1

Reviewer 1 Report

1. The author needs to follow the writing style according to MDPI.

2. Some paragraphs are hard to understand

3. Equation 1 should be a theorem or lemma. Please rewrite

4. Dataset is not described clearly

5. Explain more detail the data pre-processing

5. I suggest you add another accuracy test like MAPE, RMSE, not only RSquare

6. Practical implications should be addressed.

Author Response

Comments of Reviewer #1: 1. The author needs to follow the writing style according to MDPI.

Response of the authors: The final version of the text will be formatted according to MDPI style and standards.

 

Comments of Reviewer #1: 2. Some paragraphs are hard to understand

Response of the authors: The text has been proofread.

 

Comments of Reviewer #1: 3. Equation 1 should be a theorem or lemma. Please rewrite.

Response of the authors: Equation 1 has been rewritten as a theorem.

 

Comments of Reviewer #1: 4. Dataset is not described clearly.

Response of the authors: The description of the data was detailed.

 

Comments of Reviewer #1: 5. Explain more detail the data pre-processing

Response of the authors: A figure showing the data analysis process workflow has been added.

 

Comments of Reviewer #1: 5. I suggest you add another accuracy test like MAPE, RMSE, not only RSquare

Response of the authors: The suggested metrics are not applicable to the developed model, as the response variable is categorical with ordinal linguistic values.

 

Comments of Reviewer #1: 6. Practical implications should be addressed.

Response of the authors: Practical implications have been added.

Author Response File: Author Response.pdf

Reviewer 2 Report

The paper is a good study, however extensive proofreading is required to fix some writing issues. 

 

Author Response

Comments of Reviewer #2: The paper is a good study, however extensive proofreading is required to fix some writing issues.

Response of the authors: The text has been proofread.

Author Response File: Author Response.pdf

Reviewer 3 Report

On the second page (58th row), where the authors write about psychological learning processes, this paper might be relevant: https://www.mdpi.com/2076-3417/11/4/1744

On the second page, 123rd row: there could be an additional sentence or footnote that explains what is the goal of the Mann-Whitney-Wilcoxon test, and when it is used. 

2nd page, 132nd row: similarly, there could be a sentence about the quadratic discriminant analysis. 

3rd page, Material and method section: there could be a figure illustrating the workflow of the paper. It would help a lot to understand the paper and the methodology. 

In the first half of the 'Results and discussions' section, when the p-values are detailed about the distribution of the variables in the pass and fail groups, it would be good if there was a table that shows the statistics of the failing and passing students: for example the median (or average) of the variables in both groups and the corresponding p values of the tests. So this table could give the readers a summary of which variables have different distributions in the two groups. 

 There could also be a histogram of the most significant variables (the variables that differ the most in the two groups). This histogram could be colored by the status (fail/pass), i.e., on the same figure for one selected variable you could show the histogram of the passing and failing students with different colors and some opacity.

 

I really liked that the variables were connected to the literature and that in general there was a thorough literature review. 

 

As far as I understand, Table 1 shows the results of many univariate logistic regressions, right? This could be clarified in the caption or the main text.

Before starting the analyses, it would be good to see some exploratory data analysis and visualizations. For example, how correlated are the variables? Are there highly correlated variables? And it would also be nice if the paper had a table listing all the used variables, and what type of values can these variables take. 

Consider visualizing the dataset and/or the decisions of the AI model (page 7, row 270-273) on a 'parallel coordinates plot' https://datavizcatalogue.com/methods/parallel_coordinates.html

About Table 2 and the models: Did you optimize the parameters  (hyperparameters) or you used the default settings? It seems like CCN, DTB, and PNN are overfitted. 

For me, it was not entirely clear what are the product minimum maximum conjunction operators. There could be more introduction to these concepts. 

 

Author Response

Comments of Reviewer #3: On the second page (58th row), where the authors write about psychological learning processes, this paper might be relevant: https://www.mdpi.com/2076-3417/11/4/1744

Response of the authors: The paper was addressed.

 

Comments of Reviewer #3: On the second page, 123rd row: there could be an additional sentence or footnote that explains what is the goal of the Mann-Whitney-Wilcoxon test, and when it is used.

Response of the authors: A footnote has been added.

 

Comments of Reviewer #3: 2nd page, 132nd row: similarly, there could be a sentence about the quadratic discriminant analysis.

Response of the authors: A footnote has been added.

 

Comments of Reviewer #3: 3rd page, Material and method section: there could be a figure illustrating the workflow of the paper. It would help a lot to understand the paper and the methodology.

Response of the authors: A figure has been added.

 

Comments of Reviewer #3: In the first half of the 'Results and discussions' section, when the p-values are detailed about the distribution of the variables in the pass and fail groups, it would be good if there was a table that shows the statistics of the failing and passing students: for example the median (or average) of the variables in both groups and the corresponding p values of the tests. So this table could give the readers a summary of which variables have different distributions in the two groups.

Response of the authors: A table has been added.

 

Comments of Reviewer #3: There could also be a histogram of the most significant variables (the variables that differ the most in the two groups). This histogram could be colored by the status (fail/pass), i.e., on the same figure for one selected variable you could show the histogram of the passing and failing students with different colors and some opacity.

Response of the authors: A figure has been added.

 

Comments of Reviewer #3: I really liked that the variables were connected to the literature and that in general there was a thorough literature review.

Response of the authors: The literature review has been expanded.

 

Comments of Reviewer #3: As far as I understand, Table 1 shows the results of many univariate logistic regressions, right? This could be clarified in the caption or the main text.

Response of the authors: The information has been added to the table title.

 

Comments of Reviewer #3: Before starting the analyses, it would be good to see some exploratory data analysis and visualizations. For example, how correlated are the variables? Are there highly correlated variables? And it would also be nice if the paper had a table listing all the used variables, and what type of values can these variables take.

Response of the authors: The information has been added to the tables 1 and 2.

 

Comments of Reviewer #3: Consider visualizing the dataset and/or the decisions of the AI model (page 7, row 270-273) on a 'parallel coordinates plot' https://datavizcatalogue.com/methods/parallel_coordinates.html.

Response of the authors: A figure has been added.

 

Comments of Reviewer #3: About Table 2 and the models: Did you optimize the parameters (hyperparameters) or you used the default settings? It seems like CCN, DTB, and PNN are overfitted.

Response of the authors: This clarification was added to the text.

 

Comments of Reviewer #3: For me, it was not entirely clear what are the product minimum maximum conjunction operators. There could be more introduction to these concepts.

Response of the authors: An introduction to these concepts has been added.

Author Response File: Author Response.pdf

Reviewer 4 Report

 1)         Introduction: The introduction section does not go into enough detail about the topic and needs to be thoroughly revised. By reading a compelling introduction, readers will understand the careful thought/consideration that went into why the proposed approach may provide more persuasive results. The authors should conduct a more in-depth critical review of the literature to highlight the inadequacies of current techniques. Furthermore, briefly describe the major contributions in bullet form, just before the organization paragraph.

 

 

2)         Most of the recent works are part of the introduction section only and not the related works. The authors are advised to discuss the recent research developments in this section. The literature must be strongly updated with some relevant and recent papers focused on the fields dealt with the manuscript. Please consider references from year 2019 to 2021. Like 10.32604/csse.2023.025796, 10.3390/s20113210, 10.32604/cmc.2022.020146

 

3)         Regarding the experiments, the paper lacks detailed information on how to implement the experiments. For example, what was the simulation tool (open source or implemented by oneself ...)? Lack of how the experiment was carried out. More details should be explained to support the findings.

 

4)         To make it easier to understand the experimental findings, a study flowchart or framework diagram for suggested algorithm is required.

 

Authors should introduce their proposed research model/framework more effective, i.e., some essential brief explanation vis-à-vis the text with a total research flowchart or framework diagram for each proposed algorithm to indicate how these employed models are working to receive the experimental results. It is difficult to understand how the proposed approaches are working. For the employed data set, please provide more details illustration.

 

 

5)         A comparison with the state of art in the form of a table should be added.  What makes the proposed method suitable for this unique task? What new development to the proposed method have the authors added (compared to the existing approaches)? These points should be clarified.

 

6)         I suggest that you add some more results. Some more simulation results and some comparison of the presented scheme with other schemes. May be some figures for the simulation results or the comparisons.

 

7)         To make this study more general and extended by other scholars. Authors should provide the limitations of current work.

 

8)         Conclusion needs to be improved. The most important obtained results should be briefly and clearly mentioned through the support of numerical data in the conclusion.

Author Response

Comments of Reviewer #4: Introduction: The introduction section does not go into enough detail about the topic and needs to be thoroughly revised. By reading a compelling introduction, readers will understand the careful thought/consideration that went into why the proposed approach may provide more persuasive results. The authors should conduct a more in-depth critical review of the literature to highlight the inadequacies of current techniques. Furthermore, briefly describe the major contributions in bullet form, just before the organization paragraph.

Response of the authors: A section of related works has been added.

 

Comments of Reviewer #4: Most of the recent works are part of the introduction section only and not the related works. The authors are advised to discuss the recent research developments in this section. The literature must be strongly updated with some relevant and recent papers focused on the fields dealt with the manuscript. Please consider references from year 2019 to 2021. Like 10.32604/csse.2023.025796, 10.3390/s20113210, 10.32604/cmc.2022.020146.

Response of the authors: A section of related works has been added.

 

Comments of Reviewer #4: Regarding the experiments, the paper lacks detailed information on how to implement the experiments. For example, what was the simulation tool (open source or implemented by oneself ...)? Lack of how the experiment was carried out. More details should be explained to support the findings.

Response of the authors: Details have been added.

 

Comments of Reviewer #4: To make it easier to understand the experimental findings, a study flowchart or framework diagram for suggested algorithm is required. Authors should introduce their proposed research model/framework more effective, i.e., some essential brief explanation vis-à-vis the text with a total research flowchart or framework diagram for each proposed algorithm to indicate how these employed models are working to receive the experimental results. It is difficult to understand how the proposed approaches are working. For the employed data set, please provide more details illustration.

Response of the authors: A figure showing the data analysis process workflow has been added.

 

Comments of Reviewer #4: A comparison with the state of art in the form of a table should be added.  What makes the proposed method suitable for this unique task? What new development to the proposed method have the authors added (compared to the existing approaches)? These points should be clarified.

Response of the authors: A section of related works has been added.

 

Comments of Reviewer #4: I suggest that you add some more results. Some more simulation results and some comparison of the presented scheme with other schemes. May be some figures for the simulation results or the comparisons.

Response of the authors: Some figures and more information have been added.

 

Comments of Reviewer #4: To make this study more general and extended by other scholars. Authors should provide the limitations of current work.

Response of the authors: The limitation has been added in the text

 

Comments of Reviewer #4: Conclusion needs to be improved. The most important obtained results should be briefly and clearly mentioned through the support of numerical data in the conclusion.

Response of the authors: The conclusion has been improved.

Author Response File: Author Response.pdf