Explanation of Student Attendance AI Prediction with the Isabelle Infrastructure Framework^†

Round 1

Reviewer 1 Report

 

The paper has as a goal a system that not only predicts the absence of students in classrooms but also provides explanations to humans that are acceptable and transparent to a large degree. The researchers concentrate on presenting the formulation of the problem, and the deficiencies of current AI methods (with or without xAI) and propose their method as one that fulfils the aforementioned goals of explanation, transparency and proactive acting to support the presence of students. The paper does explain theoretically a scenario where this methodology could be beneficial, but other than that does not make any comparisons with actual state-of-the-art AI methods and their xAI counterparts to support the claim that the proposed strategy is beneficial.

 

The paper is original in the sense that logical rules on this topic are not easy to find in the literature. Nonetheless, although there are novel aspects, those are based on a constructed synthetic example without any concrete reference to performance or generalization to other scenarios and use cases that are valid for the problem of the absence of students in classrooms. It is as if the researchers picked up one situation where their suggested method is beneficial from an explanation and actionability point of view, but other than that there is no prospect of other situations and cases. What is characteristic is that there is no clear future work mentioned.

 

A work that would help the readers understand xAI taxonomy (page 5) is the following:

- Schwalbe, G., & Finzel, B. (2023). A comprehensive taxonomy for explainable artificial intelligence: a systematic survey of surveys on methods and concepts. Data Mining and Knowledge Discovery, 1-59.

https://doi.org/10.1007/s10618-022-00867-8

AI methods are also used for student modelling, like in the following paper:

- Saranti, A., Taraghi, B., Ebner, M., & Holzinger, A. (2019). Insights into learning competence through probabilistic graphical models. In Machine Learning and Knowledge Extraction: Third IFIP TC 5, TC 12, WG 8.4, WG 8.9, WG 12.9 International Cross-Domain Conference, CD-MAKE 2019, Canterbury, UK, August 26–29, 2019, Proceedings 3 (pp. 250-271). Springer International Publishing.

https://doi.org/10.1007/978-3-030-29726-8_16

 

It is not clear if the contents of section 2.1. which present valid previous research are validated or not by the new methodology. The privacy aspect is mentioned on page 2 but is not addressed afterwards. It is not clear how far human domain knowledge is helping the definition of the structure of the rules; are all domain experts agreeing in their definition? On page 5 there is a big fallacy: AI is not comparable to xAI. Those are two distinct parts; sometimes xAI methods are applied after the AI model is constructed and trained and at other times its explainability is already present by construction. It is quite unusual to present on 3.1.1. details about attack trees without having an introduction about them. The reviewers assume that the researchers are really into this topic but the readers may not. What is the connection between Kripke structures and those logical rules to Prolog rules and Inductive Logic Programming (ILP)? A kind of related work that also acknowledges the necessity of textual explanations in the form of rules for better human comprehensibility, is the following:

- Finzel, B., Saranti, A., Angerschmid, A., Tafler, D., Pfeifer, B., & Holzinger, A. (2022). Generating explanations for conceptual validation of graph neural networks. KI-Künstliche Intelligenz, 1-15.

doi: 10.1007/s13218-022-00781-7

The goal of the researchers Is to remodel the data so that one has a better explainability when one models the phenomenon. If so, how good is the prediction performance? After all, one does not just want explainability but also good predictions. How do the counterfactuals differ from the ones used in state-of-the-art AI models? It is mentioned that there are more alternatives in the counterfactual set; who defines them? Can they be automatically defined?

 

First of all, one expects that for a paper that is based on promoting transparency for humans, a UI study to evaluate the proposed solution would be a part of it. Since the researchers mention the importance of causality and counterfactual explanations, maybe they should consider measuring causability as well:

- Holzinger, A., Langs, G., Denk, H., Zatloukal, K., & Müller, H. (2019). Causability and explainability of artificial intelligence in medicine. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 9(4), e1312.

https://doi.org/10.1002/widm.1312

Secondly, the paper mentions bias and its uncovering; how does this work do it? The overall evaluation of the paper from the reviewers is that this is a preliminary work of trying to “test the waters” of a methodology, but it is not a fully grown research yet – unless some aforementioned criteria are met. 

 

The paper is well-written and has a clear structure in general. The topic switch from lines 49 to 50 is not adequate. There are no major typos found. Some expressions like „intruding“ in line 67 feel like imposing something. The reviewers suggest rather to use expressions like „inform the parents earlier“, and „discuss matters with the student earlier“. The three „!!!“ on page 3 are probably a typo.

Minor editing of English language required

Author Response

Dear Editor,

Thank you for the swift processing of our submission. Please also pass our gratitude to the
reviewers that ahve acted so swiftly and have provided very constructive and helpful reviews
in a very short time.
Please find attached below the original two reviewers comments. We have inlined our responses
and answers to these comments into the review text. To simplify the identification of our answers
we have started each line with '>' when ut is an answer.

Kind regards,

Florian Kammuller also on behalf od Dimpy Satija

RESPONSES TO REVIEWERS' COMMENTS AND SUGGESTIONS

**********
Reviewer 1:
**********

The paper has as a goal a system that not only predicts the absence of students in classrooms
but also provides explanations to humans that are acceptable and transparent to a large degree.
The researchers concentrate on presenting the formulation of the problem, and the deficiencies
of current AI methods (with or without xAI) and propose their method as one that fulfils the
aforementioned goals of explanation, transparency and proactive acting to support the presence
of students. The paper does explain theoretically a scenario where this methodology could be beneficial,
but other than that does not make any comparisons with actual state-of-the-art AI methods and their
xAI counterparts to support the claim that the proposed strategy is beneficial.
> We have now added a comparison to other xAI methods (see response below) where we also highlight some of
> the other differences to related work follwing up on the constructive suggestions by this reviewer.
 
The paper is original in the sense that logical rules on this topic are not easy to find in the literature.
Nonetheless, although there are novel aspects, those are based on a constructed synthetic example without
any concrete reference to performance or generalization to other scenarios and use cases that are valid for
the problem of the absence of students in classrooms. It is as if the researchers picked up one situation
where their suggested method is beneficial from an explanation and actionability point of view, but other
than that there is no prospect of other situations and cases. What is characteristic is that there is no
clear future work mentioned.
> We have added now a Future Work section at the end of the conclusions. There are a number of constructive
> suggestions raised by this reviewer that we discuss there (see also below comments and our responses).

A work that would help the readers understand xAI taxonomy (page 5) is the following:
- Schwalbe, G., & Finzel, B. (2023). A comprehensive taxonomy for explainable artificial intelligence: a systematic survey of surveys on methods and concepts. Data Mining and Knowledge Discovery, 1-59.
https://doi.org/10.1007/s10618-022-00867-8
> We have summarised this paper and contrasted it to our approach in Section 3.1.

AI methods are also used for student modelling, like in the following paper:
- Saranti, A., Taraghi, B., Ebner, M., & Holzinger, A. (2019). Insights into learning competence through probabilistic graphical models. In Machine Learning and Knowledge Extraction: Third IFIP TC 5, TC 12, WG 8.4, WG 8.9, WG 12.9 International Cross-Domain Conference, CD-MAKE 2019, Canterbury, UK, August 26–29, 2019, Proceedings 3 (pp. 250-271). Springer International Publishing.
https://doi.org/10.1007/978-3-030-29726-8_16
> We have summarised and contrasted this interesting additional reference to our approach in Section 2.1. 

It is not clear if the contents of section 2.1. which present valid previous research are validated or not
by the new methodology. The privacy aspect is mentioned on page 2 but is not addressed afterwards. It is
not clear how far human domain knowledge is helping the definition of the structure of the rules; are all
domain experts agreeing in their definition? On page 5 there is a big fallacy: AI is not comparable to xAI.
Those are two distinct parts; sometimes xAI methods are applied after the AI model is constructed and trained
and at other times its explainability is already present by construction. 
> Dear Reviewer, thanks for pointing out this but our aim is to say that addition of explanation makes AI
> systems stronger and acceptable as XAI is human centric.

It is quite unusual to present on 3.1.1. details about attack trees without having an introduction about them.
The reviewers assume that the researchers are really into this topic but the readers may not.
> Indeed, we have been assuming to much here. Thanks for pointing out this shortcoming.
> There is a short introduction to attack trees and Kripke structures now in Section 3.1.1 where they
> are first mentioned. In addition, we give an intuitive description of Kripke structure in the new
> related work Section 2.2 when we contrast to ILP (see also the following response to the ILP proposal).

What is the connection between Kripke structures and those logical rules to Prolog rules and
Inductive Logic Programming (ILP)?
> Thank you to the reviewer: this is a very good point and a good suggestion to make our approach easier
> understandable for a wider readership. In Section 3.2.1 where Kripke structures are introduced in our paper
> we now added a sentence that shows up the similarities to Prolog rules and ILP. In the new related work
> section 2.2 we added a paragraph summarizing ILP, showing up the similarities with out PCR algorithm but
> also contrasting it to the mathematical notion of induction underlying our framework to provide a better
> intuition for readers.

A kind of related work that also acknowledges the necessity of textual explanations in the form of rules for
better human comprehensibility, is the following:
- Finzel, B., Saranti, A., Angerschmid, A., Tafler, D., Pfeifer, B., & Holzinger, A. (2022). Generating explanations for conceptual validation of graph neural networks. KI-Künstliche Intelligenz, 1-15.
doi: 10.1007/s13218-022-00781-7
> The mentioned work is now summarized in the new related work Section 2.2 and compared to the rule based
> approach in our paper.

The goal of the researchers Is to remodel the data so that one has a better explainability when one models
the phenomenon. If so, how good is the prediction performance? After all, one does not just want explainability
but also good predictions.
> Our approach remodels not so much the data but rather the decision procedure by re-engineering the
> black box function. In so far, the prediction performance doesn't differ between the original
> black box function and the one we use in our logical model. This is, however, a good question and in
> the discussion Section 5.3 we use it to highlight some of these key aspects of our approach.

How do the counterfactuals differ from the ones used in state-of-the-art AI models? It is mentioned that
there are more alternatives in the counterfactual set; who defines them? Can they be automatically defined?
> This is a very good question. Thank you to this reviewer for raising it here. We answer it briefly
> here in this review response but also have added a paragraph after the introduction of counterfactuals
> at the end of the introduction to Section 5.1 just before the PCR algorithm in Section 5.1.1.
> Our counterfactuals are essentially the same as the counterfactuals as used in any AI model, also
> state-of-the-art ones. The difference is only that our definition is very generic in the sense that
> it doesn't use a metric specific to any modeling world in order to describe the "closedness" of worlds.
> The definition of counterfactuals in our framework (Definition 5.1) uses the "distance of possible
> worlds" given by the transitions between states (the relation "closest"). This our "abstract metric"
> can be in concrete applications be identified with more specific metrics. In our case study of
> addendance data, a possible metric would be "distance of locations".

It is mentioned that there are more alternatives in the counterfactual set; who defines them?
Can they be automatically defined?
> Indeed, it is a good question how the definition of the counterfactual set works practically. The
> advantage of the Isabelle approach is that the definition as given in the paper (and in the Isabelle
> sources) is sufficient to provide all counterfactuals for a Desirable Outcome DO as a set.
> It is however an open question what is the best way to implement this abstract definition into
> an automated procedure. This is a refinement of the given definition but can be done within Isabelle
> and code can be generated but this is future work.
> Thanks to this constructive suggestion, we have picked up this point and added it in the future work
> section at the end of the conclusions.

First of all, one expects that for a paper that is based on promoting transparency for humans, a UI study
to evaluate the proposed solution would be a part of it.
> This is a good point but it is beyond the timeframe of this revision to launch a UI study to evaluate
> the proposed solution. However, we have picked up this valuable suggestion and added a paragraph
> to the conclusions in the "Future Work" Section that clarifies our methodology as "proof of concept"
> and adds the suggested UI study as an important next step for future work to fully explore and validate
> our methodology.

Since the researchers mention the importance of causality and counterfactual explanations, maybe they should consider measuring causability as well:
- Holzinger, A., Langs, G., Denk, H., Zatloukal, K., & Müller, H. (2019). Causability and explainability of artificial intelligence in medicine. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 9(4), e1312.
https://doi.org/10.1002/widm.1312
> Thank you again for this additional suggestion that is indeed very relevant and interesting. It is very useful
> to make the terminology and methododology more precise. We have 
> added a separate paragraph recalling the concept of causability and and explanation as coined by Holzinger et al
> at the end of the related work Section 2.3 and discussed and related its relevance to our methodology.

Secondly, the paper mentions bias and its uncovering; how does this work do it?
> Thank you for raising this point.
> The uncovering of bias is a side-effect of explanation in general. A bias may be built
> into the black box classification function by data-workers who manually value decisions on the
> training data set for the black box function. This bias is revealed by the explanation. 
> To clarify, we have elaborated this point in the discussion Section 5.3.

The overall evaluation of the paper from the reviewers is that this is a preliminary work of trying to “test the waters” of a methodology, but it is not a fully grown research yet – unless some aforementioned criteria are met.
> We hope that the clarifications we have added -- following the very constructive questions and
> suggestions of this reviewer -- have contributed to clarify the benefits of our work.
> We agree with this reviewer that this work is a proof of concept for a new methodology. Clearly, to become a
> "fully grown research" it is absolutely necessary to conduct some empirical evaluation of the methodology
> as well on a larger number of datasets as well as performing the UI study that has been suggested by this
> reviewer. While we cannot provide those works now in the context of this submission revision, we added a
> corresponding summary of the valuable insights of this reviewer to the conclusions as avenues for future
> research in a separate Section also picking up there answers to earlier comments of this reviewer.

The paper is well-written and has a clear structure in general. 
> Thank you to this reviewer for appreciating the writing and structure of the paper.

The topic switch from lines 49 to 50 is not adequate.
> We resolved this by restructuring the introduction section.

There are no major typos found. Some expressions like „intruding“ in line 67 feel like imposing something. The reviewers suggest rather to use expressions like „inform the parents earlier“, and „discuss matters with the student earlier“.
The three „!!!“ on page 3 are probably a typo.
> Thanks for the valuable minot correction whoch we could all take on board.

 

Reviewer 2 Report

This paper presents a methodology on the application example of pupil attendance for constructing explanations for AI classification  algorithms. The methodology includes building a model of the application in the Isabelle Insider and Infrastructure framework (IIIf) and an algorithm (PCR) that helps to obtain a detailed logical rule to  specify the performance of the black box algorithm, hence allowing to explain it.

In this paper, an existing method is applied it to the example of pupil’s attendance in schools to illustrate and validate the proposed methodology.

The topic of this paper is understandable from the abstract. The paper contains sufficient new material to merit publication since predicting students’ attendance is a topic that has not been investigated significantly before. The introduction is understandable by their own and contain the relevant facts related to the problem of predicting students’ attendance.

 This paper should be accepted subject to the following conditions:

-          In the end of Introduction, presentation of paper by sections has to be added.

-          The contribution of the proposed methodology should be discussed in the paper.

Author Response

**********
REVIEWER 2
**********

> Thank you to this reviewer for his positive response and the constructive comments that helped to improve
> the paper.

This paper presents a methodology on the application example of pupil attendance for constructing explanations for AI classification  algorithms. The methodology includes building a model of the application in the Isabelle Insider and Infrastructure framework (IIIf) and an algorithm (PCR) that helps to obtain a detailed logical rule to  specify the performance of the black box algorithm, hence allowing to explain it.

In this paper, an existing method is applied it to the example of pupil’s attendance in schools to illustrate and validate the proposed methodology.

The topic of this paper is understandable from the abstract. The paper contains sufficient new material to merit publication since predicting students’ attendance is a topic that has not been investigated significantly before. The introduction is understandable by their own and contain the relevant facts related to the problem of predicting students’ attendance.

 This paper should be accepted subject to the following conditions:

-   In the end of Introduction, presentation of paper by sections has to be added. DONE

> We have added a new Section 1.2 "Overview of the Paper" that gives a presentation of the paper by sections

-          The contribution of the proposed methodology should be discussed in the paper.

> Similarly, we have added a new Section 1.1. "Contribution" that briefly summarizes the main contribution
> pointing out how it relates to the earlier publication.

> A big Thank you again to the two reviewers for these concrete improvement hints that are now all done.
> As a final remark, we have highlighted all additional or changed sections in the text in red.

***
END
***

Round 2

Reviewer 1 Report

All the mentioned issues from the reviewers were addressed.