Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
5.8
3.1
Journals
Information
Special Issues
Foundations and Challenges of Interpretable ML
share announcement

Foundations and Challenges of Interpretable ML

A special issue of Information (ISSN 2078-2489). This special issue belongs to the section "Artificial Intelligence".

Deadline for manuscript submissions: closed (28 February 2023) | Viewed by 22664

Special Issue Editors

Prof. Dr. Isabel Valera

E-Mail Website
Guest Editor
Department of Computer Science, Saarland University, 66123 Saarbrücken, Germany
Interests: fair and interpretable ML; probabilistics ML
Dr. Melanie F. Pradier

E-Mail Website
Guest Editor
Biological Computation Lab, Microsoft Research, Cambridge CB12FB, UK
Interests: probabilistic models; interpretable ML; healthcare applications

Special Issue Information

Dear Colleagues,

Interpretability, the ability to present information in understandable terms to humans, has received an increased interest from both academia and industry; it has also been broadly discussed by policy makers, e.g., in the EU General Data Protection Regulation (“GDPR”). This special issue focuses on interpretable machine learning (also sometimes called Explainable AI or XAI), a set of approaches which aim at providing data-driven models with increased understanding and transparency.

We welcome works on definitions, mechanisms, as well as measures of interpretability (e.g., comparative user studies covering any group of stakeholders). Survey, meta-analysis, and critical review papers that contrast and highlight domains of applicability and limitations thereof are particularly welcome. More specifically, topics of interest include but are not limited to:

  • Definition and study of interpretability
  • Human-in-the-loop approaches
  • Cognitive analysis of human learning
  • Exemplar-based reasoning
  • Logic-based and symbolic reasoning approaches
  • Causality-based approaches
  • Novel applications requiring interpretability
  • Interpretation of black-box models
  • Interpretable design via suitable proxies, i.e., modularity, sparsity
  • Interpretable supervised models
  • Interpretable unsupervised models
  • Interpretable reinforcement learning
  • Inspection and debugging approaches for ML systems
  • Transparent models for auditability

We sincerely thank Amir Karimi for his great assist in editing this special issue.

Prof. Dr. Isabel Valera
Dr. Melanie F. Pradier
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Information is an international peer-reviewed open access monthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 1600 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Published Papers (7 papers)

Download All Papers
Order results
Result details
Show export options expand_more Show export options expand_less
Select all
Export citation of selected articles as:

Research

Jump to: Review

24 pages, 14413 KiB  
Article
On the Soundness of XAI in Prognostics and Health Management (PHM)
by David Solís-Martín, Juan Galán-Páez and Joaquín Borrego-Díaz
Information 2023, 14(5), 256; https://doi.org/10.3390/info14050256 - 24 Apr 2023
Cited by 4 | Viewed by 1680
Abstract
The aim of predictive maintenance, within the field of prognostics and health management (PHM), is to identify and anticipate potential issues in the equipment before these become serious. The main challenge to be addressed is to assess the amount of time a piece [...] Read more.
The aim of predictive maintenance, within the field of prognostics and health management (PHM), is to identify and anticipate potential issues in the equipment before these become serious. The main challenge to be addressed is to assess the amount of time a piece of equipment will function effectively before it fails, which is known as remaining useful life (RUL). Deep learning (DL) models, such as Deep Convolutional Neural Networks (DCNN) and Long Short-Term Memory (LSTM) networks, have been widely adopted to address the task, with great success. However, it is well known that these kinds of black box models are opaque decision systems, and it may be hard to explain their outputs to stakeholders (experts in the industrial equipment). Due to the large number of parameters that determine the behavior of these complex models, understanding the reasoning behind the predictions is challenging. This paper presents a critical and comparative revision on a number of explainable AI (XAI) methods applied on time series regression models for PM. The aim is to explore XAI methods within time series regression, which have been less studied than those for time series classification. This study addresses three distinct RUL problems using three different datasets, each with its own unique context: gearbox, fast-charging batteries, and turbofan engine. Five XAI methods were reviewed and compared based on a set of nine metrics that quantify desirable properties for any XAI method. One of the metrics introduced in this study is a novel metric. The results show that Grad-CAM is the most robust method, and that the best layer is not the bottom one, as is commonly seen within the context of image processing. Full article
(This article belongs to the Special Issue Foundations and Challenges of Interpretable ML)
►▼ Show Figures

Figure 1

28 pages, 4234 KiB  
Article
Intrinsically Interpretable Gaussian Mixture Model
by Nourah Alangari, Mohamed El Bachir Menai, Hassan Mathkour and Ibrahim Almosallam
Information 2023, 14(3), 164; https://doi.org/10.3390/info14030164 - 3 Mar 2023
Cited by 1 | Viewed by 2103
Abstract
Understanding the reasoning behind a predictive model’s decision is an important and longstanding problem driven by ethical and legal considerations. Most recent research has focused on the interpretability of supervised models, whereas unsupervised learning has received less attention. However, the majority of the [...] Read more.
Understanding the reasoning behind a predictive model’s decision is an important and longstanding problem driven by ethical and legal considerations. Most recent research has focused on the interpretability of supervised models, whereas unsupervised learning has received less attention. However, the majority of the focus was on interpreting the whole model in a manner that undermined accuracy or model assumptions, while local interpretation received much less attention. Therefore, we propose an intrinsic interpretation for the Gaussian mixture model that provides both global insight and local interpretations. We employed the Bhattacharyya coefficient to measure the overlap and divergence across clusters to provide a global interpretation in terms of the differences and similarities between the clusters. By analyzing the GMM exponent with the Garthwaite–Kock corr-max transformation, the local interpretation is provided in terms of the relative contribution of each feature to the overall distance. Experimental results obtained on three datasets show that the proposed interpretation method outperforms the post hoc model-agnostic LIME in determining the feature contribution to the cluster assignment. Full article
(This article belongs to the Special Issue Foundations and Challenges of Interpretable ML)
►▼ Show Figures

Figure 1

22 pages, 2601 KiB  
Article
Transferring CNN Features Maps to Ensembles of Explainable Neural Networks
by Guido Bologna
Information 2023, 14(2), 89; https://doi.org/10.3390/info14020089 - 2 Feb 2023
Viewed by 1797
Abstract
The explainability of connectionist models is nowadays an ongoing research issue. Before the advent of deep learning, propositional rules were generated from Multi Layer Perceptrons (MLPs) to explain how they classify data. This type of explanation technique is much less prevalent with ensembles [...] Read more.
The explainability of connectionist models is nowadays an ongoing research issue. Before the advent of deep learning, propositional rules were generated from Multi Layer Perceptrons (MLPs) to explain how they classify data. This type of explanation technique is much less prevalent with ensembles of MLPs and deep models, such as Convolutional Neural Networks (CNNs). Our main contribution is the transfer of CNN feature maps to ensembles of DIMLP networks, which are translatable into propositional rules. We carried out three series of experiments; in the first, we applied DIMLP ensembles to a Covid dataset related to diagnosis from symptoms to show that the generated propositional rules provided intuitive explanations of DIMLP classifications. Then, our purpose was to compare rule extraction from DIMLP ensembles to other techniques using cross-validation. On four classification problems with over 10,000 samples, the rules we extracted provided the highest average predictive accuracy and fidelity. Finally, for the melanoma diagnostic problem, the average predictive accuracy of CNNs was 84.5% and the average fidelity of the top-level generated rules was 95.5%. The propositional rules generated from the CNNs were mapped at the input layer by squares in which the relevant data for the classifications resided. These squares represented regions of attention determining the final classification, with the rules providing logical reasoning. Full article
(This article belongs to the Special Issue Foundations and Challenges of Interpretable ML)
►▼ Show Figures

Figure 1

19 pages, 7816 KiB  
Article
Exploiting Distance-Based Structures in Data Using an Explainable AI for Stock Picking
by Michael C. Thrun
Information 2022, 13(2), 51; https://doi.org/10.3390/info13020051 - 21 Jan 2022
Cited by 5 | Viewed by 3090
Abstract
In principle, the fundamental data of companies may be used to select stocks with a high probability of either increasing or decreasing price. Many of the commonly known rules or used explanations for such a stock-picking process are too vague to be applied [...] Read more.
In principle, the fundamental data of companies may be used to select stocks with a high probability of either increasing or decreasing price. Many of the commonly known rules or used explanations for such a stock-picking process are too vague to be applied in concrete cases, and at the same time, it is challenging to analyze high-dimensional data with a low number of cases in order to derive data-driven and usable explanations. This work proposes an explainable AI (XAI) approach on the quarterly available fundamental data of companies traded on the German stock market. In the XAI, distance-based structures in data (DSD) that guide decision tree induction are identified. The leaves of the appropriately selected decision tree contain subsets of stocks and provide viable explanations that can be rated by a human. The prediction of the future price trends of specific stocks is made possible using the explanations and a rating. In each quarter, stock picking by DSD-XAI is based on understanding the explanations and has a higher success rate than arbitrary stock picking, a hybrid AI system, and a recent unsupervised decision tree called eUD3.5. Full article
(This article belongs to the Special Issue Foundations and Challenges of Interpretable ML)
►▼ Show Figures

Figure 1

17 pages, 5025 KiB  
Article
Beyond Importance Scores: Interpreting Tabular ML by Visualizing Feature Semantics
by Amirata Ghorbani, Dina Berenbaum, Maor Ivgi, Yuval Dafna and James Y. Zou
Information 2022, 13(1), 15; https://doi.org/10.3390/info13010015 - 30 Dec 2021
Cited by 2 | Viewed by 2772
Abstract
Interpretability is becoming an active research topic as machine learning (ML) models are more widely used to make critical decisions. Tabular data are one of the most commonly used modes of data in diverse applications such as healthcare and finance. Much of the [...] Read more.
Interpretability is becoming an active research topic as machine learning (ML) models are more widely used to make critical decisions. Tabular data are one of the most commonly used modes of data in diverse applications such as healthcare and finance. Much of the existing interpretability methods used for tabular data only report feature-importance scores—either locally (per example) or globally (per model)—but they do not provide interpretation or visualization of how the features interact. We address this limitation by introducing Feature Vectors, a new global interpretability method designed for tabular datasets. In addition to providing feature-importance, Feature Vectors discovers the inherent semantic relationship among features via an intuitive feature visualization technique. Our systematic experiments demonstrate the empirical utility of this new method by applying it to several real-world datasets. We further provide an easy-to-use Python package for Feature Vectors. Full article
(This article belongs to the Special Issue Foundations and Challenges of Interpretable ML)
►▼ Show Figures

Figure 1

26 pages, 4590 KiB  
Article
Understanding Collections of Related Datasets Using Dependent MMD Coresets
by Sinead A. Williamson and Jette Henderson
Information 2021, 12(10), 392; https://doi.org/10.3390/info12100392 - 23 Sep 2021
Cited by 2 | Viewed by 1917
Abstract
Understanding how two datasets differ can help us determine whether one dataset under-represents certain sub-populations, and provides insights into how well models will generalize across datasets. Representative points selected by a maximum mean discrepancy (MMD) coreset can provide interpretable summaries of a single [...] Read more.
Understanding how two datasets differ can help us determine whether one dataset under-represents certain sub-populations, and provides insights into how well models will generalize across datasets. Representative points selected by a maximum mean discrepancy (MMD) coreset can provide interpretable summaries of a single dataset, but are not easily compared across datasets. In this paper, we introduce dependent MMD coresets, a data summarization method for collections of datasets that facilitates comparison of distributions. We show that dependent MMD coresets are useful for understanding multiple related datasets and understanding model generalization between such datasets. Full article
(This article belongs to the Special Issue Foundations and Challenges of Interpretable ML)
►▼ Show Figures

Figure 1

Review

Jump to: Research

25 pages, 1964 KiB  
Review
A Review on Federated Learning and Machine Learning Approaches: Categorization, Application Areas, and Blockchain Technology
by Roseline Oluwaseun Ogundokun, Sanjay Misra, Rytis Maskeliunas and Robertas Damasevicius
Information 2022, 13(5), 263; https://doi.org/10.3390/info13050263 - 23 May 2022
Cited by 23 | Viewed by 5642
Abstract
Federated learning (FL) is a scheme in which several consumers work collectively to unravel machine learning (ML) problems, with a dominant collector synchronizing the procedure. This decision correspondingly enables the training data to be distributed, guaranteeing that the individual device’s data are secluded. [...] Read more.
Federated learning (FL) is a scheme in which several consumers work collectively to unravel machine learning (ML) problems, with a dominant collector synchronizing the procedure. This decision correspondingly enables the training data to be distributed, guaranteeing that the individual device’s data are secluded. The paper systematically reviewed the available literature using the Preferred Reporting Items for Systematic Review and Meta-analysis (PRISMA) guiding principle. The study presents a systematic review of appliable ML approaches for FL, reviews the categorization of FL, discusses the FL application areas, presents the relationship between FL and Blockchain Technology (BT), and discusses some existing literature that has used FL and ML approaches. The study also examined applicable machine learning models for federated learning. The inclusion measures were (i) published between 2017 and 2021, (ii) written in English, (iii) published in a peer-reviewed scientific journal, and (iv) Preprint published papers. Unpublished studies, thesis and dissertation studies, (ii) conference papers, (iii) not in English, and (iv) did not use artificial intelligence models and blockchain technology were all removed from the review. In total, 84 eligible papers were finally examined in this study. Finally, in recent years, the amount of research on ML using FL has increased. Accuracy equivalent to standard feature-based techniques has been attained, and ensembles of many algorithms may yield even better results. We discovered that the best results were obtained from the hybrid design of an ML ensemble employing expert features. However, some additional difficulties and issues need to be overcome, such as efficiency, complexity, and smaller datasets. In addition, novel FL applications should be investigated from the standpoint of the datasets and methodologies. Full article
(This article belongs to the Special Issue Foundations and Challenges of Interpretable ML)
►▼ Show Figures

Figure 1

Show export options expand_more Show export options expand_less
Select all
Export citation of selected articles as:
 
Displaying articles 1-7
Back to TopTop