Explainable AI in Manufacturing and Industrial Cyber–Physical Systems: A Survey
Abstract
:1. Introduction
1.1. What?
1.2. Why?
- Trustworthiness: In order to use the system predictions in real world applications, the user needs to be trust the applied model. Offering an explanation for a prediction is an important aspect for ensuring human trust and the effective use of ML, if the explanations are faithful and intelligible [8].
- Causality: Causality reveals the cause-and-effect relationship between feature space and possible outputs. Assigning a set of causes for an effect demands wide knowledge, as ML models try to find correlations between the features.
- Transferability: In general, a model is usually trained and tested based on limited data. One reason for pursuing model explainability is to properly utilize a trained model in another domain with similar characteristics. Hence, it can be referred to as reusability.
- Informativeness: Informativeness is the information that an XAI model provides about how it works in order to avoid misconceptions. The model explains the relations inside the box and increases the knowledge of the user regarding the internal process.
- Confidence: A measure that provides an expectation that a decision will prove to be correct or incorrect. The measure should always be assessed within a system where reliability is a concern. Therefore, an explainable model designed for this purpose must offer a confidence level for its predictions.
- Fairness: ML algorithms are products of their data and any bias in the input data will influence the attained results and avoid fair conclusions. An XAI system could reveal imbalances within the data and ensure fairness in ML models.
- Accessibility: Explainability can be considered as a tool for improving internal process of ML models. It gives the users and non-professionals the ability to tune performance based on their requirements.
- Interactivity: This goal is considered for models that require interaction with end-users. The model should describe the decision made and the choices considered, then present the explanation in straightforward, natural language to resolve ambiguity.
- Privacy Awareness: The ability to explain the logic in a model could provide a tool for assessing the privacy. An opaque model may capture sensitive data and cause a privacy breach.
1.3. How?
2. XAI in Manufacturing and Industrial Systems
2.1. Product Development
2.2. Process Control and Automation
2.3. Inventory Management
2.4. Fault Detection and Diagnosis
2.5. Predictive Maintenance
2.6. Product Quality
3. XAI in Industrial Cyber–Physical Systems
Cyber-Security
4. Discussion and Future Directions
- Focus areas of XAI in manufacturing: In the literature, predictive maintenance, fault diagnosis and prognosis emerge as the most extensively discussed fields where XAI is applied within manufacturing. Conversely, other use cases such as product development, process control, and inventory management are rarely addressed or explored in this context.
- Need for tailored explanations: It was noted that the majority of XAI explanations are primarily conveyed through feature importance. However, given the diverse range of users in the manufacturing industry, including machine operators, engineers, scientists, and managers, it is essential to tailor explanations based on the specific expertise and comprehension levels of the intended users, thus ensuring optimal understanding and usability.
- Limitations of existing XAI methods: It was observed that most of the explanations were generated using post hoc methods like SHAP and LIME, covering both global and local perspectives. Nevertheless, these methods exhibited limitations in delivering real-time insights and actionable recommendations for immediate decision-making.
- Advancing XAI techniques: There could be further advancements in XAI techniques tailored specifically for manufacturing applications. This could include the development of hybrid approaches that combine traditional AI methods with XAI techniques to achieve optimal transparency and interpretability while maintaining high performance. Besides, addressing challenges related to scalability, robustness, and compliance with industry regulations will be crucial for the widespread adoption of XAI in industrial settings.
- Real-time XAI: There could be a focus on real-time XAI capabilities to enable dynamic decision-making and adaptation in rapidly changing industrial environments.
- XAI and emerging technologies: The integration of XAI into emerging technologies such as the IoT and edge computing could open up new possibilities for enhancing transparency and interpretability in this domain.
- Regulatory compliance: In the context of ICPSs and industrial settings, adherence to industry standards and regulations is particularly critical due to the potential impact on safety, security, reliability, and performance. For example, safety-critical domains such as healthcare, automotive, and aerospace have stringent regulatory requirements governing the development and deployment of AI systems to ensure patient safety, vehicle reliability, and aviation security. Therefore, any future trends in XAI within ICPSs and industrial settings must take into account the unique regulatory landscape of each industry. This includes conducting thorough assessments of regulatory requirements, integrating compliance measures into the design and development process of AI systems, and establishing transparent mechanisms for documenting and auditing XAI implementations. Furthermore, collaboration between industry stakeholders, regulatory bodies, and AI researchers is essential to address regulatory challenges effectively and ensure that XAI techniques align with industry standards and regulations. By prioritizing compliance and transparency, organizations can build trust in AI systems and facilitate their widespread adoption across diverse industrial sectors.
5. Conclusions
Author Contributions
Funding
Conflicts of Interest
Abbreviations
AHU | Air handling unit |
ACE | Automated concept-based explanation |
AI | Artificial intelligence |
ANN | Artificial neural network |
ARM | Association rule mining |
ASV | Asymmetric Shapley values |
BB | Balanced blagging |
CAM | Class activation map |
CEM | Contrastive explanation method |
CNN | Convolutional neural network |
CP | Ceteris peribus |
CPS | Cyber–physical system |
CWConv | Continuous wavelet convolutional |
CybSec | Cyber-security |
DCAE | Deep convolutional autoencoder |
DCNN | Deep convolutional neural network |
DeconvNet | Deconvolutional network |
DeepLIFT | Deep learning important features |
DICE | Diverse counterfactual explanations |
DIFFI | Depth-based isolation forest feature importance |
DNN | Deep neural network |
DT | Decision tree |
EmSHAP | Energy-based model for Shapley value estimation |
FACE | Feasible and actionable counterfactual explanations |
FAM | Frequency activation map |
FDD | Fault detection and diagnosis |
FedeX | Federated learning based explainable |
FFNN | Feed forward neural network |
FG-CAM | Frequency-domain-based gradient-weighted class activation map |
FL | Federated learning |
FLS | Fuzzy logic system |
GAP | Global average pooling |
GNN | Graph neural networks |
GPR | Gaussian process regression |
Grad-CAM | Gradient-weighted class activation mapping |
GSX | Gumbel–Sigmoid explanator |
IAP | Interpretable anomaly prediction |
ICE | Individual conditional expectation |
ICPS | Industrial cyber–physical system |
IF | Isolation forest |
IIoT | Industrial internet of things |
InvM | Inventory management |
IntGrad | Integrated gradient |
IoT | Internet of things |
ISG | Intrinsic subgraph generation |
KBDBN | Knowledge-based deep belief network |
KNN | K-nearest neighbors |
LFCM | Learning fuzzy cognitive maps |
LGBM | Light gradient-boosting machine |
LIME | Local interpretable model-agnostic explanations |
LRP | Layer-wise relevance propagation |
LSTM | Long short-term memory |
MCD | Minimum covariance determinant |
ML | Machine learning |
MMD | Maximum mean discrepancy |
MP | Meaningful perturbation |
MSE | Mean squared error |
MV | Machine vision |
NAM | Neural additive models |
NN | Neural network |
OCTET | Object-aware counterfactual explanations |
PdM | Predictive maintenance |
PDP | Partial dependence plot |
PC | Process control |
PD | Product development |
ProtoPNet | Prototypical part network |
PS | Prototype selection |
QA | Quality assurance |
QC | Quality control |
RF | Random forest |
RMSE | Root-mean-square error |
RUL | Remaining useful life |
SG | Saliency gradient |
SHAP | Shapley additive explanations |
SmoothGrad | Smoothing gradient |
SMOTE | Synthetic minority oversampling method |
SVERL | Shapley values for explaining reinforcement learning |
SVM | Support vector machine |
TCAV | Testing with concept activation vectors |
WKN | Wavelet kernel net |
XAI | Explainable artificial intelligence |
XGB | Extreme gradient boosting |
References
- Ye, Q.; Xia, J.; Yang, G. Explainable AI for COVID-19 CT Classifiers: An Initial Comparison Study. In Proceedings of the 2021 IEEE 34th International Symposium on Computer-Based Medical Systems (CBMS), Aveiro, Portugal, 7–9 June 2021; pp. 521–526. [Google Scholar] [CrossRef]
- Guidotti, R.; Monreale, A.; Turini, F.; Pedreschi, D.; Giannotti, F. A Survey of Methods for Explaining Black Box Models. ACM Comput. Surv. 2018, 51, 1–42. [Google Scholar] [CrossRef]
- Gunning, D.; Aha, D. DARPA’s Explainable Artificial Intelligence (XAI) Program. AI Mag. 2019, 40, 44–58. [Google Scholar] [CrossRef]
- Goodman, B.; Flaxman, S. European Union Regulations on Algorithmic Decision-Making and a “Right to Explanation”. AI Mag. 2017, 38, 50–57. [Google Scholar] [CrossRef]
- Nor, A.K.M.; Pedapati, S.R.; Muhammad, M.; Leiva, V. Overview of Explainable Artificial Intelligence for Prognostic and Health Management of Industrial Assets Based on Preferred Reporting Items for Systematic Reviews and Meta-Analyses. Sensors 2021, 21, 8020. [Google Scholar] [CrossRef]
- Doshi-Velez, F.; Kim, B. Towards A Rigorous Science of Interpretable Machine Learning. arXiv 2017, arXiv:1702.08608. [Google Scholar]
- Barredo Arrieta, A.; Díaz-Rodríguez, N.; Del Ser, J.; Bennetot, A.; Tabik, S.; Barbado, A.; Garcia, S.; Gil-Lopez, S.; Molina, D.; Benjamins, R.; et al. Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI. Inf. Fusion 2020, 58, 82–115. [Google Scholar] [CrossRef]
- Ribeiro, M.T.; Singh, S.; Guestrin, C. “Why Should I Trust You?”: Explaining the Predictions of Any Classifier. In Proceedings of the KDD ’16: The 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA, 13–17 August 2016. [Google Scholar]
- El Shawi, R.; Sherif, Y.; Al-Mallah, M.; Sakr, S. Interpretability in HealthCare A Comparative Study of Local Machine Learning Interpretability Techniques. In Proceedings of the 2019 IEEE 32nd International Symposium on Computer-Based Medical Systems (CBMS), Cordoba, Spain, 5–7 June 2019; pp. 275–280. [Google Scholar] [CrossRef]
- Speith, T. A Review of Taxonomies of Explainable Artificial Intelligence (XAI) Methods. In Proceedings of the FAccT ’22, 2022 ACM Conference on Fairness, Accountability, and Transparency, New York, NY, USA, 21–24 June 2022; pp. 2239–2250. [Google Scholar] [CrossRef]
- Rong, Y.; Leemann, T.; trang Nguyen, T.; Fiedler, L.; Qian, P.; Unhelkar, V.; Seidel, T.; Kasneci, G.; Kasneci, E. Towards Human-centered Explainable AI: A Survey of User Studies for Model Explanations. arXiv 2023, arXiv:cs.AI/2210.11584. [Google Scholar] [CrossRef]
- Marcinkevičs, R.; Vogt, J.E. Interpretable and explainable machine learning: A methods-centric overview with concrete examples. WIREs Data Min. Knowl. Discov. 2023, 13, e1493. [Google Scholar] [CrossRef]
- Sofianidis, G.; Rozanec, J.M.; Mladenic, D.; Kyriazis, D. A Review of Explainable Artificial Intelligence in Manufacturing. CoRR 2021, 24, 93–113. [Google Scholar]
- Lundberg, S.; Lee, S.I. A Unified Approach to Interpreting Model Predictions. arXiv 2017, arXiv:cs.AI/1705.07874. [Google Scholar]
- Zhou, B.; Khosla, A.; Lapedriza, A.; Oliva, A.; Torralba, A. Learning Deep Features for Discriminative Localization. In Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 27–30 June 2016; pp. 2921–2929. [Google Scholar] [CrossRef]
- Upasane, S.J.; Hagras, H.; Anisi, M.H.; Savill, S.; Taylor, I.; Manousakis, K. A Big Bang-Big Crunch Type-2 Fuzzy Logic System for Explainable Predictive Maintenance. In Proceedings of the 2021 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), Luxembourg, Luxembourg, 11–14 July 2021; pp. 1–8. [Google Scholar] [CrossRef]
- Upasane, S.J.; Hagras, H.; Anisi, M.H.; Savill, S.; Taylor, I.; Manousakis, K. A Type-2 Fuzzy-Based Explainable AI System for Predictive Maintenance Within the Water Pumping Industry. IEEE Trans. Artif. Intell. 2024, 5, 490–504. [Google Scholar] [CrossRef]
- Huong, T.T.; Bac, T.P.; Ha, K.N.; Hoang, N.V.; Hoang, N.X.; Hung, N.T.; Tran, K.P. Federated Learning-Based Explainable Anomaly Detection for Industrial Control Systems. IEEE Access 2022, 10, 53854–53872. [Google Scholar] [CrossRef]
- Kong, B.O.; Kim, M.S.; Kim, B.H.; Lee, J.H. Prediction of Creep Life Using an Explainable Artificial Intelligence Technique and Alloy Design Based on the Genetic Algorithm in Creep-Strength-Enhanced Ferritic 9% Cr Steel. Met. Mater. Int. 2023, 29, 1334–1345. [Google Scholar] [CrossRef]
- Khan, I.A.; Moustafa, N.; Pi, D.; Sallam, K.M.; Zomaya, A.Y.; Li, B. A New Explainable Deep Learning Framework for Cyber Threat Discovery in Industrial IoT Networks. IEEE Internet Things J. 2022, 9, 11604–11613. [Google Scholar] [CrossRef]
- Dix, M.; Chouhan, A.; Ganguly, S.; Pradhan, S.; Saraswat, D.; Agrawal, S.; Prabhune, A. Anomaly detection in the time-series data of industrial plants using neural network architectures. In Proceedings of the 2021 IEEE Seventh International Conference on Big Data Computing Service and Applications (BigDataService), Oxford, UK, 23–26 August 2021; pp. 222–228. [Google Scholar] [CrossRef]
- Sesana, M.; Cavallaro, S.; Calabresi, M.; Capaccioli, A.; Napoletano, L.; Antonello, V.; Grandi, F. Process and Product Quality Optimization with Explainable Artificial Intelligence. In Artificial Intelligence in Manufacturing; Springer: Berlin/Heidelberg, Germany, 2024; pp. 459–477. [Google Scholar] [CrossRef]
- Morita, K.; Davies, D.W.; Butler, K.T.; Walsh, A. Modeling the dielectric constants of crystals using machine learning. J. Chem. Phys. 2020, 153, 024503. [Google Scholar] [CrossRef] [PubMed]
- Hajgató, G.; Wéber, R.; Szilágyi, B.; Tóthpál, B.; Gyires-Tóth, B.; Hős, C. PredMaX: Predictive maintenance with explainable deep convolutional autoencoders. Adv. Eng. Inform. 2022, 54, 101778. [Google Scholar] [CrossRef]
- Li, T.; Zhao, Z.; Sun, C.; Cheng, L.; Chen, X.; Yan, R.; Gao, R.X. WaveletKernelNet: An Interpretable Deep Neural Network for Industrial Intelligent Diagnosis. IEEE Trans. Syst. Man Cybern. Syst. 2022, 52, 2302–2312. [Google Scholar] [CrossRef]
- Sahakyan, M.; Aung, Z.; Rahwan, T. Explainable Artificial Intelligence for Tabular Data: A Survey. IEEE Access 2021, 9, 135392–135422. [Google Scholar] [CrossRef]
- Bodria, F.; Giannotti, F.; Guidotti, R.; Naretto, F.; Pedreschi, D.; Rinzivillo, S. Benchmarking and Survey of Explanation Methods for Black Box Models. arxiv 2021, arXiv:cs.AI/2102.13076. [Google Scholar] [CrossRef]
- Gawde, S.; Patil, S.; Kumar, S.; Kamat, P.; Kotecha, K.; Alfarhood, S. Explainable Predictive Maintenance of Rotating Machines Using LIME, SHAP, PDP, ICE. IEEE Access 2024, 12, 29345–29361. [Google Scholar] [CrossRef]
- Hasan, M.J.; Sohaib, M.; Kim, J.M. An Explainable AI-Based Fault Diagnosis Model for Bearings. Sensors 2021, 21, 4070. [Google Scholar] [CrossRef]
- Dhaou, A.; Bertoncello, A.; Gourvénec, S.; Garnier, J.; Le Pennec, E. Causal and Interpretable Rules for Time Series Analysis. In Proceedings of the KDD ’21, 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, New York, NY, USA, 14–18 August 2021; pp. 2764–2772. [Google Scholar] [CrossRef]
- Jakubowski, J.; Stanisz, P.; Bobek, S.; Nalepa, G.J. Roll Wear Prediction in Strip Cold Rolling with Physics-Informed Autoencoder and Counterfactual Explanations. In Proceedings of the 2022 IEEE 9th International Conference on Data Science and Advanced Analytics (DSAA), Shenzhen, China, 13–16 October 2022; pp. 1–10. [Google Scholar] [CrossRef]
- Ming, Y.; Xu, P.; Qu, H.; Ren, L. Interpretable and Steerable Sequence Learning via Prototypes. In Proceedings of the KDD ’19, 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Anchorage, AK, USA, 4–8 August 2019; pp. 903–913. [Google Scholar] [CrossRef]
- Tan, S.; Soloviev, M.; Hooker, G.; Wells, M.T. Tree Space Prototypes: Another Look at Making Tree Ensembles Interpretable. In Proceedings of the FODS ’20, 2020 ACM-IMS on Foundations of Data Science Conference, New York, NY, USA, 19–20 October 2020; pp. 23–34. [Google Scholar] [CrossRef]
- Kim, M.S.; Yun, J.P.; Park, P. An Explainable Convolutional Neural Network for Fault Diagnosis in Linear Motion Guide. IEEE Trans. Ind. Inform. 2021, 17, 4036–4045. [Google Scholar] [CrossRef]
- Chen, H.Y.; Lee, C.H. Vibration Signals Analysis by Explainable Artificial Intelligence (XAI) Approach: Application on Bearing Faults Diagnosis. IEEE Access 2020, 8, 134246–134256. [Google Scholar] [CrossRef]
- Yeh, C.K.; Kim, B.; Arik, S.O.; Li, C.L.; Pfister, T.; Ravikumar, P. On Completeness-aware Concept-Based Explanations in Deep Neural Networks. arXiv 2022, arXiv:cs.LG/1910.07969. [Google Scholar]
- Guidotti, R.; Monreale, A.; Matwin, S.; Pedreschi, D. Explaining Image Classifiers Generating Exemplars and Counter-Exemplars from Latent Representations. Proc. AAAI Conf. Artif. Intell. 2020, 34, 13665–13668. [Google Scholar] [CrossRef]
- Chen, C.; Li, O.; Tao, C.; Barnett, A.J.; Su, J.; Rudin, C. This looks like that: Deep learning for interpretable image recognition. In Proceedings of the 33rd International Conference on Neural Information Processing Systems, Red Hook, NY, USA, 8 December 2019; Curran Associates Inc.: Red Hook, NY, USA, 2019. [Google Scholar]
- Mollas, I.; Bassiliades, N.; Tsoumakas, G. LioNets: Local Interpretation of Neural Networks Through Penultimate Layer Decoding. In Communications in Computer and Information Science; Springer International Publishing: Berlin/Heidelberg, Germany, 2020; pp. 265–276. [Google Scholar] [CrossRef]
- Hoover, B.; Strobelt, H.; Gehrmann, S. exBERT: A Visual Analysis Tool to Explore Learned Representations in Transformers Models. arXiv 2019, arXiv:cs.CL/1910.05276. [Google Scholar]
- Lampridis, O.; Guidotti, R.; Ruggieri, S. Explaining Sentiment Classification with Synthetic Exemplars and Counter-Exemplars. In Proceedings of the Discovery Science; Appice, A., Tsoumakas, G., Manolopoulos, Y., Matwin, S., Eds.; Springer: Cham, Switzerland, 2020; pp. 357–373. [Google Scholar]
- Beechey, D.; Smith, T.M.S.; Özgür, Ş. Explaining Reinforcement Learning with Shapley Values. arXiv 2023, arXiv:cs.LG/2306.05810. [Google Scholar]
- Mothilal, R.K.; Sharma, A.; Tan, C. Explaining machine learning classifiers through diverse counterfactual explanations. In Proceedings of the ACM FAT ’20, 2020 Conference on Fairness, Accountability, and Transparency, Barcelona, Spain, 27–30 January 2020. [Google Scholar] [CrossRef]
- Ribeiro, M.T.; Singh, S.; Guestrin, C. Anchors: High-precision model-agnostic explanations. In Proceedings of the AAAI’18/IAAI’18/EAAI’18, Thirty-Second AAAI Conference on Artificial Intelligence and Thirtieth Innovative Applications of Artificial Intelligence Conference and Eighth AAAI Symposium on Educational Advances in Artificial Intelligence, New Orleans, LA, USA, 2–7 February 2018; AAAI Press: Washington, DC, USA, 2018. [Google Scholar]
- Kim, B.; Wattenberg, M.; Gilmer, J.; Cai, C.; Wexler, J.; Viegas, F.; Sayres, R. Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV). arXiv 2018, arXiv:stat.ML/1711.11279. [Google Scholar]
- Zhou, B.; Khosla, A.; Lapedriza, A.; Oliva, A.; Torralba, A. Learning Deep Features for Discriminative Localization. arXiv 2015, arXiv:cs.CV/1512.04150. [Google Scholar]
- Shrikumar, A.; Greenside, P.; Kundaje, A. Learning Important Features Through Propagating Activation Differences. arXiv 2019, arXiv:cs.CV/1704.02685. [Google Scholar]
- Lapuschkin, S.; Binder, A.; Montavon, G.; Klauschen, F.; Müller, K.R.; Samek, W. On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation. PLoS ONE 2015, 10, e0130140. [Google Scholar] [CrossRef]
- Zeiler, M.D.; Fergus, R. Visualizing and Understanding Convolutional Networks. arXiv 2013, arXiv:cs.CV/1311.2901. [Google Scholar]
- Selvaraju, R.R.; Cogswell, M.; Das, A.; Vedantam, R.; Parikh, D.; Batra, D. Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization. In Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy, 22–29 October 2017; pp. 618–626. [Google Scholar] [CrossRef]
- Poyiadzi, R.; Sokol, K.; Santos-Rodriguez, R.; De Bie, T.; Flach, P. FACE: Feasible and Actionable Counterfactual Explanations. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, New York, NY, USA, 7–9 February 2020. [Google Scholar] [CrossRef]
- Smilkov, D.; Thorat, N.; Kim, B.; Viégas, F.; Wattenberg, M. SmoothGrad: Removing noise by adding noise. arXiv 2017, arXiv:cs.LG/1706.03825. [Google Scholar]
- Chen, C.; Li, O.; Tao, C.; Barnett, A.J.; Su, J.; Rudin, C. This Looks Like That: Deep Learning for Interpretable Image Recognition. arXiv 2019, arXiv:cs.LG/1806.10574. [Google Scholar]
- Simonyan, K.; Vedaldi, A.; Zisserman, A. Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps. arXiv 2014, arXiv:cs.CV/1312.6034. [Google Scholar]
- Sundararajan, M.; Taly, A.; Yan, Q. Axiomatic Attribution for Deep Networks. arXiv 2017, arXiv:cs.LG/1703.01365. [Google Scholar]
- Goldstein, A.; Kapelner, A.; Bleich, J.; Pitkin, E. Peeking Inside the Black Box: Visualizing Statistical Learning with Plots of Individual Conditional Expectation. arXiv 2014, arXiv:stat.AP/1309.6392. [Google Scholar] [CrossRef]
- Bien, J.; Tibshirani, R. Prototype selection for interpretable classification. Ann. Appl. Stat. 2011, 5, 2403–2424. [Google Scholar] [CrossRef]
- Kim, B.; Khanna, R.; Koyejo, O.O. Examples are not enough, learn to criticize! Criticism for Interpretability. In Proceedings of the Advances in Neural Information Processing Systems; Lee, D., Sugiyama, M., Luxburg, U., Guyon, I., Garnett, R., Eds.; Curran Associates, Inc.: Red Hook, NY, USA, 2016; Volume 29. [Google Scholar]
- Fong, R.C.; Vedaldi, A. Interpretable Explanations of Black Boxes by Meaningful Perturbation. In Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy, 22–29 October 2017; IEEE: New York, NY, USA, 2017. [Google Scholar] [CrossRef]
- Dhurandhar, A.; Chen, P.Y.; Luss, R.; Tu, C.C.; Ting, P.; Shanmugam, K.; Das, P. Explanations based on the Missing: Towards Contrastive Explanations with Pertinent Negatives. arXiv 2018, arXiv:cs.AI/1802.07623. [Google Scholar]
- Chattopadhay, A.; Sarkar, A.; Howlader, P.; Balasubramanian, V.N. Grad-CAM++: Generalized Gradient-Based Visual Explanations for Deep Convolutional Networks. In Proceedings of the 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), Lake Tahoe, NV, USA, 12–15 March 2018; pp. 839–847. [Google Scholar] [CrossRef]
- Ghorbani, A.; Wexler, J.; Zou, J.; Kim, B. Towards Automatic Concept-based Explanations. arXiv 2019, arXiv:stat.ML/1902.03129. [Google Scholar]
- Lundberg, S.M.; Erion, G.; Chen, H.; DeGrave, A.; Prutkin, J.M.; Nair, B.; Katz, R.; Himmelfarb, J.; Bansal, N.; Lee, S.I. Explainable AI for Trees: From Local Explanations to Global Understanding. arXiv 2019, arXiv:cs.LG/1905.04610. [Google Scholar] [CrossRef]
- Ying, R.; Bourgeois, D.; You, J.; Zitnik, M.; Leskovec, J. GNNExplainer: Generating Explanations for Graph Neural Networks. arXiv 2019, arXiv:cs.LG/1903.03894. [Google Scholar]
- Looveren, A.V.; Klaise, J. Interpretable Counterfactual Explanations Guided by Prototypes. arXiv 2020, arXiv:cs.LG/1907.02584. [Google Scholar]
- Huang, Q.; Yamada, M.; Tian, Y.; Singh, D.; Yin, D.; Chang, Y. GraphLIME: Local Interpretable Model Explanations for Graph Neural Networks. arXiv 2020, arXiv:cs.LG/2001.06216. [Google Scholar] [CrossRef]
- Wang, H.; Wang, Z.; Du, M.; Yang, F.; Zhang, Z.; Ding, S.; Mardziel, P.; Hu, X. Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networks. arXiv 2020, arXiv:cs.CV/1910.01279. [Google Scholar]
- Frye, C.; Rowat, C.; Feige, I. Asymmetric Shapley values: Incorporating causal knowledge into model-agnostic explainability. arXiv 2021, arXiv:stat.ML/1910.06358. [Google Scholar]
- Agarwal, R.; Melnick, L.; Frosst, N.; Zhang, X.; Lengerich, B.; Caruana, R.; Hinton, G. Neural Additive Models: Interpretable Machine Learning with Neural Nets. arXiv 2021, arXiv:cs.LG/2004.13912. [Google Scholar]
- Nauta, M.; van Bree, R.; Seifert, C. Neural Prototype Trees for Interpretable Fine-grained Image Recognition. arXiv 2021, arXiv:cs.CV/2012.02046. [Google Scholar]
- Schnake, T.; Eberle, O.; Lederer, J.; Nakajima, S.; Schutt, K.T.; Muller, K.R.; Montavon, G. Higher-Order Explanations of Graph Neural Networks via Relevant Walks. IEEE Trans. Pattern Anal. Mach. Intell. 2022, 44, 7581–7596. [Google Scholar] [CrossRef]
- Zemni, M.; Chen, M.; Éloi, Z.; Ben-Younes, H.; Pérez, P.; Cord, M. OCTET: Object-aware Counterfactual Explanations. arXiv 2023, arXiv:cs.CV/2211.12380. [Google Scholar]
- Lu, C.; Zeng, J.; Xia, Y.; Cai, J.; Luo, S. Energy-based Model for Accurate Shapley Value Estimation in Interpretable Deep Learning Predictive Modeling. arXiv 2024, arXiv:cs.LG/2404.01078. [Google Scholar]
- Tilli, P.; Vu, N.T. Intrinsic Subgraph Generation for Interpretable Graph based Visual Question Answering. arXiv 2024, arXiv:cs.CL/2403.17647. [Google Scholar]
- Zafar, M.R.; Khan, N.M. DLIME: A Deterministic Local Interpretable Model-Agnostic Explanations Approach for Computer-Aided Diagnosis Systems. arXiv 2019, arXiv:cs.LG/1906.10263. [Google Scholar]
- Shankaranarayana, S.M.; Runje, D. ALIME: Autoencoder Based Approach for Local Interpretability. arXiv 2019, arXiv:cs.LG/1909.02437. [Google Scholar]
- Bramhall, S.; Horn, H.E.; Tieu, M.; Lohia, N. QLIME-A Quadratic Local Interpretable Model-Agnostic Explanation Approach. SMU Data Sci. Rev. 2020, 3, 4. [Google Scholar]
- Zhou, J.; Gandomi, A.H.; Chen, F.; Holzinger, A. Evaluating the Quality of Machine Learning Explanations: A Survey on Methods and Metrics. Electronics 2021, 10, 593. [Google Scholar] [CrossRef]
- Nauta, M.; Trienes, J.; Pathak, S.; Nguyen, E.; Peters, M.; Schmitt, Y.; Schlötterer, J.; van Keulen, M.; Seifert, C. From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AI. ACM Comput. Surv. 2023, 55, 1–42. [Google Scholar] [CrossRef]
- Samek, W.; Montavon, G.; Vedaldi, A.; Hansen, L.K.; Müller, K.R. Explainable AI: Interpreting, Explaining and Visualizing Deep Learning; Springer Nature: Cham, Switzerland, 2019; Volume 11700. [Google Scholar]
- Razavi-Far, R.; Wan, D.; Saif, M.; Mozafari, N. To Tolerate or To Impute Missing Values in V2X Communications Data? IEEE Internet Things J. 2022, 9, 11442–11452. [Google Scholar] [CrossRef]
- Hallaji, E.; Razavi-Far, R.; Saif, M. DLIN: Deep Ladder Imputation Network. IEEE Trans. Cybern. 2022, 52, 8629–8641. [Google Scholar] [CrossRef]
- Ahmed, I.; Jeon, G.; Piccialli, F. From Artificial Intelligence to Explainable Artificial Intelligence in Industry 4.0: A Survey on What, How, and Where. IEEE Trans. Ind. Inform. 2022, 18, 5031–5042. [Google Scholar] [CrossRef]
- Park, S.; Kayani, S.H.; Euh, K.; Seo, E.; Kim, H.; Park, S.; Yadav, B.N.; Park, S.J.; Sung, H.; Jung, I.D. High strength aluminum alloys design via explainable artificial intelligence. J. Alloys Compd. 2022, 903, 163828. [Google Scholar] [CrossRef]
- Yan, F.; Song, K.; Liu, Y.; Chen, S.; Chen, J. Predictions and mechanism analyses of the fatigue strength of steel based on machine learning. J. Mater. Sci. 2020, 55, 15334–15349. [Google Scholar] [CrossRef]
- Roy, I.; Feng, B.; Roychowdhury, S.; Ravi, S.K.; Umretiya, R.V.; Reynolds, C.; Ghosh, S.; Rebak, R.B.; Hoffman, A. Understanding oxidation of Fe-Cr-Al alloys through explainable artificial intelligence. MRS Commun. 2023, 13, 82–88. [Google Scholar] [CrossRef]
- Ravi, S.K.; Roy, I.; Roychowdhury, S.; Feng, B.; Ghosh, S.; Reynolds, C.; Umretiya, R.V.; Rebak, R.B.; Hoffman, A.K. Elucidating precipitation in FeCrAl alloys through explainable AI: A case study. Comput. Mater. Sci. 2023, 230, 112440. [Google Scholar] [CrossRef]
- Xiong, J.; Shi, S.Q.; Zhang, T.Y. Machine learning of phases and mechanical properties in complex concentrated alloys. J. Mater. Sci. Technol. 2021, 87, 133–142. [Google Scholar] [CrossRef]
- Yang, C.; Ren, C.; Jia, Y.; Wang, G.; Li, M.; Lu, W. A machine learning-based alloy design system to facilitate the rational design of high entropy alloys with enhanced hardness. Acta Mater. 2022, 222, 117431. [Google Scholar] [CrossRef]
- Jiménez-Luna, J.; Grisoni, F.; Schneider, G. Drug discovery with explainable artificial intelligence. Nat. Mach. Intell. 2020, 2, 573–584. [Google Scholar] [CrossRef]
- Preuer, K.; Klambauer, G.; Rippmann, F.; Hochreiter, S.; Unterthiner, T. Interpretable Deep Learning in Drug Discovery; Springer: Cham, Switzerland, 2019. [Google Scholar]
- Baum, D.; Baum, K.; Gros, T.P.; Wolf, V. XAI Requirements in Smart Production Processes: A Case Study. In Proceedings of the Explainable Artificial Intelligence; Longo, L., Ed.; Springer: Cham, Switzerland, 2023; pp. 3–24. [Google Scholar]
- Perez-Castanos, S.; Prieto-Roig, A.; Monzo, D.; Colomer-Barbera, J. Holistic Production Overview: Using XAI for Production Optimization. In Artificial Intelligence in Manufacturing: Enabling Intelligent, Flexible and Cost-Effective Production Through AI; Springer Nature: Cham, Switzerland, 2024; pp. 423–436. [Google Scholar] [CrossRef]
- Ji-Soo, H.; Yong-Min, H.; Seung-Yong, O.; Tae-Ho, K.; Hyeon-Jeong, L.; Sung-Woo, K. Injection Process Yield Improvement Methodology Based on eXplainable Artificial Intelligence (XAI) Algorithm. J. Korean Soc. Qual. Manag. 2023, 51, 55–65. [Google Scholar]
- Lee, Y.; Roh, Y. An Expandable Yield Prediction Framework Using Explainable Artificial Intelligence for Semiconductor Manufacturing. Appl. Sci. 2023, 13, 2660. [Google Scholar] [CrossRef]
- Kim, S.; Lee, K.; Noh, H.K.; Shin, Y.; Chang, K.B.; Jeong, J.; Baek, S.; Kang, M.; Cho, K.; Kim, D.W.; et al. Automatic Modeling of Logic Device Performance Based on Machine Learning and Explainable AI. In Proceedings of the 2020 International Conference on Simulation of Semiconductor Processes and Devices (SISPAD), Kobe, Japan, 23 September–6 October 2020; pp. 47–50. [Google Scholar] [CrossRef]
- Zhai, W.; Shi, X.; Wong, Y.D.; Han, Q.; Chen, L. Explainable AutoML (xAutoML) with adaptive modeling for yield enhancement in semiconductor smart manufacturing. arXiv 2024, arXiv:cs.CE/2403.12381. [Google Scholar]
- Singh, N.; Adhikari, D. AI in Inventory Management: Applications, Challenges, and Opportunities. Int. J. Res. Appl. Sci. Eng. Technol. 2023, 11, 2049–2053. [Google Scholar] [CrossRef]
- Qaffas, A.A.; Hajkacem, M.A.B.; Ncir, C.E.B.; Nasraoui, O. Interpretable Multi-Criteria ABC Analysis Based on Semi-Supervised Clustering and Explainable Artificial Intelligence. IEEE Access 2023, 11, 43778–43792. [Google Scholar] [CrossRef]
- Ntakolia, C.; Kokkotis, C.; Karlsson, P.; Moustakidis, S. An Explainable Machine Learning Model for Material Backorder Prediction in Inventory Management. Sensors 2021, 21, 7926. [Google Scholar] [CrossRef] [PubMed]
- Shajalal, M.; Boden, A.; Stevens, G. Explainable product backorder prediction exploiting CNN: Introducing explainable models in businesses. Electron. Mark. 2022, 32, 2107–2122. [Google Scholar] [CrossRef]
- Razavi-Far, R.; Kinnaert, M. Incremental Design of a Decision System for Residual Evaluation: A Wind Turbine Application*. In IFAC Proceedings Volumes, Proceedings of the 8th IFAC Symposium on Fault Detection, Supervision and Safety of Technical Processes, Mexico City, Mexico, 29–31 August 2012; Elsevier: Amsterdam, The Netherlands, 2012; Volume 45, pp. 343–348. [Google Scholar] [CrossRef]
- Razavi-Far, R.; Zio, E.; Palade, V. Efficient residuals pre-processing for diagnosing multi-class faults in a doubly fed induction generator, under missing data scenarios. Expert Syst. Appl. 2014, 41, 6386–6399. [Google Scholar] [CrossRef]
- Farajzadeh-Zanjani, M.; Razavi-Far, R.; Saif, M.; Rueda, L. Efficient feature extraction of vibration signals for diagnosing bearing defects in induction motors. In Proceedings of the 2016 International Joint Conference on Neural Networks (IJCNN), Vancouver, BC, Canada, 24–29 July 2016; pp. 4504–4511. [Google Scholar] [CrossRef]
- Razavi-Far, R.; Kinnaert, M. A multiple observers and dynamic weighting ensembles scheme for diagnosing new class faults in wind turbines. Control Eng. Pract. 2013, 21, 1165–1177. [Google Scholar] [CrossRef]
- Saeki, M.; Ogata, J.; Murakawa, M.; Ogawa, T. Visual explanation of neural network based rotation machinery anomaly detection system. In Proceedings of the 2019 IEEE International Conference on Prognostics and Health Management (ICPHM), San Francisco, CA, USA, 17–20 June 2019; pp. 1–4. [Google Scholar] [CrossRef]
- Grezmak, J.; Zhang, J.; Wang, P.; Loparo, K.A.; Gao, R.X. Interpretable Convolutional Neural Network Through Layer-wise Relevance Propagation for Machine Fault Diagnosis. IEEE Sens. J. 2020, 20, 3172–3181. [Google Scholar] [CrossRef]
- Kotriwala, A.; Klöpper, B.; Dix, M.; Gopalakrishnan, G.; Ziobro, D.; Potschka, A. XAI for Operations in the Process Industry - Applications, Theses, and Research Directions. In CEUR Workshop Proceedings, Proceedings of the AAAI 2021 Spring Symposium on Combining Machine Learning and Knowledge Engineering (AAAI-MAKE 2021), Palo Alto, CA, USA, 22–24 March 2021; Martin, A., Hinkelmann, K., Fill, H.G., Gerber, A., Lenat, D., Stolle, R., van Harmelen, F., Eds.; CEUR-WS.Org: Örebro, Sweden, 2021; Volume 2846. [Google Scholar]
- Nor, K.; Pedapati, S.R.; Muhammad, M. Application of Explainable AI (Xai) For Anomaly Detection and Prognostic of Gas Turbines with Uncertainty Quantification. Preprints 2021, 2021, 2021090034. [Google Scholar] [CrossRef]
- Brito, L.C.; Susto, G.A.; Brito, J.N.; Duarte, M.A. An explainable artificial intelligence approach for unsupervised fault detection and diagnosis in rotating machinery. Mech. Syst. Signal Process. 2022, 163, 108105. [Google Scholar] [CrossRef]
- Grezmak, J.; Wang, P.; Sun, C.; Gao, R.X. Explainable Convolutional Neural Network for Gearbox Fault Diagnosis. In Procedia CIRP, Proceedings of the 26th CIRP Conference on Life Cycle Engineering (LCE) Purdue University, West Lafayette, IN, USA, 7–9 May 2019; Elsevier: Amsterdam, The Netherlands, 2019; Volume 80, pp. 476–481. [Google Scholar] [CrossRef]
- Kim, M.S.; Yun, J.P.; Park, P. An Explainable Neural Network for Fault Diagnosis With a Frequency Activation Map. IEEE Access 2021, 9, 98962–98972. [Google Scholar] [CrossRef]
- Srinivasan, S.; Arjunan, P.; Jin, B.; Sangiovanni-Vincentelli, A.L.; Sultan, Z.; Poolla, K. Explainable AI for Chiller Fault-Detection Systems: Gaining Human Trust. Computer 2021, 54, 60–68. [Google Scholar] [CrossRef]
- Madhikermi, M.; Malhi, A.K.; Främling, K. Explainable Artificial Intelligence Based Heat Recycler Fault Detection in Air Handling Unit. In Proceedings of the Explainable, Transparent Autonomous Agents and Multi-Agent Systems; Calvaresi, D., Najjar, A., Schumacher, M., Främling, K., Eds.; Springer: Cham, Switzerland, 2019; pp. 110–125. [Google Scholar]
- Hong, C.W.; Lee, C.; Lee, K.; Ko, M.S.; Hur, K. Explainable Artificial Intelligence for the Remaining Useful Life Prognosis of the Turbofan Engines. In Proceedings of the 2020 3rd IEEE International Conference on Knowledge Innovation and Invention (ICKII), Kaohsiung, Taiwan, 21–23 August 2020; pp. 144–147. [Google Scholar] [CrossRef]
- Abid, F.B.; Sallem, M.; Braham, A. Robust Interpretable Deep Learning for Intelligent Fault Diagnosis of Induction Motors. IEEE Trans. Instrum. Meas. 2020, 69, 3506–3515. [Google Scholar] [CrossRef]
- Sun, K.H.; Huh, H.; Tama, B.A.; Lee, S.Y.; Jung, J.H.; Lee, S. Vision-Based Fault Diagnostics Using Explainable Deep Learning With Class Activation Maps. IEEE Access 2020, 8, 129169–129179. [Google Scholar] [CrossRef]
- Li, Y.F.; Liu, J. A Bayesian Network Approach for Imbalanced Fault Detection in High Speed Rail Systems. In Proceedings of the 2018 IEEE International Conference on Prognostics and Health Management (ICPHM), Seattle, WA, USA, 11–13 June 2018; pp. 1–7. [Google Scholar] [CrossRef]
- Carletti, M.; Masiero, C.; Beghi, A.; Susto, G.A. Explainable Machine Learning in Industry 4.0: Evaluating Feature Importance in Anomaly Detection to Enable Root Cause Analysis. In Proceedings of the 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC), Bari, Italy, 6–9 October 2019; pp. 21–26. [Google Scholar] [CrossRef]
- Szelążek, M.; Bobek, S.; Gonzalez-Pardo, A.; Nalepa, G.J. Towards the Modeling of the Hot Rolling Industrial Process. Preliminary Results. In Proceedings of the Intelligent Data Engineering and Automated Learning—IDEAL 2020; Analide, C., Novais, P., Camacho, D., Yin, H., Eds.; Springer: Cham, Switzerland, 2020; pp. 385–396. [Google Scholar]
- Serradilla, O.; Zugasti, E.; Cernuda, C.; Aranburu, A.; de Okariz, J.R.; Zurutuza, U. Interpreting Remaining Useful Life estimations combining Explainable Artificial Intelligence and domain knowledge in industrial machinery. In Proceedings of the 2020 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), Glasgow, UK, 19–24 July 2020; pp. 1–8. [Google Scholar] [CrossRef]
- Wang, Y.; Wang, P. Explainable machine learning for motor fault diagnosis. In Proceedings of the 2023 IEEE International Instrumentation and Measurement Technology Conference (I2MTC), Kuala Lumpur, Malaysia, 22–25 May 2023; pp. 1–6. [Google Scholar] [CrossRef]
- Gamal Al-Kaf, H.A.; Lee, K.B. Explainable Machine Learning Method for Open Fault Detection of NPC Inverter Using SHAP and LIME. In Proceedings of the 2023 IEEE Conference on Energy Conversion (CENCON), Kuching, Malaysia, 23–24 October 2023; pp. 14–19. [Google Scholar] [CrossRef]
- Gummadi, A.; Napier, J.; Abdallah, M. XAI-IoT: An Explainable AI Framework for Enhancing Anomaly Detection in IoT Systems. IEEE Access 2024, 12, 71024–71054. [Google Scholar] [CrossRef]
- Sinha, A.; Das, D. An explainable deep learning approach for detection and isolation of sensor and machine faults in predictive maintenance paradigm. Meas. Sci. Technol. 2023, 35, 015122. [Google Scholar] [CrossRef]
- Zhou, D.; Yao, Q.; Wu, H.; Ma, S.; Zhang, H. Fault diagnosis of gas turbine based on partly interpretable convolutional neural networks. Energy 2020, 200, 117467. [Google Scholar] [CrossRef]
- Oh, C.; Jeong, J. VODCA: Verification of Diagnosis Using CAM-Based Approach for Explainable Process Monitoring. Sensors 2020, 20, 6858. [Google Scholar] [CrossRef]
- Kumar, P.; Hati, A.S. Deep convolutional neural network based on adaptive gradient optimizer for fault detection in SCIM. ISA Trans. 2021, 111, 350–359. [Google Scholar] [CrossRef]
- Felsberger, L.; Apollonio, A.; Cartier-Michaud, T.; Müller, A.; Todd, B.; Kranzlmüller, D. Explainable Deep Learning for Fault Prognostics in Complex Systems: A Particle Accelerator Use-Case. In Proceedings of the Machine Learning and Knowledge Extraction; Holzinger, A., Kieseberg, P., Tjoa, A.M., Weippl, E., Eds.; Springer: Cham, Switzerland, 2020; pp. 139–158. [Google Scholar]
- Grezmak, J.; Zhang, J.; Wang, P.; Gao, R.X. Multi-stream convolutional neural network-based fault diagnosis for variable frequency drives in sustainable manufacturing systems. In Procedia Manufacturing, Proceedings of the Sustainable Manufacturing—Hand in Hand to Sustainability on Globe: Proceedings of the 17th Global Conference on Sustainable Manufacturing, Shanghai, China, 9–11 October 2020; Elsevier: Amsterdam, The Netherlands, 2020; Volume 43, pp. 511–518. [Google Scholar]
- Lee, J.; Noh, I.; Lee, J.; Lee, S.W. Development of an Explainable Fault Diagnosis Framework Based on Sensor Data Imagification: A Case Study of the Robotic Spot-Welding Process. IEEE Trans. Ind. Inform. 2022, 18, 6895–6904. [Google Scholar] [CrossRef]
- Yang, D.; Karimi, H.R.; Gelman, L. An explainable intelligence fault diagnosis framework for rotating machinery. Neurocomputing 2023, 541, 126257. [Google Scholar] [CrossRef]
- Nie, X.; Xie, G. A novel normalized recurrent neural network for fault diagnosis with noisy labels. J. Intell. Manuf. 2021, 32, 1271–1288. [Google Scholar] [CrossRef]
- Gribbestad, M.; Hassan, M.U.; Hameed, I.A.; Sundli, K. Health Monitoring of Air Compressors Using Reconstruction-Based Deep Learning for Anomaly Detection with Increased Transparency. Entropy 2021, 23, 83. [Google Scholar] [CrossRef] [PubMed]
- Brusa, E.; Cibrario, L.; Delprete, C.; Di Maggio, L.G. Explainable AI for Machine Fault Diagnosis: Understanding Features’ Contribution in Machine Learning Models for Industrial Condition Monitoring. Appl. Sci. 2023, 13, 2038. [Google Scholar] [CrossRef]
- Moosavi, S.; Razavi-Far, R.; Palade, V.; Saif, M. Explainable Artificial Intelligence Approach for Diagnosing Faults in an Induction Furnace. Electronics 2024, 13, 1721. [Google Scholar] [CrossRef]
- Amarasinghe, K.; Kenney, K.; Manic, M. Toward Explainable Deep Neural Network Based Anomaly Detection. In Proceedings of the 2018 11th International Conference on Human System Interaction (HSI), Gdansk, Poland, 4–6 July 2018; pp. 311–317. [Google Scholar] [CrossRef]
- Yu, J.; Liu, G. Knowledge extraction and insertion to deep belief network for gearbox fault diagnosis. Knowl.-Based Syst. 2020, 197, 105883. [Google Scholar] [CrossRef]
- Keleko, A.T.; Kamsu-Foguem, B.; Ngouna, R.H.; Tongne, A. Health condition monitoring of a complex hydraulic system using Deep Neural Network and DeepSHAP explainable XAI. Adv. Eng. Softw. 2023, 175, 103339. [Google Scholar] [CrossRef]
- Liu, Y.; Li, Z.; Chen, H. Artificial Intelligence-based Fault Detection and Diagnosis: Towards Application in a Chemical Process. In Proceedings of the 2023 CAA Symposium on Fault Detection, Supervision and Safety for Technical Processes (SAFEPROCESS), Yibin, China, 22–24 September 2023; pp. 1–6. [Google Scholar] [CrossRef]
- Santos, M.R.; Guedes, A.; Sanchez-Gendriz, I. SHapley Additive exPlanations (SHAP) for Efficient Feature Selection in Rolling Bearing Fault Diagnosis. Mach. Learn. Knowl. Extr. 2024, 6, 316–341. [Google Scholar] [CrossRef]
- Harinarayan, R.R.A.; Shalinie, S.M. XFDDC: Explainable Fault Detection Diagnosis and Correction framework for chemical process systems. Process Saf. Environ. Prot. 2022, 165, 463–474. [Google Scholar] [CrossRef]
- Sinha, A.; Das, D. XAI-LCS: Explainable AI-Based Fault Diagnosis of Low-Cost Sensors. IEEE Sens. Lett. 2023, 7, 1–4. [Google Scholar] [CrossRef]
- Meas, M.; Machlev, R.; Kose, A.; Tepljakov, A.; Loo, L.; Levron, Y.; Petlenkov, E.; Belikov, J. Explainability and Transparency of Classifiers for Air-Handling Unit Faults Using Explainable Artificial Intelligence (XAI). Sensors 2022, 22, 6338. [Google Scholar] [CrossRef]
- Devkar, P.; Venkatarathnam, G. Enhancing Fault Detection and Diagnosis in AHU Using Explainable AI. In Sustainability in Energy and Buildings 2023; Littlewood, J.R., Jain, L., Howlett, R.J., Eds.; Springer Nature: Singapore, 2024; pp. 131–142. [Google Scholar] [CrossRef]
- Hrnjica, B.; Softic, S. Explainable AI in Manufacturing: A Predictive Maintenance Case Study; Springer: Cham, Switzerland, 2020; pp. 66–73. [Google Scholar] [CrossRef]
- Paolanti, M.; Romeo, L.; Felicetti, A.; Mancini, A.; Frontoni, E.; Loncarski, J. Machine Learning approach for Predictive Maintenance in Industry 4.0. In Proceedings of the 2018 14th IEEE/ASME International Conference on Mechatronic and Embedded Systems and Applications (MESA), Oulu, Finland, 2–4 July 2018; pp. 1–6. [Google Scholar] [CrossRef]
- Langone, R.; Cuzzocrea, A.; Skantzos, N. Interpretable Anomaly Prediction: Predicting anomalous behavior in industry 4.0 settings via regularized logistic regression tools. Data Knowl. Eng. 2020, 130, 101850. [Google Scholar] [CrossRef]
- Cummins, L.; Sommers, A.; Ramezani, S.B.; Mittal, S.; Jabour, J.; Seale, M.; Rahimi, S. Explainable Predictive Maintenance: A Survey of Current Methods, Challenges and Opportunities. IEEE Access 2024, 12, 57574–57602. [Google Scholar] [CrossRef]
- Tchakoua, P.; Wamkeue, R.; Hasnaoui, F.; Theubou Tameghe, T.A.; Ekemb, G. New trends and future challenges for wind turbines condition monitoring. In Proceedings of the 2013 International Conference on Control, Automation and Information Sciences (ICCAIS), Nha Trang, Vietnam, 25–28 November 2013; pp. 238–245. [Google Scholar] [CrossRef]
- Matzka, S. Explainable Artificial Intelligence for Predictive Maintenance Applications. In Proceedings of the 2020 Third International Conference on Artificial Intelligence for Industries (AI4I), Irvine, CA, USA, 21–23 September 2020; pp. 69–74. [Google Scholar] [CrossRef]
- Wu, H.; Huang, A.; Sutherland, J.W. Layer-wise relevance propagation for interpreting LSTM-RNN decisions in predictive maintenance. Int. J. Adv. Manuf. Technol. 2022, 118, 963–978. [Google Scholar] [CrossRef]
- Mansouri, T.; Vadera, S. Explainable fault prediction using learning fuzzy cognitive maps. Expert Syst. 2023, 40, e13316. [Google Scholar] [CrossRef]
- Mansouri, T.; Vadera, S. A Deep Explainable Model for Fault Prediction Using IoT Sensors. IEEE Access 2022, 10, 66933–66942. [Google Scholar] [CrossRef]
- Christou, I.T.; Kefalakis, N.; Zalonis, A.; Soldatos, J. Predictive and Explainable Machine Learning for Industrial Internet of Things Applications. In Proceedings of the 2020 16th International Conference on Distributed Computing in Sensor Systems (DCOSS), Marina del Rey, CA, USA, 25–27 May 2020; pp. 213–218. [Google Scholar] [CrossRef]
- Silva, R.L.; Rudek, M.; Szejka, A.L.; Junior, O.C. Machine Vision Systems for Industrial Quality Control Inspections. In Proceedings of the Product Lifecycle Management to Support Industry 4.0; Chiabert, P., Bouras, A., Noël, F., Ríos, J., Eds.; Springer: Cham, Switzerland, 2018; pp. 631–641. [Google Scholar]
- Goldman, C.; Baltaxe, M.; Chakraborty, D.; Arinez, J. Explaining Learning Models in Manufacturing Processes. Procedia Comput. Sci. 2021, 180, 259–268. [Google Scholar] [CrossRef]
- Lee, M.; Jeon, J.; Lee, H. Explainable AI for domain experts: A post Hoc analysis of deep learning for defect classification of TFT–LCD panels. J. Intell. Manuf. 2022, 33, 1747–1759. [Google Scholar] [CrossRef]
- Senoner, J.; Netland, T.; Feuerriegel, S. Using Explainable Artificial Intelligence to Improve Process Quality: Evidence from Semiconductor Manufacturing. Manag. Sci. 2021, 68, 5557–6354. [Google Scholar] [CrossRef]
- Kharal, A. Explainable Artificial Intelligence Based Fault Diagnosis and Insight Harvesting for Steel Plates Manufacturing. arXiv 2020, arXiv:cs.AI/2008.04448. [Google Scholar]
- Meister, S.; Wermes, M.; Stüve, J.; Groves, R. Investigations on Explainable Artificial Intelligence methods for the deep learning classification of fibre layup defect in the automated composite manufacturing. Compos. Part B Eng. 2021, 224, 109160. [Google Scholar] [CrossRef]
- Lavasa, E.; Chadoulos, C.; Siouras, A.; Etxabarri Llana, A.; Rodríguez Del Rey, S.; Dalamagas, T.; Moustakidis, S. Toward Explainable Metrology 4.0: Utilizing Explainable AI to Predict the Pointwise Accuracy of Laser Scanning Devices in Industrial Manufacturing. In Artificial Intelligence in Manufacturing: Enabling Intelligent, Flexible and Cost-Effective Production Through AI; Springer Nature: Cham, Switzerland, 2024; pp. 479–501. [Google Scholar] [CrossRef]
- Hwang, C.; Lee, T. E-SFD: Explainable Sensor Fault Detection in the ICS Anomaly Detection System. IEEE Access 2021, 9, 140470–140486. [Google Scholar] [CrossRef]
- Makridis, G.; Theodoropoulos, S.; Dardanis, D.; Makridis, I.; Separdani, M.M.; Fatouros, G.; Kyriazis, D.; Koulouris, P. XAI enhancing cyber defence against adversarial attacks in industrial applications. In Proceedings of the 2022 IEEE 5th International Conference on Image Processing Applications and Systems (IPAS), Genova, Italy, 5–7 December 2022; Volume 5, pp. 1–8. [Google Scholar] [CrossRef]
- Bac, T.P.; Ha, D.T.; Tran, K.D.; Tran, K.P. Explainable Articial Intelligence for Cybersecurity in Smart Manufacturing. In Artificial Intelligence for Smart Manufacturing: Methods, Applications, and Challenges; Tran, K.P., Ed.; Springer International Publishing: Cham, Switzerland, 2023; pp. 199–223. [Google Scholar] [CrossRef]
- Sivamohan, S.; Sridhar, S.S. An optimized model for network intrusion detection systems in industry 4.0 using XAI based Bi-LSTM framework. Neural Comput. Appl. 2023, 35, 11459–11475. [Google Scholar] [CrossRef] [PubMed]
- Kundu, R.K.; Hoque, K.A. Explainable Predictive Maintenance is Not Enough: Quantifying Trust in Remaining Useful Life Estimation. Annu. Conf. Phm Soc. 2023, 15. [Google Scholar] [CrossRef]
- Klamert, V.; Schmid-Kietreiber, M.; Bublin, M. A deep learning approach for real time process monitoring and curling defect detection in Selective Laser Sintering by infrared thermography and convolutional neural networks. In Procedia CIRP, Proceedings of the 12th CIRP Conference on Photonic Technologies [LANE 2022], Furth, Germany, 4–8 September 2022; Elsevier: Amsterdam, The Netherlands, 2022; Volume 111, pp. 317–320. [Google Scholar] [CrossRef]
- Hanchate, A.; Bukkapatnam, S.T.; Lee, K.H.; Srivastava, A.; Kumara, S. Explainable AI (XAI)-driven vibration sensing scheme for surface quality monitoring in a smart surface grinding process. J. Manuf. Processes 2023, 99, 184–194. [Google Scholar] [CrossRef]
- Javaid, M.; Haleem, A.; Singh, R.P.; Suman, R.; Gonzalez, E.S. Understanding the adoption of Industry 4.0 technologies in improving environmental sustainability. Sustain. Oper. Comput. 2022, 3, 203–217. [Google Scholar] [CrossRef]
- Yu, Z.; Gao, H.; Cong, X.; Wu, N.; Song, H.H. A Survey on Cyber–Physical Systems Security. IEEE Internet Things J. 2023, 10, 21670–21686. [Google Scholar] [CrossRef]
- Alguliyev, R.; Imamverdiyev, Y.; Sukhostat, L. Cyber-physical systems and their security issues. Comput. Ind. 2018, 100, 212–223. [Google Scholar] [CrossRef]
- Farajzadeh-Zanjani, M.; Hallaji, E.; Razavi-Far, R.; Saif, M. Generative-Adversarial Class-Imbalance Learning for Classifying Cyber-Attacks and Faults - A Cyber-Physical Power System. IEEE Trans. Dependable Secur. Comput. 2022, 19, 4068–4081. [Google Scholar] [CrossRef]
- Farajzadeh-Zanjani, M.; Hallaji, E.; Razavi-Far, R.; Saif, M. Generative adversarial dimensionality reduction for diagnosing faults and attacks in cyber-physical systems. Neurocomputing 2021, 440, 101–110. [Google Scholar] [CrossRef]
- Kim, S.; Park, K.J.; Lu, C. A Survey on Network Security for Cyber–Physical Systems: From Threats to Resilient Design. IEEE Commun. Surv. Tutor. 2022, 24, 1534–1573. [Google Scholar] [CrossRef]
- Hoenig, A.; Roy, K.; Acquaah, Y.; Yi, S.; Desai, S. Explainable AI for Cyber-Physical Systems: Issues and Challenges. IEEE Access 2024, 12, 73113–73140. [Google Scholar] [CrossRef]
- Le, D.; Vung, P.; Nguyen, H.; Dang, T. Visualization and Explainable Machine Learning for Efficient Manufacturing and System Operations. Smart Sustain. Manuf. Syst. 2019, 3, 20190029. [Google Scholar] [CrossRef]
- Wang, M.; Zheng, K.; Yang, Y.; Wang, X. An Explainable Machine Learning Framework for Intrusion Detection Systems. IEEE Access 2020, 8, 73127–73141. [Google Scholar] [CrossRef]
- Barnard, P.; Marchetti, N.; DaSilva, L.A. Robust Network Intrusion Detection Through Explainable Artificial Intelligence (XAI). IEEE Netw. Lett. 2022, 4, 167–171. [Google Scholar] [CrossRef]
- Houda, Z.A.E.; Brik, B.; Khoukhi, L. “Why Should I Trust Your IDS?”: An Explainable Deep Learning Framework for Intrusion Detection Systems in Internet of Things Networks. IEEE Open J. Commun. Soc. 2022, 3, 1164–1176. [Google Scholar] [CrossRef]
- Zebin, T.; Rezvy, S.; Luo, Y. An Explainable AI-Based Intrusion Detection System for DNS Over HTTPS (DoH) Attacks. IEEE Trans. Inf. Forensics Secur. 2022, 17, 2339–2349. [Google Scholar] [CrossRef]
- Oseni, A.; Moustafa, N.; Creech, G.; Sohrabi, N.; Strelzoff, A.; Tari, Z.; Linkov, I. An Explainable Deep Learning Framework for Resilient Intrusion Detection in IoT-Enabled Transportation Networks. IEEE Trans. Intell. Transp. Syst. 2023, 24, 1000–1014. [Google Scholar] [CrossRef]
- Gaspar, D.; Silva, P.; Silva, C. Explainable AI for Intrusion Detection Systems: LIME and SHAP Applicability on Multi-Layer Perceptron. IEEE Access 2024, 12, 30164–30175. [Google Scholar] [CrossRef]
- Jeong, S.; Lee, S.; Lee, H.; Kim, H.K. X-CANIDS: Signal-Aware Explainable Intrusion Detection System for Controller Area Network-Based In-Vehicle Network. IEEE Trans. Veh. Technol. 2024, 73, 3230–3246. [Google Scholar] [CrossRef]
- Arreche, O.; Guntur, T.R.; Roberts, J.W.; Abdallah, M. E-XAI: Evaluating Black-Box Explainable AI Frameworks for Network Intrusion Detection. IEEE Access 2024, 12, 23954–23988. [Google Scholar] [CrossRef]
- Shtayat, M.M.; Hasan, M.K.; Sulaiman, R.; Islam, S.; Khan, A.U.R. An Explainable Ensemble Deep Learning Approach for Intrusion Detection in Industrial Internet of Things. IEEE Access 2023, 11, 115047–115061. [Google Scholar] [CrossRef]
- Javeed, D.; Gao, T.; Kumar, P.; Jolfaei, A. An Explainable and Resilient Intrusion Detection System for Industry 5.0. IEEE Trans. Consum. Electron. 2024, 70, 1342–1350. [Google Scholar] [CrossRef]
- Arisdakessian, S.; Wahab, O.A.; Mourad, A.; Otrok, H.; Guizani, M. A Survey on IoT Intrusion Detection: Federated Learning, Game Theory, Social Psychology, and Explainable AI as Future Directions. IEEE Internet Things J. 2023, 10, 4059–4092. [Google Scholar] [CrossRef]
- Neupane, S.; Ables, J.; Anderson, W.; Mittal, S.; Rahimi, S.; Banicescu, I.; Seale, M. Explainable Intrusion Detection Systems (X-IDS): A Survey of Current Methods, Challenges, and Opportunities. IEEE Access 2022, 10, 112392–112415. [Google Scholar] [CrossRef]
- Moustafa, N.; Koroniotis, N.; Keshk, M.; Zomaya, A.Y.; Tari, Z. Explainable Intrusion Detection for Cyber Defences in the Internet of Things: Opportunities and Solutions. IEEE Commun. Surv. Tutor. 2023, 25, 1775–1807. [Google Scholar] [CrossRef]
- Chen, T.C.T. Explainable Artificial Intelligence (XAI) in Manufacturing. In Explainable Artificial Intelligence (XAI) in Manufacturing: Methodology, Tools, and Applications; Springer International Publishing: Cham, Switzerland, 2023; pp. 1–11. [Google Scholar] [CrossRef]
- Islam, M.R.; Ahmed, M.U.; Barua, S.; Begum, S. A Systematic Review of Explainable Artificial Intelligence in Terms of Different Application Domains and Tasks. Appl. Sci. 2022, 12, 1353. [Google Scholar] [CrossRef]
Models | XAI Methods | References |
---|---|---|
Bayesian Network | Intrinsic | [118] |
Fuzzy Logic System | Intrinsic | [16] |
Isolation Forest | SHAP, Local-DIFFI | [22,110,119] |
RF | SHAP, ELI5, LIME, CEM | [120,121,122,123,124] |
CNN | CAM, grad-CAM, FAM, SHAP, LRP | [34,35,106,107,111,112,115,117,125,126,127,128,129,130,131,132] |
RNN | LRP | [133] |
LSTM | SHAP, MAE | [115,134] |
Bi-LSTM | SHAP | [115] |
Bayesian DL | SHAP | [109] |
KNN | SHAP | [29,135] |
ANN | LIME | [114,122] |
DNN | SHAP, LIME, LRP, KBDBN, CEM | [124,136,137,138,139] |
SVM | SHAP, LIME, DIFFI, CEM | [114,122,124,135,140,141] |
XGB | LIME, SHAP | [113,125,142,143,144,145] |
Reference | Year | Use Case | Algorithm | Data Type | XAI Approach | XAI Output |
---|---|---|---|---|---|---|
[85] | 2020 | PD | XGB and LGBM | Tabular | SHAP | Feature Importance |
[84] | 2022 | PD | Keras-based DNN | Chemical composition | LIME | Feature Importance |
[86] | 2023 | PD | NN | Tabular | SHAP | Feature Importance |
[87] | 2023 | PD | GPR | Tabular | SHAP | Feature Importance |
[96] | 2020 | PC | RF, XGB, Catboost | Tabular | SHAP | Feature Importance |
[93] | 2023 | PC | RF | Tabular | LIME | Feature Importance |
[94] | 2023 | PC | XGB and LGBM | Tabular | SHAP | Feature Importance |
[95] | 2023 | PC | RF | Tabular | SHAP | Feature Importance |
[100] | 2021 | InvM | RF, KNN, LGBM, XGB, BB, NN, LR, SVM | Tabular | SHAP | Feature Importance |
[101] | 2022 | InvM | CNN | Tabular | SHAP, LIME | Feature Importance |
[99] | 2023 | InvM | SS-E-k-means | Tabular | SHAP | Feature Importance |
[106] | 2019 | FDD | CNN | Vibration | grad-CAM | Visual Explanation |
[114] | 2019 | FDD | SVM and DNN | Temperature and Speed | LIME | Feature Importance |
[111] | 2019 | FDD | DCNN | Vibration | LRP | Visual Explanation |
[35] | 2020 | FDD | CNN | Vibration | grad-CAM | Visual Explanation |
[107] | 2020 | FDD | CNN | Vibration | LRP | Visual Explanation |
[115] | 2020 | FDD | CNN, LSTM, Bi-LSTM | Temperature, Pressure and Speed | SHAP | Feature Importance |
[116] | 2020 | FDD | deep-SincNet | Current | Intrinsic | Temporal and Spectral Presentation |
[117] | 2020 | FDD | CNN | Vibration | CAM | Visual Explanation |
[110] | 2021 | FDD | KNN, IF, etc. | Vibration | SHAP and Local-DIFFI | Feature Importance |
[29] | 2021 | FDD | KNN | Vibration | SHAP | Feature Importance |
[34] | 2021 | FDD | CNN | Vibration | grad-CAM | Visual Explanation |
[112] | 2021 | FDD | CNN | Vibration | FAM | Visual Explanation |
[109] | 2021 | FDD | DNN | Temperature, Pressure, Speed and Position | SHAP | Feature Importance |
[113] | 2021 | FDD | XGB | Temperature, Flow and Power | LIME | Feature Importance |
[25] | 2021 | FDD | WKN | Vibration | Intrinsic | Feature Map |
[142] | 2022 | FDD | XGB | Tabular | SHAP | Feature Importance |
[125] | 2023 | FDD | CNN-XGB | Vibration | SHAP | Feature Importance |
[137] | 2018 | CybSec | DNN | Tabular | LRP | Feature Importance |
[163] | 2021 | CybSec | Bi-LSTM | Tabular | SHAP | Feature Importance |
[20] | 2021 | CybSec | Conv-LSTM | Tabular | LIME | Feature Importance |
[164] | 2022 | CybSec | CNN | Image | grad-CAM, LIME | Visual Explanation |
[165] | 2023 | CybSec | LSTM | Tabular | SHAP | Feature Importance |
[166] | 2023 | CybSec | BiLSTM | Tabular | SHAP, LIME | Feature Importance |
[146] | 2020 | PdM | LGBM | Tabular | Intrinsic | Feature Importance |
[148] | 2020 | PdM | Regularized Logistic Regression | Temperature, Pressure, and Speed | Intrinsic | Feature Importance |
[16] | 2021 | PdM | FLS | Pressure and Current | Intrinsic | Rules |
[24] | 2022 | PdM | PCA + DCAE | Tabular | Intrinsic | Feature Importance |
[167] | 2023 | PdM | XGB, RF, LR, FFNN | Tabular | SHAP, LIME, Anchor | Rules, Feature Importance |
[17] | 2024 | PdM | FLS | Pressure, vibration, ultrasonic, and Current | Intrinsic | Rules |
[28] | 2024 | PdM | SVM, RF, DT, KNN | Vibration, current, and temperature | LIME, SHAP, PDP, ICE | Feature Importance |
[157] | 2021 | QA | CNN | Tabular | CAM | Visual Explanation |
[158] | 2021 | QA | CNN | Image | LRP and DT | Visualization and Rules |
[159] | 2021 | QA | Non-linear Meta Model | Tabular | SHAP | Feature Importance |
[160] | 2020 | QA | RF | Tabular | ARM | Rules |
[161] | 2021 | QA | CNN | Image | SHAP, Smooth IG, grad-CAM | Visual Explanation |
[168] | 2022 | QA | CNN | Image | grad-CAM | Visual Explanation |
[169] | 2023 | QA | CNN | Image | LIME | Feature Importance |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Moosavi, S.; Farajzadeh-Zanjani, M.; Razavi-Far, R.; Palade, V.; Saif, M. Explainable AI in Manufacturing and Industrial Cyber–Physical Systems: A Survey. Electronics 2024, 13, 3497. https://doi.org/10.3390/electronics13173497
Moosavi S, Farajzadeh-Zanjani M, Razavi-Far R, Palade V, Saif M. Explainable AI in Manufacturing and Industrial Cyber–Physical Systems: A Survey. Electronics. 2024; 13(17):3497. https://doi.org/10.3390/electronics13173497
Chicago/Turabian StyleMoosavi, Sajad, Maryam Farajzadeh-Zanjani, Roozbeh Razavi-Far, Vasile Palade, and Mehrdad Saif. 2024. "Explainable AI in Manufacturing and Industrial Cyber–Physical Systems: A Survey" Electronics 13, no. 17: 3497. https://doi.org/10.3390/electronics13173497