Machine Learning and Knowledge Extraction

Bioinspired computing methods, such as Artificial Neural Networks (ANNs), play a significant role in machine learning. This is particularly evident in smart manufacturing, where ANNs and their derivatives, like deep learning, are widely used for pattern recognition and adaptive control. However, ANNs sometimes fail to achieve the desired results, especially when working with small datasets. To address this limitation, this article presents the effectiveness of DNA-Based Computing (DBC) as a complementary approach. DBC is an innovative machine learning method rooted in the central dogma of molecular biology that deals with the genetic information of DNA/RNA to protein. In this article, two machine learning approaches are considered. In the first approach, an ANN was trained and tested using time series datasets driven by long and short windows, with features extracted from the time domain. Each long-window-driven dataset contained approximately 150 data points, while each short-window-driven dataset had approximately 10 data points. The results showed that the ANN performed well for long-window-driven datasets. However, its performance declined significantly in the case of short-window-driven datasets. In the last approach, a hybrid model was developed by integrating DBC with the ANN. In this case, the features were first extracted using DBC. The extracted features were used to train and test the ANN. This hybrid approach demonstrated robust performance for both long- and short-window-driven datasets. The ability of DBC to overcome the ANN’s limitations with short-window-driven datasets underscores its potential as a pragmatic machine learning solution for developing more effective smart manufacturing systems, such as digital twins. Full article

Deep neural networks (DNNs) are highly effective across many domains but are sensitive to noisy or corrupted training data. Existing noise mitigation strategies often rely on strong assumptions about noise distributions or require costly retraining, limiting their scalability. Inspired by machine unlearning, we propose a novel framework that integrates attribution-guided data partitioning, neuron pruning, and targeted fine-tuning to enhance robustness. Our method uses gradient-based attribution to probabilistically identify clean samples without assuming specific noise characteristics. It then applies sensitivity-based neuron pruning to remove components most susceptible to noise, followed by fine-tuning on the retained high-quality subset. This approach jointly addresses data and model-level noise, offering a practical alternative to full retraining or explicit noise modeling. We evaluate our method on CIFAR-10 image classification and keyword spotting tasks under varying levels of label corruption. On CIFAR-10, our framework improves accuracy by up to 10% (F-FT vs. retrain) and reduces retraining time by 47% (L-FT vs. retrain), highlighting both accuracy and efficiency gains. These results highlight its effectiveness and efficiency in noisy settings, making it a scalable solution for robust generalization. Full article

Knee osteoarthritis (KOA) is a most prevalent chronic muscoloskeletal disorder causing pain and functional impairment. Accurate predictions of KOA evolution are important for early interventions and preventive treatment planning. In this paper, we propose a novel dynamic hypergraph-based risk model (DyHRM) which integrates the encoder–decoder (ED) architecture with hypergraph convolutional neural networks (HGCNs). The risk model is used to generate longitudinal forecasts of KOA incidence and progression based on the knee evolution at a historical stage. DyHRM comprises two main parts, namely the dynamic hypergraph gated recurrent unit (DyHGRU) and the multi-view HGCN (MHGCN) networks. The ED-based DyHGRU follows the sequence-to-sequence learning approach. The encoder first transforms a knee sequence at the historical stage into a sequence of hidden states in a latent space. The Attention-based Context Transformer (ACT) is designed to identify important temporal trends in the encoder’s state sequence, while the decoder is used to generate sequences of KOA progression, at the prediction stage. MHGCN conducts multi-view spatial HGCN convolutions of the original knee data at each step of the historic stage. The aim is to acquire more comprehensive feature representations of nodes by exploiting different hyperedges (views), including the global shape descriptors of the cartilage volume, the injury history, and the demographic risk factors. In addition to DyHRM, we also propose the HyGraphSMOTE method to confront the inherent class imbalance problem in KOA datasets, between the knee progressors (minority) and non-progressors (majority). Embedded in MHGCN, the HyGraphSMOTE algorithm tackles data balancing in a systematic way, by generating new synthetic node sequences of the minority class via interpolation. Extensive experiments are conducted using the Osteoarthritis Initiative (OAI) cohort to validate the accuracy of longitudinal predictions acquired by DyHRM under different definition criteria of KOA incidence and progression. The basic finding of the experiments is that the larger the historic depth, the higher the accuracy of the obtained forecasts ahead. Comparative results demonstrate the efficacy of DyHRM against other state-of-the-art methods in this field. Full article

While neural networks can solve complex geometric problems, as demonstrated by systems like AlphaGeometry, we have limited understanding of how they internally represent and reason about spatial relationships. In this work, we investigate how neural networks develop internal spatial understanding by training Graph Neural Networks and Transformers to predict point positions on a discrete 2D grid from geometric constraints that describe hidden figures. We show that both models develop interpretable internal representations that mirror the geometric structure of the problems they solve. Specifically, we observe that point embeddings self-organize into 2D grid structures during training, and during inference, the models iteratively construct the hidden geometric figures within their embedding spaces. Our analysis reveals how reasoning complexity correlates with prediction accuracy, and shows that models solve constraints through an iterative refinement process, which might resemble continuous optimization. We also find that Graph Neural Networks prove more suitable than Transformers for this type of structured constraint reasoning and scale more effectively to larger problems. These findings provide initial insights into how neural networks can develop structured understanding and contribute to their interpretability. Full article

Multimodal medical data provides a wide and real basis for disease diagnosis. Computer-aided diagnosis (CAD) powered by artificial intelligence (AI) is becoming increasingly prominent in disease diagnosis. CAD for multimodal medical data requires addressing the issues of data fusion and prediction. Traditionally, the prediction performance of CAD models has not been good enough due to the complicated dimensionality reduction. Therefore, this paper proposes a fusion and prediction model—EPGC—for multimodal medical data based on graph neural networks. Firstly, we select features from unstructured multimodal medical data and quantify them. Then, we transform the multimodal medical data into a graph data structure by establishing each patient as a node, and establishing edges based on the similarity of features between the patients. Normalization of data is also essential in this process. Finally, we build a node prediction model based on graph neural networks and predict the node classification, which predicts the patients’ diseases. The model is validated on two publicly available datasets of heart diseases. Compared to the existing models that typically involve dimensionality reduction, classification, or the establishment of complex deep learning networks, the proposed model achieves outstanding results with the experimental dataset. This demonstrates that the fusion and diagnosis of multimodal data can be effectively achieved without dimension reduction or intricate deep learning networks. We take pride in exploring unstructured multimodal medical data using deep learning and hope to make breakthroughs in various fields. Full article

Nanoparticles have found widespread application across diverse fields, including agriculture and animal husbandry. However, a persistent challenge in laboratory-based studies involving nanoparticle exposure is the limited availability of experimental data, which constrains the robustness and generalizability of findings. This study presents a comprehensive analysis of the impact of zinc oxide nanoparticles (ZnO NPs) in feed on elemental homeostasis in male Wistar rats. Using correlation-based network analysis, a correlation graph weight value of 15.44 and a newly proposed weighted importance score of 1.319 were calculated, indicating that a dose of 3.1 mg/kg represents an optimal balance between efficacy and physiological stability. To address the issue of limited sample size, synthetic data generation was performed using generative adversarial networks, enabling data augmentation while preserving the statistical characteristics of the original dataset. Machine learning models based on fully connected neural networks and kernel ridge regression, enhanced with a custom loss function, were developed and evaluated. These models demonstrated strong predictive performance across a ZnO NP concentration range of 1–150 mg/kg, accurately capturing the dependencies of essential element, protein, and enzyme levels in blood on nanoparticle dosage. Notably, the presence of toxic elements and some other elements at ultra-low concentrations exhibited non-random patterns, suggesting potential systemic responses or early indicators of nanoparticle-induced perturbations and probable inability of synthetic data to capture the true dynamics. The integration of machine learning with synthetic data expansion provides a promising approach for analyzing complex biological responses in data-scarce experimental settings, contributing to the safer and more effective application of nanoparticles in animal nutrition. Full article

This paper introduces a novel framework for Distributionally Robust Bayesian Optimization (DRBO) with continuous context that integrates optimal transport theory and entropic regularization. We propose the sampling from the Wasserstein Barycenter Bayesian Optimization (SWBBO) method to deal with uncertainty about the context; that is, the unknown stochastic component affecting the observations of the black-box objective function. This approach captures the geometric structure of the underlying distributional uncertainty and enables robust acquisition strategies without incurring excessive computational costs. The method incorporates adaptive robustness scheduling, Lipschitz regularization, and efficient barycenter construction to balance exploration and exploitation. Theoretical analysis establishes convergence guarantees for the robust Bayesian Optimization acquisition function. Empirical evaluations on standard global optimization problems and real-life inspired benchmarks demonstrate that SWBBO consistently achieves faster convergence, good final regret, and greater stability than other recently proposed methods for DRBO with continuous context. Indeed, SWBBO outperforms all of them in terms of both optimization performance and robustness under repeated evaluations. Full article

Recent advancements in generative AI have fostered the development of highly adept Large Language Models (LLMs) that integrate diverse data types to empower decision-making. Among these, multimodal retrieval-augmented generation (RAG) applications are promising because they combine the strengths of information retrieval and generative models, enhancing their utility across various domains, including clinical use cases. This paper introduces AlzheimerRAG, a multimodal RAG application for clinical use cases, primarily focusing on Alzheimer’s disease case studies from PubMed articles. This application incorporates cross-modal attention fusion techniques to integrate textual and visual data processing by efficiently indexing and accessing vast amounts of biomedical literature. Our experimental results, compared to benchmarks such as BioASQ and PubMedQA, yield improved performance in the retrieval and synthesis of domain-specific information. We also present a case study using our multimodal RAG in various Alzheimer’s clinical scenarios. We infer that AlzheimerRAG can generate responses with accuracy non-inferior to humans and with low rates of hallucination. Full article

Accurate and high-resolution spatio-temporal prediction of PM_2.5 concentrations remains a significant challenge for air pollution early warning and prevention. Advanced artificial intelligence (AI) technologies, however, offer promising solutions to this problem. A spatio-temporal prediction model is designed in this study, which is built upon a seq2seq architecture. This model employs an improved graph convolutional neural network to capture spatially dependent features, integrates time-series information through a gated recurrent unit, and incorporates an attention mechanism to achieve PM_2.5 concentration prediction. Benefiting from high-resolution satellite remote sensing data, the regional, multi-step and high-resolution prediction of PM_2.5 concentration in Beijing has been performed. To validate the model’s performance, ablation experiments are conducted, and the model is compared with other advanced prediction models. The experimental results show our proposed Spatio-Temporal Graph Convolutional Network with Attention Mechanism (STGCA) outperforms comparison models in multi-step forecasting, achieving root mean squared error (RMSE), mean absolute error (MAE) and mean absolute percentage error (MAPE) of 4.21, 3.11 and 11.41% for the first step, respectively. For subsequent steps, the model also shows significant improvements. For subsequent steps, the model also shows significant improvements, with RMSE, MAE and MAPE values of 5.08, 3.69 and 13.34% for the second step and 6.54, 4.61 and 16.62% for the third step, respectively. Additionally, STGCA achieves the index of agreement (IA) values of 0.98, 0.97 and 0.95, as well as Theil’s inequality coefficient (TIC) values of 0.06, 0.08 and 0.10 proving its superiority. These results demonstrate that the proposed model offers an efficient technical approach for smart air pollution forecasting and warning in the future. Full article

The data landscape has changed, as more and more information is produced in the form of continuous data streams instead of stationary datasets. In this context, several online machine learning techniques have been proposed with the aim of automatically adapting to changes in data distributions, known as drifts. Though effective in certain scenarios, contemporary techniques do not generalize well to different types of data, while they also require manual parameter tuning, thus significantly hindering their applicability. Moreover, current methods do not thoroughly address drifts, as they mostly focus on concept drifts (distribution shifts on the target variable) and not on data drifts (changes in feature distributions). To confront these challenges, in this paper, we propose an AutoML Pipeline for Streams (AML4S), which automates the choice of preprocessing techniques, the choice of machine learning models, and the tuning of hyperparameters. Our pipeline further includes a drift detection mechanism that identifies different types of drifts, therefore continuously adapting the underlying models. We assess our pipeline on several real and synthetic data streams, including a data stream that we crafted to focus on data drifts. Our results indicate that AML4S produces robust pipelines and outperforms existing online learning or AutoML algorithms. Full article

Novel techniques in evolutionary optimization, simulation, and machine learning enable a broad analysis of domains like fluid dynamics, in which computation is expensive and flow behavior is complex. This paper introduces the concept of full domain analysis, defined as the ability to efficiently determine the full space of solutions in a problem domain and analyze the behavior of those solutions in an accessible and interactive manner. The goal of full domain analysis is to deepen our understanding of domains by generating many examples of flow, their diversification, optimization, and analysis. We define a formal model for full domain analysis, its current state of the art, and the requirements of its sub-components. Finally, an example is given to show what can be learned by using full domain analysis. Full domain analysis, rooted in optimization and machine learning, can be a valuable tool in understanding complex systems in computational physics and beyond. Full article

This paper examines the integration of machine learning (ML) techniques in professional football, focusing on two key areas: (i) player and team performance, and (ii) match outcome prediction. Using a systematic methodology, this study reviews 172 papers from a five-year observation period (2019–2024) to identify relevant applications, focusing on the analysis of game actions (free kicks, passes, and penalties), individual and collective performance, and player position. A predominance of supervised learning, deep learning, and hybrid models (which integrate several ML techniques) is observed in the ML categories. Among the most widely used algorithms are decision trees, extreme gradient boosting, and artificial neural networks, which focus on optimizing sports performance and predicting outcomes. This paper discusses challenges such as the limited availability of public datasets due to access and cost restrictions, the restricted use of advanced visualization tools, and the poor integration of data acquisition devices, such as sensors. However, it also highlights the role of ML in addressing these challenges, thereby representing future research opportunities. Furthermore, this paper includes two illustrative case studies: (i) predicting the date Cristiano Ronaldo will reach 1000 goals, and (ii) an example of predicting penalty shoots; these examples demonstrate the practical potential of ML for performance monitoring and tactical decision-making in real-world football environments. Full article

Accurate segmentation of deep brain structures is critical for preoperative planning in such neurosurgical procedures as Deep Brain Stimulation (DBS). Previous research has showcased successful pipelines for segmentation from T1-weighted (T1w) Magnetic Resonance Imaging (MRI) data. Nevertheless, the role of T2-weighted (T2w) MRI data has been underexploited so far. This study proposes and evaluates a fully automated deep learning pipeline based on nnU-Net for the segmentation of eight clinically relevant deep brain structures. A heterogeneous dataset has been prepared by gathering 325 paired T1w and T2w MRI scans from eight publicly available sources, which have been annotated by means of an atlas-based registration approach. Three 3D nnU-Net models—unimodal T1w, unimodal T2w, and multimodal (encompassing both T1w and T2w)—have been trained and compared by using 5-fold cross-validation and a separate test set. The outcomes prove that the multimodal model consistently outperforms the T2w unimodal model and achieves comparable performance with the T1w unimodal model. On our dataset, all proposed models significantly exceed the performance of the state-of-the-art DBSegment tool. These findings underscore the value of multimodal MRI in enhancing deep brain segmentation and offer a robust framework for accurate delineation of subcortical targets in both research and clinical settings. Full article

Interactions between human behavior, legal regulations, and monitoring technology in road traffic systems provide an everyday example of complex biosocial–technical systems. In this paper, a study is reported that investigated the potential for a thrifty world model to predict consequences from choices about road traffic system design. Colloquially, the term thrifty means economical. In physics, the term thrifty is related to the principle of least action. Predictions were made with algebraic machine learning, which combines predefined embeddings with ongoing learning from data. The thrifty world model comprises three categories that encompass a total of only eight system design choice options. Results indicate that the thrifty world model is sufficient to encompass biosocial–technical complexity in predictions of where and when it is most likely that accidents will occur. Overall, it is argued that thrifty world models can provide a practical alternative to large photo-realistic world models, which can contribute to explainable artificial intelligence (AI) and to frugal AI. Full article

According to a recent experimental phenomenology–information processing theory, the sensory strength, or vividness, of visual mental images self-reported by human observers reflects the intensive variation in subjective time duration during the process of generation of said mental imagery. The primary objective of this study was to test the hypothesis that a biologically plausible essential multilayer perceptron (MLP) architecture can validly map the phenomenological categories of subjective time duration onto levels of subjectively self-reported vividness. A secondary objective was to explore whether this type of neural network cognitive modeling approach can give insight into plausible underlying large-scale brain dynamics. To achieve these objectives, vividness self-reports and reaction times from a previously collected database were reanalyzed using multilayered perceptron network models. The input layer consisted of six levels representing vividness self-reports and a reaction time cofactor. A single hidden layer consisted of three nodes representing the salience, task positive, and default mode networks. The output layer consisted of five levels representing Vittorio Benussi’s subjective time categories. Across different models of networks, Benussi’s subjective time categories (Level 1 = very brief, 2 = brief, 3 = present, 4 = long, 5 = very long) were predicted by visual imagery vividness level 1 (=no image) to 5 (=very vivid) with over 90% success in classification accuracy, precision, recall, and F1-score. This accuracy level was maintained after 5-fold cross validation. Linear regressions, Welch’s t-test for independent coefficients, and Pearson’s correlation analysis were applied to the resulting hidden node weight vectors, obtaining evidence for strong correlation and anticorrelation between nodes. This study successfully mapped Benussi’s five levels of subjective time categories onto the activation patterns of a simple MLP, providing a novel computational framework for experimental phenomenology. Our results revealed structured, complex dynamics between the task positive network (TPN), the default mode network (DMN), and the salience network (SN), suggesting that the neural mechanisms underlying temporal consciousness involve flexible network interactions beyond the traditional triple network model. Full article

Vision-based navigation is a common solution for the critical challenge of GPS-denied Unmanned Aerial Vehicle (UAV) operation, but a research gap remains in the autonomous discovery of robust landmarks from aerial survey imagery needed for such systems. In this work, we propose a framework to fill this gap by identifying visually distinctive urban buildings from aerial survey imagery and curating them into a landmark database for GPS-free UAV localization. The proposed framework constructs semantically rich embeddings using intermediate layers from a pre-trained YOLOv11n-seg segmentation network. This novel technique requires no additional training. An unsupervised landmark selection strategy, based on the Isolation Forest algorithm, then identifies objects with statistically unique embeddings. Experimental validation on the VPAIR aerial-to-aerial benchmark shows that the proposed max-pooled embeddings, assembled from selected layers, significantly improve retrieval performance. The top-1 retrieval accuracy for landmarks more than doubled compared to typical buildings (0.53 vs. 0.31), and a Recall@5 of 0.70 is achieved for landmarks. Overall, this study demonstrates that unsupervised outlier selection in a carefully constructed embedding space yields a highly discriminative, computation-friendly set of landmarks suitable for real-time, robust UAV navigation. Full article

Predicting undergraduate student success is critical for informing timely interventions and improving outcomes in higher education. This study leverages over a decade of historical data from Louisiana State University (LSU) to forecast graduation outcomes using advanced machine learning techniques, with a focus on convolutional autoencoders (CAEs). We detail the data processing and transformation steps, including feature selection and imputation, to construct a robust dataset. The CAE effectively extracts meaningful latent features, validated through low-dimensional t-SNE visualizations that reveal clear clusters based on class labels, differentiating students likely to graduate from those at risk. A two-year gap strategy is introduced to ensure rigorous evaluation and simulate real-world conditions by predicting outcomes on unseen future data. Our results demonstrate the promise of CAE-derived embeddings for dimensionality reduction and computational efficiency, with competitive performance in downstream classification tasks. While models trained on embeddings showed slightly reduced performance compared to raw input data, with accuracies of 83% and 85%, respectively, their compactness and computational efficiency highlight their potential for large-scale analyses. The study emphasizes the importance of rigorous preprocessing, feature engineering, and evaluation protocols. By combining these approaches, we provide actionable insights and adaptive modeling strategies to support robust and generalizable predictive systems, enabling educators and administrators to enhance student success initiatives in dynamic educational environments. Full article

Rust’s growing popularity in high-integrity systems requires automated vulnerability detection in order to maintain its strong safety guarantees. Although Rust’s ownership model and compile-time checks prevent many errors, sometimes unexpected bugs may occasionally pass analysis, underlining the necessity for automated safe and unsafe code detection. This paper presents Rust-IR-BERT, a machine learning approach to detect security vulnerabilities in Rust code by analyzing its compiled LLVM intermediate representation (IR) instead of the raw source code. This approach offers novelty by employing LLVM IR’s language-neutral, semantically rich representation of the program, facilitating robust detection by capturing core data and control-flow semantics and reducing language-specific syntactic noise. Our method leverages a graph-based transformer model, GraphCodeBERT, which is a transformer architecture pretrained model to encode structural code semantics via data-flow information, followed by a gradient boosting classifier, CatBoost, that is capable of handling complex feature interactions—to classify code as vulnerable or safe. The model was evaluated using a carefully curated dataset of over 2300 real-world Rust code samples (vulnerable and non-vulnerable Rust code snippets) from RustSec and OSV advisory databases, compiled to LLVM IR and labeled with corresponding Common Vulnerabilities and Exposures (CVEs) identifiers to ensure comprehensive and realistic coverage. Rust-IR-BERT achieved an overall accuracy of 98.11%, with a recall of 99.31% for safe code and 93.67% for vulnerable code. Despite these promising results, this study acknowledges potential limitations such as focusing primarily on known CVEs. Built on a representative dataset spanning over 2300 real-world Rust samples from diverse crates, Rust-IR-BERT delivers consistently strong performance. Looking ahead, practical deployment could take the form of a Cargo plugin or pre-commit hook that automatically generates and scans LLVM IR artifacts during the development cycle, enabling developers to catch vulnerabilities at an early stage in the development cycle. Full article

In this study, we introduce an enhanced hybrid Autoencoder–Dense–Transformer Neural Network (AE-DTNN) model for developing an effective intrusion detection system (IDS) aimed at improving the performance and robustness of threat detection strategies within a rapidly changing and increasingly complex network landscape. The Autoencoder component restructures network traffic data, while a stack of Dense layers performs feature extraction to generate more meaningful representations. The Transformer network then facilitates highly precise and comprehensive classification. Our strategy incorporates adaptive synthetic sampling (ADASYN) for both binary and multi-class classification tasks, complemented by the edited nearest neighbors (ENN) technique and the use of class weights to mitigate class imbalance issues. In experiments conducted on the NF-BoT-IoT-v2 dataset, the AE-DTNN-based IDS achieved outstanding performance, with 99.98% accuracy in binary classification and 98.30% in multi-class classification. On the NSL-KDD dataset, the model reached 98.57% accuracy for binary classification and 97.50% for multi-class classification. Additionally, the model attained 99.92% and 99.78% accuracy in binary and multi-class classification, respectively, on the CSE-CIC-IDS2018 dataset. These results demonstrate the exceptional effectiveness of the proposed model in contrast to conventional approaches, highlighting its strong potential to detect a broad range of network intrusions with high reliability. Full article