Machine Learning and Knowledge Extraction

The reciprocal relationship between Large Language Models (LLMs) and Knowledge Graphs (KGs) highlights their synergistic potential in enhancing artificial intelligence (AI) applications. LLMs, with their natural language understanding and generative capabilities, support the automation of KG construction through entity recognition, relation extraction, and schema generation. Conversely, KGs serve as structured and interpretable data sources that improve the transparency, factual consistency and reliability of LLM-based applications, mitigating challenges such as hallucinations and lack of explainability. This study conducts a systematic literature review of 77 studies to examine AI methodologies supporting LLM–KG integration, including symbolic AI, machine learning, and hybrid approaches. The research explores diverse applications spanning healthcare, finance, justice, and industrial automation, revealing the transformative potential of this synergy. Through in-depth analysis, this study identifies key limitations in current approaches, including challenges in scalability with maintaining dynamic and real-time Knowledge Graphs, difficulty in adapting general-purpose LLMs to specialized domains, limited explainability in tracing model outputs to interpretable reasoning, and ethical concerns surrounding bias, fairness, and transparency. In response, the study highlights potential strategies to optimize LLM–KG synergy. The findings from this study provide actionable insights for researchers and practitioners aiming for robust, transparent, and adaptive AI systems to enhance knowledge-driven AI applications through LLM–KG integration, further advancing generative AI and explainable AI (XAI) applications. Full article

Particle tracking velocimetry (PTV) forms the basis for many fluid dynamic experiments, in which individual particles are tracked across multiple successive images. However, when the experimental setup involves high-speed, high-density particles that are indistinguishable and follow complex or unknown flow fields, matching particles between images becomes significantly more challenging. Reliable PTV algorithms are crucial in such scenarios. Previous work has demonstrated that the Self-Organizing Map (SOM) machine learning approach offers superior outcomes on complex-plasma data compared with traditional methods, though its performance is sensitive to hyperparameter calibration, which requires optimization for specific flow scenarios. In this article, we describe how the dependence of the various hyperparameters on different flow scenarios was studied and the optimal settings for diverse flow conditions were identified. Based on these results, automatic hyperparameter calibration was implemented in the PTV framework. Furthermore, the SOM’s performance was directly compared with that of the preceding conventional PTV method, Trackpy, for complex plasmas using synthetic data. Finally, as a new approach to identifying incorrectly matched particle traces, a Long Short-Term Memory (LSTM) neural network was developed to sort out all inaccuracies to further improve the outcome. Combined with automatic hyperparameter calibration, outlier detection and additional computational speed optimization, this work delivers a robust, versatile and efficient framework for PTV analysis. Full article

We used machine learning (ML) to characterize 894,154 medical records of outpatient visits from the Veterans Administration Central Data Warehouse (VA CDW) by the likelihood of assignment of 200 International Classification of Diseases (ICD) code blocks. Using four different predictive models, we found the ML-derived predictions for the code blocks were consistently more effective in predicting death or 90-day rehospitalization than the assigned code block in the record. We reviewed records of ICD chapter assignments. The review revealed that the ML-predicted chapter assignments were consistently better than those humanly assigned. Impact factor analysis, a method of explanation of AI findings that was developed in our group, demonstrated little effect on any one assigned ICD code block but a marked impact on the ML-derived code blocks of kidney disease as well as several other morbidities. In this study, machine learning was much better than human code assignment at predicting the relatively rare outcomes of death or rehospitalization. Future work will address generalizability using other datasets, as well as addressing coding that is more nuanced than that of the categorization provided by code blocks. Full article

Large language models (LLMs) have recently demonstrated notable performance, particularly in addressing the challenge of extensive data requirements when training traditional forecasting models. However, these methods encounter significant challenges when applied to high-dimensional and domain-specific datasets. These challenges primarily arise from inability to effectively model inter-variable dependencies and capture variable-specific characteristics, leading to suboptimal performance in complex forecasting scenarios. To address these limitations, we propose ADTime, an adaptive LLM-based approach for multivariate time series forecasting. ADTime employs advanced preprocessing techniques to identify latent relationships among key variables and temporal features. Additionally, it integrates temporal alignment mechanisms and prompt-based strategies to enhance the semantic understanding of forecasting tasks by LLMs. Experimental results show that ADTime outperforms state-of-the-art methods, reducing MSE by 9.5% and MAE by 6.1% on public datasets, and by 17.1% and 13.5% on domain-specific datasets. Furthermore, zero-shot experiments on real-world refinery datasets demonstrate that ADTime exhibits stronger generalization capabilities across various transfer scenarios. These findings highlight the potential of ADTime in advancing complex, domain-specific time series forecasting tasks. Full article

In electron microscopy, obtaining low-noise images is often difficult, especially when examining biological samples or delicate materials. Therefore, the suppression of noise is essential for the analysis of such noisy images. State-of-the-art image denoising methods are dominated by supervised Convolution neural network (CNN)-based methods. However, supervised CNNs cannot be used if a noise-free ground truth is unavailable. To address this problem, we propose a method that uses re-occurring patterns in images. Our proposed method does not require noise-free images for the denoising task. Instead, it is based on the idea that averaging images with the same signal having independent noise suppresses the overall noise. In order to evaluate the performance of our method, we compare our results with other state-of-the-art denoising methods that do not require a noise-free image. We show that our method is the best for retaining fine image structures. Additionally, we develop a confidence map for evaluating the denoising quality of the proposed method. Furthermore, we analyze the time complexity of the algorithm to ensure scalability and optimize the algorithm to improve the runtime efficiency. Full article

Regression is a fundamental task in machine learning, and neural networks have been successfully employed in many applications to identify underlying regression patterns. However, they are often criticised for their lack of interpretability and commonly referred to as black-box models. Feature selection approaches address this challenge by simplifying datasets through the removal of unimportant features, while improving explainability by revealing feature importance. In this work, we leverage mathematical programming to identify the most important features in a trained deep neural network with a ReLU activation function, providing greater insight into its decision-making process. Unlike traditional feature selection methods, our approach adjusts the weights and biases of the trained neural network via a Mixed-Integer Linear Programming (MILP) model to identify the most important features and thereby uncover underlying relationships. The mathematical formulation is reported, which determines the subset of selected features, and clustering is applied to reduce the complexity of the model. Our results illustrate improved performance in the neural network when feature selection is implemented by the proposed approach, as compared to other feature selection approaches. Finally, analysis of feature selection frequency across each dataset reveals feature contribution in model predictions, thereby addressing the black-box nature of the neural network. Full article

Due to their considerable costs, deep neural networks (DNNs) are valuable assets that need to be protected in terms of intellectual property (IP). From this statement, DNN watermarking gains significant interest since it allows DNN owners to prove their ownership. Various methods that embed ownership information in the model behavior have been proposed. They need to fill several requirements, among them the security, which represents an attacker’s difficulty in breaking the watermarking scheme. There is also the robustness requirement, which quantifies the resistance against watermark removal techniques. The problem is that the proposed methods generally fail to meet these necessary standards. This paper presents RoSe-Mix, a robust and secure deep neural network watermarking technique designed for black-box settings. It addresses limitations in existing DNN watermarking approaches by integrating key features from two established methods: RoSe, which uses cryptographic hashing to ensure security, and Mixer, which employs image Mixup to enhance robustness. Experimental results demonstrate that RoSe-Mix achieves security across various architectures and datasets with a robustness to removal attacks exceeding 99%. Full article

This paper explores the potential use of Large Language Models (LLMs), such as ChatGPT, Google Gemini, and Microsoft Copilot, in threat hunting, specifically focusing on Living off the Land (LotL) techniques. LotL methods allow threat actors to blend into regular network activity, which makes detection by automated security systems challenging. The study seeks to determine whether LLMs can reliably generate effective queries for security tools, enabling organisations with limited budgets and expertise to conduct threat hunting. A testing environment was created to simulate LotL techniques, and LLM-generated queries were used to identify malicious activity. The results demonstrate that LLMs do not consistently produce accurate or reliable queries for detecting these techniques, particularly for users with varying skill levels. However, while LLMs may not be suitable as standalone tools for threat hunting, they can still serve as supportive resources within a broader security strategy. These findings suggest that, although LLMs offer potential, they should not be relied upon for accurate results in threat detection and require further refinement to be effectively integrated into cybersecurity workflows. Full article

In today’s rapidly evolving transportation infrastructure, developing long-lasting, high-performance pavement materials remains a significant priority. Integrating machine learning (ML) techniques provides a transformative approach to optimizing asphalt mix design and performance prediction. This study investigates the use of waste plastics, including Polyethylene Terephthalate (PET), High-Density Polyethylene (HDPE), and Polyvinyl Chloride (PVC), as modifiers in asphalt concrete to enhance durability and mechanical performance. A predictive modeling approach was employed to estimate the bulk-specific gravity (G_mb) of asphalt concrete using various ML techniques, including Artificial Neural Networks (ANNs), Support Vector Machines (SVMs), Gaussian Processes (GPs), and Reduced Error Pruning (REP) Tree. The accuracy of each model was evaluated using statistical performance metrics, including the correlation coefficient (CC), scatter index (SI), mean absolute error (MAE), and root mean square error (RMSE). The results demonstrate that the ANN model outperformed all other ML techniques, achieving the highest correlation (CC = 0.9996 for training, 0.9999 for testing) and the lowest error values (MAE = 0.0004, RMSE = 0.0006, SI = 0.00026). A comparative analysis between actual and predicted G_mb values confirmed the reliability of the proposed ANN model, with minimal error margins and superior accuracy. Additionally, sensitivity analysis identified bitumen content (BC) and volume of bitumen (V_b) as the most influential parameters affecting G_mb, emphasizing the need for precise parameter optimization in asphalt mix design. This study demonstrates the effectiveness of machine learning-driven predictive modeling in optimizing sustainable asphalt mix design, offering a cost-effective, time-efficient, and highly accurate alternative to traditional experimental methods. Full article

Waste heat recovery is a critical strategy for optimizing energy consumption and reducing greenhouse gas emissions. In this context, the circular economy highlights the importance of this practice as a key tool to enhance energy efficiency, minimize waste, and decrease environmental impact. Artificial neural networks are particularly well-suited for managing nonlinearities and complex interactions among multiple variables, making them ideal for controlling a double-stage absorption heat transformer. This study aims to simultaneously optimize both user-defined parameters. Levenberg–Marquardt and scaled conjugated gradient algorithms were compared from five to twenty-five neurons to determine the optimal operating conditions while the coefficient of performance and the gross temperature lift were simultaneously maximized. The methodology includes R2024a MATLAB© programming, real-time data acquisition, visual engineering environment software, and flow control hardware. The results show that applying the Levenberg–Marquardt algorithm resulted in an increase in the correlation coefficient (R) at 20 neurons, improving the thermodynamic performance and enabling greater energy recovery from waste heat. Full article

As climate change transforms our environment and human intrusion into natural ecosystems escalates, there is a growing demand for disease spread models to forecast and plan for the next zoonotic disease outbreak. Accurate parametrization of these models requires data from diverse sources, including the scientific literature. Despite the abundance of scientific publications, the manual extraction of these data via systematic literature reviews remains a significant bottleneck, requiring extensive time and resources, and is susceptible to human error. This study examines the application of a large language model (LLM) as an assessor for screening prioritisation in climate-sensitive zoonotic disease research. By framing the selection criteria of articles as a question–answer task and utilising zero-shot chain-of-thought prompting, the proposed method achieves a saving of at least 70% work effort compared to manual screening at a recall level of 95% (NWSS@95%). This was validated across four datasets containing four distinct zoonotic diseases and a critical climate variable (rainfall). The approach additionally produces explainable AI rationales for each ranked article. The effectiveness of the approach across multiple diseases demonstrates the potential for broad application in systematic literature reviews. The substantial reduction in screening effort, along with the provision of explainable AI rationales, marks an important step toward automated parameter extraction from the scientific literature. Full article

(1) Background: This paper intends to accomplish a comparative study and analysis regarding the multiclass classification of facial thermal images, i.e., in three classes corresponding to predefined emotional states (neutral, happy and sad). By carrying out a comparative analysis, the main goal of the paper consists in identifying a suitable algorithm from machine learning field, which has the highest accuracy (ACC). Two categories of images were used in the process, i.e., images with Gaussian noise and images with “salt and pepper” type noise that come from two built-in special databases. An augmentation process was applied to the initial raw images that led to the development of the two databases with added noise, as well as the subsequent augmentation of all images, i.e., rotation, reflection, translation and scaling. (2) Methods: The multiclass classification process was implemented through two subsets of methods, i.e., machine learning with random forest (RF), support vector machines (SVM) and k-nearest neighbor (KNN) algorithms and deep learning with the convolutional neural network (CNN) algorithm. (3) Results: The results obtained in this paper with the two subsets of methods belonging to the field of artificial intelligence (AI), together with the two categories of facial thermal images with added noise used as input, were very good, showing a classification accuracy of over 99% for the two categories of images, and the three corresponding classes for each. (4) Discussion: The augmented databases and the additional configurations of the implemented algorithms seems to have had a positive effect on the final classification results. Full article

Equatorial plasma bubbles (EPBs) are regions of depleted electron density that form in the Earth’s ionosphere due to Rayleigh–Taylor instability. These bubbles can cause signal scintillation, leading to signal loss and errors in position calculations. EPBs can be detected in images captured by All-Sky Imager (ASI) systems. This study proposes a low-cost automatic detection method for EPBs in ASI data that can be used for both real-time detection and classification purposes. This method utilizes Two-Dimensional Principal Component Analysis (2DPCA) with Recursive Feature Elimination (RFE), in conjunction with a Random Forest machine learning model, to create an Explainable Artificial Intelligence (XAI) model capable of extracting image features to automatically detect EPBs with the lowest possible dimensionality. This led to having a small-sized and extremely fast-trained model that could be used to identify EPBs within the captured ASI images. A set of 2458 images, classified into two categories—Event and Empty—were used to build the database. This database was randomly split into two subsets: a training dataset (80%) and a testing dataset (20%). The produced XAI model demonstrated slightly higher detection accuracy compared to the standard 2DPCA model while being significantly smaller in size. Furthermore, the proposed model’s performance has been evaluated and compared with other deep learning baseline models (ResNet18, Inception-V3, VGG16, and VGG19) in the same environment. Full article

Non-negative Matrix Factorization (NMF) has gained popularity due to its effectiveness in clustering and feature selection tasks. It is particularly valuable for managing high-dimensional data by reducing dimensionality and providing meaningful semantic representations. However, traditional NMF methods may encounter challenges when dealing with noisy data, outliers, or when the underlying manifold structure of the data is overlooked. This paper introduces an innovative approach called SGRiT, which employs Stiefel manifold optimization to enhance the extraction of latent features. These learned features have been shown to be highly informative for clustering tasks. The method leverages a spectral decomposition criterion to obtain a low-dimensional embedding that captures the intrinsic geometric structure of the data. Additionally, this paper presents a solution for addressing the Stiefel manifold problem and utilizes a Riemannian-based trust region algorithm to optimize the loss function. The outcome of this optimization process is a new representation of the data in a transformed space, which can subsequently serve as input for the NMF algorithm. Furthermore, this paper incorporates a novel subspace graph regularization term that considers high-order geometric information and introduces a sparsity term for the factor matrices. These enhancements significantly improve the discrimination capabilities of the learning process. This paper conducts an impartial analysis of several essential NMF algorithms. To demonstrate that the proposed approach consistently outperforms other benchmark algorithms, four clustering evaluation indices are employed. Full article

The electrical storm optimization (ESO) algorithm, inspired by the dynamic nature of electrical storms, is a novel population-based metaheuristic that employs three dynamically adjusted parameters: field resistance, field intensity, and field conductivity. Field resistance assesses the spread of solutions within the search space, reflecting strategy diversity. The field intensity balances the exploration of new territories and the exploitation of promising areas. The field conductivity adjusts the adaptability of the search process, enhancing the algorithm’s ability to escape local optima and converge on global solutions. These adjustments enable the ESO to adapt in real-time to various optimization scenarios, steering the search toward potential optima. ESO’s performance was rigorously tested against 60 benchmark problems from the IEEE CEC SOBC 2022 suite and 20 well-known metaheuristics. The results demonstrate the superior performance of ESOs, particularly in tasks requiring a nuanced balance between exploration and exploitation. Its efficacy is further validated through successful applications in four engineering domains, highlighting its precision, stability, flexibility, and efficiency. Additionally, the algorithm’s computational costs were evaluated in terms of the number of function evaluations and computational overhead, reinforcing its status as a standout choice in the metaheuristic field. Full article

This study investigates the integration of diverse data modalities within deep learning ensembles for Android malware classification. Android applications can be represented as binary images and function call graphs, each offering complementary perspectives on the executable. We synthesise these modalities by combining predictions from convolutional and graph neural networks with a multilayer perceptron. Empirical results demonstrate that multimodal models outperform their unimodal counterparts while remaining highly efficient. For instance, integrating a plain CNN with 83.1% accuracy and a GCN with 80.6% accuracy boosts overall accuracy to 88.3%. DenseNet-GIN achieves 90.6% accuracy, with no further improvement obtained by expanding this ensemble to four models. Based on our findings, we advocate for the flexible development of modalities to capture distinct aspects of applications and for the design of algorithms that effectively integrate this information. Full article

We introduce a weakly supervised segmentation approach that leverages class activation maps and the Segment Anything Model to generate high-quality masks using only classification data. A pre-trained classifier produces class activation maps that, once thresholded, yield bounding boxes encapsulating the regions of interest. These boxes prompt the SAM to generate detailed segmentation masks, which are then refined by selecting the best overlap with automatically generated masks from the foundational model using the intersection over union metric. In a polyp segmentation case study, our approach outperforms existing zero-shot and weakly supervised methods, achieving a mean intersection over union of 0.63. This method offers an efficient and general solution for image segmentation tasks where segmentation data are scarce. Full article

The growing sophistication of cyber threats necessitates robust and interpretable intrusion detection systems (IDS) to safeguard network security. While machine learning models such as Decision Tree (DT), Random Forest (RF), k-Nearest Neighbors (K-NN), and XGBoost demonstrate high effectiveness in detecting malicious activities, their interpretability decreases as their complexity and accuracy increase, posing challenges for critical cybersecurity applications. Local Interpretable Model-agnostic Explanations (LIME) is widely used to address this limitation; however, its reliance on normal distribution for perturbations often fails to capture the non-linear and imbalanced characteristics of datasets like CIC-IDS-2018. To address these challenges, we propose a modified LIME perturbation strategy using Weibull, Gamma, Beta, and Pareto distributions to better capture the characteristics of network traffic data. Our methodology improves the stability of different ML models trained on CIC-IDS datasets, enabling more meaningful and reliable explanations of model predictions. The proposed modifications allow for an increase in explanation fidelity by up to 78% compared to the default Gaussian approach. Pareto-based perturbations provide the best results. Among all distributions tested, Pareto consistently yielded the highest explanation fidelity and stability, particularly for K-NN ($R^2$ = 0.9971, S = 0.9907) and DT ($R^2$ = 0.9267, S = 0.9797). This indicates that heavy-tailed distributions fit well with real-world network traffic patterns, reducing the variance in attribute importance explanations and making them more robust. Full article

This research paper presents novel approaches for detecting credit card risk through the utilization of Long Short-Term Memory (LSTM) networks and XGBoost algorithms. Facing the challenge of securing credit card transactions, this study explores the potential of LSTM networks for their ability to understand sequential dependencies in transaction data. This research sheds light on which model is more effective in addressing the challenges posed by imbalanced datasets in credit risk assessment. The methodology utilized for imbalanced datasets includes the use of the Synthetic Minority Oversampling Technique (SMOTE) to address any imbalance in class distribution. This paper conducts an extensive literature review, comparing various machine learning methods, and proposes an innovative framework that compares LSTM with XGBoost to improve fraud detection accuracy. LSTM, a recurrent neural network renowned for its ability to capture temporal dependencies within sequences of transactions, is compared with XGBoost, a formidable ensemble learning algorithm that enhances feature-based classification. By meticulously carrying out preprocessing tasks, constructing competent training models, and implementing ensemble techniques, our proposed framework demonstrates unwavering performance in accurately identifying fraudulent transactions. The comparison of LSTM and XGBoost shows that LSTM is more effective for our imbalanced dataset. Compared with XGBOOST’s 97% accuracy, LSTM’s accuracy is 99%. The final result emphasizes how crucial it is to select the optimal algorithm based on particular criteria within financial concerns, which will ultimately result in more reliable and knowledgeable credit score decisions. Full article