Search Results (367)

Domain shift poses a major challenge in many Machine Learning applications due to variations in data acquisition protocols, particularly in the medical field. Test-time augmentation (TTA) can solve the domain shift issue and improve robustness by aggregating predictions from multiple augmented versions of the same input. However, TTA may inadvertently generate unrealistic or Out-of-Distribution (OOD) samples that negatively affect prediction quality. In this work, we introduce a filtering procedure that removes from the TTA images all the OOD samples whose representations lie far from the training data distribution. Moreover, all the retained TTA images are weighted inversely to their distance from the training data. The final prediction is provided by a Self-Ensemble with Confidence, which is a lightweight ensemble strategy that fuses predictions from the original and retained TTA samples using a weighted soft voting scheme, without requiring multiple models or retraining. This method is model-agnostic and can be integrated with any deep learning architecture, making it broadly applicable across various domains. Experiments on cross-domain leukocyte classification benchmarks demonstrate that our method consistently improves over standard TTA and Baseline inference, particularly when strong domain shifts are present. Ablation studies and statistical tests confirm the effectiveness and significance of each component. Full article

Cyberattacks include Structured Query Language Injection (SQLi), which represents threats at the level of web applications that interact with the database. These attacks are carried out by executing SQL commands, which compromise the integrity and confidentiality of the data. In this paper, a machine learning (ML)-based model is proposed for identifying SQLi attacks. The authors propose a two-stage personalized software processing pipeline as a novel element. Although individual techniques are known, their structured combination and application in this context represent a novel approach to transforming raw SQL queries into input features for an ML model. In this research, a dataset consisting of 90,000 SQL queries was constructed, comprising 17,695 legitimate and 72,304 malicious queries. The dataset consists of synthetic data generated using the GPT-4o model and data from a publicly available dataset. These were processed within a pipeline proposed by the authors, consisting of two stages: syntactic normalization and the extraction of the eight semantic features for model training. Also, within the research, several ML models were analyzed using the Azure Machine Learning Studio platform. These models were paired with different sampling algorithms for selecting the training set and the validation set. Out of the 15 training-sampling algorithm combinations, the Voting Ensemble model achieved the best performance. It achieved an accuracy of 96.86%, a weighted AUC of 98.25%, a weighted F1-score of 96.77%, a weighted precision of 96.92%, and a Matthews correlation coefficient of 89.89%. These values demonstrate the model’s ability to classify queries as legitimate or malicious. The attack identification rate was only 15 malicious queries missed out of a total of 7200, and the number of false alarms was 211 cases. The results confirm the possibility of integrating this algorithm into an additional security layer within an existing web application architecture. In practice, the authors suggest adding an extra layer of security using synthetic data. Full article

Background: Accurate differentiation of Parkinson’s disease (PD) from healthy aging is crucial for timely intervention and effective management. Postural sway abnormalities are prominent motor features of PD. Quantitative stabilometry and machine learning (ML) offer a promising avenue for developing objective markers to support the diagnostic process. This study aimed to develop and validate high-performance ML models to classify individuals with PD and age-matched healthy older adults (HOAs) using a comprehensive set of stabilometric parameters. Methods: Thirty-seven HOAs (mean age 70 ± 6.8 years) and 26 individuals with idiopathic PD (Hoehn and Yahr stages 2–3, on medication; mean age 66 years ± 2.9 years), all aged 60–80 years, participated. Stabilometric data were collected using a force platform during quiet stance under eyes-open (EO) and eyes-closed (EC) conditions, from which 34 parameters reflecting the time- and frequency-domain characteristics of center-of-pressure (COP) sway were extracted. After data preprocessing, including mean imputation for missing values and feature scaling, three ML classifiers (Random Forest, Gradient Boosting, and Support Vector Machine) were hyperparameter-tuned using GridSearchCV with three-fold cross-validation. An ensemble voting classifier (soft voting) was constructed from these tuned models. Model performance was rigorously evaluated using 15 iterations of stratified train–test splits (70% train and 30% test) and an additional bootstrap procedure of 1000 iterations to derive reliable 95% confidence intervals (CIs). Results: Our optimized ensemble voting classifier achieved excellent discriminative power, distinguishing PD from HOAs with a mean accuracy of 0.91 (95% CI: 0.81–1.00) and a mean Area Under the ROC Curve (AUC ROC) of 0.97 (95% CI: 0.92–1.00). Importantly, feature analysis revealed that anteroposterior sway velocity with eyes open (V-AP) and total sway path with eyes closed (TOD_EC, calculated using COP displacement vectors from its mean position) are the most robust and non-invasive biomarkers for differentiating the groups. Conclusions: An ensemble ML approach leveraging stabilometric features provides a highly accurate, non-invasive method to distinguish PD from healthy aging and may augment clinical assessment and monitoring. Full article

Soft robots demonstrate significant advantages in applications within complex environments due to their unique material properties and structural designs. However, they also face challenges in fault diagnosis, such as nonlinearity, time variability, and the difficulty of precise modeling. To address these issues, this paper proposes a fault diagnosis method based on multimodal spatiotemporal features and ensemble learning. First, a sliding-window Kalman filter is utilized to eliminate noise interference from multi-source signals, constructing separate temporal and spatial representation spaces. Subsequently, an adaptive weight strategy for feature fusion is applied to train a heterogeneous decision tree model, followed by a dynamic weighted voting mechanism based on confidence levels to obtain diagnostic results. This method optimizes the feature extraction and fusion process in stages, combined with a dynamic ensemble strategy. Experimental results indicate a significant improvement in diagnostic accuracy and model robustness, achieving precise identification of faults in soft robots. Full article

Myocardial infarction (MI) remains a leading cause of in-hospital mortality. Early identification of high-risk patients is essential for improving clinical outcomes and optimizing hospital resource allocation. This study presents a machine learning framework for predicting mortality following MI using a publicly available dataset of 1700 patient records, and after excluding records with over 20 missing values and features with more than 300 missing entries, the final dataset included 1547 patients and 113 variables, categorized as binary, categorical, integer, or continuous. Missing values were addressed using denoising autoencoders for continuous features and variational autoencoders for the remaining data. In contrast, feature selection was performed using Random Forest, and PowerTransformer scaling was applied, addressing class imbalance by using SMOTE. Twelve models were evaluated, including Focal-Loss Neural Networks, TabNet, XGBoost, LightGBM, CatBoost, Random Forest, SVM, Logistic Regression, and a voting ensemble. Performance was assessed using multiple metrics, with SVM achieving the highest F1 score (0.6905), ROC-AUC (0.8970), and MCC (0.6464), while Random Forest yielded perfect precision and specificity. To assess generalizability, a subpopulation external validation was conducted by training on male patients and testing on female patients. XGBoost and CatBoost reached the highest ROC-AUC (0.90), while Focal-Loss Neural Network achieved the best MCC (0.53). Overall, the proposed framework outperformed previous studies in key metrics and maintained better performance under demographic shift, supporting its potential for clinical decision-making in post-MI care. Full article

This paper addresses the problem of recognising and classifying hand movements to control an upper-limb exoskeleton. To solve this problem, a multisensory system based on the fusion of data from electromyography (EMG) sensors, inertial measurement units (IMUs), and virtual reality (VR) trackers is proposed, which provides highly accurate detection of users’ movements. Signal preprocessing (noise filtering, segmentation, normalisation) and feature extraction were performed to generate input data for regression and classification models. Various machine learning algorithms are used to recognise motor activity, ranging from classical algorithms (logistic regression, k-nearest neighbors, decision trees) and ensemble methods (random forest, AdaBoost, eXtreme Gradient Boosting, stacking, voting) to deep neural networks, including convolutional neural networks (CNNs), gated recurrent units (GRUs), and transformers. The algorithm for integrating machine learning models into the exoskeleton control system is considered. In experiments aimed at abandoning proprietary tracking systems (VR trackers), absolute position regression was performed using data from IMU sensors with 14 regression algorithms: The random forest ensemble provided the best accuracy (mean absolute error = 0.0022 metres). The task of classifying activity categories out of nine types is considered below. Ablation analysis showed that IMU and VR trackers produce a sufficient informative minimum, while adding EMG also introduces noise, which degrades the performance of simpler models but is successfully compensated for by deep networks. In the classification task using all signals, the maximum result (99.2%) was obtained on Transformer; the fully connected neural network generated slightly worse results (98.4%). When using only IMU data, fully connected neural network, Transformer, and CNN–GRU networks provide 100% accuracy. Experimental results confirm the effectiveness of the proposed architectures for motor activity classification, as well as the use of a multi-sensor approach that allows one to compensate for the limitations of individual types of sensors. The obtained results make it possible to continue research in this direction towards the creation of control systems for upper exoskeletons, including those used in rehabilitation and virtual simulation systems. Full article

Emotion recognition in speech is essential for enhancing human–computer interaction (HCI) systems. Despite progress in Bangla speech emotion recognition, challenges remain, including low accuracy, speaker dependency, and poor generalization across emotional expressions. Previous approaches often rely on traditional machine learning or basic deep learning models, struggling with robustness and accuracy in noisy or varied data. In this study, we propose a novel multi-stream deep learning feature fusion approach for Bangla speech emotion recognition, addressing the limitations of existing methods. Our approach begins with various data augmentation techniques applied to the training dataset, enhancing the model’s robustness and generalization. We then extract a comprehensive set of handcrafted features, including Zero-Crossing Rate (ZCR), chromagram, spectral centroid, spectral roll-off, spectral contrast, spectral flatness, Mel-Frequency Cepstral Coefficients (MFCCs), Root Mean Square (RMS) energy, and Mel-spectrogram. Although these features are used as 1D numerical vectors, some of them are computed from time–frequency representations (e.g., chromagram, Mel-spectrogram) that can themselves be depicted as images, which is conceptually close to imaging-based analysis. These features capture key characteristics of the speech signal, providing valuable insights into the emotional content. Sequentially, we utilize a multi-stream deep learning architecture to automatically learn complex, hierarchical representations of the speech signal. This architecture consists of three distinct streams: the first stream uses 1D convolutional neural networks (1D CNNs), the second integrates 1D CNN with Long Short-Term Memory (LSTM), and the third combines 1D CNNs with bidirectional LSTM (Bi-LSTM). These models capture intricate emotional nuances that handcrafted features alone may not fully represent. For each of these models, we generate predicted scores and then employ ensemble learning with a soft voting technique to produce the final prediction. This fusion of handcrafted features, deep learning-derived features, and ensemble voting enhances the accuracy and robustness of emotion identification across multiple datasets. Our method demonstrates the effectiveness of combining various learning models to improve emotion recognition in Bangla speech, providing a more comprehensive solution compared with existing methods. We utilize three primary datasets—SUBESCO, BanglaSER, and a merged version of both—as well as two external datasets, RAVDESS and EMODB, to assess the performance of our models. Our method achieves impressive results with accuracies of 92.90%, 85.20%, 90.63%, 67.71%, and 69.25% for the SUBESCO, BanglaSER, merged SUBESCO and BanglaSER, RAVDESS, and EMODB datasets, respectively. These results demonstrate the effectiveness of combining handcrafted features with deep learning-based features through ensemble learning for robust emotion recognition in Bangla speech. Full article

Background/Objectives: Clinicians routinely rely on periapical radiographs to identify root-end disease, but interpretation errors and inconsistent readings compromise diagnostic accuracy. We, therefore, developed an explainable, multimodal AI framework that (i) fuses two data modalities, deep CNN embeddings and radiomic texture descriptors that are extracted only from lesion-relevant pixels selected by Grad-CAM, and (ii) makes every prediction transparent through dual-layer explainability (pixel-level Grad-CAM heatmaps + feature-level SHAP values). Methods: A dataset of 2285 periapical radiographs was processed using six CNN architectures (EfficientNet-B1/B4/V2M/V2S, ResNet-50, Xception). For each image, a Grad-CAM heatmap generated from the penultimate layer of the CNN was thresholded to create a binary mask that delineated the region most responsible for the network’s decision. Radiomic features (first-order, GLCM, GLRLM, GLDM, NGTDM, and shape2D) were then computed only within that mask, ensuring that handcrafted descriptors and learned embeddings referred to the same anatomic focus. The two feature streams were concatenated, optionally reduced by principal component analysis or SelectKBest, and fed to random forest or XGBoost classifiers; five-view test-time augmentation (TTA) was applied at inference. Pixel-level interpretability was provided by the original Grad-CAM, while SHAP quantified the contribution of each radiomic and deep feature to the final vote. Results: Raw CNNs achieved a ca. 52% accuracy and AUC values near 0.60. The multimodal fusion raised performance dramatically; the Xception + radiomics + random forest model achieved a 95.4% accuracy and an AUC of 0.9867, and adding TTA increased these to 96.3% and 0.9917, respectively. The top ensemble, Xception and EfficientNet-V2S fusion vectors classified with XGBoost under five-view TTA, reached a 97.16% accuracy and an AUC of 0.9914, with false-positive and false-negative rates of 4.6% and 0.9%, respectively. Grad-CAM heatmaps consistently highlighted periapical regions, while SHAP plots revealed that radiomic texture heterogeneity and high-level CNN features jointly contributed to correct classifications. Conclusions: By tightly integrating CNN embeddings, mask-targeted radiomics, and a two-tiered explainability stack (Grad-CAM + SHAP), the proposed system delivers state-of-the-art lesion detection and a transparent technique, addressing both accuracy and trust. Full article

This study proposes CEL-DL-Bagging (Cross-Entropy Loss-optimized Deep Learning Bagging), a multi-model fusion framework that integrates cross-entropy loss-weighted voting with Bootstrap Aggregating (Bagging). First, we develop a lightweight recognition architecture by embedding a salient position attention (SPA) mechanism into four base networks (YOLOv5s-cls, EfficientNet-B0, MobileNetV3, and ShuffleNetV2), significantly enhancing discriminative feature extraction for disease patterns. Our experiments show that these SPA-enhanced models achieve consistent accuracy gains of 0.8–1.7 percentage points, peaking at 97.86%. Building on this, we introduce DB-CEWSV—an ensemble framework combining Deep Bootstrap Aggregating (DB) with adaptive Cross-Entropy Weighted Soft Voting (CEWSV). The system dynamically optimizes model weights based on their cross-entropy performance, using SPA-augmented networks as base learners. The final integrated model attains 98.33% accuracy, outperforming the strongest individual base learner by 0.48 percentage points. Compared with single models, the ensemble learning algorithm proposed in this study led to better generalization and robustness of the ensemble learning model and better identification of rice diseases in the natural background. It provides a technical reference for applying rice disease identification in practical engineering. Full article

Cybersecurity threats are becoming increasingly sophisticated, frequent, and diverse, posing a major risk to critical infrastructure, public trust, and digital economies. Traditional intrusion detection systems often struggle with detecting novel or rare attack types, particularly when data availability is limited or heterogeneous. The current study tries to address these challenges by proposing a unified, multimodal threat detection framework that leverages the combination of synthetic data generation through Generative Adversarial Networks (GANs), advanced ensemble learning, and transfer learning techniques. The research objective is to enhance detection accuracy and resilience against zero-day, botnet, and image-based malware attacks by integrating multiple data modalities, including structured network logs and malware binaries, within a scalable and flexible pipeline. The proposed system features a dual-branch architecture: one branch uses a CNN with transfer learning for image-based malware classification, and the other employs a soft-voting ensemble classifier for tabular intrusion detection, both trained on augmented datasets generated by GANs. Experimental results demonstrate significant improvements in detection performance and false positive reduction, especially when multimodal outputs are fused using the proposed confidence-weighted strategy. The findings highlight the framework’s adaptability and practical applicability in real-world intrusion detection and response systems. Full article

Information is the primary driver of progress in today’s world, especially given the vast amounts of data available for extracting meaningful knowledge. The motivation for addressing the problem of forensic analysis—specifically the validity of decision making in multi-criteria contexts—stems from its limited coverage in the existing literature. Methodologically, machine learning and ensemble models represent key trends in this domain. Datasets used for such purposes can be either real or synthetic, with synthetic data becoming particularly valuable when real data is unavailable, in line with the growing use of publicly available Internet data. The integration of these two premises forms the central challenge addressed in this paper. The proposed solution is a three-layer ensemble model: the first layer employs multi-criteria decision-making methods; the second layer implements multiple machine learning algorithms through an optimized asymmetric procedure; and the third layer applies a voting mechanism for final decision making. The model is applied and evaluated through a case study analyzing the U.S. Army’s decision to replace the Colt 1911 pistol with the Beretta 92. The results demonstrate superior performance compared to state-of-the-art models, offering a promising approach to forensic decision analysis, especially in data-scarce environments. Full article

Hyperspectral remote sensing plays a crucial role in guiding and supporting various mineral prospecting activities. Combined with artificial intelligence, hyperspectral remote sensing technology becomes a powerful and versatile tool for a wide range of mineral exploration activities. This study investigates the effectiveness of ensemble learning (EL) algorithms for lithological classification and mineral exploration using EnMAP hyperspectral imagery (HSI) in a semi-arid region. The Moroccan Anti-Atlas mountainous region is known for its complex geology, high mineral potential and rugged terrain, making it a challenging for mineral exploration. This research applies core and heterogeneous ensemble learning methods, i.e., boosting, stacking, voting, bagging, blending, and weighting to improve the accuracy and robustness of lithological classification and mapping in the Moroccan Anti-Atlas mountainous region. Several state-of-the-art models, including support vector machines (SVMs), random forests (RFs), k-nearest neighbors (k-NNs), multi-layer perceptrons (MLPs), extra trees (ETs) and extreme gradient boosting (XGBoost), were evaluated and used as individual and ensemble classifiers. The results show that the EL methods clearly outperform (single) base classifiers. The potential of EL methods to improve the accuracy of HSI-based classification is emphasized by an optimal blending model that achieves the highest overall accuracy (96.69%). The heterogeneous EL models exhibit better generalization ability than the baseline (single) ML models in lithological classification. The current study contributes to a more reliable assessment of resources in mountainous and semi-arid regions by providing accurate delineation of lithological units for mineral exploration objectives. Full article

Adversarial attacks in Natural Language Processing (NLP) present a critical challenge, particularly in sentiment analysis, where subtle input modifications can significantly alter model predictions. In search of more robust defenses against adversarial attacks on sentimental analysis, this research work introduces two novel defense mechanisms: the Lexicon-Based Random Substitute Model (LRSM) and the Word-Variant Voting Model (WVVM). LRSM employs randomized substitutions from a dataset-specific lexicon to generate diverse input variations, disrupting adversarial strategies by introducing unpredictability. Unlike traditional defenses requiring synonym dictionaries or precomputed semantic relationships, LRSM directly substitutes words with random lexicon alternatives, reducing overhead while maintaining robustness. Notably, LRSM not only neutralizes adversarial perturbations but occasionally surpasses the original accuracy by correcting inherent model misclassifications. Building on LRSM, WVVM integrates LRSM, Frequency-Guided Word Substitution (FGWS), and Synonym Random Substitution and Voting (RS&V) in an ensemble framework that adaptively combines their outputs. Logistic Regression (LR) emerged as the optimal ensemble configuration, leveraging its regularization parameters to balance the contributions of individual defenses. WVVM consistently outperformed standalone defenses, demonstrating superior restored accuracy and F1 scores across adversarial scenarios. The proposed defenses were evaluated on two well-known sentiment analysis benchmarks: the IMDB Sentiment Dataset and the Yelp Polarity Dataset. The IMDB dataset, comprising 50,000 labeled movie reviews, and the Yelp Polarity dataset, containing labeled business reviews, provided diverse linguistic challenges for assessing adversarial robustness. Both datasets were tested using 4000 adversarial examples generated by established attacks, including Probability Weighted Word Saliency, TextFooler, and BERT-based Adversarial Examples. WVVM and LRSM demonstrated superior performance in restoring accuracy and F1 scores across both datasets, with WVVM excelling through its ensemble learning framework. LRSM improved restored accuracy from 75.66% to 83.7% when compared to the second-best individual model, RS&V, while the Support Vector Classifier WVVM variation further improved restored accuracy to 93.17%. Logistic Regression WVVM achieved an F1 score of 86.26% compared to 76.80% for RS&V. These findings establish LRSM and WVVM as robust frameworks for defending against adversarial text attacks in sentiment analysis. Full article

The emergence of the Internet of Vehicles (IoV) has revolutionized intelligent transportation and communication systems. However, IoV presents many complex and ever-changing security challenges and thus requires robust cybersecurity protocols. This paper comprehensively describes and evaluates ensemble learning approaches for multi-class intrusion detection systems in the IoV environment. The study evaluates several approaches, such as stacking, voting, boosting, and bagging. A comprehensive review of the literature spanning 2020 to 2025 reveals important trends and topics that require further investigation and the relative merits of different ensemble approaches. The NSL-KDD, CICIDS2017, and UNSW-NB15 datasets are widely used to evaluate the performance of Ensemble Learning-Based Intrusion Detection Systems (ELIDS). ELIDS evaluation is usually carried out using some popular performance metrics, including Precision, Accuracy, Recall, F1-score, and Area Under Receiver Operating Characteristic Curve (AUC-ROC), which were used to evaluate and measure the effectiveness of different ensemble learning methods. Given the increasing complexity and frequency of cyber threats in IoV environments, ensemble learning methods such as bagging, boosting, and stacking enhance adaptability and robustness. These methods aggregate multiple learners to improve detection rates, reduce false positives, and ensure more resilient intrusion detection models that can evolve alongside emerging attack patterns. Full article

Gait recognition is a reliable biometric approach that uniquely identifies individuals based on their natural walking patterns. It is widely used to recognize individuals who are challenging to camouflage and do not require a person’s cooperation. The general face-based person recognition system often fails to determine the offender’s identity when they conceal their face by wearing helmets and masks to evade identification. In such cases, gait-based recognition is ideal for identifying offenders, and most existing work leverages a deep learning (DL) model. However, a single model often fails to capture a comprehensive selection of refined patterns in input data when external factors are present, such as variation in viewing angle, clothing, and carrying conditions. In response to this, this paper introduces a fusion-based multi-model gait recognition framework that leverages the potential of convolutional neural networks (CNNs) and a vision transformer (ViT) in an ensemble manner to enhance gait recognition performance. Here, CNNs capture spatiotemporal features, and ViT features multiple attention layers that focus on a particular region of the gait image. The first step in this framework is to obtain the Gait Energy Image (GEI) by averaging a height-normalized gait silhouette sequence over a gait cycle, which can handle the left–right gait symmetry of the gait. After that, the GEI image is fed through multiple pre-trained models and fine-tuned precisely to extract the depth spatiotemporal feature. Later, three separate fusion strategies are conducted, and the first one is decision-level fusion (DLF), which takes each model’s decision and employs majority voting for the final decision. The second is feature-level fusion (FLF), which combines the features from individual models through pointwise addition before performing gait recognition. Finally, a hybrid fusion combines DLF and FLF for gait recognition. The performance of the multi-model fusion-based framework was evaluated on three publicly available gait databases: CASIA-B, OU-ISIR D, and the OU-ISIR Large Population dataset. The experimental results demonstrate that the fusion-enhanced framework achieves superior performance. Full article