Search Results (108)

With the increasing sophistication of Artificial Intelligence (AI), traditional digital steganography methods face a growing risk of being detected and compromised. Adversarial attacks, in particular, pose a significant threat to the security and robustness of hidden information. To address these challenges, this paper proposes a novel AI-based steganography framework designed to enhance the security of concealed messages within digital images. Our approach introduces a multi-stage embedding process that utilizes a sequence of encoder models, including a base encoder, a residual encoder, and a dense encoder, to create a more complex and secure hiding environment. To further improve robustness, we integrate Wavelet Transforms with various deep learning architectures, namely Convolutional Neural Networks (CNNs), Bayesian Neural Networks (BNNs), and Graph Convolutional Networks (GCNs). We conducted a comprehensive set of experiments on the FashionMNIST and MNIST datasets to evaluate our framework’s performance against several adversarial attacks. The results demonstrate that our multi-stage approach significantly enhances resilience. Notably, while CNN architectures provide the highest baseline accuracy, BNNs exhibit superior intrinsic robustness against gradient-based attacks. For instance, under the Fast Gradient Sign Method (FGSM) attack on the MNIST dataset, our BNN-based models maintained an accuracy of over 98%, whereas the performance of comparable CNN models dropped sharply to between 10% and 18%. This research provides a robust and effective method for developing next-generation secure steganography systems. Full article

The deployment of deep learning models in Internet of Things (IoT) systems is increasingly threatened by adversarial attacks. To address the challenge of effectively detecting adversarial examples generated by Generative Adversarial Networks (AdvGANs), this paper proposes an unsupervised detection method that integrates spatial statistical features and multidimensional distribution characteristics. First, a collection of adversarial examples under four different attack intensities was constructed on the CIFAR-10 dataset. Then, based on the VGG16 and ResNet50 classification models, a dual-module collaborative architecture was designed: Module A extracted spatial statistics from convolutional layers and constructed category prototypes to calculate similarity, while Module B extracted multidimensional statistical features and characterized distribution anomalies using the Mahalanobis distance. Experimental results showed that the proposed method achieved a maximum AUROC of 0.9937 for detecting AdvGAN attacks on ResNet50 and 0.9753 on VGG16. Furthermore, it achieved AUROC scores exceeding 0.95 against traditional attacks such as FGSM and PGD, demonstrating its cross-attack generalization capability. Cross-dataset evaluation on Fashion-MNIST confirms its robust generalization across data domains. This study presents an effective solution for unsupervised adversarial example detection, without requiring adversarial samples for training, making it suitable for a wide range of attack scenarios. These findings highlight the potential of the proposed method for enhancing the robustness of IoT systems in security-critical applications. Full article

Image classification is central to computer vision, supporting applications from autonomous driving to medical imaging, yet state-of-the-art convolutional neural networks remain constrained by heavy floating-point operations (FLOPs) and parameter counts on edge devices. To address this accuracy–efficiency trade-off, we propose a unified lightweight framework built on a pruning-aware coordinate attention block (PACB). PACB integrates coordinate attention (CA) with L1-regularized channel pruning, enriching feature representation while enabling structured compression. Applied to MobileNetV3 and RepVGG, the framework achieves substantial efficiency gains. On GTSRB, MobileNetV3 parameters drop from 16.239 M to 9.871 M (–6.37 M) and FLOPs from 11.297 M to 8.552 M (–24.3%), with accuracy improving from 97.09% to 97.37%. For RepVGG, parameters fall from 7.683 M to 7.093 M (–0.59 M) and FLOPs from 31.264 M to 27.918 M (–3.35 M), with only ~0.51% average accuracy loss across CIFAR-10, Fashion-MNIST, and GTSRB. Complexity analysis further confirms PACB does not increase asymptotic order, since the additional CA operations contribute only lightweight lower-order terms. These results demonstrate that coupling CA with structured pruning yields a scalable accuracy–efficiency trade-off under hardware-agnostic metrics, making PACB a promising, deployment-ready solution for mobile and edge applications. Full article

Predictive Coding (PC) frameworks claim to model recognition via prediction–error loops, but they often lack explicit biological implementation of fast familiar recognition and impose latency that limits real-time robotic control. We begin with Experience-Based Pattern Matching (EBPM), a biologically grounded mechanism inspired by neural engram reactivation, enabling near-instantaneous recognition of familiar stimuli without iterative inference. Building upon this, we propose Dynamic Integrative Matching and Encoding (DIME), a hybrid system that relies on EBPM under familiar and low-uncertainty conditions and dynamically engages PC when confronted with novelty or high uncertainty. We evaluate EBPM, PC, and DIME across multiple image datasets (MNIST, Fashion-MNIST, CIFAR-10) and on a robotic obstacle-course simulation. Results from multi-seed experiments with ablation and complexity analyses show that EBPM achieves minimal latency (e.g., ~0.03 ms/ex in MNIST, ~0.026 ms/step in robotics) but poor performance in novel or noisy cases; PC exhibits robustness at a high cost; DIME delivers strong trade-offs—boosted accuracy in familiar clean situations (+4–5% over EBPM on CIFAR-10), while cutting PC invocations by ~50% relative to pure PC. Our contributions: (i) formalizing EBPM as a neurocomputational algorithm built from biologically plausible principles, (ii) developing DIME as a dynamic EBPM–PC integrator, (iii) providing ablation and complexity analyses illuminating component roles, and (iv) offering empirical validation in both perceptual and embodied robotic scenarios—paving the way for low-latency recognition systems. Full article

We study motif-based optimization for sparse multilayer perceptrons (MLPs), where weights are shared and updated at the level of small neuron groups (‘motifs’) rather than individual connections. Building on Sparse Evolutionary Training (SET), our approach reduces the number of unique parameters and redundant multiply–accumulate operations by exploiting block-structured sparsity. Across Fashion-MNIST and a lung X-ray dataset, our Motif-SET improves training/inference efficiency with modest accuracy trade-offs, and we provide a principled recipe to choose motif size based on accuracy and efficiency budgets. We further compare against representative modern sparse training and compression methods, analyze failure modes such as overly large motifs, and outline real-world constraints on mobile/embedded targets. Our results and ablations indicate that motif size $m=2$ often offers a strong balance between compute and accuracy under resource constraints. Full article

Quantum-enhanced machine learning, encompassing both quantum algorithms and quantum-inspired classical methods such as tensor networks, offers promising tools for extracting structure from complex, high-dimensional data. In this work, we study the training dynamics of Matrix Product State (MPS) classifiers applied to three-class problems, using both fashion MNIST and hyperspectral satellite imagery as representative datasets. We investigate the phenomenon of grokking, where generalization emerges suddenly after memorization, by tracking entanglement entropy, local magnetization, and model performance across training sweeps. Additionally, we employ information-theory tools to gain deeper insights: transfer entropy is used to reveal causal dependencies between label-specific quantum masks, while O-information captures the shift from synergistic to redundant correlations among class outputs. Our results show that grokking in the fashion MNIST task coincides with a sharp entanglement transition and a peak in redundant information, whereas the overfitted hyperspectral model retains synergistic, disordered behavior. These findings highlight the relevance of high-order information dynamics in quantum-inspired learning and emphasize the distinct learning behaviors that emerge in multi-class classification, offering a principled framework to interpret generalization in quantum machine learning architectures. Full article

Quantization-aware training (QAT) has emerged as a key strategy for enabling efficient deep learning inference on resource-constrained platforms. Yet, most existing approaches rely on static, manually selected numeric formats—fixed-point or floating-point—and fixed bit-widths, limiting their adaptability and often requiring extensive design effort or architecture search. In this work, we introduce a novel QAT framework that breaks this rigidity by jointly learning, during training, both the numeric representation format and the associated bit-widths in an end-to-end differentiable manner. At the core of our method lies a unified parameterization that is capable of emulating both fixed- and floating-point arithmetic, paired with a bit-aware loss function that penalizes excessive precision in a hardware-aligned fashion. We demonstrate that our approach achieves state-of-the-art trade-offs between accuracy and compression on MNIST, CIFAR-10, and CIFAR-100, reducing average bit-widths to as low as 1.4 with minimal accuracy loss. Furthermore, FPGA implementation using Xilinx FINN confirms over $5\times$ LUT and $4\times$ BRAM savings. This is the first QAT method to unify numeric format learning with differentiable precision control, enabling highly deployable, precision-adaptive deep neural networks. Full article

Activation functions play a crucial role in ensuring training stability, convergence speed, and overall performance in both convolutional and attention-based networks. In this study, we introduce two novel activation functions, each incorporating a sine component and a constraint term. To assess their effectiveness, we replace the activation functions in four representative architectures—VGG16, ResNet50, DenseNet121, and Vision Transformers—covering a spectrum from lightweight to high-capacity models. We conduct extensive evaluations on four benchmark datasets (CIFAR-10, CIFAR-100, MNIST, and Fashion-MNIST), comparing our methods against seven widely used activation functions. The results consistently demonstrate that our proposed functions achieve superior performance across all tested models and datasets. From a design application perspective, the proposed functional periodic structure also facilitates rich and structurally stable activation visualizations, enabling designers to trace model attention, detect surface biases early, and make informed aesthetic or accessibility decisions during interface prototyping. Full article

Federated learning enables collaborative model training across distributed devices while preserving data privacy. However, in real-world environments such as smart cities, heterogeneous and resource-constrained edge devices often render existing methods impractical. Low-power sensors and cameras struggle to complete full-model training, while high-performance devices remain idly waiting for others. Knowledge distillation approaches rely on public datasets that are rarely available or poorly aligned with urban data, which limits their effectiveness in deployment. These limitations lead to inefficiencies, unstable convergence, and poor adaptability in diverse urban networks. Partial training alleviates some challenges by allowing clients to train submodels tailored to their capacity, but existing methods still incur high computational costs for identifying important parameters and suffer from uneven parameter updates, reducing model effectiveness. To address these challenges, we propose Parameter-Level Dynamic Submodel Extraction (PLDSE), a lightweight and adaptive framework for federated learning. PLDSE estimates parameter importance using gradient-based scores on a server-side validation set, reducing overhead while accurately identifying critical parameters. In addition, it integrates a rolling scheduling mechanism to rotate unselected parameters, ensuring full coverage and consistent model updates. Experiments on CIFAR-10, CIFAR-100, and Fashion-MNIST demonstrate superior accuracy and faster convergence, with PLDSE achieving 62.82% on CIFAR-100 under low heterogeneity and 61.51% under high heterogeneity, outperforming prior methods. Full article

We explore the sparsity and prunability of multi-layer perceptrons (MLPs) trained using the Forward-Forward (FF) algorithm, an alternative to backpropagation (BP) that replaces the backward pass with local, contrastive updates at each layer. We analyze the sparsity of the weight matrices during training using multiple metrics, and test the prunability of FF networks on the MNIST, FashionMNIST and CIFAR-10 datasets. We also propose FFLib—a novel, modular PyTorch-based library for developing, training and analyzing FF models along with a suite of FF-based architectures, including FFNN, FFNN+C and FFRNN. In addition to structural sparsity, we describe and apply a new method for visualizing the functional sparsity of neural activations across different architectures using the HSV color space. Moreover, we conduct a sensitivity analysis to assess the impact of hyperparameters on model performance and sparsity. Finally, we perform pruning experiments, showing that simple FF-based MLPs exhibit significantly greater robustness to one-shot neuron pruning than traditional BP-trained networks, and a possible 8-fold increase in compression ratios while maintaining comparable accuracy on the MNIST dataset. Full article

Inspired by the interpretability of Kolmogorov–Arnold Networks (KANs), a novel Pixel-level Feature Selection (PFS) model based on KANs (PFSKANs) is proposed as a fundamentally distinct alternative from trainable Convolutional Neural Networks (CNNs) and transformers in the computer vision tasks. We modify the simplification techniques of KANs to detect key pixels with high contribution scores directly at the input image. Specifically, a trainable selection procedure is intuitively visualized and performed only once, since the obtained interpretable pixels can subsequently be identified and dimensionally standardized using the proposed mathematical approach. Experiments on the image classification tasks using the MNIST, Fashion-MNIST, CIFAR-10, and CIFAR-100 datasets demonstrate that PFSKANs achieve comparable performance to CNNs in terms of accuracy, parameter efficiency, and training time. Full article

Neural Architecture search (NAS) aims to automate the design process of Deep Neural Networks, reducing the Deep Learning (DL) expertise required and avoiding a trial-and-error process. Nonetheless, one of the main drawbacks of NAS is the high consumption of computational resources. Consequently, efficient evaluation methods (EEMs) to assess the quality of candidate architectures are an open research problem. This work tests various EEMs in the Deep Genetic Algorithm (DeepGA), including early stopping, population memory, and training-free proxies. The Fashion MNIST, CIFAR-10, and CIFAR-100 datasets were used for experimentation. The results show that population memory has a valuable impact on avoiding repeated evaluations. Additionally, early stopping achieved competitive performance while significantly reducing the computational cost of the search process. The training-free configurations using the Logsynflow and Linear Regions proxies, as well as a combination of both, were only partially competitive but dramatically reduced the search time. Finally, a comparison of the architectures and hyperparameters obtained with the different algorithm configurations is presented. The training-free search processes resulted in deeper architectures with more fully connected layers and skip connections than the ones obtained with accuracy-guided search configurations. Full article

In the previous article, we introduced a neural network framework based on symmetric differential equations. This novel framework exhibits complete symmetry, endowing it with perfect mathematical properties. While we have examined some of the system’s mathematical characteristics, a detailed discussion of the network training methodology has not yet been presented. Drawing on the principles of the traditional backpropagation algorithm, this study proposes an alternative training approach that utilizes differential equation signal propagation instead of chain rule derivation. This approach not only preserves the effectiveness of training but also offers enhanced biological interpretability. The foundation of this methodology lies in the system’s reversibility, which stems from its inherent symmetry—a key aspect of our research. However, this method alone is insufficient for effective neural network training. To address this, we further introduce a distributed Proportional–Integral–Derivative (PID) control approach, emphasizing its implementation within a closed system. By incorporating this method, we achieved both faster training speeds and improved accuracy. This approach not only offers novel insights into neural network training but also extends the scope of research into control methodologies. To validate its effectiveness, we apply this method to the MNIST (Modified National Institute of Standards and Technology database) and Fashion-MNIST, demonstrating its practical utility. Full article

In this study, we present a comparative analysis of gradient descent-based optimizers frequently used in Convolutional Neural Networks (CNNs), including SGD, mSGD, RMSprop, Adadelta, Nadam, Adamax, Adam, and the recent EVE optimizer. To explore the interaction between optimization strategies and activation functions, we systematically evaluate all combinations of these optimizers with four activation functions—ReLU, LeakyReLU, Tanh, and GELU—across three benchmark image classification datasets: CIFAR-10, Fashion-MNIST (F-MNIST), and Labeled Faces in the Wild (LFW). Each configuration was assessed using multiple evaluation metrics, including accuracy, precision, recall, F1-score, mean absolute error (MAE), and mean squared error (MSE). All experiments were performed using k-fold cross-validation to ensure statistical robustness. Additionally, two-way ANOVA was employed to validate the significance of differences across optimizer–activation combinations. This study aims to highlight the importance of jointly selecting optimizers and activation functions to enhance training dynamics and generalization in CNNs. We also consider the role of critical hyperparameters, such as learning rate and regularization methods, in influencing optimization stability. This work provides valuable insights into the optimizer–activation interplay and offers practical guidance for improving architectural and hyperparameter configurations in CNN-based deep learning models. Full article

Background/Objectives: Convolutional Neural Networks (CNNs), while effective in tasks such as image classification and language processing, often experience overfitting and inefficient training due to static, structure-agnostic regularization techniques like traditional dropout. This study aims to address these limitations by proposing a more dynamic and context-sensitive dropout strategy. Methods: We introduce Probabilistic Feature Importance Dropout (PFID), a novel regularization method that assigns dropout rates based on the probabilistic significance of individual features. PFID is integrated with adaptive, structured, and contextual dropout strategies, forming a unified framework for intelligent regularization. Results: Experimental evaluation on standard benchmark datasets including CIFAR-10, MNIST, and Fashion MNIST demonstrated that PFID significantly improves performance metrics such as classification accuracy, training loss, and computational efficiency compared to conventional dropout methods. Conclusions: PFID offers a practical and scalable solution for enhancing CNN generalization and training efficiency. Its dynamic nature and feature-aware design provide a strong foundation for future advancements in adaptive regularization for deep learning models. Full article