Search Results (844)

The active or adjustable journal bearings are designed with unique mechanisms to reduce the rotor-bearing system lateral vibrations by adjusting their damping and stiffness. The article provides a comprehensive review of the literature, outlining the structure and findings of studies on active bearings. Over the years, various kinds of adjustable bearing designs have been developed with unique operational mechanisms. Such bearing designs include adjustable pad sectors, externally adjustable pads, active oil injection through pad openings, and flexible deformable sleeves. These modifications enhance the turbine shaft line’s performance by increasing the system’s overall stability. The detailed review in this paper highlights the characteristics of bearings, along with the key advantages, limitations, and potential offered by active control across different bearing types. The efficiency of any rotor system can be greatly enhanced by optimally selecting the adjustable bearing parameters. These adjustable bearings have demonstrated a unique capability to modify the hydrodynamic operation within the bearing clearances. Experimental studies and simulation approaches were also utilized to optimize bearing geometries, lubrication regimes, and control mechanisms. The use of advanced controllers like PID, LQG, and Deep Q networks further refined the stability. The concluding section of the article explores potential avenues for the future development of active bearings. Full article

In modern intelligent buildings, the control of Building Energy Systems (BES) faces increasing complexity in balancing energy costs, thermal comfort, and operational flexibility. Traditional centralized or flat deep reinforcement learning (DRL) methods often fail to effectively handle the multi-timescale dynamics, large state–action spaces, and strict constraint satisfaction required for real-world energy systems. To address these challenges, this paper proposes an energy policy-aware layered control architecture that combines Virtual Energy Storage System (VESS) modeling with a novel Dynamic Constraint-Aware Policy Optimization (DCPO) algorithm. The VESS is modeled based on the thermal inertia of building envelope components, quantifying flexibility in terms of virtual power, capacity, and state of charge, thus enabling BES to behave as if it had embedded, non-physical energy storage. Building on this, the BES control problem is structured using a hierarchical Markov Decision Process, in which the upper level handles strategic decisions (e.g., VESS dispatch, HVAC modes), while the lower level manages real-time control (e.g., temperature adjustments, load balancing). The proposed DCPO algorithm extends actor–critic learning by incorporating dynamic policy constraints, entropy regularization, and adaptive clipping to ensure feasible and efficient policy learning under both operational and comfort-related constraints. Simulation experiments demonstrate that the proposed approach outperforms established algorithms like Deep Q-Networks (DQN), Deep Deterministic Policy Gradient (DDPG), and Twin Delayed DDPG (TD3). Specifically, it achieves a 32.6% reduction in operational costs and over a 51% decrease in thermal comfort violations compared to DQN, while ensuring millisecond-level policy generation suitable for real-time BES deployment. Full article

In the context of industrial applications, ensuring medium access control is a fundamental challenge. Industrial IoT devices are resource-constrained and must guarantee reliable communication while reducing energy consumption. The IEEE 802.15.4e standard proposed time-slotted channel hopping (TSCH) to meet the requirements of the industrial Internet of Things. TSCH relies on time synchronization and channel hopping to improve performance and reduce energy consumption. Despite these characteristics, configuring an efficient schedule under varying traffic conditions and interference scenarios remains a challenging problem. The exploitation of reinforcement learning (RL) techniques offers a promising approach to address this challenge. AI enables TSCH to dynamically adapt its scheduling based on real-time network conditions, making decisions that optimize key performance criteria such as energy efficiency, reliability, and latency. By learning from the environment, reinforcement learning can reconfigure schedules to mitigate interference scenarios and meet traffic demands. In this work, we compare various reinforcement learning (RL) algorithms in the context of the TSCH environment. In particular, we evaluate the deep Q-network (DQN), double deep Q-network (DDQN), and prioritized DQN (PER-DQN). We focus on the convergence speed of these algorithms and their capacity to adapt the schedule. Our results show that the PER-DQN algorithm improves the packet delivery ratio and achieves faster convergence compared to DQN and DDQN, demonstrating its effectiveness for dynamic TSCH scheduling in Industrial IoT environments. These quantifiable improvements highlight the potential of prioritized experience replay to enhance reliability and efficiency under varying network conditions. Full article

To improve the tracking accuracy and the adaptability of intelligent vehicles in various road conditions, an adaptive model predictive controller combining reinforcement learning is proposed in this paper. Firstly, to solve the problem of control accuracy decline caused by a fixed prediction time domain, a low-computational-cost adaptive prediction horizon strategy based on a two-dimensional Gaussian function is designed to realize the real-time adjustment of prediction time domain change with vehicle speed and road curvature. Secondly, to address the problem of tracking stability reduction under complex road conditions, the Deep Q-Network (DQN) algorithm is used to adjust the weight matrix of the Model Predictive Control (MPC) algorithm; then, the convergence speed and control effectiveness of the tracking controller are improved. Finally, hardware-in-the-loop tests and real vehicle tests are conducted. The results show that the proposed adaptive predictive horizon controller (DQN-AP-MPC) solves the problem of poor control performance caused by fixed predictive time domain and fixed weight matrix values, significantly improving the tracking accuracy of intelligent vehicles under different road conditions. Especially under variable curvature and high-speed conditions, the proposed controller reduces the maximum lateral error by 76.81% compared to the unimproved MPC controller, and reduces the average absolute error by 64.44%. The proposed controller has a faster convergence speed and better trajectory tracking performance when tested on variable curvature road conditions and double lane roads. Full article

With the expansion of vehicle-to-everything (V2X) networks and the rising demand for intelligent services, vehicle edge computing encounters heightened requirements for more efficient task offloading. This study proposes a task offloading technique that utilizes federated collaboration and multi-agent deep reinforcement learning to reduce system latency and energy consumption. The task offloading issue is formulated as a Markov decision process (MDP), and a framework utilizing the Multi-Agent Dueling Double Deep Q-Network (MAD3QN) is developed to facilitate agents in making optimal offloading decisions inside intricate environments. Secondly, Federated Learning (FL) is implemented during the training phase, leveraging local training outcomes from many vehicles to enhance the global model, thus augmenting the learning efficiency of the agents. Experimental results indicate that, compared to conventional baseline algorithms, the proposed method decreases latency and energy consumption by at least 10% and 9%, respectively, while enhancing the average reward by at least 21%. Full article

This paper presents the design and implementation of an Intelligent Home Energy Management System in a smart home. The system is based on an economically decentralized hybrid concept that includes photovoltaic technology, a proton exchange membrane fuel cell, and a hydrogen refueling station, which together provide a reliable, secure, and clean power supply for smart homes. The proposed design enables power transfer between Vehicle-to-Home (V2H) and Home-to-Vehicle (H2V) systems, allowing electric vehicles to function as mobile energy storage devices at the grid level, facilitating a more adaptable and autonomous network. Our approach employs Double Deep Q-networks for adaptive control and forecasting. A Multi-Agent System coordinates actions between home appliances, energy storage systems, electric vehicles, and hydrogen power devices to ensure effective and cost-saving energy distribution for users of the smart grid. The design validation is carried out through MATLAB/Simulink-based simulations using meteorological data from Tunis. Ultimately, the V2H/H2V system enhances the utilization, reliability, and cost-effectiveness of residential energy systems compared with other management systems and conventional networks. Full article

To improve the traffic efficiency of connected and automated vehicles (CAVs) in on-ramp merging areas, this study proposes a novel region-level multi-agent reinforcement learning framework, Regionally Coordinated Nash-Advantage Decomposition Deep Q-Network with Conflict-Aware Q Fusion (RC-NashAD-DQN). Unlike existing vehicle-level control methods, which suffer from high computational overhead and poor scalability, our approach abstracts on-ramp and main road areas as region-level control agents, achieving coordinated yet independent decision-making while maintaining control precision and merging efficiency comparable to fine-grained vehicle-level approaches. Each agent adopts a value–advantage decomposition architecture to enhance policy stability and distinguish action values, while sharing state–action information to improve inter-agent awareness. A Nash equilibrium solver is applied to derive joint strategies, and a conflict-aware Q-fusion mechanism is introduced as a regularization term rather than a direct action-selection tool, enabling the system to resolve local conflicts—particularly at region boundaries—without compromising global coordination. This design reduces training complexity, accelerates convergence, and improves robustness against communication imperfections. The framework is evaluated using the SUMO simulator at the Taishan Road interchange on the S1 Yongtaiwen Expressway under heterogeneous traffic conditions involving both passenger cars and container trucks, and is compared with baseline models including C-DRL-VSL and MADDPG. Extensive simulations demonstrate that RC-NashAD-DQN significantly improves average traffic speed by 17.07% and reduces average delay by 12.68 s, outperforming all baselines in efficiency metrics while maintaining robust convergence performance. These improvements enhance cooperation and merging efficiency among vehicles, contributing to sustainable urban mobility and the advancement of intelligent transportation systems. Full article

Considering the dynamic RCS characteristics of stealthy UAVs, we proposed a stealthy UAV path planning algorithm based on the Double Deep Q-Network (DDQN). By introducing the reinforcement learning model that can interact with the environment, the stealth UAV adjusts the path planning strategy through the rewards obtained from the environment to design the optimal path in real-time. Specifically, by considering the effect of RCS from different angles on the detection probability of the air defense radar, the stealth UAV realizes the iterative optimization of the path planning scheme to improve the reliability of the penetration path. Under the guidance of a goal-directed composite reward function proposed, the convergence speed of the stealth UAV path planning algorithm is improved. The simulation results show that the stealth UAV can reach the target position with the optimal path while avoiding the threat zone. Full article

Accurate state-of-charge (SOC) estimation is crucial in smart-building energy management systems, where rooftop photovoltaics and lithium-ion energy storage systems must be coordinated to align renewable generation with real-time demand. This paper introduces a novel, modular hybrid framework for SOC estimation, which synergistically combines the predictive power of artificial neural networks (ANNs), the logical consistency of finite state automata (FSA), and an adaptive dynamic supervisor layer. Three distinct ANN architectures—feedforward neural network (FFNN), long short-term memory (LSTM), and 1D convolutional neural network (1D-CNN)—are employed to extract comprehensive temporal and spatial features from raw data. The inherent challenge of ANNs producing physically irrational SOC values is handled by processing their raw predictions through an FSA module, which constrains physical validity by applying feasible transitions and domain constraints based on battery operational states. To further enhance the adaptability and robustness of the framework, two advanced supervisor mechanisms are developed for model selection during estimation. A lightweight rule-based supervisor picks a model transparently using recent performance scores and quick signal heuristics, whereas a more advanced double deep Q-network (DQN) reinforcement-learning supervisor continuously learns from reward feedback to adaptively choose the model that minimizes SOC error under changing conditions. This RL agent dynamically selects the most suitable ANN+FSA model, significantly improving performance under varying and unpredictable operational conditions. Comprehensive experimental validation demonstrates that the hybrid approach consistently outperforms raw ANN predictions and conventional extended Kalman filter (EKF)-based methods. Notably, the RL-based supervisor exhibits good adaptability and achieves lower error results in challenging high-variance scenarios. Full article

Background/Objectives: Sepsis is a severe medical condition characterized by a dysregulated host response to infection, with potentially fatal outcomes, requiring early diagnosis and rapid intervention. The limitations of traditional sepsis identification methods, as well as the complexity of clinical data generated in intensive care, have driven increased interest in applying artificial intelligence in this field. The aim of this narrative review article is to analyze how artificial intelligence is being used in the diagnosis and prognosis of sepsis, to present the most relevant current models and algorithms, and to discuss the challenges and opportunities related to integrating these technologies into clinical practice. Methods: We conducted a structured literature search for this narrative review, covering studies published between 2016 and 2024 in databases such as PubMed/Medline, Scopus, Web of Science, IEEE Xplore, and Google Scholar. The review covered models based on machine learning (ML), deep neural networks (DNNs), Recurrent Neural Networks (RNNs), and clinical alert systems implemented in hospitals. The clinical data sources used, algorithms applied, system architectures, and performance outcomes are presented. Results: Numerous artificial intelligence models demonstrated superior performance compared to conventional clinical scores (qSOFA, SIRS), achieving AUC values above 0.90 in predicting sepsis and mortality. Systems such as Targeted Real-Time Early Warning System (TREWS) and InSight have been clinically validated and have significantly reduced the time to treatment initiation. However, challenges remain, such as a lack of model transparency, algorithmic bias, difficulties integrating into clinical workflows, and the absence of external validation in multicenter settings. Conclusions: Artificial intelligence has the potential to transform sepsis management through early diagnosis, risk stratification, and personalized treatment. A responsible, multidisciplinary approach is necessary, including rigorous clinical validation, enhanced interpretability, and training of healthcare personnel to effectively integrate these technologies into everyday practice. Full article

As energy systems transition toward high shares of variable renewable generation, local energy communities (ECs) are increasingly relevant for enabling demand-side flexibility and self-sufficiency. This shift is particularly evident in the residential sector, where the deployment of photovoltaic (PV) systems is rapidly growing. While mixed-integer linear programming (MILP) remains the standard for operational optimization and demand response in such systems, its computational burden limits scalability and responsiveness under real-time or uncertain conditions. Reinforcement learning (RL), by contrast, offers a model-free, adaptive alternative. However, its application to real-world energy system operation remains limited. This study explores the application of a Deep Q-Network (DQN) to a real residential EC, which has received limited attention in prior work. The system comprises three single-family homes sharing a centralized heating system with a thermal energy storage (TES), a PV installation, and a grid connection. We compare the performance of MILP and RL controllers across economic and environmental metrics. Relative to a reference scenario without TES, MILP and RL reduce energy costs by 10.06% and 8.78%, respectively, and both approaches yield lower total energy consumption and CO₂-equivalent emissions. Notably, the trained RL agent achieves a near-optimal outcome while requiring only 22% of the MILP’s computation time. These results demonstrate that DQNs can offer a computationally efficient and practically viable alternative to MILP for real-time control in residential energy systems. Full article

The energy management strategy (EMS) is a core technology for improving the fuel economy of hybrid electric vehicles (HEVs). However, the coexistence of both discrete and continuous control variables, along with complex physical constraints in HEV powertrains, presents significant challenges for the design of efficient EMSs based on deep reinforcement learning (DRL). To further enhance fuel efficiency and coordinated powertrain control under complex driving conditions, this study proposes a hierarchical DRL-based EMS. The proposed strategy adopts a layered control architecture: the upper layer utilizes the soft actor–critic (SAC) algorithm for continuous control of engine torque, while the lower layer employs a deep Q-network (DQN) for discrete gear selection optimization. Through offline training and online simulation, experimental results demonstrate that the proposed strategy achieves fuel economy performance comparable to dynamic programming (DP), with only a 3.06% difference in fuel consumption. Moreover, it significantly improves computational efficiency, thereby enhancing the feasibility of real-time deployment. This study validates the optimization potential and real-time applicability of hierarchical reinforcement learning for hybrid control in HEV energy management. Furthermore, its adaptability is demonstrated through sustained and stable performance under long-duration, complex urban bus driving conditions. Full article

As IoT (internet of things) devices grow in prominence, safeguarding them from cyberattacks is becoming a pressing challenge. To bootstrap IoT security, device identification or authentication is crucial for establishing trusted connections among devices without prior trust. In this regard, radio frequency fingerprinting (RFF) is gaining attention because it is more efficient and requires fewer computational resources compared to resource-intensive cryptographic methods, such as digital signatures. RFF works by identifying unique manufacturing defects in the radio circuitry of IoT devices by analyzing over-the-air signals that embed these imperfections, allowing for the identification of the transmitting hardware. Recent studies on RFF often leverage advanced classification models, including classical machine learning techniques such as K-Nearest Neighbor (KNN) and Support Vector Machine (SVM), as well as modern deep learning architectures like Convolutional Neural Network (CNN). In particular, CNNs are well-suited as they use multidimensional mapping to detect and extract reliable fingerprints during the learning process. However, a significant limitation of these approaches is that they require large datasets and necessitate retraining when new devices not included in the initial training set are added. This retraining can cause service interruptions and is costly, especially in large-scale IoT networks. In this paper, we propose a novel solution to this problem: RFF using Siamese networks, which eliminates the need for retraining and allows for seamless authentication in IoT deployments. The proposed Siamese network is trained using in-phase and quadrature (I/Q) samples from 10 different Software-Defined Radios (SDRs). Additionally, we present a new algorithm, the Similarity-Based Embedding Classification (SBEC) for RFF. We present experimental results that demonstrate that the Siamese network effectively distinguishes between malicious and trusted devices with a remarkable 98% identification accuracy. Full article

In the cybersecurity attack and defense space, the “attacker” and the “defender” form a dynamic and symmetrical adversarial pair. Their strategy iterations and capability evolutions have long been in a symmetrical game of mutual restraint. We will introduce modern Intrusion Detection Systems (IDSs) from the defender’s side to counter the techniques designed by the attacker (APT attack). One major challenge faced by IDS is to identify complex attack paths from a vast provenance graph. By constructing an attack behavior tracking graph, the interactions between system entities can be recorded, but the malicious activities of attackers are often hidden among a large number of normal system operations. Although traditional methods can identify attack behaviors, they only focus on the surface association relationships between entities and ignore the deep causal relationships, which limits the accuracy and interpretability of detection. Existing graph anomaly detection methods usually assign the same weight to all interactions, while we propose a Causal Autoencoder for Graph Explanation (CAGE) based on reinforcement learning. This method extracts feature representations from the traceability graph through a graph attention network(GAT), uses Q-learning to dynamically evaluate the causal importance of edges, and highlights key causal paths through a weight layering strategy. In the DARPA TC project, the experimental results conducted on the selected three datasets indicate that the precision of this method in the anomaly detection task remains above 97% on average, demonstrating excellent accuracy. Moreover, the recall values all exceed 99.5%, which fully proves its extremely low rate of missed detections. Full article

In recent years, self-autonomous intelligent transportation applications such as drones and autonomous vehicles have seen rapid development and deployment across various countries. Within the domain of artificial intelligence, self-autonomous agents are defined as software entities capable of independently operating drones in an intelligent transport system (ITS) without human intervention. The integration of these agents into autonomous vehicles and their deployment across distributed cloud networks have increased significantly. These systems, which include drones, ground vehicles, and aircraft, are used to perform a wide range of tasks such as delivering passengers and packages within defined operational boundaries. Despite their growing utility, practical implementations face significant challenges stemming from the heterogeneity of network resources, as well as persistent issues related to security, privacy, and processing costs. To overcome these challenges, this study proposes a novel blockchain-enabled self-autonomous intelligent transport system designed for drone workflow applications. The proposed system architecture is based on a remote method invocation (RMI) client–server model and incorporates a serverless computing framework to manage processing costs. Termed the self-autonomous blockchain-enabled cost-efficient system (SBECES), the framework integrates a client and system agent mechanism governed by Q-learning and deep-learning-based policies. Furthermore, it incorporates a blockchain-based hash validation and fault-tolerant (HVFT) mechanism to ensure data integrity and operational reliability. A deep reinforcement learning (DRL)-enabled adaptive scheduler is utilized to manage drone workflow execution while meeting quality of service (QoS) constraints, including deadlines, cost-efficiency, and security. The overarching objective of this research is to minimize the total processing costs that comprise execution, communication, and security overheads, while maximizing operational rewards and ensuring the timely execution of drone-based tasks. Experimental results demonstrate that the proposed system achieves a 30% reduction in processing costs and a 29% improvement in security and privacy compared to existing state-of-the-art solutions. Full article