Search Results (12)

Cyber coercion, where legitimate users are forced to perform actions under duress, poses a serious insider threat to modern organizations, especially to critical infrastructure. Traditional security controls and monitoring tools struggle to distinguish coerced actions from normal user actions. In this paper, we propose a cyber coercion detection system that analyzes a user’s activity using an integrated large language model (LLM) to evaluate contextual cues from user commands or actions and current policies and procedures. If the LLM indicates coercion, behavioral methods, such as keystroke dynamics and mouse usage patterns, and physiological signals such as heart rate are analyzed to detect stress or anomalies indicative of duress. Experimental results show that the LLM-assisted multimodal approach shows potential in detecting coercive activity with and without detected coercive communication, where multimodal biometrics assist the confidence of the LLM in cases in which it does not detect coercive communication. The proposed system may add a critical detection capability against coercion-based cyber-attacks, providing early warning signals that could inform defensive responses before damage occurs. Full article

Pilot visual behavior safety assessment is a cross-disciplinary technology that analyzes pilots’ gaze behavior and neurocognitive responses. This paper proposes a multimodal analysis method for pilot visual behavior safety, specifically for cognitive state decoding. This method aims to achieve a quantitative and efficient assessment of pilots’ observational behavior. Addressing the subjective limitations of traditional methods, this paper proposes an observational behavior detection model that integrates facial images to achieve dynamic and quantitative analysis of observational behavior. It addresses the “Midas contact” problem of observational behavior by constructing a cognitive analysis method using multimodal signals. We propose a bidirectional long short-term memory (LSTM) network that matches physiological signal rhythmic features to address the problem of isolated features in multidimensional signals. This method captures the dynamic correlations between multiple physiological behaviors, such as prefrontal theta and chest-abdominal coordination, to decode the cognitive state of pilots’ observational behavior. Finally, the paper uses a decision-level fusion method based on an improved Dempster–Shafer (DS) evidence theory to provide a quantifiable detection strategy for aviation safety standards. This dual-dimensional quantitative assessment system of “visual behavior–neurophysiological cognition” reveals the dynamic correlations between visual behavior and cognitive state among pilots of varying experience. This method can provide a new paradigm for pilot neuroergonomics training and early warning of vestibular-visual integration disorders. Full article

With the improvement in living standards and the aging of the population, the development of thin, light, and unobtrusive electronic skin devices is accelerating. These electronic devices combine the convenience of wearable electronics with the comfort of a skin-like fit. They are used to acquire multimodal physiological signal data from the wearer and real-time transmission of signals for vital signs monitoring, health dynamics warning, and disease prevention. These capabilities impose unique requirements on material selection, signal transmission, and data processing for such electronic devices. Firstly, this review provides a systematic introduction to nanomaterials, conductive hydrogels, and liquid metals, which are currently used in human health monitoring. Then, it introduces the solution to the contradiction between wireless data transmission and flexible electronic skin devices. Then, the latest data processing progress is briefly described. Finally, the latest research advances in electronic skin devices based on medical scenarios are presented, and their current development, challenges faced, and future opportunities in the field of vital signs monitoring are discussed. Full article

The increasing deployment of wind energy systems, particularly offshore wind farms, necessitates advanced monitoring and maintenance strategies to ensure optimal performance and minimize downtime. Supervisory Control And Data Acquisition (SCADA) systems have become indispensable tools for monitoring the operational health of wind turbines, generating vast quantities of time series data from various sensors. Anomaly detection techniques applied to this data offer the potential to proactively identify deviations from normal behavior, providing early warning signals of potential component failures. Traditional model-based approaches for fault detection often struggle to capture the complexity and non-linear dynamics of wind turbine systems. This has led to a growing interest in data-driven methods, particularly those leveraging machine learning and deep learning, to address anomaly detection in wind energy applications. This study focuses on the development and application of a semi-supervised, multivariate anomaly detection model for horizontal axis wind turbines. The core of this study lies in Bidirectional Long Short-Term Memory (BI-LSTM) networks, specifically a BI-LSTM autoencoder architecture, to analyze time series data from a SCADA system and automatically detect anomalous behavior that could indicate potential component failures. Moreover, the approach is reinforced by the integration of the Isolation Forest algorithm, which operates in an unsupervised manner to further refine normal behavior by identifying and excluding additional anomalous points in the training set, beyond those already labeled by the data provider. The research utilizes a real-world dataset provided by EDP Renewables, encompassing two years of comprehensive SCADA records collected from a single offshore wind turbine operating in the Gulf of Guinea. Furthermore, the dataset contains the logs of failure events and recorded alarms triggered by the SCADA system across a wide range of subsystems. The paper proposes a multi-modal anomaly detection framework orchestrating an unsupervised module (i.e., decision tree method) with a supervised one (i.e., BI-LSTM AE). The results highlight the efficacy of the BI-LSTM autoencoder in accurately identifying anomalies within the SCADA data that exhibit strong temporal correlation with logged warnings and the actual failure events. The model’s performance is rigorously evaluated using standard machine learning metrics, including precision, recall, F1 Score, and accuracy, all of which demonstrate favorable results. Further analysis is conducted using Cumulative Sum (CUSUM) control charts to gain a deeper understanding of the identified anomalies’ behavior, particularly their persistence and timing leading up to the failures. Full article

Ship seismic wave signals represent one of the most critical physical field characteristics of vessels. To achieve the high-precision detection of ship seismic wave field signals in marine environments, an underwater seismic wave signal detection system was designed. The system adopts a three-stage architecture consisting of watertight instrument housing, a communication circuit, and a buoy to realize high-capacity real-time data transmissions. The host computer performs the collaborative optimization of multi-modal hardware architecture and adaptive signal processing algorithms, enabling the detection of ship targets in oceanic environments. Through verification in a water tank and sea trials, the system successfully measured seismic wave signals. An improved ALE-LOFAR (Adaptive Line Enhancer–Low-Frequency Analysis) joint framework, combined with DEMON (Demodulation of Envelope Modulation) demodulation technology, was proposed to conduct the spectral feature analysis of ship seismic wave signals, yielding the low-frequency signal characteristics of vessels. This scheme provides an important method for the covert monitoring of shallow-sea targets, providing early warnings of illegal fishing and ensuring underwater security. Full article

The risk of serious crashes is notably higher among young and novice drivers. This increased risk is due to several factors, including a lack of recognition of dangerous situations, an overestimation of driving skills, and vulnerability to peer pressure. Recently, advanced driver assistance systems (ADAS) have been integrated into vehicles to help mitigate crashes linked to these factors. While numerous studies have examined ADAS broadly, few have specifically investigated its effects on young and novice drivers. This study aimed to address that gap by exploring ADAS’s impact on these drivers. Most studies in this review conclude that ADAS is beneficial for young and novice drivers, though some research suggests its impact may be limited or even negligible. Tailoring ADAS to address the unique needs of young drivers could enhance both the system’s acceptance and reliability. The review also found that unimodal warnings (e.g., auditory or visual) are as effective as multimodal warnings. Of the different types of warnings, auditory and visual signals proved the most effective. Additionally, ADAS can influence young drivers’ car-following behavior; for instance, drivers may maintain greater safety buffers or drive closely to avoid alarm triggers, likely due to perceived system unreliability. Aggressive drivers tend to benefit most from active ADAS, which actively intervenes to assist the driver. Future research could explore the combined effects of multiple ADAS functions within a single vehicle on young and novice drivers to better understand how these systems interact and impact driver behavior. Full article

Driving warning systems are of great help in notifying emergencies. Based on the results of former studies as well as the multisensory integration effect (MIE), the current meta-analysis investigated the effectiveness of utilizing unimodal (i.e., auditory, visual, and tactile) and multimodal (i.e., bimodal and trimodal) driving warning systems in drivers’ response time. Sixty eligible articles representing 308 individual studies were included in this meta-analysis. The results showed: First, both auditory warnings (pooled Hedges’ g = 0.98, 95% CI: 0.34 to 1.61, p < 0.01) and tactile warnings (pooled Hedges’ g = 0.77, 95% CI: 0.22 to 1.32, p < 0.01) were found to reduce the response time significantly compared to no warning, but visual warnings did not produce significant benefit; Second, tactile warnings outperformed the visual warnings (pooled Hedges’ g = 0.74, 95% CI: 0.11 to 1.37, p < 0.05); Third, auditory-tactile bimodal warnings surpassed unimodal warnings (p < 0.05); Fourth, drivers’ response time under trimodal warning conditions were shorter than that under bimodal warning conditions but not in a significant level. Overall, the results support multisensory redundant signal effect hypothesis in multimodal conditions. Current study provides a quantitative understanding of the effectiveness of driving warnings and could contribute to the design of related technologies. Full article

In recent years, implantable sensors have been extensively researched since they allow localized sensing at an area of interest (e.g., within the vicinity of a surgical site or other implant). They allow unobtrusive and potentially continuous sensing, enabling greater specificity, early warning capabilities, and thus timely clinical intervention. Wireless remote interrogation of the implanted sensor is typically achieved using radio frequency (RF), inductive coupling or ultrasound through an external device. Two categories of implantable sensors are available, namely active and passive. Active sensors offer greater capabilities, such as on-node signal and data processing, multiplexing and multimodal sensing, while also allowing lower detection limits, the possibility to encode patient sensitive information and bidirectional communication. However, they require an energy source to operate. Battery implantation, and maintenance, remains a very important constraint in many implantable applications even though energy can be provided wirelessly through the external device, in some cases. On the other hand, passive sensors offer the possibility of detection without the need for a local energy source or active electronics. They also offer significant advantages in the areas of system complexity, cost and size. In this review, implantable passive sensor technologies will be discussed along with their communication and readout schemes. Materials, detection strategies and clinical applications of passive sensors will be described. Advantages over active sensor technologies will be highlighted, as well as critical aspects related to packaging and biocompatibility. Full article

Semi-autonomous vehicles (AVs) enable drivers to engage in non-driving tasks but require them to be ready to take control during critical situations. This “out-of-the-loop” problem demands a quick transition to active information processing, raising safety concerns and anxiety. Multimodal signals in AVs aim to deliver take-over requests and facilitate driver–vehicle cooperation. However, the effectiveness of auditory, visual, or combined signals in improving situational awareness and reaction time for safe maneuvering remains unclear. This study investigates how signal modalities affect drivers’ behavior using virtual reality (VR). We measured drivers’ reaction times from signal onset to take-over response and gaze dwell time for situational awareness across twelve critical events. Furthermore, we assessed self-reported anxiety and trust levels using the Autonomous Vehicle Acceptance Model questionnaire. The results showed that visual signals significantly reduced reaction times, whereas auditory signals did not. Additionally, any warning signal, together with seeing driving hazards, increased successful maneuvering. The analysis of gaze dwell time on driving hazards revealed that audio and visual signals improved situational awareness. Lastly, warning signals reduced anxiety and increased trust. These results highlight the distinct effectiveness of signal modalities in improving driver reaction times, situational awareness, and perceived safety, mitigating the “out-of-the-loop” problem and fostering human–vehicle cooperation. Full article

In case of dangerous driving, the in-vehicle robot can provide multimodal warnings to help the driver correct the wrong operation, so the impact of the warning signal itself on driving safety needs to be reduced. This study investigates the design of multimodal warnings for in-vehicle robots under driving safety warning scenarios. Based on transparency theory, this study addressed the content and timing of visual and auditory modality warning outputs and discussed the effects of different robot speech and facial expressions on driving safety. Two rounds of experiments were conducted on a driving simulator to collect vehicle data, subjective data, and behavioral data. The results showed that driving safety and workload were optimal when the robot was designed to use negative expressions for the visual modality during the comprehension (SAT 2) phase and speech at a rate of 345 words/minute for the auditory modality during the comprehension (SAT 2) and prediction (SAT 3) phases. The design guideline obtained from the study provides a reference for the interaction design of driver assistance systems with robots as the interface. Full article

In autonomous driving vehicles, the driver can engage in non-driving-related tasks and does not have to pay attention to the driving conditions or engage in manual driving. If an unexpected situation arises that the autonomous vehicle cannot manage, then the vehicle should notify and help the driver to prepare themselves for retaking manual control of the vehicle. Several effective notification methods based on multimodal warning systems have been reported. In this paper, we propose an advanced method that employs alarms for specific conditions by analyzing the differences in the driver’s responses, based on their specific situation, to trigger visual and auditory alarms in autonomous vehicles. Using a driving simulation, we carried out human-in-the-loop experiments that included a total of 38 drivers and 2 scenarios (namely drowsiness and distraction scenarios), each of which included a control-switching stage for implementing an alarm during autonomous driving. Reaction time, gaze indicator, and questionnaire data were collected, and electroencephalography measurements were performed to verify the drowsiness. Based on the experimental results, the drivers exhibited a high alertness to the auditory alarms in both the drowsy and distracted conditions, and the change in the gaze indicator was higher in the distraction condition. The results of this study show that there was a distinct difference between the driver’s response to the alarms signaled in the drowsy and distracted conditions. Accordingly, we propose an advanced notification method and future goals for further investigation on vehicle alarms. Full article

Entomological radars are important for scientific research of insect migration and early warning of migratory pests. However, insects are hard to detect because of their tiny size and highly maneuvering trajectory. Generalized Radon–Fourier transform (GRFT) has been proposed for effective weak maneuvering target detection by long-time coherent detection via jointly motion parameter search, but the heavy computational burden makes it impractical in real signal processing. Particle swarm optimization (PSO) has been used to achieve GRFT detection by fast heuristic parameter search, but it suffers from obvious detection probability loss and is only suitable for single target detection. In this paper, we convert the realization of GRFT into a multimodal optimization problem for insect multi-target detection. A novel niching method without radius parameter is proposed to detect unevenly distributed insect targets. Species reset and boundary constraint strategy are used to improve the detection performance. Simulation analyses of detection performance and computational cost are given to prove the effectiveness of the proposed method. Furthermore, real observation data acquired from a Ku-band entomological radar is used to test this method. The results show that it has better performance on detected target amount and track continuity in insect multi-target detection. Full article