Search Results (186)

In response to the redundant spatio-temporal modeling and insufficient adaptation to dynamic decision-making in eye-tracking interaction of smart home interfaces, a smart home interface eye-tracking response optimization model based on spatio-temporal Transformer and gate control cross-attention is proposed. It adapts the physiological characteristics of eye-tracking jumps through dynamic sparse attention gating to compress computational redundancy and combines multi-objective reinforcement learning attention modulation to construct a closed-loop decision-making mechanism, optimizing interface parameters in real-time. Experiments showed that the model reduced eye-tracking trajectory prediction error by 23.7% compared to advanced benchmarks, increased the success rate of adapting to dynamic mutation scenarios to 89.2%, and controlled performance fluctuations within 2.3% under noise interference. In high-fidelity user testing, the accuracy of cross-task gaze transfer reached 93.4%, the failure rate of glare interference was optimized to 2.4%, and the user cognitive load index was reduced by 27.9%. Its resource consumption and energy consumption were reduced by 26.7% and 44.9%, respectively, while its posture deviation tolerance remained at 3.5°. The sparse spatio-temporal modeling of the spatio-temporal adaptive Transformer module and the enhanced gating mechanism of the hierarchical gated cross-attention module work together to break through the limitations of traditional methods in computational efficiency and dynamic feedback, providing high-precision and low-latency eye-tracking interaction solutions for smart home interface systems, and promoting the practical evolution of personalized human–machine collaborative control. Full article

Gaze estimation plays a crucial role in human-computer interaction and behavior analysis. However, in dynamic scenes, rigid head movements and rapid gaze shifts pose significant challenges to accurate gaze prediction. Most existing methods either process single-frame images independently or rely on long video sequences, making it difficult to simultaneously achieve strong performance and high computational efficiency. To address this issue, we propose DGAGaze, a gaze estimation framework based on a difference-driven spatiotemporal attention mechanism. This framework uses a geometry-aware temporal alignment module to mitigate interference from rigid head movements, compensating for them through pose estimation and affine feature warping, thereby achieving explicit decoupling between global head motion and local eye motion. Based on the aligned features, inter-frame differences are used to adjust spatial and channel attention weights, enhancing motion-sensitive representations without introducing an additional temporal modeling layer. Extensive experiments on the EyeDiap and Gaze360 datasets demonstrate the effectiveness of the proposed approach. DGAGaze achieves improved gaze estimation accuracy while maintaining a lightweight architecture based on a ResNet-18 backbone, outperforming existing state-of-the-art methods. Full article

Postural control relies on the integration of visual, vestibular, and somatosensory inputs under biomechanical constraints. Conventional Romberg testing provides limited quantitative insight, particularly regarding directional control and sensory dependence. Wearable inertial measurement units (IMUs) enable portable, multidimensional assessment of postural sway. Thirty healthy adults (15 females, 15 males) completed a modified Romberg protocol with systematic manipulation of stance (normal, tandem), visual condition (eyes open, eyes closed), and arm position (arms at sides, arms forward), including both left and right leading foot during tandem stance. Whole-body kinematics were recorded using a full-body IMU system comprising 17 wireless sensors. Center-of-mass (CoM) trajectories were derived from a 23-segment biomechanical model, and linear, spatial, and nonlinear sway metrics were computed. Statistical analyses were conducted using repeated-measures ANOVA, with significance set at p < 0.05. Visual deprivation significantly increased sway path length, mean sway velocity, and sway area across all stance conditions (p < 0.001). Tandem stance elicited greater mediolateral sway than normal stance (p < 0.001). Romberg ratios exceeded unity for all metrics and were significantly higher in tandem stance (p < 0.01). Arm position effects were negligible in normal stance but showed significant Vision × Arm interactions during tandem stance (p < 0.05). Leading foot position had no significant main effects. Combining a modified Romberg protocol with full-body IMU-based CoM analysis enables sensitive characterization of sensory dependence and directional postural control. Tandem stance with visual deprivation increases mediolateral postural demands under reduced base-of-support conditions, providing a more challenging context for evaluating directional postural control. Full article

Eye-tracking is a key enabling technology for smart eyewear, supporting hands-free interaction, accessibility, and context-aware human–machine interfaces under strict constraints on size, power consumption, and computational complexity. While camera-based solutions provide high accuracy, their integration into lightweight and low-power wearable platforms remains challenging. This paper is a feasibility study for the design, simulation, and experimental evaluation of a photosensor oculography (PSOG) eye-tracking system that is fully integrated into an eyewear frame, based on near-infrared (NIR) emitters and photodiodes. The proposed approach combines simulation-driven optimization of the optical constellation, a multi-frequency modulation and demodulation scheme enabling parallel source discrimination and robust ambient-light rejection, and a resource-efficient signal acquisition pipeline suitable for embedded implementation. Eye rotations in azimuth and elevation are inferred from differential reflectance patterns of ocular regions (sclera, iris, and pupil) using lightweight regression techniques, including shallow neural networks and Gaussian process regression, selected to balance estimation accuracy with computational and power constraints. System performance is evaluated using a controllable artificial-eye platform under defined geometric and illumination conditions, enabling repeatable assessment of gaze-estimation accuracy and algorithmic behavior. Sub-degree errors are achieved in this controlled setting, demonstrating the feasibility and potential effectiveness of the proposed architecture. Practical considerations for translation to real-world smart eyewear, including human-subject validation, anatomical variability, calibration strategies, and embedded deployment, are discussed and identified as directions for future work. By detailing the optical design methodology, modulation strategy, and algorithmic trade-offs, this work clarifies the distinct contributions of the proposed PSOG system relative to existing frame-integrated and camera-free eye-tracking approaches, and provides a foundation for further development toward wearable and augmented-reality applications. Full article

Background/Objectives: Successful interception of moving targets requires combining predictive control, which anticipates future target states, and reactive control, which compensates for ongoing sensory discrepancies. How these components evolve over time and are distributed across gaze and manual behavior remains unclear. We aimed to explore the time-resolved dynamics of predictive control during continuous interception and to dissociate eye and hand contributions. Methods: Human participants intercepted a moving target in a two-dimensional arena using a joystick while eye movements were recorded. Target speed was systematically varied, and visual information was selectively reduced by occluding either the target or the user-controlled cursor. Predictive control was assessed using two complementary metrics: a geometric strategy index capturing moment-to-moment spatial lead or lag relative to target motion, applied separately to gaze and manual trajectories, and root mean square error (RMSE) computed relative to current and forward-shifted target positions to quantify predictive alignment. Results: Successful interception was characterized by structured, speed-dependent transitions between predictive and reactive control rather than a fixed strategy. Predictive alignment emerged early and was dynamically reweighted as temporal constraints increased. Gaze and manual behavior showed complementary but partially dissociable predictive signatures. Occluding the target decreased predictive alignment, whereas occluding the user-controlled cursor had comparatively minor effects, indicating strong reliance on internal state estimation rather than continuous visual feedback of the effector. Conclusions: Predictive and reactive control are continuously and dynamically reweighted during interception. Their interaction unfolds within single trials and depends on target dynamics and sensory availability. These findings provide quantitative evidence for time-resolved coordination between anticipatory and feedback-driven control mechanisms in goal-directed behavior. Full article

Virtual Reality (VR) has emerged as a powerful medium for immersive human–computer interaction, where users’ emotional and experiential states play a pivotal role in shaping engagement and perception. However, existing affective computing approaches often model emotion recognition and immersion estimation as independent problems, overlooking their intrinsic coupling and the structured relationships underlying multimodal physiological signals. In this work, we propose a modality-aware multi-task learning framework that jointly models emotion recognition and immersion estimation from a graph-structured and symmetry-aware interaction perspective. Specifically, heterogeneous physiological and behavioral modalities—including eye-tracking, electrocardiogram (ECG), and galvanic skin response (GSR)—are treated as relational components with structurally symmetric encoding and fusion mechanisms, while their cross-modality dependencies are adaptively aggregated to preserve interaction symmetry at the representation level and introduce controlled asymmetry at the task-optimization level through weighted multi-task learning, without introducing explicit graph neural network architectures. To support reproducible evaluation, the VREED dataset is further extended with quantitative immersion annotations derived from presence-related self-reports via weighted aggregation and factor analysis. Extensive experiments demonstrate that the proposed framework consistently outperforms recurrent, convolutional, and Transformer-based baselines. Compared with the strongest Transformer baseline, the proposed framework yields consistent relative performance gains of approximately 3–7% for emotion recognition metrics and reduces immersion estimation errors by nearly 9%. Beyond empirical improvements, this study provides a structured interpretation of multimodal affective modeling that highlights symmetry, coupling, and controlled symmetry breaking in multi-task learning, offering a principled foundation for adaptive VR systems, emotion-driven personalization, and dynamic user experience optimization. Full article

Eye tracking has become a central method in human–computer interaction (HCI), supported by advances in sensing technologies and AI-based gaze analysis. Despite this rapid growth, a comprehensive and up-to-date overview of eye-tracking research across the broader HCI landscape remains lacking. This study combines records from Web of Science (WoS) and Scopus to analyse 1033 publications on eye tracking in HCI published between 2020 and 2025. After merging and deduplicating the datasets, we conducted bibliometric network analyses (keyword co-occurrence, co-citation, co-authorship, and source mapping) using VOSviewer and performed a qualitative content analysis of the 50 most-cited papers. The literature is dominated by journal articles and conference papers produced by small- to medium-sized research teams (mean: 3.9 authors per paper; h-index: 29). Keyword and overlay visualisations reveal four principal research axes: deep-learning-based gaze estimation; XR-related interaction paradigms within HCI; cognitive load and human factors; and usability- and accessibility-oriented interface design. The most-cited studies focus on gaze interaction in immersive environments, deep learning for gaze estimation, multimodal interaction, and physiological approaches to assessing cognitive load. Overall, the findings indicate that eye tracking in HCI is evolving from a measurement-oriented technique into a core enabling technology that supports interaction design, cognitive assessment, accessibility, and ethical considerations such as privacy. This review identifies research gaps and outlines future directions for benchmarking practices, real-world deployments, and privacy-preserving gaze analytics in HCI. Full article

This study examines the integration of affective computing within immersive environments, virtual reality (VR), augmented reality (AR), and mixed reality (MR), to support affective user experience (AUX) in individuals with autism spectrum disorder (ASD). Twenty-eight published empirical studies were analyzed following PRISMA guidelines, focusing on affective modalities, immersive technologies, methodological approaches, and intervention outcomes. Results indicate that immersive systems increasingly incorporate physiological sensing, eye-tracking, behavioral analytics, and, to a lesser extent, facial and speech recognition. Although 89% of studies rely on unimodal affective signals, emerging multimodal frameworks demonstrate enhanced adaptability and real-time emotional responsiveness. VR remains the predominant platform due to its high immersive capacity and controlled manipulation of social stimuli, while AR support interaction in everyday contexts, offering higher accessibility. Across studies, immersive affective systems show consistent benefits in emotion recognition, anxiety reduction, engagement, and social communication. However, the field is limited by small sample sizes, restricted real-world contextual relevance, and a lack of standardized AUX evaluation frameworks. This review identifies methodological gaps and proposes future research directions involving adaptive affective systems, low-cost sensors, and inclusive, longitudinal designs aimed at achieving emotionally intelligent, scalable, and context-aware immersive interventions for people with ASD. Full article

Parkinson’s disease (PD) is a multifactorial neurodegenerative disorder characterized by complex interactions across molecular, cellular, circuit, and behavioral scales. While experimental and clinical studies have provided critical insights into PD pathology, integrating these heterogeneous data into coherent mechanistic frameworks and translational strategies remains a major challenge. Computational modeling offers a powerful approach to bridge these scales, enabling the systematic investigation of disease mechanisms, candidate biomarkers, and therapeutic strategies. In this review, we survey state-of-the-art computational approaches applied to PD, spanning molecular dynamics and biophysical models, cellular- and circuit-level network models, systems and abstract-level simulations of basal ganglia function, and whole-brain and data-driven models linked to clinical phenotypes. We highlight how multiscale and hybrid modeling strategies connect α-synuclein pathology, mitochondrial dysfunction, oxidative stress, and dopaminergic degeneration to alterations in neural dynamics and motor and non-motor symptoms. We further discuss the role of computational models in biomarker discovery, including imaging, electrophysiological, and digital biomarkers. In particular, eye-movement-based measures are highlighted as quantitative, reproducible behavioral signals that provide principled constraints for individualized computational modeling. We also review the emerging impact of computational approaches on drug discovery, target prioritization, and in silico clinical trials. Finally, we examine future directions toward personalized and precision medicine in PD, emphasizing digital twin frameworks, longitudinal validation, and the integration of patient-specific data with mechanistic and data-driven models. Together, these advances underscore the growing role of computational modeling as an integrative and hypothesis-generating framework, with the long-term goal of supporting data-constrained predictive approaches for biomarker development and translational applications. Full article

Understanding how tacit knowledge embedded in visual materials is accessed and utilized during evaluation tasks remains a key challenge in human–computer interaction and visual expertise research. Although eye-tracking studies have identified systematic differences between experts and novices, findings remain inconsistent, particularly in art-related visual evaluation contexts. This study examines whether tacit aspects of visual evaluation can be inferred from gaze behavior by comparing individuals with and without formal art education. Visual evaluation was assessed using a structured, prompt-based task in which participants inspected artistic images and responded to items targeting specific visual elements. Eye movements were recorded using a screen-based eye-tracking system. Areas of Interest (AOIs) corresponding to correct-answer regions were defined a priori based on expert judgment and item prompts. Both AOI-level metrics (e.g., fixation count, mean, and total visit and gaze durations) and image-level metrics (e.g., fixation count, saccade count, and pupil size) were analyzed using appropriate parametric and non-parametric statistical tests. The results showed that participants with an art-education background produced more fixations within AOIs, exhibited longer mean and total AOI visit and gaze durations, and demonstrated lower saccade counts than participants without art education. These patterns indicate more systematic and goal-directed gaze behavior during visual evaluation, suggesting that formal art education may shape tacit visual evaluation strategies. The findings also highlight the potential of eye tracking as a methodological tool for studying expertise-related differences in visual evaluation processes. Full article

This paper presents the design and feasibility evaluation of a low-cost, head-mounted pupil dilation tracking system based on computer vision. The proposed solution employs a standard webcam and active infrared illumination, enabling stable eye image acquisition under controlled lighting conditions. The developed image processing pipeline incorporates adaptive contrast enhancement and geometric pupil detection, allowing for the estimation of relative changes in pupil diameter in real time. System evaluation was conducted in a controlled experiment involving 24 participants performing an N-back task with emotional modulation, a well-established paradigm for eliciting task-evoked pupillary responses under constant working-memory demands. The results revealed statistically significant changes in relative pupil dilation in response to stimuli with varying emotional valence during a working memory task, confirming the system’s ability to capture task-evoked pupillary responses (TEPRs). The proposed system constitutes a low-cost research tool for studies of task engagement and physiological responses in the context of human–computer interaction and psychophysiology, with a focus on the analysis of functional pupilometric changes. Full article

Understanding how learners process data visualizations with seductive details is essential for improving comprehension and engagement. This study examined the influence of task-relevant and task-irrelevant seductive details on attentional distribution and comprehension in the context of data story learning, using COVID-19 data visualizations as experimental materials. A gaze-based methodology was applied, using eye-movement data and saliency maps to visualize learners’ attentional patterns while processing bar graphs with varying embellishments. Results showed that task-relevant seductive details supported comprehension for learners with higher visuospatial abilities by guiding attention toward textual information, while task-irrelevant details hindered comprehension, particularly for those with lower visuospatial abilities who focused disproportionately on visual elements. Working memory capacity emerged as a significant predictor of attentional distribution. Additionally, repeated exposure to data visualizations enhanced participants’ ability to recognize visualization types, improving efficiency and reducing reliance on legends and supplementary text. Overall, this study highlights the cognitive mechanisms underlying visualization processing in data story learning and provides practical implications for education, human–computer interaction, and adaptive technology design, emphasizing the importance of tailoring visualization strategies to individual learner differences. Full article

This study presents DeepVisionAnalytics, an integrated framework that combines eye tracking, OpenCV-based computer vision (CV), and machine learning (ML) to support objective analysis of consumer behaviour in visually driven tasks. Unlike conventional self-reported surveys, which are prone to cognitive bias, recall errors, and social desirability effects, the proposed approach relies on direct behavioural measurements of visual attention. The system captures gaze distribution and fixation dynamics during interaction with products or interfaces. It uses AOI-level eye tracking metrics as the sole behavioural signal to infer candidate choice under constrained experimental conditions. In parallel, OpenCV and ML perform facial analysis to estimate demographic attributes (age, gender, and ethnicity). These attributes are collected independently and linked post hoc to gaze-derived outcomes. Demographics are not used as predictive features for choice inference. Instead, they are used as contextual metadata to support stratified, segment-level interpretation. Empirical results show that gaze-based inference closely reproduces observed choice distributions in short-horizon, visually driven tasks. Demographic estimates enable meaningful post hoc segmentation without affecting the decision mechanism. Together, these results show that multimodal integration can move beyond descriptive heatmaps. The platform produces reproducible decision-support artefacts, including AOI rankings, heatmaps, and segment-level summaries, grounded in objective behavioural data. By separating the decision signal (gaze) from contextual descriptors (demographics), this work contributes a reusable end-to-end platform for marketing and UX research. It supports choice inference under constrained conditions and segment-level interpretation without demographic priors in the decision mechanism. Full article

This paper presents a novel hybrid multimodal deep learning model for robust and real-time eye gaze estimation. Accurate gaze tracking is essential for advancing human–computer interaction (HCI) and assistive technologies, but existing methods often struggle with environmental variations, require extensive calibration, and are computationally intensive. Our proposed model, GazeNet-HM, addresses these limitations by synergistically fusing features from RGB, depth, and infrared (IR) imaging modalities. This multimodal approach allows the model to leverage complementary information: RGB provides rich texture, depth offers invariance to lighting and aids pose estimation, and IR ensures robust pupil detection. Furthermore, we introduce a personalized adaptation module that dynamically fine-tunes the model to individual users with minimal calibration data. To ensure practical deployment, we employ advanced model compression techniques, enabling real-time inference on resource-constrained embedded systems. Extensive evaluations on public datasets (MPIIGaze, EYEDIAP, Gaze360) and our collected M-Gaze dataset demonstrate that GazeNet-HM achieves state-of-the-art performance, reducing the mean angular error by up to 27.1% compared to leading unimodal methods. After model compression, the system achieves a real-time inference speed of 32 FPS on an embedded Jetson Xavier NX platform. Ablation studies confirm the contribution of each modality and component, highlighting the effectiveness of our holistic design. Full article

Background: Amyotrophic lateral sclerosis (ALS) progressively impairs motor function, compromising speech and limiting communication. Augmentative and alternative communication (AAC) is essential to maintain autonomy, social participation, and quality of life for people with ALS (PALS). This review maps technological developments in AAC, from low-tech tools to advanced brain–computer interface (BCI) systems. Methods: We conducted a scoping review following the PRISMA extension for scoping reviews. PubMed, Web of Science, SciELO, MEDLINE, and CINAHL were screened for studies published up to 31 August 2025. Peer-reviewed RCT, cohort, cross-sectional, and conference papers were included. Single-case studies of invasive BCI technology for ALS were also considered. Methodological quality was evaluated using JBI Critical Appraisal Tools. Results: Thirty-seven studies met inclusion criteria. High-tech AAC—particularly eye-tracking systems and non-invasive BCIs—were most frequently studied. Eye tracking showed high usability but was limited by fatigue, calibration demands, and ocular impairments. EMG- and EOG-based systems demonstrated promising accuracy and resilience to environmental factors, though evidence remains limited. Invasive BCIs showed the highest performance in late-stage ALS and locked-in syndrome, but with small samples and uncertain long-term feasibility. No studies focused exclusively on low-tech AAC interventions. Conclusions: AAC technologies, especially BCIs, EMG and eye-tracking systems, show promise in supporting autonomy in PALS. Implementation gaps persist, including limited attention to caregiver burden, healthcare provider training, and the real-world use of low-tech and hybrid AAC. Further research is needed to ensure that communication solutions are timely, accessible, and effective, and that they are tailored to functional status, daily needs, social participation, and interaction with the environment. Full article