Search Results (26)

As the demand for marine resource development escalates, underwater robots have gained prominence as a technological alternative to human participation in deep-sea exploration, resource assessments, and other intricate tasks, underscoring their academic and engineering importance. Traditional underwater robots, however, typically exhibit limited resilience to environmental disturbances and are readily obstructed or interfered with by aquatic vegetation, sediments, and other physical impediments. This paper examines the biological locomotion mechanisms of black ghostfish, which utilize undulatory fins and flapping wings, and presents a coupled undulatory-flapping propulsion strategy to facilitate rapid movement and precise posture adjustment in underwater robots. A multimodal undulatory-flapping bio-inspired underwater robotic platform is proposed, with a systematic explanation of its structure and motion principles. Additionally, kinematic and dynamic models for coordinated propulsion with multiple actuators are developed, and the robot’s performance under various driving modes is evaluated using computational fluid dynamics simulations. The simulation outcomes confirm the viability of the developed dynamic model. A prototype was constructed, and a PID-based control algorithm was developed to assess the robot’s performance in linear movement, turning, and other behaviors in both undulatory fin and flapping modes. Experimental findings indicate that the robot, functioning in undulatory fin propulsion mode at a frequency of 2.5 Hz, attains a velocity of 0.35 m/s, while maintaining attitude angle fluctuation errors within ±5°. In the flapping propulsion mode, precise posture modifications can be executed. These results validate the feasibility of the proposed multimodal bio-inspired underwater robot design and provide a new approach for the development of high-performance, autonomous bio-inspired underwater robots. Full article

This paper presents the design and evaluation of a sound-driven robot quiz system that enhances fairness and engagement in educational human–robot interaction (HRI). The system integrates a real-time sound-based first-responder detection mechanism with gamified multimodal feedback, including verbal cues, music, gestures, points, and badges. Motivational design followed the Octalysis framework, and the system was evaluated using validated scales from the Technology Acceptance Model (TAM), the Intrinsic Motivation Inventory (IMI), and the Godspeed Questionnaire. An experimental study was conducted with 32 university students comparing the proposed multimodal system combined with sound-driven first quiz responder detection to a sequential turn-taking quiz response with a verbal-only feedback system as a baseline. Results revealed significantly higher scores for the experimental group across perceived usefulness (M = 4.32 vs. 3.05, d = 2.14), perceived ease of use (M = 4.03 vs. 3.17, d = 1.43), behavioral intention (M = 4.24 vs. 3.28, d = 1.62), and motivation (M = 4.48 vs. 3.39, d = 3.11). The sound-based first-responder detection system achieved 97.5% accuracy and was perceived as fair and intuitive. These findings highlight the impact of fairness, motivational feedback, and multimodal interaction on learner engagement. The proposed system offers a scalable model for designing inclusive and engaging educational robots that promote active participation through meaningful and enjoyable interactions. Full article

The possibility of satisfying special geometric conditions, either through their architecture or through their configuration, makes a mechanism acquire changeable motion characteristics (kinematotropic or metamorphic behavior, multi-mode operation capability, etc.) that are of interest. Aligning revolute (R)-pair axes is one of such special conditions. In spherical linkages, only R-pairs, whose axes share a common intersection (spherical motion center (SMC)), are present. Investigating how R-pair axes can become collinear in a spherical mechanism leads to the identification of those that exhibit changeable motion features. This approach is adopted here to select non-redundant spherical mechanisms coming from regular polygons that are foldable/deployable and have a wide enough workspace for performing motion tasks. This analysis shows that the ones with hexagonal architecture prevail over the others. These results are exploitable in many contexts related to field robotics (aerospace, machines for construction sites, deployable antennas, etc.) Full article

Motion-capture data provide high accuracy but are difficult to obtain, necessitating dataset augmentation. To our knowledge, no prior study has investigated few-shot generative models for motion-capture data that address both quality and diversity. We tackle the diversity loss that arises with extremely small datasets (n ≤ 10) by applying transfer learning and continual learning to retain the rich variability of a larger pretraining corpus. To assess quality, we introduce MFMMD (Motion Feature-Based Maximum Mean Discrepancy)—a metric well-suited for small samples—and evaluate diversity with the multimodality metric. Our method embeds an Elastic Weight Consolidation (EWC)-based regularization term in the generator’s loss and then fine-tunes the limited motion-capture set. We analyze how the strength of this term influences diversity and uncovers motion-specific characteristics, revealing behavior that differs from that observed in image-generation tasks. The experiments indicate that the transfer learning pipeline improves generative performance in low-data scenarios. Increasing the weight of the regularization term yields higher diversity in the synthesized motions, demonstrating a marked uplift in motion diversity. These findings suggest that the proposed approach can effectively augment small motion-capture datasets with greater variety, a capability expected to benefit applications that rely on diverse human-motion data across modern robotics, animation, and virtual reality. Full article

Personality is a fundamental psychological trait that exerts a long-term influence on human behavior patterns and social interactions. Automatic personality recognition (APR) has exhibited increasing importance across various domains, including Human–Robot Interaction (HRI), personalized services, and psychological assessments. In this study, we propose a multimodal personality recognition model that classifies the Big Five personality traits by extracting features from three heterogeneous sources: audio processed using Wav2Vec2, video represented as Skeleton Landmark time series, and text encoded through Bidirectional Encoder Representations from Transformers (BERT) and Doc2Vec embeddings. Each modality is handled through an independent Self-Attention block that highlights salient temporal information, and these representations are then summarized and integrated using a late fusion approach to effectively reflect both the inter-modal complementarity and cross-modal interactions. Compared to traditional recurrent neural network (RNN)-based multimodal models and unimodal classifiers, the proposed model achieves an improvement of up to 12 percent in the F1-score. It also maintains a high prediction accuracy and robustness under limited input conditions. Furthermore, a visualization based on t-distributed Stochastic Neighbor Embedding (t-SNE) demonstrates clear distributional separation across the personality classes, enhancing the interpretability of the model and providing insights into the structural characteristics of its latent representations. To support real-time deployment, a lightweight thread-based processing architecture is implemented, ensuring computational efficiency. By leveraging deep learning-based feature extraction and the Self-Attention mechanism, we present a novel personality recognition framework that balances performance with interpretability. The proposed approach establishes a strong foundation for practical applications in HRI, counseling, education, and other interactive systems that require personalized adaptation. Full article

Bio-inspired design, a paradigm-shifting methodology that translates evolutionary mechanisms into engineering solutions, has established itself as a cornerstone for pioneering innovation in multifaceted technological systems. Despite its promise, the inherent complexity of biological systems and interdisciplinary knowledge gaps hinder the effective translation of biological principles into practical engineering solutions. This study introduces a structured framework integrating large language models (LLMs) with a function–behavior–characteristic–environment (F-B-C-E) paradigm to systematize biomimetic design processes. We propose a standardized F-B-C-E knowledge model to formalize biological strategy representations, coupled with a BERT-based pipeline for automated inversion of biological strategies into engineering applications. To optimize strategy selection, a hybrid decision-making methodology combining VIKOR multi-criteria analysis and rank correlation is developed. The framework’s functional robustness is validated via aquatic robotic system implementations, wherein three biomimetic propulsion modalities—oscillatory caudal propulsion, pulsed hydrodynamic thrust generation, and autonomous peristaltic locomotion—demonstrate quantifiable enhancements in locomotion efficiency and environmental adaptability metrics. These results underscore the robustness of the proposed inversion methodology in resolving intricate engineering problems through systematic biomimetic translation. Full article

Characterizing imaging performance requires a multidisciplinary approach that evaluates various interconnected parameters, including dosage optimization and dynamic accuracy. Radiation dose and dynamic accuracy are challenged by patient motion that results in poor image quality. These challenges are more prevalent in the brain/cardiac pediatric patient imaging, as they relate to excess radiation dose that may be associated with various complications. Scanning vulnerable pediatric patients ought to eliminate anesthesia due to critical risks associated in some cases with intracranial hemorrhages, brain strokes, and congenital heart disease. Some pediatric imaging, however, requires prolonged scanning under anesthesia. It can often be a laborious, suboptimal process, with limited field of view and considerable dose. High dynamic accuracy is also necessary to diagnose tissue’s dynamic behavior beyond its static structural morphology. This study presents several performance characterization experiments from a new robotic multimodal imaging system using specially designed calibration methods at different system configurations. Additional musculoskeletal imaging and imaging from a pediatric brain stroke patient without anesthesia are presented for comparisons. The findings suggest that the system’s large dynamically controlled gantry enables scanning at full patient movement and with important improvements in scan times, accuracy, radiation dose, and the ability to image brain structures without anesthesia. This could position the system as a potential transformative tool in the pediatric interventional imaging landscape. Full article

In recent years, artificial intelligence (AI) has increasingly utilized speech and video signals for emotion recognition, facial recognition, and depression detection, playing a crucial role in mental health assessment. However, the AI-driven research on detecting depression severity remains limited, and the existing models are often too large for lightweight deployment, restricting their real-time monitoring capabilities, especially in resource-constrained environments. To address these challenges, this study proposes a lightweight and accurate multimodal method for detecting depression severity, aiming to provide effective support for smart healthcare systems. Specifically, we design a multimodal detection network based on speech and video signals, enhancing the recognition of depression severity by optimizing the cross-modal fusion strategy. The model leverages Long Short-Term Memory (LSTM) networks to capture long-term dependencies in speech and visual sequences, effectively extracting dynamic features associated with depression. Considering the behavioral differences of respondents when interacting with human versus robotic interviewers, we train two separate sub-models and fuse their outputs using a Mixture of Experts (MOE) framework capable of modeling uncertainty, thereby suppressing the influence of low-confidence experts. In terms of the loss function, the traditional Mean Squared Error (MSE) is replaced with Negative Log-Likelihood (NLL) to better model prediction uncertainty and enhance robustness. The experimental results show that the improved AI model achieves an accuracy of 83.86% in depression severity recognition. The model’s floating-point operations per second (FLOPs) reached 0.468 GFLOPs, with a parameter size of only 0.52 MB, demonstrating its compact size and strong performance. These findings underscore the importance of emotion and facial recognition in AI applications for mental health, offering a promising solution for real-time depression monitoring in resource-limited environments. Full article

Artificial agents are expected to increasingly interact with humans and to demonstrate multimodal adaptive emotional responses. Such social integration requires both perception and production mechanisms, thus enabling a more realistic approach to emotional alignment than existing systems. Indeed, existing emotion recognition methods rely on behavioral signals, predominantly facial expressions, as well as non-invasive brain recordings, such as Electroencephalograms (EEGs) and functional Magnetic Resonance Imaging (fMRI), to identify humans’ emotions, but accurate labeling remains a challenge. This paper introduces a novel approach examining how behavioral and physiological signals can be used to predict activity in emotion-related regions of the brain. To this end, we propose a multimodal deep learning network that processes two categories of signals recorded alongside brain activity during conversations: two behavioral signals (video and audio) and one physiological signal (blood pulse). Our network enables (1) the prediction of brain activity from these multimodal inputs, and (2) the assessment of our model’s performance depending on the nature of interlocutor (human or robot) and the brain region of interest. Results demonstrate that the proposed architecture outperforms existing models in anterior insula and hypothalamus regions, for interactions with a human or a robot. An ablation study evaluating subsets of input modalities indicates that local brain activity prediction was reduced when one or two modalities are omitted. However, they also revealed that the physiological data (blood pulse) achieve similar levels of predictions alone compared to the full model, further underscoring the importance of somatic markers in the central nervous system’s processing of social emotions. Full article

Computational thinking (CT) currently has been mainly explored using programming robots and conducted in K12 education. In early childhood education, arts have a significant place in children’s learning, expression and cognitive development. Specifically, creating pop-up storybooks is a child-friendly activity. Our study aimed to explore the combination of CT and art, and to develop talented children’s CT abilities through a multimodal literacies educational approach, which referred to using different skills (i.e., writing, drawing, making, and storytelling) in the art activity. A total of 12 talented children were selected to participate in a pop-up storybook production workshop using a convenience sampling method. We adopted an observation method to capture talented children’s CT behaviors, generating a total of 180 min of activity videos and collecting fieldnotes and the children’s worksheets and artworks for the data analysis. Based on a content analysis, we found that talented children enhanced their CT development in multiple modes and practiced seven CT skills. In conclusion, our study emphasizes the importance of art in children’s education and provides new insights for subsequent research on children’s CT education. Full article

Odor source localization (OSL) technology allows autonomous agents like mobile robots to localize a target odor source in an unknown environment. This is achieved by an OSL navigation algorithm that processes an agent’s sensor readings to calculate action commands to guide the robot to locate the odor source. Compared to traditional ‘olfaction-only’ OSL algorithms, our proposed OSL algorithm integrates vision and olfaction sensor modalities to localize odor sources even if olfaction sensing is disrupted by non-unidirectional airflow or vision sensing is impaired by environmental complexities. The algorithm leverages the zero-shot multi-modal reasoning capabilities of large language models (LLMs), negating the requirement of manual knowledge encoding or custom-trained supervised learning models. A key feature of the proposed algorithm is the ‘High-level Reasoning’ module, which encodes the olfaction and vision sensor data into a multi-modal prompt and instructs the LLM to employ a hierarchical reasoning process to select an appropriate high-level navigation behavior. Subsequently, the ‘Low-level Action’ module translates the selected high-level navigation behavior into low-level action commands that can be executed by the mobile robot. To validate our algorithm, we implemented it on a mobile robot in a real-world environment with non-unidirectional airflow environments and obstacles to mimic a complex, practical search environment. We compared the performance of our proposed algorithm to single-sensory-modality-based ‘olfaction-only’ and ‘vision-only’ navigation algorithms, and a supervised learning-based ‘vision and olfaction fusion’ (Fusion) navigation algorithm. The experimental results show that the proposed LLM-based algorithm outperformed the other algorithms in terms of success rates and average search times in both unidirectional and non-unidirectional airflow environments. Full article

Since the emergence of ChatGPT, research on large language models (LLMs) has actively progressed across various fields. LLMs, pre-trained on vast text datasets, have exhibited exceptional abilities in understanding natural language and planning tasks. These abilities of LLMs are promising in robotics. In general, traditional supervised learning-based robot intelligence systems have a significant lack of adaptability to dynamically changing environments. However, LLMs help a robot intelligence system to improve its generalization ability in dynamic and complex real-world environments. Indeed, findings from ongoing robotics studies indicate that LLMs can significantly improve robots’ behavior planning and execution capabilities. Additionally, vision-language models (VLMs), trained on extensive visual and linguistic data for the vision question answering (VQA) problem, excel at integrating computer vision with natural language processing. VLMs can comprehend visual contexts and execute actions through natural language. They also provide descriptions of scenes in natural language. Several studies have explored the enhancement of robot intelligence using multimodal data, including object recognition and description by VLMs, along with the execution of language-driven commands integrated with visual information. This review paper thoroughly investigates how foundation models such as LLMs and VLMs have been employed to boost robot intelligence. For clarity, the research areas are categorized into five topics: reward design in reinforcement learning, low-level control, high-level planning, manipulation, and scene understanding. This review also summarizes studies that show how foundation models, such as the Eureka model for automating reward function design in reinforcement learning, RT-2 for integrating visual data, language, and robot actions in vision-language-action models, and AutoRT for generating feasible tasks and executing robot behavior policies via LLMs, have improved robot intelligence. Full article

Robotic walking devices can be used for intensive exercises to enhance gait rehabilitation therapies. Mixed Reality (MR) techniques may improve engagement through immersive and interactive environments. This article introduces an MR-based multimodal human–robot interaction strategy designed to enable shared control with a Smart Walker. The MR system integrates virtual and physical sensors to (i) enhance safe navigation and (ii) facilitate intuitive mobility training in personalized virtual scenarios by using an interface with three elements: an arrow to indicate where to go, laser lines to indicate nearby obstacles, and an ellipse to show the activation zone. The multimodal interaction is context-based; the presence of nearby individuals and obstacles modulates the robot’s behavior during navigation to simplify collision avoidance while allowing for proper social navigation. An experiment was conducted to evaluate the proposed strategy and the self-explanatory nature of the interface. The volunteers were divided into four groups, with each navigating under different conditions. Three evaluation methods were employed: task performance, self-assessment, and observational measurement. Analysis revealed that participants enjoyed the MR system and understood most of the interface elements without prior explanation. Regarding the interface, volunteers who did not receive any introductory explanation about the interface elements were mostly able to guess their purpose. Volunteers that interacted with the interface in the first session provided more correct answers. In future research, virtual elements will be integrated with the physical environment to enhance user safety during navigation, and the control strategy will be improved to consider both physical and virtual obstacles. Full article

Prey-predator interactions play a pivotal role in elucidating the evolution and adaptation of various organism’s traits. Numerous approaches have been employed to study the dynamics of prey-predator interaction systems, with agent-based methodologies gaining popularity. However, existing agent-based models are limited in their ability to handle multi-modal interactions, which are believed to be crucial for understanding living organisms. Conversely, prevailing prey-predator integration studies often rely on mathematical models and computer simulations, neglecting real-world constraints and noise. These elusive attributes, challenging to model, can lead to emergent behaviors and embodied intelligence. To bridge these gaps, our study designs and implements a prey-predator interaction scenario that incorporates visual and olfactory sensory cues not only in computer simulations but also in a real multi-robot system. Observed emergent spatial-temporal dynamics demonstrate successful transitioning of investigating prey-predator interactions from virtual simulations to the tangible world. It highlights the potential of multi-robotics approaches for studying prey-predator interactions and lays the groundwork for future investigations involving multi-modal sensory processing while considering real-world constraints. Full article

Simulation and optimization have become common tasks in engineering practice due to their advantages, namely cost reduction and unlimited testing prior to manufacturing. Over the last few years, personal computers have become powerful enough to run complex simulations. On the other hand, the industry has seen an increase in automation, where repetitive tasks carried out by humans in the past are gradually being replaced by robotic systems. Those robotic systems usually involve a robotic arm, a gripper, and a control system. This article presents a methodology for the simulation and optimization of existing engineering parts, i.e., based on reverse engineering. The models were subjected to static loadings and free vibration (modal) analysis in the finite element method (FEM) software ANSYS Workbench 2021 R2. The adaptive multi-objective optimization algorithm was also applied in ANSYS Workbench 2021 R2. The effectiveness of the proposed methodology was evaluated, and the outcome was that significant improvement could be achieved in terms of both the static and dynamic behavior of the analyzed part. Full article