This article is concerned with digital, multimodal feedback that supports learning and assessment within education. Drawing on the research literature alongside a case study from a postgraduate program in digital education, I argue that approaching f...
Digital musical instruments have become increasingly prevalent in musical creation and production. Optimizing their usability and, particularly, their expressiveness, has become essential to their study and practice. The absence of multimodal feedbac...
Most of the information a user obtains through a computer is visual and/or auditory. Providing synchronized haptic information in addition to visual and/or auditory information can significantly enhance user experience and perception of virtual objec...
This paper presents the design and evaluation of a sound-driven robot quiz system that enhances fairness and engagement in educational human–robot interaction (HRI). The system integrates a real-time sound-based first-responder detection mechan...
Written corrective feedback (CF) could pave the way for L2 development, especially when embedded in multimodality. Building on prior research, this descriptive study drew a relationship between specific types of errors that were most successfully rev...
While technology has helped improve process efficiency in several domains, it still has an outstanding debt to education. In this article, we introduce NAIRA, a Multimodal Learning Analytics platform that provides Real-Time Feedback to foster collabo...
Historically, the field of discourse marker research has moved from relying on intuition to more and more ecological data, with written, spoken, and now multimodal corpora available to study these pervasive pragmatic devices. For some topics, video i...
Recent studies have indicated that terahertz time-domain spectroscopy (THz-TDS) can stably and efficiently acquire output spectra using an affordable and compact multimode laser diode (MMLD) with delayed optical feedback as the light source. This res...
Pumping stations have undergone significant modernization and digitalization in recent decades. However, traditional virtual inspections often prioritize the visual experience and fail to effectively represent the haptic physical properties of device...
By performing motor-imagery tasks, for example, imagining hand movements, Motor-Imagery based Brain-Computer Interfaces (MI-BCIs) users can control digital technologies, for example, neuroprosthesis, using their brain activity only. MI-BCI users need...
In recent years, cross-modal retrieval has garnered significant attention due to its potential to bridge heterogeneous data modalities, particularly in aligning visual content with natural language. Despite notable progress, existing methods often st...
Reliable detection of contact states, such as the “mating” of connectors, is crucial for high-quality teleoperated assembly. Conventional systems relying solely on visual and continuous force feedback often fail to convey these discrete h...
Developments in sensor technology, artificial intelligence, and network technologies like 5G has made remote operation a valuable method of controlling various types of machinery. The benefits of remote operations come with an opportunity to access h...
Road safety for vulnerable road users, particularly cyclists, remains a critical global issue. This study explores the potential of multimodal visual and haptic interaction technologies to improve cyclists’ perception of and responsiveness to t...
The role of the sense of touch in Human–Computer Interaction as a channel for feedback in manipulative processes is investigated through the research presented here. The paper discusses how information and feedback as generated by the computer...
The study of extreme events (EEs) in photonics has expanded significantly due to straightforward implementation conditions. EEs have not been discussed systematically, to the best of our knowledge, in the chaotic dynamics of a Fabry–Perot laser...
In this paper, we explore an approach to feedback which could allow those learning creative digital media practices in remote and asynchronous environments to receive rich, multi-modal, and interactive feedback upon their creative artefacts. We propo...
Deep reinforcement learning (DRL) and large foundation models (FMs) have reshaped modern artificial intelligence (AI) by enabling systems that learn from interaction while leveraging broad generalization and multimodal reasoning capabilities. This su...
The article presents a contextual survey of eight contributions in the special issue Musical Interactions (Volume I) in Multimodal Technologies and Interaction. The presentation includes (1) a critical examination of what it means to be musical, to d...
As autonomous driving technologies advance, the need for human-in-the-loop systems becomes increasingly critical to ensure safety, adaptability, and public confidence. This paper presents the design and evaluation of a context-aware immersive teleope...
Individuals with disabilities and persons operating in inaccessible environments can greatly benefit from the aid of robotic manipulators in performing daily living activities and other remote tasks. Users relying on robotic manipulators to interact...
Immersive virtual reality (VR) systems are becoming widely used for education with three-dimensional (3D) information. Specifically, three-dimensional spaces to create virtual environments can help increase students’ learning interest and abili...
Industry 4.0 has intensified the skills gap in industrial automation education, with graduates requiring extended on boarding periods and supplementary training investments averaging USD 11,500 per engineer. This paper introduces VisFactory, a multim...
Dexterous robotic grasping with multifingered hands remains a critical challenge in non-visual environments, where diverse object geometries and material properties demand adaptive force modulation and tactile-aware manipulation. To address this, we...
Robotic-assisted surgery (RAS) provides enhanced dexterity and visualisation but remains constrained by the absence of clinically meaningful palpation and haptic feedback. This perspective examines palpation sensing in RAS from an engineering and sys...
This paper describes flight planning supported by modeling, guidance, and feedback control for an electric Vertical Take-Off and Landing (eVTOL) QuadPlane small Uncrewed Aircraft System (sUAS). Five Lift+Cruise sUAS waypoint types are defined and use...
The evolution of Human–Computer Interaction (HCI) has laid the foundation for more immersive and dynamic forms of communication between humans and machines. Building on this trajectory, this work introduces a significant advancement in the doma...
This paper proposes a kinesthetic–tactile fusion feedback system based on virtual interaction. Combining the results of human fingertip deformation characteristics analysis and an upper limb motion mechanism, a fingertip tactile feedback device...
Most haptic actuators available on the market today can generate only a single modality of stimuli. This ultimately limits the capacity of a kinaesthetic haptic controller to deliver more expressive feedback, requiring a haptic controller to integrat...
A complete set of diverse haptic feedbacks is essential for a highly realistic and immersive virtual environment. In this sense, a multi-mode haptic interface that simultaneously generates multiple kinds of haptic signals is highly desirable. In this...
Objective: In this study, we introduce a multi-modal sensing and feedback framework aimed at assisting clinicians during endovascular surgeries and catheterization procedures. This framework utilizes state-of-the-art imaging and sensing sub-systems t...
The rapid advancement of flexible sensor technology has profoundly transformed neural interface research, enabling multimodal information acquisition, real-time neurochemical and electrophysiological signal monitoring, and adaptive closed-loop regula...
Smart assistive technologies such as sensor-based footwear and walking aids offer promising opportunities for gait rehabilitation through real-time feedback and patient-centered monitoring. While biofeedback applications show great potential, current...
Research on Accessible Digital Musical Instruments (ADMIs) has highlighted the need for participatory design methods, i.e., to actively include users as co-designers and informants in the design process. However, very little work has explored how pre...
This paper presents an intelligent navigation wearable device for visually impaired individuals. The system aims to improve their independent travel capabilities and reduce the negative emotional impacts associated with visible disability indicators...
Aging often leads to elderly gait characterized by slower speeds, shorter strides, and increased cycle; improving gait can significantly enhance the quality of life. Early gait training can help reduce gait impairment later on. Augmented reality (AR)...
Rendering surface materials to provide realistic tactile sensations is a key focus in haptic interaction research. However, generating texture maps and designing corresponding haptic feedback often requires expert knowledge and significant effort. To...
Effectively managing visual search tasks across multiple spatial regions during daily activities such as driving, cycling, and navigating complex environments often overwhelms visual processing capacity, increasing the risk of errors and missed criti...
Current advancements in music technology enable the creation of customized Digital Musical Instruments (DMIs). This paper presents a systematic review of Accessible Digital Musical Instruments (ADMIs) in inclusive music practice. History of research...
Numerous technological solutions have been proposed to promote piano learning and teaching, but very few with market success. We are convinced that users’ needs should be the starting point for an effective and transdisciplinary development process o...
Intelligent agents that can interact with users using natural language are becoming increasingly common. Sometimes an intelligent agent may not correctly understand a user command or may not perform it properly. In such cases, the user might try a se...
Multimodal interaction technology has become a key aspect of remote education by enriching student engagement and learning results as it utilizes the speech, gesture, and visual feedback as various sensory channels. This publication reflects on the l...
This paper presents a system that utilizes vision transformers and multimodal feedback modules to facilitate navigation and collision avoidance for the visually impaired. By implementing vision transformers, the system achieves accurate object detect...
Multimodal remote sensing data classification can enhance a model’s ability to distinguish land features through multimodal data fusion. In this context, how to help models understand the relationship between multimodal data and target tasks ha...
Sensors can monitor physical attributes and record multimodal data in order to provide feedback. The application calligraphy trainer, exploits these affordances in the context of handwriting learning. It records the expert’s handwriting perform...
This paper presents the development and evaluation of a smartphone application designed to improve accessibility for blind users. It uses the lightweight EfficientDet-lite2 model and the comprehensive COCO dataset in order to provide real-time object...
This study investigated to what extent multimodal data can be used to detect mistakes during Cardiopulmonary Resuscitation (CPR) training. We complemented the Laerdal QCPR ResusciAnne manikin with the Multimodal Tutor for CPR, a multi-sensor system c...
The efficient multi-modal fusion of data streams from different sensors is a crucial ability that a robotic perception system should exhibit to ensure robustness against disturbances. However, as the volume and dimensionality of sensory-feedback incr...
Urinary tract infections (UTIs) are among the most common reasons patients seeking health care and antibiotics to be prescribed in primary care. However, general practitioners’ (GPs) guideline adherence is low. The RedAres randomised controlled...
Light propagation in multimode fibers is known to experience various nonlinear effects, which are being actively studied. One of the interesting effects is the brightness enhancement at the Raman conversion of the multimode beam in graded index (GRIN...
of 6