Skip to Content
You are currently on the new version of our website. Access the old version .
  • 3.3
    Impact Factor
  • 6.6
    CiteScore
  • 23 days
    Submission to First Decision
  • 5 days
    Acceptance to Publication

Multimodal Technologies and Interaction

Multimodal Technologies and Interaction is an international, peer-reviewed, open access journal on multimodal technologies and interaction published monthly online by MDPI.

Get Alerted

Add your email address to receive forthcoming issues of this journal.

All Articles (948)

  • Article
  • Open Access

Free-hand interaction in Mixed Reality (MR) can become difficult as the interaction technique, target geometry, and spatial context vary. We present FAIR-MR, a failure-aware adaptive interaction system that responds to unsuccessful interaction through geometric interventions. Two user studies characterize interaction difficulty across gestures and spatial conditions, and then compare Static, Precomputed, and Online Bandit strategies for selecting interventions. Interaction success varied across gestures, with target geometry affecting interaction techniques differently. The adaptation strategies produced distinct intervention and interaction patterns. Within Online Bandit, intervention estimates became increasingly differentiated with experience, and greater pre-decision differentiation was associated with subsequent recovery. However, this policy development did not produce higher aggregate success than the other strategies. These findings reveal how interaction conditions, intervention strategies, and learned policy behavior shape adaptive interaction. FAIR-MR contributes a system-level approach connecting interaction difficulty, intervention selection, interface changes, transition costs, and recovery.

Conceptual overview of FAIR-MR. Study 1 characterizes interaction difficulty across interaction context and technique, providing an empirical basis for the conditions and interventions evaluated in Study 2. Following an observed interaction failure, FAIR-MR connects adaptation decisions and applied interventions with subsequent recovery.
  • Article
  • Open Access

User-Centered Design and Exploratory Evaluation of an Immersive Virtual Reality Application for Volcano and Rover Activities with Students Aged 11–12

  • Rubén De Jesús Rivera-Medina,
  • Huizilopoztli Luna-García and
  • Klinge Orlando Villalba-Condori
  • + 4 authors

Science, technology, engineering, and mathematics (STEM) education in Mexico currently offers an opportunity to adopt immersive virtual reality (VR) as a supplement to teaching methods in elementary education. This article presents the user-centered design (UCD) and exploratory classroom evaluation of Vulcanus, an immersive VR application for volcano-related STEM activities aimed at elementary school students aged 11 to 12. The application combines guided audiovisual content, hand tracking, rover assembly, and robotic exploration tasks. Forty-eight students participated (24 per condition); the 24 VR users completed questionnaires on experience, discomfort, and usability. The results showed an adapted experience score of 32.12/40; the adapted 10-item distress scale yielded a descriptive mean of 5.37/30, and an adapted System Usability Scale (SUS) score of 69.23/100. The classroom intervention included historical knowledge questionnaires; however, their scores were excluded from the main findings because the original open-ended response scoring could not be independently audited. The contribution of this article is the design and exploratory evaluation of an educational VR application with hand tracking.

Flow in the UCD process.
  • Article
  • Open Access

The rapid development of online learning platforms has made human–computer interaction increasingly important in modern educational and work environments. However, task-oriented interactive experiences remain underexplored, particularly slide-presentation tasks involving university students acting as presenters. In this study, 61 university students (aged 19–23) completed three eye-tracking sub-experiments embedded in online slide-presentation and speaking tasks; eye-movement and questionnaire data were collected to investigate interface layout, emoji use, visualization of audience feedback, and personal status assessment design. Results show that the horizontal layout enabled rapid capture of text and image information; people prefer detailed explanations when making a choice; emoji do not lead to faster attention to the text but did not produce measurable differences in emotional or attentional indices; adding a feedback module does not necessarily promote positive emotions; however, positive feedback can. Combining physiological measures with reflective feedback is essential for understanding emotional experience. These findings provide quantitative evidence for designing interactive interfaces and emotional support in online learning, aiming to improve interaction efficiency, deepen the understanding of users’ emotional experiences, and promote engagement and user retention among online meeting users.

Material design of Experiment 1-1: Different interactive texts in the audio settings dialog.
  • Article
  • Open Access

Background/Objectives: Interactive tourism services require service objectives to be translated into meaningful visitor experiences. This study examines how design teams represented visitor motivations in interaction concepts and how documented formative feedback informed refinement. Methods: A retrospective cross-case analysis examined eight industry-engaged projects addressing membership participation or post-visit re-engagement in an anonymised resort context. Project reports, prototype representations, and evaluation accounts were compared. SUS records were examined for documentary coverage rather than used to rank prototypes. Jiaobei provided an illustrative case. Results: Membership concepts embedded benefits within practical and social activities, whereas post-visit concepts proposed cultural, narrative, identity-related, and memory-based reasons to reconnect. Reported feedback informed choices about information visibility, navigation, and participation. Jiaobei linked a proposed physical memento and gesture-based ritual to digital content; navigation feedback was associated with an added map overview. Conclusions: Four provisional principles concern valued activities, voluntary return, complementary modality roles, and recoverable navigation. Their contribution is design-oriented and context-grounded. Incomplete evaluation records, unverified multimodal implementation, and absent post-revision measurements limit empirical claims about usability gains and sustained engagement.

Jiaobei interaction architecture and evidence boundaries. (Upper panel): an analytical diagram based on B1; dashed elements indicate proposed physical or sensing functions, while solid elements represent documented software states and navigation relationships. (Lower panel): scenario images illustrating the envisioned physical carrier and interaction sequence; these images do not establish functional NFC integration or gesture recognition. Voice diary and AR features were proposed extensions outside the evaluated software scope.

Highly Accessed Articles

News & Conferences

Latest Issues

Open for Submission

Journal Sections

XFacebookLinkedIn
Multimodal Technol. Interact. - ISSN 2414-4088