Skip to Content
  • 3.3
    Impact Factor
  • 6.6
    CiteScore
  • 23 days
    Submission to First Decision
  • 5 days
    Acceptance to Publication

Multimodal Technologies and Interaction

Multimodal Technologies and Interaction is an international, peer-reviewed, open access journal on multimodal technologies and interaction published monthly online by MDPI.

Get Alerted

Add your email address to receive forthcoming issues of this journal.

All Articles (951)

  • Article
  • Open Access

Digital mental health interventions increasingly use narrative, game design, personalisation and behavioural data to support engagement, but it remains unclear how these features can be integrated without weakening fidelity to the original evidence-based intervention approach. This study examined interdisciplinary perspectives on a narrative-based serious game and meaningful engagement framework for preventative digital mental health grounded in principles informed by Self Help Plus (SH+) and acceptance and commitment therapy (ACT). A two-round electronic Delphi (eDelphi) study was conducted involving participants providing human–computer interaction (HCI), mental health and potential user perspectives. Fifteen participants completed Round 1, and following attrition, twelve completed Round 2. Consensus required at least 70% agreement or disagreement and an interquartile range of no more than 1. Round 1 assessed agreement with core framework assumptions and established consensus on 6/12 items. Round 2 re-rated the six unresolved items with controlled feedback, introduced five clarification items and elicited refinement priorities. Round 1 showed broad support for narrative, game design, behavioural observation, transferability and flexible pacing, but identified tensions around translation of SH+/ACT principles, oversimplification, narrative complexity, behavioural data sufficiency, context specificity and autonomy. Of the six original items re-rated in Round 2, only interactive translation of SH+/ACT principles (A1) reached consensus. Five new clarification items separately reached consensus on keeping the core SH+/ACT content and processes stable while adapting delivery style, examples, pacing, narrative framing and interaction format. These items also supported adjustable narrative depth, multidimensional engagement assessment, principle level transferability and guided autonomy. These judgements do not validate the integrated framework. They provide partial conceptual support for its development and indicate that further development should proceed through a modular minimum viable product (MVP) and empirical testing rather than immediate deployment.

Round 1 directional distribution across original items. Disagreement combines scores 1–2, neutrality is 3 and agreement combines 4–5. C indicates consensus and NC no consensus under the specified directional-percentage and IQR rule. Bars show descriptive directional percentages; consensus was assessed using the percentage and IQR rule in Table 2. Item codes A1-F2 identify the original statements listed in Table 2 and Appendix B.
  • Article
  • Open Access

Teachers increasingly consult large language model (LLM) chatbots, yet little is known about the feedback guidance these conversational interfaces give physical education (PE) teachers, or whether it depends on the chatbot or the language. We submitted one prompt, describing a low-skilled child who repeatedly fails a forward roll and a teacher tempted to reprimand him publicly, to ChatGPT, Gemini, Claude, and DeepSeek in English, French, and Arabic. In this exploratory single-run study, a directed content analysis of the 12 responses (109 scripted feedback statements) described and compared chatbots and languages. All chatbots rejected the public reprimand and prescribed a chin tuck and a simpler task. However, only half explained the cause of the error, two-thirds of the technical cues combined several points, 95% of the technical cues directed attention to the body, and six responses contradicted their own guidance to give one cue at a time. Praise was often unspecific, one statement invited the student’s view, and several responses flattered the teacher. Descriptively, the chatbot accounted for 58% of the dissimilarity between coded responses and the language for 12%, and these shares cannot be separated from run-to-run variation. Within these limits, teachers may wish to check diagnosis, cue load, and autonomy support.

Binary code profiles of the 12 responses (four chatbots by three languages). Each column is one response and each row one of the 33 codes, grouped by theme. A filled cell means the code was present in the response. EN, English; FR, French; AR, Arabic.
  • Article
  • Open Access

Free-hand interaction in Mixed Reality (MR) can become difficult as the interaction technique, target geometry, and spatial context vary. We present FAIR-MR, a failure-aware adaptive interaction system that responds to unsuccessful interaction through geometric interventions. Two user studies characterize interaction difficulty across gestures and spatial conditions, and then compare Static, Precomputed, and Online Bandit strategies for selecting interventions. Interaction success varied across gestures, with target geometry affecting interaction techniques differently. The adaptation strategies produced distinct intervention and interaction patterns. Within Online Bandit, intervention estimates became increasingly differentiated with experience, and greater pre-decision differentiation was associated with subsequent recovery. However, this policy development did not produce higher aggregate success than the other strategies. These findings reveal how interaction conditions, intervention strategies, and learned policy behavior shape adaptive interaction. FAIR-MR contributes a system-level approach connecting interaction difficulty, intervention selection, interface changes, transition costs, and recovery.

Conceptual overview of FAIR-MR. Study 1 characterizes interaction difficulty across interaction context and technique, providing an empirical basis for the conditions and interventions evaluated in Study 2. Following an observed interaction failure, FAIR-MR connects adaptation decisions and applied interventions with subsequent recovery.
  • Article
  • Open Access

User-Centered Design and Exploratory Evaluation of an Immersive Virtual Reality Application for Volcano and Rover Activities with Students Aged 11–12

  • Rubén De Jesús Rivera-Medina,
  • Huizilopoztli Luna-García and
  • Klinge Orlando Villalba-Condori
  • + 4 authors

Science, technology, engineering, and mathematics (STEM) education in Mexico currently offers an opportunity to adopt immersive virtual reality (VR) as a supplement to teaching methods in elementary education. This article presents the user-centered design (UCD) and exploratory classroom evaluation of Vulcanus, an immersive VR application for volcano-related STEM activities aimed at elementary school students aged 11 to 12. The application combines guided audiovisual content, hand tracking, rover assembly, and robotic exploration tasks. Forty-eight students participated (24 per condition); the 24 VR users completed questionnaires on experience, discomfort, and usability. The results showed an adapted experience score of 32.12/40; the adapted 10-item distress scale yielded a descriptive mean of 5.37/30, and an adapted System Usability Scale (SUS) score of 69.23/100. The classroom intervention included historical knowledge questionnaires; however, their scores were excluded from the main findings because the original open-ended response scoring could not be independently audited. The contribution of this article is the design and exploratory evaluation of an educational VR application with hand tracking.

Flow in the UCD process.

Highly Accessed Articles

News & Conferences

Latest Issues

Open for Submission

Journal Sections

XFacebookLinkedIn
Multimodal Technol. Interact. - ISSN 2414-4088