Perspectives and Applications of Multimodal Artificial Intelligence and Big Data

Dear Colleagues,

A vast amount of data are generated by humans through smartphones, the Web, and social media, as well as from sensors. These data are highly heterogeneous, often combining text, images, audio, video, and multiple sensory inputs simultaneously.

Artificial intelligence (AI) systems play a crucial role in enhancing humans’ ability to monitor, understand, analyse, and generate knowledge from these diverse data sources. In particular, multimodal AI facilitates the integration of two or more modalities (such as speech, gesture, and facial expressions) to understand complex relationships among different data types. This approach enables more comprehensive results and the more effective fusion of various data formats, leading to improved decision-making. This contrasts with traditional AI models, which typically focus on a single type of data input. Multimodal AI has applications across several domains. Multimodal AI is particularly transformative in monitoring applications, where it enables real-time, adaptive, and context-aware systems capable of processing information from multiple sources simultaneously, allowing for personalized interactions. This is especially relevant in robotics, where multimodal AI can significantly enhance human–robot interactions by simultaneously processing data from different sensors, contributing to more effective outcomes in both personal and industrial settings. In assistive robotics, for example, multimodal monitoring allows for robots to interpret and respond to human emotions, gestures, and speech while simultaneously analysing environmental data. This leads to more effective and responsive robotic assistants that can support individuals with disabilities, the elderly, or those in need of specialized care. In industrial and collaborative robotics, multimodal AI improves safety, efficiency, and adaptability by allowing for robots to understand human intentions and react accordingly in dynamic environments.

Beyond robotics, multimodal AI finds applications in various domains. In marketing, multimodal AI can help define more personalized and effective campaigns by analyzing data from videos, texts, and images found on social media. Likewise, integrating text analysis, voice tone detection, and facial expression interpretation can improve customer service interactions with chatbots, leading to more natural and empathetic conversations. Additionally, multimodal AI strengthens disaster management by combining heterogeneous data from sensors, social media, and satellite imagery for real-time situational awareness. In healthcare, the integration of patients' vital signs with diagnostic data with multimodal AI may lead to more accurate diagnoses and predictions, improving patient outcomes.

Despite its advantages, multimodal AI introduces several challenges that need to be addressed, including the proper alignment of various data types, interpretation issues (such as ambiguities), missing data, and difficulties around making results interpretable and trustworthy for users. Addressing these challenges is crucial to unlocking the full potential of multimodal AI in critical applications.

This Special Issue invites submissions that focus on the methods, applications, challenges, and perspectives of Multimodal AI across a wide range of application areas, with a particular interest in multimodal monitoring and robotics for assistance, among other areas.

Contributions are welcome to take the form of original research, advancements, developments, and experiments in the following fields (this list is not exhaustive):

• Multimodal AI in customer service;
• Multimodal AI in social media;
• Multimodal AI in healthcare;
• Multimodal AI in assistive and industrial robotics;
• Multimodal AI for real-time monitoring and decision-making;
• Generative AI and multimodal data;
• Predictive AI and multimodal data;
• Advanced human–robot interactions;
• AI in multimodal user interfaces;
• AI in multimodal interactions;
• User-centric multimodal AI;
• Multimodal explainable AI;
• Multimodal trustworthy AI.

Dr. Maria Chiara Caschera
Dr. Francesca Cordella
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Big Data and Cognitive Computing is an international peer-reviewed open access monthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 1800 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

• multimodal AI
• artificial intelligence
• data analysis
• human-robot interaction