Multimodal Emotion Recognition

Dear Colleagues,

The goal of automatic emotion recognition is to detect and recognize affect from low-level sensorial cues. The processing of behavioral and emotional signals usually involves facial analysis, body posture, speech/vocalization, as well as biomeasurements and analysis of brain signals. Most of the times, proper emotional models are used for emotion recognition. These can be coarsely divided into two categories: discrete and continuous emotion models. One of the most universally recognized and widely used discrete emotion models is based on the six basic emotions (sadness, happiness, fear, anger, surprise, and disgust), as proposed by Paul Ekman. More recently, the concept of compound emotion (e.g., surprisingly happy or happily disgusted) has also been explored in AI. There are also many studies using dimensional spaces (e.g., valence, arousal), mainly due to the fact that discrete models, although easily applicable in classification problems, do not always fully describe every emotion-enriched experience, and their label-based character limits their applicability in domains where the intensity of the emotion is important.

For a more accurate emotion recognition, many works have proposed multimodal fusion approaches, combining various cues (e.g., visual, audio, wearable devices, brain signals). Efforts in developing related methods, however, often face challenges due to a significant lack of proper datasets for training and testing. Thus, one of the most typical problems faced by researchers in affective computing is that they train their systems on one dataset and, while they achieve accurate results on it, they fail to achieve high accuracies when applying the same model on a different dataset. This becomes even more challenging when data are captured in non-controlled, spontaneous conditions. Regarding AI methods for emotion recognition, various techniques have been proposed, with the most recent studies focusing on end-to-end, deep-learning topologies as a way to take advantage of as much training data as possible, aiming at high accuracies in datasets captured in the wild.

This Special Issue is looking for high-quality research contributions in one or more of the following domains:

• New computational models in multimodal emotion recognition, using deep-learning topologies
• Personalized emotional models and multimodality
• Domain adaptation across datasets in emotion recognition
• Domain adaptation across modalities in emotion recognition
• New multimodal datasets for emotion recognition

Dr. Stylianos (Stelios) Asteriadis
Dr. Enrique Hortal
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 250 words) can be sent to the Editorial Office for assessment.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-anonymized peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Multimodal Technologies and Interaction is an international peer-reviewed open access monthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 1800 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.