Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
6.8
3.9
Journals
Sensors
Special Issues
Sensors for Affective Computing and Sentiment Analysis
sensors-logo
Submit to Sensors Review for Sensors Propose a Special Issue

Journal Menu

► Journal Menu

Journal Browser

► Journal Browser
share announcement

Sensors for Affective Computing and Sentiment Analysis

A special issue of Sensors (ISSN 1424-8220). This special issue belongs to the section "Physical Sensors".

Deadline for manuscript submissions: closed (31 January 2020) | Viewed by 108393

Special Issue Editors

Prof. Dr. Antonio Fernández-Caballero

E-Mail Website
Guest Editor
Instituto de Investigación en Informática de Albacete, Universidad de Castilla-La Mancha, 02071 Albacete, Spain
Interests: pattern recognition; human–computer interaction; affective computing; computer vision; multi-sensor fusion
Special Issues, Collections and Topics in MDPI journals
Dr. Arturo Martínez-Rodrigo

E-Mail Website
Guest Editor
Universidad de Castilla-La Mancha, Cuenca, Spain
Interests: signal processing; physiological sensors; sensors networks; Internet of Things; embedded systems

Special Issue Information

Dear Colleagues,

Emotions are essential in human–human communication, cognition, learning, and rational decision-making processes. However, human–machine interfaces (HMIs) are still not able to understand human sentiments and react accordingly. With the aim of endowing HMIs with the emotional intelligence they lack, the affective computing science focuses on the development of artificial intelligence by means of the analysis of affects and emotions, such that systems and sensors could be able to recognize, interpret, process, and simulate human sentiments.

Nowadays, the evaluation of electrophysiological signals plays a key role in the advancement towards that purpose, as they are an objective representation of the emotional state of an individual. Hence, interest in physiological variables like electroencephalogram, electrocardiogram, or electrodermal activity, among many others, has notably grown in the field of affective states detection. Furthermore, emotions have also been widely identified by means of the assessment of speech characteristics and facial gestures of people under different sentimental conditions.

This Special Issue, Sensors for Affective Computing and Sentiment Analysis, is intended to be a venue for researchers that are interested in the development and/or use of physical sensors in those areas of expertise related to sentiment analysis, who want to initiate their studies or are currently working on this topic. Hence, manuscripts introducing new proposals based on physical sensors for the analysis of physiological measures, facial recognition, speech recognition, or natural language processing are welcome in this Special Issue of Sensors for Affective Computing and Sentiment Analysis.

Prof. Dr. Antonio Fernández-Caballero
Dr. Arturo Martínez-Rodrigo
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Sensors is an international peer-reviewed open access semimonthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2600 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

  • Sensors for affective computing
  • Sensors for sentiment analysis
  • Sensors for ubiquitous and pervasive computing
  • Sensors for ambient intelligence
  • Sensors for ambient assisted living
  • Sensors for physiological computing
  • Internet of things sensors
  • Sensors for natural language processing
  • Brain–-computer interfaces
  • Biofeedback and neurofeedback systems
  • Wearable systems
  • Applications and case studies

Published Papers (16 papers)

Download All Papers
Order results
Result details
Show export options expand_more Show export options expand_less
Select all
Export citation of selected articles as:

Research

29 pages, 3056 KiB  
Article
Crowd of Oz: A Crowd-Powered Social Robotics System for Stress Management
by Tahir Abbas, Vassilis-Javed Khan, Ujwal Gadiraju, Emilia Barakova and Panos Markopoulos
Sensors 2020, 20(2), 569; https://doi.org/10.3390/s20020569 - 20 Jan 2020
Cited by 18 | Viewed by 5887
Abstract
Coping with stress is crucial for a healthy lifestyle. In the past, a great deal of research has been conducted to use socially assistive robots as a therapy to alleviate stress and anxiety related problems. However, building a fully autonomous social robot which [...] Read more.
Coping with stress is crucial for a healthy lifestyle. In the past, a great deal of research has been conducted to use socially assistive robots as a therapy to alleviate stress and anxiety related problems. However, building a fully autonomous social robot which can deliver psycho-therapeutic solutions is a very challenging endeavor due to limitations in artificial intelligence (AI). To overcome AI’s limitations, researchers have previously introduced crowdsourcing-based teleoperation methods, which summon the crowd’s input to control a robot’s functions. However, in the context of robotics, such methods have only been used to support the object manipulation, navigational, and training tasks. It is not yet known how to leverage real-time crowdsourcing (RTC) to process complex therapeutic conversational tasks for social robotics. To fill this gap, we developed Crowd of Oz (CoZ), an open-source system that allows Softbank’s Pepper robot to support such conversational tasks. To demonstrate the potential implications of this crowd-powered approach, we investigated how effectively, crowd workers recruited in real-time can teleoperate the robot’s speech, in situations when the robot needs to act as a life coach. We systematically varied the number of workers who simultaneously handle the speech of the robot (N = 1, 2, 4, 8) and investigated the concomitant effects for enabling RTC for social robotics. Additionally, we present Pavilion, a novel and open-source algorithm for managing the workers’ queue so that a required number of workers are engaged or waiting. Based on our findings, we discuss salient parameters that such crowd-powered systems must adhere to, so as to enhance their performance in response latency and dialogue quality. Full article
(This article belongs to the Special Issue Sensors for Affective Computing and Sentiment Analysis)
►▼ Show Figures

Figure 1

18 pages, 2070 KiB  
Article
Brain and Body Emotional Responses: Multimodal Approximation for Valence Classification
by Jennifer Sorinas, Jose Manuel Ferrández and Eduardo Fernandez
Sensors 2020, 20(1), 313; https://doi.org/10.3390/s20010313 - 06 Jan 2020
Cited by 16 | Viewed by 3873
Abstract
In order to develop more precise and functional affective applications, it is necessary to achieve a balance between the psychology and the engineering applied to emotions. Signals from the central and peripheral nervous systems have been used for emotion recognition purposes, however, their [...] Read more.
In order to develop more precise and functional affective applications, it is necessary to achieve a balance between the psychology and the engineering applied to emotions. Signals from the central and peripheral nervous systems have been used for emotion recognition purposes, however, their operation and the relationship between them remains unknown. In this context, in the present work, we have tried to approach the study of the psychobiology of both systems in order to generate a computational model for the recognition of emotions in the dimension of valence. To this end, the electroencephalography (EEG) signal, electrocardiography (ECG) signal and skin temperature of 24 subjects have been studied. Each methodology has been evaluated individually, finding characteristic patterns of positive and negative emotions in each of them. After feature selection of each methodology, the results of the classification showed that, although the classification of emotions is possible at both central and peripheral levels, the multimodal approach did not improve the results obtained through the EEG alone. In addition, differences have been observed between cerebral and peripheral responses in the processing of emotions by separating the sample by sex; though, the differences between men and women were only notable at the peripheral nervous system level. Full article
(This article belongs to the Special Issue Sensors for Affective Computing and Sentiment Analysis)
►▼ Show Figures

Figure 1

8 pages, 775 KiB  
Article
General Mental Health Is Associated with Gait Asymmetry
by Hanatsu Nagano, Eri Sarashina, William Sparrow, Katsuyoshi Mizukami and Rezaul Begg
Sensors 2019, 19(22), 4908; https://doi.org/10.3390/s19224908 - 10 Nov 2019
Cited by 7 | Viewed by 3104
Abstract
Wearable sensors are being applied to real-world motion monitoring and the focus of this work is assessing health status and wellbeing. An extensive literature has documented the effects on gait control of impaired physical health, but in this project, the aim was to [...] Read more.
Wearable sensors are being applied to real-world motion monitoring and the focus of this work is assessing health status and wellbeing. An extensive literature has documented the effects on gait control of impaired physical health, but in this project, the aim was to determine whether emotional states associated with older people’s mental health are also associated with walking mechanics. If confirmed, wearable sensors could be used to monitor affective responses. Lower limb gait mechanics of 126 healthy individuals (mean age 66.2 ± 8.38 years) were recorded using a high-speed 3D motion sensing system and they also completed a 12-item mental health status questionnaire (GHQ-12). Mean step width and minimum foot-ground clearance (MFC), indicative of tripping risk, were moderately correlated with GHQ-12. Ageing and variability (SD) of gait parameters were not significantly correlated with GHQ-12. GHQ-12 scores were, however, highly correlated with left-right gait control, indicating that greater gait symmetry was associated with better mental health. Maintaining good mental health with ageing may promote safer gait and wearable sensor technologies could be applied to gait asymmetry monitoring, possibly using a single inertial measurement unit attached to each shoe. Full article
(This article belongs to the Special Issue Sensors for Affective Computing and Sentiment Analysis)
►▼ Show Figures

Figure 1

11 pages, 1243 KiB  
Article
Comparative Evaluation of the Autonomic Response to Cognitive and Sensory Stimulations through Wearable Sensors
by Alessandro Tonacci, Lucia Billeci, Elisa Burrai, Francesco Sansone and Raffaele Conte
Sensors 2019, 19(21), 4661; https://doi.org/10.3390/s19214661 - 27 Oct 2019
Cited by 11 | Viewed by 2788
Abstract
Psychological stress is known to activate the autonomic nervous system (ANS), thus representing a useful target to be monitored to understand the physiological, unconscious effect of stress on the human body. However, little is known about how differently the ANS responds to cognitive [...] Read more.
Psychological stress is known to activate the autonomic nervous system (ANS), thus representing a useful target to be monitored to understand the physiological, unconscious effect of stress on the human body. However, little is known about how differently the ANS responds to cognitive and sensory stimulations in healthy subjects. To this extent, we enrolled 23 subjects and administered a stress protocol consisting of the administration of sensory (olfactory) and cognitive (mathematical) stressors. Autonomic parameters were unobtrusively monitored through wearable sensors for capturing electrocardiogram and skin conductance signals. The results obtained demonstrated an increase of the heart rate during both stress protocols, with a similar decrease of the heart rate variability. Cognitive stress test appears to affect the autonomic parameters to a greater extent, confirming its effects on the human body. However, olfactory stimulation could be useful to study stress in specific experimental settings when the administration of complex cognitive testing is not feasible. Full article
(This article belongs to the Special Issue Sensors for Affective Computing and Sentiment Analysis)
►▼ Show Figures

Figure 1

37 pages, 2227 KiB  
Article
Detecting and Monitoring Hate Speech in Twitter
by Juan Carlos Pereira-Kohatsu, Lara Quijano-Sánchez, Federico Liberatore and Miguel Camacho-Collados
Sensors 2019, 19(21), 4654; https://doi.org/10.3390/s19214654 - 26 Oct 2019
Cited by 111 | Viewed by 25039
Abstract
Social Media are sensors in the real world that can be used to measure the pulse of societies. However, the massive and unfiltered feed of messages posted in social media is a phenomenon that nowadays raises social alarms, especially when these messages contain [...] Read more.
Social Media are sensors in the real world that can be used to measure the pulse of societies. However, the massive and unfiltered feed of messages posted in social media is a phenomenon that nowadays raises social alarms, especially when these messages contain hate speech targeted to a specific individual or group. In this context, governments and non-governmental organizations (NGOs) are concerned about the possible negative impact that these messages can have on individuals or on the society. In this paper, we present HaterNet, an intelligent system currently being used by the Spanish National Office Against Hate Crimes of the Spanish State Secretariat for Security that identifies and monitors the evolution of hate speech in Twitter. The contributions of this research are many-fold: (1) It introduces the first intelligent system that monitors and visualizes, using social network analysis techniques, hate speech in Social Media. (2) It introduces a novel public dataset on hate speech in Spanish consisting of 6000 expert-labeled tweets. (3) It compares several classification approaches based on different document representation strategies and text classification models. (4) The best approach consists of a combination of a LTSM+MLP neural network that takes as input the tweet’s word, emoji, and expression tokens’ embeddings enriched by the tf-idf, and obtains an area under the curve (AUC) of 0.828 on our dataset, outperforming previous methods presented in the literature. Full article
(This article belongs to the Special Issue Sensors for Affective Computing and Sentiment Analysis)
►▼ Show Figures

Figure 1

15 pages, 2102 KiB  
Article
Two-Wired Active Spring-Loaded Dry Electrodes for EEG Measurements
by Seungchan Lee, Younghak Shin, Anil Kumar, Kiseon Kim and Heung-No Lee
Sensors 2019, 19(20), 4572; https://doi.org/10.3390/s19204572 - 21 Oct 2019
Cited by 9 | Viewed by 5347
Abstract
Dry contact electrode-based EEG acquisition is one of the easiest ways to obtain neural information from the human brain, providing many advantages such as rapid installation, and enhanced wearability. However, high contact impedance due to insufficient electrical coupling at the electrode-scalp interface still [...] Read more.
Dry contact electrode-based EEG acquisition is one of the easiest ways to obtain neural information from the human brain, providing many advantages such as rapid installation, and enhanced wearability. However, high contact impedance due to insufficient electrical coupling at the electrode-scalp interface still remains a critical issue. In this paper, a two-wired active dry electrode system is proposed by combining finger-shaped spring-loaded probes and active buffer circuits. The shrinkable probes and bootstrap topology-based buffer circuitry provide reliable electrical coupling with an uneven and hairy scalp and effective input impedance conversion along with low input capacitance. Through analysis of the equivalent circuit model, the proposed electrode was carefully designed by employing off-the-shelf discrete components and a low-noise zero-drift amplifier. Several electrical evaluations such as noise spectral density measurements and input capacitance estimation were performed together with simple experiments for alpha rhythm detection. The experimental results showed that the proposed electrode is capable of clear detection for the alpha rhythm activation, with excellent electrical characteristics such as low-noise of 1.131 μVRMS and 32.3% reduction of input capacitance. Full article
(This article belongs to the Special Issue Sensors for Affective Computing and Sentiment Analysis)
►▼ Show Figures

Figure 1

26 pages, 4321 KiB  
Article
Impact of Physiological Signals Acquisition in the Emotional Support Provided in Learning Scenarios
by R. Uria-Rivas, M. C. Rodriguez-Sanchez, O. C. Santos, J. Vaquero and J. G. Boticario
Sensors 2019, 19(20), 4520; https://doi.org/10.3390/s19204520 - 17 Oct 2019
Cited by 9 | Viewed by 3541
Abstract
Physiological sensors can be used to detect changes in the emotional state of users with affective computing. This has lately been applied in the educational domain, aimed to better support learners during the learning process. For this purpose, we have developed the AICARP [...] Read more.
Physiological sensors can be used to detect changes in the emotional state of users with affective computing. This has lately been applied in the educational domain, aimed to better support learners during the learning process. For this purpose, we have developed the AICARP (Ambient Intelligence Context-aware Affective Recommender Platform) infrastructure, which detects changes in the emotional state of the user and provides personalized multisensorial support to help manage the emotional state by taking advantage of ambient intelligence features. We have developed a third version of this infrastructure, AICARP.V3, which addresses several problems detected in the data acquisition stage of the second version, (i.e., intrusion of the pulse sensor, poor resolution and low signal to noise ratio in the galvanic skin response sensor and slow response time of the temperature sensor) and extends the capabilities to integrate new actuators. This improved incorporates a new acquisition platform (shield) called PhyAS (Physiological Acquisition Shield), which reduces the number of control units to only one, and supports both gathering physiological signals with better precision and delivering multisensory feedback with more flexibility, by means of new actuators that can be added/discarded on top of just that single shield. The improvements in the quality of the acquired signals allow better recognition of the emotional states. Thereof, AICARP.V3 gives a more accurate personalized emotional support to the user, based on a rule-based approach that triggers multisensorial feedback, if necessary. This represents progress in solving an open problem: develop systems that perform as effectively as a human expert in a complex task such as the recognition of emotional states. Full article
(This article belongs to the Special Issue Sensors for Affective Computing and Sentiment Analysis)
►▼ Show Figures

Figure 1

18 pages, 538 KiB  
Article
Explicit and Implicit Responses to Tasting Drinks Associated with Different Tasting Experiences
by Daisuke Kaneko, Maarten Hogervorst, Alexander Toet, Jan B. F. van Erp, Victor Kallen and Anne-Marie Brouwer
Sensors 2019, 19(20), 4397; https://doi.org/10.3390/s19204397 - 11 Oct 2019
Cited by 23 | Viewed by 3153
Abstract
Probing food experience or liking through verbal ratings has its shortcomings. We compare explicit ratings to a range of (neuro)physiological and behavioral measures with respect to their performance in distinguishing drinks associated with different emotional experience. Seventy participants tasted and rated the valence [...] Read more.
Probing food experience or liking through verbal ratings has its shortcomings. We compare explicit ratings to a range of (neuro)physiological and behavioral measures with respect to their performance in distinguishing drinks associated with different emotional experience. Seventy participants tasted and rated the valence and arousal of eight regular drinks and a “ground truth” high-arousal, low-valence vinegar solution. The discriminative power for distinguishing between the vinegar solution and the regular drinks was highest for sip size, followed by valence ratings, arousal ratings, heart rate, skin conductance level, facial expression of “disgust,” pupil diameter, and Electroencephalogram (EEG) frontal alpha asymmetry. Within the regular drinks, a positive correlation was found between rated arousal and heart rate, and a negative correlation between rated arousal and Heart Rate Variability (HRV). Most physiological measures showed consistent temporal patterns over time following the announcement of the drink and taking a sip. This was consistent over all nine drinks, but the peaks were substantially higher for the vinegar solution than for the regular drinks, likely caused by emotion. Our results indicate that implicit variables have the potential to differentiate between drinks associated with different emotional experiences. In addition, this study gives us insight into the physiological temporal response patterns associated with taking a sip. Full article
(This article belongs to the Special Issue Sensors for Affective Computing and Sentiment Analysis)
►▼ Show Figures

Figure 1

13 pages, 842 KiB  
Article
The Role of Perceived Control in the Psychophysiological Responses to Disgust of Subclinical OCD Women
by Miguel Ángel Serrano, Vicent Rosell-Clari and Gemma García-Soriano
Sensors 2019, 19(19), 4180; https://doi.org/10.3390/s19194180 - 26 Sep 2019
Cited by 1 | Viewed by 2307
Abstract
Obsessive‒compulsive disorder (OCD), and especially contamination obsessions and washing compulsions, has been related to disgust. However, when its cardiovascular correlates have been studied, contradictory results have been found, including heart rate accelerations and decelerations. The aim of this study is to analyze emotional, [...] Read more.
Obsessive‒compulsive disorder (OCD), and especially contamination obsessions and washing compulsions, has been related to disgust. However, when its cardiovascular correlates have been studied, contradictory results have been found, including heart rate accelerations and decelerations. The aim of this study is to analyze emotional, cognitive, and cardiovascular responses in nonclinical (control) and subclinical participants with obsessive‒compulsive contamination/washing symptoms when confronted with a disgusting stimulus. Twenty-seven participants (14 subclinical OCD) completed a behavioral avoidance task with a contamination-based stimulus while their heart rate and subjective variables were measured. Results showed heart rate reductions in both samples, whereas subjective measures reflected higher disgust, anxiety, dirtiness, and emotional valence in the subclinical sample. However, at the same time, the sense of dominance was lower in the control group. In conclusion, our results support a heart rate deceleration during exposure to a disgusting stimulus dissociated from the subjective experience. Full article
(This article belongs to the Special Issue Sensors for Affective Computing and Sentiment Analysis)
►▼ Show Figures

Figure 1

26 pages, 10171 KiB  
Article
Detecting Moments of Stress from Measurements of Wearable Physiological Sensors
by Kalliopi Kyriakou, Bernd Resch, Günther Sagl, Andreas Petutschnig, Christian Werner, David Niederseer, Michael Liedlgruber, Frank H. Wilhelm, Tess Osborne and Jessica Pykett
Sensors 2019, 19(17), 3805; https://doi.org/10.3390/s19173805 - 03 Sep 2019
Cited by 123 | Viewed by 14922
Abstract
There is a rich repertoire of methods for stress detection using various physiological signals and algorithms. However, there is still a gap in research efforts moving from laboratory studies to real-world settings. A small number of research has verified when a physiological response [...] Read more.
There is a rich repertoire of methods for stress detection using various physiological signals and algorithms. However, there is still a gap in research efforts moving from laboratory studies to real-world settings. A small number of research has verified when a physiological response is a reaction to an extrinsic stimulus of the participant’s environment in real-world settings. Typically, physiological signals are correlated with the spatial characteristics of the physical environment, supported by video records or interviews. The present research aims to bridge the gap between laboratory settings and real-world field studies by introducing a new algorithm that leverages the capabilities of wearable physiological sensors to detect moments of stress (MOS). We propose a rule-based algorithm based on galvanic skin response and skin temperature, combing empirical findings with expert knowledge to ensure transferability between laboratory settings and real-world field studies. To verify our algorithm, we carried out a laboratory experiment to create a “gold standard” of physiological responses to stressors. We validated the algorithm in real-world field studies using a mixed-method approach by spatially correlating the participant’s perceived stress, geo-located questionnaires, and the corresponding real-world situation from the video. Results show that the algorithm detects MOS with 84% accuracy, showing high correlations between measured (by wearable sensors), reported (by questionnaires and eDiary entries), and recorded (by video) stress events. The urban stressors that were identified in the real-world studies originate from traffic congestion, dangerous driving situations, and crowded areas such as tourist attractions. The presented research can enhance stress detection in real life and may thus foster a better understanding of circumstances that bring about physiological stress in humans. Full article
(This article belongs to the Special Issue Sensors for Affective Computing and Sentiment Analysis)
►▼ Show Figures

Figure 1

21 pages, 572 KiB  
Article
An Empirical Study Comparing Unobtrusive Physiological Sensors for Stress Detection in Computer Work
by Fatema Akbar, Gloria Mark, Ioannis Pavlidis and Ricardo Gutierrez-Osuna
Sensors 2019, 19(17), 3766; https://doi.org/10.3390/s19173766 - 30 Aug 2019
Cited by 23 | Viewed by 5350
Abstract
Several unobtrusive sensors have been tested in studies to capture physiological reactions to stress in workplace settings. Lab studies tend to focus on assessing sensors during a specific computer task, while in situ studies tend to offer a generalized view of sensors’ efficacy [...] Read more.
Several unobtrusive sensors have been tested in studies to capture physiological reactions to stress in workplace settings. Lab studies tend to focus on assessing sensors during a specific computer task, while in situ studies tend to offer a generalized view of sensors’ efficacy for workplace stress monitoring, without discriminating different tasks. Given the variation in workplace computer activities, this study investigates the efficacy of unobtrusive sensors for stress measurement across a variety of tasks. We present a comparison of five physiological measurements obtained in a lab experiment, where participants completed six different computer tasks, while we measured their stress levels using a chest-band (ECG, respiration), a wristband (PPG and EDA), and an emerging thermal imaging method (perinasal perspiration). We found that thermal imaging can detect increased stress for most participants across all tasks, while wrist and chest sensors were less generalizable across tasks and participants. We summarize the costs and benefits of each sensor stream, and show how some computer use scenarios present usability and reliability challenges for stress monitoring with certain physiological sensors. We provide recommendations for researchers and system builders for measuring stress with physiological sensors during workplace computer use. Full article
(This article belongs to the Special Issue Sensors for Affective Computing and Sentiment Analysis)
►▼ Show Figures

Figure 1

32 pages, 10353 KiB  
Article
Predicting Depression, Anxiety, and Stress Levels from Videos Using the Facial Action Coding System
by Mihai Gavrilescu and Nicolae Vizireanu
Sensors 2019, 19(17), 3693; https://doi.org/10.3390/s19173693 - 25 Aug 2019
Cited by 78 | Viewed by 11421
Abstract
We present the first study in the literature that has aimed to determine Depression Anxiety Stress Scale (DASS) levels by analyzing facial expressions using Facial Action Coding System (FACS) by means of a unique noninvasive architecture on three layers designed to offer high [...] Read more.
We present the first study in the literature that has aimed to determine Depression Anxiety Stress Scale (DASS) levels by analyzing facial expressions using Facial Action Coding System (FACS) by means of a unique noninvasive architecture on three layers designed to offer high accuracy and fast convergence: in the first layer, Active Appearance Models (AAM) and a set of multiclass Support Vector Machines (SVM) are used for Action Unit (AU) classification; in the second layer, a matrix is built containing the AUs’ intensity levels; and in the third layer, an optimal feedforward neural network (FFNN) analyzes the matrix from the second layer in a pattern recognition task, predicting the DASS levels. We obtained 87.2% accuracy for depression, 77.9% for anxiety, and 90.2% for stress. The average prediction time was 64 s, and the architecture could be used in real time, allowing health practitioners to evaluate the evolution of DASS levels over time. The architecture could discriminate with 93% accuracy between healthy subjects and those affected by Major Depressive Disorder (MDD) or Post-traumatic Stress Disorder (PTSD), and 85% for Generalized Anxiety Disorder (GAD). For the first time in the literature, we determined a set of correlations between DASS, induced emotions, and FACS, which led to an increase in accuracy of 5%. When tested on AVEC 2014 and ANUStressDB, the method offered 5% higher accuracy, sensitivity, and specificity compared to other state-of-the-art methods. Full article
(This article belongs to the Special Issue Sensors for Affective Computing and Sentiment Analysis)
►▼ Show Figures

Graphical abstract

43 pages, 3102 KiB  
Article
Game-Calibrated and User-Tailored Remote Detection of Stress and Boredom in Games
by Fernando Bevilacqua, Henrik Engström and Per Backlund
Sensors 2019, 19(13), 2877; https://doi.org/10.3390/s19132877 - 28 Jun 2019
Cited by 14 | Viewed by 6533
Abstract
Emotion detection based on computer vision and remote extraction of user signals commonly rely on stimuli where users have a passive role with limited possibilities for interaction or emotional involvement, e.g., images and videos. Predictive models are also trained on a group level, [...] Read more.
Emotion detection based on computer vision and remote extraction of user signals commonly rely on stimuli where users have a passive role with limited possibilities for interaction or emotional involvement, e.g., images and videos. Predictive models are also trained on a group level, which potentially excludes or dilutes key individualities of users. We present a non-obtrusive, multifactorial, user-tailored emotion detection method based on remotely estimated psychophysiological signals. A neural network learns the emotional profile of a user during the interaction with calibration games, a novel game-based emotion elicitation material designed to induce emotions while accounting for particularities of individuals. We evaluate our method in two experiments ( n = 20 and n = 62 ) with mean classification accuracy of 61.6%, which is statistically significantly better than chance-level classification. Our approach and its evaluation present unique circumstances: our model is trained on one dataset (calibration games) and tested on another (evaluation game), while preserving the natural behavior of subjects and using remote acquisition of signals. Results of this study suggest our method is feasible and an initiative to move away from questionnaires and physical sensors into a non-obtrusive, remote-based solution for detecting emotions in a context involving more naturalistic user behavior and games. Full article
(This article belongs to the Special Issue Sensors for Affective Computing and Sentiment Analysis)
►▼ Show Figures

Figure 1

16 pages, 1178 KiB  
Article
Predicting Group Contribution Behaviour in a Public Goods Game from Face-to-Face Communication
by Ehsan Othman, Frerk Saxen, Dmitri Bershadskyy, Philipp Werner, Ayoub Al-Hamadi and Joachim Weimann
Sensors 2019, 19(12), 2786; https://doi.org/10.3390/s19122786 - 21 Jun 2019
Cited by 8 | Viewed by 3999
Abstract
Experimental economic laboratories run many studies to test theoretical predictions with actual human behaviour, including public goods games. With this experiment, participants in a group have the option to invest money in a public account or to keep it. All the invested money [...] Read more.
Experimental economic laboratories run many studies to test theoretical predictions with actual human behaviour, including public goods games. With this experiment, participants in a group have the option to invest money in a public account or to keep it. All the invested money is multiplied and then evenly distributed. This structure incentivizes free riding, resulting in contributions to the public goods declining over time. Face-to-face Communication (FFC) diminishes free riding and thus positively affects contribution behaviour, but the question of how has remained mostly unknown. In this paper, we investigate two communication channels, aiming to explain what promotes cooperation and discourages free riding. Firstly, the facial expressions of the group in the 3-minute FFC videos are automatically analysed to predict the group behaviour towards the end of the game. The proposed automatic facial expressions analysis approach uses a new group activity descriptor and utilises random forest classification. Secondly, the contents of FFC are investigated by categorising strategy-relevant topics and using meta-data. The results show that it is possible to predict whether the group will fully contribute to the end of the games based on facial expression data from three minutes of FFC, but deeper understanding requires a larger dataset. Facial expression analysis and content analysis found that FFC and talking until the very end had a significant, positive effect on the contributions. Full article
(This article belongs to the Special Issue Sensors for Affective Computing and Sentiment Analysis)
►▼ Show Figures

Figure 1

31 pages, 4611 KiB  
Article
Evaluating and Validating Emotion Elicitation Using English and Arabic Movie Clips on a Saudi Sample
by Sharifa Alghowinem, Roland Goecke, Michael Wagner and Areej Alwabil
Sensors 2019, 19(10), 2218; https://doi.org/10.3390/s19102218 - 14 May 2019
Cited by 13 | Viewed by 4964
Abstract
With the advancement of technology in both hardware and software, estimating human affective states has become possible. Currently, movie clips are used as they are a widely-accepted method of eliciting emotions in a replicable way. However, cultural differences might influence the effectiveness of [...] Read more.
With the advancement of technology in both hardware and software, estimating human affective states has become possible. Currently, movie clips are used as they are a widely-accepted method of eliciting emotions in a replicable way. However, cultural differences might influence the effectiveness of some video clips to elicit the target emotions. In this paper, we describe several sensors and techniques to measure, validate and investigate the relationship between cultural acceptance and eliciting universal expressions of affect using movie clips. For emotion elicitation, a standardised list of English language clips, as well as an initial set of Arabic video clips are used for comparison. For validation, bio-signal devices to measure physiological and behavioural responses associated with emotional stimuli are used. Physiological and behavioural responses are measured from 29 subjects of Arabic background while watching the selected clips. For the six emotions’ classification, a multiclass SVM (six-class) classifier using the physiological and behavioural measures as input results in a higher recognition rate for elicited emotions from Arabic video clips (avg. 60%) compared to the English video clips (avg. 52%). These results might reflect that using video clips from the subjects’ culture is more likely to elicit the target emotions. Besides measuring the physiological and behavioural responses, an online survey was carried out to evaluate the effectiveness of the selected video clips in eliciting the target emotions. The online survey, having on average 220 respondents for each clip, supported the findings. Full article
(This article belongs to the Special Issue Sensors for Affective Computing and Sentiment Analysis)
►▼ Show Figures

Figure 1

14 pages, 1886 KiB  
Article
Using Eye Tracking to Assess Gaze Concentration in Meditation
by Kang-Ming Chang and Miao-Tien Wu Chueh
Sensors 2019, 19(7), 1612; https://doi.org/10.3390/s19071612 - 03 Apr 2019
Cited by 6 | Viewed by 4658
Abstract
An important component of Heart Chan Meditation is gaze concentration training. Here, we determine whether eye tracking can be used to assess gaze concentration ability. Study participants (n = 306) were requested to focus their gaze on the innermost of three concentric circles [...] Read more.
An important component of Heart Chan Meditation is gaze concentration training. Here, we determine whether eye tracking can be used to assess gaze concentration ability. Study participants (n = 306) were requested to focus their gaze on the innermost of three concentric circles for 1 min while their eye movements were recorded. Results suggest that participants with high scores on gaze concentration accuracy and precision had lower systolic blood pressure and higher sleep quality, suggesting that eye tracking may be effective to assess and train gaze concentration within Heart Chan Meditation. Full article
(This article belongs to the Special Issue Sensors for Affective Computing and Sentiment Analysis)
►▼ Show Figures

Graphical abstract

Show export options expand_more Show export options expand_less
Select all
Export citation of selected articles as:
 
Displaying articles 1-16
Back to TopTop