Emotion Recognition: An Evaluation of ERP Features Acquired from Frontal EEG Electrodes

Round 1

Reviewer 1 Report

The preset manuscript reports an attempt to classify emotional states along arousal/valence axis based on ERP elicited by presentation of IAPS pictures. By  SVM polynomial classifier, the authors succeeded in classifying four emotional states above levels reported in the previous studies. 

Major Points:

In my understanding, the authors used only 40 images. They say that some of them "had to be repeated", but do not specify how many trials had been conducted to obtain ERP in each of the four classes. Worse, the authors used "a mechanically connected keyboard" to synchronize EEG data and timing of stimulus onset, which is quite unusual in ERP research. These make me doubt whether the authors actually succeeded in recording clear ERP components (without artifact correction by ICA).

Before evaluating the soundness of the results, I have to see some example ERPs of individual participants that should possibly included as figures or supplementary materials. 

It reads from the manuscript that the authors made no attempts to mitigate the influences of blink  and eye-movement artifacts (aside from instructing the participants to refrain from blinking and moving their eyes). It is quite conceivable that rate of blink and eye-movement pattern differ depending on emotional state. Additionally, if trial number is small, these artifacts easily change the waveforms, hence measured amplitude/latency of ERP component. I strongly recommend removing ocular artefacts at preprocessing. 

Minor Points:

"1. Introduction": sentences with almost identical meaning are repeated several times in this section. Needs substantial editing. 

"2. Related Studies": Too detailed and cumbersome. The authors had better delete some details and get straight to the point. Adding sub-header would be helpful.

"5. Preprocessing Operations on EEG and Feature Selection"

Specify in which electrode each ERP component was measured. 

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Reviewer 2 Report

The manuscript proposes a methodology for emotion recognition based on EEG signals. The authors have obtained ERP recordings from 24 subjects and evaluated their methodology on 4 binary classification problems. The objective is clear and of scientific interest and the manuscript is well-written, with good structure and interesting outcomes. I enjoyed reading it. Below are some minor comments:

1. In the Introduction, the authors describe very well the emotional behavior and introduce the reader to the manuscript’s objective adequately. However, it should be stated that the EEG is a diagnostic tool for epilepsy and helps in the differential diagnosis of several other neurological disorders (e.g. dementia, lupus erythematosus etc) to avoid any possible misunderstanding on the use of EEG. Thus, please insert the following recently published studies on EEG-based brain disorder diagnosis:

Usman, S. M., Khalid, S., & Bashir, Z. (2021). Epileptic seizure prediction using scalp electroencephalogram signals. Biocybernetics and Biomedical Engineering, 41(1), 211-220.

Tzimourta, K. D., Christou, V., Tzallas, A. T., Giannakeas, N., Astrakas, L. G., Angelidis, P., ... & Tsipouras, M. G. (2021). Machine Learning Algorithms and Statistical Approaches for Alzheimer's Disease Analysis Based on Resting-State EEG Recordings: A Systematic Review. International Journal of Neural Systems, 2130002-33.

And also, the following recently published studies on EEG-based emotion recognition:

Li, Y., & Zheng, W. (2021). Emotion Recognition and Regulation Based on Stacked Sparse Auto-Encoder Network and Personalized Reconfigurable Music. Mathematics, 9(6), 593.

Kong, T., Shao, J., Hu, J., Yang, X., Yang, S., & Malekian, R. (2021). EEG-Based Emotion Recognition Using an Improved Weighted Horizontal Visibility Graph. Sensors, 21(5), 1870.

2. I would suggest Lines 234-245 to form a separate paragraph describing the EEG data acquisition system.
3. Line 235 “MP150 data ac-235 quisition system provided by Biopac.” Please insert a reference.
4. Line 253. Please replace the word “wished” with “aimed”
5. The text “We did not use ICA and artifact removal techniques in accordance with conclusion of Jenke et al. (2014) as these 311 operations did not reportedly impact the classification results considerably but added to 312 offline processing time.” (Lines 310-313) discusses the EEG preprocessing method; thus, it belongs to Discussion.
6. The text “Frantzidis et al. (2010) achieved the best four class classification accuracy using ERPs as attributes. Moreover, the methodology used has not been validated by any researcher to the best of our knowledge. The emotion evocation method, the attribute selection method and emotion classification technique are in fact influenced by the study of Frantzidis et al. (2010).” (Lines 225-229) discusses the EEG methodology; thus, it belongs to Discussion.
7. Line 327. Please insert the full term “Support Vector Machines” the first time it is shown in the manuscript and then the abbreviation “SVM”. Also, a small paragraph describing the main aspects of SVM should be included. Same with 10-fold cross-validation (Line 329)
8. Line 361. Please insert a gap between the words “fromfrontal”
9. The caption of Figure 5 should claim that this is a flowchart of the three-classifier methodology
10. Please cite the referring study in Line 390 “The results of course are better than an existing study.”
11. At Table 11 please fix the alignment of the column “Negative Predictive Value (%)”
12. References must be numbered in order of appearance in the text. In the text, reference numbers should be placed in square brackets [ ], and placed before the punctuation; for example [1], [1–3] or [1,3]. Please follow the official journal template and revise.

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Reviewer 3 Report

This paper proposes a method to analyze and classify (human) emotions by representing them in a 2-dim space with the two dimensions arousal and valence. The classification into the four resulting quadrants (classes) is based on the use of two types of Event Related Potential (ERP) attributes (average ERPs with latencies and difference of average ERPs and latencies) acquired from frontal EEG electrodes of 24 human subjects. Using three classifiers, a very high accuracy of classification has been achieved.

Although I am not an expert in EEG analysis, I agree with the proposal and acknowledge its analytical value, especially the high accuracy of classification, but I have issues with the scientific, theoretical embedding. My conclusion is that the authors did a very good job with respect to the methods and results, and therefore I recommend publication. I only would like to recommend revision of the Introduction, which seems to me being rather selective and not covering the scientific background of the work well enough.

At the beginning of the Introduction the authors present a most commonly accepted definition of emotion, but a few sentences later say that “It is very widely admitted that modern day researchers and scholars do not agree on the definition of emotions”. This is quite confusing and needs to be revised. Especially problematic is the end of the Introduction, with referring to someone who claims that “a proper definition of emotion is not at all necessary for fruitful research”. I do not agree, because how would then anyone be able to describe emotions and interpret (physiological, behavioral etc.) data? EEG waves to not speak for themselves! It is our duty as researchers to translate data into concepts, to attach meaning to them. Without a definition, that is, without a concept of what emotions are, we would not be able to speak about them. And if we remain in a state of confusion and disagreement, we will never be able to properly communicate and productively contribute to an understanding of the phenomena that are important to us. Despite the obvious fact that the emotion construct has a varied, fuzzy and complex nature, we would benefit a lot if we try to construct a definition, whether it is descriptive or prescriptive (Widen, S.C., Russell, J.A., 2010. Descriptive and prescriptive definitions of emotion. Emot. Rev. 2, 377–378).

A prescriptive definition is the better option, as it is a definition of the concept or construct of emotion that is used to pick out the set of events that a scientific theory of emotion purports to explain. It uses key features of what people think of (i.e., descriptively define as) emotions and say “this is what I am talking about when I use the term emotion in my research” (e.g.; Izard, C.E., 2010. The many meanings/aspects of emotion: definitions, functions, activation and regulation. Emot. Rev. 2, 363–370). Exactly this is what I would like to see the authors of this study doing in the Introduction.

In recent years a number of researchers have offered definitions either explicitly for the purpose of investigating emotion in animals and humans alike (Lang, P.J., 2010. Emotion and motivation: toward consensus definitions and a common research purpose. Emot. Rev. 2, 229–233; LeDoux, J., Phelps, L., Alberini, C., 2016. What we talk about when we talk about emotions. Cell 167, 1443–1445; Rolls, E.T., 2014. Emotion and Decision-Making Explained. Oxford University Press, Oxford) or implicitly, focusing on particular, objectively identifiable facets of emotional processing in non-human species (e.g., Panksepp, J., 1982. Toward a general psycho-biological theory of emotions. Behav. Brain Sci. 5, 407–422.).

For instance, Anderson and Adolphs (2014. A framework for studying emotions across species. Cell 157, 187–200) define emotion very broadly as “an internal CNS state that gives rise to physiological, behavioural, cognitive (& subjective) responses”. Central to their definition is the proposal that (human) emotions comprise four key components or building blocks: Valence, Scalability, Persistence and Generalisation. These building blocks evolved to serve many independent behavioral and cognitive functions, but over time they have combined in the brain to produce what we humans now regard as emotion states. It would be interesting to see if these four dimensions could also be used by Singh and Singh to classify human emotions.

The authors of this manuscript refer to, and use as basis of their work, the study by Frantzidis et al. (2010). They classified neurophysiological data into four emotional classes, namely, high valence high-arousal (HVHA), low-valence high-arousal (LVHA), high-valence low-arousal (HVLA) and low-valence low-arousal (LVLA). But actually, this 2-dim emotion space has been developed long before (e.g., Russell, J. A. & Barrett, L. F. 1999 Core affect, prototypical emotional episodes, and other things called emotion: dissecting the elephant. J. Pers. Soc. Psychol. 76, 805 – 819; Watson, D., Wiese, D., Vaidya, J. & Tellegen, A. 1999 The two general activation systems of affect: structural findings, evolutionary considerations, and psychobiological evidence. J. Pers. Soc. Psychol. 76, 820–838). Unfortunately, these original proposals are not mentioned in this manuscript by Singh and Singh.

Also, the work of Stanley & Meyer (2009. Two-dimensional affective space: a new approach to orienting the axes. Emotion 9, 214–237) is relevant here, as these authors have defined emotions in terms of these two fundamental dimensions, valence and arousal. And the subjective experiences that can be characterized in terms of these valence and arousal dimensions have been labelled core affect, which should also be mentioned in the Introduction (Russell, J. A. 2003 Core affect and the psychological construction of emotion. Psychol. Rev. 110, 145 – 172; Barrett, L. F., Mesquita, B., Ochsner, K. N. & Gross, J. J. 2007 The experience of emotion. Ann. Rev. Psychol. 58, 373 – 403). Later in the manuscript, Singh and Singh do mention Russell’s model of Circumplex, but there is no reference.

To summarize, I would like to see the original, core publications for a) the definition of emotions and b) the classification of emotions properly described in the Introduction in order to set the stage and to provide the scientific background of this study.

Author Response

Please see the attachment.

Author Response File: Author Response.docx