EEG-Based Emotion Recognition Using Logistic Regression with Gaussian Kernel and Laplacian Prior and Investigation of Critical Frequency Bands

Abstract

Emotion plays a nuclear part in human attention, decision-making, and communication. Electroencephalogram (EEG)-based emotion recognition has developed a lot due to the application of Brain-Computer Interface (BCI) and its effectiveness compared to body expressions and other physiological signals. Despite significant progress in affective computing, emotion recognition is still an unexplored problem. This paper introduced Logistic Regression (LR) with Gaussian kernel and Laplacian prior for EEG-based emotion recognition. The Gaussian kernel enhances the EEG data separability in the transformed space. The Laplacian prior promotes the sparsity of learned LR regressors to avoid over-specification. The LR regressors are optimized using the logistic regression via variable splitting and augmented Lagrangian (LORSAL) algorithm. For simplicity, the introduced method is noted as LORSAL. Experiments were conducted on the dataset for emotion analysis using EEG, physiological and video signals (DEAP). Various spectral features and features by combining electrodes (power spectral density (PSD), differential entropy (DE), differential asymmetry (DASM), rational asymmetry (RASM), and differential caudality (DCAU)) were extracted from different frequency bands (Delta, Theta, Alpha, Beta, Gamma, and Total) with EEG signals. The Naive Bayes (NB), support vector machine (SVM), linear LR with L1-regularization (LR_L1), linear LR with L2-regularization (LR_L2) were used for comparison in the binary emotion classification for valence and arousal. LORSAL obtained the best classification accuracies (77.17% and 77.03% for valence and arousal, respectively) on the DE features extracted from total frequency bands. This paper also investigates the critical frequency bands in emotion recognition. The experimental results showed the superiority of Gamma and Beta bands in classifying emotions. It was presented that DE was the most informative and DASM and DCAU had lower computational complexity with relatively ideal accuracies. An analysis of LORSAL and the recently deep learning (DL) methods is included in the discussion. Conclusions and future work are presented in the final section.

1. Introduction

Affective computing defined by Picard [1] is a multidisciplinary research field that relates to computer science, psychology, neuroscience, and cognitive science. Levenson [2] believed that during natural selection, emotions were preserved for the necessity of rapid response mechanisms when facing different environmental threats. Emotion plays a nuclear role in human behavior, such as perception, attention, decision-making, and communication [3]. Positive emotions contribute to healthy life and efficient work, while negative emotions may result in health problems [4].

Emotion recognition methods include two main categories, according to the methods humans communicate emotions, including body expressions, and physiological signals. Body expressions are physical manifestations and easy to be collected. Theorists argue that each emotion corresponds to its unique somatic response [1]. However, human physical manifestations are easily affected by the user’s cultural background and social environment [4]. The physiological signals [3,4] are internal signals, such as electroencephalogram (EEG), electrocardiogram (ECG), heart rate (HR), electromyogram (EMG), and galvanic skin response (GSR). According to Connon’s theory [5], the emotion changes are associated with quick responses in physiological signals coordinated by the autonomic nervous systems (ANS). This makes the physiological signals not easily controlled and overcome the shortcomings of body expressions [4]. Physiological signals have been widely applied in many studies for emotion recognition [3,4]. These physiological signals, including ECG and EMG, are still not a direct reaction to emotion changes. According to psychology and neurophysiology, emotion generation and activity have a close relationship with the activity of the cerebral cortex. Thus, EEG signals effectively reflect the brain electrical activity, and have been widely applied in many fields, including cognitive performance prediction [6], mental load analysis [7,8], mental fatigue assessment [9], recommendation system [10] and decoding visual stimuli [11,12].

Recently, the field of EEG-based emotion recognition has attracted a lot of interest, including Brain-Computer Interaction (BCI) systems, basic emotion theories, and machine learning algorithms [13,14]. In machine learning, the definition of the emotion model is necessary to describe the objective function of the algorithms. There are mainly two kinds of models [3], discrete emotion spaces and continuous emotion models. Among these models, the valence-arousal model by Russell [15] has been widely used in emotion recognition for its simplicity to establish assessment criteria. The progress of EEG-based emotion recognition also includes feature extraction, feature selection, dimension reduction, and classification algorithms [13,14]. After the pre-processing of original EEG signals, the current work is to extract and select informative features to enhance the discriminative signal characteristics. Traditionally, feature extraction and selection are based on neuroscience and cognition science [16]. For example, frontal asymmetry in Alpha band power for differentiating valence level has attracted lots of interest in neuroscience research [17]. Besides neuro-scientific assumptions, computation methods in machine learning are also applied for feature extraction and selection in EEG-based emotion recognition [16,18]. Several studies transformed the pre-processed EEG-signal into various analysis domains, including time, frequency, statistical, and spectral domains [19]. It should be noted that only one feature extraction method is not suitable for various applications and BCI systems [19]. Although the most informative EEG features for emotion classification are still being researched, the power features obtained from different bands are widely recognized as the most popular features. In these studies [20,21,22], power spectral density (PSD) from EEG signals worked well for identifying emotional states. However, feature extraction usually generates high-dimensional and abundant features. Feature selection and dimension reduction are necessary to avoid over-specification and to reduce computational burden [3]. Compared to filter and wrapper methods for feature selection, the dimension reduction methods, e.g., principal component analysis (PCA), and Fisher linear discriminant (FLD), are more efficient. For further information about feature selection and dimension reduction, we refer the reader to [23,24]. Many machine learning algorithms have been introduced as EEG-based emotion classifiers, such as support vector machine (SVM) [25,26], Naive Bayes (NB) [27], K-nearest neighbors (KNN), linear discriminant analysis (LDA), random forest (RF), and artificial neural networks (ANN). Among these methods, SVM based on spectral features, e.g., PSD, is the most widely applied approach. In [25], SVM was used to classify the joy, sadness, anger, and pleasure feelings based on the EEG signals from 12 symmetric electrodes pairs. SVM was used in [26] for emotion recognition with the accuracies 32% and 37% in valence and arousal dimensions, respectively. A Gaussian NB in [27] was used to classify low/high valence, and arousal emotion with precision of 57.6% and 62.0%, respectively.

Recently, deep learning (DL) methods have been introduced for EEG-based emotion classification [28,29]. The study [30,31] proposed deep belief network (DBN) to discriminate positive, neutral, and negative emotions. The experimental results show that DBN performs better than SVM and KNN. In [32], after an effective pre-processing method instead of traditional feature extraction methods, a hybrid neural network combining convolutional neural network (CNN) and recurrent neural network (RNN) is proposed to learn spatial-temporal representation from the pre-processed EEG recordings. The proposed pre-processing strategy improves the emotion recognition accuracies by about 33% and 30% for valence and arousal dimensions, respectively. In [33], a deep CNN (DCNN) model is introduced to learn discriminative representations from the combined features in the raw time domain, after normalization, and in the frequency domain. The obtained emotion classification accuracies are higher than the traditionally best bagging tree (BT) classifier. The study [34] proposed a hierarchical bidirectional gated recurrent unit (GRU) network with an attention mechanism. The proposed scheme learned more significant representation from EEG sequences and the accuracies obtained on cross-subject emotion classification task outperformed the long short time memory (LSTM) network by 4.2% and 4.6% in valence and arousal dimensions, respectively. Compared to traditional shallow methodologies, the DL models remove the signal pre-processing and feature extraction/selection progress, and are more suitable for affective representation [35,36]. However, the DL methods cannot reveal the relationship between emotional states and EEG signals for being like a black box [37]. Moreover, the training of DL networks is extremely computationally time-consuming, which limits their practical applications in real-time emotion recognition [3].

As aforementioned, the field of affective computing has developed a lot over the past several years, including the incorporation of DL methodologies. However, the modeling and recognition of emotional states is still an unexplored problem [13,14]. EEG-based emotion recognition is still faced with several challenges, including fuzzy boundaries between emotions.

Note that logistic regression (LR) [38] has been widely used as a statistical learning model in pattern recognition and machine learning, as well as in EEG signal processing. In [39], LR trained with EEG power spectral features was used for automatic epilepsy diagnosis. The work in [40] further used wavelet transform to extract effective representation from non-stationary EEG records and adopted LR as a classifier to identify epileptic and non-epileptic seizures. In [41], regularized linear LR was trained using the raw EEG signal without feature extraction to classify imaginary movements. In [42], LR with L2-penalization to avoid overfitting was trained using spectral power features from intracranial EEG (iEEG) signals for the analysis of the brain’s encoding states and memory performance. The study in [43] further incorporated t-distributed stochastic neighbor embedding (tSNE) for dimension reduction of iEEG signals, and the learned L2-regularized LR classier was used for predicting memory encoding success. Despite the above studies, the potential of the LR model for EEG-based emotion recognition is still not fully explored.

In this present study, we systematically introduced the logistic regression (LR) algorithm with Gaussian kernel and Laplacian prior [44,45,46] for EEG-based emotion recognition. Different from these LR classifiers, Gaussian radial basis function (RBF) kernel was used to enhance the data separability in the transformed space [46]. Moreover, Laplacian prior promoting the sparsity of logistic regressors was acted as L1-regularization [44]. This prior forces many components of logistic regressors to be zero. Thus, the learned logistic regressors with sparseness control the complexity of the LR classifier and consequently avoids over-specification in EEG-based emotion recognition. The logistic regression via variable splitting and augmented Lagrangian (LORSAL) algorithm [45] was introduced to optimize the logistic regressors for lower computational complexity. Thus, the introduced LR method is abbreviated as LORSAL. For overall evaluation of the LORSAL classifier, various power spectral features and features calculated by combinations of electrodes were used as input for the classifiers. The conventional NB, SVM, linear LR with L1-regularization (LR_L1), linear LR with L2-regularization (LR_L2) were used for comparison to evaluate the performance of the LORSAL classifier. This paper also presents an investigation of critical frequency bands [47,48] and an analysis of the effect of extracted features for EEG-based emotion classification.

The rest of this paper is organized as follows. Section 2 presents the materials and methods, including the dataset for emotion analysis using EEG, physiological and video signals (DEAP), various features extracted from the EEG signals, the introduced LR model with Gaussian kernel and Laplacian prior, and the LORSAL algorithm to learn LR regressors. The experimental results are shown in Section 3. The introduced method is evaluated in the task of subject-dependent emotion recognition in valence and arousal dimensions, and the compared methods include NB, SVM, LR_L1, and LR_L2. Section 4 gives the discussion and a further comparison of LORSAL and the DL methods. Related conclusion and future work are presented in Section 5.

2. Materials and Methods

2.1. DEAP Dataset and Pre-Processing

This study was performed on the dataset DEAP developed by researchers at Queen Mary University of London [27]. This dataset is publicly available (http://www.eecs.qmul.ac.uk/mmv/ datasets/deap/index.html) and consists of multimodal physiological signals for human emotion analysis. It contains, in total, 32 EEG-channel recordings and eight peripheral signals of 32 subjects (50 percent females, aged between 19 and 37). The carefully selected 40 1-min videos were used as emotion elicitation materials [27]. As shown in Figure 1, the 2D valence-arousal emotion model by Russell [15] was used to quantitatively describe emotional states. The first dimension, valence, ranges from unpleasant to pleasant, and the second dimension, arousal, changes from bored to excited. Therefore, the valence-arousal model can describe most variations in human emotion changes. The well-known self-assessment manikins (SAM) [49] (shown in Figure 2) were adopted for self-assessment along the valence and arousal dimensions, and the corresponding discrete rating values change from 1 to 9, which can be used as identification labels in emotion analysis tasks [27]. In this paper, the first 32-channel EEG records (marker in Figure 3) in the DEAP dataset preprocessed in MATLAB format were used. The EEG signals were preprocessed by down-sampling from 512 Hz to 128 Hz, and then band-pass filtering with 4–45 Hz.

In this work, two different binary classification problems were posed for subject-dependent emotion recognition: The discrimination of low/high valence (LV/HV), and low/high arousal (LA/HA). The subjects’ ratings (scaling from 1 to 9) by SAM in the experiments [27] were used as the ground truth and the threshold was selected as 5 to divide the rating values into two categories: LV/HV and LA/HA. The time duration of one trail for each subject in the preprocessed EEG sequences is 63 s, in which the first 3 s are baseline signals before watching video elicitations. The 3 s sequences were removed to obtain the stimulus-related dynamics. The remaining 60-s EEG signals (thus, 7680 readings in each EEG channel, in total) were segmented into sixty 1 s epochs. Thus, there were 40 * 60 EEG epochs, in total, for each participant. Each subject-dependent EEG data had a dimensionality of 128 (sampling points) * 32 (EEG channels) * 2400 (EEG epochs). Finally, we obtained the labeled EEG signals with the dimension of 2400 for each subject. In this paper, for each subject, 10% of labeled epochs were used to train the emotion classifier, and the remaining 90% for test. For example, the constructed EEG dataset for the first participant consisted of 960 LV and 1440 HV epochs. Then, 10% of samples were randomly selected from LV and HV samples, respectively, and 240 epochs were selected for training. Ten-fold cross-validation was used to evaluate the introduced LORSAL classifier, and the compared traditional methods.

2.2. Feature Extraction

In this study, various power spectral features in the frequency domain and features calculated by combinations of electrodes were extracted from the constructed EEG signals. The extraction of prominent statistical characteristics is important for emotion recognition. The physiological signals, e.g., EEG, are characterized with high complexity and non-stationarity, and power spectral density (PSD) [20,21,22] from different frequency bands is the most well-known applicable statistical feature in the task of emotion analysis. This benefits from the assumption that EEG signals are stationary for the duration of a trail [50]. Many studies in neuroscience and psychology [51] suggest that these five frequency bands are closely linked to psychological activities, including the emotion activity: Delta (1 Hz–3 Hz), Theta (4 Hz–7 Hz), Alpha (8 Hz–13 Hz), Beta (14 Hz–30 Hz), and Gamma (31–50 Hz). The fast Fourier transform (FFT) can be applied using discrete Fourier transform (DFT) [52], while the common alternatives are short-time Fourier transform (STFT) [53,54]. PSD features are extracted from the above five frequency bands using 256-point STFT and a sliding 0.5 s Hanning window with 0.25 s overlapping along 1 s epoch for each EEG channel.

Differential entropy (DE) [55,56] is a measurement of the complexity of a continuous random variable by extending the Shannon entropy concept [57]. These studies by Zheng et al. [47,48] and Duan et al. [56] introduced DE for emotion classification using EEG low/high-frequency patterns.

The original formula of DE is defined as

$$h(X)=-{\displaystyle {\int }_{X}f(x)\log (f(x))dx},$$

(1)

and DE when a random variable X obeys the Gaussian distribution N(μ, σ²) can be simply given as:

$$h(X)=-{\displaystyle {\int }_{-\infty }^{+\infty }\frac{1}{\sqrt{2\pi {\sigma }^2}}\exp \frac{{(x-\mu )}^2}{2{\sigma }^2}\log \frac{1}{\sqrt{2\pi {\sigma }^2}}\exp \frac{{(x-\mu )}^2}{2{\sigma }^2}dx}=\frac{1}{2}\log 2\pi e{\sigma }^2,$$

(2)

where π and e are constants. According to [55], given a certain frequency band, DE equals to the logarithmic spectral energy for a fixed-length EEG recording. Thus, the DE features are calculated in the five frequency bands as for the PSD features.

In the literature [58,59], the asymmetric brain activity between the left and right hemispheres is of high relation with emotions. In the studies [47,48], the differential asymmetry (DASM) and rational asymmetry (RASM) features were defined as the differences and ratios of the DE features of hemisphere asymmetric electrodes. Here, 14 pairs of asymmetric electrodes are selected to calculate DASM and RASM: Fp1-Fp2, F7-F8, F3-F4, T7-T8, P7-P8, C3-C4, P3-P4, O1-O2, AF3-AF4, FC5-FC6, FC1-FC2, CP5-CP6, CP1-CP2, and PO3-PO4. The DASM and RASM features are given as

$$\mathrm{DASM}=\mathrm{DE}(X_{left})-\mathrm{DE}(X_{right}),$$

(3)

and

$$\mathrm{RASM}=\mathrm{DE}(X_{left})/\mathrm{DE}(X_{right})$$

(4)

respectively. Due to the studies suggested in [58,59], the emotional states are closely linked to the spectral differences of brain activity between frontal and posterior brain regions. The definition of differential caudality (DCAU) features [47,48] was also adopted in this paper to characterize the spectral asymmetry in frontal-posterior direction. The DCAU features are given as the differences between 11 pairs of frontal-posterior electrodes: FC5-CP5, FC1-CP1, FC2-CP2, FC6-CP6, F7-P7, F3-P3, Fz-Pz, F4-P4, F8-P8, Fp1-O1, Fp2-O2. The formulation of DCAU is defined as

$$\mathrm{DCAU}=\mathrm{DE}(X_{frontal})-\mathrm{DE}(X_{posterior}).$$

(5)

The dimensions of the PSD, DE, DSAM, RASM, and DCAU features are 160 (32 channels * 5 bands), 160 (32 channels * 5 bands), 70 (14 pairs of electrodes * 5 bands), 70 (14 pairs of electrodes * 5 bands), 55 (11 pairs of electrodes * 5 bands), respectively. For simplicity, the above-extracted features were used directly and separately as input for the introduced and compared recognition methods.

2.3. Logistic Regression with Gaussian Kernel and Laplacian Prior

Logistic regression (LR) has been a common statistical learning model in pattern recognition and machine learning [38]. Strictly speaking, the applications of LR in EEG signal analysis is not new, as illustrated in the above introduction [39,40,41,42,43]. Despite this, the potential of the LR model for EEG-based emotion recognition has not been fully exploited. In this paper, we systematically introduced the logistic regression (LR) algorithm with Gaussian kernel and Laplacian prior [44,45,46] for emotion recognition with EEG signals.

The goal of a supervised learning algorithm is to train a classifier using training samples in order to recognize the label of an input feature vector from different classes. In EEG-based emotion recognition, the major task is to assign the input EEG signals to one of the given classes. Especially in this study, two binary classification problems were posed for subject-dependent emotion recognition: The classification of LV/HV emotions, and LA/HA emotions.

Using a multinomial LR (MLR) model [38,44], the probability that the input feature x_i belongs to emotion class k is written as

$$p(y_i=k|{\mathbf{x}}_i,\mathit{w})=\frac{\exp ({\mathit{w}}^{(k)}\mathbf{h}({\mathbf{x}}_i))}{{\displaystyle {\sum }_{k=1}^K\exp ({\mathit{w}}^{(k)}\mathbf{h}({\mathbf{x}}_i))}},$$

(6)

where x_i is the feature vector extracted from the original EEG sequences, and h(x_i) indicates a vector of functions of the input feature vector x_i, and $\mathit{w}\equiv {[{\mathit{w}}^{{(1)}^{\mathrm{T}}},\dots ,{\mathit{w}}^{{(K)}^{\mathrm{T}}}]}^{\mathrm{T}}$ is the logistic regressors. For binary classification tasks (K = 2), this is known as LR model, for K > 2, the usual designation is MLR [44]. Although emotion recognition in this paper is binary classification, the formula of MLR is presented here for completeness. On one hand, this does not affect the understanding of the model, on the other hand, this makes it easy to extend to the cases when handling multiple emotion classes.

Note that the function h(x_i) can be linear or nonlinear. For the latter case, kernel functions can be selected to further enhance the separability of extracted features in the transformed space. In this study, the Gaussian kernel is utilized, given by

$$K({\mathbf{x}}_i,{\mathbf{x}}_j)=\exp (-\parallel {\mathbf{x}}_i-{\mathbf{x}}_j\parallel /(2{\rho }^2)),$$

(7)

In this paper, the training of the LR classifier using labeled EEG epochs amounts to estimate the class densities and learn the logistic regressor w. Following the formulation of the sparse MLR (SMLR) algorithm in [44], the solution of w is given by the maximum a posteriori (MAP) estimate

$$\widehat{\mathit{w}}=\arg \underset{\mathit{w}}{\max }\ell (\mathit{w})+\log p(\mathit{w}),$$

(8)

where $\ell (\mathit{w})$ indicates the log-likelihood function given as following:

$$\ell (\mathit{w})=\log {\displaystyle \prod _{i=1}^{L}p(y_i|{\mathbf{x}}_i,\mathit{w})},$$

(9)

where L denotes the number of training samples, and

$$p(\mathit{w})\propto \exp (-\lambda \parallel \mathit{w}{\parallel }_1),$$

(10)

denotes the Laplacian prior, where $\parallel \mathit{w}{\parallel }_1$ indicates the L1 norm of w, and l is the regularization parameter. The Laplacian prior forces the sparsity on the logistic regressors w, and promote many components of w equal to zero [45,46]. The obtained sparse regressor reduces the complexity of the LR classifier and, therefore, avoids over-specification in EEG-based emotion classification.

The convex problem in Equation (8) is difficult to optimize for the nonquadratic property of the term $\ell (\mathit{w})$ and the non-smoothness of the term logp(w). The studies in [44,60] decomposed the problem in Equation (8) into a sequence of quadratic problems using a majorization-minimization scheme [61]. The SMLR algorithm optimizes each quadratic problem with the complexity of O(((L + 1)K)³) [44]. The fast SMLR (FSMLR) [62] is more efficient by applying a block-based Gauss–Seidel iterative procedure to estimate w. Thus, the FSMLR algorithm is K² faster than SMLR with the complexity of O((L + 1) ³K).

In this work, the logistic regression via variable splitting and augmented Lagrangian (LORSAL) [45] algorithm is introduced to solve the LR regressors in Equation (8). LORSAL has been proposed for hyperspectral image classification (HSI) in remote sensing community [45,46]. The complexity of LORSAL is O((L + 1) ²K) for each quadratic problem, compared to the O(((L + 1)K)³) and O((L + 1) ³K) complexities of the SMLR and FSMLR algorithms. Note that in this paper, we might use LORSAL directly to indicate the introduced LR with Gaussian kernel and Laplacian prior.

3. Experimental Results

In this work, we systematically investigated the classification performance of the introduced LORSAL method compared with four classifiers, Naive Bayes (NB) [27], support vector machine (SVM) [25,26], linear LR with L1-regularization (LR_L1), linear LR with L2-regularization (LR_L2) for the binary classification of the LV/HV and LA/HA emotional states. These features, including PSD, DE, DASM, RASM, and DCAU, were extracted from the EEG-signals and used directly as inputs for the classifier. The NB in MATLAB was employed as in [27]. The LIBLINEAR [63] software was adopted for the implementation of the LR_L1, and LR_L2 classifiers, respectively, with the default cost parameter. The LIBSVM [64] tool was utilized to implement the SVM classifier by using the linear kernel with default parameters. For simplicity, the parameters for Gaussian kernel and Laplacian prior in the LORSAL method were set as default in [46]. Such parameter settings may be not optimal for EEG-based emotion recognition, but present ideal classification performance in the experiments.

3.1. Overall Classification Accuracy

The mean accuracies and standard deviations obtained by different classifiers in valence dimension for different features extracted from five frequency bands (Delta, Theta, Alpha, Beta, and Gamma) and the total frequency bands are tabulated in

Table 1. The mean precisions and standard deviations (%) of the classification of LV/HV emotions obtained by the compared classifiers for different features extracted from different frequency bands.

Feature	Classifier	Delta	Theta	Alpha	Beta	Gamma	Total
PSD	NB	54.18/5.39	53.94/5.24	54.25/5.08	58.85/7.63	61.23/8.48	60.51/6.71
	SVM	51.14/14.31	47.57/15.75	52.35/16.04	62.97/12.18	64.15/12.19	69.04/5.91
	LR_L1	44.80/4.63	45.23/4.72	44.00/5.51	38.41/8.75	36.18/9.67	34.43/9.52
	LR_L2	44.57/4.88	45.11/4.92	44.14/5.49	38.47/8.63	36.38/9.58	34.38/9.57
	LORSAL	56.45/5.79	56.19/5.34	58.29/5.56	64.95/6.78	68.30/7.98	63.29/6.54
DE	NB	54.27/3.62	53.90/3.53	54.62/4.07	59.19/5.59	61.52/6.51	61.04/5.76
	SVM	52.08/13.54	49.76/14.14	52.04/13.41	62.91/10.23	65.88/9.93	69.55/6.58
	LR_L1	44.04/4.80	44.67/4.73	43.16/5.34	38.22/8.67	36.08/9.52	33.69/9.88
	LR_L2	43.86/4.88	44.44/4.69	43.15/5.41	38.15/8.64	36.06/9.52	33.81/9.59
	LORSAL	61.79/4.55	58.34/3.90	59.20/4.04	67.06/6.95	72.93/7.30	77.17/6.37
DASM	NB	54.81/6.61	53.97/5.53	54.07/6.12	60.18/6.16	62.27/6.88	62.36/5.66
	SVM	45.64/14.11	44.66/14.77	44.83/14.66	56.24/13.18	60.95/12.14	64.48/10.21
	LR_L1	45.32/4.11	45.77/3.89	45.41/4.86	40.68/7.45	38.83/8.79	36.90/8.82
	LR_L2	45.35/4.04	46.03/3.95	45.59/4.66	40.77/7.53	38.84/8.68	37.11/8.80
	LORSAL	58.38/3.58	55.21/3.55	55.42/3.46	60.31/5.39	64.63/6.22	71.63/4.94
RASM	NB	51.17/6.55	51.61/7.37	51.02/6.98	54.90/8.57	58.08/9.47	55.65/6.77
	SVM	37.41/15.45	37.23/14.33	37.35/13.87	40.56/16.24	47.29/17.02	48.17/17.72
	LR_L1	49.39/6.07	48.19/5.55	48.17/6.24	45.29/6.11	42.39/7.44	42.71/6.64
	LR_L2	49.45/4.75	47.92/4.30	47.95/4.73	45.44/5.87	42.53/6.97	42.69/6.37
	LORSAL	49.31/8.79	51.68/8.71	51.68/8.98	52.55/10.36	57.38/9.69	51.67/7.53
DCAU	NB	53.98/6.22	52.97/6.73	53.72/6.03	59.61/5.70	61.97/6.52	61.95/5.52
	SVM	43.40/13.46	41.11/13.93	43.91/14.39	55.47/13.83	59.57/12.67	63.48/9.93
	LR_L1	45.94/4.24	46.26/4.11	45.82/4.41	41.38/7.32	39.19/8.29	37.41/8.21
	LR_L2	46.03/4.23	46.45/4.11	45.74/4.38	41.33/7.24	39.21/8.35	37.45/8.22
	LORSAL	57.01/3.38	54.68/3.59	55.20/3.55	59.38/5.61	63.54/6.21	69.89/4.89

. It should be noted that ‘Total’ in Table 1 denotes the features by concatenating all different features from all frequency bands. Given the same features extracted from the EEG signals, the accuracy metrics of the black bold font in Table 1 indicate the highest accuracies obtained by different classifiers for each frequency band, while the precision metrics with gray background denote the highest precisions obtained by compared methods for all kinds of frequency bands. The LORSAL methods obtained the highest accuracy of 77.17% for the DE feature from the total frequency bands among all the compared classifiers. Under the same case, the highest classification accuracy obtained by SVM is 69.55%, while the best accuracy by NB is 62.36% for the DASM feature from the total frequency bands.

The SVM classifier is the most widely applied approach based on spectral features, especially PSD. Table 1 shows that the performance of SVM is second only to LORSAL on all features extracted from total frequency bands, and the corresponding accuracies are 69.04%, 69.55%, 64.48%, 48.17%, and 63.48% for the PSD, DE, DASM, RASM, and DCAU features. In study [27], the NB classifier obtained an accuracy of 57.6% in the valence dimension. In this study, the mean precisions obtained by NB are approximately between 60% and 62% for the PSD, DE, DASM, and DCAU features from the total frequency bands.

However, the best accuracies obtained by LR_L1 and LR_L2 are approximately 46%, which is significantly lower than those obtained by NB, SVM, and LORSAL. Although the LR_L1 and LR_L2 adopted L1-regularization and L2-regularization during the optimization of LR regressors to avoid over-specification, the assumption of linear separability does not hold for the extracted features from EEG signals. The average precisions obtained by the LORSAL have significant improvement over these by LR_L1 after incorporating the Gaussian kernel. The Gaussian kernel can enhance the data separability in the transformed space and meanwhile, the Laplacian prior can promote sparsity on the learned LR regressor and avoid over-specification of the selected training EEG epochs.

For the classification task of LV/HV emotions, the LORSAL methods present the best classification accuracies, 77.17%, 71.63%, and 69.89% on the DE, DASM, and DCAU features from total frequency bands, which are higher than SVM by about 8%, 7%, and 6%. The SVM and NB classifiers perform best by accuracies 69.04% and 55.65% on the PSD and RASM features from ‘Total’ bands, respectively. As is shown in

Table 2. The mean precisions and standard deviations (%) of the classification of LA/HA emotions obtained by the compared classifiers for different features extracted from different frequency bands.

Feature	Classifier	Delta	Theta	Alpha	Beta	Gamma	Total
PSD	NB	54.22/5.37	53.08/5.66	53.82/5.30	55.39/8.03	58.44/8.94	57.54/6.15
	SVM	50.83/15.93	48.07/15.6	49.95/15.65	58.05/15.31	58.82/15.08	68.60/8.07
	LR_L1	47.74/6.07	47.96/5.66	47.46/6.49	45.34/9.97	44.11/11.67	43.55/14.04
	LR_L2	47.66/6.31	47.94/5.78	47.42/6.41	45.47/9.96	44.03/11.64	43.58/14.23
	LORSAL	57.45/6.75	56.91/6.86	58.82/6.23	63.99/6.47	67.75/7.05	61.62/6.64
DE	NB	54.13/3.76	53.43/3.68	54.08/3.76	56.97/4.81	58.75/5.33	58.46/5.00
	SVM	46.90/14.53	45.03/14.93	47.51/14.57	55.88/13.34	63.33/12.79	69.92/7.94
	LR_L1	47.29/6.52	47.69/5.93	46.96/6.81	45.26/10.88	44.16/12.99	43.10/15.21
	LR_L2	47.38/6.40	47.81/5.89	46.91/6.88	45.29/11.03	44.12/13.03	43.25/15.00
	LORSAL	61.97/4.64	58.18/3.94	59.35/3.97	66.57/6.67	72.73/7.62	77.03/6.20
DASM	NB	54.67/6.10	54.78/6.33	54.31/7.68	58.11/5.75	60.46/5.28	60.34/4.50
	SVM	43.95/13.24	42.84/13.38	43.02/12.53	51.24/14.69	56.42/13.68	61.79/12.34
	LR_L1	48.07/5.75	48.16/4.98	47.91/5.34	45.99/8.41	45.45/10.67	44.60/12.33
	LR_L2	48.13/5.64	48.05/5.18	47.76/5.38	45.93/8.45	45.43/10.72	44.78/12.22
	LORSAL	58.13/3.75	55.14/3.62	55.18/3.37	59.70/4.94	64.54/5.87	71.20/4.96
RASM	NB	51.31/7.57	51.92/7.12	51.01/6.47	54.25/6.52	55.26/6.84	53.57/4.25
	SVM	36.59/13.04	35.31/11.31	36.18/12.49	37.79/14.05	42.61/17.12	43.46/15.61
	LR_L1	49.45/6.07	49.07/4.97	50.18/4.32	49.14/5.75	47.83/7.35	47.94/7.39
	LR_L2	49.44/5.08	49.16/4.41	50.19/4.22	49.05/5.73	47.92/7.21	48.10/7.31
	LORSAL	49.09/10.60	50.93/10.79	50.15/10.95	50.14/11.36	53.51/11.07	50.67/8.33
DCAU	NB	55.09/7.10	53.86/6.91	53.98/7.42	56.98/7.23	60.14/5.99	59.98/4.50
	SVM	42.34/13.47	41.56/13.26	43.19/13.26	50.64/13.76	55.06/14.99	60.04/12.79
	LR_L1	48.19/5.42	48.36/4.90	48.01/5.18	46.34/8.24	45.29/10.05	44.47/11.45
	LR_L2	48.21/5.31	48.38/5.04	47.86/5.14	46.29/8.13	45.33/9.93	44.61/11.51
	LORSAL	57.16/3.69	54.49/3.65	55.18/3.46	58.09/4.99	62.68/5.60	68.48/4.93

, the performance of the five classifiers on classifying LA/HA emotions is similar to the case of LV/HV classification. The introduced LORSAL method performed best on the DE, DASM, and DCAU features from ‘Total’ bands, and the corresponding accuracies outperformed these of SVM by about 7%, 9%, and 8%. In addition, the performance of LORSAL was obviously better than that of the compared LR_L1, and LR_L2 methods in the arousal dimension. The incorporated Gaussian kernel and Laplacian prior improved the distinguishing ability of LORSAL in emotion recognition task based on EEG signals.

For a more comprehensive comparison of the NB, SVM, and LORSAL approaches,

Table 3. The mean metrics and standard deviations (%) of precision, recall, and F1 for the binary classification of LV/HV and LA/HA emotions obtained by the compared classifiers for different features extracted the total frequency bands.

Feature	Classifier	Valence			Arousal
		Precision	Recall	F1	Precision	Recall	F1
PSD	NB	60.51/6.71	56.65/4.99	48.84/9.54	57.54/6.15	54.95/4.29	46.86/8.54
	SVM	69.04/5.91	65.62/6.55	65.24/7.39	68.60/8.07	61.09/6.17	60.41/7.47
	LORSAL	63.29/6.54	62.15/6.65	61.84/7.27	61.62/6.64	59.02/5.53	58.46/6.54
DE	NB	61.04/5.76	60.31/5.19	58.83/5.48	58.46/5.00	58.96/5.05	55.84/5.74
	SVM	69.55/6.58	66.93/6.50	66.89/7.15	69.92/7.94	63.50/6.57	63.45/7.54
	LORSAL	77.17/6.37	76.79/6.21	76.90/6.27	77.03/6.20	76.15/6.14	76.47/6.14
DASM	NB	62.36/5.66	62.18/5.54	61.73/5.57	60.34/4.50	60.79/4.90	59.76/4.77
	SVM	64.48/10.21	63.01/7.15	62.01/9.11	61.79/12.34	59.06/6.36	57.49/8.46
	LORSAL	71.63/4.94	71.38/4.92	71.43/4.92	71.20/4.96	70.68/5.00	70.82/4.94
RASM	NB	55.65/6.77	53.72/4.64	46.01/7.92	53.57/4.25	52.38/2.95	40.70/6.37
	SVM	48.17/17.72	52.47/5.24	43.24/9.56	43.46/15.61	50.25/0.98	40.00/3.72
	LORSAL	51.67/7.53	51.35/2.98	46.69/5.49	50.67/8.33	50.60/2.11	45.32/3.94
DCAU	NB	61.95/5.52	61.65/5.44	61.26/5.48	59.98/4.50	60.14/4.50	59.36/4.37
	SVM	63.48/9.93	62.09/6.51	60.99/8.59	60.04/12.79	58.21/6.32	56.12/8.76
	LORSAL	69.89/4.89	69.68/4.77	69.71/4.81	68.48/4.93	68.11/4.90	68.19/4.88

tabulated the average values and standard deviations of precision, recall, and F1, for the binary emotion classification problem of LV/HV and LA/HA, respectively, when different features were extracted from the total frequency bands. The introduced LORSAL method obtained the best precisions (77.17%/6.37% for LV/HV, and 77.03%/6.20% for LA/HA), the best recalls (76.79%/6.21% for LV/HV, and 76.15%/6.14% for LA/HA), and the best F1 values (76.90%/6.27% for LV/HV, and 76.47%/6.14% for LA/HA), for EEG-based emotion recognition. In summary, the above analysis suggests the application of LORSAL on the DE features extracted from ‘Total’ bands for EEG-based emotion recognition. For simplicity, we will focus on comparing the performance of LORSAL with NB and SVM in the following subsections.

3.2. Investigation of Critical Frequency Bands

In this study, the informative features were extracted from different frequency bands (Delta, Theta, Alpha, Beta, Gamma, and Total) for EEG-based emotion recognition. Thus, we present an investigation of the critical frequency bands in EEG signals for emotion processing. Figure 4a–f and Figure 5a–f show the mean precisions obtained by LORSAL, SVM, and NB for the classification of LV/HV and LA/HA, respectively, when the frequency bands alternated among Delta, Theta, Alpha, Beta, Gamma, and Total. Gamma and Beta are more informative than other frequency bands (Delta, Theta, and Alpha). For example, among the first five frequency bands, the LOSAL obtained the highest accuracies of 72.93% and 67.06% on Gamma and Beta from the DE features in valence classification, the best accuracies of 72.73% and 66.57% on Gamma and Beta from the DE features in arousal recognition.

There is not always a causal relationship between features with high recognition accuracies and emotions. Koelstra et al. [27] presented an investigation about the causal relationship between the emotions and their EEG signals on the DEAP dataset. The average frequency power of trials was calculated over the bands Theta, Alpha, Beta, and Gamma (between 3 and 47 Hz). The Spearman correlated coefficients were tabulated [27] to present the statistical correlation of the power changes of EEG sequences and subject ratings. Similar research has been done by Zheng [47], we focused on analyzing the informative neural patterns associated with recognizing different emotions. Especially, the Fisher ratio was used to investigate the critical frequency bands for discriminating different emotions. The Fisher ratio has been used in pattern recognition for class separability measure and feature selection [65,66,67], as well as in emotion classification [3,4,13,14,27]. The higher values of the Fisher ratio mean more informative neural patterns and features related to emotion recognition. It is defined as the ratio of interclass difference to the intraclass spread

$$F_n(\mathrm{L},\mathrm{H})=\frac{{({\mathrm{m}}_{Ln}-{\mathrm{m}}_{Hn})}^2}{{\sigma }_{Ln}^2+{\sigma }_{Hn}^2},$$

(11)

where L and H mean two different emotion, e.g., LV/HV or LA/HA, and ${\mathrm{m}}_{Ln}$, ${\mathrm{m}}_{Hn}$, ${\sigma }_{Ln}^2$, and ${\sigma }_{Hn}^2$ denote the mean and variance of the n-th dimension of the EEG feature belonging to emotions L and H, respectively. Thus, $F_n(\mathrm{L},\mathrm{H})$ indicates the class separability between emotions L and H for the n-th dimension of the extracted feature.

Given the extracted feature and the specific frequency bands, the mean Fisher ratio $F(\mathrm{L},\mathrm{H})$ is calculated by averaging the values of $F_n(\mathrm{L},\mathrm{H})$ over all the EEG channels (e.g., PSD and DE) or combinations of electrodes (e.g., DASM, RASM, and DCAU)

$$F(\mathrm{L},\mathrm{H})=\frac{1}{N}{\displaystyle \sum _{n=1}^N{F}_n(\mathrm{L},\mathrm{H})},$$

(12)

where N is the number of EEG channels or combinations of electrodes.

Figure 6 shows the Fisher ratio of the extracted PSD, DE, DASM, RASM, and DCAU feature along different frequency bands by averaging the values over all subjects in valence and arousal dimensions, respectively. In addition,

Table 4. Fisher ratio of different frequency bands by averaging the values over all features and subjects in valence and arousal dimensions, respectively.

Emotion	Valence					Arousal
Frequency Bands	Delta	Theta	Alpha	Beta	Gamma	Delta	Theta	Alpha	Beta	Gamma
PSD	0.046	0.005	0.014	0.155	0.414	0.045	0.005	0.013	0.124	0.418
DE	0.047	0.052	0.077	0.242	0.256	0.051	0.048	0.062	0.173	0.197
DASM	0.057	0.063	0.070	0.240	0.317	0.068	0.073	0.067	0.186	0.243
RASM	0.004	0.050	0.037	0.131	0.115	0.004	0.065	0.052	0.095	0.080
DCAU	0.059	0.062	0.064	0.224	0.307	0.062	0.064	0.067	0.195	0.244

illustrates the Fisher ratio over different frequency bands by averaging the values over all features and subjects in valence and arousal dimensions, respectively. The Fisher ratio values shown in Table 4 are calculated by further averaging the values presented in Figure 6 over the five different frequency bands. The following subsection presents comprehensively an analysis including the EEG neural patterns associated with emotions in previous studies [27,47,48], the critical frequency bands and the informative features for emotion recognition. Specific frequency ranges are highly related to certain brain activities. There are neuroscience findings [68,69] revealing that the Alpha bands in EEG signals associate with attentional processing, while the Beta bands associate with emotional and cognitive progress. In the previous studies on the DEAP dataset by Koelstra et al. [27], they reported negative correlations in the Theta, Alpha, and Gamma bands for arousal, and strong correlations in all investigated frequency bands for valence. Similarly, Onton et al. [70] found a positive correlation of valence and the Beta and Gamma bands.

From Figure 6 and Table 4, we found that the Gamma and Beta bands obtained higher values of the Fisher ratio than the other frequency bands. This means that the features extracted over Gamma and Beta bands are more effective for discriminating different emotions, as is in accordance with the classification accuracies by the compared classifiers illustrated in Figure 5. Similarly, Li and Lu [71] showed that the EEG Gamma bands are appropriate for emotion recognition when images are used for emotion elicitation. The studies by [47] Zheng and Lu found specific neural patterns in high-frequency bands for distinguishing negative, neutral, and positive emotions. For negative and neutral emotions, the energy of Beta and Gamma frequency bands decreases, while positive emotions present higher energy of these two frequency bands. Their experimental results [47] on SJTU Emotion EEG Dataset (SEED) showed that the KNN, LR_L2, SVM, and DBN classifiers performed better on Gamma and Beta frequency bands than other bands for the PSD, DE, DASM, RASM, and DCAU features. This showed the informativeness of EEG Gamma and Beta bands for emotion recognition with film clips as stimuli in the SEED dataset. The emotion elicitation materials in the DEAP dataset are one-minute videos [27]. Our experimental results in DEAP and findings were in accordance with these previous studies [47,48]. Additionally, the total frequency bands concatenating all the original five frequency bands can further improve the emotion recognition performance, and the LORSAL obtained the highest mean accuracies in valence and arousal dimensions from the DE, DASM, and DCAU features, as is consistent with the results in studies [47,48].

3.3. Effect of Extracted Features

This subsection presents an analysis of the effects of different features on the average accuracies for emotion recognition based on the EEG signal. When the extracted features alternate among PSD, DE, DASM, RASM, and DCAU, the mean precisions obtained by LORSAL, SVM, and NB for the classification of LV/HV and LA/HA are shown in Figure 7a–f and Figure 8a–f. All the LORSAL, SVM, and NB classifiers performed best for the DE features. Among all classifiers, LORSAL obtained the highest accuracies of 77.17% and 77.03% in valence and arousal dimensions, respectively, for the DE features extracted from total frequency bands. In addition, the highest precisions obtained by SVM are 69.55% and 69.92%, respectively, for DE from total frequency bands. The DE features denote the complexity of continuous random variables [55,56,57]. The EEG signals are characterized by higher low-frequency energy over high-frequency energy, and consequently, DE can distinguish EEG sequences according to low- and high-frequency energy. These results agree with the findings in [47,48], and further show the superiority of the DE features in EEG-based emotion classification.

Moreover, the DASM and DCAU features provide relatively ideal performance compared to the PSD and DE features. DASM and DCAU are asymmetric features and the former findings showed the effectiveness of asymmetrical brain activity along left-right and frontal-posterior directions in emotion analysis. It is noted that the dimensions of the DASM and DCAU features are 70 and 55, respectively, which are fewer than that of PSD and DE with 160-dimension features. This makes DASM and DCAU more competitive in computational complexity. The experimental results were also consistent with the findings by Zheng and Lu [47,48].

4. Discussion

Although the area of affective computing has developed a lot over the past years, the topic of EEG-based emotion recognition is still a challenging problem. This paper introduced LR with Gaussian kernel and Laplacian prior for EEG-based emotion recognition. The Gaussian kernel enhances the EEG data separability in the transformed space, and the Laplacian prior controls the complexity of the learned LR regressor in the training process. The LORSAL algorithm was introduced to optimize the LR with Gaussian kernel and Laplacian prior for its low computational complexity. Various spectral power features in the frequency domain and features by combining the asymmetrical electrodes, PSD, DE, DASM, RASM, and DCAU, were extracted for the Delta, Theta, Alpha, Beta, Gamma and Total frequency bands using 256-point STFT from the segmented 1 s EEG epochs.

The experiments were conducted on the publicly available DEAP dataset, and the performance of the introduced LORSAL methods was compared with the NB, SVM, LR_L1, and LR_L2 classifiers. The experimental results showed that LORSAL presented the best accuracies of 77.17% and 77.03% in valence and arousal dimensions, respectively, on the DE features from the total frequency bands, while the SVM classifiers obtained the second-highest accuracies of 69.55% and 69.92%. The other evaluation metrics obtained by LORSAL, SVM, and NB were also tabulated in the paper. The introduced LORSAL method also presented the best Recall (76.79% and 77.03% in valence and arousal, respectively) and F1 (76.90% and 76.47% in valence and arousal, respectively). The previous experimental results showed the superiority of the introduced LORSAL method for EEG-based emotion recognition compared to the NB, SVM, LR_L1, and LR_L2 approaches.

This paper also showed an investigation of the critical frequency bands for EEG-based emotion recognition. In this study, the informative features are captured from different frequency bands: Delta, Theta, Alpha, Beta, Gamma, and Total. The previous neuroscience studies showed that specific frequency band ranges are associated with specific brain activities. For example, the EEG Alpha frequency bands are related to attentional processing, whereas the Beta bands are a reflection of emotional and cognitive processing. The experimental results showed that the LORSAL, SVM, and NB classifiers performed better on the Gamma and Beta frequency bands than other bands for different features. The comparison of the Fisher ratio also showed the effectiveness of Gamma and Beta bands in emotion recognition. The findings in this study are in accordance with the previous work about critical bands investigation [47,48].

Additionally, the effects of different features, PSD, DE, DASM, RASM, and DCAU, on the emotion classification results were also analyzed in this paper. Experimental results show that the compared approaches, LORSAL, SVM, and NB obtained superior precision metrics on the DE features over other features. This shows the effectiveness of the DE features in distinguishing low- and high-frequency energy in EEG sequences. Meanwhile, the DASM and DCAU features presented relatively ideal classification accuracies compared to the PSD features. It is noted that DASM and DCAU have the advantages of less time consumption for their lower dimensionality than PSD and DE.

For a more comprehensive analysis, -

Table 5. Comparison of the introduced LORSAL methods, the other shallow classifiers, and the deep learning approaches for EEG-based emotion recognition of LV/HV and LA/HA on DEAP dataset.

Classifier			Valence	Arousal	Description
NB		by Koelstra et al. [27]	57.6	62.0	2-class classification for valence and arousal, and within-subject emotion recognition.
Bayesian weighted-log-posterior		by Yoon et al. [72]	70.9	70.1
SVM+mRMR		by Atkinson et al. [73]	73.41	73.06
Segment level decision fusion		by Rozgić et al. [74]	76.9	68.4
CNN+RNN		by Li et al. [75]	72.06	74.12
DNN		by Tripathi et al. [76]	75.78	73.12
CNN			81.41	73.36
LSTM-RNN		by Alhagry et al. [77]	85.65	85.45
3D-CNN		by Salama et al. [78]	87.44	88.49
GELM		by Zheng et al. [48]	69.7		4-class classification in VA space.
SVM	+Raw	by Chen et al. [33]	0.5590	0.7525	2-class classification for valence and arousal, and within-subject emotion recognition, and AUC (Area Under ROC Curve) used for evaluation.
	+Norm		0.5591	0.5590
	+PSD		0.7596	0.5531
	+PSD+Raw		0.9234	0.9462
	+PSD+Norm		0.7460	0.7353
CVCNN	+Raw		0.6221	0.6012
	+Norm		0.6551	0.6176
	+PSD		0.9307	0.88.51
	+PSD+Raw		0.9933	0.9988
	+PSD+Norm		1.00	1.00
GSCNN	+Raw		0.6242	0.5902
	+Norm		0.6394	0.5987
	+PSD		0.8875	0.8802
	+PSD+Raw		0.9933	0.9930
	+PSD+Norm		1.00	1.00
GSLTCNN	+Raw		0.6717	0.6175
	+Norm		0.6350	0.5670
	+PSD		0.8523	0.8390
	+PSD+Raw		0.9946	0.9958
	+PSD+Norm		1.00	1.00
CNN	+Raw	by Chen et al. [34]	57.2	56.3	2-class classification for valence and arousal, and cross-subject emotion recognition
LSTM			63.7	61.9
H-ATT-BGRU			67.9	66.5
NB	+DE	in our study	61.04	58.46	2-class classification for valence and arousal, and within-subject emotion recognition
SVM			69.55	69.92
MLR_L1			33.69	43.10
MLR_L2			33.81	43.25
LORSAL			77.17	77.03

showed a comparison of the introduced LORSAL methods, the other shallow classifiers, and the deep learning approaches for EEG-based emotion recognition of LV/HV and LA/HA on DEAP dataset. In single-trial classification by Koelstra et al. [27], the NB after feature selection using Fisher’s linear discrimination, obtained the accuracies of 57.65% and 62.0% in valence and arousal dimensions. In [72], the Bayesian weighted-log-posterior function optimized with the perceptron convergence algorithm presented average precisions of 70.9% and 70.1% for valence and arousal. For within-subject emotion recognition of LV/HV and LA/HA, Atkinson et al. [73] presented the accuracies of 73.41% and 73.06% using minimum-Redundancy- Maximum-Relevance (mRMR) for feature selection. Rozgić et al. [74] performed classification using segment level decision fusion and presented precisions of 76.9% and 69.4% to discriminate LV/HV and LA/HA emotions. In the studies by Zheng et al. [48], the discriminative graph regularized extreme learning machine (GELM) with DE features achieved the highest average accuracies of 69.67% for 4-class classification in VA emotion space. The introduced LORSAL classifier presented ideal evaluation metrics for EEG emotion recognition, including the compared NB, SVM, LR_L1, and LR_L2 methods in the experiments.

Recently, deep learning (DL) methods have been used for EEG-based emotion classification [28,29]. In [75], a hybrid DL model combining CNN and RNN learned task-related features from grid-like EEG frames and achieved the accuracies of 72.06% and 74.12 for valence and arousal. The DNN and CNN models by Tripathi et al. [76] achieved the precisions of 75.78%, 73.12%, 81.41%, and 73.36% along valence and arousal dimensions, respectively. The classification accuracies for valence and arousal were over 85% using LSTM-RNN by Alhagry et al. [77], and over 87% using 3D-CNN by Salama et al. [78]. More recently, Chen et al. [33,34] have researched a lot on the combination of DL models and various features. As tabulated in Table 5, computer vision CNN (CVCNN), global spatial filter CNN (GSCNN), and global space local time filter (GSLTCNN) [33] presented obvious improvements with concatenating PSD, raw EEG features, and normalized EEG signals. In [34], the proposed hierarchical bidirectional gated recurrent unit (H-ATT-BGRU) network performed better on raw EEG signals than CNN and LSTM, and the obtained accuracies in valence and arousal dimensions were 67.9% and 66.5% for 2-class cross-subject emotion recognition. For more details about the DL architectures applied in the DEAP data, readers may refer to the literature [33,34,75,76,77,78]. Compared to traditional shallow methods, the DL schemes remove the signal pre-processing and feature extraction/selection progress, and are more suitable for affective representation [35,36]. However, the DL methods cannot reveal the relationship between emotional states and EEG signals for being like a black box [37].

However, more importantly, the training of DL networks is extremely time-consuming, which limits their practical applications in real-time emotion recognition [3]. Craik et al. [28] stated that from practical issues, the DL methods have problems of very long computation, and the vanishing/ exploding gradients, and their practical application need extra graphic processing unit (GPU). Roy et al. [29] pointed out that from a practical point-of-view, the hyperparameter search of a DL algorithm often takes up a lot of time for training. Additionally, Craik et al. [28] and Roy et al. [29] make comprehensive reviews on the recent DL schemes.

To illustrate the time efficiency, the average training time of the compared NB, SVM, MLR_L1, MLR_L2, and LORSAL methods are shown in

Table 6. Running time of the compared NB, SVM, MLR_L1, MLR_L2, and LORSAL methods in terms of second and the average time-consumption of feature extraction is 68.15 s.

Classifier	Valence					Arousal
	PSD	DE	DASM	RASM	DCAU	PSD	DE	DASM	RASM	DCAU
NB	4.37	4.36	1.91	1.92	1.51	4.42	4.41	1.93	1.93	3.44
SVM	4.53	4.37	2.34	2.47	2.04	4.26	4.16	2.26	2.34	3.45
MLR_L1	0.14	0.13	0.04	0.04	0.03	0.17	0.15	0.04	0.04	0.08
MLR_L2	0.12	0.11	0.04	0.04	0.03	0.13	0.11	0.04	0.04	0.07
LORSAL	3.75	3.89	3.87	3.69	3.85	3.74	3.86	3.86	3.69	3.87

. The average running time for STFT-based feature extraction is 68.15 s. In our experiment, all the programs are performed using on a computer with an Intel Core i5-4590 of 3.30 GHz and 8.00-GB RAM. LORSAL takes just no more than 4 s for training, and the computing time is in the same order as the compared traditional shallow methods. As mentioned earlier, the complexity of LORSAL is O((L + 1) ²K) for each quadratic problem, where L is the number of EEG epochs used for training and K is the number of emotion classes. As shown in Table 5, the time-consumptions of LORSAL on DE, PSD, DASM, RASM, and DCAU (with different dimensions 160, 160, 70, 70, and 55) are nearly the same. Given limited computational resources, or with portable devices, the introduced LORSAL algorithm has higher time efficiency than DL methods and can present better performance than the compared shallow methods.

5. Conclusions and Future Work

This paper systematically investigates the introduced LORSAL algorithm for the EEG-based emotion class. Additionally, the critical frequency bands, Delta, Theta, Alpha, Beta, Gamma, and the effectiveness of different features, PSD, DE, DASM, RASM, and DCAU, on emotion recognition are also analyzed. The LORSAL classifier performs better than the compared shallow methods and has the superiority of time efficiency compared to the recent DL approaches.

The performance and application of LORSAL-based emotion recognition should be further researched in future work. More informative and representative features can be used in LORSAL. As shown in Table 5, in the research by Chen et al. [33], SVM achieved higher values of AUC (Area Under ROC Curve), 0.9234 and 0.9426, for classifying LV/HV and LA/HA emotions by concatenating PSD and raw pre-processed EEG signals than with other features. We will try to integrate different features to train the LORSAL classifier. The future attempts include the application of LORSAL for 4-class emotion classification in VA space, as the studies in [48]. A further comparison of LORSAL and DL methods, and the combination of their advantages in feature extraction and avoiding overfitting will be investigated. Future work could also include applying LORSAL on multimodal information, e.g., fNIRS, and other physiological signals in brain activity analysis [79,80].