Exploring the Interplay between Facial Expression Recognition and Emotional Intelligence in Adults: Implications for Novel Learning Methods

Abstract

Background: Considering the widespread use of personal protective equipment (PPE) during the COVID-19 pandemic, which conceals facial expressions, this study aims (1) to explore facial expression recognition ability; (2) to assess the ability to read facial expressions under different conditions of face coverage; and (3) to investigate the role of emotional intelligence in facial expression recognition. This multidimensional exploration provides vital insights for creating novel learning methods, emphasizing the pivotal role of emotions. Methods: The study sample comprised 124 Italian adult subjects, consisting of 71 females (57.3%) and 53 males (42.7%), with a mean age of 35.31 years. The present investigation employed the Emotional Intelligence Scale (EIS) and an image-based questionnaire administered via the online platform Google Forms. Student’s t-test and one-way ANOVA were computed with SPSS (v. 29). Results: Significant differences were found based on gender, profession, and level of education in face expression recognition and emotional intelligence. The implications for learning processes are discussed. Conclusion: This study unveils the intricate dynamics of emotion recognition, gender, occupation, level of education’s influence, and emotional intelligence. By incorporating these insights into educational programs, a transformative process can be initiated, nurturing not only empathy, cultural insight, and emotional intelligence but also catalyzing the evolution toward inclusive and emotionally equipped societies.

1. Introduction

Over the past several years, there has been a growing acknowledgment of the pivotal role played by emotional intelligence (EI) and facial emotion recognition across a variety of social contexts [1], encompassing the familial, friendship, occupational, romantic, and learning spheres [2,3,4]. According to Mayer and Salovey (1997), emotional intelligence is a person’s ability to perceive, express, understand, use, and manage emotions in oneself (personal intelligence) and in others (social intelligence), which leads to adaptive behavior [5]. In other words, emotional intelligence is the ability to understand and regulate emotions. On the other hand, face processing is one of the most important functions of the human visual system, and it is essential for normal social functioning [6].

These skills are increasingly recognized as critical elements for effective interpersonal interaction and empathetic comprehension [7].

However, these competencies have specifically encountered a paradigm shift with the emergence of personal protective equipment (PPE) and a reduction in risky hand-to-face habits [8] as prophylactic measures to mitigate the transmission of the SARS-CoV-2 virus responsible for COVID-19. While indispensable for safeguarding public health, surgical masks in particular have introduced a nuanced dimension to the dynamics of social interaction. These facial coverings, while instrumental in curbing viral spread, inadvertently occlude a substantial portion of the face, thereby impinging upon the intricate repertoire of emotional expression conveyed through facial cues [9,10]. This diminished visibility of facial expressions presents a multifaceted challenge, impacting not only individual emotional conveyance but also cross-cultural nuances of interpersonal communication and connection. Furthermore, empirical investigations have demonstrated the discernible impact of mask utilization on the accurate classification of emotional cues and perceptual attributes such as trustworthiness, likability, and perceived proximity [11]. This novel facet of facial concealment has prompted inquiries into circumstances wherein occluded facial features, particularly ocular cues, as exemplified by sunglasses, impact the intricacies of emotion recognition. An important study conducted in South Korea elucidated a hierarchical schema, with the mouth emerging as a dominant vector for the interpretation of emotions such as happiness, sadness, anger, and disgust, while ocular cues take precedence in discerning expressions of fear [12]. Moreover, additional research has revealed that the use of facial masks has the potential to affect the perception and identification of emotions. This suggests a possibility of miscommunication [13]. Additionally, when essential facial cues are obscured by mask usage, the ability to recognize emotions is notably hindered. Among the emotions, sadness and anger appear to be the most prominently affected [14]. Moreover, with the introduction of digital learning modalities, on the one hand, as a response to the health emergency and, on the other hand, as an evolution of education itself, despite it representing an opportunity [15,16], the challenge of being able to distinguish emotional states via facial expressions in virtual settings has gained importance. In fact, some evidence has shown that the role of the body is essential in emotion recognition, even if facial occlusions are present [17]. In an era of profound transformations in the conduit of emotional transmission, a need has emerged to clarify the interaction between emotional intelligence and different conditions of facial covering, either caused by emergencies (surgical masks) or contingency (sunglasses, etc.). In order to fill this gap, this paper aims (1) to explore facial expression recognition ability; (2) to assess the ability to read facial expressions under different conditions of face coverage; and (3) to investigate the role of emotional intelligence in facial expression recognition.

Through this multidimensional exploration, crucial information can be gathered to structure novel learning strategies that take into account the paradigm shift that each of us has been exposed to as a result of the historical events we have experienced and that focus on the crucial role of emotions.

2. Materials and Methods

2.1. Participants

The study sample comprised 124 Italian adult subjects, recruited via non-probabilistic snowball sampling. It consisted of 71 females (57.3%) and 53 males (42.7%). The age of the participants ranged from 19 to 69 years, with a mean age of 35.31 years (SD = 12.87). For the levels of education considered in this investigation, the participants were distributed as follows: 4.8% had a middle school certificate (n= 6), 46% had graduated high school (n = 57), 21% had bachelor’s degrees (n = 26), and 28.2% had master’s degrees (n = 35). Concerning sample professions, 40.3% were “Not Employed” (n = 50), 27.4% were “Employed” (n = 34), 10.5% were “Teachers” (n = 13), 8.9% were “Self-employed” (n = 11), 9.7% were “Healthcare Professionals” (n = 12), and 3.2% were “Armed Forces” (n = 4).

2.2. Procedure

Data were collected online by administering the psychometric tools from 30 March 2022 to 22 July 2022.

Their administration was carried out in a single session using Google Forms, and their completion took about 20 min.

2.3. Psychometric Tools

The following demographic variables were recorded for each participant: gender, age, level of education, and occupation.

The first psychometrically administered instrument was the Self-Report Emotional Intelligence Test, a self-report measure of emotional skills elaborated by Schutte and co-workers [18] and validated in the Italian version by Craparo, Magnano, and Faraci (2014) [19]. It is a unidimensional scale, with higher scores indicating a greater level of EI (with Cronbach’s alpha reaching 0.89). According to Mayer and Salovey (1997) [20], the SREIT considers EI an ability and not a trait and represents each of the following categories: the evaluation and expression of self-emotions and those in others, the self-regulation of emotions and regulation in others, and the use of emotions in problem solving. The SREIT consists of 33 items evaluated with a 5-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree). Examples of items are the following: “I congratulate others when they have done something right”, “It is hard for me to understand why people feel the way they feel”, “I know why my feelings change” [21].

The second instrument employed in this study was a questionnaire comprising a total of 49 images. These images were derived from the Radboud Faces Database (RaFD) [22], which has a high median of 88%, indicating a skewed distribution of agreement values, with more than 76% of images having an agreement ≥ 80%. It portrays two frontal-facing models, one for males and one for females. Each model displays seven emotions through facial expressions (disgust, contempt, happiness, fear, anger, surprise, sadness). On the faces of the selected photos, a surgical mask, a pair of sunglasses, and a niqab (“a religious garment and visible symbol worn by Muslim women” [23] covering the entire face except for the eye area) were digitally added using computer software. This procedure resulted in seven conditions:

• Uncovered male model (for 7 emotions);
• Uncovered female model (for 7 emotions);
• Male model with a surgical mask (for 7 emotions);
• Female model with a surgical mask (for 7 emotions);
• Male model with sunglasses (for 7 emotions);
• Female model with sunglasses (for 7 emotions);
• Female model with a niqab (for 7 emotions).

2.4. Statistical Analysis

Data analysis was conducted using the SPSS software (version 29.0). Student’s t-test and one-way ANOVA were computed. The following independent variables were considered: age, gender, educational attainment, and occupation.

3. Results

Regarding facial expression recognition, out of the total 49 conditions included within the facial expression questionnaire, in

Table 1. (A) Facial expression questionnaire (female faces)—frequency of correct and incorrect answers; (B) facial expression questionnaire (male faces)—frequency of correct and incorrect answers.

(A)
Condition	Score	n	%
10.F.NIQ.DG.	0	117	94.4%
	1	7	5.6%
14.F.OCC.P.	0	93	75.0%
	1	31	25.0%
23.F.MASC.DP.	0	74	59.70%
	1	50	40.3%
47.F.NIQ.DP.	0	81	65.3%
	1	43	34.7%
48.F.MASC.DG.	0	95	76.6%
	1	29	23.4%
(B)
Condition	Score	n	%
2.M.VISO.P.	0	85	68.5%
	1	39	31.5%
9.M.MASC.P.	0	120	96.8%
	1	4	3.2%
13.M.VISO.T.	0	92	74.2%
	1	32	25.8%
18.M.MASC.T	0	95	76.6%
	1	29	23.4%
22.M.MASC.DG.	0	93	75.0%
	1	31	25.0%
30.M.OCC.T.	0	92	74.2%
	1	32	25.8%
46.M.OCC.P.	0	81	65.3%
	1	43	34.7%
49.M.MASC.DP.	0	96	77.4%
	1	28	22.6%

Condition: 10.F.NIQ.DG. (female with niqab expresses disgust); 14.F.OCC.P. (female with sunglasses expresses fear); 23.F.MASC.DP. (female with mask expresses contempt); 47.F.NIQ.DP. (female with niqab expresses contempt); 48.F.MASC.DG (female with mask expresses disgust). Condition: 2.M.VISO.P. (male with uncovered face expresses fear); 9.M.MASC.P. (male with mask expresses fear); 13.M.VISO.T. (male with uncovered face expresses sadness); 18.M.MASC.T (male with mask expresses sadness); 22.M.MASC.DG. (male with mask expresses disgust); 30.M.OCC.T. (male with sunglasses expresses sadness); 46.M.OCC.P. (male with sunglasses expresses fear); 49.M.MASC.DP. (male with mask expresses contempt).

(conditions with female faces) and Table 1 (conditions with male faces), provided below, only the conditions (13 out of 49) with more incorrect responses (score = 0) than correct responses (score = 1) are displayed.

With regard to the gender of the models pictured in the images chosen for the construction of the questionnaire, from the above-obtained scores, it is possible to observe that the incorrect answers were attributed more to the conditions (8 out of 13) in which emotions were expressed by the male model compared with the incorrect answers that were attributed to the conditions (5 out of 13) in which emotions were expressed by the female model. With regard to emotions, on the other hand, it is possible to observe that the highest percentages in the incorrect responses, compared with the correct ones, are present in the cases where the models express disgust (3 out of 13), contempt (3 out of 13), sadness (3 out of 13), and fear (4 out of 13). From the above results, it is also possible to see that, conversely, a higher number of incorrect responses were not found in the images in which the following emotions were expressed: surprise, happiness, and anger. In relation to the conditions, the highest percentages of wrong answers, compared with the correct ones, are present in all conditions: uncovered face (2 out of 13), niqab (2 out of 13), sunglasses (2 out of13), surgical mask (6 out of 13).

Analyzing gender differences,

Table 2. Facial expression questionnaire—differences by gender (n = 124).

Condition	Gender	M	ds	t	p
4.M.VISO.F.	1	0.94	0.232	−2.04	0.045
	2	1.00	0.000
6.M.OCC.R.	1	0.75	0.438	2.57	0.011
	2	0.53	0.504
12.F.VISO.S.	1	0.90	0.300	−2.39	0.018
	2	1.00	0.000
13.M.VISO.T.	1	0.34	0.476	2.39	0.018
	2	0.15	0.361
18.M.MASC.T	1	0.31	0.466	2.35	0.021
	2	0.13	0.342
30.M.OCC.T.	1	0.37	0.485	3.30	0.001
	2	0.11	0.320
33.F.OCC.F.	1	0.93	0.258	−1.99	0.049
	2	1.00	0.000
46.M.OCC.P.	1	0.46	0.502	3.31	0.001
	2	0.19	0.395

Condition: 4.M.VISO.F. (male with uncovered face expresses happiness); 6.M.OCC.R. (male with sunglasses expresses anger); 12.F.VISO.S. (female with uncovered face expresses surprise); 13.M.VISO.T. (male with uncovered face expresses sadness); 18.M.MASC.T. (male with mask expresses sadness); 30.M.OCC.T. (male with sunglasses expresses sadness); 33.F.OCC.F. (female with sunglasses expresses happiness); 46.M.OCC.P. (male with sunglasses expresses fear). Gender: 1 = female; 2 = male.

highlights that participants belonging to the male gender scored significantly higher in recognizing the expression of happiness in male faces with uncovered faces. In addition, they also scored significantly higher on average in the ability to recognize the emotion of happiness in female faces with sunglasses and, finally, the emotion of surprise in female faces with uncovered faces. As for the female gender, on the other hand, participants belonging to this category showed a significantly greater ability to recognize the emotion of sadness in male faces with uncovered faces, sunglasses, and face masks. In addition, significantly higher scores were also found within the same sample in the recognition of the emotions of anger and fear in male faces with sunglasses.

Concerning differences based on profession,

Table 3. Facial expression reading questionnaire—differences by profession (n = 124).

Condition	Profession	M	ds	f	p
6.M.OCC.R.	1	0.76	0.431	2.39	0.042
	2	0.65	0.485
	3	0.54	0.519
	4	0.55	0.522
	5	0.67	0.492
	6	0.00	0.000
9.M.MASC.P.	1	0.00	0.000	5.38	0.000
	2	0.03	0.171
	3	0.00	0.000
	4	0.27	.467
	5	0.00	0.000
	6	0.00	0.000
23.F.MASC.DP.	1	0.52	0.505	2.92	0.016
	2	0.32	0.475
	3	0.23	0.439
	4	0.09	0.302
	5	0.67	0.492
	6	0.25	0.500
25.M.VISO.S.	1	0.96	0.198	2.28	0.051
	2	1.00	0.000
	3	1.00	0.000
	4	1.00	0.000
	5	1.00	0.000
	6	0.75	0.500
28.M.VISO.DP.	1	0.60	0.495	2.71	0.023
	2	0.62	0.493
	3	0.15	0.376
	4	0.55	0.522
	5	0.75	0.452
	6	0.25	0.500
48.F.MASC.DG.	1	0.16	0.370	2.71	0.024
	2	0.18	0.387
	3	0.23	0.439
	4	0.64	0.505
	5	0.33	0.492
	6	0.25	0.500

Condition: 6.M.OCC.R. (male with sunglasses expresses anger); 9.M.MASC.P. (male with mask expresses fear); 23.F.MASC.DP.(female with mask expresses contempt); 25.M.VISO.S. (male with uncovered face expresses surprise); 28.M.VISO.DP. (male with uncovered visage expresses contempt); 48.F.MASC.DG. (female with mask expresses disgust). Profession: 1 = “Not Employed”; 2 = “Employee”; 3 = “Teacher”; 4 = “Self-employed”; 5 = “Healthcare Professionals”; 6 = “Armed Forces”.

highlights that participants belonging to the category of “Self-employed” scored significantly higher in recognizing the expression of fear in male faces with a mask. The same subjects also obtained similarly significant scores in the greater ability to recognize the expression of disgust in female faces with a mask. Subjects in the “Not Employed” category, on the other hand, were significantly better at recognizing the expression of anger in male faces with sunglasses. In addition, participants belonging to the “Healthcare Professions” category were significantly better at recognizing the emotion of contempt expressed by a female face with a face mask and a male face with an uncovered face. In conclusion, subjects from all occupational groups, except for the “Not Employed” and “Armed Forces” categories, were significantly better at recognizing the emotion of surprise in male faces with uncovered faces.

Regarding the scores derived from the responses given on the scale of EI and regarding the differences between the sample groups divided by gender (

Table 4. Emotional intelligence—differences by gender (n = 124).

Condition	Gender	n	M	ds	t	p
EI	1	71	3.69	0.283	1.58	0.118
	2	53	3.60	0.328	1.54	0.126

Gender: 1 = female; 2 = male.

), no significant difference was found regarding EI in the responses between males and females.

Dividing the sample for level of education (

Table 5. Emotional intelligence—differences by level of education (n = 124).

Condition	Level of Education	n	M	ds	f	p
EI	1	6	3.96	0.359	2.76	0.045
	2	57	3.61	0.318
	3	26	3.62	0.227
	4	35	3.69	0.301

Education qualification: 1 = middle school certificate; 2 = high school graduation; 3 = bachelor’s degree; 4 = master’s degree.

), those holding a “middle school certificate” achieved a higher average on the EI scale compared with the other categories of level of education.

Analyzing the scores obtained from the responses to the EI scale, no significant difference by profession was found (

Table 6. Emotional intelligence—differences by profession (n = 124).

Condition	Profession	n	M	ds	f	p
EI	1	50	3.59	0.307	0.01	0.303
	2	34	3.68	0.299
	3	13	3.74	0.310
	4	11	3.73	0.374
	5	12	3.73	0.223
	6	4	3.65	0.250

Profession: 1 = “Not Employed”; 2 = “Employed”; 3 = “Teacher”; 4 = “Self-employed”; 5 = “Healthcare Professionals”; 6 = “Armed Forces”.

).

Additionally, the sample was divided between high and low EI scores (

Table 7. Percentiles of the EI scores.

	5	10	25	50	75	90	95
Weighted average (definition 1)	31,515	32,424	34,318	36,667	38,409	40,152	41,742
Turkey’s Test			34,394	36,667	38,383

). In total, 28 subjects presented a low score in EI, falling in the 5–25 percentile since they obtained an average score ranging from 3.1515 to 3.4318. Indeed, 31 subjects fell in the 75–95 percentile since they obtained an average score ranging from 3.8409 to 4.1742. The rest of the participants (n = 65) who were part of the survey had average EI scores.

Table 8. Facial expression questionnaire—differences by high or low EI percentile (n = 59).

Condition	EI Percentile	n	M	ds	t	p
11.M.MASC.F.	1	28	0.84	0.374	−2.40	0.019
	2	31	1.00	0.000
13.M.VISO.T	1	28	0.39	0.495	2.39	0.020
	2	31	0.13	0.341
14.F.OCC.P.	1	28	0.06	0.250	−2.11	0.039
	2	31	0.26	0.445
35.F.OCC.S.	1	28	0.94	0.250	−2.11	0.039
	2	31	0.97	0.180

Condition: 11.M.MASC.F (male with mask expresses happiness); 13.M.VISO.T. (male with uncovered face expresses sadness); 14.F.OCC.P. (female with sunglasses expresses fear); 35.F.OCC.S. (female with sunglasses expresses surprise). EI percentile: 1 = low score; 2 = high score.

highlights those participants belonging to the sample with low EI who obtained significantly higher scores in recognizing the emotion of sadness in male faces with uncovered faces. As for participants belonging to the sample with high EI, they obtained significantly higher scores in recognizing the emotion of happiness in male faces with face masks. Similarly, they also obtained significantly higher scores regarding the ability to recognize the emotions of fear and surprise in female faces with sunglasses.

4. Discussion

The results of this article show that emotions like disgust, contempt, sadness, and fear consistently elicited higher percentages of incorrect responses compared with correct ones. Conversely, emotions like surprise, happiness, and anger were better recognized. These patterns indicate a differential recognition ability for specific emotions, which could be due to inherent cultural or societal influences [24]. This evidence invites one to think critically about the societal, cultural, and psychological factors that shape our perceptions. Educators can utilize these examples to engage students in discussions about cognitive biases, social conditioning, and the nuances of emotion recognition, fostering a deeper understanding of human behavior.

Furthermore, this study sheds light on several significant findings related to gender, emotions, and facial expression recognition. Among the 49 conditions, 13 conditions were characterized by more incorrect responses than correct ones. Interestingly, the incorrect answers were predominantly attributed to conditions where emotions were expressed by male models (8 out of 13), as opposed to female models (5 out of 13). This observation suggests potential gender differences in emotion expression intensity. Women seem to express emotions with greater intensity than men, as the scientific literature has shown [25,26]. In more detail, the analysis of gender differences in facial expression recognition indicated that males scored significantly higher in recognizing happiness in male faces without obstructions. Similarly, males exhibited superior recognition of happiness in female faces with sunglasses and surprise in female faces without obstructions. Conversely, females outperformed males in recognizing sadness in male faces without obstructions, with sunglasses, and with face masks. Additionally, females excelled at recognizing anger and fear in male faces with sunglasses. These data highlight gender differences in emotional expression and recognition. However, the state of the art presents contradictory data, as while some research confirms gender differences in emotion recognition [27], other evidence does not support this finding [28]. However, it seems essential to promote gender awareness and sensitivity. Educational institutions should integrate the above findings into diversity and gender sensitivity training, encouraging students to reflect on their biases and work toward an equitable understanding of emotions across genders.

Moreover, occupation-related distinctions were also highlighted. However, they appear to be fuzzy data, without a comparison in the scientific literature. It is likely that these results were reached depending on the type of sample in the study. For this reason, while it appears that occupation may play a role in the ability to recognize facial expressions, further studies exploring this type of relationship in detail are needed.

Regarding emotional intelligence (EI), in contrast with the scientific evidence, no significant gender-based differences were observed in EI scores [29]. Similarly, no significant differences were found in EI scores across different professions. However, subjects with low EI scores were notably more adept at recognizing sadness in male faces without obstructions, while high EI scorers excelled at recognizing happiness in male faces with masks, as well as fear and surprise in female faces with sunglasses. These data can be confirmed by other evidence that has shown that having a good knowledge and management of one’s emotional dimension can be relevant to facial emotion recognition [5]; in fact, a higher level of EI implies faster and better emotion recognition [30]. Most recent findings indicate that Emotion Information Processing (EIP) has been introduced as a novel element of EI. This lends support to the concept that individuals with high EI possess superior skills in processing emotions during the initial stages of perceiving emotional information [31]. This study’s findings emphasize the multifaceted nature of facial expression recognition, influenced by gender, emotions, occupation, and emotional intelligence. These insights contribute to our understanding of how different factors shape human emotion recognition abilities and highlight potential areas for intervention or training to enhance accuracy and reduce biases in emotion recognition. Moreover, incorporating the discussed results into the learning process has the potential to not only enrich educational experiences but also empower individuals with valuable skills for understanding and navigating emotions, fostering empathy, and contributing to more inclusive and emotionally intelligent societies.

5. Conclusions

In conclusion, this study unveils the intricate dynamics of emotion recognition, gender influence, and emotional intelligence. The findings highlight the importance of understanding societal and cultural factors that shape emotion interpretation, emphasizing the need for critical thinking and gender sensitivity in education. This study also stresses the diverse nature of emotion recognition across occupations, urging further research in this area. Moreover, the significant role of emotional intelligence in recognizing specific emotions emphasizes its value in enhancing interpersonal understanding. By incorporating these insights into educational programs, a transformative process can unfold, nurturing not only empathy, cultural acumen, and emotional intelligence but also catalyzing the evolution toward societies that are both inclusive and emotionally adept.

6. Limitations

This study presents some limitations. Firstly, the sample size is noteworthy. In addition, the snowball sampling method also poses limitations. Indeed, it would be advisable to prefer stratified sampling for future studies in order to make the results obtained more reliable and more generalizable. Secondly, despite providing valuable insights into the factors affecting emotion recognition, it might not be fully representative of all cultural, gender, occupational, and emotional intelligence variations because of its specific sample and methodology. In addition, using a monofactorial psychometric instrument to assess EI did not allow us to thoroughly analyze the correlations of this construct with facial emotion recognition. For this reason, it will be essential for future research to be able to adopt a multifactorial instrument for the assessment of EI.

Further research and in-depth investigations are needed to address these limitations and provide a more comprehensive understanding of the intricate factors influencing emotion perception and recognition.