Development of a Drawing Set for the Achievement Emotions Adjective List (DS-AEAL): Preliminary Data on a Pictorial Instrument for Children

Abstract

This work investigated the goodness of a Drawing Set for assessing children’s achievement emotions, to be used together with a short form of the Achievement Emotions Adjective List (DS-AEAL). We considered control-value theory as the main theoretical framework. In Study 1, we developed a set of 10 drawings of faces representing enjoyment, pride, hope, relief, relaxation, anxiety, anger, shame, sadness, and boredom, involving 259 adults as raters. In Study 2, we administered a matching task and a labelling task to 89 adults. The results supported the goodness of the correspondence between the DS-AEAL and the verbal labels. In Study 3, we proposed the same tasks to 192 7-year-olds and 10-year-olds. We found age differences, with lower performance for younger children in line with their less-developed abilities in recognition and recall. Overall, recognition and recall were better for primary compared to secondary emotions. Notwithstanding their preliminary nature, our results support the goodness of the DS-AEAL to assess achievement emotions in various learning contexts, together with the corresponding verbal labels. It can satisfy research and educational purposes, primarily in academic contexts such as the school, where reliable, valid, and easy-to-administer methods are essential.

1. Introduction

Emotions felt by children in relation to school learning have a great impact on both their wellbeing and achievement [1,2,3,4]. Their knowledge is the basis for implementing actions to support them, with cascading effects at a variety of levels on students’ lives. Self-report instruments are among the most accurate ways to measure emotions related to learning, given their potential to guarantee access to people’s internal psychological world [5]. However, previous research has disregarded the development of instruments for assessing these emotions, specifically those devised for children, integrating verbal and non-verbal stimuli, and therefore benefitting from the advantages of presenting both kinds of information [6].

To fill this gap in the current literature, our research had the objective of presenting preliminary data on the development of a pictorial instrument for assessing children’s emotions related to learning, i.e., the Drawing Set-Achievement Emotions Adjective List (DS-AEAL). This instrument differs from others in the literature because it uses drawings together with verbal labels, with related advantages [6]; it addresses a specific age group, that is, school-aged children, for whom research about nature and correlates of achievement emotions is still underexplored [7,8,9,10,11,12,13]; it is focused on the measurement of a wide range of emotions related to learning; and it comprises both primary and secondary emotions strictly related to learning environments, extending previous research [14]. Moreover, we used the developed instrument to explore differences in 7-year-olds and 10-year-olds’ abilities to recognise and recall emotions.

1.1. The Control-Value Theory of Achievement Emotions

Emotions focused on learning activities or outcomes are referred to as achievement emotions [1,2,4]. They can be described as resorting to at least two underlying dimensions, i.e., valence—permitting to distinguish between positive and negative emotions—and activation—differentiating between activating and deactivating emotions. Achievement emotions play a crucial role in learning contexts because they are strictly intertwined with a range of motivational, cognitive, and behavioural dimensions and are associated with both students’ wellbeing and achievement [1,2,4]. According to Pekrun’s control-value theory, they are conceptualised in terms of antecedents, including mainly control and value appraisals, and consequences, such as cognitive performance. The impact of achievement emotions on the latter is, in turn, mediated by motivational and self-regulatory processes. While initially documented mainly for university students, these relations have been in part confirmed also with primary and secondary school students [7,8,9,10,11,12]. However, knowledge about a wide range of young children’s achievement emotions and their relation with other constructs is still lacking [3], also due to the paucity of measures specifically developed to assess them during the school years.

1.2. Measurement of Achievement Emotions

Achievement emotions are usually assessed by means of self-report questionnaires. Notwithstanding their limitations, such as desirability biases or memory distortions, self-report instruments are still the primary tools for gathering insights about people’s inner states [5]. Some questionnaires include items describing different emotional components, such as the affective, cognitive, motivational, and physiological components. An example of a largely used questionnaire of this kind is the Achievement Emotions Questionnaire (AEQ) [15]. This 232-item questionnaire permits the assessment of nine achievement emotions, namely three positive activating emotions (i.e., enjoyment, pride, and hope), one positive deactivating emotion (i.e., relief), three negative activating emotions (i.e., anxiety, anger, and shame), and two negative deactivating emotions (i.e., hopelessness and boredom).

However, the AEQ is quite long, and the complexity of its contents makes it difficult to use with children. To assess achievement emotions in this age range, Lichtenfeld and colleagues [7] adapted the AEQ for measuring three achievement emotions (enjoyment, anxiety, and boredom) with a total of 28 items, developing the Achievement Emotions Questionnaire-Elementary School (AEQ-ES). The AEQ-ES also includes pictorial supports depicting faces with five levels of increasing intensity. However, it focuses on a low number of emotions, and the authors did not report data about the process conducted in the development of the drawings of the faces.

Other questionnaires include adjectives instead of items, with the advantage of brevity. For example, the Achievement Emotions Adjective List (AEAL) is formed by a list of 30 adjectives, with three adjectives for each of the 10 achievement emotions [12]. The latter comprises at least two emotions for each of the four quadrants resulting from the combination between valence and activation—i.e., positive activating emotions: enjoyment, pride, and hope; positive deactivating emotions: relief and relaxation; negative activating emotions: anxiety, anger, and shame; and negative deactivating emotions: sadness or hopelessness and boredom. The AEAL was developed with a sample of higher secondary school students. However, there is also a shorter version with only one adjective for each emotion, which was used with children attending primary school [10,13].

It is worth reporting that a recently Revised version of the AEQ (AEQ-R) permits the measurement of a higher number of achievement emotions pertaining to the four quadrants compared to the original version. Nevertheless, the AEQ-R still maintains quite a large number of items, i.e., 79, and a linguistic formulation understandable to adults or, at the most, to adolescents [4].

1.3. Children’s Ability to Recognise and Recall Emotions

When evaluating children’s emotions, instruments accompanying verbal labels with graphical devices have several advantages, for example, favouring more direct access to the semantic network in which the emotional information is stored [6]. This is of key relevance during development, particularly for young children but also for impaired ones, such as nonverbal or traumatised ones, whose ability to interpret verbal messages could be less accurate than the ability to interpret facial expressions [16].

Such instruments can be used relatively early, given the precocious development of facial recognition abilities, serving the adaptive functions of detecting individuals’ emotions that, in turn, favour the appropriate responses to the social context [17]. At the age of two or three, children recognise the facial expressions of primary or basic emotions (e.g., enjoyment, fear, anger, sadness). The latter form a limited set of emotions that are typically manifested and recognised universally across cultures, are discrete, have a pattern of neural and bodily expressed components, and have a feeling or motivational component [18]. While different theorists still discuss the number of primary emotions or families of primary emotions, there is a large consensus about including among them at least four emotions, i.e., enjoyment, fear, anger, and sadness [19,20,21,22]. During preschool age, children acquire the ability to represent knowledge about expressions of primary emotions through verbal labels, permitting preschoolers to recognise them [23,24,25]. They also develop the same abilities about secondary or complex emotions (or families of secondary or complex emotions), i.e., those emotions that require social experiences for their construction (such as pride or shame), a little later [23,24,25]. At the same time, children’s ability to read cues conveyed by faces, which enables perception and recognition of facial expressions, is markedly less accurate than adults’, and, at least for some emotions, it develops with different time trajectories during late childhood and adolescence [26,27]. Specifically, 7-year-olds and 10-year-olds’ accuracy in recognising facial expressions of primary emotions is lower than adults’; however, while 7-year-olds and 10-year-olds do not differ in accuracy, they differ in their speed, with older children faster than younger [26]. The progress in recognition abilities could be linked to improved efficiency in both face-specific and overall processing capacities [26].

As regards the ability to recall emotions, the literature on the development of the use of a psychological (and specifically an emotional) lexicon indicates that children are able to produce verbal labels about primary emotions already at 2–3 years [23,24,25,28,29]. During the preschool and school years, they gradually refine the ability to describe verbally all the primary emotions and also the secondary emotions, acquiring, for example, the ability to describe mixed emotions (i.e., emotions of opposite valence that coexist in the same person in the same situation) only at the end of the primary school years [23,24,25,28,29]. This further suggests that children’s abilities to use verbal labels to describe emotions still mature during the school years.

Overall, recognition abilities usually develop earlier compared to recall abilities, mirroring the distinction between comprehension and production abilities. We know, for example, that between 2 and 3 years, the passive vocabulary of a child is larger than their active vocabulary [30]. Similarly, the ability to identify primary emotions by categorising facial expressions develops earlier compared to the ability to label them [31,32].

1.4. Graphic Scales to Assess Children’s Emotions

In the literature, there are several sets of stimuli representing facial expressions of emotions; however, they are rarely specifically devoted to children. We anticipate that in this paper we did not have the objective of conducting an exhaustive research review about this topic, and therefore, as follows, we referred only to some examples of scales. They mainly focus on primary emotions and are composed of graphical stimuli like photographs and computer-generated faces, or—less frequently—drawings (e.g., [6,16,26,33,34,35,36,37,38,39,40]). Among the sets developed specifically for children, there is, for example, the Picture-Set of Young Children’s Affective Facial Expressions (PSYCAFE) [36], a picture-set of children’s faces comprising portraits of males and females from 4 to 6 years elaborated digitally from real pictures, referring to six primary emotions and one neutral face. Also, the Dartmouth Database of Children’s Faces (DDCF) [33], the Child Affective Facial Expression (CAFE) [38], the EU-Emotion Stimulus Set [40], and the Child Emotion Facial Expression Set (ChildEFES) [39], assessing a variety of primary emotions, were specifically devised for children, but they include photographs of real persons (also children). When developing a graphical scale for assessing emotions, it is worth considering that drawings are more flexible than photographs, as, for example, the same version of a drawing can be used with children of different ages, avoiding the proliferation of versions adapted to participants’ characteristics, which may become a confounding variable. Moreover, drawings are particularly familiar to children and are characterised by higher ecological validity than schematic or computerised faces [35].

There are also scales that use pictograms to assess affective states in children, a method that cannot be considered new [41]. For example, the Self-Reported Nonverbal Emotion-Measurement Instrument for Children (SNEMIC) [42] is formed by a set of cartoon puppets that represent both facial and bodily expressions of primary emotions and are presented together with a 4-point response scale. From the literature about the development of emotion knowledge, it is possible to identify more than one set of drawings, including faces of primary emotions, that were used to study preschool and school-aged children’s emotional recognition. They include, for example, the Emotion Matching Task [43], the Affect Knowledge Test [44], the Test of Emotion Comprehension [45], or developed ad hoc stimuli [46]. Nevertheless, most of them comprised only drawings of primary emotions.

More recent research has focused on the development of pictorial scales formed by emojis, both with adults [47] and children [48]. However, while assessing emojis’ valence and arousal has proved feasible—some studies also ordered emojis by quantifying their positive or negative valence [49]—research still needs to be conducted to develop scales including discrete emotions represented through emojis.

Focusing on pictorial scales devised to assess achievement emotions, we find that scarce attention has been paid to developing sets of stimuli (in terms of drawings or other graphical materials) including a wide range of emotions, which could be used to evaluate children’s emotional reactions related to learning contexts, their antecedents, or their consequences. Lichtenfeld and colleagues’ [7] AEQ-ES included five faces with increasing intensity for enjoyment, anxiety, and boredom but did not report the process through which they developed the sets. Another of the rare studies in which achievement emotions were assessed by using drawings demonstrated that cues from the context can convey an emotional meaning that can be misleading in the process of detecting the intended emotion [50]. Even if this study involved adult participants, its results suggest introducing contextual elements in pictorial stimuli to assess emotions only when they are strictly necessary. In another study, Raccanello and Bianchetti [14] tested a preliminary version of an instrument including 10 achievement emotions represented through drawings of faces. Both adults and children (i.e., 7-year-olds and 10-year-olds) were administered three tasks with increasing cognitive complexity: a magnitude task (n = 46), a matching task (n = 47), and a labelling task (n = 53). Most of the participants gave adequate answers for all the emotions in the first two tasks. However, in the labelling task, there were some difficulties with some secondary emotions, specifically pride, hope, relief, and boredom.

1.5. The Present Research

To fill in gaps in the current literature, the aim of this work was to develop a set of drawings representing a large number of achievement emotions to be used with children together with the corresponding verbal labels, benefitting from the advantages of presenting both non-verbal and verbal information [47,51]. In addition, we explored differences in children’s abilities to recognise and recall the emotions represented in the set, involving a sample of 7-year-olds and 10-year-olds.

Based on Raccanello and Bianchetti’s proposal [14], we developed a pictorial instrument for assessing 10 achievement emotions. The set of drawings, called DS-AEAL, was intended to be used together with the AEAL [12] in its short form [10,13], with each of the 10 drawings of faces corresponding to each of the achievement emotions measured through verbal labels by the AEAL, namely enjoyment, pride, hope, relief, relaxation, anxiety, anger, shame, sadness, and boredom. While the advantages of using graphical rather than verbal stimuli have already been documented—e.g., reliance on intuitive attribution rather than verbalisation [47]—it is worth noting that combining graphical and verbal stimuli could integrate the benefits of both modalities, and it has already proved useful, for example, with children with specific problems such as dyslexia [51]. Moreover, this combination could be particularly appropriate for assessing secondary emotions, for which there is no universal association between non-verbal indicators conveyed through facial expressions and the corresponding emotions.

In Study 1, we developed a set of 10 drawings with the aid of a professional illustrator. The faces corresponding to the families of primary emotions (i.e., enjoyment and relaxation for the family of enjoyment; anxiety for the family of fear; anger for the family of anger; sadness for the family of sadness) were drawn on the basis of Ekman’s indications about facial expressions of emotions [17]. For the faces corresponding to the families of secondary emotions (i.e., pride, hope, relief, shame, and boredom), we involved a sample of adults to identify the most adequate drawings among four different options. We also note that, with the exception of shame, the latter emotions were particularly difficult to label in previous studies with similar methodologies (e.g., [14]). In Study 2, we explored the goodness of the DS-AEAL through two tasks (i.e., a matching task and a labelling task) involving a sample of adults. In Study 3, we administered the same tasks to a sample of 7-year-olds and 10-year-olds to investigate age differences in their abilities to recognise and label emotions. In both Study 2 and Study 3, we paid particular attention to possible differences between primary and secondary emotions. In Study 3, we focused on 7-year-olds and 10-year-olds because the literature about the development of the abilities to understand emotions indicated a certain number of differences between the abilities of these two age groups, in particular concerning secondary emotions [23,24,25]. In order to detect differences in these abilities, we decided to involve students at the beginning and end of primary school. In the Italian context, primary school includes five years, and we involved students attending the second and fourth grades. Even if it could have been interesting to consider as participants first-graders instead of second-graders, we chose not to do that because we needed that the participants had some basic capacities about reading and writing, which are usually not mastered yet by first-graders.

2. Study 1

2.1. Aims

The main aim was to develop a set of 10 drawings to be used together with the AEAL in its short form, i.e., the DS-AEAL. For primary emotions, we asked a professional illustrator to draw the corresponding faces, with one version for each emotion. For secondary emotions, we aimed to select the best drawing among four alternative options involving a sample of university students as adult raters.

2.2. Materials and Methods

2.2.1. Participants

The convenience sample included 259 adults (M = 20:11, range: 18:9–47:8; 93% female) attending university courses at the University of Verona in northern Italy. They were characterised by a range of socio-economic statuses. All the participants gave their written consent. The whole project followed the American Psychological Association (APA) ethical guidelines and was approved by the Ethical Committee of the Department of Human Sciences, University of Verona (protocol number 8209).

2.2.2. Instruments and Procedure

For primary emotions, we asked a professional illustrator to draw five drawings of faces, each corresponding to enjoyment, relaxation, anxiety, anger, and sadness, on the basis of Ekman’s indications [17] and the corresponding drawings in Raccanello and Bianchetti’s work [14]. To favour participants’ responses, there was both a male and a female version of each drawing.

For secondary emotions, we asked her to draw four alternative drawings for pride, hope, relief, shame, and boredom, always with a male and a female version (see

Table 1. Female version of the face options for the five secondary emotions.

Emotion	Option A	Option B	Option C	Option D
Pride
Hope
Relief
Shame
Boredom

for the female version). For hope, relief, and boredom, the face was integrated by inserting a corresponding gesture. We administered to the participants a selection task developed ad hoc, presenting them with the four alternatives. For each emotion, they had to rate the four options according to how much each face corresponded to the given emotion (i.e., “Observe the following four faces carefully: Order them from the one that corresponds more (1) to the one that corresponds less (4) to pride/hope/relief/shame/boredom”), using a 4-point scale (1 = “maximum correspondence” and 4 = “minimum correspondence”). We recoded the responses about each option to identify the one that was selected more frequently as the first choice (0 = “not selected as the first choice” and 1 = “selected as the first choice”).

2.2.3. Data Analysis

For all the analyses (in all three studies), we used the R software, version 4.4.0 [52]. We ran five generalised linear mixed models (GLMMs), with the face option (four levels) as the categorical fixed within-subject effect, the participants as the random effect, and the proportion of faces selected as the first choice as the dependent variable. We used the binomial family and the logit link function. We carried out post hoc Bonferroni tests, which give the possibility of lowering the level of significance taking into account the number of comparisons, reducing the instance of a false positive (error I type). We calculated the odds ratio (OR) as the effect size: OR lower than 1.68, 3.47, and 6.71 are equivalent, respectively, to Cohen’s d lower than 0.2 (small effect), 0.5 (medium effect), and 0.8 (large effect) [53].

2.3. Results

We reported in

Table 2. Means (M), standard errors (SE), 95% confidence intervals (CI), key Bonferroni post hoc tests (with z-values and levels of significance, p), and odds ratio (OR) for the face options for the five secondary emotions.

Emotion	Descriptive Statistics	Option A	Option B	Option C	Option D	Bonferroni Post Hoc Tests	OR
Pride	M	0.50	0.10	0.31	0.08	A vs. B: z = 9.13, p < 0.001	9.32
	SE	0.03	0.02	0.03	0.02	A vs. C: z = 4.27, p = 0.001	2.19
	95% CI	[0.44, 0.56]	[0.07, 0.14]	[0.26, 0.37]	[0.06, 0.13]	A vs. D: z = 9.30, p < 0.001	10.73
Hope	M	0.24	0.20	0.19	0.38	D vs. A: z = 3.31, p = 0.006	0.53
	SE	0.03	0.03	0.02	0.03	D vs. B: z = 4.31, p < 0.001	0.42
	95% CI	[0.19, 0.30]	[0.16, 0.25]	[0.14, 0.24]	[0.32, 0.44]	D vs. C: z = 4.72, p < 0.001	0.38
Relief	M	0.09	0.86	0.03	0.02	B vs. A: z = 14.68, p < 0.001	0.02
	SE	0.02	0.02	0.01	0.01	B vs. C: z = 13.12, p < 0.001	194.35
	95% CI	[0.06, 0.13]	[0.81, 0.90]	[0.02, 0.06]	[0.01, 0.04]	B vs. D: z = 11.84, p < 0.001	314.68
Shame	M	0.13	0.06	0.65	0.17	C vs. A: z = 11.16, p < 0.001	0.08
	SE	0.02	0.01	0.03	0.02	C vs. B: z = 11.49, p < 0.001	0.03
	95% CI	[0.09, 0.17]	[0.04, 0.09]	[0.59, 0.70]	[0.13, 0.22]	C vs. D: z = 10.52, p < 0.001	9.27
Boredom	M	0.15	0.54	0.24	0.07	B vs. A: z = 8.92, p < 0.001	0.15
	SE	0.02	0.03	0.03	0.02	B vs. C: z = 6.96, p < 0.001	3.80
	95% CI	[0.11, 0.20]	[0.48, 0.60]	[0.19, 0.30]	[0.04, 0.10]	B vs. D: z = 10.11, p < 0.001	17.01

the descriptive statistics about the proportions of selected options for each emotion.

For each GLMM, there was a significant effect of the face option: pride, χ²(3, n = 258) = 136.60, p < 0.001; hope, χ²(3, n = 259) = 29.80, p < 0.001; relief, χ²(3, n = 259) = 361.77, p < 0.001; shame, χ²(3, n = 259) = 233.64, p < 0.001; and boredom, χ²(3, n = 259) = 149.53, p < 0.001. The post hoc tests indicated that there was always one drawing that was selected significantly more frequently than the other three (see Table 2 for key Bonferroni post hoc tests and OR).

2.4. Discussion

We selected, respectively for pride, hope, relief, shame, and boredom, the options A, D, B, C, and B. For each of the five secondary emotions, we included the selected drawings in the DS-AEAL, together with the drawings depicting the primary emotions. We reported the male version of the DS-AEAL in Figure 1 and the female version in Figure 2. Overall, this preliminary study permitted us to obtain the set of drawings for the 10 achievement emotions of the DS-AEAL, including both the drawings of faces and the corresponding labels.

3. Study 2

3.1. Aims

The aim was to explore whether adult participants perceived the 10 drawings of the DS-AEAL that resulted from Study 1 as corresponding to the hypothesised emotion labels. We used a matching task and a labelling task. For the matching task, we expected that most of the participants adequately recognised each drawing as associated with the hypothesised verbal label, corresponding to one of the 10 achievement emotions already anticipated in Study 1 (Hypothesis 1). For the labelling task, we expected that most of the participants produced an appropriate emotional term for each of the 10 drawings (Hypothesis 2).

3.2. Materials and Methods

3.2.1. Participants

The convenience sample included 89 adults attending the University of Verona in northern Italy. We involved 44 students (M = 22:2, range: 18:00–49:8; 66% female) for the matching task and 45 students (M = 23:2, range: 18:11–44:2; 67% female) for the labelling task. They were characterised by a range of socio-economic statuses. All the participants gave their written consent.

3.2.2. Instruments and Procedure

In the matching task, the participants had to match the 10 drawings of the DS-AEAL with the corresponding verbal labels (i.e., “Draw a line between each face and the emotion that you think represents it”), namely enjoyment, pride, hope, relief, relaxation, anxiety, anger, shame, sadness, and boredom. The drawings were divided into two sheets according to the valence of the emotions, i.e., in one sheet, we included the five positive emotions, and in a second sheet, the five negative emotions. This was performed because we presented the same task to children (see Study 3), and therefore, we needed to use a task with a relatively low cognitive load (that would have been higher presenting all the drawings together). For each of the 10 emotions, the answer was coded as 0 = “inadequate” in case of a missing or wrong response, or 1 = “adequate” in case of a correct response.

In the labelling task, the participants had to write the name of the emotion represented by each of the 10 drawings of the DS-AEAL (i.e., “Under each drawing, write the name of the emotion expressed by the face”). Each answer was coded as 0 = “inadequate” (when the label was missing or inaccurate) or 1 = “adequate” (when the label was accurate, including synonyms). Two judges coded all the responses for reliability (mean agreement: 95%).

Each task was preceded by a familiarisation phase exemplifying how to respond to the stimuli and the questions. In such a phase, we used some drawings of emotions adapted from previous research as stimuli [14]. In all the tasks, the face order was randomised and kept constant.

3.2.3. Data Analysis

For each task, we conducted 10 binomial tests separately for each emotion. Respectively, we compared the proportion of responses adequately recognising each drawing versus those coded as inadequate (for the matching task) and the proportion of responses adequately describing each drawing versus those coded as inadequate (for the labelling task).

3.3. Results

We reported frequencies, proportions, 95% CI, and level of significance (p) about the binomial tests in

Table 3. Frequencies of adequate responses, proportions, 95% CI, and level of significance for binomial tests (p), by emotion (enjoyment, pride, hope, relief, relaxation, anxiety, anger, shame, boredom, and sadness) and task (matching and labelling).

Emotion	Task	Frequency	Proportion	95% CI	p
Enjoyment	Matching	43	0.98	[0.88, 0.99]	<0.001
	Labelling	45	1.00	[0.92, 1.00]	<0.001
Pride	Matching	43	0.98	[0.88, 0.99]	<0.001
	Labelling	33	0.73	[0.58, 85]	0.002
Hope	Matching	43	0.98	[0.88, 0.99]	<0.001
	Labelling	30	0.67	[0.51, 0.80]	0.036
Relief	Matching	43	0.98	[0.88, 0.99]	<0.001
	Labelling	32	0.71	[0.56, 0.84]	0.007
Relaxation	Matching	43	0.98	[0.88, 0.99]	<0.001
	Labelling	37	0.82	[0.68, 0.92]	<0.001
Anxiety	Matching	42	0.96	[0.85, 0.99]	<0.001
	Labelling	34	0.76	[0.61, 0.87]	<0.001
Anger	Matching	43	0.98	[0.88, 0.99]	<0.001
	Labelling	42	0.93	[0.82, 0.99]	<0.001
Shame	Matching	42	0.96	[0.85, 0.99]	<0.001
	Labelling	39	0.87	[0.73, 0.95]	<0.001
Sadness	Matching	43	0.98	[0.88, 0.99]	<0.001
	Labelling	44	0.98	[0.88, 0.99]	<0.001
Boredom	Matching	43	0.98	[0.88, 0.99]	<0.001
	Labelling	39	0.87	[0.73, 0.95]	<0.001

, separately for each emotion and for the matching and labelling tasks. For the matching task, all the binomial tests were significant with p < 0.001, demonstrating that most of the participants matched the drawings to the hypothesised verbal labels. For the labelling task, the binomial tests were significant, with p < 0.05 for hope, p < 0.01 for pride and relief, and p < 0.001 for enjoyment, relaxation, anxiety, anger, shame, sadness, and boredom. Therefore, most of the raters (and for enjoyment, all of them) produced the emotional word corresponding to the label that had been hypothesised by the experimenters.

3.4. Discussion

Overall, our findings indicated that the participants adequately recognised and named all the drawings of the DS-AEAL, confirming both Hypotheses 1 and 2.

Given the advantages of having the possibility to present different images for the same emotion (see, for example, the dataset with photos developed by Negrão and colleagues [39], consisting of almost 1000 stimuli for assessing seven primary emotions), we report in the Supplementary Materials (SM) the instructions and drawings of the DS-AEAL, using both the versions of the drawings tested in this study and other two versions, separately for males and females. These other versions have the same graphical characteristics as the tested versions and differ from them only for the hair. We included both the English (three male versions: Figures S1–S3; three female versions: Figures S4–S6) and the Italian materials (three male versions: Figures S7–S9; three female versions: Figures S10–S12). Therefore, overall, it is possible to use a dataset with six drawings for each achievement emotion, three for males and three for females.

4. Study 3

4.1. Aims

The aim was to explore 7-year-olds and 10-year-olds’ abilities to recognise and label facial expressions corresponding to 10 achievement emotions using the AEAL-DS developed through Studies 1 and 2. We used the same two tasks described in Study 2: a matching and a labelling task. For both tasks, we expected a better performance at increasing age (Hypothesis 3), reflecting older children’s more refined abilities. Moreover, we investigated possible differences between the two tasks according to emotion type. We hypothesised that primary emotions would be recognised and recalled better than secondary emotions (Hypothesis 4).

4.2. Materials and Methods

4.2.1. Participants

We involved a convenience sample of 192 children, i.e., 87 7-year-olds attending the second grade and 105 10-year-olds attending the fifth grade of a primary school in northern Italy. For the matching task, there were 41 7-year-olds (M = 7:4, range: 6:11–7:11; 51% female) and 60 10-year-olds (M = 10:5, range: 9:11–11:3; 35% female). For the labelling task, there were 46 7-year-olds (M = 7:5, range: 7:0–7:11; 72% female) and 45 10-year-olds (M = 10:4, range: 9:8–10:9; 49% female). They came from families with a range of socio-economic statuses. We obtained written parental consent for their participation.

4.2.2. Instruments and Procedure

We contacted the principal of a primary school to ask for availability for study participation. Then the principal asked the school’s teachers. After obtaining their availability, the students whose parents or legal guardians signed the informed consent form participated in the research.

We used the same two tasks from Study 2. The tasks were administered in class during regular school time. The experimenter read aloud all the instructions in order to help students comprehend the tasks and to diminish the rate of missing answers. In the end, all the children received a diploma as a gift for their participation.

4.2.3. Data Analysis

We ran two GLMMs, one for the matching task and one for the labelling task. Age (7-year-olds and 10-year-olds) was the fixed between-subject effect, emotion (enjoyment, pride, hope, relief, relaxation, anxiety, anger, shame, sadness, and boredom) was the fixed within-subject effect, the participants were the random effect (nested into age), and the adequacy of the response was the dependent variable. We used the binomial family and the logit link function. We carried out post hoc Bonferroni tests and calculated the OR as the effect size.

4.3. Results

For the matching task, the GLMM indicated a significant effect of age, χ²(1, n = 101) = 6.11, p = 0.014, with higher adequacy, z = 2.72, p = 0.007, OR = 0.25, for 10-year-olds (M = 0.97, SE = 0.01, 95% CI [0.93, 0.98]) compared to 7-year-olds (M = 0.88, SE = 0.04, 95% CI [0.77, 0.94]). Also, a significant effect of emotion emerged, χ²(9, n = 101) = 45.08, p < 0.001 (see Figure 3 for mean responses and 95% CI). It is worth noting that all the drawings were matched to adequate labels, with proportions ranging from 0.82 to 0.99, with the lowest value for anxiety and the highest one for anger. The post hoc tests indicated that anxiety was matched less adequately than enjoyment, sadness, boredom, and anger; hope less adequately than boredom and anger, as well as relief; and shame less adequately than anger (with values ranging from z = −4.44, p < 0.001, OR = 0.50 to z = −3.25, p = 0.050, OR = 4.28).

For the labelling task, the GLMM revealed a significant effect of age, χ²(1, n = 91) = 9.78, p = 0.002, with higher adequacy, z = 3.33, p < 0.001, OR = 0.38, for older children (M = 0.78, SE = 0.04, 95% CI [0.70, 0.85]) compared to younger children (M = 0.58, SE = 0.05, 95% CI [0.48, 0.67]). There was also a significant effect of emotion: χ²(9, n = 91) = 152.31, p < 0.001 (see Figure 3 for mean proportions and 95% CI). Overall, shame was the emotion with the lowest mean proportion of adequate responses (15%), while sadness was the one with the highest value (98%). Examining the post hoc tests, we found that shame, pride, anxiety, relief, and hope (these emotions had proportions ranging from 0.15 to 0.39, and they did not significantly differ among them) had significantly lower values (ranging from z = −7.11, p < 0.001, OR = 204.00 to z = −3.68, p = 0.011, OR = 0.30) than relaxation, boredom, anger, enjoyment, and sadness (these emotions had proportions ranging from 0.69 to 0.98, and they did not significantly differ among them). In addition, relaxation had significantly lower values than anger (z = −3.92, p = 0.004, OR = 11.30), enjoyment (z = −4.20, p = 0.001, OR = 16.50), and sadness (z = −4.19, p = 0.001, OR = 23.70). These effects were moderated by a significant two-way interaction between age and emotion, χ²(1, n = 91) = 43.81, p < 0.001. Comparing the two age levels, we found that the post hoc tests were significant for shame, z = 3.72, p = 0.037, OR = 0.05, and for anxiety, z = 3.66, p = 0.048, OR = 0.14, with higher values for older children compared to younger children (Figure 4).

4.4. Discussion

Through both the matching and the labelling tasks, we documented the presence of some developmental differences between 7-year-olds and 10-year-olds, supporting Hypothesis 3. Moreover, our findings indicated, notwithstanding some exceptions, that both 7-year-olds and 10-year-olds were better at recognising and recalling primary emotions than secondary emotions, in line with Hypothesis 4.

5. General Discussion

Our research aimed at offering preliminary data to support the goodness of an instrument for assessing children’s achievement emotions, the DS-AEAL. The set was intended to be used with the AEAL [12] in its short form [10,13]. The main theoretical framework on which we based our studies was Pekrun’s control-value theory of achievement emotions [1,2,4]. The relevance of this work derives from the fact that the developed instrument permits to fill gaps in the current literature by offering materials to be utilised to assess achievement emotions (a) by integrating the advantages of using non-verbal and verbal stimuli together [6]; (b) by extending the still scarce knowledge about their nature and correlates with school-aged children (for exceptions see [7,8,9,10,11,12]); (c) by referring to all the four quadrants resulting from the combination between valence and arousal [1,2,4], also considering the differences between primary and secondary emotions; and (d) by exploring further how children’s ability to recognise and recall emotions develops along the school years.

5.1. Development of the DS-AEAL

In Study 1, we developed the set of drawings that were included in the DS-AEAL. A professional illustrator drew five faces corresponding to four primary emotions (i.e., enjoyment and relaxation for the family of enjoyment; anxiety for the family of fear; anger for the family of anger; and sadness for the family of sadness) based on Ekman’s indications [17]. She then drew four versions of faces for each of the five families of secondary emotions (i.e., pride, hope, relief, shame, and boredom). For four of them, the corresponding graphical representation had also resulted in ambiguity in previous research [14]. The ratings of the four versions from a sample of adults permitted us to select the more adequate drawing for each secondary emotion. Therefore, we obtained a set of 10 drawings of 10 faces (with a male and a female version), forming the DS-AEAL, referring to 10 achievement emotions pertaining to the four quadrants identifiable by combining the dimensions of valence and arousal and including both primary and secondary emotions. This phase permitted us to overcome some limitations of previous instruments about drawings of secondary emotions, based on the judgement of quite a large sample of adults.

The goodness of the correspondence between the set of 10 drawings and the verbal labels was investigated in Study 2. We administered a matching and a labelling task to two samples of adults, whose performance in facial recognition tasks is documented as more accurate than children’s [26,27]. We found that, for all the 10 emotions, most of the raters (and all of them for enjoyment in the labelling task) judged the faces as adequately representing the intended emotion, supporting both Hypotheses 1 and 2. In the SM, we reported the materials and the drawings of the DS-AEAL to make available the set of drawings together with the instructions of the AEAL [12] in their short versions [10,13]. We also reported in the SM the drawings related to two other versions of the set for both males and females, to be used in case it is necessary to present more than one pictorial stimulus within a specific assessment (in English and in Italian). Overall, to our knowledge, this study is the first to develop a set of drawings about a large number of achievement emotions to be used together with verbal labels by gathering data about the goodness of the correspondence between the drawings and the intended labels.

Finally, in Study 3, we documented some developmental differences between 7-year-olds and 10-year-olds. In the matching task, relying on children’s recognition abilities, older children performed better than younger children. Also, in the labelling task, in which children’s performance was related to their ability to use the emotional lexicon, older children outperformed younger children, but this happened in particular for shame and anxiety. Differences in shame could be explained by reasoning about the later appearance of secondary emotions compared to primary emotions during development. As for anxiety, we could speculate that the documented increase in anxiety, especially in learning contexts, during the school years [7,8,9,10,11], could be connected to the fact that mastery of anxiety-related lexicon increases, particularly comparing 7-year-olds and 10-year-olds, and therefore this leads to better performance for older children. However, this speculation should be examined by further research with larger samples, focusing specifically on the school context. Therefore, overall, the results of the matching and labelling tasks supported Hypothesis 3, in line with the literature documenting a refinement of both recognition and recall abilities, in particular about emotions, during the school years [23,24,25,26,28,29]. Concerning the differences between primary and secondary emotions, children’s performance differed between the two tasks. On the whole, enjoyment, anger, sadness, and boredom were among the emotions with higher scores in both tasks. In the matching task, all the emotions had scores higher than 0.82, indicating that children mastered quite well their abilities to recognise emotions from facial expressions. Moreover, it is worth noting that anxiety was the least recognised emotion, differing significantly from enjoyment, sadness, boredom, and anger. In addition, hope and relaxation were significantly less recognised than boredom and anger, and shame less than anger. In the labelling task, children named more adequately the group of emotions including sadness, enjoyment, anger, boredom, and relaxation (reported here in decreasing order, beginning with the one with the highest adequacy score) compared to the remaining emotions (in decreasing order: hope, relief, anxiety, pride, and shame). While the presence of sadness, enjoyment, anger, and relaxation in the first group was in line with Hypothesis 4 and can be explained by resorting to the fact that they are all primary emotions, the result about boredom was somehow unexpected. Nevertheless, it could be interpreted, considering that the introduction of a gesture conveying boredom was probably a good means to externalise the intended emotion. Acknowledging the described exceptions, taken together, these findings indicate that children of the examined age find it easier to recognise and recall primary rather than secondary emotions, supporting Hypothesis 4. Again, these differences could, in part, be in line with the later development of the latter compared to primary emotions [18]. Future research should investigate this issue better, focusing, for example, on the school context and examining differences in achievement emotions during the school years while also taking into account the characteristics of primary and secondary emotions. Finally, it is worth underlying that the findings of Study 3 confirm that the drawings of the DS-AEAL should be used together with the corresponding labels, given that, for children, the sole presentation of the drawings could sometimes lead to suggesting an emotion different from the one intended by the researchers, probably due to the developing recognition and recall abilities of children of the considered age.

5.2. Limitations and Future Directions

Our work suffers from some limitations. First, the samples of raters involved in judging the goodness of the DS-AEAL (Studies 1 and 2) were recruited within a specific cultural context. Further studies should test the goodness of the set by involving raters from different cultural backgrounds, given that especially gestures associated with secondary emotions could have different meanings in different cultures. Second, the sample size for Studies 2 and 3 was relatively reduced. Future research should explore adults and children’s performance at the same tasks, involving a larger number of participants. Third, our data presented preliminary information about the correspondence between the 10 drawings of the DS-AEAL and the intended labels, but they do not permit us to make statements about the reliability and validity of the research findings gathered through the use of the drawings together with the AEAL. Future studies should provide information on both the reliability and validity of using the DS-AEAL to assess students’ achievement emotions in a variety of contexts. Fourth, we explored age differences through a cross-sectional design; future studies should use longitudinal ones. Fifth, for some secondary emotions (i.e., hope, relief, and boredom), the drawings include a gesture beyond the facial expression, and it is worth noting that the meaning of gestures could also be culture-dependent. To avoid the influence of contextual elements different from the sole face [50], future instruments could include gestures for all the emotions and test their comprehension with culturally diverse samples. Sixth, it is worth mentioning that the graphic scales that we cited in the theoretical part of this paper were reported only as examples. We did not aim at conducting a research review on this issue, and we are also aware of the fact that research is developing many graphic instruments in line with the rapid changes in technology and, in particular, artificial intelligence. The graphic scale that we developed could benefit in the future from further improvements, taking into account such recent technological developments as well as the potentialities of artificial intelligence. Seventh, we did not check directly with the participants whether they preferred to work with male or female materials, and this could have biassed some results; future research with the DS-AEAL should consider this issue, for example, asking directly to the involved people if they would like to see drawings of males or females. Eighth, we did not gather data about the validity of the DS-AEAL, for example, using other already-developed instruments for assessing achievement emotions or measuring their correlates, such as control and value appraisals, emotion regulation, or performance in learning tasks. Future research could use the DS-AEAL with primary school students to provide evidence about the external validity of the scale.

5.3. Practical Implications

Concerning its educational implications, this research offers the first version of an instrument, the DS-AEAL, which can help extend theoretical research about achievement emotions with young children. The possibility of using a brief and easy-to-administer instrument to assess children’s achievement emotions, combining the benefits of non-verbal and verbal stimuli [6], is the basis for developing applied research in this field. Through such research, it is possible to devise and implement interventions that, starting with reliable and valid measures of achievement emotions, enable teachers and professionals interested in students’ wellbeing to develop actions to support positive emotions and accept and discard negative ones. While the standards for evidence-based interventions have long been acknowledged [54], there is still room for proposing new interventions regarding young children’s achievement emotions, of which there are relatively few. Among such standards, a key role is played by the use of valid instruments. Many studies focused on the development of instruments utilising verbal labels, but a range of instruments which include the corresponding drawings is not available. Such instruments give the possibility to favour children’s answers due to their immediacy, and so they also constitute a viable way for extending theoretical research about achievement emotions with young children, which is relatively scarce. And such research, in turn, is one of the bases for developing evidence-based interventions. In other terms, all professionals interested in children’s wellbeing at school would benefit from having the possibility to detect students’ affect through valid instruments that rely on multiple channels, such as non-verbal and verbal ones.

In addition, the availability of instruments of this kind permits the extension of knowledge about the nature and correlates of achievement emotions for children with atypical development [16], an issue that, currently, is largely neglected by researchers. However, we did not gather data about the specific abilities of children with different impairments while using the DS-AEAL. We could speculate that in cases of difficulties in verbal language understanding, this instrument could be a resource to access the inner world of those children who maintain a certain level of understanding concerning non-verbal communication. Nevertheless, future research should investigate for which types of difficulties the adoption of this instrument could be useful and for which types it could not be.

6. Conclusions

Overall, we documented the goodness of the present version of the DS-AEAL, confirming the correspondence between the pictorial representations and the hypothesised labels of achievement emotions. In addition, our study shed light on some developmental differences in recognition and recall abilities related to emotional faces for school-age children, also taking into account the distinction between primary and secondary emotions. Notwithstanding possible limitations related to the characteristics of self-reported data, this instrument could be used in a variety of learning contexts to assess children’s 10 different achievement emotions. It could satisfy both research and educational purposes, primarily in academic contexts such as the school, where reliable, valid, and easy-to-administer methods are essential.