Validation of the Scale of Emotional States in the Physical Education Context

Abstract

The purpose of the present study was to validate an instrument of student emotional experiences in the Spanish Physical Education context. The sample of participants consisted of 864 secondary education students from various educational institutions of Spain who ranged in age from 13 to 19 years. To assess the psychometric properties of the Scale of Emotions in Physical Education (SEPE), various types of analyses were conducted. The factor structure of the SEPE was examined through confirmatory factorial analysis in relation to two models. In the first model, it was proposed that the eight first order factors, which represented the eight emotional states, would be correlated amongst each other. In the second model, an eight-factor model with two higher order factors was proposed, with these higher order factors representing distinct sets of positive and negative emotions. The results provide support the presence of an eight-factor second order model which consisted of sets of four positive emotions and four negative emotions. These results provide evidence for the reliability and validity of the SEPE within the Spanish Physical Education context.

1. Background

Considerable advances have been made in recent years in the understanding of psychological and motivational characteristics of students in the Physical Education (PE) context. In this regard, Schutz and Pekrun [1] commented that there has been notable interest displayed by researchers in understanding emotional and affective processes in students. Nonetheless, research to date in this area has been relatively fragmented as there have been theories and corresponding research focused on specific emotions, such as anxiety [2], and studies on the specific effects of certain emotions [3].

Damasio [4] characterized emotion in terms of the collection of psychological and physical responses that have the purpose of reestablishing a state of mental and physical equilibrium in the face of external circumstances or in response to challenging social circumstances. These responses take place spontaneously and automatically and are usually reflected in a series of affective, cognitive, expressive, and motivational components [5,6]. Taking anxiety as an example, the affective component refers to the sense of uneasiness and nervous feelings, whereas the cognitive component refers to worry. In turn, the motivational component involves an evasive tendency, or the lack of desire to confront the circumstances that need to be faced. Finally, the expressive component refers to anxious facial expressions or other somatic responses that reflect an underlying state of unease.

The Individual Zones of Optimal Functioning (IZOF) model developed by Hanin [7] argues that emotion is a component of the psycho-biosocial state conceptualized as a situational, multimodal, and dynamic manifestation of human functioning. This model presents five basic dimensions (form, content, intensity, time, and context) in order to describe the optimal and dysfunctional structure of the individual, as well as the dynamics of emotional experiences that are related to performance. The model provides the functional explanation of the dynamics of the relationships between emotions and performance based on a detailed description of people’s subjective experiences. This in the educational context is especially important because teachers who assist their students in the self-regulation of emotions generally face three problems: (a) the identification of emotional states related to individual performance success versus poor performance; (b) the understanding of the relationships between emotions and performance; and (c) the selection of emotional self-regulation techniques [8].

In general, emotions activate or inhibit certain types of behavior given that people tend to experience common sensations across similar contexts that are associated with positive or negative affect, which can lead to maintenance or dropout from sport [9]. As such, it becomes extremely important to recognize and understand the emotional processes that take place given the evident influence upon academic outcomes. In the Physical Education context, for example, anxiety and boredom have been found to have an inverse relationship with involvement in PE classes [10], and conversely, enjoyment is found to positively correlate with level of involvement in PE [11].

It is essential to have the capacity to effectively measure emotional states given their known influence on the social, emotional, and personal development of individuals in addition to their behavioral implications in various contexts. Pekrun, Goetz, Titz, and Perry [12] developed the Achievement Emotions Questionnaire (AEQ) which assesses emotional valence, activation, and focus at the moment in which the experience takes place. This instrument was designed with reference to three subscales that reflect emotional characteristics in classes, in learning, and in test-taking and consists of a total of 231 items. A direct adaptation of this questionnaire would be problematic given the number of items that it contains. Furthermore, the AEQ was created for the context of the ordinary classroom (e.g., mathematics, literature, science) which differs in its dynamics from Physical Education classes [13]. As such, it is necessary to develop an instrument that is more specific to Physical Education classes.

On the other hand, a shorter instrument to assess emotions in the context of Physical Education is the PANASN (Positive and Negative Affect Schedule; [14]), used in numerous investigations [15,16]. This questionnaire is composed of 20 items distributed in proportion between two factors: positive affection and negative affection. Each of these items refers to a unique emotion; therefore, there is no multidimensional value to value the different emotions expressed by students during EF classes. In addition, the questionnaire confuses emotion with moods [17] and was not designed for application in the field of EF [7].

Various studies have revealed that positive emotional experiences can be accompanied by affective and social benefits and can stimulate personal growth. The authors of the present study reviewed theory and research and sought to identify those emotions that were most likely to be experienced by Physical Education students. The emotions selected included anxiety, which is defined by Eysenck and Fajkowska [18] as a negative emotional state and which is related to worry and nervousness. Based on the State and Trait Anxiety Inventory (STAI), a distinction is made between trait and anxiety state [19]. The trait anxiety is a relatively permanent characteristic of personality, which is reflected in latency to react with the state of anxiety-ness; the state of anxiety is a situational and transitory reaction, characterized specifically by a cognitive state of recurrent worry by the possible failure or bad performance in the task, and by the aversive consequences that it can have on the decrease of one’s self-esteem and own social undervaluation. Generally, this reaction leads to feelings of nervousness, tension, and physiological activation [20]. Previous research indicates that anxiety can result from social and familial influences and negative personal experiences among other factors that can increase the likelihood of the appearance of anxiety and extend to affect the level of student involvement in classes of Physical Education [21,22]. By contrast, calmness is an emotional state in which worry is not present during any given moment in time [23]. To date, however, studies of this emotion have yet to be conducted in the Physical Education environment [24,25]. Some research indicates that physical activity contributes to a state of calmness or tranquility either during or following the activity.

Boredom is also one of the emotions that were included on the instrument. From the psychological paradigm, boredom is often defined as a state of reduced excitement that occurs in a situation of understimulation [26]. More concretely, from an educational constructivist perspective, boredom is defined as a negative emotion that impedes a student’s learning experiences [27]. Thus, the perception of boredom is likely to result in less control of attention, slow processing of information, and consequently, in students’ resistance to assimilation of content, ability, and school [28]. This emotion has been studied extensively in classes of Physical Education [29,30] and is characterized by a lack of interest or enjoyment and limited motivation to do the activity due to students’ perceptions that the level of challenge is low in relation to their ability. A contrasting emotional state is enjoyment, which is reflected in the experience of happiness and enthusiasm. This emotion has been included in various studies [31,32] which generally reveal that when students feel personal control and intrinsic motivation, they are more likely to experience enjoyment in PE classes.

According to the embarrassment theory of Modigliani [33], embarrassment is like a sentiment of fragility because of the belief that one’s own self believes they are less competent than others. This emotion is experienced with some assiduity during PE classes, since failure or comparison with others is usually a trigger for that emotion, although it can be attenuated when one’s behavior is not as deficient as it seems or there is no eye contact. In this sense, a study by Trigueros et al. [34] shows that students who experience embarrassment during EF classes will feel more inhibited in class, negatively affecting their motivation and reducing their participation in class. A contrasting effect is experienced under circumstances in which students feel confidence. The two main conceptual approaches to the study of confidence are the theory of self-efficacy [35] and the confidence model [36]. Self-efficacy is defined as “the belief in one’s own capacities to organize and execute the courses of action required to produce certain achievements” [35]. By extension, confidence is defined as belief in the individual’s ability to succeed [36]. Both theories indicate that information that students use to judge their abilities is a critical factor in developing and maintaining a stable level of confidence [35,37]. Various studies [38,39] have demonstrated that this emotion can have an important role in contributing to class involvement during PE classes and subsequent physical activity involvement beyond the school day.

The emotional states of hopelessness and pride are also included in the questionnaire. These emotions refer to the state of mood and illusion in what is being done. According to the theory of hopelessness [40], this emotion is manifested by the presence of negative expectations before a highly valued exit or by feelings of impotence before the improbability of modifying the reached results, being able to manifest this emotion immediately before or after the event. On the other hand, pride refers to positive expectations of oneself or to self-esteem. However, the few studies in the area of Physical Education that have analyzed these emotions [41,42] suggest that hopelessness is negatively related to motivation and vice versa because it is linked to unfavorable perceptions of competence and autonomy in classes. As such, students experiencing hopelessness feel that they will not be able to meet the challenges that are present.

In accordance with these considerations, the present investigation was intended to assess the psychometric properties of an unpublished instrument called the Scale of Emotional States in Physical Education questionnaire (SEPE) (see Appendix A) to determine whether positive and negative emotional states actually represent two distinct categories of emotion during PE classes. The factor structure of the SEPE was examined through confirmatory factory analysis in relation to two models. In the first model, it was proposed that the eight first order factors, which represented the eight emotional states included in the survey, would be correlated amongst each other. In the second model, an eight-factor model with two higher order factors was proposed, with these higher order factors representing distinct sets of positive and negative emotions. In addition, tests of factorial invariance were conducted in relation to gender and the internal consistency of the individual subscales was examined.

2. Methods

2.1. Participants

The participants in this study were 864 secondary education students (438 males and 436 females) who ranged in age from 13 to 19 years (M = 15.34 y, SD = 1.12 y) and who were in attendance at various educational institutions (public and private) in the south of the province of Almería. In order to assess the temporal stability of the SEPE, a separate independent sample of 281 secondary education students who ranged in age from 15 to 18 years (M = 16.14 y; SD = 0.78 y) was used. These students completed the instrument on two occasions two weeks apart.

2.2. Instruments

The SEPE instrument was examined through confirmatory factor analysis to identify the factor structure of the instrument. The stem for each item on the 40-item questionnaire was: “During PE classes…”, and responses were provided along a 7-point scale that ranged from “1” (“completely disagree”) to “7” (“completely agree”). We opted for a Likert scale of 1 to 7 because on a scale of only 5 points, respondents tend to avoid the 2 extreme options, thus obtaining very little variation. In this way, adding more levels results in more diverse ratings [43].

2.3. Procedure

This study followed all procedural ethics with regard to the ethical treatment of human participants. Permission was requested and approved by the Bioethics Committee in Human Research of the University of Almeria (Ref: UALBIO 2019/014). Furthermore, we obtained written informed consent from the parents/legal guardians of all participants of the study, because they were underage students. Additionally, the parents/legal guardians were previously informed of the objective and the procedure of this study in writing.

In the present study, we made use of the Delphi method (see [44]), which consists of a technique of obtaining information, based on the consultation of experts in an area, in order to obtain the most reliable consensus opinion of the consulted group. These experts are individually submitted to a series of in-depth questionnaires that are interspersed with feedback from the group. In this way, 56 items were initially developed with the intention of developing and including a range of emotional experiences relevant to the Physical Education context. As a first step, four expert researchers with expertise in the study of emotions in PE area were contacted to solicit their participation. These individuals then contributed to the identification of relevant emotions in Physical Education and completed a review of existing instruments [18,23,25,31,34]. Fifty-six relevant questionnaire items were developed from this list, while redundant items were not retained.

To evaluate each of the items from the original pool of items, these four individuals then evaluated each of the items for uniqueness and clarity. Items were eliminated from many of the subscales due to a lack of clarity, redundancy or due to overlap with other items which resulted in a final questionnaire that consisted of 40 items. These 40 items were distributed across eight factors and their corresponding subscales.

2.4. Data Analysis

An assessment of the psychometric properties of the SEPE was conducted through various statistical processes. As a first step, a confirmatory factor analysis (CFA) was conducted to test the structure of the model comprising eight emotional dimensions with two higher order factors representing the four positive and four negative emotions. Subsequently, a multigroup analysis was conducted to assess invariance relative to gender. Finally, all descriptive statistics were examined, and the internal consistency of the subscale was assessed through Cronbach’s alpha statistic. Temporal stability was assessed through intraclass correlations (ICC). Statistical packages SPSS 24.0 and AMOS 21.0 were used to conduct these analyses.

Given the high value for the Mardia coefficient (314.63) across the various CFAs, the maximum likelihood estimation method was used along with the bootstrapping procedure. In this case, estimates were not greatly affected by normality issues, which were considered robust [45]. For the purpose of accepting or rejecting the proposed models, a set of model fit indices were employed, including: χ2/df, Comparative Fit Index (CFI), Tucker Lewis Index (TLI), Incremental Fit Index (IFI), Root Mean Square Error of Approximation (RMSEA), as well as 90% confidence intervals (IC) and Standardized Root Mean Square Residual (SRMR).

Given that χ2 is very sensitive to sample size [46], the value of χ2/df was also considered with values less than 5.0 considered to be acceptable [47]. The incremental fit indices (CFI, TLI, and IFI) revealed a good fit with values equal to or exceeding 0.90 [48], while the RMSEA and SRMR error indices are considered acceptable if their values are less than 0.08 [49,50].

3. Results

3.1. Confirmatory Factor Analysis

The eight-factor structure of the model was examined in relation to the various fit indices and yielded corresponding values of: χ2 (712, N = 864) = 2446.07, p = 0.001; χ2/df = 3.43; CFI = 0.85; TLI = 0.83; IFI = 0.85; and RMSEA = 0.074 (IC 90% = 0.071–0.077); SRMR = 0.086. An examination of these fit indices and the residual standardized covariances indicated the possibility of areas for improvement given that residual values of some items correlated with residual values of other items with standardized residual values exceeding 2.00 [46]. Four items were subsequently eliminated from each of the factors of boredom, hopelessness, calmness, and anxiety. Following the elimination of these four items, there was an improvement to the indices of fit: χ2 (566, N = 864) = 1352.35, p = 0.001; χ2/df = 2.39; CFI = 0.92; TLI = 0.91; IFI = 0.92; RMSEA = 0.056 (IC 90% = 0.052–0.060); and SRMR= 0.067. In the revised model, the standardized residual values of each of the items were below 2.0, and the standardized factor loadings were statistically significant (p < 0.001) and ranged from 0.33 to 0.90. Subsequently, individual items with loadings below 0.50 were eliminated from two factors (Pride and Enjoyment). Excluding these two items resulted in an improvement to the fit indices in the model: χ2 (499, N = 864) = 1018.30, p = 0.001; χ2/df = 2.04; CFI = 0.94; IFI = 0.94; TLI= 0.94; RMSEA = 0.048 (IC 90% = 0.044–0.053); and SRMR = 0.048. This model was finalized as such (Figure 1) as each of the standardized residual values were consequently below 2.0, and standardized factor loadings were statistically significant (p < 0.001) and ranged between 0.57 and 0.90.

Following the revision of the model, a higher-order model was examined in which the four positive emotions and the four negative emotions constituted distinct first-order factors. Fit indices for this model were similar to the previously tested model and acceptable: χ2 (523, N = 864) = 1013.55, p = 0.001, χ2/df = 2.11; CFI = 0.94; TLI = 0.94; IFI = 0.94; RMSEA = 0.050 (IC 90% = 0.046–0.054); and SRMR = 0.065. Each of the standardized factor loadings was significant (p < 0.001), including 0.86 for Shame; 0.87 for Boredom; 0.93 for Hopelessness; and 0.63 for Anxiety for the Negative Emotion constructs. For the Positive Emotion constructs, these weights were 0.69 for Confidence; 0.54 for Calmness; 0.85 for Enjoyment; and 0.67 for Pride. The correlation between the two higher order factors was r = −0.34.

3.2. Analysis of Gender Invariance

A multigroup analysis was conducted to compare the factor structures of the eight-factor model with the two-factor, higher-order model.

Table 1. Multigroup invariance model across gender.

Eight-factor first order model
Models	χ2	df	χ2/df	Δχ2	Δdf	CFI	TLI	IFI	RMSEA (IC 90%)	SRMR
Model 1	1653.95	998	1.66			0.94	0.94	0.94	0.039	0.042
Model 2	1684.27	1024	1.65	30.32	26	0.93	0.93	0.93	0.038	0.041
Model 3	1760.28	1060	1.66	106.33	62 **	0.93	0.92	0.93	0.039	0.042
Model 4	1889.14	1094	1.72	235.19	96 **	0.91	0.92	0.91	0.041	0.045
Two-factor higher order model
Models	χ2	df	χ2/df	Δχ2	Δdf	CFI	TLI	IFI	RMSEA (IC 90%)	SRMR
Model 1	1759.59	1046	1.68			0.92	0.92	0.92	0.039	0.079
Model 2	1795.06	1072	1.68	35.48	26	0.92	0.92	0.92	0.039	0.079
Model 3	1797.79	1073	1.68	38.20	27	0.92	0.92	0.92	0.039	0.079
Model 4	1807.69	1076	1.68	48.10	30 *	0.92	0.92	0.92	0.039	0.081
Model 5	1835.34	1084	1.69	75.75	38 **	0.92	0.92	0.92	0.040	0.081
Model 6	1965.44	1118	1.76	205.85	72 **	0.91	0.91	0.91	0.041	0.082

* p < 0.05; ** p < 0.001.

displays the various fit indices for each of the models. There were no significant differences between the unrestricted model (Model 1) and the mean weighted invariance model (Model 2). In turn, differences were found between Model 1 (unrestricted model) and Model 3 (structurally invariant covariance model) and between Model 1 and Model 4 (invariant model of mean residuals). The lack of significant differences between Model 1 and Model 2 attained the minimum criteria for acceptance of invariance relative to gender [51].

For the purpose of examining the factor structure of the eight-factor model, as well as the two-factor higher order model, a multigroup analysis of invariance was conducted across gender. Table 1 provides the various fit indices for the various models. For the eight-factor model, significant differences were not found between the unrestricted model (Model 1) and the weighted mean invariance model (Model 2). Differences were encountered between Models 1 and 3 (invariant structural covariance model) and Model 4 (mean residual invariant model). The lack of significant differences between Models 1 and 2 was sufficient to consider the proposed model to be invariant with respect to gender [51].

With respect to the higher-order model, significant differences were not found between the unrestricted model (Model 1), the weighted mean invariance model (Model 2), and the invariant structural covariance model (Model 3). The results indicated significant differences among Models 1 and 4 (mean residual invariant model) as well as with Model 5. These results provide support as well for invariance across gender for the higher-order model.

3.3. Descriptive Statistics

As can be observed in

Table 2. Descriptive statistics and bivariate correlations.

Factor	M	SD	1	2	3	4	5	6	7	8
1. Embarrassment	2.35	0.96		0.56 **	0.67 **	0.50 **	−0.12 *	−0.13 **	−0.14 **	−0.18 **
2. Boredom	2.44	1.05			0.66 **	0.41**	−0.15 **	−0.18 **	−0.16 **	−0.23 **
3. Hopelessness	1.97	1.05				0.51**	−0.14 **	−0.21 **	−0.19 **	−0.27 **
4. Anxiety	2.11	1.23					−0.22 **	−0.22 **	−0.29 **	−0.31 **
5. Pride	4.82	1.55						0.38 **	0.59 **	0.51 **
6. Confidence	4.94	1.27							0.42 **	0.50 **
7. Calmness	4.61	1.44								0.68 **
8. Enjoyment	4.60	1.26

* p < 0.01; ** p < 0.001. Note: M = Mean; SD = Standard deviation.

, the mean values overall for the positive emotions were higher than the mean values for negative emotions, with the Confidence subscale demonstrating the highest individual subscale mean value (M = 4.94). Negative emotion subscales demonstrated lower overall means, most notably on the Hopelessness subscale (M = 1.97).

3.4. Assessment of Internal Consistency and Temporal Stability

An assessment of the internal consistency of the scale was subsequently conducted, as well as an assessment of the scale’s stability over time. Cronbach’s alpha statistic was used to assess internal consistency and the associated alpha values for the subscales that were: 0.76 for Boredom, 0.82 for Hopelessness, 0.84 for Embarrassment, 0.85 for Enjoyment, 0.86 for Pride, 0.90 for Confidence, 0.91 for Anxiety, and 0.92 for Calmness. For the entire positive and negative emotions subscales, the associated Cronbach alpha values were 0.80 for the positive emotions scale and 0.82 for the negative emotions scale.

With regard to the analysis of temporal stability, an independent sample was used in which intraclass correlation coefficients (ICC) and confidence intervals (CI) were examined. For the analysis of temporal stability, data from an independent sample were obtained on two separate occasions, and the findings were assessed with regard to intraclass correlation coefficients (ICC) and confidence intervals (CI). Temporal stability was supported for each of the SEPE subscales, including Embarrassment = 0.85 (CI = 0.81–0.90); Pride = 0.85 (CI = 0.82–0.89); Boredom = 0.88 (CI = 0.87–0.93); Hopelessness = 0.86 (CI = 0.80–0.94); Anxiety = 0.86 (CI = 0.82–0.91); Confidence = 0.82 (CI = 0.77–0.88); Calmness = 0.87 (CI = 0.86–0.91); and Enjoyment = 0.78 (CI = 0.75–0.81).

4. Discussion

The purpose of the present study was to develop an appropriate instrument for the measurement of those emotional states that are most commonly experienced in the Physical Education environment. Eight different emotional states were included: embarrassment, anxiety, boredom, hopelessness, hope, enjoyment, confidence, and calmness.

To conduct the analyses, it was necessary to examine various psychometric indices including factor validity, internal consistency, and temporal stability of the SEPE instrument. The goal was to create a practical and brief questionnaire which appropriately represented the positive and negative types of emotions that students most commonly experience in classes of Physical Education. This process led to the generation of a valid and reliable instrument that is multidimensional and which was invariant for males and females.

The confirmatory factor analysis provided support for a 34-item eight factor model for the SEPE. Fit indices were acceptable, and temporal stability was also in evidence. The subscales attained adequate internal consistency, with Cronbach alpha values exceeding 0.70, which is commonly accepted as reflective of internal consistency [52]. The confirmatory factor analysis revealed that the set of negative emotions were correlated positively amongst each other, as was the set of positive emotions. The positive and negative emotions were inversely correlated with each other, as anticipated.

In accordance with these findings, it seems probable that the Physical Education context evokes various emotional responses among students as an outgrowth of their own personal experiences in this context and in relation to the favorability of their interactions with teachers and peers, among other factors. In turn, students’ psychological well-being is likely to be influenced by the nature of these experiences [10,53]. It is also likely that the satisfaction of psychological needs in Physical Education would contribute to the development of emotional growth and a resulting improvement in student perceptions of Physical Education classes. Logically, more favorable emotional experiences should extend to other additional educational settings beyond a given Physical Education classroom [54]. In addition, greater involvement in regular physical activity outside of the classroom might also result from a more favorable perception of their experiences in Physical Education.

An additional important finding from the present investigation was that the CFA results supported the existence of a two factor higher-order model in which positive emotions and negative emotions loaded on different factors. From a methodological point of view, this knowledge could result in the use of emotional assessment specific to either positive or negative emotions, if needed. In this sense, this result is shown according to the IZOF model [7], where it is stated that emotions possess a high degree of interindividual variability in the intensity and content that accompany successful and deficient individual performances. In this way, and according to the model, students can present positive and negative emotions depending on the individual differences in the level of available resources, the capacity of the students to efficiently achieve and use their motor skills, and the individual capacity of the students to compensate for the lack or insufficiency of knowledge and motor skills (little preparation for an exam or activity). The use of the global measure could prove worthwhile where emotional characteristics in PE are examined in relation to other variables. It should also be noted that the two-factor higher order model, as well as the eight-factor model, demonstrated invariance across gender and acceptable internal consistency of each subscale.

Limitations

Although the results of the present student provided psychometric support for the SEPE, there are also a few limitations that should be noted. First, the present sample was one of convenience, and thus, this pattern of findings should be examined with other groups of students who may vary by age, grade, and other variables given that instrument validation is a continuous process. Second, future research could attempt to determine the predictive validity of the instrument through comparison with studies using similar scales (e.g., AEQ), as well as to attempt to identify and determine psychological and behavioral predictors towards commitment to PE classes, adherence to physical activity practice, adoption of a healthy lifestyle, prosocial skills, etc.

5. Conclusions

In closing, the findings from the present study provided support for the validity and reliability of the SEPE instrument as a multidimensional measure of emotion in Physical Education classes. It is essential that we recognize the importance of assessing emotional states in our students in these classes so that we can better understand how to increase the likelihood of favorable and reduce the likelihood of unfavorable emotional experiences in this context. By doing so, we can increase the likelihood that these individuals will engage in physical activity outside of the school context and improve their quality of life.