*Article* **Spanish Adaptation of Motivational Climate in Education Scale with University Students**

#### **Antonio Granero-Gallegos and María Carrasco-Poyatos \***

Department of Education, Faculty of Education Sciences, Health Research Center, University of Almeria, 04120 Almeria, Spain; agranoero@ual.es

**\*** Correspondence: carrasco@ual.es; Tel.: +34-950-214773

Received: 23 May 2020; Accepted: 7 June 2020; Published: 9 June 2020

**Abstract:** The aim of this study was to provide evidence of the validity and reliability of the dimensionality of the Spanish adaptation of two correlated subscales to assess motivational climate in the university education context: Mastery motivational climate and performance motivational climate. Two different studies with cross-sectional design and different samples of university students were used to accomplish this research (Study 1: 181 students, mean age = 20.83, *SD* = 1.76; Study 2: 354 students, mean age = 21.84; *SD* = 1.98). In Study 1, Exploratory Structural Equation Modeling, Confirmatory Factor Analysis (CFA), and reliability analysis of the scale were conducted. In Study 2, CFA, reliability analysis, discriminant validity, temporal stability, factorial invariance across gender, and nomological validity were managed through a regression model measuring the relationships between democratic and autocratic behavior, mastery climate, and performance climate. The final version of the Motivational Climate in Education Scale showed acceptable goodness of fit and values of discriminant validity, reliability, temporal stability, and invariance across gender. According to its nomological validity, democratic behavior was shown to be a statistically significant predictor of mastery climate, and the teacher's autocratic behavior was shown to be a statistically significant predictor of performance climate. This scale is a valid and reliable instrument to assess mastery climate and performance climate in the Spanish university educational context.

**Keywords:** validation; mastery; task; performance; ego

#### **1. Introduction**

In recent years, various studies have shown that teaching style could affect motivation during the students' training process [1–3] and, in addition, as other authors state [4], decades of research confirm that one of the keys to determining the quality of the students' experience is the motivational climate created by teachers in their classes. The relationship between dispositional and environmental factors has been demonstrated, such as the motivational climate created by other people (e.g., the teacher) [5], with different academic variables: Motivation [6,7], the perception of competence and autonomy [8,9], school commitment [10], adolescent adjustment [11], academic achievement [12], or academic performance [13]. For this reason, the role of the teacher is recognized amongst these environmental factors [14], and the motivational climate that their role generates is so important that some authors emphasize that it may be responsible for the students' academic success or failure [15].

The Achievement Goal Theory (AGT) [14,16,17], framed within the cognitive–social theories from an interactional perspective, has also developed research in the educational field. This theory starts from the basic idea that people are intentional organisms directed by our objectives, and that we rationally act according to them. In achievement settings, such as the educational environment, the main objective is to show competence [14]. These achievement goals are related to motivational patterns of adaptation and maladjustment, and constitute the main mechanism for determining

whether success or, conversely, failure results [14,16]. The classroom motivational climate is assumed to affect individual outcomes through the student's individual achievement goal, which is established according to the surrounding goal climate [18].

According to the AGT theory, two motivational climates predominate in the social contexts of achievement situations, such as classes in the educational environment: The mastery climate (also the task-involving motivational climate) and/or the performance climate (the ego-involving motivational climate) [19–21]. An ego-involving motivational climate is associated with performance and with normative standards, and judges a student's performance compared to that of others in the class [22]. Both teachers and students tend to adopt ability-demonstration goals, and this is associated with boredom, minimal effort, negative classroom experiences, and lack of teacher support [23–25]. On the other hand, this motivational theory argues that, in social contexts promoting involvement in learning and mastery (i.e., task-involving), the teacher values cooperation, effort, and improvement, which favor cognitive, affective, and adaptive behavioral responses on the part of the students [26]. The perception of this mastery climate is focused on personal improvement and achievement, and is based on the belief that success comes from work and perseverance [22]. Student achievement has been shown to be more effective when teachers strive to create a climate of learning and mastery rather than when they are more concerned with avoiding failure or demonstrating ability [27–29].

The motivational classroom climate generated by the teacher has been widely studied in different educational contexts, especially in primary [8,30] and secondary education [9,31–33]. Some studies have also been carried out in the university context [34,35].

Although several scales have been used to measure the classroom motivational climate in the Spanish non-university educational context, such as the Learning and Performance Orientations in Physical Education Classes Questionnaire [36] or the Classroom Motivational Climate Questionnaire [37], no scale has been adapted to measure the motivational climate generated by the teacher in university classrooms. It should be noted that the Motivational Orientation and Climate Scale [34] from the original Norwegian version of the Perception of Success Questionnaire [38] has recently been validated in Spanish (from the Dominican Republic, Latin America) [34]; this measures both the orientation and the motivational climate. However, despite the importance of studying the motivational climate and the influence on various variables at the academic level, a scale that specifically measures the university student's perception of the mastery climate (i.e., task-involving) and the performance climate (i.e., ego-involving) has not been validated in the Spanish educative context. This is the present work's main contribution to the international literature, since it is intended to adapt and validate a scale that allows one to specifically measure these two variables in Spanish classrooms, including university classrooms.

For all of the above, two different studies have been proposed, in which it is hypothesized that the factorial structure of two perceived motivational climate subscales will show adequate goodness of fit, and that the two factors will negatively correlate. Furthermore, the teacher's democratic behavior is expected to be a predictor of the mastery climate, while the teacher's autocratic behavior is expected to be a predictor of the performance climate. Therefore, the objective of the present study is to provide evidence of reliability and validity for the dimensionality of two correlated subscales from the Spanish version, measuring the motivational climate in university education. The "Strengthening the Reporting of Observational Studies in Epidemiology" (STROBE) initiative [39] was used for the descriptions of Study 1 and Study 2.

#### **2. Study 1**

#### *2.1. Materials and Methods*

#### 2.1.1. Study Design

This is an observational, descriptive, cross-sectional, and non-randomized study. Spanish university students from the University of Almeria (Spain) participated in the first study. The data were collected

during the first semester of the 2019/2020 course. The inclusion criterion to participate in the research was to be a graduate student in a presential university course. The exclusion criteria were: (i) Not giving consent for the use of the data in research and (ii) not completely filling out the data collection form.

#### 2.1.2. Outcomes and Instruments

Motivational Climate in Education: In order to assess the motivational climate in educative classrooms, two dimensions that assess motivational climate perceptions were selected from the Norwegian version of the Perception of Success Questionnaire [38]. These two dimensions are made up of seven items and measure the motivational climate perceived by university students in class. The mastery motivational climate dimension (four items) implies that students are encouraged to improve their skills in relation to their own achievement level (e.g., "the teacher expects us to learn new skills and gain new knowledge"), while the performance motivational climate dimension (three items) stimulates competitiveness and social comparison (e.g., "students are encouraged to outperform others"). For the responses, a Likert scale was used from 1 (strongly disagree) to 5 (strongly agree).

#### 2.1.3. Procedure

Firstly, the items from the two subscales of the Motivational Orientation and Climate Scale (MOC)—mastery motivational climate and performance motivational climate [38]—were translated using back translation [40]. Two translators translated the seven items into Spanish. Subsequently, two further translators proceeded to translate the items back into their original language (back translation). To judge the accuracy of the translation, the degree of coincidence with the original version was considered. The final version obtained was analyzed by a group of four education experts [41] to guarantee the items' adequate design in the construct measure, which was intended to measure and maintain the original meaning [40]. The experts evaluated the relevance and understanding of each item on a scale from 1 (strongly disagree) to 4 (strongly agree). If the mean item scores were <2.5, they were reviewed. If an item was not classified by at least three of the four experts within the theoretical dimensions of the scale, it was reviewed again. The global agreement of four experts on the relevance and understanding of the items was measured with the Intraclass Correlation Coefficient (ICC), using a mixed effects model and assuming an absolute agreement definition; the values obtained were: ICC = 0.85 for relevance and ICC = 0.92 for understanding. The Spanish version was administered to 45 higher education students between the ages of 18 and 30, all of whom expressed full understanding of the items. Thus, the final version of the Spanish Motivation Climate in Education Scale (MCES) was obtained.

Subsequently, the professors and those in charge at the education faculty were contacted asking for their collaboration and to give information regarding the purpose of the research. Students were informed by email to participate in the study. The instrument was finally administered via an online form, in which the importance of the research and how to complete the scale were briefly explained, as well as informing participants that they could leave the study at any time. All subjects gave their informed consent to be included before participating in the study. The study was conducted in accordance with the Helsinki Declaration, and the protocol was approved by the Bioethics Committee at the University of Almeria.

#### 2.1.4. Risk of Bias Assessment

In terms of bias control, it should be noted that there was no sample randomization since convenience sampling was followed. However, there was blinding between the participants and the researchers in charge of data treatment and analysis. Regarding selection bias, it was indicated that participation in the study was voluntary and the communication with students was by email.

#### 2.1.5. Sample Size

Regarding the sample size (*n* = 181), the provisions set by Carretero-Dios and Pérez [42] were complied with in relation to the preliminary study of the items for scale validation: Between 50 and 100 participants with characteristics similar to those of the population, and between 5 and 10 participants for each item on the scale. According to the mean standard deviation established in a previous study [38] (SD = 1.12, performance motivational climate; 0.80, mastery motivational climate) and an estimated error (d) of 0.17 (performance motivational climate) and 0.12 (mastery motivational climate), a valid sample size for a 95% confidence interval (CI) was between 167 and 171 (*<sup>n</sup>* <sup>=</sup> *CI*<sup>2</sup> <sup>×</sup> *<sup>d</sup>* 2 /*SD*<sup>2</sup> ). A total of 181 students completed the study.

#### 2.1.6. Data Analysis

The factorial structure was evaluated using Exploratory Structural Equation Modeling (ESEM) and a Confirmatory Factor Analysis (CFA). After verifying each item of the factorial structure and obtaining a final model, a CFA was performed to verify the factorial structure of the ESEM. The factors were correlated taking into account that the data were collected from different classes or groups of students; to avoid the non-dependence of the observations, the Mplus cluster option and COMPLEX function were used. The ESEM model was estimated by considering the recommendations of Marsh, Morin, Parker, and Kaur [43] so that all of the rotated loadings were freely estimated, and the Geomin oblique rotation was chosen with an epsilon value of 0.5 in order to facilitate the subsequent comparison of the factorial structure obtained [44]. Furthermore, in the absence of a secondary factor loading of >0.32, primary factor loadings of >0.50 were considered adequate [45] to maintain an item in a factor. Standardized factorial loadings (λ) were reported.

The models (ESEM and CFA) were tested using the robust maximum likelihood estimation method for continuous variables (MLR) [46], and the evaluation of the models was based on the following goodness-of-fit indexes: χ 2 /df ratio values, the Comparative Fit Index (CFI), the Tucker–Lewis Index (TLI), the Root Mean Square Error of Approximation (RMSEA) with its 90% confidence interval (CI), and the Standardized Root Mean Square Residual (SRMR). For the χ 2 /df ratio, values <2.0 or <5.0 were considered excellent [47] or acceptable [48], respectively, <0.95 or between 0.90 and 0.95 (CFI and TLI), below 0.06 or 0.10 (RMSEA), respectively, indicating an excellent or marginally acceptable fit [49], and SRMR values <0.08 [48]. The descriptive statistics were calculated with SPSS 24.0, and the rest of the models (ESEM and CFA) were performed with Mplus 7.0.

The reliability of the scale was evaluated using different parameters: Composite reliability using McDonald's ω [50], Average Variance Extracted (AVE) [51], and Cronbach's alpha (α). Compared with the values traditionally used to assess reliability, such as α, the ω has the advantage of considering the strength of association between the items and the constructs, as well as the measurement errors specific to each item [52]. Reliability values >0.70 were considered acceptable. In addition, a temporal stability analysis was performed using the ICC and its 95% CI, considering values ≥0.70 as being adequate [53].

#### *2.2. Results*

#### 2.2.1. Participants

A total of 181 university students participated, aged between 19 and 25 years old (M = 20.83; SD = 1.76). Of these, 76.2% were male. A further 19 students did not give their consent to participate in the data collection, so they were not included in the study. The data were collected in December 2019. There were no missing values in the data of the included sample.

#### 2.2.2. Exploratory Structural Equation Modeling

The descriptive statistics of the items and the ESEM results are shown in Table 1. The ESEM (i.e., the preliminary exploratory analysis) verified the factorial structure of the MCES in two factors. The ESEM results showed high primary factorial loadings (>0.67) and secondary factorial loadings

below 0.04. The two factors were made up of seven items—four items in factor 1 (F1) and three items in factor 2 (F2); they showed the following goodness of fit: χ 2 /df = 1.00, p = 0.288; CFI = 0.99; TLI = 0.99; RMSEA = 0.034 (90% CI = 0.000; 0.098), SRMR = 0.014 (Table 1).

**Table 1.** Item descriptions and exploratory structural equation modeling of the Motivation Climate in Education Scale (MCES).


Note. ESEM = Exploratory Structural Equation Modeling; M = mean; SD = standard deviation; Q1 = skewness; Q2 = kurtosis; F1 = mastery motivational climate; F2 = performance motivational climate; λ = standardized factor loadings; factor loadings >0.50 (primary) are highlighted in bold.

#### 2.2.3. Confirmatory Factor Analysis

Next, a CFA was performed with the seven-item model consisting of two correlated factors (Table 2). The model's goodness-of-fit indices were excellent: χ 2 /df = 2.00, *p* = 0.023; CFI = 0.978; TLI = 0.965; RMSEA = 0.071 (90% CI = 0.026; 0.113), SRMR = 0.053. The standardized factorial loadings of the CFA can be verified in Table 1. Likewise, the reliability values were as follows: The mastery motivational climate (F1)—ω = 0.89, α = 0.88, and AVE = 0.65; the performance motivational climate (F2)—ω = 0.87, α = 0.87, and AVE = 0.70.

**Table 2.** Confirmatory factor analysis, standardized factor loadings for the confirmatory factor analysis (CFA) solutions.


Note: CFA = Confirmatory Factorial Analysis; F1 = mastery motivational climate; F2 = performance motivational climate; \*\* *p* < 0.001.

#### **3. Study 2**

#### *3.1. Materials and Methods*

#### 3.1.1. Study Design

This is an observational, descriptive, cross-sectional, and non-randomized study. Spanish university students from the University of Almeria (Spain) participated in this second study, but were from different courses from those in Study 1. The data were collected during the second semester of the 2019/2020 course. The inclusion criterion was to be a graduate student of a presential-mode university degree. The exclusion criteria were: (i) Not giving consent for the use of data in research, (ii) not completely filling out the data collection form, and (iii) having participated in Study 1.

#### 3.1.2. Outcomes and Instruments

Motivational Climate: The MCES version used was the same as in Study 1.

The Scale of Democratic Behavior and Autocratic Behavior (ECDA): The Democratic Behavior (e.g., "encourages students to make suggestions on how to give classes") and Autocratic Behavior (e.g., "presents their ideas in a forceful way") subscales were used; they were validated in the Spanish university context [54] and were derived from the original Leadership Scale for Physical Education [55]. Each factor is represented by three items that are answered using a Likert-type scale from 1 (never) to 5 (always). In the present study, this scale showed an excellent goodness of fit: χ 2 /df = 1.34, *p* = 0.220; CFI = 0.99; TLI = 0.99; RMSEA = 0.031 (90% CI = 0.000, 0.074), SRMR = 0.02. Reliability: Democratic behavior—ω = 0.76, α = 0.85; AVE = 0.56; autocratic behavior—ω = 0.76, α = 0.78; AVE = 0.52.

#### 3.1.3. Procedure

The professors and those in charge of the education faculty were contacted to request collaboration and to report on the purpose of the investigation. The education students were asked by email to participate in the study. The instrument was finally administered via an online form, in which the importance of the research and how to complete the scale were briefly explained, as well as informing participants that they could leave the study at any time. All subjects gave their informed consent to be included before participating in the study. The study was conducted in accordance with the Helsinki Declaration, and the protocol was approved by the Bioethics Committee at the University of Almeria.

#### 3.1.4. Risk of Bias Assessment

Related to risk of bias, in this second study, it should be noted that there was no sample randomization, since convenience sampling was followed. Despite this, there was blinding between the participants and the researchers in charge of data treatment and analysis. Regarding selection bias, it was indicated that participation in the study was voluntary, and the communication with students was by email.

#### 3.1.5. Sample Size

With the total sample (*n* = 354), the confirmatory analysis requirements [42] of 10 participants for each item on the scale were met. For the nomological validity analysis, an a priori power analysis conducted using the Free Statistics Calculator v.4.0 software [56] indicated that a minimum sample size of 166 would be sufficient for the model structure to detect small effect sizes (i.e., f<sup>2</sup> = 0.10) with a power level of 0.95 and a two-tailed significance level of α = 0.05 in a structural equation model (SEM) with four latent variables and thirteen observed variables. A total of 354 students completed the study.

#### 3.1.6. Data Analysis

The descriptive statistics and correlations were calculated with SPSS 24 (IBM, Chicago, IL, USA). The CFA model to verify the factorial structure of the MCES from Study 1 was calculated

with Mplus 7.0 [57]; hence, the two factors were correlated: Task-involving climate (items 1, 3, 5, 7) and performance-involving climate (items 2, 4, 6). Insomuch as the data were collected from different classrooms, and to avoid the likely non-independence of the observations, the cluster option and the COMPLEX function of Mplus were used. Standardized factor loadings (λ) were reported. The model (CFA) was tested employing the MLR estimation method [46]. The same combination of fit indices was used as in Study 1 in order to check the model's goodness of fit (χ 2 /df, CFI, TLI, RMSEA, SRMR). Scale score reliability estimates were also computed using three parameters: ω for composite reliability [50], AVE [51], and α. In addition, a temporal stability analysis was performed using the ICC and its 95% CI, considering values ≥0.70 to be adequate [53]. To obtain evidence supporting the discriminant validity of the constructs, the heterotrait–monotrait ratio (HTMT) of the correlations between factors was obtained, and values <0.85 were considered adequate [58]. The MCES invariance was tested across sex, employing the MLR estimation. Four progressively more restrictive models were run for each of the two factors: (1) Configural invariance, (2) weak invariance (i.e., invariance of the factor loadings/cross-loadings), (3) strong measurement (i.e., invariance of the factor loadings/cross-loadings and intercepts), and (4) strict invariance (i.e., invariance of the factor loadings/cross-loadings, intercepts, and residual variances). In regards to the measurement invariance, the nested models were compared taking into account the changes (∆) in the goodness-of-fit indices (i.e., increases in RMSEA of at least 0.015 or decreases in CFI and TLI of at least 0.010 indicated a lack of invariance) [59]. To provide evidence of nomological validity, we conducted regression model analysis (direct effects) between factors of democratic behavior and autocratic behavior (latent variables) on mastery motivational climate and performance motivational climate (latent variables).

#### *3.2. Results*

#### 3.2.1. Participants

The participants were 354 university students (59.6% male) from Education Sciences, aged between 19 and 29 years old (*M* = 21.84; *SD* = 1.98) from the University of Almeria, Spain. The participants were different from those in Study 1. In the data collected (January–February 2020), there were no missing values from the total sample. Apart from the total sample, twelve questionnaires were discarded because they were incomplete, and seven people did not consent to participate in this study.

#### 3.2.2. Structure and Reliability Factor

Table 3 shows the descriptive statistics and correlations of the MCES items. The standardized factor loadings of the four items for the task motivational climate (F1) and the three items for the performance motivational climate (F2) of the MCES are shown in Figure 1. The seven-item model with CFA achieved an acceptable fit, as demonstrated by the goodness-of-fit values: χ 2 /df = 4.00, *p* < 0.0001; CFI = 0.967; TLI = 0.946; RMSEA = 0.082 (90% CI = 0.067, 0.119), SRMR = 0.044. The reliability analyses showed the following values: Mastery motivational climate: ω = 0.90, α = 0.89, AVE = 0.68; performance motivational climate: ω = 0.89, α = 0.89, AVE = 0.72. The temporal stability analysis was evaluated using the ICC for the two factors, with values obtained >0.82, for which the instrument was administered to an independent sample on two occasions with an interval of four weeks between the two data collections. Regarding the discriminant validity, the value of the HTMT proportion in the correlation between the different dimensions was <0.85 [58].


**Table 3.** Descriptive statistics and correlations for the items of MCES.

Note. \*\**p* < 0.001; F1 = mastery motivational climate; F2 = performance motivational climate.

**Figure 1.** Confirmatory Factor Analysis—standardized factor loadings and residual variances. Note: The ellipses represent the latent factors and the rectangles represent the items; \*\* *p* < 0.001.

#### 3.2.3. Measurement Invariance

The invariance according to gender (i.e., male=211, female=143) of the MCES was evaluated based on the CFA model, the results of which are shown in Table 4. Starting with a configural invariance (M0) model, invariance constraints were progressively added to the loading factors (i.e., weak invariance, M1), intercepts (i.e., strong invariance, M2), and residual variances (i.e., strict invariance). The values of these restrictive models were acceptable, except for the strict invariance, since the results were outside the cut-off values in the CFI. The weak and strong configural invariance models did not exceed the recommendations for RMSEA (∆ > 0.015), CFI (∆ > 0.01), and TLI (∆ > 0.01), but, as can be seen in Table 4, the strict invariance showed a decrease that slightly exceeded the limits of the recommended values (∆CFI = 0.013). However, the RMSEA and TLI values did not exceed the MCES recommended limits for invariance by gender in university students.

1


**Table 4.** Invariance test across gender for the MCES.

Note. χ <sup>2</sup> = Chi square; df = degrees of freedom; RMSEA = root mean square error of approximation; 90% CI = 90% confidence interval of the RMSEA; CFI = comparative fit index; TLI = Tucker–Lewis index; \* *p* < 0.01.

#### 3.2.4. Nomological Validity

The regression model (SEM) was performed with the bootstrapping technique (5000 samplings) because the Mardia coefficient presented a high value (18.78). Thirteen observed variables and four latent variables were introduced into the analysis. The results of the regression model estimated sample demonstrated a good fit with the data: χ 2 /df = 2.89, *p* < 0.001; CFI = 0.954; TLI = 0.954; RMSEA = 0.073 (90% CI = 0.061, 0.086), SRMR = 0.048. The factor loadings of the indicators across all of the model's latent variables ranged between 0.48 and 0.89. In our model (see Figure 2), the direct relationships between democratic behavior and the performance motivational climate were not significant, nor were the relationships between autocratic behavior and the mastery motivational climate. Table 5 also presents the point estimates as well as the bias-corrected bootstrapped 95% confidence intervals for the mediated effects. A significant positive relationship and high effect size was found between democratic behavior and the mastery motivational climate (β = 0.81), and between autocratic behavior and the performance motivational climate (β = 0.70). The model accounted for 70% of the variance in the mastery motivational climate and 38% of the variance in the performance motivational climate.

**Figure 2.** Regression model measuring the relationships of democratic and autocratic behavior with mastery motivational climate and performance motivational climate. Note: \*\* *p* < 0.001; broken lines represent non-significant relations.



Note: B = regression weights; SE = standard error; β = standardized regression weights; Est = estimate; IC = confidence intervals; LL = lower limit; UL = upper limit; R<sup>2</sup> = variance.

#### **4. Discussion**

The objective of this study was to provide psychometric evidence on the dimensionality of the Spanish version of two correlated subscales of the MOC [38] in order to measure the perceived motivational climate with higher education students (MCES). Through the psychometric analysis of the scale of two factors correlated by two different studies, the final version of the MCES shows adequate levels of factor validity, discriminant validity, reliability, and temporal stability, as well as gender-based invariance. Thus, the first hypothesis is fulfilled. Furthermore, for nomological validity, democratic behavior was shown as a predictor of the mastery motivational climate, whereas the teacher's autocratic behavior was shown as a predictor of the performance motivational climate. Accordingly, the second hypothesis posed in this study is also fulfilled.

The present work focused on validating the two motivational climate factors (i.e., mastery and performance) of the original Stornes and Bru scale [38] for use in education. Like other validated scales for measuring the motivational climate in other contexts [36,60,61], a factorial structure with only these two correlated factors has been used. The adaptation of this scale to the educational context with university students has presented adequate values for the goodness-of-fit indices of the CFA. The original Stornes and Bru scale [38] also presented adequate goodness-of-fit indices at the psychometric level, but with four factors. In this regard, it should be noted that the four-factor original by Stornes and Bru [38] has recently been validated in Portuguese and Latin American Spanish (The Dominican Republic) by Gutiérrez, Tomás, Gómez, and Moll [62] to measure the orientation and the motivational climate in students from 14 to 18 years of age, and that, in general, it has presented adequate psychometric fits, both in the Portuguese version with the Angolan population and in the version with the Dominican Republic population. Likewise, Gutiérrez and Tomás [34] adapted this four-factor scale to the university population in the Dominican Republic, although this validation presents some indices below the acceptable value, such as the TLI value and the χ 2 /df ratio.

Regarding reliability, the results of the present study demonstrate the high reliability of the two motivational climate dimensions, with values for α and composite reliability (ω) >0.86. In the study by Stornes and Bru [38], the α values for these two factors showed adequate reliability with scores >0.70; on the other hand, in the Dominican Republic university education version by Gutiérrez and Tomás [34], both the mastery climate factor and the performance climate factor presented α values <0.70, although the composite reliability values were adequate (>0.70). In the aforementioned adaptation by Gutiérrez and Tomás [34], the performance climate also presented α values below 0.70 in both the Angolan and Dominican Republic student samples, and the mastery climate achieved α values of <0.70 in the Angolan sample.

For the AVE values, the present study showed adequate indices in the two dimensions of the motivational climate with values >0.67, taking into account that the cut-off for the minimum acceptable values is >0.50 [53]. The scale also demonstrated adequate temporal stability. These are notable contributions made by this research to the literature, since none of the cited versions of this scale [34,38,62] show AVE values for these factors or a temporal stability analysis for the scale. In relation to the discriminant validity, it should be pointed out that an analysis using the HTMT ratio has also been performed in the present study, showing adequate values. The other cited studies do not conduct this HTMT analysis, although the correlation found between the mastery climate and performance climate was negative.

In relation to the analysis of invariance by gender, the scale is invariant at the configural level—weak and strong—although not in the strict invariance model. In the other four-factor versions of this scale, no factor invariance by gender analysis has been shown, although Gutiérrez et al. [62] presented invariance by country.

Regarding the relationship of the two motivational climate dimensions with teacher leadership, the results show coherence with the previous literature, given that democratic behavior is predictive of the mastery motivational climate, while autocratic behavior is predictive of the performance motivational climate. These results are in line with previous studies, such as that by Mohammadzade, Zardoshtian, and Hossini [63], which demonstrated the positive predictive relationship between autocratic behaviors by the coach and the performance climate, whereas this relationship was not significant between democratic behaviors and the performance climate. Likewise, Alfermann, Lee, and Würth [64] (in athletes) and Bekiari [65] (in students) found a high and positive correlation between the teacher's democratic behavior and the motivational climate towards mastery, while the relationship between the teacher's autocratic behavior and the motivational climate towards performance was also positive and significant. Authors such as Bari´c and Bucik [66] and Smith, Fry, Ethington, and Li [67] (in athletes) and Habibullah and Sinha [68] (in the business environment) also highlighted the positive and significant relationships between democratic behavior and the mastery climate and between authoritarian behavior and the performance climate.

Despite the strengths and relevant results in relation to the psychometric properties of the MCES, this study also presents certain limitations that must be pointed out. First, the sample was selected according to convenience, not randomly, so the results obtained cannot be generalized. Second, the sample analyzed was made up of university students from a single university. Future research should examine the psychometry of this scale in students belonging to different educational levels (i.e., primary or secondary education) and other universities to determine if the factorial structure remains invariant based on this variable.

#### **5. Conclusions**

The Motivational Climate in Education Scale constitutes a valid and reliable instrument for evaluating the mastery motivational climate and the performance motivational climate during the teaching–learning process in the Spanish educational context. This two-correlated-factor version of the instrument presents a better fit than the four-factor scale (including the dispositional orientation and the motivational climate), and could contribute to more deeply analyzing the influence of the motivational climate generated by the teacher on the motivation, commitment, and academic achievement of students within their training process. These aspects are interesting from the teaching point of view, since a well-channeled motivational climate (i.e., mastery climate) based on work, consistency, improvement, and self-improvement could contribute to a greater teaching efficiency and favor students' cognitive, affective, and behavioral responses.

**Author Contributions:** Conceptualization, A.G.-G.; data curation, M.C.-P.; formal analysis, A.G.-G.; investigation, A.G.-G. and M.C.-P.; methodology, A.G.-G.; project administration, A.G.-G. and M.C.-P.; writing—original draft, A.G.-G. and M.C.-P.; writing—review and editing, A.G.-G. and M.C.-P. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Acknowledgments:** This work was carried out with the aid received from the call of Grupos Docentes de Innovación y Buenas Prácticas, University of Almeria; biennium 2020–2021 [Project: Metodologías activas y aprendizaje competencial en educación superior (Active methodologies and competency learning in higher education)].

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**


© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
