*Article* **Development and Psychometric Properties of a Scale to Measure Resilience among Portuguese University Students: Resilience Scale-10**

**Jacinto Jardim 1,\* , Anabela Pereira <sup>2</sup> and Ana Bártolo <sup>2</sup>**


**Abstract:** Higher education students lack skills to deal in a positive way with the crises inherent to the developmental phase in which they find themselves. However, the complexity of the current global context adds an urgent need for them to be resilient. For this purpose, it is necessary to evaluate their capacity for resilience, which requires the use of instruments that are easy to access, useful, simple, and fast. In this sense, this study aimed to develop and validate a scale to assess students' resilience in the face of adversity. The scale was administered to a sample of 2030 Portuguese higher education students. The results obtained pointed towards a factorial structure composed of two factors named "self-determination" and "adaptability", which showed good internal consistency. Therefore, this scale proved to be a valid measure to assess resilience among the university population. Future studies may consider this variable as an intervention target since it can be a predictor of success in phases of change and crisis.

**Keywords:** resilience; validity; higher education; resilience scale-10

#### **1. Introduction**

Given the continuous transformation of society and the unpredictability of its future, knowing how to deal with adversities has become an essential competence to survive healthy and with quality. This capacity, called resilience, has gained relevance in scientific research [1–6].

The resilience has been defined from different perspectives: As an interactive concept which is concerned with the combination of serious risk experiences and a relatively positive psychological outcome despite those experiences [7]; as a universal capacity that allows a person, group or community to prevent, minimize or overcome the harmful effects of adversity [8,9]; as the capacity of a dynamic system to adapt successfully to disturbances that threaten the system function, viability, or development [10]; as the outcome from negotiations between individuals and their environments for the resources to define themselves as healthy amidst conditions collectively viewed as adverse [11]; and also as the "ability to use knowledge, attitudes and skills in order to prevent, minimize or overcome the harmful effects of crises and adversities" [2] (p. 167). Therefore, a resilient person, facing a stressful or adverse situation, manages to use his or her personal resources by adopting behaviors that help being successful in that circumstance.

If the complexity of the current global context calls for psychological support for the general population [12], in the case of higher education students, the need for this competence is further justified by the fact that they have to face the crises inherent to the developmental stage in which they find themselves, and the obstacles of the education system itself, as well as the unpredictability of the labor market, in which they will soon be integrated [13–16].

**Citation:** Jardim, J.; Pereira, A.; Bártolo, A. Development and Psychometric Properties of a Scale to Measure Resilience among Portuguese University Students: Resilience Scale-10. *Educ. Sci.* **2021**, *11*, 61. https://doi.org/10.3390/ educsci11020061

Academic Editor: Maria José Sousa Received: 14 January 2021 Accepted: 1 February 2021 Published: 7 February 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

Previous studies point towards the need to promote this capacity among higher education students as it facilitates adaptation to changes and ambiguous situations, the recovery from exhausting and tiring conditions, as well as the proficiency in maintaining a calm, clarity of objectives and guidance in hostile situations. Additionally, it enables the student to think strategically and make appropriate decisions in times of pressure, to perform complex tasks, and to adopt flexible behaviors in problem solving. In this domain, several intervention programs have been evaluated as effective [17–19]. For example, the program You Can Do It! [18], which has been widely applied in multiple Australian schools, presents resilience as the basis for social and emotional learning. Nevertheless, research indicates that the effectiveness of a program depends a lot on the type of activities implemented [19]. Therefore, resilience is a concept that should continue to be investigated, so that the programs made available to young students include pedagogical strategies and resources more oriented to the promotion of this capacity.

In order to assess resilience and determine the criteria for the functioning of the resilient person, several scales have been developed and validated [3,20–25]. Most of the existing scales use mainly clinical criteria, as proposed by Anaut [17,18]: The level of anxiety and depression, the level of social competence, school and intellectual success, and clinical symptoms. Therefore, they focus on behavioral and cognitive assessments, personality traits, and psychological disorders. In turn, Sætren et al. [20] (p. 3) proposed an evaluation through the "three-factor model of personal resiliency", that reflects aspects of the individual's personal experience in three dimensions of his or her own development: "Sense of mastery, sense of relatedness, and emotional reactivity". Another instrument also used to assess resilience is the one by Prince-Embury [25], called resiliency scales for children and adolescents (RSCA), which assesses psychological symptoms and psychological status. RSCA comprises three scales: Sense of control—assesses personal attributes such as optimism, self-efficacy, and adaptability; relationship capacity—assesses trust, support, comfort, and tolerance; and emotional reactivity—assesses sensitivity, recovery, and injury.

Therefore, most of the aforementioned instruments address resilience at the clinical level, with predominance to traumatic and health situations of children and adolescents. Additionally, they are often complex scales in terms of completion and interpretation, which makes both application and accessibility difficult in contexts where a large number of individuals need to be evaluated, as in the case of higher education. In order to bridge this gap, we aimed to develop and assess the psychometric proprieties of a new self-assessment scale—resilience scale-10 (RS-10)—to measure resilience in a non-clinical community. More specifically, the purposes of this study were to (i) develop a brief measure of resilience appropriate for university students; and (ii) examine the factor structure and reliability of this scale.

#### **2. Methods**

#### *2.1. Development of the Portuguese Version of the RS-10*

This study consisted of two phases, beginning with the scale development phase aiming at identifying the appropriate items to include in a self-report measure. A set of items related to resilience was collected, considering the following parameters: Knowledge obtained in the literature; objectives of the instrument to be built; population to be evaluated; dimensions of the construct to be evaluated; and behavioral aspects of the construct to be included in the evaluation [26]. Additionally, in order to define the RS-10 items, we used as a reference the questionnaires developed by Grotberg [8,26] and Anaut [1,19]. Based on this process, 30 initial items were obtained. These 30 items were presented to a panel of experts with seven researchers/university professors in the areas of psychology, education, and management, obtaining a reduced final version including only 10 items. The 10 selected items were included since they were more directly related to the ability to deal with adversities in daily life, without a focus on traumatic situations. This version went through a process of spoken reflection carried out with a class of 3rd year students from a private university in Porto, consisting of 22 students (7 male and 15 female). These

participants were instructed to report their subjective perception of personal skills to face adversity. Each item was answered using a 5-point (1 = never; 2 = rarely; 3 = sometimes; 4 = almost always; 5 = always) response scale, so possible scores ranged from 10 to 50. Higher scores indicated a higher level of resilience. The application of the questionnaire lasted approximately 10 min and, subsequently, questions related to the instructions for completing the instrument, suitability of the included items, order of the items, and eventual doubts related to the semantics were discussed with the students (see the final version in Table 1; see also Appendix A). Thereafter, the scale's psychometric properties were assessed using a sample of Portuguese higher education students.


**Table 1.** Description of the items of the resilience scale-10 (RS-10).

#### *2.2. Validating the RS-10: Sample*

The sample included 2030 Portuguese university students over 18 years old (M = 21.11; SD = 2.00; range: 18–26 years) and the majority were female (77.1%). Students were recruited from several Portuguese Higher Education institutions. Most university students were based in higher education institutions located in the North of Portugal (51.5%). Participants attended different training areas, namely social sciences (59.8%), education, health and tertiary services (33.1%), and exact sciences such as engineering, computer science, mathematics, building and agriculture, and transformation industry (7.1%). Among the students, 71.8% were undergraduates and 28.2% were master students. This study has been carried out in accordance with the Declaration of Helsinki and informed consent was obtained from all the participants.

#### **3. Data Analysis and Results**

#### *3.1. Properties of Items*

A preliminary analysis was conducted in order to assess descriptive statistics, normality, and non-multicollinearity at the item level (see Table 2). The mean response for the 10 items was 3.62 (SD = 0.20). No deviations from the normal distribution were found, as the kurtosis and skewness scores for each item fell within −2 and 2. All the items presented significant positive corrected item-total correlations (≥0.42). The internal consistency of

the total scale was good (α = 0.866) and there was a low variation in reliability if items were deleted. All inter-correlations among all the items were below 0.65 suggesting no multicollinearity. Based on this, all the items were retained from subsequent analyses.



#### *3.2. Factorial Validity*

To test the factorial structure of the RS-10, a combination of the principal components analysis (PCA) and confirmatory factor analysis (CFA) was conducted. The sample was randomly split into two samples through the randomization function on the Statistical Package for Social Sciences, version 24 (SPSS Inc., Chicago, IL, USA), resulting in 2030 observations in total, 791 for the PCA and 1239 for the CFA.

PCA using the oblique rotation method suggested a two-factor structure. The Kaiser-Meyer-Olkin (KMO) measure presented a value of 0.897 and Barlett's test of sphericity was significant (χ <sup>2</sup> = 3412.112, *p* < 0.001). Factor 1 comprised items 1, 2, 3, 4, and 5 (5 items), which we termed as "self-determination", based on the three sources of resilience features labelled by Grotberg [8]. Factor 2 included five items (6, 7, 8, and 9, 10), which we termed as "adaptability". These two factors explained together 60.70% of the total variance. Table 3 shows the factor loadings (>0.60) and communalities (>0.40). Additionally, a good internal consistency for each factor was estimated (Factor 1: Cronbach's α = 0.843; Factor 2: Cronbach's α = 0.813).



The two-factor model derived from PCA was then cross-validated. Mardia's kurtosis coefficient of 20.95 with a critical ratio of 23.80 indicated that the data were multivariate non-normal. Therefore, CFA was performed using the maximum likelihood estimation (ML) with bootstrapping (1000 resamples) to generate accurate estimations of standard errors with accompanying confidence intervals (bias-corrected at the 95% confidence level). To assess the overall model fit, we use the following parameters: Root mean square error of approximation (RMSEA), comparative fit index (CFI), and the standardized root mean

square residual (SRMR). The criteria for an acceptable model fit were CFI ≥ 0.95; RMSEA ≤ 0.10 [27], and SRMR ≤ 0.08 [28]. The chi-square test (χ 2 ) was reported, but not used to check the model fit due to its sensibility to large samples [29]. Data obtained from the CFA showed that the two-factor model fitted well descriptively (CFI = 0.957; RMSEA = 0.068 (90% CI 0.059–0.076); SRMR = 0.040). All the standardized factor loadings (see Figure 1) of the item parcels were statistically significant showing that question items were good indicators for each factor. ≥ 0.95; RMSEA ≤ 0.10 [27], and SRMR ≤ 0.08 [28]. The chi-square test (χ2) was reported, but not used to check the model fit due to its sensibility to large samples [29]. Data obtained from the CFA showed that the two-factor model fitted well descriptively (CFI = 0.957; RMSEA = 0.068 (90% CI 0.059–0.076); SRMR = 0.040). All the standardized factor loadings (see Figure 1) of the item parcels were statistically significant showing that question items were good indicators for each factor.

7 0.730 0.568 8 0.792 0.673 9 0.786 0.607 10 0.705 0.437

The two-factor model derived from PCA was then cross-validated. Mardia's kurtosis coefficient of 20.95 with a critical ratio of 23.80 indicated that the data were multivariate non-normal. Therefore, CFA was performed using the maximum likelihood estimation (ML) with bootstrapping (1000 resamples) to generate accurate estimations of standard errors with accompanying confidence intervals (bias-corrected at the 95% confidence level). To assess the overall model fit, we use the following parameters: Root mean square error of approximation (RMSEA), comparative fit index (CFI), and the standardized root mean square residual (SRMR). The criteria for an acceptable model fit were CFI

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 5 of 8

**Figure 1.** Confirmatory factor analysis: Two-factor solution using a part of the sample (database 2; *n* = 1239). **Figure 1.** Confirmatory factor analysis: Two-factor solution using a part of the sample (database 2; *n* = 1239).

#### **4. Discussion and Conclusions**

**4. Discussion and Conclusions** Universities normally have to prepare students to face the multiple difficulties they have to face, both in academia and in the subsequent period of integration in the labor market. However, there are deficiencies regarding the acquisition of skills appropriate to face these challenges [16,30–32]. In times of crisis, as experienced globally today, these are Universities normally have to prepare students to face the multiple difficulties they have to face, both in academia and in the subsequent period of integration in the labor market. However, there are deficiencies regarding the acquisition of skills appropriate to face these challenges [16,30–32]. In times of crisis, as experienced globally today, these are even more evident.

even more evident. In order to carry out an effective intervention in this domain, it is important to evaluate each student individually in order to activate different strategies for them. This argument supported the need to develop a brief and accessible measure for the assessment of resilience in higher education students identifying the degree of their ability to cope with adversity. Our analysis showed that RS-10 is a valid measure and can contribute to the understanding of the perception of personal skills to face adversity among university students. The scale demonstrated a meaningful and strong factor structure, with items effectively measuring the factors, as shown by the loadings. According to the results, the two-factor model identified in the factor analysis indicated that RS-10 can capture the dimensions of self-determination and adaptability that adequately summarize resilience.

With self-determination, despite the difficulties, obstacles, discouragements, and failures, the individual is sure to complete the previously outlined tasks. Therefore, selfdetermination allows goals to be achieved and personal projects and ambitions to be realized. In addition, according to several authors, both from the perspective of clinical psychology, organizations and human resources management, self-determination is decisive in carrying out projects, despite the obstacles [33–37].

In turn, adaptability means the ability to change, whenever necessary, the strategy in order to achieve the previously determined objective. As such, it presupposes mental flexibility and emotional plasticity in order to better face unpredictable and unexpected situations. This is a construct that explains the success of those entrepreneurs who identify opportunities and implement projects. Furthermore, it is a characteristic of societies where innovation predominates, such as the current one we live in. Moreover, according to recent research, flexibility is an essential construct to integrate work teams, to lead people in organizations, and to maintain adequate levels of health and well-being [36,38,39]. Similar to other scales, RS-10 has good internal consistency (α = 0.866), with the factor referring to self-determination showing a Cronbach's α = 0.843 and the factor relating to adaptability revealed a Cronbach's α = 0.813. For example, the Portuguese version of the resilience scale of Wagnild and Young [19], composed of 25 items, which evaluates adolescents presented a Cronbach's alpha of 0.86, and the Brazilian version presented a Cronbach's alpha of 0.80 [23]. In turn, "The Multidimensional Teachers' Resilience Scale" [40] presents the following Cronbach's alpha values: Motivational = 0.78; emotional = 0.72; social = 0.74; professional = 0.75. Furthermore, the original version of the Brief Resilience Scale [18] demonstrated good internal consistency, with Cronbach's alpha values between 0.80 and 0.91. Although the RS-10 shows promising psychometric properties, the scale's overall validity needs to be interpreted in the light of potential limitations. First, the main limitation of this study has to do with the sample, which is only Portuguese, thus we suggest a transcultural validation. Furthermore, the Portuguese version of this self-reporting tool must be translated and validated in other languages. Second, there was no assessment of the reproducibility/repeatability. In addition, an important next step is to assess the convergent validity of the RS-10 through the correlations of the scale with theoretically related constructs such as self-efficacy [33,40] and self-esteem [41,42].

Perceived self-efficacy while as a positive belief allows you to carry out new or difficult tasks in order to obtain desired results. This vision of self-confidence, associated with self-control allows us to deal with stress-inducing situations and promote resilience.

Overall, the RS-10 was developed with input from university students and is a quickresponse instrument appropriate for use across different training areas and degree levels. However, the generalizability of the study results should be better established in future studies in additional settings. Despite this, the study presents opportunities for empirical studies in the field of resilience and we believe that the use of this instrument in clinical practice may contribute to intervene effectively next to students, promoting their capacity for self-determination and adaptability, which are crucial to face the adversities inherent to human nature and the evolutionary dynamics of the history of society.

**Author Contributions:** Conceptualization, J.J., A.P., and A.B.; investigation, J.J., A.P., and A.B.; writing—original draft, J.J., A.P., and A.B.; writing—review and editing, J.J., A.P., and A.B. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Institutional Review Board Statement:** The study was conducted according to the guidelines of the Declaration of Helsinki, and approved by a ethics committee.

**Informed Consent Statement:** Informed consent was obtained from all subjects involved in the study.

**Data Availability Statement:** The data that support the findings of this study are available from the corresponding author [J.J.], upon reasonable request.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **Appendix A**

**Table A1.** Resilience scale-10 (RS-10) (Jacinto Jardim, Anabela Pereira, and Ana Bártolo, 2021) \*.


The following sentences refer to a variety of skills one may think to possess to a smaller or larger degree. Please rate each sentence considering how frequently you think in the way it describes, using the following rating scale: 1 = never; 2 = rarely; 3 = sometimes; 4 = almost always; 5 = always. \* Authorization to use this instrument should be asked from jacinto.jardim@uab.pt.

#### **References**


### *Article* **Higher Education and Employability Skills: Barriers and Facilitators of Employer Engagement at Local Level**

**Fátima Suleman <sup>1</sup> , Pedro Videira 1,\* and Emília Araújo <sup>2</sup>**


**Abstract:** This research examines the barriers and facilitators to employers' engagement with higher education institutions. The data were collected through interviews with a set of employers (*n* = 19) in the Northern region of Portugal, V.N.de Famalicão, in 2019. We begin by exploring employers' engagement activities as a potential solution to address local-level skill problems. Empirical evidence suggests that the engagement activities are mostly passive as firms use higher education largely as a recruitment channel. The differences in organizational goals and culture are the most cited barriers to the lack of more active engagement. Some efforts have recently been made to strengthen the ties between higher education and employers, notably through a local multi-stakeholder partnership as a potential broker. However, it will take time for this to bear fruit and contribute to reducing skill gaps and shortages. The data show that despite employers' apparent willingness, more effort must be made to encourage active engagement.

**Keywords:** higher education; employers' engagement; employability; multi-stakeholder partnership

#### **1. Introduction**

In recent decades, the collaboration between higher education (HE) and industry has grown significantly [1] fueled by marked changes in the modes of knowledge production and innovation [2,3] as well as increasing societal demands and expectations from higher education institutions (HEIs). This collaboration is based on the recognition that HEIs are an essential source of technology, innovation, and human capital [4]. Under the label of a third mission, HEIs are expected to interact with firms at the national and regional levels to promote innovation and competitiveness in the knowledge economy [5].

Only more recently have academics started to pay attention to other forms of collaboration regarding graduates' skills and training [4,6], namely in curriculum design and delivery [7], and in promoting employability skills [8]. Our study adds insights to this research agenda by examining the employers' perception of the barriers and facilitators to engaging with HEIs. More specifically, this research focuses on the engagement activities implemented by employers to develop employability skills and explores the factors that facilitate or hinder such engagement. These issues are raised by HEIs but also by employers themselves in the literature. This is a timely issue in the context of massification of HE and the skill shortages and gaps reported by employers worldwide; however scientific research is still scarce and geographically limited.

Graduate employability has become one of HE's institutional missions and it has been pursued through different means both inside and outside the classroom [9,10]. HEIs have been pressured to provide graduates with skills that match the economic and labor market skill needs and foster their employability [11,12] in a context of rising graduate unemployment levels and persistent skill problems and gaps reported by employers in different settings. The employers' engagement with HE in education and training has been promoted as a vehicle to bridge the divide between the supply and demand of higher-level

**Citation:** Suleman, F.; Videira, P.; Araújo, E. Higher Education and Employability Skills: Barriers and Facilitators of Employer Engagement at Local Level. *Educ. Sci.* **2021**, *11*, 51. https://doi.org/10.3390/ educsci11020051

Academic Editor: Eleanor Dommett Received: 30 December 2020 Accepted: 27 January 2021 Published: 30 January 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

skills [13,14], foster graduates' employability, and reduce skill mismatch, thus providing employers with a timely supply of required skills in the economy.

However, in spite of the growing interest in both graduate employability and the engagement between employers and HEIs in the development of skills and training, relatively little academic research has been undertaken on the types of engagement activities developed or the factors that facilitate or hinder them [13–15]. In addition, the literature focuses on few national cases, such as the UK and Australia; research from other European countries is remarkably scarce. Furthermore, the perception of employers deserves proper scrutiny since it provides HEIs with information on the barriers and facilitators to developing engagement. This paper contributes to this literature by attempting to answer the following questions: In which activities do employers engage with HE? What are the major obstacles and facilitators of this engagement?

The empirical analysis draws on qualitative data collected in 2019 through interviews with a set of employers (*n* = 19) from a city that has developed a multiple stakeholder partnership aimed at connecting different local stakeholders to promote employability, entrepreneurship, and innovation in the region. We have therefore studied employer engagement at a local/regional level in order to ascertain the extent to which this type of partnership fosters closer ties between HE and employers. This qualitative material was examined through content analysis which allowed us to categorize the skill problems faced by these employers at the graduate level, the engagement activities which they develop with HE, and the drivers and barriers to those engagement activities.

The rest of the paper is organized as follows. The next section provides an overview of the literature on employers' engagement with HE in education and training and the role of HE in developing employability skills, a brief overview of the national and regional contexts regarding HE as well as of the local partnership implemented by the city council. Section 3 is devoted to the methodology before reporting the empirical findings in Section 4. Finally, Section 5 provides a discussion of those findings as well as some policy implications.

#### **2. Theoretical Background**

#### *2.1. Tackling Skill Problems through Employers' Engagement with Higher Education*

The cooperation between higher education institutions (HEIs) and firms has grown significantly in recent times [1,4] fueled by increasing global competition, changes in modes of knowledge production and innovation as well as in the roles attributed to higher education (HE) by policy-makers and societies. This collaboration is expected to provide benefits to different stakeholders [16] by increasing firms' innovation and competitiveness in knowledge economies [5], improving graduates' employability [8], and providing new streams of financing to HEIs in the face of decreasing budgets [17]. Academic interest in university–business collaboration has thus increased dramatically [1] along with the recognition of HEIs as an essential source of technology, innovation, and human capital [4] in the triple-helix model of knowledge production [2] and its subsequent theoretical revisions [3].

However, academic interest has focused mostly on R&D and related activities [1], while collaboration with the development of skills has received less attention [4,6]. The literature on employer engagement attempts to fill this gap but it is mostly limited to a set of countries. Although people may interpret employer engagement with HE in training and education in different ways [18], it essentially comprises responses that help the upskilling of workers or the development of tools that enhance employability [19]. It is reported in the literature that employability is often assumed to be synonymous with work-readiness [12] and HEIs are trying to involve employers to prepare graduates for the world of work. The move towards making employers insiders and key actors in transforming employability into graduate employment [20] has only emerged since the 1990s and it aims to both improve graduates' employability prospects and provide a better response to economic and labor market imperatives [9] by promoting closer ties between the supply of skills and labor market demand.

Employer engagement may be enacted in different forms and through various activities, notably information exchange, internships and using HE as a recruitment channel, participation in job fairs, assessment of the quality of graduates, training opportunities, advice on curriculum and courses, co-design of certain courses, participating in the governance bodies of HEIs and co-funding as a partner.

Some authors further distinguish between "active" and "passive" forms of engagement [13]. A more "passive" engagement involves a simple market transaction and focuses on the acquisition of HEIs' products and services. Examples of the latter may be found in information exchange activities where employers turn to HE to provide information to access the best graduates [21], participation in job fairs [22], or through graduate recruitment [18]. In the context of information exchangers and recruiters, employers are viewed as passive stakeholders that provide information about skills and take advantage of channels to access the most skilled candidates.

Other initiatives imply a more "active engagement", which involves collaboration and the definition of what is being delivered by higher education [13] both at the undergraduate and graduate levels [23] as well as in lifelong learning [4]. A more active engagement allows firms to have far greater influence on the supply of skills and thus contributes to reducing skill problems. Active forms of engagement include, for example, work-based learning, which is often incorporated in the HE curriculum [14,24]. Work-based learning is essentially a partnership between employers and HE institutions to deliver courses and provide students with work experience opportunities [25] and has been found to provide all stakeholders with the most potential benefits [24].

The design, delivery, and assessment of tailored courses [26] is a further example of active engagement. Ref. [27] (p. 2010) label it an "anticipative strategy", which indicates that employers participate in the supply of skills; they work with universities to conceive courses and expect HEIs to provide them with bespoke graduates. Finally, some employers participate in one or more of the governing bodies of HEIs. Once again, this helps employers influence the supply of skills as advisers [28], but according to ref. [29] (p. 2007), it should also involve their financial contribution. Co-funding aims to reduce public expenditure and fosters higher education expansion. However, [10] found evidence of employers' persistent unwillingness to make financial contributions to the development of higher education programs.

#### *2.2. Barriers to Engagement: Higher Education Institutions and Employers*

Another stream of the literature examines the employers' engagement with higher education in order to make this interaction more efficient with a win-win outcome. Under the label of barriers and facilitators [14,15,25], this stream of literature focuses on issues that stakeholders from HE and employers raise in order to facilitate the engagement activities and outcomes.

Available literature provides a set of cultural and structural barriers faced by both HEIs and employers [13,18,30]. Barriers within HE refer to the cultural mismatch that often reduces the willingness to engage with employers to ensure the supply of suitable skills. It is said that HE has a certain disdain for business-like activities and places a strong focus on academic activities. Communication difficulties are another widely reported as employers and HE lack a common language.

Some barriers seem to be structural because they relate to the HE system, notably: the disciplinary-based curriculum; the adaptation of pedagogical methods; lack of flexibility to provide tailor-made courses or curriculum; timing of the response to requirements; or poor customer services. In sum, higher education's response needs to meet, rather than conflict with, the employers' and learners' needs in order to obtain a strategic fit.

Additionally, studies have cited the need for an appropriate learning package, that is, higher education should build or adapt training to better fit the employers' specific needs. This often involves bespoke courses and of course rejects the one-size-fits-all programs. However, HE often has insufficient information about employers' skill needs and engagement involves a trade-off between costs. Ref. [18] note that engagement is costly and risky for HEIs so they may prefer to look for alternatives or prioritize other activities. Moreover, academics' engagement activities are currently not generally acknowledged in career progression within academia [4] and thus often result from sporadic and informal contacts [31] that are not institutionalized.

Employers also impose barriers to engagement. Firstly, the nature and intensity of employers' engagement with HEI vary, notably in line with their workforce composition and specific skill needs, their product or service, and the market competition [13]. Secondly, it is necessary to have a culture of learning and perceive engagement as an investment; whereas the business strategy is often to pursue immediate profit, the benefits of engagement take time to become evident [18]. Thirdly, not only do employers find it difficult to enumerate skill requirements but these may change over time [9], and their main demand is for soft skills and sound work attitudes [23]. Finally, engagement is easier when employers need graduates from certain fields of education, notably engineering [32], chemistry, or health [13].

The reported barriers indicate that a cultural fit between organizations is required from the outset, and it not only calls for changes in work practices and mindsets on both sides but also a share of values. Ref. [19] (p. 2013) highlights the tension between entrepreneurial and academic languages and this creates barriers to collaboration. The literature therefore also examines what facilitates engagement.

#### *2.3. Facilitators of Engagement: Higher Education Institutions and Employers*

The HEIs are faced with two interrelated issues. On the one hand, they must proactively encourage engagement by approaching employers, involving them in learning, and providing information about the benefits [33]. It is essential that HEI and the employer develop proper communication and a shared understanding [18]. On the other hand, the system itself needs to be adapted. Ref. [25] (p. 2015) stress that staff from all levels must be enthusiastic about the engagement, in particular in the case of workplace learning.

Employers have a selective approach to HEIs and trust is one of the major issues. For example, the willingness to invest in skill development depends on the employer's knowledge about the programs supplied by the HEI [30]. Alumni can also be used to build a bridge with the HEI to ensure that collaboration is beneficial. The literature documents the fact that the gratitude towards the academy felt by alumni impacts their willingness to engage [34]. Employers may also develop collaboration with individual academics and follow them across HEIs [13]; that is, they mostly interact informally and this means alumni play a decisive role.

On the other hand, employers prioritize the geographical proximity of HEIs [35]. Geographical and social proximity between HE and firms may help in informal relations and information collection [13]. Furthermore, ref. [15] note that engagement is facilitated by the ability to sustain the partnership. This entails equality among partners, but the complexity of this varies in line with the number of members involved; small partnerships are easier to manage, while larger ones require more clearly defined roles.

Employers' engagement with HE is therefore far from straightforward [19] and involves a continuum that goes from an understanding of specific employers' skill requirements as well as the different levels of skills of students and employees to the implementation of mechanisms to develop such skills. All this process requires a culture of trust and commitment to the defined goals; engagement of skilled staff to interact with employers; flexible systems and working methods adapted to the diverse workforce. Only when these conditions are fulfilled do they work as facilitators; otherwise, they function as barriers and invalidate or contribute to making the engagement inefficient. Ultimately, both the decision taken by employers to meaningfully engage with HE and the potential results of that engagement will depend on their assessment of the perceived costs and benefits of that engagement [13]. Some argue that engagement should involve public investment [36], while others suggest there should be multiple funding sources as they facilitate engagement and acknowledge the relevance of the activities [37].

In sum, in spite of the growing interest shown by different stakeholders in graduates' employability and particularly in the collaboration between employers and HEIs in actual skill development and training, relatively little academic research has been conducted on the types of engagement activities developed, the factors that facilitate or hinder those activities, as well the perceptions of both HE professionals and particularly of employers about these relationships [10,38,39].

A profusion of policy reports have been written at the behest of governments [40] or employers' associations [14] and in some countries, namely the UK and Australia, a number of academic studies have already been made on this subject [41]. However, literature is scarce for other European countries, although some have made huge investments in HE. Furthermore, to the best of our knowledge, there are still no studies in the academic literature on the employers' perceptions of engagement or the factors that facilitate or impede engagement or make it more efficient.

#### *2.4. The National and Regional Contexts*

At the national level, Portugal has made substantial investments and progress in both HE and vocational training in recent decades. There has been a trend of massification in HE since the 1990s, with a sharp increase in enrolment rates from around 157,000 students in 1990 to almost 400,000 in 2019. Over roughly the same period, enrolment rates among 30–34-year-olds have also more than doubled, going from 15.1% in 1992 to 36.2% in 2019 and drawing close to the EU average which stood at 40.3% in 2019 [42]. Other recent trends have seen the diversification and differentiation of the system with the expansion of the private and polytechnic subsystems, and the adoption of the Bologna model in 2006 and of a managerialist reform in 2007 that brought Portuguese HE more in line with the trends in Europe and beyond [43].

Over this period, post-graduate and doctoral enrolment [44], as well as research activities, also increased substantially along with research outputs at the system level [45]. On the other hand, the Portuguese education and training system has been found to be excessively centralized, leaving little room to accommodate regional needs [46,47]. Recent legislation has acknowledged the potential benefits of decentralization and has tried to promote the greater involvement of local and regional authorities and stakeholders in addressing the specific skill needs of sub-national economies. However, up until now, the engagement of these local actors has been mostly limited to advice or consultations initiated by the central government [47].

The relatively sparse and fragmented literature on employer engagement with HE in education and training in Portugal has reported somewhat mixed findings. Ref. [39] (p. 107) acknowledge that the collaboration between employers and HEIs is at an embryonic stage, and this is a pervasive feature. Some authors report examples of a variety of activities of engagement, including the design and delivery of courses and shared governance [28]. Nevertheless, passive forms of engagement, such as hiring candidates or job advertisements [28] and internships [48], are widespread among employers in Portugal.

Given this low level of cooperation, a regional multi-stakeholder partnership, Famalicão Made IN, was formed by the city council of V. N. Famalicão with the aim of connecting different local stakeholders to promote employability, entrepreneurship, and innovation in the region through suitable responses from education and training systems. V. N. Famalicão is in a small but vibrant industry-based region, ranking third in the country's export volume and 2nd in gross added value in manufacturing industries [49]. The main industrial sectors include textiles, metallurgy/machinery and polymers (namely for use as automobile components), and agri-food industries and the unemployment levels are traditionally below the national average. Although the region has a few large industrial firms, namely in the textile and automobile components sectors, the majority are small and medium-sized firms in keeping with the typical profile of firms in Portugal [50].

In the region, there are four HEIs, two universities, one public and other private, and two public polytechnics. The universities are more generalist in what they offer (but with a strong component of engineering degrees) while the polytechnics are more vocational in nature (as is usually the case for polytechnics in the Portuguese system) and with a stronger focus on industry-related degrees. Not surprisingly, these institutions in the more immediate vicinity are the ones with whom the firms in our sample will develop greater ties both for recruitment and R&D purposes. Other institutions in the wider Northern region with whom our firms also develop some ties include also public and private universities and polytechnics.

The primary role of Famalicão Made IN is to build bridges between employers and other actors, such as education and training institutions at all levels, local employment services, municipal and intermunicipal institutions, and ultimately to mediate the relationship between local actors and national policymakers. In the context of employer engagement with HE, this partnership can be viewed as an agent that facilitates the cooperation with firms, especially medium and small enterprises [51].

This program was officially launched in 2013 and was initially focused primarily on VET provision as key to boosting employability and addressing the persistent skill shortages experienced by firms in the regional ecosystem. Over time, it has diversified both its scope of action and the stakeholders engaged in it. The collaboration with local HE is thus more recent and targets joint R&D projects conducted by firms as well as initiatives to reduce skill shortages and gaps.

#### **3. Methods and Data**

The data were gathered in the course of an ongoing project undertaken in Portugal to analyze and reflect on employers' engagement with HEIs as a possible avenue for the reduction of persistent skill mismatches at the graduate level. The paper draws on primary qualitative data gathered from face-to-face semi-structured interviews with human resource managers and owners of 19 industry-based firms located in the northern region of Portugal (county of V.N. de Famalicão). The interviews were conducted by two members of the research team, lasted between 1 and 2 h and were fully transcribed. The questions were intended to draw data on (i) recruitment strategies; (ii) perceptions on the preparation of graduates for the world of work and skill problems faced by these firms; (iii) solutions for skill problems (among which training and recruitment policies, relations established with schools and HE and barriers and facilitators to the engagement with these training organizations); and (iv) characterization data of these firms.

This qualitative material was examined through content analysis in order to gain a better understanding of the actual engagement activities developed by local firms with HEIs and the major obstacles and facilitators of those relations. Following the literature review and content analysis of the interviews, we defined several categories for each analytical dimension: skill problems (shortage and gaps); engagement activities regarding the acquisition, training, and assessment of skills, as well as R&D activities; barriers to that engagement, namely cultural/organizational barriers, and facilitators related to personal/social, geographical proximity. A single table containing the selected analytical dimensions was then produced and all the relevant excerpts from the interviews were coded into those categories. This has allowed a more systematic and comparable overview of the analytical data from the interviews. Smaller excerpts were selected for use throughout the paper to illustrate the employers' perceptions of the subjects under analysis.

The firms were selected by convenience sampling and the sample is therefore not statistically representative. However, we have included firms from the most representative sectors (namely textiles, metallurgy, and agri-food) of this strongly industry-based region, with different characteristics in terms of size and years of activity (Table 1). However, the sample is somewhat skewed in relation to size as it includes a higher proportion of large firms than is present in the region which, as noted, is comprised largely of small and medium firms. Most of the firms in the sample have been actively recruiting graduates in

the past three years, notably in different engineering fields, ICT professionals, and, in some cases, also from management and product design areas.


**Table 1.** Characterization of firms.

#### **4. Results**

This section will start with a brief overview of the main skill shortages and gaps faced by employers in this regional setting. As employers' engagement with HE does not take place in a vacuum, the forms and intensity of these activities are likely to be influenced by either the skill problems at the graduate level or the perceived costs and benefits associated with that collaboration. We will therefore delve deeper into the actual engagement activities developed with HE before examining the perceived barriers and facilitators to that engagement in the next section.

#### *4.1. Employers Engagement with HE at the Regional Level*

Skill problems include skill shortages, translated into hard-to-fill vacancies, and skill deficits or gaps which relate to the employers' perceptions of graduates' level of preparation for the world-of-work. These gaps were further typified under the categories of soft skills, work attitudes/maturity, and technical skills.

Regarding skill shortages and hard-to-fill vacancies, a non-negligible proportion of the employers (8 of 19) refer to difficulties in finding and retaining graduate employees in the region, giving rise to constraints in their activity. Most of these vacancies are found in technical areas such as Engineering (F6; F15; F18; F1; F11), ICT (F6; F10; F12), and Physics (F15). Three sets of factors are reported by the sampled employers to explain skill shortages. First, they blame HEIs for a skill shortage that causes an undersupply of graduates in these areas, despite the massification of HE. Second, the good labor market conditions, notably low level of unemployment and the greater bargaining power of graduates, which increase the labor costs: "*it's a region of full employment [* . . . *] in all areas of expertise it is now very difficult and it's the candidates who choose where they want to go and what they want to do*." (F16); "*I would say in engineering nowadays you are only unemployed if you choose to be (* . . . *) people are much more selective today*" (F15). Finally, the brain drain of young qualified people is a problem that affects the sampled firms. The demand for talented youngsters from large neighboring cities and the international labor market makes it more difficult for employers from the relatively small town where these firms are based to attract and retain a skilled workforce.

In terms of graduates' preparation for the world-of-work, there is widespread consensus that HE graduates are well endowed with technical skills, "*technical skills are excellent*" (F8), and "*nowadays students have a far higher skill level than when I finished university*" (F6). However, employers claim graduates are poorly prepared in soft skills and work attitudes (12/19). They report graduates often lack soft skills such as written and oral communication, problem-solving, and transversal skills in general: "*In terms of technical skills, they are better prepared but what they lack is some behavioral and social skills*" (F6); "*I feel that teamwork is an issue as is communication; the ability to communicate both orally and in writing is definitely a problem*". (F13); "*Nowadays, we give as much value to behavioral skills as we do to technical skills, something which did not happen a few years ago*" (F3).

The work attitudes, commitment, and behavioral skills of recent graduates are a major concern (15/19). These characteristics are highly valued by the firms and are widely regarded as instrumental for the use of other technical skills to the benefit of the organization (F4; F11; F3, F2). Thus, HEIs are often blamed for not sufficiently addressing soft skills and attitudes, alongside the technical skills (F6; F1): "*They [graduates] have difficulties in interaction and communication (* . . . *) but I don't see the universities addressing this issue of soft skills*" (F1).

The next question is the employers' willingness and/or ability to engage with HE to tackle skill problems. Table 2 displays the engagement activities of the sampled employers, as well as the barriers and facilitators of this engagement. As can be seen, HEIs serve as a recruitment channel to access the best candidates and engage in the training of graduates through internships. All firms develop at least one of these types of interaction and a non-negligible number (9/19) engages in three. Firms sometimes also participate in job fairs and events at the university (F16; F5).


**Table 2.** Firms' engagement activities with HE and barriers/facilitators to engagement.

The internship programs are regarded as an opportunity to provide some real-world training to students while allowing for candidates to be screened to create a pool of talent. The internships target both graduates (masters and PhD) and non-graduates and occur throughout the year or in the summer. Internships also result sometimes from students visiting the firm (F16). F4 regrets that some students have never had any contact with an organization and, consequently, are unaware of the world of work. These internship programs are often implemented through formal agreements between firms and HEIs and are reported as beneficial by the sampled employers:

"*That has been the way recently, we establish protocols with the universities in the region to receive interns and then recruitment comes from those visits (* . . . *) the truth is that they are very receptive to what we have to say just as we are very receptive to what they have to show us.*" (F16);

"*Yes, we work with different internship programs and we have our own program too. We like to receive those interns, especially because in nine months or so they will enter the labor market. And if we can have this contact before and the student can also get to know the reality of a company, it can facilitate their choice afterwards*" (F1);

"*We have some people coming here for internships. Some come to do their Master thesis. They usually have to face a problem and then come up with a solution. (* . . . *) It is good, it is a way of evaluating people and knowing if they are good enough to stay.*" (F18).

Activities such as teaching, collaboration in course design, or participating in HE governance bodies are scarcely reported by employers. However, they intend to engage more actively in these activities and some firms, namely medium and larger firms, are sometimes invited for this kind of collaboration. In such cases, these types of firm are consulted in the design and demanded an assessment of the course: "*Well, I know that one of our engineers here participates in one of these [governance] bodies (* . . . *) sometimes they ask to use some of our machines, they have visits here or we donate some equipment. I know they sometimes discuss some new degrees, pedagogic content, what makes more sense or not*" (F19). Nevertheless, employers call on the expertise of HE institutions if they are unable to properly prepare graduates: "*We have people with a lot of know-how that want to pass on that knowledge [to graduates] but they don't know how to do it (* . . . *) so what we did was a partnership with the* . . . *Business School*" (F17).

However, this is a random activity, often resulting from the employers' interpersonal relations with HE professionals, so it is far from a systematic and internalized process. Overall, the sampled employers indicate that there is room to deepen the engagement and they are willing to do so. Nevertheless, as we will see later, there are barriers that hamper closer ties.

The collaboration in R&D activities attracts wide attention and prevails in 8/19 firms. It should be noted that the firms in the sample are industry-based and need to develop new products and technologies, which entails close interaction with HEIs. At the same time, firms and HEIs have strong incentives and a long history of cooperation in R&D: "*Nowadays we have two R&D projects in cooperation as well as several in the 2020 [program] in the areas of innovation and markets, especially because we have opened up to cooperation with universities. Beforehand secrecy was the soul of business but not anymore* . . . " (F7). "*In R&D we have seven people, three of whom are doctorates (* . . . *) we will always have to resort to research centers because they have other skills that we don't have*." (F3). The engagement not only involves R&D but also post-graduate training and sometimes the recruitment of master or PhD graduates that participate in the development of products and technologies. A sectoral pattern emerges in the collaboration with HE in R&D in that firms from the agri-food and textile sectors seem to be more engaged in collaborative R&D than those from other sectors, namely metallurgy, machinery, and components for the automobile industry. One possible explanation is that the textile industry in the region has undergone a strong technological upgrade in recent decades that has been widely recognized, and therefore resorts more to this sort of collaboration. However, these results should be treated with caution because other industries, namely those related to the automobile industry, may be more prone to producing in-house R&D (or in collaboration with other firms in the sector).

In sum, the reported engagement activities show that employers are aware of the relevance of HEIs as skill suppliers and often contact them to acquire talented people. However, they are still far from participating systematically in the skill formation process, although some are trying to do so. As noted, the sampled employers are available and intend to extend their engagement, so it should be possible to make progress in the future. Currently, R&D continues to be the major activity of contact between employers and HEIs.

#### *4.2. Barriers and Facilitators to Employers' Engagement with HE*

The sampled employers reported a set of barriers and facilitators of engagement. More specifically, cultural differences that prevent fruitful communication between HEIs and firms are perceived as the major issue. HEIs are generally accused of being distant from firms (F16; F6; F8; F7; F18; F4; F11, F2) and from the world-of-work. "*The world is moving at one pace and universities are another*" (F6); "*I believe the universities are still distant (* . . . *) the students should be put in contact with firms sooner in their university trajectory as they are in programs abroad where the connections with firms start in the first year of college*" (F8). HEIs continue to be focused on academic activities and disregard the requirements of employers. According to these employers, it is the HEIs that fail in setting shared goals and a common language that would increase proximity and develop ties: "*We need a stronger connection with reality (* . . . *) and that does not happen. It does not happen on either side because firms try to get closer and then often lose their patience because of bureaucracies (* . . . *) and universities often use firms just for statistics so that they can say they have links with firms and present those numbers*" (F2). This detachment exacerbates the mismatch between the skills acquired at HE and those required by employers (F8; F18; F11; F2). Two employers (F3; F11) highlighted their willingness to collaborate with vocational schools. They noted the benefits of working with vocational training institutions rather than with universities or polytechnics, notably the flexibility to adapt the curriculum and pedagogical methods, and teachers' knowledge of skill requirements that help a better match of the skill supply.

Some employers recognize that some steps have been taken in recent years to overcome organizational barriers (F16; F15; F18; F19), and some HEIs try to be more responsive to firms' needs and engage more actively with them. However, for these firms, HEIs have not yet provided an adequate response to employers' apparent willingness to engage more actively with HE, and the costs of these engagement activities incurred by firms still exceed the potential benefits: "*The relations have improved, I'm not going to say they haven't, but they still need to be closer*" (F16); "*I believe relations are increasingly better and I see HEIs making strides to come and ask the firms, something which didn't happen before (* . . . *)* [HEIs] *are proactively trying to get closer to firms*" (F15); However, the trade-off between costs and benefits is at the heart of the discussion. "*We must also see what is the economic benefit for us (* . . . *) generally the end result is more of a burden and a loss for our activity*" (F19). Employers suggest engagement is a risky activity that has uncertain benefits.

The barriers are not limited to HEIs with some firms (F11; F2) referring that they refrain from establishing closer links to HE. Others acknowledge that their willingness to engage is often lessened by the above-mentioned barriers and they must take initiatives to improve the ties with HE. Faced with the trade-off between costs and benefits, the sampled employers propose solutions to increase effective engagement.

The sampled employers enumerate some facilitators: They believe that personal contacts, notably through alumni and teachers, are efficient ways of finding appropriate partners inside the HEIs and conducting research projects and/or finding talented candidates: "*We are not approached. Fortunately, we have a relationship (* . . . *) with universities and polytechnics, with teachers and alumni that allows us to implement our normal* [hiring] *processes. What we feel is that without these actions we would not be contacted either by universities or polytechnics*" (F11). However, F3 reports that "*Our relations with HEIs happen in two ways: either we proactively contact them because we have a specific need and then relations are established, or there is someone here at the firm that has a good relationship with someone at the university and then the collaboration follows through that different channel*." In other words, personal contacts appear to be a facilitator for closer ties between HEIs and the world of work.

Geographical proximity is an additional facilitator reported by the sampled employers. Regional universities easily create networks with local employers since they have a deeper knowledge of the region and employers' needs. Consequently, employers refer to local interactions rather than national and distant partners (F16; F6; F15; F7; F10; F1; F12; F3). On the one hand, geographical proximity eases face-to-face interactions and access to information about institutions and people able and willing to establish partnerships. On

the other hand, it helps create a pool of talents of young graduates in HEIs closer to their homes and searching for job opportunities in the local or regional labor market. Some employers speak about these advantages:

"*We work a lot with the [local] University (..) we give priority to the [local] University because of a partnership we established several years ago, because of geographic proximity (* . . . *) we have a lot of people here from that university. We have a good relationship with the presidency and the vice-presidency and great proximity also with the school of engineering" (F15). "We work with several institutions on account of proximity*, [local] *University, the Polytechnic* . . . [all in the Northern region of Portugal]" (F1). However, in addition to fruitful experiences, some underline the specificity of education programs and technological specificities of certain HEIs (F13; F18). Others are "*available to collaborate with any HEI as long as the attitude is appropriate*" (F4; F7; F8)

We now turn to the relationship between engagement activities and the barriers or facilitators reported by the sampled employers. One group is made up of firms that use HEIs as a recruitment channel but acknowledge that cultural barriers probably prevent other types of engagement (F8, F1, F2). Others follow the same strategy but take advantage of personal contacts and geographical proximity to overcome such barriers (F16, F6); use only of the proximity to overcome them (F7, F1, F4); or use only interpersonal contacts (F11). In other words, barriers are compensated by some facilitators, especially to allow firms to access talented graduates.

It is interesting to note that F19 is actively engaged with HEI and this was facilitated by personal contacts; however, he/she refers to skill shortages and admits that young graduates lack work attitudes and maturity. Furthermore, firms must also tackle cultural barriers to develop collaboration in R&D. This is given as the major factor inhibiting stronger university-business collaboration.

Finally, regarding the role of Famalicão Made IN, the local partnership, many employers recognize that it has already positively impacted R&D collaboration between HE and firms, access to funding, the visibility of firms, and the county and it has facilitated access to other local and national institutions and decision-makers. Some of the R&D partnerships and engagement activities have indeed resulted from post-graduate training work and knowledge interactions, and the county is currently trying to expand these projects. However, most employers are also aware that much work is required to address local skill problems and, more importantly, to align the supply of and demand for higher-level skills, as well as to overcome existing barriers between firms and HEIs. It should be noted that Famalicão Made IN has very recently taken steps to work with HE, and it can become an efficient broker to reduce cultural barriers in the future.

#### **5. Discussion and Conclusions**

This research contributes to our understanding of two key but often neglected questions in the literature: Are employers willing and able to actively engage with HEIs to become a viable solution for skill problems at the graduate level? What are the main barriers and facilitators of the engagement activities with HE experienced by firms?

Our results show that the sampled employers use HEIs as a recruitment channel and set aside active forms of engagement that would allow them to directly influence the supply of skills. These results are in line with the relatively sparse and fragmented literature on employer engagement with HE in education and training in Portugal, for example, [28,34,48]. However, despite the sampled employers' willingness to develop more active forms of engagement, some barriers remain.

Given the data displayed in Table 2, the efficiency of engagement activities must be questioned. We note that F19 laments skill shortages and a lack of graduates despite being engaged in active strategies. These examples confirm that there is no one-size-fitsall solution for skill problems [28]. Active engagement, through shared governance, for example, does not seem to be the answer either.

When examining the employers' perception of the barriers and facilitators of engagement, we found that a cultural mismatch between business strategy and academic language is the major barrier [19]. HE and the sampled firms seem unable to fully communicate through a common language and goals. This probably prevents active engagement and two employers (F3; F11) expressed their preference for vocational education institutions that show greater flexibility and responsiveness than HE. These firms noted structural barriers related to the strategy of the curriculum and pedagogical methods [18] that differentiate HE from other levels of education.

This raises the question of whether HE should change its goals and language so that it can cooperate closely with the world of work? Previous research has shown that the expectations of employers vary, for example, [28] with some employers preferring to hire ready-to-work candidates provided by HEIs, while others deliver training and ensure that newly hired workers are endowed with specific skills. In fact, the training policies of the employers in our sample involved preparing their workforce and using internships as the key to screening and training graduates.

Turning to the facilitators of engagement, the sampled employers emphasized the role of trust in developing sustainable collaboration. Some know which institutions have responses to their technical skills requirements [30]. Although HEIs' proactive approach to employers was noted [33], all employers are also aware of the costs of engagement [10] and acknowledged the trade-off between costs and benefits [13]. This leads us to the discussion of public investment in the engagement activities [36]. We agree that it involves multiple sources [37] and the stakeholders involved must acknowledge the relevance and the economic and social impacts of investing in engagement. Public intervention would probably be translated into regulations that guarantee equity among different types of HEI and different types of employers. The reform of Portuguese HE in 2007 strived to engage external stakeholders at non-executive governance bodies of HEIs. However, to date, their participation is still minimal to non-existent and is sometimes referred to as imaginary or non-interfering friends [52].

Our study found that the sampled employers reported geographical proximity as well as personal and social contacts; this later appeared as a major factor for successful cooperation. It is the alumni that sometimes open the door to HEIs and support collaboration [34]. So, when the sampled employers had suitable partners [15], they were able to access talented graduates. The multi-stakeholder partnership was also seen as a broker to help build bridges between HE and employers [25].

Geographical proximity interacts with this [13] as informal relations in a small region seem to be the perfect recipe for encouraging collaboration and communication. Finally, while the multi-stakeholder partnership attempts to build bridges between HE and the market [51], employers acknowledged that further work is required to improve the partnership's role as a broker to ease cooperation.

The reported facilitators raise trust which may encourage employers to become stakeholders that can help foster graduates' employability and create opportunities for the world of work to be embedded in HE. However, the development of employability skills is a shared responsibility [28] and HE has to fulfill multiple missions and respond to economic as well as social and cultural needs.

Although the findings achieved thus far are interesting, they should be interpreted with caution. This is a case study of one region using a sample of firms solely from industry sectors. Future research should therefore explore the skill problems of other sectors and preferably of other regions. This might be useful to ascertain the national and regional level differences of skill problems, as well as the solutions and the barriers/facilitators to tackle them. Policymakers, HEIs, and employers should understand that multiple solutions are required to reduce skill mismatch, shortages, and gaps, and all stakeholders are responsible for finding appropriate answers. However, each stakeholder has its own culture and language, and efforts should be taken to improve communication and interaction between them.

**Author Contributions:** Conceptualization, F.S. and P.V.; Data curation, E.A.; Methodology, F.S., P.V. and E.A.; Writing—Original draft, F.S. and P.V.; Writing—Review & editing, F.S., P.V. and E.A. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research was funded by Fundação para a Ciência e a Tecnologia: Project BRIGHET 'Bringing Together Higher Education, Training, and Job Quality' FCT-PTDC/SOC-SOC/30016/2017.

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Acknowledgments:** We would like to thank the employers, local stakeholders and members of Famalicão Made IN for sharing their invaluable time, experience and materials without which we would not have been able to conduct this research.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**


### *Article* **Teaching Sentiment in Emergency Online Learning—A Conceptual Model**

**Domingos Martinho 1,\* , Pedro Sobreiro <sup>2</sup> and Ricardo Vardasca <sup>1</sup>**


**Abstract:** Due to the COVID-19 pandemic, higher education institutions with a face-to-face model have found themselves in the contingency of migrating to online learning. This study explores the perspective of all the lecturers at a Portuguese private higher education institution who were invited to participate, regardless of their research area, in this questionnaire. It aims to propose and test a conceptual model that combines attitudes, preferred activities, and technological experience with the sentiment about the impact of this experience on students' learning process, on their teaching activity, and on the strategy of higher education institutions. An online questionnaire was conducted to 65 lecturers engaging in emergency online lecturing. The obtained results showed that lecturers reveal a positive attitude towards online lecturing, tend to prefer activities in which they feel most comfortable in face-to-face lecturing, and consider having technological experience useful for online activities. Lecturers have a positive sentiment about the impact of online learning on students' learning, their faculty career, and the strategy of higher education institutions. The proposed conceptual model test shows that the model has well-fitting conditions. The results confirm the hypotheses formulated: namely, the predictive effect of attitude, preferred activities, and technological experience on sentiment. Faculty engagement in emergency online lecturing shows that the members are available to participate in the changing process, and the proposed conceptual model can be used to assess this readiness.

**Keywords:** COVID-19; emergency online learning; emergency online teaching; higher education; lecturers; online learning; Portugal; sentiment analysis

#### **1. Introduction**

The COVID-19 pandemic has affected higher education institutions (HEIs) in their activities in order to promote the protection of their lecturers, staff, and students in a public health emergency. The institutions had no alternative but to cancel all face-to-face lectures, including labs and other learning experiences, and to determine that lecturers completely switch the courses to emergency online learning, reducing contacts and thereby preventing the spread of the virus.

This teaching model that many call "emergency remote teaching" [1], includes the use of totally remote teaching solutions, mediated by the internet, to ensure activities that would otherwise be taught in a face-to-face form, returning to this format once the crisis or emergency is overcome [1]. The followed model seems similar to the online learning that has been stated by Anderson [2], referring to a teaching and learning type in which: (1) the student and the lecturer are at physical distance; (2) student–content, student–lecturer and student–student interactions are mediated by technology; and (3) some type of support is provided [2].

In the COVID-19 context, higher education lecturers were challenged by the need for the adoption of online learning practices, for which the majority were not prepared [3], and

**Citation:** Martinho, D.; Sobreiro, P.; Vardasca, R. Teaching Sentiment in Emergency Online Learning—A Conceptual Model. *Educ. Sci.* **2021**, *11*, 53. https://doi.org/10.3390/ educsci11020053

Academic Editor: Han Reichgelt Received: 13 January 2021 Accepted: 27 January 2021 Published: 30 January 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

there were no indications that they were interested in using it [4]. The faculty members had to prepare and teach their lectures from home, with all the practical and technical challenges that this entails, and often without adequate technical support [1]. In addition to the lack of required online specific pedagogical competences, it is generally agreed that in a normal situation, the challenge to effectively transfer what is taught in a face-to-face classroom to an online version remains a problem [3]. Most of these lecturers, who normally develop their activities face-to-face, do not reveal an interest in online learning (only about 30% to 35% consider this option) [4,5]. This position is caused by the lack of motivation and incentives resulting from various obstacles that can be summarized as technological readiness [6–8], absence of organizational incentive to compensate for extra work [9,10], and the prejudices related to the value of online teaching [5,11,12].

In a normal situation, the most relevant motivations for adopting online learning are related to the concern of reaching new audiences, diversifying the HEI's offer, and contributing to the management of organizational change and the positioning of the HEI´s offer in the context of online education [9,11–13].

In the emergency caused by COVID-19, lecturers needed, overnight, to use tools with which they felt comfortable [14]. Face-to-face lecturers thus needed to develop online teaching activities in order to avoid the collapse of the teaching and learning process. In this situation, lecturers adopted emergency remote teaching that. as stated by Hodges [1] (p. 6), "is a temporary shift of instructional delivery to an alternate delivery mode due to crisis circumstances". Emergency online teaching is different from all other situations in which online teaching and learning activities are planned by lecturers who have online teaching skills. For many of these lecturers with little or no experience in online teaching, the option was to transport the typical activities they developed for face-to-face teaching to the online environment and, gradually, introduce activities that would allow more meaningful learning [15]. Despite the skills and support limitations, lecturers have a positive sentiment about emergency online learning [16,17].

The present investigation focuses on the motivations of lecturers with no, or little, experience in online teaching. Without any other option, these lecturers were required to adopt emergency online teaching. In order to address this great challenge, lecturers changed their attitude towards online education, their favorite activities, and technological experience. This study aims to investigate whether these feelings and skills affect online teaching sentiment. It aims to understand how lecturers perceive the impact of this experience on students' learning, on their teaching activity, and in the development of HEI online learning strategy.

The document is organized into six sections: the present section, which introduces the research topic, the motivation, and the aim; the following section, which presents the conceptual model and hypothesis for the research; the methodology is then described, followed by the sections of the obtained results, its discussion, and final remarks in the conclusions.

#### **2. Conceptual Model and Hypotheses**

From the existing literature, several theories and models have emerged that have in common the objective of explaining the intention to use technologies through the relationship between latent, including external and outcome, variables [18,19] Although these models have been developed with the aim of explaining and predicting the acceptance of computer technologies in general, they have been adapted with a view for their application in more specific contexts, such as online teaching and learning [20,21].

Contrary to previous studies, this study is based on the migration from face-to-face to emergency online education. It was carried out without the lecturers involved having had any opportunity to carry out any type of training, and these had only minimal support. They were limited to providing access to the platforms and technologies used. For this study, a conceptual model is proposed that combines factors that can be measured when face-to-face lecturers have transferred their activities to emergency online learning, namely:

(1) online teaching attitude (OTA); (2) preferred online activities (POA); (3) technological experience (TEX); and (4) online teaching sentiment (OTS).

#### *2.1. Online Teaching Attitude (OTA)*

The attitude towards online teaching and learning is identical to that shown in other pre-pandemic studies [18–21]. It consists of appraising individuals' positive or negative feelings (evaluative affect) about the use of online education [9,18,19]. The following two hypotheses are proposed:

**Hypothesis 1 (H1).** *OTA positively affects OTS.*

**Hypothesis 2 (H2).** *OTA positively affects POA.*

#### *2.2. Preferred Online Activities (POA)*

The activities proposed by lecturers in emergency online learning, with which most did not have previous experience, ended up following those recommended in the existing literature. They choose to diversify the activities and the materials used, thus seeking to correspond to the different student learning profiles [22,23]. The activities preferred by lecturers when migrating activities to emergency online teaching can be compared with the concept of self-efficacy. According Jo et al. [24] (p. 50), "self-efficacy reports to lecturers' personal beliefs about their abilities and skills". It seems normal that lecturers prefer the activities in which they feel more qualified and competent. Thus, Hypothesis 3 (H3) is suggested: POA positively affects OTS.

#### *2.3. Technological Experience (TEX)*

Technological experience identifies the degree of technological readiness [25] of the lecturers from their perspective [26]. As mentioned by Abdullah and Ward [27], experience plays an important role in the adoption of online education and can be defined as "the amount and type of computer skills acquired by a person over time" [27] (p. 34). For Joo et al. [24], "it is important for lecturers to have enough time and opportunities to practice new technologies until they feel comfortable enough to use the technology and perceive that technology". In a context in which lecturers did not have that time, technological experience seems to be an important factor that can influence online teaching sentiment [28]. The following three hypotheses are proposed:

**Hypothesis 4 (H4).** *TEX positively affects OTS.*

**Hypothesis 5 (H5).** *TEX positively affects OTA.*

#### **Hypothesis 6 (H6).** *TEX positively affects POA.*

#### *2.4. Online Teaching Sentiment (OTS)*

According to Liu [29] (p.15), "sentiment is the underlying feeling, attitude, evaluation, or emotion associated with an opinion", which is represented by three aspects: the type, orientation, and intensity of the sentiment. In the context of this work, the lexicon-based approach that involves calculating the orientation of feeling from the semantic orientation of words or phrases was used. The orientation of the sentiment can be positive, neutral, or negative. Neutral means the absence of sentiment or no sentiment or opinion [29,30]. Sentiment intensity is an important aspect for the classification of the feeling associated with a sentence [31]. For example, "good is weaker than excellent, and dislike is weaker than detest" [29] (p. 16).

Sentiment analysis is studied in many different contexts, with machine learning and natural language processing being the most common techniques [32]. In the current research, sentiment analysis was based on processing natural language and extracting information that examine phrases and assign to each one of them a sentiment polarity (positive, negative, neutral) [29,33]. By this way, the opinions expressed by lecturers in relation to the impact of the online emergency teaching was assessed in three aspects: (1) impact on students' learning; (2) impact on their future teaching activity; (3) impact on the future HEI online learning strategy.

Based on the previous theoretical variables, the conceptual model with the relationships between all the factors that influence OTS is presented in Figure 1.

**Figure 1.** Conceptual model.

#### **3. Methodology**

#### *3.1. Participants*

The participants (*n* = 65) were lecturers form a Portuguese private HEI. This HEI has a total of 98 lecturers that were invited to participate in the questionnaire. The link to questionnaire was sent to everyone through e-mail message, along with an introduction about the research objectives.

#### *3.2. Data Colletion*

The data were collected through online surveys from April to May 2020. The aggregated response rate was 79%, and the final sample consisted of 66% of the reference population. From 98 potential respondents, 78 questionnaires were answered by respondents, of which 13 were rejected because of missing values.

#### *3.3. Lecturers' Personal Information/Demographic Data*

In the total of sample of lecturers, the percentage of females was 40%, while that of males was 60%. A total of 1.5% of lecturers were up to 29 years of age, 13.8% from 30 to 39 years of age, 46.2% between 40 to 49 years of age, 21.5% between 50 to 59 years of age, and 16.9% were 60 years of age or older. In terms of the academic qualifications of the lecturers, 23.1% of participants held bachelor's degrees, 33.8% held master's degrees, and 43.1% held a doctoral degree. The teaching experience shows that 18.5% had up to 4 years, 20% had from 5 to 9 years, 24.6% had 10 to 19 years of experience, and 36.9% disclosed 20 or more years of experience.

#### *3.4. Survey Instrument and Structure*

The questionnaire consisted of six sections. The first section intended to characterize the respondents. In the second section, respondents were asked about their attitude toward online teaching and learning with a 5-point Likert scale (1—lower; 2—sometimes lower; 3—no significant differences; 4—sometimes higher; 5—higher). The third section was to evaluate the degree of preference/satisfaction with the online activities. A 10-point end defined scale with ratings from null (1) to high (10) was chosen, in order to produce increased sensitivity of the measurement instrument [34]. In the fourth section, respondents were asked to self-assess their technology skills. A 4-point Likert scale was adopted (1—none; 2—up to 3 years; 3—from 3 to 6 years; 4—more than 6 years).

The fifth section of the questionnaire survey presents three open questions about the impact of emergency online teaching and learning in the present and in the future of (1) student´s learning, (2) teaching activities, and (3) online learning and teaching in HEI strategy. These questions are intended to collect data for sentiment analysis about online learning and teaching. Table 1 presents the constructs of each section and the sources which inspired them.

**Table 1.** Constructs and their inspiration sources.


(\*) Open questions.

#### *3.5. Pilot Study for the Questionnaire*

A pilot study was conducted to check the reliability of the questionnaire items. The sample size was set based on 20% of the aggregated sample size of this study (98 lecturers) and thus adhered strictly to the research criteria. Cronbach's alpha test was utilized for the computation of internal reliability [40] through IBM SPSS Statistics v26, in order to judge the outcomes of the pilot study. A value of 0.7 was taken to be an acceptable value for the reliability coefficient, considering the model for social science research [41–43]. The appropriate findings are shown in Table 2.

**Table 2.** Cronbach´s alpha value for pilot study.


#### *3.6. Sentiment Analysis*

The fifth section of the questionnaire presents three open questions about the impact of emergency online teaching and learning in the future of (1) student´s learning, (2) teaching activities, and (3) online learning and teaching in HEI strategy. These questions are intended to collect data for sentiment analysis towards online learning and teaching.

There are many applications and enhancements on sentiment analysis algorithms that have been proposed in the last few years [33]. For this work the OpLexicon 3.0 was used. It is a sentiment lexicon for the Portuguese language, built using multiple sources of information, and has four categories of words: verbs, adjectives, hashtag, and emoticons. The lexicon is constituted of around 32,000 polarized words classified by their morphological category and annotated with positive (1), negative (−1), and neutral (0) polarities [30,38].

The sentiment analysis was developed in R [44] following the following steps represented in Figure 2: (1) the words are extracted from each answer of the open questions in the questionnaire; (2) verification of whether the word is present in the OpLexicon and determination of the polarity; (3) the sum of the polarity of the word in the answer is determined; and the final step is (4) to convert the sum of polarity to a Likert scale.

**Figure 2.** Flow-chart representing the determination of the polarity of the open questions.

The conversion to a Likert scale was based in the following Algorithm 1, where each answer is processed after the determination of the cut points (median values) used to convert to a scale aligned with the other questions of the survey:


As an example, considering the opinion "I consider that my adaptation was made in a smooth way", the next step is the processing of each word: "I (1) consider (2) that (3) my (4) adaptation (5) was (6) made (7) in (8) a (9) smooth (10) way (11)". To determine the polarity of each word, OpLexicon 3.0 was used. In the example given, only the word "smooth" (10) returns value 1 (positive polarity) from OpLexicon; all the other words do not have an associated polarity, returning "word is not present in dataset". The algebraic sum of the returned values is 1. Consequently, this answer would get a polarity value of 1. After performing this step, an algorithm is developed following the "Likert calculation", calculating the median of the negative and positive words in each question: (1) negative values less or equal to the negative values median were assigned one, (2) negative values less than zero and greater than median were assigned two, (3) 0 (neutral) was assigned three, (4) positive values and less than positive median were assigned four, and positive

values greater than positive mean wrtr assigned five. The null values were replaced by 0 representing the absence of an answer.

#### *3.7. SPSS and SmartPLS 3*

The demographic data was evaluated with the aid of IBM SPSS Statistics v26. Smart-PLS 3 software was used with a graphical user-interface to estimate the PLS-SEM models [45]. This tool can cope with smaller sample size (<100), non-normal data, exploratory research for the same effect size and model complexity, and it can more easily specify formative constructs [46,47].

#### *3.8. Adjustment Quality for the SEM Model*

The following fit measures were considered to assess the adjustment quality of the model:


The values presented in Tables 3–5 show that the model has well-fitting conditions.

**Table 3.** Adjustment quality for the Structural Equation Modeling SEM model.



**Table 4.** Discriminate validity.

Diagonal values (in bold) are Composite reliability (CR).

#### **Table 5.** F-square.


Finally, evaluating the predictive validity or Stone–Geisser indicator for the accuracy of the adjusted model. Q<sup>2</sup> > 0 implies the model has predictive relevance [46,52] (Table 6).

#### **Table 6.** Predictive validity (Q<sup>2</sup> ).


SSO—sum of squares errors using mean for prediction; SSE—sum of squares prediction error.

#### **4. Results**

#### *4.1. Online Teaching Attitude*

The results showed that respondents have a positive attitude towards online teaching. The item "I have the same availability for online as for face-to-face teaching "(OTA3) has an average of 3.71, while the item "quality of online education in relation to face-to-face education" (OTA1) has 3.25, and the item "I like online education in the same way as face-to-face education" (OTA2) has an average of 3.14 (Table 7).

**Table 7.** Online teaching attitude.


#### *4.2. Preferred Online Activities*

Lecturers revealed greater preference for "online sessions" (POA4) with a mean of 8.48, "oral presentations" (POA3) with 7.66, and "written assignments" (POA2) with 7.34 (Table 8).


**Table 8.** Preferred online activities.

(\*) Standard-deviation.

#### *4.3. Technological Experience*

Respondents showed high experience in the use of "online meeting systems" (TEX1), with an average of 3.94. The remaining items evaluated obtained average values above 3.0 (Table 9).

**Table 9.** Technological experience.


(\*) Standard-deviation.

#### *4.4. Sentiment Analysis*

The results of sentiment analysis of the open questions allowed the identification of their sentiment value, as exemplified in Table 10, for impact on lecturers' careers.

**Table 10.** Example of qualitative sentiment for the impact on lecturer´s careers.


\* The answers related to the impact on lecturers' careers were translated to English to allow better comprehension.

The impact of online learning on students' learning has approximately 9 responses with a negative sentiment (14%), as well as a neutral sentiment with 14 answers (21%), and 30 responses with a positive sentiment (46%). The sentiment in relation to teaching activities

has 11 responses with a negative sentiment (17%), 11 answers of neutral sentiment (17%). and 31 positive sentiment responses (47%). In relation to the higher education institution, there are 3 answers with a negative sentiment (5%), 12 responses of neutral sentiment (18%), and 34 positive sentiment responses (52%).

The opinion in relation to the impact of online learning in the institution strategy is the one with a higher percentage of positive sentiment (52%), as opposed to 5% who expressed positive sentiment. The opinion in relation to teaching activities has the highest percentage of positive sentiment (17%) as well as neutral sentiment. The overall sentiment distribution is represented in Figure 3.

**Figure 3.** Frequency of the sentiment identified.

#### *4.5. PLS Analysis*

The path coefficients of the prediction model were positive in POA (0.390), and they were negative in OTA (−0.169) to the latent variable of OTS. TEX coefficients to the prediction model were positives to the latent variables of OTA and POA. These results show that TEX has direct and indirect (via OTA (0.160) and POA (0.427)) effects on OTS.

The model also presented OTS1 (student online learning) (0.772), OST2 (teaching career development) (0.866), and OTS3 (online learning in HEI) (0.867), which had positive path coefficients to OTS (Figure 4).

Specific indirect effects are show in the Table 11.



**Figure 4.** Partial least squares structure model (inner path coefficients and outer weights).

#### **5. Discussion**

A questionnaire was conducted with the participation of 66% (*n* = 65) of all lecturers (98) for a Portuguese private HEI who developed their activities in an emergency online teaching environment. The study examined their attitude toward online teaching, what online activities they most value, and investigated whether technological experience influences these attitude and preferences. The opinions of these lecturers in relation to emergency online teaching, namely their impact on students' learning, their professional development, and the development of HEI strategy was also examined. Finally, a conceptual model was proposed and tested to assess the effect of attitudes, activities, and technological experience on online teaching sentiment. In the following points, the results obtained in relation to the previous literature are discussed.

#### *5.1. Attitude toward Online Teaching*

The results showed that lecturers have a positive attitude towards emergency online teaching, showing an identical availability to face-to-face teaching. This conclusion coincides with other studies conducted in an emergency online teaching that show that lecturers report more on the advantages of distance education [53]. This is reinforced by the results obtained in the analysis of the impact of online teaching sentiment on teaching and students' learning.

Based on this conclusion, at least in an emergency situation, lecturers do not question the value of online teaching. Although this is not the same type of education, these conclusions are more positive than the results obtained in a normal situation when questioning face-to-face lectures about their availability and acceptance of online teaching [5,11].

#### *5.2. Preferred Activities*

The most preferred activities of lecturers ("online sessions", "oral presentations", and "written assignments") confirm the García-Peñalvo et al. study [15] and reveal that lecturers relied on the "tools" they dominated and only later did they begin to use resources more adjusted to online teaching and learning. This strategy is confirmed by Rapanta et al. [14], who state that many non-specialist online lecturers have chosen to focus on materials/resources that they would use anyway to teach the course content, regardless of whether they are face-to-face or online.

Despite the difficulties related to the emergency online teaching that cannot be compared with "normal" online teaching, some of the options found can be problematized. However, as concluded by Spoel et al. [54], there was the attempt to provide students with the basic ingredients for learning (online lectures, group activities, discussion forums, etc.) that reveal concern with diversification, thus seeking to correspond to the different student learning profiles [22,23].

This adaptability seems to confirm Anderson in that "an excellent e-teacher is an excellent teacher" [2] (p.360), possessing pedagogical skills that allow them to understand the teaching process, in order to be able to make the best use of the range of activities they have at their disposal.

#### *5.3. Technological Experience*

Pre-pandemic studies [7,8,10] show that technological readiness can be a factor that conditions the participation of lecturers in online teaching. Although these conclusions cannot be directly transposed to emergency online education, results show that the participants in this study had technological experience in some of the tools for the development of online activities.

#### *5.4. Sentiment Analysis*

Lecturers have a positive or neutral sentiment about the impact of emergency online learning on students' learning. These findings are similar to others, where it was concluded that lecturers expressed a favorable opinion about the students' academic performance during the COVID-19 pandemic outbreak [16,17]. The findings of this study are slightly more positive than the results reported by Tartavulea et al. [55], which concluded that emergency online teaching has an overall moderate positive impact on the educational process, albeit the overall effectiveness of the online educational experience is perceived to be lower than in the case of face-to-face teaching.

Likewise, lecturers expressed a neutral or positive sentiment regarding the impact of emergency online teaching on their professional activity. In addition to showing high availability for online teaching, lecturers do not refer to the eventual need for compensation for the required additional work caused by transposition of face-to-face to emergency online teaching, as studies about online teaching reveal [9].

Lecturers thus seem to prefer to take advantage of the professional development opportunity that the situation offers [4]. These conclusions reveal a positive stance that HEIs that intend to invest in online teaching strategies cannot miss. Studies carried out in a pandemic situation have not focused on this aspect, so it is not possible to make comparisons with similar situations. Despite this, there is pre-pandemic literature that shows that lecturers do not consider online teaching as having a positive impact on their careers [4,13].

The results verified in the sentiment analysis about the impact of emergency online teaching for the future development of the HEI are in line with other studies which were carried out outside the emergency context, and where the contribution to organizational change and positioning of the HEI offer are the aspects most frequently pointed out by lecturers on the adoption online teaching [9,11–13]. The extra time and effort invested by lecturers in emergency online teaching can explain the positive perception regarding the impact on HEI strategy [54].

#### *5.5. Conceptual Model*

The results of the conceptual model test show that the model has well-fitting conditions. In relation to each of the tested hypotheses it is concluded that five of the six hypotheses have been confirmed (Table 12). The obtained values show that the effect of POA and EXT on OTS, and TEX on POA are strong (>0.35), while the effects of OTA and TEX on POA are moderate (>0.15) [56].


**Table 12.** Hypothesis results.

#### *5.6. Limitations*

Some limitations of the present study must be highlighted. First, the study was carried out in an institution with 98 lecturers, of which 66% submitted valid responses (no missing data). The sample size (*n* = 65) represents the HEI population, but with all respondents belonging to a single HEI, the study does not allow generalizing the results for Portuguese HEIs.

Another limitation of the study is the fact that the results are based on the respondents' perceptions, which may cause a bias. Although it was clarified that the survey results would only be used for the purposes of the survey, respondents may be tempted to choose the "correct" answer or the more socially desired answer, thus being vulnerable to distortions [42].

The way in which the transition from face-to-face to the emergency online teaching was carried out may justify why lecturers expressed a greater degree of preference for lectures (online sessions). This preference, by itself, could indicate that they merely transposed the "bad" face-to-face practices to the online environment, namely the face-to-face expository sessions. However, despite this greater preference, there is a significant degree of adherence to other activities, namely oral presentations, written assignments (in group), discussion forums, and chat. The diversity and characteristics of these activities can enhance student–lecturer or student–student interaction, leaving good indications about teaching and learning process [2].

The conditions available were certainly not the same in all institutions, just as they are not the same in the face-to-face context. These differences may have affected, to a greater or lesser extent, the quality of the solutions adopted and should be considered as a moderating factor when extending the study to other HEIs.

#### **6. Conclusions**

After the emergency online teaching experiences related to COVID-19 pandemic situation, lecturers acquired an experience that will mark their teaching life forever. As the storm passes and face-to-face classes are resumed in a normal environment, HEIs can expect less resistance and more enthusiasm for online teaching from their lecturers [3,4]. So that this enthusiasm does not fade away, it will be necessary to support the training of lecturers by providing them with the skills and competences they require to act in the context of online education. Hybrid approaches integrating online teaching with faceto-face activities can represent a significant improvement when many studies reveal that online education constitutes a key factor for the development of HEIs [4].

This work only reflects the perspective of the lecturers. In parallel, another study is being carried out that will reflect the students' perspective and that will allow a comparison between the two perspectives to be established.

Further research, ideally expanding the sample size with participation of lecturers from different HEIs, is required to verify whether the proposed model continues to maintain theoretical validity. In the same way, this extension will allow confirmation of the findings. **Author Contributions:** Conceptualization, D.M. and P.S.; methodology, D.M.; software, D.M. and P.S.; validation, D.M., P.S. and R.V.; formal Analysis, D.M. and P.S.; investigation, D.M. and P.S.; resources, R.V.; data curation, D.M. and P.S.; writing–original draft preparation, D.M., P.S. and R.V.; supervision, D.M.; project administration, R.V. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Institutional Review Board Statement:** The study was conducted according to the guidelines of the Declaration of Helsinki, and has obtained prior approval of the Ethic Committee of the ISLA Santarém (2020-002).

**Informed Consent Statement:** Informed consent was obtained from all subjects involved in the study.

**Data Availability Statement:** The data presented in this study are available on request from the corresponding author. The data are not publicly available.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**


### *Article* **Mechatronics: Experiential Learning and the Stimulation of Thinking Skills**

**Maki K. Habib 1,\* , Fusaomi Nagata <sup>2</sup> and Keigo Watanabe <sup>3</sup>**


**Abstract:** The development of experiential learning methodologies is gaining attention, due to its contributions to enhancing education quality. It focuses on developing competencies, and build-up added values, such as creative and critical thinking skills, with the aim of improving the quality of learning. The interdisciplinary mechatronics field accommodates a coherent interactive concurrent design process that facilitates innovation and develops the desired skills by adopting experiential learning approaches. This educational learning process is motivated by implementation, assessment, and reflections. This requires synergizing cognition, perception, and behavior with experience sharing and evaluation. Furthermore, it is supported by knowledge accumulation. The learning process with active student's engagement (participation and investigation) is integrated with experimental systems that are developed to facilitate experiential learning supported by properly designed lectures, laboratory experiments, and integrated with course projects. This paper aims to enhance education, learning quality, and contribute to the learning process, while stimulating creative and critical thinking skills. The paper has adopted a student-centered learning approach and focuses on developing training tools to improve the hands-on experience and integrate it with project-based learning. The developed experimental systems have their learning indicators where students acquire knowledge and learn the target skills through involvement in the process. This is inspired by collaborative knowledge sharing, brainstorming, and interactive discussions. The learning outcomes from lectures and laboratory experiments are synergized with the project-based learning approach to yield the desired promising results and exhibit the value of learning. The effectiveness of the developed experimental systems along with the adopted project-based learning approach is demonstrated and evaluated during laboratory sessions supporting different courses at Sanyo-Onoda City University, Yamaguchi, Japan, and at the American University in Cairo.

**Keywords:** mechatronics; experiential learning; creative thinking; critical thinking; robotics; thinking skills; subsumption architecture; mobile robot; educational system; project-based learning

### **1. Introduction**

The current era of globalization and the high competition environment has set up a high demand for talent and intelligence requirements. The education quality represents the core elements of success in such competition. Mechatronics classified as a "unifying interdisciplinary and intelligent engineering science paradigm" [1], due to its capability to comprehend and synergize technologies with modern engineering sciences that exhibit quality performance, stimulate creativity and innovations, focus on the interaction of interdisciplinary knowledge, and building up thinking skills through the learning process. Furthermore, it is highly considered as a philosophy and a new way of dynamic work/practices that support new ways of learning to lead to new concurrent design methodologies and innovations [1,2]. Many studies have been conducted to understand

**Citation:** Habib, M.K.; Nagata, F.; Watanabe, K. Mechatronics: Experiential Learning and the Stimulation of Thinking Skills. *Educ. Sci.* **2021**, *11*, 46. https://doi.org/ 10.3390/educsci11020046

Academic Editor: Maria José Sousa Received: 31 December 2020 Accepted: 20 January 2021 Published: 26 January 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

the mechatronics education process and its comprehensive role to establish quality educational requirements and fulfilling future needs creatively. Essential parts of mechatronics education demand to complement theoretical knowledge with practical skills and experience at different levels of the educational process. Such relation aims to develop skills finding creative and innovative solutions, hands-on experience, and practical knowledge, decision-making capabilities at individual and teamwork levels. Hence, among the key features of mechatronics skill-building learning and education is the integration of problem/project-based learning within its structural requirements.

Mechatronics evolved as an engineering science discipline [3] that best reflects interdisciplinary concepts into practice, synergizing knowledge contributed by different disciplines in an innovative way. Mechatronics, with its effective educational process, is contributing to developing capable future creative engineers. Liliana and Florina [4] address the synergy that can be achieved between education, knowledge, and innovation through mechatronics education and its impact on future needs. Acar and Parkin [5] reviewed aspects of mechatronics education and training while comparing the effectiveness of generalist and specialist engineering education.

Researchers published work focused on developing technical tools and systems to enhance experience and skills through training. Akdogan et al. [6] introduced a study tool for teaching and training with a focus on stepper motor as a driving system and its associated techniques using a microcontroller and suitable interface. Camposaragna et al. [7] presented mechatronics educational activity at Bergamo University demonstrated using course projects aiming to teach mechatronics by designing Cartesian plotters. A mechatronics system was developed by Lee and Jung [8] to support learning by using intelligent control techniques. To illustrate the mechatronics intelligent system, an inverted pendulum system was designed and implemented using a neuro-fuzzy control method, based on the Tagaki-Sugeno model. Stockmans-Daou [9], a development system, was introduced, and it constitutes a framework along with automated tools that help to develop reconfigurable mechatronics platforms by combining LEGO Mindstorms systems with a focus on robotics education. Besides, Tokuyasu [10] used Mindstorms platforms and introduced mechatronics installations that provide functional blocks with different capabilities, motors, and sensors. Mindstorms with their sensors are widely considered as an effective toolkit for mechatronics education. Stark et al. [11] developed a low-cost personal laboratory setup supporting learning mechatronics control theory is featured by portability and accessibility. Furthermore, Yilmaz and Tuncalp [12] introduced a mixed learning model that is Web-based and integrates a range of perceptual techniques and methods for effective mechatronics education.

Besides the mentioned work, Sell et al. [13] introduced a methodology with its technical concept that helps students to use mechatronics platforms to support coursework and hands-on experiments through the Internet independent of physical place. The developed platform was demonstrated in Estonia and Germany, and the reflections of the experience were reported by Grover et al. [14]. For the purpose of enhancing microcontroller skills, an undergraduate course was designed for students with different backgrounds. This course was based on a low-cost Arduino platform. Pratumsuwan [15] presented a framework for the final year mechatronics project.

This paper aims to enhance education and learning quality and contributes to learning while stimulating creative and critical thinking skills. The paper has adopted a studentcentered learning approach and focused on developing training tools that contribute to improve the hands-on experience and integrate it with project-based learning. This enables students to apply the acquired knowledge in the project to develop new ideas and thinking skills. For this purpose, four educational and experimental systems are proposed to stimulate the learning process of undergraduate students. The developed experimental systems, along with the adopted project-based learning approach, are evaluated in actual laboratory sessions supporting different courses at Sanyo-Onoda City University, Yamaguchi, Japan, and at the American University in Cairo.

#### **2. Paper Contributions and Methodology**

The key contributions of this paper can be listed as follow:


The adopted methodology that has been used to achieve this aims to:


#### **3. Experiential Learning and Thinking Skills**

The trends guiding education for the 21st century are student-centered, experiential learning, technology-infused, and inquiry-based supported by compassion and empathy. Experiential learning is an active student-led learning process through direct engagement supported by experience, analysis, and reflections. It can also be described as an applied creativity, a learning approach that helps to identify the ability of skill and knowledge adaptation needs and applies the required changes to improve and maximize performance. Therefore, experiential learning combined with creative thinking represents the top important requirements featuring future careers.

Creativity is a collection of critical skills that facilitate divergently thinking and enables the brain to generate new concepts, new ideas, and new ways to solve problems. Creativity skill-building can be achieved through different efforts, such as readings, interactive discussion, brainstorming, exercises, experiments, team-based projects, etc. This helps students to assess real problems available in real life in new ways and enables them to find out the best solution, novel product design, and the way to adapt and adopt new technologies. Building creative thinking as a cognitive process enables innovative capabilities that enhance the learning process yielding innovation.

Engineering education supported by doing through experiments and project-based learning methodologies has direct contributions to motivate students' interest, improve their hands-on experience, and positively influence their learning capabilities. Moreover, it facilitates discussion and social interaction, encourage, inspire, and challenge students to improve their ability to apply the acquired knowledge. This approach enables students to use the added value knowledge to develop new skills to design and provide innovative solutions to problems in the real world. Such engagement supported by students' active efforts helps them to build up experience by doing and learning from it, leading to stimulate the brain, enhance thinking skills, and establish creative abilities to develop innovative solutions. In addition, a team-based project helps to create and enhance employability skills at the individual and group levels. Such an approach forms the foundation of experiential learning that aims to conduct hands-on experiments and projects, test and troubleshoots, gather and analyze data, evaluate outcomes, reflect/observe creative behaviors abstracted from the learning novel experience. The creative behaviors stimulate new ideas and solutions that contribute to the enhancement of the learning process and in developing new thinking skills. Furthermore, this has a direct impact on developing professional skills and contribute to achieving the quality of innovative learning, enabling the transformation of new concepts into reality with added value processes, functions, services, or products [16–23].

#### **4. Mechatronics and the Development of Creative Thinking Skills**

As an interdisciplinary field, mechatronics is evolved into a philosophy supporting new ways of thinking, interdisciplinary knowledge synergy, work and practices, skills, and innovations [2]. Mechatronics considers education as a social process with a focus to synergize knowledge, establish interactive communications, develop functional skills (through experiments and projects), stimulate motivation, facilitate new teaching techniques, and effectively contribute to the enhancement of student's independent learning skills with the stimulation of strongly couples critical and creative activities as individuals and in teams to bring complementary dimensions to the learning process. This should be supported by an effective exploratory learning environment that focuses on creativity while stimulating a student's motivation. In mechatronics education, the learning process is directly linked with functional skill development that is directly coupled with thinking and creativity. These skills enable thinkers to solve complex problems and work efficiently across different disciplines. In general, current human thinking is not that efficient, based on judgment more than ideas, design, knowledge, and experience. Thinking and creativity are skills that can be developed. These skills require learning it, practicing it, paying attention to it, and gaining confidence. Creative design is a constructive process that looks to develop and implement ideas with new values. Hence, it is essential that students and researchers (individuals and teams) develop their knowledge, experience, and accumulate the acquired creative thinking skills. Besides, engaging them to develop new innovative concurrent design capabilities.

Creative mechatronics as a concurrent and interactive design process helps to facilitate the learning process that leads to the generation of new ideas and solutions to fulfill the demand to maximize the level of creativity supported by creative thinking skills. Evidence of lateral thinking (horizontal) that is classified as deliberate creative skills aiming to understand existing challenges or opportunities, open new possibilities, maximize problemsolving agility for better or new solutions, and develop new ideas by trying to brainstorm new novel solutions, concepts, and perceptions, while understanding governing rules and constraints. In addition, evidence of vertical thinking involves logical thinking and reasoning, and it includes different functional principles in terms of accuracy, reliability, and geometric structure besides the application of general scientific design principles. In real problem-solving circumstances, a combination of lateral and vertical approaches should be considered to produce the best results. Moreover, practical design with experimental work is an integral part to stimulate thinking, while building up thinking skills. Integrating problem-solving approaches together with creative thinking practices into the concurrent design process of engineering and science projects/products demand not only to change the learning environment and the teaching methods, but it also requires to adopt a new holistic assessment, and evaluation methods [24,25]. Such learning environments contribute to establishing motivation, create curiosity and challenges, imagination, and success in understanding science, engineering, and technology [2].

#### **5. Mechatronics Experimental Educational Systems**

A set of experimental systems were developed in association with this paper. The developed experimental system is featured by the integration of the necessary knowledge and skills demanded in mechatronics as an interdisciplinary field. The developed experimental hardware and associated software alone are not making a major difference compared to the adopted educational learning process as a whole. Each experiment in the list of the laboratory work represents an integrated process that includes: Experiment set up and troubleshooting, running and results' evaluation, answering attached design problems according to the design requirements. This is going to challenge the gained skills throughout the experiment while stimulating students to develop new skills. Furthermore, it leads to improving writing and oral communication skills through the assigned presentation sessions.

#### *5.1. Mechatronics Experimental System I*

Figure 1 depicts the mechatronics experimental system I that consists of LEDs aboard showing LEDs as ON or OFF, a two wheels' mobile robot with position control. The developed mechatronics experimental system I aims to support hands-on experiments as a part of experimental sessions of the course "Experiments I for Mechanical Engineering" required for second-year students [18]. This course is a three credit-hour course, and it constitutes sixteen laboratory sessions divided into five modules associated with five topics. These are:


Each module constitutes three laboratory sessions. There are about 10 students per each laboratory session, which is divided into five groups. In general, two students per one group. The remaining laboratory session is dedicated to the group presentation on a selected module and for each group. The laboratory sessions are conducted weekly, and the duration of each is five hours. Accordingly, the laboratory module titled mechatronics experiment I is covered by three laboratory sessions, and that requires a total of 15 h (3 sessions × 5 h).

Figure 2 presents the board layout of the experimental LEDs. The board has 24 LEDs that arrayed into 8 LEDs per row, as depicted in Figure 3, showing the two output ports B and C, respectively. Lighting and timely flashing patterns can be set using hexadecimal code as required, while the flashing period can be adjusted using Window's timer interrupt or by external triggering. The experiment helps students to learn how to use input-output ports and to design LEDs' flashing using varieties of different flashing patterns by writing

proper computer programs to achieve the desired flashing according to the status of binary input received through sensors or by other means and checked through the input port. Five hours are allotted for this experiment covering hardware and programming skill requirements. Note that the input port A is used to receive the operational state of binary sensors. *Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 6 of 23

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 6 of 23

**Figure 1.** Presents the mechatronics experimental system I5.1.1 Input-Output ports and LEDs flashing experiment: Programing with a timer interrupt. **Figure 1.** Presents the mechatronics experimental system I5.1.1 Input-Output ports and LEDs flashing experiment: Programing with a timer interrupt. through the input port. Five hours are allotted for this experiment covering hardware and programming skill requirements. Note that the input port A is used to receive the operational state of binary sensors.

Figure 2 presents the board layout of the experimental LEDs. The board has 24 LEDs

**Figure 2.** The board layout of the experimental LEDs. **Figure 2.** The board layout of the experimental LEDs. *Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 7 of 23

The position control module to control a stepper motor is shown in Figure 4. A photo-interrupter is used and fixed together with the leadscrew that is mechanically coupled with the stepper motor. The sensor detects the position of the leadscrew movement through slit positions aligned under the base frame of the leadscrew. The pulse signals generated by the sensor during the movement are used to calculate the actual moved distance and compare it with the pulses sent to drive the stepping motor in steps equivalent to the total desired distance. This experiment helps students to program the motion and understand the effect of stepper motor resolution and the leadscrew resolution with their impact on the traveled distance and the position accuracy. With the implemented position control module and the leadscrew mechanism, each step of the used stepper motor has a resolution of 1.8 degrees/step, and this causes the leadscrew to move by 6.25 *µ*m, based on its thread structure. The desired number of pulses signal representing the desired distance is sent to the motor through port A, and the output port B is used to monitor the actual movement using the photo-coupler (output port B, see Figure 3). Table 1 shows the interpretation of the binary code displayed at port B to that corresponding motor motion action. Students can learn from the experiment how to generate pulses used to drive the motor, how to control the stepper motor, and track the current position of the moving leadscrew. Besides, students (after understanding the hardware structure) work to develop computer programs to control the leadscrew while counting the number of detected slits and find out the position accordingly. Five hours are allotted to conduct this

**Figure 2.** The board layout of the experimental LEDs. **Figure 3.** The implemented two 8 bits' output ports. **Figure 3.** The implemented two 8 bits' output ports.

experiment.

**Figure 4.** The position control module with leadscrew, stepper motor, and light sensor.

#### 5.1.1. Experiment on Position Control of a Stepper Motor photo-interrupter is used and fixed together with the leadscrew that is mechanically cou-

5.1.1. Experiment on Position Control of a Stepper Motor

**Figure 3.** The implemented two 8 bits' output ports.

7 6 5 4 3 2 1 0

7 6 5 4 3 2 1 0

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 7 of 23

The position control module to control a stepper motor is shown in Figure 4. A photointerrupter is used and fixed together with the leadscrew that is mechanically coupled with the stepper motor. The sensor detects the position of the leadscrew movement through slit positions aligned under the base frame of the leadscrew. The pulse signals generated by the sensor during the movement are used to calculate the actual moved distance and compare it with the pulses sent to drive the stepping motor in steps equivalent to the total desired distance. This experiment helps students to program the motion and understand the effect of stepper motor resolution and the leadscrew resolution with their impact on the traveled distance and the position accuracy. With the implemented position control module and the leadscrew mechanism, each step of the used stepper motor has a resolution of 1.8 degrees/step, and this causes the leadscrew to move by 6.25 µm, based on its thread structure. The desired number of pulses signal representing the desired distance is sent to the motor through port A, and the output port B is used to monitor the actual movement using the photo-coupler (output port B, see Figure 3). Table 1 shows the interpretation of the binary code displayed at port B to that corresponding motor motion action. Students can learn from the experiment how to generate pulses used to drive the motor, how to control the stepper motor, and track the current position of the moving leadscrew. Besides, students (after understanding the hardware structure) work to develop computer programs to control the leadscrew while counting the number of detected slits and find out the position accordingly. Five hours are allotted to conduct this experiment. pled with the stepper motor. The sensor detects the position of the leadscrew movement through slit positions aligned under the base frame of the leadscrew. The pulse signals generated by the sensor during the movement are used to calculate the actual moved distance and compare it with the pulses sent to drive the stepping motor in steps equivalent to the total desired distance. This experiment helps students to program the motion and understand the effect of stepper motor resolution and the leadscrew resolution with their impact on the traveled distance and the position accuracy. With the implemented position control module and the leadscrew mechanism, each step of the used stepper motor has a resolution of 1.8 degrees/step, and this causes the leadscrew to move by 6.25 *µ*m, based on its thread structure. The desired number of pulses signal representing the desired distance is sent to the motor through port A, and the output port B is used to monitor the actual movement using the photo-coupler (output port B, see Figure 3). Table 1 shows the interpretation of the binary code displayed at port B to that corresponding motor motion action. Students can learn from the experiment how to generate pulses used to drive the motor, how to control the stepper motor, and track the current position of the moving leadscrew. Besides, students (after understanding the hardware structure) work to develop computer programs to control the leadscrew while counting the number of detected slits and find out the position accordingly. Five hours are allotted to conduct this experiment.

The position control module to control a stepper motor is shown in Figure 4. A

Output port B

Output port C

**Figure 4.** The position control module with leadscrew, stepper motor, and light sensor. **Figure 4.** The position control module with leadscrew, stepper motor, and light sensor.



5.1.2. Use of Mobile Robot for Line-Tracing Experiment

Figure 5 shows the used mobile robot with two actuated wheels and equipped with four diffuse light sensors at the front side. The two wheels of the robot are driven by two

stepper motors. The motion of the stepper motors is excited by the binary data coming from received through the output ports B and C to energize the coils of the motors. Furthermore, Figure 5 shows ω and v representing the rotational and the forward translational velocities of the robot, respectively. The robot's rotational velocity is calculated from the angular velocity of both wheels as: stepper motors. The motion of the stepper motors is excited by the binary data coming from received through the output ports B and C to energize the coils of the motors. Furthermore, Figure 5 shows ω and v representing the rotational and the forward translational velocities of the robot, respectively. The robot's rotational velocity is calculated from the angular velocity of both wheels as:

Figure 5 shows the used mobile robot with two actuated wheels and equipped with four diffuse light sensors at the front side. The two wheels of the robot are driven by two

**Table 1.** 4-bit binary code given to the lower four bits' port B corresponding to the motion type of

$$
\omega = R \frac{(\omega\_1 - \omega\_2)}{L} \tag{1}
$$

where *ω*1 and *ω*2 are the wheels' rotational velocities 1 and 2, respectively, and the mobile robot is described with respect to its coordinate frame; 2*L* represents the thread distance connecting the centers of the two wheels, while *R* represents the wheel radius. Furthermore, when generating forward/backward linear motions, it requires only to set the required robot linear velocity (*v*) and find out the linear velocity of each wheel, *v*1, *v*2, as, where *ω*1 and *ω*2 are the wheels' rotational velocities 1 and 2, respectively, and the mobile robot is described with respect to its coordinate frame; 2*L* represents the thread distance connecting the centers of the two wheels, while *R* represents the wheel radius. Furthermore, when generating forward/backward linear motions, it requires only to set the required robot linear velocity (*v*) and find out the linear velocity of each wheel, *v*1, *v*2, as,

$$\begin{aligned} v &= R \frac{(\omega\_1 + \omega\_2)}{2} \\ v\_1 &= R\omega\_1, v\_2 = R\omega\_2 \end{aligned} \tag{2}$$

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 8 of 23

0110 Low level signal for clockwise motion 0111 High level signal for clockwise motion 0010 Low level signal for counterclockwise motion 0011 High level signal for counterclockwise motion

1000 Excitation of motor is OFF

5.1.2. Use of Mobile Robot for Line-Tracing Experiment

the stepper motor.

**Figure 5.** The mobile robot with two actuated wheels and equipped with four diffuse light sensors. **Figure 5.** The mobile robot with two actuated wheels and equipped with four diffuse light sensors.

The integrated light reflectors with the robot are used to detect floor color (black or white). The detection status is monitored by checking reflected outputs going to the input port A. The integrated light reflectors with the robot are used to detect floor color (black or white). The detection status is monitored by checking reflected outputs going to the input port A.

In this part of the experiment, the aim is to enable students to learn how to write basic programs that generate forward/backward motions and instruct the robot to turn left/right In this part of the experiment, the aim is to enable students to learn how to write basic programs that generate forward/backward motions and instruct the robot to turn left/right while understanding the basic kinematics of a two wheels' mobile robots. In this experiment, students learn how to program the robot to generate the required motion, test and run the programs. Moreover, to write programs that enable the robot to trace a line using the integrated light sensors. Five hours are allotted to this experiment.

#### *5.2. Mechatronics Experimental System II*

The second developed mechatronics experimental system is shown in Figure 6, and it consists of three analog sensors that help to measure distance, temperature, and light intensity. Moreover, it has interfaces for AD/DA conversion; also, it is equipped with an LCD panel. Beside this, it has one DC motor and one stepper motor, and a photo-interrupter sensor. The principle hardware is of this experiment is provided through C-TASK Company. The developed experimental module "mechatronics experiment II" system is used to fulfill the needs demanded by the experimental sessions of the course "Experiments II

for Mechanical Engineering" offered at a third year level [26]. The Experiments II for Mechanical Engineering course is a four credit-hour course, and it is conducted for two semesters (one year). It constitutes twenty-six laboratory sessions (13 per each semester). The laboratory sessions are conducted weekly, and the duration of each laboratory session is five hours. the needs demanded by the experimental sessions of the course "Experiments II for Mechanical Engineering" offered at a third year level [26]. The Experiments II for Mechanical Engineering course is a four credit-hour course, and it is conducted for two semesters (one year). It constitutes twenty-six laboratory sessions (13 per each semester). The laboratory sessions are conducted weekly, and the duration of each laboratory session is five hours.

while understanding the basic kinematics of a two wheels' mobile robots. In this experiment, students learn how to program the robot to generate the required motion, test and run the programs. Moreover, to write programs that enable the robot to trace a line using the inte-

The second developed mechatronics experimental system is shown in Figure 6, and it consists of three analog sensors that help to measure distance, temperature, and light intensity. Moreover, it has interfaces for AD/DA conversion; also, it is equipped with an LCD panel. Beside this, it has one DC motor and one stepper motor, and a photo-interrupter sensor. The principle hardware is of this experiment is provided through C-TASK Company. The developed experimental module "mechatronics experiment II" system is used to fulfill

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 9 of 23

grated light sensors. Five hours are allotted to this experiment.

*5.2. Mechatronics Experimental System II* 

The laboratory sessions cover twelve modules supporting twelve topics. Two laboratory sessions are assigned for each module topic. Besides, a laboratory session is assigned for presentation at the end of each semester. The overall topic modules within the course are listed, based on each semester as follows:

For the first semester:


For the second semester:


The first module during the week of mechatronics experimental system II aims to help students learn how to use different analog sensors and convert their output signals to digital through the use of A/D conversion technique. Students in the second week learn how to design a closed-loop feedback control facilitated by a PID control law to control the wheel's rotational velocity. Velocity control software is developed and tested by students for the purpose of this experiment. In addition, the D/A converter used in this experiment is characterized by 8 bits output, and it can accommodate analog signal ranging at least from 0 to +3.3 V. Furthermore, it is necessary to practice how to adjust the output voltage from the D/A to match the need to drive the DC motor with 256 levels (8 bits) of speed control resolution. The DC motor velocity is controlled using a proportional-integral (PI) controller given by:

$$
\pi(k+1) = \pi(k) + \Delta\pi(k)\tag{3}
$$

$$
\Delta \tau(k) = K\_p (v\_d - v\_s(k)) + K\_i \sum\_{n=1}^k (v\_d - v\_s(n)) \tag{4}
$$

where *τ* (*k*) represents the DC motor torque's output at the k discrete time; *Kp* represents the proportional gain, and *Ki* represents the integral gain of the PI controller; *v<sup>d</sup>* and *vs*(*k*) are the desired rotational velocity and the actual rotational velocity measured at *k* discrete time, as shown in Figure 7. The actual rotational velocity is measured by counting the number of detected holes at the wheel boundary using the optical sensor. *Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 11 of 23

**Figure 7.** Wheel with a hole and a sensor to measure a DC motor's velocity. **Figure 7.** Wheel with a hole and a sensor to measure a DC motor's velocity.

*5.3. Mechatronics Experimental System III*  Grasping Force and Joint Velocity Control Using 4-DoF Robot After going through the laboratory sessions of mechatronics experiment systems I and II, students would be able to reflect and build up their creative thinking skills through This experiment enables students to design and implement velocity feedback controller described by Equations (3) and (4), respectively. Besides, students experimentally learn to use and understand the characteristics of PI controllers and their control impact on both dynamic parameters and the steady-state error.

project-based learning with each course or dedicated for the fourth year senior project

(ABOT), and it is shown in Figure 8. The students used this robot to understand and demonstrate its operation and then learn how to develop a controller to control its joint velocities using Pulse Width Modulation (PWM) technique. In addition, the project aims to enable students to understand the principles of grasping force control using a force sensor placed at the inner sides of the gripper and demonstrate experimentally by programing the robot for this purpose. The experimental layout of the robot is also shown in Figure 8. When the applied force by the gripper force increases, the resistance is representing the physical property of the sensor decreases. Hence, the robot's controller can be programmed to monitor the change in the force by measuring the voltage across the force sensor. The measured voltage is conditioned and converted to digital through the AGB65-

ADC converter module.

### *5.3. Mechatronics Experimental System III* Grasping Force and Joint Velocity Control Using 4-DoF Robot

After going through the laboratory sessions of mechatronics experiment systems I and II, students would be able to reflect and build up their creative thinking skills through project-based learning with each course or dedicated for the fourth year senior project course as a requirement for their graduation. The robot's arm is a PC-based 4-DOF, robot and each joint is driven by a DC motor. An example of used robots is called Arm Robot (ABOT), and it is shown in Figure 8. The students used this robot to understand and demonstrate its operation and then learn how to develop a controller to control its joint velocities using Pulse Width Modulation (PWM) technique. In addition, the project aims to enable students to understand the principles of grasping force control using a force sensor placed at the inner sides of the gripper and demonstrate experimentally by programing the robot for this purpose. The experimental layout of the robot is also shown in Figure 8. When the applied force by the gripper force increases, the resistance is representing the physical property of the sensor decreases. Hence, the robot's controller can be programmed to monitor the change in the force by measuring the voltage across the force sensor. The measured voltage is conditioned and converted to digital through the AGB65-ADC converter module. *Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 12 of 23

**Figure 8.** 4-DoFarm robot used to learn force and joint velocity control techniques. **Figure 8.** 4-DoFarm robot used to learn force and joint velocity control techniques.

Figure 9a demonstrates the robots' grasp force control of a green softball. The force control is achieved by controlling either the gripper's opening or closing velocity according to the measured output of the force sensor. Furthermore, Figure 9b demonstrates the desired and actual force control result. The measured force values at each time discrete are plotted to represent the feedback of the grasping motion. Figure 9a demonstrates the robots' grasp force control of a green softball. The force control is achieved by controlling either the gripper's opening or closing velocity according to the measured output of the force sensor. Furthermore, Figure 9b demonstrates the desired and actual force control result. The measured force values at each time discrete are plotted to represent the feedback of the grasping motion.

(**a**) The robot demonstrates a grasp force control of a green softball

**Figure 8.** 4-DoFarm robot used to learn force and joint velocity control techniques.

plotted to represent the feedback of the grasping motion.

Figure 9a demonstrates the robots' grasp force control of a green softball. The force control is achieved by controlling either the gripper's opening or closing velocity according to the measured output of the force sensor. Furthermore, Figure 9b demonstrates the desired and actual force control result. The measured force values at each time discrete are

(**a**) The robot demonstrates a grasp force control of a green softball

(**b**) Plot represent the measured force values at each time discrete

**Figure 9.** Force control result using three film-type sensors. **Figure 9.** Force control result using three film-type sensors.

#### *5.4. Mechatronics Experimental System IV*  Multiple Mobile Robots System *5.4. Mechatronics Experimental System IV* Multiple Mobile Robots System

needs.

This section introduces the mechatronics experimental system IV. System IV uses a number of mobile robots to enable these robots to learn the principles of subsumption control architecture and the way to use it to develop schooling behavior. The subsumption control architecture represents the early contributions to developing behavior-based control introduced by Brooks in 1986 [27], and it highly influences the navigation aspect of autonomous robotics development. A robot with three wheels locomotion is shown in Figure 10a, and the is a multi-robotic system using the three wheels robot is shown in Figure 10b. This system is used for the fourth year senior project to enhance their learning of new critical thinking skills through project-based learning, e.g., to develop a new structure of subsumption solving schooling behavior. With such learning activity, students begin to familiarize themselves, and practice the subsumption architecture concept and introduce technical methods to develop the structure of reactive behavior systems. This This section introduces the mechatronics experimental system IV. System IV uses a number of mobile robots to enable these robots to learn the principles of subsumption control architecture and the way to use it to develop schooling behavior. The subsumption control architecture represents the early contributions to developing behavior-based control introduced by Brooks in 1986 [27], and it highly influences the navigation aspect of autonomous robotics development. A robot with three wheels locomotion is shown in Figure 10a, and the is a multi-robotic system using the three wheels robot is shown in Figure 10b. This system is used for the fourth year senior project to enhance their learning of new critical thinking skills through project-based learning, e.g., to develop a new structure of subsumption solving schooling behavior. With such learning activity, students begin to familiarize themselves, and practice the subsumption architecture concept and introduce technical methods to develop the structure of reactive behavior systems. This starts from low level behavior to a higher level behavior using layered sets of rules that

starts from low level behavior to a higher level behavior using layered sets of rules that

1 Move to the direction of PSD sensor 1 2 Move to the direction of PSD sensor 2 3 Move to the direction of PSD sensor 3 4 Move to the direction of PSD sensor 4 5 Move to the direction of PSD sensor 5 6 Move to the direction of PSD sensor 6 7 Rotate to clockwise direction 8 Rotate to counterclockwise direction

Table 2 shows the list of command codes and the associated reaction behaviors (reflex actions) that support the navigation of the mobile robots. An agent represents a primitive basic behavior with an assigned priority. Each agent is designed to generate a

reflex action based on the sensory information within its priority level.

**Table 2.** A list of primitive reaction behaviors (reflex actions) selected by mobile robots. **Command Codes Corresponding Reflex Actions**  0 Halt at a position

low,

where ప

ሶ = ሾప

compose reactive behaviors with reference to the demand of the environmental changes and needs. frame, పଶሶ = ሾ0 1ሿ் represents the normalized velocity for moving to Position Sensitive Detector (PSD) sensor 2, while *v* is the magnitude of the robot's linear velocity.

పଶሶ

ሶ ሿ் represents the linear velocity described in the *i-*th robot, coordinate

‖పଶሶ ‖ (5)

ప ሶ =

The developed subsumption-based controller is illustrated in Figure 11. It is composed of four agents, in which the higher execution level priority is assigned to the upper agent. Each small circle with the letter 'S' inside represents a suppression node from a higher priority level to deactivate agent behavior(s) with low priority when more than one behavior agent is activated. The experimental scene of schooling behavior is shown in Figure 12. When schooling pattern modes of the mobile robots are selected, the robots regularly move to follow the inner side of the circular fence while trying to keep a constant distance between the robots and the fence, and also between the robots. This mode enables the behavior of the robots to mimic the carps' movement in a circular artificial Japanese pond. Besides, in case a robot moves along the circular fence in a counterclockwise direction, the control law resulted from "Move forward" agent can be stated as bel-

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 14 of 23

(**a**) A robot with three wheels locomotion

(**b**) Multi-robots of the three wheels locomotion

**Figure 10.** Three wheeled multiple mobile robots used for the graduation study. **Figure 10.** Three wheeled multiple mobile robots used for the graduation study.

**Figure 11.** The design of subsumption based control architecture for schooling behavior.

the sensors are converted to digital using the A/D converter module.

ሶ

"Turn to left or right" as

where ሶ

The configuration of the two tactile flex sensors and the six PSD sensors is illustrated in Figure 13. The sensing distance range of the PSD sensor is between a maximum of 80 cm and a minimum of 10 cm. In addition, Figure 14 presents the tactile flex sensor used with each robot. The degree of bending is measured, and the obtained analog signals from

The robot's orientation of a robot is controlled when by the higher priority agent

indicates to the rotational velocity; *di*1(*k*) and *di*6(*k*) are the measured values

by the 1st and 6th PSD sensors of the *i*-th robot, respectively; *K<sup>ϕ</sup>* represents the gain used to keep and control the orientation of the robot parallel to the inner circular fence.

= ఝ(() − ଵ()) (6)

Table 2 shows the list of command codes and the associated reaction behaviors (reflex actions) that support the navigation of the mobile robots. An agent represents a primitive basic behavior with an assigned priority. Each agent is designed to generate a reflex action based on the sensory information within its priority level.


**Table 2.** A list of primitive reaction behaviors (reflex actions) selected by mobile robots.

The developed subsumption-based controller is illustrated in Figure 11. It is composed of four agents, in which the higher execution level priority is assigned to the upper agent. Each small circle with the letter 'S' inside represents a suppression node from a higher priority level to deactivate agent behavior(s) with low priority when more than one behavior agent is activated. The experimental scene of schooling behavior is shown in Figure 12. When schooling pattern modes of the mobile robots are selected, the robots regularly move to follow the inner side of the circular fence while trying to keep a constant distance between the robots and the fence, and also between the robots. This mode enables the behavior of the robots to mimic the carps' movement in a circular artificial Japanese pond. Besides, in case a robot moves along the circular fence in a counterclockwise direction, the control law resulted from "Move forward" agent can be stated as bellow,

$$\dot{X}\_i = v \frac{\dot{x}\_{i2}}{||\dot{x}\_{i2}||} \tag{5}$$

where . *X<sup>i</sup>* = - . *x<sup>i</sup> y<sup>i</sup> T* represents the linear velocity described in the *i*-th robot, coordinate frame, . *xi*<sup>2</sup> = - 0 1 *<sup>T</sup>* represents the normalized velocity for moving to Position Sensitive Detector (PSD) sensor 2, while *v* is the magnitude of the robot's linear velocity. (**b**) Multi-robots of the three wheels locomotion **Figure 10.** Three wheeled multiple mobile robots used for the graduation study.

**Figure 11.** The design of subsumption based control architecture for schooling behavior. **Figure 11.** The design of subsumption based control architecture for schooling behavior.

The configuration of the two tactile flex sensors and the six PSD sensors is illustrated

The robot's orientation of a robot is controlled when by the higher priority agent

indicates to the rotational velocity; *di*1(*k*) and *di*6(*k*) are the measured values

by the 1st and 6th PSD sensors of the *i*-th robot, respectively; *K<sup>ϕ</sup>* represents the gain used to keep and control the orientation of the robot parallel to the inner circular fence.

= ఝ(() − ଵ()) (6)

the sensors are converted to digital using the A/D converter module.

ሶ

"Turn to left or right" as

where ሶ

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 16 of 23

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 16 of 23

**Figure 12.** Schooling behavior of multi-robots based on subsumption architecture. **Figure 12.** Schooling behavior of multi-robots based on subsumption architecture.

The configuration of the two tactile flex sensors and the six PSD sensors is illustrated in Figure 13. The sensing distance range of the PSD sensor is between a maximum of 80 cm and a minimum of 10 cm. In addition, Figure 14 presents the tactile flex sensor used with each robot. The degree of bending is measured, and the obtained analog signals from the sensors are converted to digital using the A/D converter module. **Figure 12.** Schooling behavior of multi-robots based on subsumption architecture.

**Figure 13.** Six PSD sensors are attached to the mobile robot, where PSD 2 position is at the front side of the robot. **Figure 13.** Six PSD sensors are attached to the mobile robot, where PSD 2 position is at the front side of the robot.

**Figure 14.** Tactile (flex) sensor with resistance as a physical property. **Figure 14.** Tactile (flex) sensor with resistance as a physical property.

Besides this, the agent behavior "Avoid objects" actuates the robot's wheels to avoid collisions when an object, including other robots, is detected within the effective The robot's orientation of a robot is controlled when by the higher priority agent "Turn to left or right" as

$$
\dot{\varphi}\_i = K\_\varphi(d\_{i\theta}(k) - d\_{i1}(k))\tag{6}
$$

when an object is detected by the flex sensor within a range of 10 cm by measuring the readings of fi1(k) and fi2(k). Students can learn the principles of how to build different levels of priorities of agent behaviors for single and multiple mobile robots through the where . *ϕi* indicates to the rotational velocity; *di*1(*k*) and *di*6(*k*) are the measured values by the 1st and 6th PSD sensors of the *i*-th robot, respectively; *Kφ* represents the gain used to keep and control the orientation of the robot parallel to the inner circular fence.

use of the subsumption architecture. Finally, students' performance associated with lab experiments was assessed using two indicators and the results are shown in Table 3. The results showed positive and effective outcomes. **Table 3.** Showing students' performance assessment indicators of conducting lab experiments. Besides this, the agent behavior "Avoid objects" actuates the robot's wheels to avoid collisions when an object, including other robots, is detected within the effective PSD range. Furthermore, the agent behavior "Slowly turn to left or right" has to be activated when an object is detected by the flex sensor within a range of 10 cm by measuring the readings of fi1(k) and fi2(k). Students can learn the principles of how to build different levels of priorities of agent behaviors for single and multiple mobile robots through the use of the subsumption architecture.

**Indicator 1**: Ability to Integrate and Test Different Mechatronic Laboratory Experiments from experimental Modules and Basic Components (%) Finally, students' performance associated with lab experiments was assessed using two indicators and the results are shown in Table 3. The results showed positive and effective outcomes.

Assessment Category 2019 2018 2017

Very Good 14 15 22

Excellent 69 71 65 **Table 3.** Showing students' performance assessment indicators of conducting lab experiments.

**Indicator 1:**

Good 17 14 12 Satisfactory 0 0 0 Ability to Integrate and Test Different Mechatronic Laboratory Experiments from experimental Modules and Basic Components (%)


Good 8 8 9 Satisfactory 0 0 0 Unsatisfactory 0 0 0 **Table 3.** *Cont.*


### **Indicator 2:**

#### **6. Project-Based Learning**

A project-based learning strategy is a comprehensive learning approach aiming to motivate and enhance the capacity of a student's thinking skills [28–32]. The adopted project-based learning approach is an integrated approach demand active student engagement in the investigation that focuses on considering realistic, complex, and open-ended, unstructured problems to coordinate all learning activities to find out innovative solutions to a problem. It is an integrated and holistic learning process the constitutes multiple milestones. These milestones start from team formation, specify the problem to solve, objectives, plan development, debating ideas, constraints, functional requirements (software and hardware), conceptual design, specifications, details design, implementation and troubleshooting, integration, demonstration, conclusions, reporting, and communicating the findings.

Project-based learning is student-centered, and it is facilitated by an instructor who motivates students to learn by applying the acquired knowledge, experience, and interdisciplinary skills through active and interactive engagement for the acquisition of new deeper knowledge and enhance thinking skills, and help to find new ideas and new solutions to solve interdisciplinary real-world challenges and problems that require to seek and use skills and knowledge from different knowledge domains. In addition, project-based learning through its activities helps team members to learn and inspire from each other through collaboration and facilitate deeper learning to represent knowledge and finding new ideas and solutions.

The presented four developed mechatronics experimental systems are used to build up acquired knowledge through experiments. The aim is to integrate the acquired knowledge foundation and enables students to demonstrate freely their skills through practice. It also facilitates a new added value experience with interdisciplinary knowledge from different perspectives while engaging them actively in the project activities. The challenges arise on how to apply such knowledge, skills, and experience through a new context to solving challenging real-life problems with which students never dealt with before. Besides this, it is also necessary to build team-based learning skills to meet professional challenges. Hence, project-based learning is the way to deal with such needs and requirements and fulfill the targeted educational values.

In project-based learning covered by this paper, students were divided into teams, and a coordinator is selected for each team from within to follow up the team interactive and collaborative work and communicate with the instructor. The target is to reflect on and improve their learning added values by developing creative and critical thinking skills, improving writing and communication skills by emphasizing student interest, while giving them the freedom and the responsibility to deliberate through brainstorming and deciding their project topic related to products or systems for each team separately. It is essential to have the communication line open between the instructor and each of the project teams to discuss their concerns, suggestions, and inquiries while receiving feedback on their reports

and reflections. However, the following milestones represent the key stages conducted successfully for the adopted project-based learning approach:


According to the experience of the conducted experimental systems, problem-solving and project-based learning during the last three years. Three key indicators were assessed and related to the size of the class between 25–30 students. Table 4 presents the students' performance assessment associated with three performance indicators.

**Table 4.** Showing students' performance assessment indicators of project-based learning.


**Table 4.** *Cont.*

**Indicator 2:** Ability to Apply Problem Solving Skills and Develop Software Programs to Support Project Solution's Needs (%)


#### **Indicator 3:**

Ability to Build, Integrate and Test a Mechatronic Project-Based System from its idea, Designing to Implementation using Basic Elements (%)


Finally, and through the obtained experience in the field, the following guidelines have been concluded to help to guide project-based learning and maximize the learning outcomes:


process according to the work progress, the team with the support of the coordinator assigns work distribution among the members for each milestone. Each assigned work should be documented and reported to all team members,


#### **7. Conclusions**

This paper focused on the importance of experiential learning and its contributions to utilize acquired knowledge and experience to develop creative and critical thinking skills through its added value activities aiming to improve the quality of students' learning outcomes. In order to support achieving such mainstream goals, experimental laboratory content was designed. The laboratory content was facilitated by the development of four mechatronics experimental systems. The developed systems were used as an effective exploratory learning environment with a focus to enhance the outcomes of the integrated learning process, motivate students through practical design and implementation work, solving real-life problems and stimulate thinking to find out new ways and solutions to solve problems and form new ideas that establish the base of building creative thinking skills which can accommodate the ability for innovations and inventions. These systems enabled students to synergize theoretical knowledge with practical mechatronics technological experience using sensors, motor, control, controllers design, information processing and programming while steaming up students' motivation and enjoyment. Furthermore, a project-based learning approach both individual and team-based were integrated with the developed concepts and the experimental mechatronics systems to enhance students' innovation capabilities through a thinking-based design process.

Although each of the developed systems is simple and primitive, integrating these systems with experiments, knowledge, experience, and skills besides other hardware and software can encourage students to solve complex and realistic problems. Moreover, build complex behavioral agent levels using the primitive ones, improve performance and enhance the outcomes of the learning process. The effectiveness of the developed mechatronics experimental systems I, II, III, and IV were evaluated through the experimental laboratory sessions and project-based learning at the Sanyo-Onoda City University, Yamaguchi, Japan, and at the American University in Cairo. Finally, students were divided into teams and asked to select a problem/topic and apply project-based learning with no given list of project titles. The projects' topics were generated through brainstorming sessions and interactive student-centered discussion by the team members with the final endorsement by the instructor. The selected project topic then went through all necessary design thinking, interactive collaboration, and development processes while the faculty facilitates this process. Finally, students made their reflections about the experience of project-based learning and list the newly concluded knowledge and skills obtained through this exercise, and share this experience with other project groups through an open team-based presentation.

The conclusions through the experiences with project-based learning led to developing effective guidelines that specify the best practices associated with the framework of achieving high-quality learning. Project-based learning should highlight and focus on the value of learning and the learning competencies that lead to better achievements, and not just on the learning as a process. This differentiates project-based learning from just conducting projects. Project-based learning should give students the space to voice out their ideas, discuss them, have fun, interact, and raise concerns and inquiries while they need to be inspired for high-level achievements. The project-based learning, with such an open space of skills and content integration will have (besides the value of learning) a good social impact. Finally, the experience through this work highlight the needs to start shifting to student centered learning techniques. The methodology that encourages students' effective engagement by contributing to lectures, assignments, laboratory experiments, and projects, while the instructor must facilitate, update, and guide the learning process. Furthermore, assessing students' performance and activities in a course should be considered holistically by looking at students' achievements as a whole.

To address the educational and teaching needs during the Covid-19 pandemic, the developed mechatronics experimental systems can be integrated remotely as an option to support remote teaching.

**Author Contributions:** Conceptualization, M.K.H. and F.N.; methodology, M.K.H., F.N. and K.W.; formal analysis, M.K.H. and F.N.; investigation, M.K.H., F.N. and K.W.; resources, M.K.H. and F.N.; data curation, M.K.H. and F.N.; writing—original draft preparation, M.K.H., F.N. and K.W.; writing—review and editing, M.K.H. and F.N.; project administration, M.K.H. and F.N.; supervision, M.K.H. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Data Availability Statement:** Not applicable.

**Acknowledgments:** The authors would like to thank Sakakibara Kaoru, C-TASK Co. Ltd. for his help to provide the hardware.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**


### *Article* **Student Burnout: A Case Study about a Portuguese Public University**

**Sofia Salgado <sup>1</sup> and Manuel Au-Yong-Oliveira 1,2,3,\***


**Abstract:** Burnout is increasingly present in organizations and in the most diverse professions, namely, in university students. Burnout can have negative repercussions on their well-being and can even lead them to abandon their studies. The objective of the study focuses on academic burnout and taking medication as a consequence of the requirements of the academic path of students at a Portuguese public university. To achieve this goal, a quantitative methodology was used, consisting of the distribution of a questionnaire to a sample of students from the analyzed university. The first study questionnaire obtained 207 responses, all valid. To perform the analysis of the quantitative data, the program IBM SPSS Statistics, version 25 was used. Inferential statistics were used, namely, Student *t*-test and one-way ANOVA (parametric tests), Spearman's correlation coefficient, and the Chi-square test, to test the previously defined research hypotheses. Among the variables for which statistically significant relationships with burnout were found, the following stand out: the arithmetic mean (course average); the professional situation; participation in extracurricular activities; the practice and frequency of physical exercise; the choice and expectations regarding the course; the uncertainty felt about the professional future; the evaluation of the relationship with colleagues.

**Citation:** Salgado, S.; Au-Yong-Oliveira, M. Student Burnout: A Case Study about a Portuguese Public University. *Educ. Sci.* **2021**, *11*, 31. https://doi.org/ 10.3390/educsci11010031

Received: 30 December 2020 Accepted: 11 January 2021 Published: 15 January 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

**Keywords:** academic burnout; stress; higher education; quantitative research; coping strategies; medication

#### **1. Introduction**

All individuals need to acquire fundamental knowledge and skills throughout their lives, and in particular, to exercise a profession. In an increasingly demanding world, the attendance of higher education is, for many, essential. However, nowadays, there are high levels of stress associated with university life and the environment of higher education institutions [1], and university students, exposed to highly competitive environments and sometimes precarious working conditions (in the case of student workers), are among the large portion of society that suffers from this complex state of mind.

According to [2], it seems frequent that, when entering university, students start by feeling only low levels of stress, caused, above all, by the uncertainties associated with the chosen training path and the type of instruction they face in the University education. The author states that this same stress tends, however, to increase with the progression in the academic path and with the intensification of the required work. The growing burden of study and responsibility also tends to add social and family pressures related to the desire for students to finish their study cycles successfully [3], as well as pressures derived from economic factors, cultural factors, and even aspects related to the personality of each student [4].

Among the factors that cause stress in young people are the significant changes that students go through in the transition periods from primary to basic education, from basic to secondary education and, of course, from secondary to university education. It is known

that it is precisely during these transition periods that students are most concerned with failure, with the future and with aspects such as the expectations of their parents, the possible end of a relationship or the distance from friends [5], all sources of pressure that can generate conflicts within your family.

On the other hand, among the various factors that generate stress and anxiety in modern society, there are also issues related to employability, namely, the unemployment rate of individuals with a higher education degree, which can have severe social, economic, and political consequences [6].

It is therefore imperative not only to understand what the consequences of stress are in a university context, but also to study how the growing concern of these students can cause harmful effects on their academic performance.

It is known that, today, one of the most common negative effects in the university community is academic burnout [7,8], which is a response to stress related to this specific environment, and which affects "the development, understanding and satisfaction of the student with their education and academic life" ([9], p. 4).

The academic burnout syndrome can be understood as the state in which a person is when subjected to long periods of work and stress, particularly caused by the questionable working conditions that he faces in the environment of the university institution in which he is inserted. These conditions can materialize in the lack of resources, in the high competitiveness among colleagues and in so many other factors that negatively influence the performance of any activities carried out in this area [10].

Now, all the peculiarities inherent to university life can cause students to develop syndromes such as burnout, directly associated with fatigue and emotional exhaustion [11], or other disorders of psychological well-being, related to the level of happiness and ideal personal development of students [12], a path that, in turn, must be free of psychological diseases and that implies reaching its full potential.

In the specific case of burnout, the student ends up facing academic life feeling disinterest and frustration, which, according to [4], has physical and emotional manifestations that are often accompanied by evasive behaviors and symptoms that mirror this discomfort. Some of these symptoms are identified: constant tiredness and fatigue; mental exhaustion; lack of ability to nurture their personal relationships; social distancing; complex mood states (such as anxiety, irritability and, in some cases, mild depression); difficulty in being focused or attentive during any task; weight loss and muscle spasms; hormonal and metabolic disorders; allergies and migraines; and insomnia and abuse of psychotropic and narcotic substances, alcohol or other drugs [10].

As can be seen from the symptoms linked to the aforementioned physical and psychological suffering, burnout is a worrying state of health, with many harmful effects [13]. Many of these symptoms are associated, in the case of university students, with a worse understanding of the subjects taught, with the low expectations of these students, with the (eventual) failure to obtain the desired academic degree and with the growing tendency to abandon the professional career for which the degree was initially chosen, as mentioned by authors such as [14,15].

However, and although it is difficult to avoid stress in some scenarios, certain authors, such as [5], believe that it is possible to learn to live with it, reducing the harmful social consequences and the pathologies that so often result from it.

In this sense, higher education institutions have an essential role in preserving the health of their students, since they are the key element in the higher education sector. Students participate in the process of producing scientific knowledge and are, at the same time, users of the teaching processes involved in it, representing an important part of today's society, considered to have a critical spirit and creativity [5].

#### *Study Objectives and Research Questions*

In view of the introduction above, with regard to burnout, the aim of this study is to fill a gap in the scientific literature on this topic. Thus, an analysis of academic burnout in a Portuguese public university is carried out and the frequency with which university students of that institution take medication, from multivitamins to stimulants and/or hypnotics, hypnotics are sleeping pills [16], is investigated in order to try to respond to the demands inherent to that environment.

When tracing the current scenario, it appears that the impact of academic burnout is one of the main factors that affect the professional and personal success of students. For this reason, it is intended to propose, following this analysis, some important recommendations to reduce burnout and to improve the quality of life of students, through the implementation of reforms in higher education.

Ref. [17] refer that, in an empirical study, it is crucial to clarify the research question that is intended to be answered and that will lead to the research work. With this in mind, in order to achieve the empirical knowledge objectives listed above, the following research question are defined: "Do students at the Portuguese public university experience burnout and take or have they taken any medication during their academic career?".

The subject of the present investigation is relevant, since, in Portugal, a significant number of investigations directed to this theme has not been carried out, still less focusing on the public institution studied herein, according to the research carried out in the database SciVerse Scopus (Elsevier) (bibliographic database). Its relevance is also shown by the growing challenge that burnout and taking medication represent for universities, as will be seen below. Both phenomena have relevant economic and social implications, since students who are in a state of burnout are more likely to give up their studies and, therefore, will not be able to make a good contribution to society, as they will not develop their skills and capabilities to their full potential.

In this sense, the main research objectives of the present study are


The objectives are achieved through the application of a quantitative data collection methodology, more precisely through the application of a questionnaire to students of the Portuguese public university under analysis and through a literature review on the subject, based on a previous documentary survey.

#### **2. Literature Review**

#### *2.1. Burnout Definitions*

There are many studies that present definitions of burnout, in its broadest sense, and there are many nuances around them. However, it was [18] who, during the exercise of his activity in psychology, in the 1970s, popularized the concept of burnout, after witnessing a decrease in motivation and commitment in individuals who volunteered at a mental health clinic.

Ref. [18] also referred that, in the work context, burnout is a combination of factors, such as chronic emotional tiredness, physical tiredness and a lack of interest in work activity, low personal fulfillment and insensitivity in the care and attention towards the user. This idea was corroborated by [19], who defined the syndrome as the result of

excessive stress in the work context, in a context in which evidence of emotional tiredness and lack of emotional resources is evident and which is manifested by: a negative response and disinterest in other people; loss of idealism (depersonalization); and a reduction in feelings of competence and job performance (low personal achievement).

Ref. [19–21] state that burnout is a state that results from a continued irreconcilability between an individual and, at least, one of the six dimensions of work listed below:


According to Velasco, J.C. [22], burnout is characterized, not only by the reduction of motivation and job satisfaction, but also by health problems, family and social conflicts, and by the decrease in productivity of individuals who suffer from it.

There are also authors, such as Rosales, Y. [23] who choose to categorize the effects of burnout in three groups: the psychosomatic, the behavioral and the emotional. In the first group, the author includes symptoms such as a change in heart rate, extreme fatigue, or headaches. In the second, it includes drug or alcohol abuse and absenteeism. Finally, in the third group, Rosales, Y. gives as examples the lack of motivation to study and the withdrawal from studies in the academic context.

As can be seen, although there is no consensus regarding the definition of burnout, there is agreement that this syndrome is the result of an effect and response to chronic stress in the work or academic context, which is characterized mainly by three dimensions: exhaustion, depersonalization and low personal fulfillment [24]. Ref. [25] even face burnout as a public health problem and, therefore, recognize that is has significant severity.

#### *2.2. Academic Burnout and Factors Responsible for Burnout*

Both burnout at work and academic burnout are a complex result (motivational, cognitive, emotional, and behavioral response) of a process of accumulation of chronic stress, developing due to the particularities of the context, academic or professional, and of the individual [26].

Ref. [27] indicate that, in the work context, the imbalance in professional life, overwork, depression, interpersonal conflicts, and a decreased sense of self-worth are some of the risk factors associated with the occurrence of the syndrome.

In an academic context, there are many factors responsible for its presence. According to some studies carried out in Australia, Canada, and the United Kingdom, the experience of the transition from secondary to higher education is, for example, a stress promoter, during and from the first year of post-secondary studies, with a propensity to increase in individuals entering university, for financial reasons, or associated with the competitiveness they face, among others [28].

Academic burnout is one of the most important educational research themes today, mainly because its prevalence has been increasing over the years [27,29]. The analysis of academic burnout can be the first step to understand the behavior and academic performance of university students, as well as to understand their level of commitment to

learning, their level of participation in the world of science and how it can affect the enthusiasm for their training [30].

According to [25,31], academic burnout consists of the feeling of cognitive and emotional exhaustion caused by the high demands of a university. Among them, university students are required to attend classes, perform assignments, exams and presentations, actively participate in the classroom, and interact with people and the educational establishment.

This excessive academic burden can have serious consequences on the physical and mental health of the student, who may not yet have an appropriate psychological structure to respond adequately. Thus, feelings of incapacity and incompetence on the part of the students may arise and, of course, a posture of disbelief, not only in relation to teachers and colleagues, but also in relation to studies and the objective of their training [31].

In addition to the academic background, studies such as those by [32], among many others, point to the fact that the social and professional pressures related to the financing of higher education, school performance, and poor relationships with colleagues and teachers are some of the factors that make college students more vulnerable to the development of this syndrome. *Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 6 of 35

> As can be deduced, there are many variables associated with burnout and, in this sense, authors such as [33] establish, in the particular case of academic burnout, categories of academic, environmental, and/or social context and intrapersonal variables (Figure 1). In Figure 1 it is possible to observe each of these categories of variables, illustrated by some examples. and efficiency of administrative staff; organizational inflexibility; the difficulty in collaborating in decision-making; and the absence of an academic association, are other difficulties that arise for students, at the organizational level [9].


**Figure 1.** Categories of variables associated with academic burnout and respective examples [33]. **Figure 1.** Categories of variables associated with academic burnout and respective examples [33].

On the other hand, there are certain peculiarities of the student's work that may be controllable, or that are subject to the intervention of the higher education institution. Some of them are, the overload of disciplines and high academic demands; the frequent absence of explanations and clarifications indispensable to the elaboration of academic activities and work that imply greater concentration, or whose contents are complex; de-The existing research on the predictive variables of burnout focuses essentially on the work context and not so much on the academic environment [9]. Ref. [34] present three groups of predictive variables related to the development of burnout in the work context: of an organizational, social, and individual nature, which can also be applied to the academic context.

manding and/or absentee teachers; the student's lack of control or autonomy over his work; the inappropriate distribution of the workload throughout the semester; and carrying out training activities and/or curricular internships at the same time as classes take There are some risk factors with regard to organizational aspects, which hinder the students' good academic performance, and which positively correlate with higher levels of burnout [34], among them, the difficulties with library and reprography services; the

must take into consideration the type of profession that he intends to pursue, or the course he attends, since burnout is more prevalent in some of the professions and/or courses, as

namely the absence of family support and the absence of friendships [36].

Regarding the social risk factors, in the academic field, which may precede or allow foreseeing the appearance of the burnout syndrome, empirical investigations emphasize the following: the poor relations between students and teachers, the absence of feedback from teachers, reduced solidarity and companionship, competitiveness and conflicts with colleagues, non-participation in cultural or recreational activities, and recreational spaces that are difficult to access. In addition, interpersonal factors also play a crucial role,

On the other hand, [9] point out other aspects as more viable to predict burnout cases in university students, namely, sex (given that women are more effective and men more cynical in relation to their training), certain traits of personality (anxiety, stiffness, perfectionism, and low levels of self-efficacy; self-efficacy being an individual's belief or confidence in their own ability to perform a particular task or solve a particular problem [37]), and certain methods of measuring student academic performance, such as skill deficits, habits, and study methods (for example, to do a job, organize time, or speak in public). In

place [35].

will be seen below [35].

absence of logistical support and infrastructure, namely, classrooms with scarce computer equipment, inappropriate ventilation, poor lighting or inappropriate furniture; or poor transport conditions for traveling to the school [9].

Additionally, the lack of financial support to study, in particular, the difficulty in obtaining scholarships provided or channeled by the university; the insufficient empathy and efficiency of administrative staff; organizational inflexibility; the difficulty in collaborating in decision-making; and the absence of an academic association, are other difficulties that arise for students, at the organizational level [9].

On the other hand, there are certain peculiarities of the student's work that may be controllable, or that are subject to the intervention of the higher education institution. Some of them are, the overload of disciplines and high academic demands; the frequent absence of explanations and clarifications indispensable to the elaboration of academic activities and work that imply greater concentration, or whose contents are complex; demanding and/or absentee teachers; the student's lack of control or autonomy over his work; the inappropriate distribution of the workload throughout the semester; and carrying out training activities and/or curricular internships at the same time as classes take place [35].

It is important to note that, when analyzing the student's task or activity factors, one must take into consideration the type of profession that he intends to pursue, or the course he attends, since burnout is more prevalent in some of the professions and/or courses, as will be seen below [35].

Regarding the social risk factors, in the academic field, which may precede or allow foreseeing the appearance of the burnout syndrome, empirical investigations emphasize the following: the poor relations between students and teachers, the absence of feedback from teachers, reduced solidarity and companionship, competitiveness and conflicts with colleagues, non-participation in cultural or recreational activities, and recreational spaces that are difficult to access. In addition, interpersonal factors also play a crucial role, namely the absence of family support and the absence of friendships [36].

On the other hand, [9] point out other aspects as more viable to predict burnout cases in university students, namely, sex (given that women are more effective and men more cynical in relation to their training), certain traits of personality (anxiety, stiffness, perfectionism, and low levels of self-efficacy; self-efficacy being an individual's belief or confidence in their own ability to perform a particular task or solve a particular problem [37]), and certain methods of measuring student academic performance, such as skill deficits, habits, and study methods (for example, to do a job, organize time, or speak in public). In addition, feeling anxious during exams, having poor social and cognitive abilities, and low prospects for academic success, as well as less motivation or satisfaction with the studies, are other aspects that, according to these authors, may precede a condition of burnout.

The identification of academic burnout, according to [38], may involve assessing the longevity of each student's studies, as well as recognizing certain attitudes, guided by cynicism and incompetence. In addition, some studies, such as those by [20,39,40], prove that people's personality, their social support and coping strategies are variables that allow for the predicting of burnout. However, there do not seem to be many investigations that analyze these variables in university students [41].

In short, it is possible to affirm that academic burnout is characterized by three dimensions: emotional exhaustion, disbelief, and low professional effectiveness, data confirmed by the analysis of several samples of university students from different countries and areas of training [31,32]. University students suffer from emotional exhaustion as a result of a feeling of exhaustion in face of the demand for studies. Disbelief, on the other hand, is present when a cynical and distant attitude towards studies is developed, and low professional effectiveness arises when the student cultivates the idea that he is incompetent [32].

#### *2.3. Implications and Consequences of Burnout*

Several studies show high stress, anxiety and depression among university students. In an attempt to respond to increasingly intense academic and social demands, namely, meeting new people, facing new responsibilities, adapting to new circumstances and increasing work demands [42], these students are more susceptible to syndromes such as academic burnout and to all the personal and social consequences that they can experience in their lives [43].

As ref. [44] point out, academic burnout can make it difficult for higher education students, expected professionals of the future, to graduate, which in turn can have a negative impact in the way they will exercise their professions or even lead to total failure with regard to finishing their cycle of studies and, later, entering the world of work.

In fact, according to certain authors, in university students, this syndrome is associated with poor cognitive performance [45], with reduced academic achievement or dropping out of school [46], and also to depressive symptoms [47] and even suicidal thoughts [13].

Now, if burnout causes low productivity and less student motivation [48], then it is negatively related to the performance of these same students [49]. Hence, students who suffer from burnout are more likely to manifest anger, difficulties in overcoming obstacles and, also, sadness [50].

On the other hand, university students involved in their study cycle in a relaxed and positive way have a better academic performance [51], showing themselves to be energetic, happy and, consequently, being more successful in their learning path.

According to [52], there are still several studies that prove that burnout has several effects, which can be severe, on the well-being and health of individuals.

In fact, burnout syndrome appears to be a considerable predictor of various illnesses and health disorders such as hypercholesterolemia, type 2 diabetes, obesity, coronary heart disease, cardiovascular disorders, musculoskeletal pain, prolonged fatigue, headaches, insomnia, gastrointestinal problems, breathing problems, and psychological changes, such as depression or mental disorders, among others [52].

According to the aforementioned authors, there is also a link between burnout and the adoption of unhealthy lifestyle habits, materialized, for example, in the increase in alcohol consumption and inactivity. In addition, other signs of the burnout syndrome are, according to [53], occupational problems, such as job dissatisfaction (for example, looking for a new job), absenteeism or disability.

#### *2.4. Burnout, Motivation, and Low Academic Performance in University Students*

The main objective of students is to obtain a good academic performance, which is what defines the progression in their studies. However, this performance is the result of an emerging product, composed of interdependent factors of various types: volitional, affective, cognitive, behavioral, and psychosocial, of the student and their learning circumstances. These factors include the conditions of the teaching institution itself, which also interfere with the student's teaching–learning relationship; the student's knowledge or study skills; and the pedagogical training materials provided for this performance [33,54].

If, according to [55], one of the causes for burnout may be the loss of motivation, according to [56], on the contrary, an extreme motivation is a kind of prerequisite for suffering from burnout. Therefore, the question of whether low or high motivation is a very limiting (if not controversial) aspect to assess the likelihood that an individual will suffer burnout [57] seems pertinent.

Ref. [58] ensures that students who are intrinsically motivated are less likely to suffer burnout than students who have extrinsic motivation. Ref. [59] corroborate this idea, stating that extrinsic motivation even tends to increase academic burnout, and an intrinsically motivated student is less susceptible to the development of this syndrome [57].

According to [60] extrinsic motivation comes from outside a person; that is, the person works to obtain external rewards, money being the most common of them. Intrinsic motivation, on the other hand, is the internal desire to do something, the person being

driven by interest, satisfaction, and the challenge of his own work, and not by external pressures.

For the author, intrinsic motivation is more important in creativity and in the work context than extrinsic motivation. In the same direction, studies by [61] corroborate the idea that intrinsic motivation plays an important role in overcoming the negative cycle of burnout. Additionally, according to [62], individual motivations have a considerable influence on the burnout cycle. Other authors such as [63] have even shown that the relationship between workload and burnout in university students is almost non-existent, but that academic motivation can effectively prevent burnout.

Therefore, the role that motivation plays in learning seems to be indisputable, namely, because it is responsible for the good involvement of students in academic activities, which require effort and dedication [64–66].

Refs. [67,68] point out four major sets of motivational theories as those that can explain the role of motivation in learning: (1) the first, focuses on beliefs about competence and expectations of students, both in terms of their success as well as their failure to obtain results; (2) the second, focuses on the motivations that lead individuals to perform certain activities (that is, the objectives with which they perform them, the intrinsic and extrinsic motivation or the values that motivate their involvement in these activities); (3) the third, encompasses students' expectations and values; and (4) the last set of motivational theories takes into account the relationship between motivational and cognitive processes.

It is crucial to understand motivation as a multidimensional construct that is explained based on the influence of the social environment and cognitive components [64–66], being a variable with a very important role both in theory and in practice, and which parents, guardians, teachers, and educators themselves often turn to, in order to clarify students' behaviors, their school performance and the quality of their learning [69–71].

Albeit it is also important to realize that academic motivation is a process that requires activation, direction and persistence, and in which the components of value, expectation and affection play a major role [72]. Note, for example, that when a student is confronted with a particular academic task, he may wonder about the reason or objective for which he should perform it; the greater the interest and usefulness you see in it, the greater the value you will attach to it and the greater your involvement in it. At the same time, the student will feel, or not, capable of carrying out the task in question, taking into account his personal resources, which may not be sufficient, making it necessary for him to be able to cultivate a certain belief that he is qualified to perform everything that this task implies. Finally, the third dimension of motivation to consider is the affective component; that is, the one according to which the student gets involved in a task and assesses the feelings and emotions that it provokes: if he feels confident, he will have the courage to complete the task successfully; if the task provokes anxiety, tension, or a bad mood, the student is more likely to abandon it early [73].

Refs. [74,75] even mention that motivation is a predictor of school performance as relevant as intelligence, and [76] add that one of the reasons why students apply themselves in tasks and learning activities, is related to the academic goals that direct their behaviors, since these are considered key predictors of performance [77].

In addition, it is expected that, as the student progresses in his studies, he will present different goals of achievement, due to his perception of the various curricular disciplines or the relevance of their contents to his vocational projects [69,71] In this sense, [77,78] concluded that students oriented towards intrinsic goals have a better academic performance, while students whose goals focus on the self, end up showing worse performance.

Consider also what [79] refer about motivation in adolescence. According to these authors, in this phase of life, motivation is evidenced by different patterns depending on the gender of the students, which influences the way they approach the tasks they face, especially in the case of the female sex, which seems to have high levels of motivation for work. Age also appears to be a variable that influences motivational patterns.

Based on the theoretical model of [80], tested by [81], ref. [70] then developed the Inventory of Academic Goals (IMA), which was adapted and validated for basic and secondary education, and for higher education teaching (IMA-ES), in order to assess students' motivation. The IMA for Basic Education and for Secondary Education consists of a self-report instrument composed of 22 questions randomly distributed over four dimensions: (1) learning-oriented goals; (2) goals aimed at avoiding social pressure in the school context; (3) goals oriented towards concrete objectives; and (4) goals aimed at avoiding social pressure in the family context. The IMA-ES also consists of a self-report instrument, this one with 17 items distributed over three dimensions: (1) learning-oriented goals; (2) goals aimed at avoiding social pressure; and (3) performance-oriented goals [82].

In this inventory, the learning goals are related to intrinsic motivation; that is, when the student learns the contents taught in order to improve their knowledge and obtain new skills and abilities. The goals aimed at avoiding social pressure result from learning with the aim of obtaining the approval of family members, teachers, or peers. Finally, performance-oriented goals are associated with the student's tendency to study only with the objective of obtaining good academic performance, that is, good grades [82].

There is unanimity on the part of the researchers with regard to the various variables that interfere with the student's academic performance, it being possible to classify them as institutional, personal and social determinants. However, it could seem evident that students with high levels of exhaustion, a greater sense of disbelief and less professional effectiveness obtain poor results in their exams [4,35,83,84] disagree with these studies and claim that burnout is not a good predictor of lower academic performance.

It is important to say that this link (not always clear) between academic performance and burnout can be justified by the use of different methods in the evaluation of performance, carried out in divergent investigations, namely, the fact that some empirical studies use the academic arithmetic mean as the sole criterion of analysis, which is notoriously insufficient [9].

Despite this, most studies are unanimous in emphasizing that burnout is negatively related to satisfaction with studies [4], professional maturity [85], the desire to abandon studies [34,86], happiness in the face of the academic path [34], and expectations of successful learning [87].

In conclusion, [9] state that the effect of burnout on academic performance is not yet fully explained, but that it appears as a relevant indicator in the decline of numerous aspects associated with performance and satisfaction with the studies themselves, as in the forecast of school dropout by the university student. In summary, the various empirical studies tend to identify that students who suffer from burnout result in poor academic performance.

#### *2.5. Academic Burnout in the Portuguese Context: What Do the Studies Say?*

With regard to Portugal, [88] carried out a study with the objective of determining the incidence of burnout in Portuguese higher education. Therefore, a questionnaire was distributed by the universities of the 20 districts of Portugal (including the universities of the archipelagos of Madeira and the Azores), through the Qualtrics platform and through the student associations of the respective institutions. This questionnaire was based on the Maslach Burnout Inventory Student Survey (MBI-SS), which will be explained below, and included 15 ordinal self-report items, whose response scale varied between "0—Never or No time" to "6—All days or Always".

According to this same study, 64.9% of the students at the University of Aveiro suffer from burnout, the highest value in all the analyzed universities; by contrast, at the Polytechnic Institute of Viana do Castelo only 15% of students suffer from burnout. Beja, Viana do Castelo, Faro, Portalegre, Vila Real, and Bragança are districts with average levels of burnout. Setúbal, Aveiro, Évora, Coimbra, Porto, Santarém, Lisbon, Viseu, Castelo Branco, Braga, and Leiria are the districts where students have the highest levels of burnout.

Ref. [88] conclude that all districts have average levels of academic involvement, with students in the Exact Sciences courses showing the lowest levels of academic involvement and the second highest burnout value among the different areas of study analyzed in the referred work (Human Sciences, Exact Sciences, Health Sciences, and Biological Sciences). In this study, Biological Sciences (at the top), the Exact Sciences and the Health Sciences are those with the highest levels of burnout. On the other hand, university students in courses in the Humanities area are those who have a greater academic involvement and those who suffer less from burnout.

The authors also concluded that female students have marginally higher values of both involvement and burnout, compared to male students.

Finally, ref. [88] point out that psychological variables, such as academic involvement and burnout, are absolutely determinant factors for the academic success of university students, being directly related to students' self-efficacy, as well as their own performance problems and school dropout.

#### **3. Research Hypotheses**

In the present research work, the following research hypotheses were defined:


*hand, students who choose the course by vocation and/or taste have lower levels of burnout than those who did not choose the course for the aforementioned reason;*


#### **4. Materials and Methods**

We shall now present the methodology and procedures adopted in the present study. Thus, the present work follows a quantitative data collection methodology, more precisely through the application of a questionnaire to students of the Portuguese public university in question.

The literature review on the topic was made based on a previous documentary research.

The objective of the study was to verify how many students experience academic burnout and take (or have already taken) medication due to the requirements of the academic environment.

In the first question of the questionnaire, it was questioned whether the participants studied (or not) at the analyzed public institution. If the participants did not study at that institution, they were sent to the end of the questionnaire and could not answer any more questions. Only participants who studied at the public institution analyzed could complete the entire questionnaire and were sent to the following questions.

The sample of this study is non-probabilistic and is also a convenience sample (not entirely random as the authors used their connections and networks to reach as many students as possible, in the Portuguese public university, having also been aided by various student bodies and associations, as described below); its approach was intended to provide a broad view of the academic environment in question, in order to achieve the objective of the study.

#### *4.1. Participants*

In total, 207 students from the analyzed university participated in the study, who were asked to answer a questionnaire, of which, 90% of the participants were between 18 and 25 years old and about 8% were between 26 and 35 years old, with about 31% of the participants being male and 69% female. It should also be noted that 91% of respondents were of Portuguese nationality, and 100% were resident in Portugal, with around 74% attending the university under analysis and the remaining 26% of respondents attending polytechnics at the same institution (the institution has four polytechnics). Of these students, the majority, 61%, have a first cycle (degree).

Regarding the study area, it was decided to divide the courses into four main study areas, namely, Biological Sciences, Health Sciences, Exact Sciences, and Human/Social Sciences, to facilitate the interpretation of the results by the reader, as well as the statistical analysis, and in order to meet the study carried out by [88], with whose investigation it is intended, in the end, to compare the results.

Thus, most students, about 32%, attend courses related to the area of Human/Social Sciences, 28% attend courses in the area of the Exact Sciences, 23% in the area of Biological Sciences, and about 17% are in the Health Sciences area. Most students, 29%, are in the 1st year and around 24% are in the 2nd year of higher education (in their degree). About 56% of the students have an arithmetic average of between 13 and 15.

Only 27% of the surveyed students had a paid professional occupation. Most of the students in this study, 33%, have a gross monthly income between 1000 and 1499 euros. However, about 25% have a gross monthly income between 1500 and 2499 euros and 24% have an income between 600 and 999 euros.

#### *4.2. Data Collection Instruments*

In order to understand if university students of the chosen educational institution experience academic burnout, as well as if they take (or have already taken) medication during their academic career and to identify their feelings regarding the course and the institution they attend, a questionnaire was applied to university students from a Portuguese public higher education institution. The survey consists of 55 closed-answer questions and four open-answer questions (Appendix A).

This questionnaire uses some questions from the Maslach Burnout Inventory—Student Survey (MBI-SS), adapted by [51], translated and adapted to the Portuguese language by [31]. It is a self-report scale consisting of 15 questions referring to the three subscales of academic burnout (emotional exhaustion, disbelief, and professional effectiveness) and the feelings and emotions that students experience in the school context.

Here, respondents express how often they feel what each of the 15 questions suggests, on an ordinal seven-point scale, which ranges from 0, "never" to 6, "always". The reliability of the factors was assessed by measuring the internal consistency of the Cronbach's alpha coefficient (α) and, in its entirety, the MBI-SS scale presents an α = 0.79 [89].

Since the central point of the scale is "regularly", the total of the burnout score was calculated using the average of the questions of the three subscales, considering that the students presented burnout when this total score was higher than three, as defended by [88]. Regarding the three subscales, for the purposes of this study, questions 38 to 42 refer to the emotional exhaustion subscale, questions 43 to 46 refer to disbelief and, finally, questions 49 to 54 refer to professional effectiveness.

In the subscale dedicated to professional effectiveness, the higher the values, the lower the levels of burnout; that is, the higher the values, the lower the ineffectiveness of individuals. For this reason, the response scale for the items in this subscale has been inverted, as suggested by the study by [90] and as is done in the study by [88], a study with which we intend, in the final stretch, to compare the results of the present study. The remaining questions were elaborated from scratch, based on the consulted literature.

It should also be noted that, initially, sociodemographic information was collected through the questionnaire.

#### *4.3. Data Collection Procedures*

In the study, a questionnaire on burnout and taking medication was distributed to university students.

After elaborating the questionnaire, tests were carried out on the data collection instruments, asking education specialists to evaluate the questionnaire in terms of the time to fill it in and the clarity of the questions, and they were also asked whether or not they had learned anything useful in order to understand whether it would be necessary to make any changes to the questionnaire.

After validation and the elaboration of the necessary corrections, the associative nuclei of the various courses, sports nuclei and administrative services of the university and the polytechnics under analysis were contacted, as well as organizations created in the academic environment, in order to request the dissemination of the questionnaire to the students. The questionnaire was disseminated through social networks via the centers and services mentioned.

It should also be noted that the data collection was carried out respecting all applicable ethical principles, especially with regard to the anonymity of the respondents and the confidentiality of the investigation. There was an incentive for student participation through the drawing of tickets for the RFM SOMNII summer festival at Praia do Relógio in Figueira da Foz (a three-day general pass).

The questionnaire was online (online Google form format) since 30 March 2020 and until 30 April 2020, having obtained 207 responses, all of which valid.

#### *4.4. Statistical Procedures*

To perform the analysis of the quantitative data, the program IBM SPSS Statistics, version 25 was used. Inferential statistics were used, namely, Student's *t*-test and one-way ANOVA (parametric tests), Spearman's correlation coefficient, and the Chi-square test to test previously defined research hypotheses.

The variables were characterized using the mean and standard deviation. The level of significance was set at a value of *p* < 0.05. Regarding Spearman's Correlation Coefficient, it is also worth noting that [91] refers that correlations above 0.40 are considered strong, while correlations that vary between 0.20 and 0.40 are considered moderate. Correlations below 0.20 are considered weak. This was precisely the criterion considered in the analysis carried out in the present investigation.

#### **5. Results**

As the values of the asymmetry (*Sk* = 0.219) and kurtosis (*Ku* = −0.522) coefficients of the burnout score are less than one, it is possible to assume that the data have an approximately normal distribution, as can be seen in Table 1. Thus, parametric tests can be used, such as Student's *t*-test and ANOVA [92,93].

**Table 1.** Descriptive analysis of the burnout score (*n* = 207).


Regarding the reliability analysis, the Alfa Cronbach coefficients were calculated for the burnout score (α = 0.901) and for the respective subscales: emotional exhaustion (α = 0.929), disbelief (α = 0.916) and professional effectiveness (α = 0.830).

The values of the coefficients are above the cut-off line (0.70) and the values of the item-total correlations corrected above 0.30. It was also verified whether the output of an item improved the internal consistency of the scale (Cronbach's Alpha without the item), as suggested by [92], as shown in Table 2.



As mentioned, it was considered that students had burnout when the average score of all questions was higher than three. It is concluded that, in the total sample, 29% of the students surveyed have burnout. It is also observed that about 37% of the students are cognitively and emotionally exhausted, 17% have a cynical attitude towards their studies, colleagues and teachers, and about 36% feel incapacity and professional inefficiency.

#### *5.1. Relationship between Burnout and Sociodemographic Variables*

Student's *t*-test was used to verify if there were differences in academic burnout due to sociodemographic variables.

There are no statistically significant differences between men (*M* = 2.54; *SD* = 0.94) and women (*M* = 2.45; *SD* = 0.92) regarding burnout (*p* = 0.531). Thus, it is concluded that sex does not influence the levels of burnout.

In order to carry out this analysis, age was recoded in only two classes: from "18 to 25 years" and "26 years or more", since the respondents were concentrated in the greatest number in these age groups and to facilitate the organization of data; additionally, there are also no statistically significant differences with regard to burnout according to the age group (*p* = 0.156), which means that students between 18 and 25 years old do not have higher levels of burnout (*M* = 2.51; *SD* = 0.91) than students aged 26 or older (*M* = 2.20; *SD* = 1.01).

To carry out this analysis, the variable educational qualifications was recoded into two categories: "Degree" and "Other higher qualifications". With regard to educational qualifications, the differences are close to statistical significance (*p* = 0.065), that is, there seems to be a trend towards higher levels of burnout in undergraduate students than in students of other study cycles. However, it cannot be considered that there are statistically significant differences between the first cycle (*M* = 2.57; *SD* = 0.96) and the academically higher study cycles (*M* = 2.33; *SD* = 0.85), in the levels of burnout.

Finally, note that there are differences in academic burnout depending on the student's professional situation, and students who do not work (*M* = 2.56; *SD* = 0.94) have higher levels of burnout (*p* = 0.046) than students who are student workers (*M* = 2.27; *SD* = 0.85).

#### *5.2. Relationship between Burnout and Student Participation in Extracurricular Activities and Physical Exercise*

Student's *t*-test was used to verify if there were differences in academic burnout due to the participation of students in extracurricular activities and due to the practice of physical exercise.

As it turns out, there are statistically significant differences in academic burnout according to the participation of students in extracurricular activities (*p* = 0.015). Students who do not participate in extracurricular activities have higher levels of burnout (*M* = 2.70; *SD* = 0.96) than students who participate in these activities (*M* = 2.37; *SD* = 0.89).

As far as physical exercise is concerned, there are statistically significant differences in academic burnout (*p* = 0.021). Students who participate in sports activities show lower levels of burnout (*M* = 2.42; *SD* = 0.87) than those who do not practice any type of physical exercise (*M* = 2.70; *SD* = 1.07).

#### *5.3. Relationship between Burnout and the Transition from Secondary Education to Higher Education, Choice of Course, and Professional Future*

Student's *t*-test was used to check if there were differences in academic burnout regarding the transition from secondary education to higher education and the choice of course and professional future.

In fact, there are statistically significant differences in burnout due to the students' perception of the negative impact of sudden freedom brought on by the transition from secondary education to higher education (*p* = 0.015). Students who consider that this freedom has affected them negatively while students have higher levels of burnout (*M* = 2.88; *SD* = 0.84) than students who consider that this aspect has not affected them negatively (*M* = 2.42; *SD* = 0.92).

In addition, there are statistically significant differences in academic burnout according to the reasons that led students to choose the course they attend. Students who chose the course they attend based on the employability rate (*M* = 2.49; *SD* = 0.86) do not have higher levels of burnout (*p* = 0.917) than those who did not choose this option (*M* = 2.47; *DP* = 0.95). However, students who choose the course by vocation and/or taste (*M* = 2.42; *DP* = 0.92) have lower levels of burnout (*p* = 0.038) than those who did not choose the course for this reason (*M* = 2.80; *SD* = 0.93).

There are also statistically significant differences in the levels of burnout according to the uncertainty felt in relation to the professional future (*p* = 0.004). After all, students who feel uncertain about their future work show higher levels of burnout (*M* = 2.56; *SD* = 0.93) than those who do not feel any uncertainty (*M* = 2.04; *SD* = 0.79).

#### *5.4. Relationship between Burnout and the Existence of a Recent Diagnosis of Depression, Intake, and Type of Medication*

Student's *t*-test was used to verify if there were differences in academic burnout due to the existence of a recent diagnosis of depression and taking medication.

Therefore, it was observed that there are statistically significant differences in burnout due to the presence of a diagnosis of depression (*p* < 0.001), since students who were recently diagnosed with depression have a higher prevalence of this syndrome (*M* = 3.40; *SD* = 0.73).

There are also statistically significant differences in academic burnout due to taking medication (*p* < 0.001), with students taking medication having higher levels of burnout (*M* = 2.72; *SD* = 0.90). Students who take antidepressants have higher levels of burnout than those who do not take this type of medication (*p* < 0.001). In turn, students who take anxiolytics (*p* < 0.001), food supplements/multivitamins (*p* = 0.024), and sleeping medication (*p* = 0.046) have higher levels of burnout compared to those who do not take this type of medication.

In addition, there are statistically significant differences in academic burnout according to the reason for taking medication. For example, students who take medication to combat symptoms of emotional exhaustion (*p* < 0.001) and to improve their academic achievement (*p* = 0.010) have higher levels of burnout when compared to students who do not take it for these reasons.

It should also be noted that there are statistically significant differences in academic burnout due to the fact that students know (or do not know) other colleagues who take sleeping pills, since students who do not know colleagues who take this type of medication have higher levels burnout (*p* = 0.021).

#### *5.5. Relationship between Burnout and the Tendency to Drop Out of the Course or Studies and Aspects to Improve Psychological Well-Being*

Student's *t*-test was used to check if there were differences in academic burnout due to the tendency of students to drop out of the course or studies and the aspects to improve their level of psychological well-being in the school they attend.

In fact, it is noticed that there are statistically significant differences in burnout according to the fact that the student has considered giving up the course or studies, concluding that students who have considered giving up the course (*p* < 0.001) or studies (*p* < 0.001) have higher levels of burnout.

There are also differences in burnout due to the aspects that students consider that would help to improve their level of psychological well-being at the university. In fact, students who report that the reduction in the number of hours of daily and weekly classes (*p* = 0.005), the reformulation of teaching methods (*p* < 0.001), and a wider range of curricular options (*p* = 0.048) would improve their psychological well-being have higher levels of burnout.

Regarding the option concerning the availability of other means that provide psychological support to students, the differences are close to statistical significance (*p* = 0.071).

#### *5.6. Relationship between Burnout and Study Areas, Year of Higher Education and Arithmetic Mean of Course*

ANOVA was used to check if there were differences in academic burnout according to the areas of study of the students, the year of higher education they attend and their arithmetic average (grades) of their course.

There are no statistically significant differences between the study areas regarding burnout (*F* = 1.379; *p* = 0.250). In other words, the study area has no influence on burnout levels, which means that students in the areas of Biological Sciences, Exact Sciences, and Health do not have higher levels of academic burnout than the rest.

In addition, there are also no statistically significant differences in academic burnout according to the year of higher education they attend. Students in the first and last years of college do not have higher levels of burnout (*F* = 0.776; *p* = 0.542), when compared to students who attend the remaining years. In other words, the year of higher education does not influence the levels of burnout.

Regarding the arithmetic mean of course, there are statistically significant differences in academic burnout (*F* = 3.369; *p* = 0.036). Thus, students with a low average, from 10 to 12 points (*M* = 2.71; *SD* = 0.98) have higher levels of burnout than students with a higher average, from 16 to 20 points (*M* = 2.22; *SD* = 0.96). In sum, the multiple comparisons tests show that there is a trend towards a decrease in burnout levels with the increase in the arithmetic mean of the course (*p* = 0.029).

Note, however, that, for this analysis, the arithmetic mean variable of the course was recoded into three categories: "From 10 to 12 points", "From 13 to 15 points" and "From 16 to 20 points" given the classes in which the greatest number of responses were concentrated and to facilitate the organization of the data.

#### *5.7. Relationship between Burnout and the Frequency with Which You Take Medication*

ANOVA was used to check if there were differences in academic burnout depending on the frequency with which he takes medication.

There are statistically significant differences in academic burnout according to the frequency of taking medication (*F* = 5.449; *p* < 0.001). Multiple comparison tests show that students who take this type of medication about once a month or once a week have significantly higher levels of burnout than students who do not take it (*p* = 0.006).

It should be noted that in order to carry out this analysis, the variable of the frequency with which students take medication was recoded into four categories: "Only during the assessment periods", "About once a month or once a week", " I do not take any medication" and "Every day" as these are the categories in which the greatest number of responses were concentrated and to facilitate the organization of the data.

#### *5.8. Correlation between Academic Burnout and Students' Gross Monthly Income, Frequency with Which They Practice Physical Exercise, and Their Initial Expectations Regarding the Course*

Spearman's Correlation Coefficient was used to check if there was a correlation between academic burnout and the students' gross monthly income, the frequency with which they practice physical exercise, and their initial expectations in relation to the course.

In view of the data obtained, it was concluded that there is no statistically significant association between gross monthly income (*rsp* = −0.020; *p* = 0.776) and burnout; that is, that a lower gross monthly income is not necessarily associated with higher burnout levels.

On the other hand, there is a weak negative correlation, statistically significant, between the frequency with which students practice physical exercise and the levels of burnout (*rsp* = −0.170, *p* = 0.014); that is, the higher the frequency with which they exercise, the lesser are their burnout levels.

It can also be said that there is a moderate negative correlation, statistically significant, between the classification that students attribute to the course they entered in view of their initial expectations and academic burnout. In fact, the higher the student's initial expectations, the lower the levels of burnout (*rsp* = −0.267; *p* < 0.001).

#### *5.9. Correlation between Academic Burnout and the Perception of the University's Material Conditions, the Classification Given by Students to Teachers, and Their Relationship with Them*

The Spearman Correlation Coefficient was used to check if there was a correlation regarding academic burnout due to the perception, on the part of students, of the material conditions of the university attended, as well as the burnout and the classification they attribute to their teachers and their relationship with them.

There is a weak, statistically significant negative correlation between burnout and the perception of the material conditions of the university attended; that is, when students have a perception that the material conditions of their university are superior, they suffer less from burnout (*rsp* = −0.137; *p* = 0.048).

In addition, there is a moderate, statistically significant, negative correlation between burnout and the rating that students assign to their teachers (*rsp* = −0.349; *p* < 0.001). It can therefore be said that students who give teachers better ratings have lower levels of burnout.

On the other hand, there is no association between burnout and the classification that students attribute to the relationship with their teachers (*rsp* = 0.023; *p* = 0.745). This means that higher ratings regarding the relationship between students and teachers are not associated with lower levels of burnout.

#### *5.10. Correlation between Academic Burnout and the Relationship of Students with Their Colleagues, the Ability to Withstand the Pressure of Studies, and the Classification of Their Performance*

Spearman's Correlation Coefficient was used to check if there was a correlation regarding academic burnout and the way students classify their relationship with colleagues, their ability to withstand the pressure caused by studies and the way they classify their academic performance.

There is a weak, statistically significant negative correlation between burnout and the rating given by students to their relationship with colleagues (*rsp* = −0.188; *p* = 0.007), that is, the higher the rating, the lower the levels of student burnout.

In addition, there is a strong, statistically significant negative correlation between burnout and the fact that students know (or do not know how) to withstand the pressures of their study cycle. That is, the more students think they are able to withstand the pressures of their study cycle, the lower the levels of burnout (*rsp* = −0.522; *p* < 0.001).

There is also a moderate, statistically significant negative correlation between burnout and the classification of students' academic performance. This means that the lower the rating of their academic performance, the higher the levels of burnout (*rsp* = −0.366; *p* < 0.001) that they suffer.

#### *5.11. Relationship between Medication Intake and Course Arithmetic Mean*

The Chi-square test was used to check if there was an association between taking medication and the course arithmetic mean (the grades of the student) (Tables 3 and 4).


**Table 3.** Calculation of the chi-square.

**Table 4.** Calculation of the chi-square.


Degrees of freedom: 2.

Critical value from the table at the 5% level: 5.991 (which is greater than the calculated critical value 3.44).

Conclusion: there is no association between the variables. The variables are independent—there is no statistically significant relationship between the arithmetic mean of the course obtained by the students and medication intake.

#### **6. Discussion**

Among the sociodemographic variables studied, the potential effects on the burnout levels of sex (H1), age (H2), and gross monthly income (H8) were investigated based on the formulated hypotheses.

For the sample analyzed, it was not possible to attribute an influence of the gender variable to the levels of burnout. Thus, hypothesis 1 was not validated, contrary to the results of [9,88]. Note, however, that the former found only marginally higher levels of burnout in females, and that the latter attributed risk factors associated with personality characteristics to the sexes, namely, greater perceived effectiveness on the part of women, and more cynical about studies by men. Although both studies may indicate an apparent trend towards their results, the influence of sex on burnout levels does not seem clear, which was also evident in the present study. To confirm this trend, further studies would be needed, with larger and standardized samples.

Additionally, the age of the students did not reveal any influence on the burnout levels, although the literature refers that one of the groups most affected by the syndrome is the millennials [94]. In this study, higher levels of burnout were not found among students aged 18 to 25 years, when compared with those of students aged 26 or older, so research hypothesis 2 was not validated. Remember that the definition of millennial is not consensual, so the age range considered in this study may not be representative of that generation. On the other hand, the entire sample consisted of university students and, regardless of the age range referring to millenials, not all will be university students, so it would be necessary to extend the study universe to extra-university realities in order to be able to correctly measure the levels of the generation as a whole.

According to the research hypothesis 8, it was expected that lower gross monthly income would be associated with higher levels of burnout, as suggested by [33], who emphasize that the lack of financial support is a variable associated with academic burnout. Studies such as those by [32] point to the fact that the related social and professional pressures with higher education funding being a factor that makes university students more vulnerable to academic burnout. However, this was not verified in this study, since a lower gross monthly income was not significantly associated with higher levels of burnout; thus, not validating hypothesis 8. The financial support that the University in analysis provides and the fact that it is located in a geographical area where the cost of living is not high can explain this result.

With regard to academic variables, the educational qualifications (H3), the study areas (H4), the year of higher education attended (H5), the arithmetic average of the course (H6) and the professional situation were considered for analysis (H7).

No significant differences were found in the burnout syndrome between educational qualifications, areas of study or year of higher education attended; therefore, hypotheses 3, 4, and 5 were not validated. The fact that there are no significant differences in relation to educational qualifications and the year in higher education can be explained by the possible uniformity of academic load throughout the university course; that is, students continue with the same workload and suffer from stress throughout the course. As it turned out, in the course of their training, it is understood that students are, on the one hand, reformulating their goals, and, on the other, being confronted with the reality of their course and the profession they intend to exercise as mentioned by the authors [69,71,95]. These perceptions that are being fed may lead students to continually renew the pressure on themselves, or that, on the contrary, they will gradually lose their motivation and enthusiasm for the path they have been following. In any case, stress levels can be replenished and thus maintained, for different reasons, throughout the course.

Ref. [88] found that students of courses in the area of Biological and Exact Sciences are the ones who reveal the highest levels of burnout among the various areas of study and that those in the Humanities are those who suffer less from burnout; this was not evidenced by the present study. This result can be explained by the distribution of the study areas in the sample, it being verified that the majority of respondents, about 32%, were studying in the area of Human/Social Sciences, 28% in the area of Exact Sciences, around 17% in the area of Health Sciences and the remaining 23% were in areas related to Biological Sciences. In the study by [88], 33% attended courses in the Humanities area, about 30% in the Exact Sciences area, 27% in the Health Sciences area, and only about 10%

in the Biological Sciences area. The main differences in the two distributions are, therefore, in the areas of Health Sciences and Biological Sciences; that is, in the present study there was an excess of 13% of students in the area of Biological Sciences and 10% less in the area of Health Sciences.

As for the arithmetic mean, it was found that students with a lower average, from 10 to 12 points, have higher levels of burnout than students with higher averages, from 16 to 20 points. This result validates hypothesis 6 and is in line with what is reported by the authors [45,46], who indicate that the syndrome is related to poor cognitive performance and decreased academic achievement. It is understandable that this is the case, since academic burnout, as we have seen, is characterized precisely by a lack of motivation to study and perform the required tasks, in addition to physical symptoms such as constant tiredness and fatigue and migraines, thus impacting the output of students who suffer from it.

The influence on burnout levels was also analyzed according to the professional situation, verifying that students who do not work have higher levels of burnout than those who are worker-students. Thus, hypothesis 7 has not been validated. It would be plausible to consider that student workers could have higher levels of burnout, compared to students who do not engage in any paid professional activity, because they have to reconcile their studies with their profession, and are therefore faced with more challenges. However, according to the results obtained in the present study, and since no references to similar analyses were found in the revised literature, the possibility of students with worker-student status having better developed or adapted mechanisms such as coping and emotional intelligence, managing to better manage stress in relation to students who have less challenges to manage.

In addition to the sociodemographic and academic variables analyzed above, we tried to understand the impact that students' perceptions about the transition from secondary to higher education, and their options regarding the course, can have on the level of burnout, considering that the transition experience is an imminent cause of stress during the first year of post-secondary studies [28]. In fact, it was proven that there are differences in the manifestations of academic burnout due to the students' perception about the negative impact of the sudden freedom brought by the transition from secondary education to higher education, thus validating hypothesis 12. Students who consider that this freedom has affected them in a negative way have higher levels of burnout, as [28] refer. Freedom brings responsibility and, in this sense, young people may not yet be mature enough to properly manage emotions, relationships and even studies, without a support network as tight as up until then; that is, there are lower levels of emotional intelligence and resilience that, as mentioned earlier, act as a kind of protection against the syndrome.

The conditions of the educational institution itself are also pointed out in the literature as factors related to the burnout syndrome [9,10,33]. It was found, in the present investigation, that students who have a perception that the material conditions of their university are superior suffer less from burnout. Additionally, the higher (better) the relationship with their colleagues, the lower the burnout levels of students. Thus, hypotheses 15 and 18 were validated, which is in agreement with the conclusions of [36], regarding the contribution to the appearance of burnout of factors such as reduced solidarity and companionship, competition and disagreements with colleagues. However, higher classifications regarding the relationship between students and teachers were not, in this study, associated with lower levels of burnout, contrary to what was indicated by the authors [36], who report that poor relationships between students and teachers and the lack of feedback from teachers, are academic social risk factors that can precede or predict the appearance of burnout. Therefore, hypothesis 17 was not validated.

Another factor that is pointed out in the literature as a predictor of academic burnout felt by students is the competence of their teachers [96]. Further, in the present study, it was concluded that students who attribute better ratings to their teachers show lower levels of burnout, which validates hypothesis 16. Students who feel that teachers are not competent may tend to develop a more cynical attitude towards the course, and/or become discouraged, feeling that the discipline or course does not meet expectations, which, in turn, may influence your motivation and commitment to studies. Students who are unmotivated and with worse academic performance are more susceptible to developing the syndrome.

The impact that the reasons for choosing the course can have on the levels of burnout was also analyzed, concluding that the students who chose the course they attend taking into account the employability rate do not present significantly higher levels of burnout than those for which the choice did not consider this criterion. However, students who chose the course by vocation and/or taste have lower burnout levels than those who did not choose the course they are attending for that reason. Therefore, hypothesis 13 was partially validated, and the assumption that the vocation or taste for the course may be linked to greater motivation on the part of the students, and thus constitute a protective factor against this syndrome as mentioned in the literature seems plausible. Remember that [95] affirm that the choice of the profession to be exercised is not fully conscious, ending up not being a factor directly linked to the students' decision. Factors which do weigh, the authors point out, for example, are economic reasons, reputation or the desire to help others.

In the present study, the correlation between burnout and the fact that students feel they know (or not) how to withstand the pressures of their study cycle was also assessed. The more students think they are able to withstand the pressures of their study cycle, the lower their burnout levels. In addition, students who report that the reduction in the number of hours of daily and weekly classes, the reformulation of teaching methods and a wider range of curricular options as factors that would improve their psychological well-being, in the university context, present greater burnout levels. Hypotheses 28 and 30 have therefore been validated.

Ref. [96] refer to the low expectations of students with regard to their course as one of the factors that most result in burnout in Portuguese university students. Ref. [95] state that, by understanding, during their academic career, that the course or future profession does not meet the expectations created, the student develops a sense of disappointment that, consequently, can also lead to frustration, discontent and stress, with a tendency to increase continuously, and lead to a burnout scenario. If the scenario gets worse, and if effective coping strategies and their implementation are neglected, a burnout scenario will most likely arise. Similar results were obtained in the present study, since it was concluded that the higher the initial expectations of the student, the lower the levels of burnout, therefore hypothesis 14 was validated.

Issues related to employability and the perception of the absence of offers in the labor market were also pointed out as factors that generate stress and anxiety in modern society and associated with academic burnout [6,33]. The present study also found that students who feel uncertain about their future work show higher levels of burnout than those who do not manifest this uncertainty, thus validating hypothesis 19. This uncertainty felt by students may be partly explained by the crisis that we are facing, as a result of the COVID-19 pandemic, namely, regarding its repercussions in the labor market, in which it was already difficult for graduates to enter.

In Portugal, young people have more and more academic training, according to the data in [97], which show that, in 2019, there were 83,193 individuals with recent degrees (students who completed their bachelor's, master's, doctorate, or specialization course in that year). In Portuguese society, until the 1980s, having a higher education was associated with ensuring a well-paid job. However, the widespread entry into higher education and the resulting increase in graduated individuals caused significant changes in this scenario. Portugal has a small and diversified economy, there is little labor, and this reality, combined with constant advances in technology and automation, contributes to the difficulty of finding jobs by young people. Most young people are unable to obtain employment in their area of training, or if they do, it is poorly paid. Nowadays, the number of young people who have a regular full-time job is decreasing, and those who have it are hardly able to achieve

a degree of security in the long term; that is, a large part survives through occasional jobs in the short term (part-time), with little or no guarantees and contractual rights, which results in the possibility of losing jobs unexpectedly due to economic crises and/or the employer (being fired with no forewarning). These situations provoke feelings of insecurity, uncertainty and fear in this generation, with serious psychological consequences. Due to the social and economic aspects mentioned, most young people postpone the decisions and responsibilities inherent in adult life, remaining for long periods semi-dependent on their parents, with all the social and personal repercussions that arise from it [98]. According to [99], the youth unemployment rate in the European Union increased by two percentage points between 2019 and 2020, reaching 17%.

Other factors considered important in preventing burnout are extracurricular activities and physical exercise. This study points in this direction, since it was concluded that students who participate in extracurricular activities or practice physical exercise have lower levels of burnout, and, in addition, the greater the frequency with which they practice physical exercise, the lower these levels are. These results allow us to validate the research hypotheses 9, 10 and 11, and are congruent with those obtained by [36], who showed that non-participation in cultural or recreational activities can precede or predict the appearance of burnout.

As already mentioned, burnout syndrome is related to depressive symptoms [47] and even suicidal thoughts [13]. The analysis of the data in the present study showed that students who were recently diagnosed with depression have higher levels of burnout, so hypothesis 20 was validated.

The need to take medication was pointed out by [96] as a predictor of academic burnout experienced by university students. In the present study, it was found that students who take medication show higher levels of burnout, and those who take it about once a month or once a week have significantly higher levels of burnout than students who do not take any medication. Regarding the type of medication, those who take antidepressants have higher levels of burnout than those who do not; those who take anxiolytics or sleeping medication, or food supplements/multivitamins, have higher levels of burnout compared to those who do not take this type of medication. In this study, it was also determined that students who take medication to combat the symptoms of emotional exhaustion (related to burnout) and to improve their academic performance have higher levels of burnout when compared to students who do not take it for these reasons. Thus, research hypotheses 21, 22, 23, and 24 were validated. This result can be justified by the social pressure that is exerted by society and by family members (related to the desire that the students conclude their study cycles successfully), and also by the increasing responsibility [3], economic pressures, cultural, and even aspects related to the students' personality [4]. It would be pertinent to understand if the levels of burnout are higher in students who take more than one type of medication at the same time.

It should also be noted that students who do not know other colleagues who take sleeping medication have been shown to have higher levels of burnout, which validates hypothesis 25. A possible explanation for these results may be the fact that difficulty in sleeping and/or the medication taken are felt as taboo topics, and not knowing specific coping strategies via such medication may be detrimental (given the importance of a good night's sleep).

Note, however, that no relationship was found between the arithmetic mean obtained by students and taking medication. Therefore, hypothesis 26 has not been validated. Considering, as indicated above, that taking medication can be enhanced by the social pressure exerted, this result can be justified by the idea that this pressure is exerted on students whatever their course average; that is, if the student has a lower average, you will feel pressure to go up; if it is average, you may be required to improve it to reach the highest level; if it is already high, you will feel pressure to maintain it.

Ref. [96] point out the student's subjective academic performance and the intention to give up his study cycle as relevant predictive factors for the academic burnout felt by

students. In this sense, it was found, in the present study, that students who consider giving up the course or studies have higher levels of burnout, and therefore hypothesis 27 was validated. Regarding the classification of their academic performance, it was proved that the better they classify their academic performance, the lower the burnout levels of the students, which validates the hypothesis 29.

Finally, hypothesis 30 was also validated. There are differences between academic burnout and the fact that students consider that improving some aspects of the university's functioning would help to increase their level of psychological well-being, with students considering that reducing the number of hours of daily and weekly classes, the reformulation of teaching methods, a wider range of curricular options, and the availability of other means that provide psychological support would improve their psychological well-being are the students who have higher levels of burnout.

#### **7. Conclusions**

Based on the results obtained, it was not possible to establish a relationship between the occurrence of burnout and sociodemographic variables, such as gender, age group, and gross monthly income, or academic variables, such as educational qualifications, areas of study, year in higher education, and the relationship with teachers.

However, there are statistically significant differences between burnout and the following factors: the arithmetic mean of the students' course; your professional situation; the way they perceive the transition to higher education; the material conditions of the educational institution; their assessment of their relationship with colleagues; the rating they attribute to the competence of teachers; the reasons for choosing the course; students' ability to withstand the pressure of their study cycle; the aspects that they believe can improve their level of psychological well-being at the university; your expectations regarding the course; uncertainty regarding the professional future; their participation in extracurricular activities; the practice of physical exercise (and the frequency of this practice); a recent depressive clinical condition; taking medication (in particular antidepressants, anxiolytics, multivitamins and/or hypnotics) and the frequency with which they take it; the reason for taking medication (alleviating symptoms of emotional exhaustion and improving school performance); the knowledge of colleagues who take sleeping medication; consider giving up the course or studies; and the way they rate their own academic performance.

In contrast, among the factors that seem to protect students from experiencing burnout are academic involvement, intrinsic motivation, coping strategies and social support, emotional intelligence, and resilience.

It should be noted that the present study showed that about 29% of the university students at the analyzed university show signs of burnout. Higher education institutions urgently need to take measures to remedy this situation, which constitutes a serious public health problem and which has several consequences, namely that of hampering the training of young people. Among these measures, rethinking, and reformulating teaching will be essential.

The results obtained provide an important practical contribution to the management of the satisfaction of university students and to improve their quality of life, since they alert to several factors that can serve as a basis for correcting failures and implementing improvements, increasing permanence and success in the analyzed higher education institution and, potentially, in other institutions.

In terms of practical contribution, it is suggested that the educational institution analyzed and the other higher education institutions implement some of the suggestions listed below to increase student satisfaction and students' quality of life:


about 21 h of classes per week (similar to Poland). The European average is only 17 h. However, there are countries in the North where these numbers are lower, for example, in Sweden (10 h) and/or Norway (13 h). However, if study time is added to this time, a national university student spends, on average, about 46 h in class and in study. Regarding worker-students, the number rises to the 63 h per week that worker-students occupy between classes, study, and their profession, which justifies that it is very hard to work and study simultaneously in such a national context [100];


In short, future research may relate the variables burnout and taking medication with others not addressed in this study, such as student personality traits (anxiety, rigidity, perfectionism, and self-efficacy), the social support system in which he/she is inserted, and the coping strategies used by him/her.

**Author Contributions:** Conceptualization, S.S. and M.A.-Y.-O.; methodology, S.S. and M.A.-Y.-O.; software, S.S.; validation, M.A.-Y.-O.; formal analysis, S.S. and M.A.-Y.-O.; investigation, S.S.; resources, S.S. and M.A.-Y.-O.; data curation, S.S.; writing—original draft preparation, S.S.; writing review and editing, M.A.-Y.-O.; visualization, S.S. and M.A.-Y.-O.; supervision, M.A.-Y.-O.; project

administration, S.S. and M.A.-Y.-O.; funding acquisition, M.A.-Y.-O. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Institutional Review Board Statement:** The administration of the university being studied was contacted at various stages of the study. The administration of the university approved of, and encouraged, the publication of the study. The study was conducted according to the guidelines of the Declaration of Helsinki. Complete anonymity of the survey respondents was guaranteed.

**Informed Consent Statement:** Informed consent was obtained from all subjects involved in the study.

**Data Availability Statement:** Data is contained within the article. Certain data is available on request due to ethical restrictions.

**Acknowledgments:** The authors would like to thank the respondents to the questionnaire. Without the respondents' time and effort this study would not have been possible.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **Appendix A. Online Questionnaire (Google Forms) Distributed to University Students at the Portuguese Public University**

Burnout is a state of physical and mental exhaustion caused by professional/academic life.

This questionnaire was carried out with the aim of analyzing whether university students have experienced, at some point in their academic life, the burnout syndrome. It also serves to understand whether these same students took (or still take) any medication during their academic career and, if so, what motivated them or motivates them to do so.

Your data will be treated confidentially, being used for scientific purposes only.


**1.** Are you a student at the Portuguese public university? \*

Yes

No

**2.** Gender \*


**3.** Age \*

18–25

```
 26–35
```

```
 36–45
```

```
 46–55
```
Other: \_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_

**4.** Country of origin (place of birth): \* \_\_\_\_\_\_\_\_\_\_\_\_\_\_

**5.** Country of residence: \* \_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_

**6.** What is your educational establishment? \*

Portuguese public university


**9.** What year of higher education do you attend? \*


**10.** Currently, what is your average course? \*

 10–12 13–15

16–18

19–20

Other: \_\_\_\_\_\_\_\_\_\_\_\_\_\_

**11.** Do you have any paid work? \*

Yes

No

**12.** What is the gross monthly income of your household (in euros)? \*

 Less than 600 euros € 600–€ 999 € 1000–€ 1499 € 1500–€ 2499 € 2500–€ 3499 € 3500–€ 4499 € 4500–€ 5499


**13.** Who do you live with? \*

Alone

Parents

Grandparents

Other family members

Other: \_\_\_\_\_\_\_\_\_\_\_

**14.** Who mainly finances your studies? \*

Parents and/or other family members

Himself

Scholarship and/or other social support

Other: \_\_\_\_\_\_\_\_\_\_\_

**15.** Do you participate in extracurricular activities? \*

 Yes No

**16.** What are the extracurricular activities in which you participate? \*

#### **Check all that apply**

Associations

Academic Tuna

Academic Practice and other related activities

Events organized by the university

I do not participate in extracurricular activities

Other: \_\_\_\_\_\_\_\_\_\_\_

**17.** Do you consider that the fact of participating in many extracurricular activities negatively affects your studies? \*

 Yes No

**18.** Do you practice physical exercise? \*

 Yes No

**19.** How often do you exercise? \*

 Once a month Once a week Two to three times a week Every day I don't practice physical activity

**20.** Do you consider that, in the transition from secondary education to higher education, most students gain more freedom in a relatively sudden way? \*

 Yes No

**21.** Do you consider that this freedom affected you negatively, as a student? \*

Yes

No

**22.** What are the reasons that led you to choose the course you are attending? \*

**Check all that apply**

 Social pressure Employability rate Pressure from family and/or friends Vocation and/or taste Other: \_\_\_\_\_\_\_\_\_\_\_

**23.** In relation to your initial expectations, how would you rate the course you are attending? \*

Worse

Like

Better

**24.** In general, how do you rate the material conditions of the university you attend? \*

More

Reasonable

Good

Excellent

**25.** In general, how do you rate your teachers? \*

Incompetent

Reasonable

Good

Excellent

**26.** How do you rate your relationship with most of your teachers? \*


**27.** And how do you rate your relationship with your colleagues? \*

Bad

Fair

Good

Excellent

**28.** Do you feel uncertain about your professional future? \*

 Yes No

**29.** Have you recently been diagnosed with depression due to the demands of your studies? \*

Yes

No

**30.** Have you ever needed to take medication in order to meet the demands of your studies? \*

 Yes No

**31.** What medications did you take/take? \*

**Check all that apply**

Antidepressants

Anxiolytics (medicines for anxiety)

 Food supplements/multivitamins to improve your mental performance (that is, to improve your memory and concentration), increase your energy and, consequently, improve your academic results

"Smart drug" (Piracetam)

Creatine

Stimulants (for example, Adderall or Ritalin)

Sleep medications (for example, Diazepam, Estazolam, Alprazolam and Zolpidem)

I never took any medication during my studies

Other: \_\_\_\_\_\_\_\_\_\_\_

**32.** How often do you take/took these medications? \*

Every day

About once a month

About once a week

Only during evaluation periods

I don't take any medication

Other: \_\_\_\_\_\_\_\_\_\_\_

**33.** Why are you taking/taking this medication? \*

#### **Check all that apply**

Social pressure

Symptoms of emotional exhaustion

Influence of friends and/or family

To improve my academic achievement

By medical prescription

I don't take any medication

Other: \_\_\_\_\_\_\_\_\_\_\_

**Check all that apply**  ☐ Social pressure

☐ Social pressure

☐ By medical prescription

☐ Other: \_\_\_\_\_\_\_\_\_\_\_

**Check all that apply**  Social pressure

**Check all that apply** 

**Check all that apply**  Social pressure

**Check all that apply** 

**Check all that apply**  ☐ Social pressure

**Check all that apply** 

**Check all that apply** 

**Check all that apply** 

☐ Other: \_\_\_\_\_\_\_\_\_\_\_

☐ Other: \_\_\_\_\_\_\_\_\_\_\_

Other: \_\_\_\_\_\_\_\_\_\_\_

☐ Other: \_\_\_\_\_\_\_\_\_\_\_

☐ Allergies ☐ Buzzing ☐ Palpitations

☐ Allergies ☐ Buzzing ☐ Palpitations

☐ Allergies Buzzing Palpitations

> Allergies Buzzing Palpitations

☐ Allergies ☐ Buzzing ☐ Palpitations

☐ Yes ☐ No

☐ Yes ☐ No

Ritalin? \* ☐ Yes ☐ No

Ritalin? \* ☐ Yes ☐ No

☐ Yes ☐ No

☐ Yes ☐ No

☐ Yes ☐ No

Ritalin? \* ☐ Yes No

Ritalin? \* ☐ Yes

☐ Yes ☐ No

☐ Yes ☐ No

me tense.

me tense.

me tense.

university.

university.

me tense.

classes at the university.

me tense.

me tense.

university.

university.

**41.** Attending a class and/or studying makes

classes at the university.

**41.** Attending a class and/or studying makes

**41.** Attending a class and/or studying makes

**40.** I feel tired when I get up in the morning and think I have to face another day at the

**40.** I feel tired when I get up in the morning and think I have to face another day at the

**39.**exhausted at the end of a day of

**41.** Attending a class and/or studying makes

classes at the university.

**41.** Attending a class and/or studying makes

**41.** Attending a class and/or studying makes

**40.** I feel tired when I get up in the morning and think I have to face another day at the

**40.** I feel tired when I get up in the morning and think I have to face another day at the

**39.**exhausted at the end of a day of

me tense.

me tense.

me tense.

university.

university.

**Emotional exhaustion \*** 

**Emotional exhaustion \*** 

☐ To improve my academic achievement

☐ Symptoms of emotional exhaustion

☐ Buzzing

☐ Yes ☐ No

Buzzing

☐ Yes ☐ No

Ritalin? \* ☐ Yes ☐ No

Ritalin? \* ☐ Yes ☐ No

☐ Yes ☐ No

☐ Yes ☐ No

☐ Yes ☐ No

Ritalin? \* ☐ Yes No

Ritalin? \* ☐ Yes

☐ Yes ☐ No

☐ Yes ☐ No

Estazolam, Alprazolam or Zolpidem? \*

**41.** Attending a class and/or studying makes

classes at the university.

**41.** Attending a class and/or studying makes

**41.** Attending a class and/or studying makes

**40.** I feel tired when I get up in the morning and think I have to face another day at the

**40.** I feel tired when I get up in the morning and think I have to face another day at the

**39.**exhausted at the end of a day of

me tense.

me tense.

me tense.

university.

university.

**Check all that apply**  ☐ Social pressure

**Check all that apply**  ☐ Social pressure

☐ By medical prescription

**34.** Have you ever experienced side effects like: \* ☐ I don't take any medication ☐ I don't take any medication ☐ I don't take any medication ☐ I don't take any medication ☐ I don't take any medication ☐ I don't take any medication ☐ Influence of friends and/or family ☐ To improve my academic achievement ☐ Influence of friends and/or family ☐ To improve my academic achievement ☐ Influence of friends and/or family ☐ To improve my academic achievement ☐ Influence of friends and/or family ☐ To improve my academic achievement ☐ Influence of friends and/or family ☐ To improve my academic achievement ☐ Influence of friends and/or family ☐ To improve my academic achievement **Check all that apply Check all that apply Check all that apply Check all that apply Check all that apply**  *Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35 *Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35 *Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35 *Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35 *Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35 *Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35 *Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

**Check all that apply** 

**Check all that apply** 

Social pressure

☐ Other: \_\_\_\_\_\_\_\_\_\_\_

☐ By medical prescription

☐ Gastrointestinal disorders

☐ By medical prescription

**Check all that apply**  ☐ Social pressure

**Check all that apply**  ☐ Social pressure

☐ Symptoms of emotional exhaustion ☐ Influence of friends and/or family ☐ To improve my academic achievement

☐ Symptoms of emotional exhaustion

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

☐ Symptoms of emotional exhaustion ☐ Influence of friends and/or family ☐ To improve my academic achievement

☐ Symptoms of emotional exhaustion

#### **Check all that apply** ☐ Other: \_\_\_\_\_\_\_\_\_\_\_ ☐ Other: \_\_\_\_\_\_\_\_\_\_\_ ☐ Other: \_\_\_\_\_\_\_\_\_\_\_ ☐ By medical prescription ☐ By medical prescription ☐ By medical prescription ☐ By medical prescription ☐ Symptoms of emotional exhaustion Social pressure Social pressure Social pressure *Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35 *Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35 *Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35 *Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35 *Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35 *Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

 Gastrointestinal disorders **34.** Have you ever experienced side effects like: \* **34.** Have you ever experienced side effects like: \* **34.** Have you ever experienced side effects like: \* **34.** Have you ever experienced side effects like: \* ☐ I don't take any medication ☐ I don't take any medication ☐ I don't take any medication ☐ I don't take any medication ☐ Influence of friends and/or family ☐ Symptoms of emotional exhaustion ☐ Influence of friends and/or family ☐ Symptoms of emotional exhaustion ☐ Influence of friends and/or family ☐ Symptoms of emotional exhaustion ☐ Influence of friends and/or family **Check all that apply Check all that apply**  *Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

☐ By medical prescription

**Check all that apply**  ☐ Social pressure

**Check all that apply**  ☐ Social pressure

 Allergies **Check all that apply Check all that apply**  ☐ Other: \_\_\_\_\_\_\_\_\_\_\_ ☐ Other: \_\_\_\_\_\_\_\_\_\_\_ **Check all that apply** 

 Buzzing ☐ Gastrointestinal disorders ☐ Gastrointestinal disorders ☐ Gastrointestinal disorders ☐ To improve my academic achievement ☐ To improve my academic achievement ☐ To improve my academic achievement Social pressure Social pressure

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

☐ Symptoms of emotional exhaustion ☐ Influence of friends and/or family ☐ To improve my academic achievement

☐ Symptoms of emotional exhaustion

 Palpitations ☐ Allergies ☐ Allergies **34.** Have you ever experienced side effects like: \* **34.** Have you ever experienced side effects like: \* **34.** Have you ever experienced side effects like: \* ☐ By medical prescription ☐ By medical prescription ☐ By medical prescription Symptoms of emotional exhaustion Symptoms of emotional exhaustion Symptoms of emotional exhaustion **Check all that apply Check all that apply** 

 Convulsions or confusional states ☐ Buzzing ☐ Buzzing ☐ Buzzing **Check all that apply Check all that apply Check all that apply Check all that apply**  ☐ Gastrointestinal disorders ☐ I don't take any medication ☐ I don't take any medication ☐ I don't take any medication ☐ I don't take any medication ☐ I don't take any medication Other: \_\_\_\_\_\_\_\_\_\_\_ Influence of friends and/or family Influence of friends and/or family Influence of friends and/or family Influence of friends and/or family Influence of friends and/or family To improve my academic achievement ☐ Social pressure ☐ Social pressure ☐ Social pressure ☐ Social pressure

☐ Allergies

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

 Excessive drowsiness ☐ Palpitations ☐ Palpitations ☐ Palpitations ☐ Gastrointestinal disorders ☐ Gastrointestinal disorders ☐ Gastrointestinal disorders ☐ Gastrointestinal disorders Other: \_\_\_\_\_\_\_\_\_\_\_ Other: \_\_\_\_\_\_\_\_\_\_\_ Other: \_\_\_\_\_\_\_\_\_\_\_ To improve my academic achievement To improve my academic achievement To improve my academic achievement To improve my academic achievement ☐ Symptoms of emotional exhaustion ☐ Symptoms of emotional exhaustion ☐ Symptoms of emotional exhaustion ☐ Symptoms of emotional exhaustion

 Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpitations, nausea and vomiting, disorientation, hallucinations and even seizures ☐ Convulsions or confusional states ☐ Excessive drowsiness ☐ Convulsions or confusional states ☐ Excessive drowsiness ☐ Convulsions or confusional states ☐ Excessive drowsiness ☐ Convulsions or confusional states ☐ Excessive drowsiness ☐ Palpitations ☐ Convulsions or confusional states ☐ Excessive drowsiness ☐ Palpitations ☐ Convulsions or confusional states ☐ Excessive drowsiness ☐ Palpitations ☐ Convulsions or confusional states ☐ Excessive drowsiness ☐ Allergies ☐ Buzzing ☐ Allergies ☐ Buzzing ☐ Allergies ☐ Buzzing ☐ Allergies ☐ Buzzing ☐ Allergies ☐ Buzzing ☐ Allergies ☐ Buzzing **34.** Have you ever experienced side effects like: \* **34.** Have you ever experienced side effects like: \* **Check all that apply 34.** Have you ever experienced side effects like: \* **Check all that apply 34.** Have you ever experienced side effects like: \* **Check all that apply 34.** Have you ever experienced side effects like: \* **Check all that apply 34.** Have you ever experienced side effects like: \* **Check all that apply 34.** Have you ever experienced side effects like: \* **Check all that apply**  By medical prescription I don't take any medication By medical prescription I don't take any medication By medical prescription I don't take any medication By medical prescription I don't take any medication By medical prescription I don't take any medication By medical prescription I don't take any medication By medical prescription I don't take any medication ☐ Influence of friends and/or family ☐ To improve my academic achievement ☐ Influence of friends and/or family ☐ To improve my academic achievement ☐ Influence of friends and/or family ☐ To improve my academic achievement ☐ Influence of friends and/or family ☐ To improve my academic achievement ☐ Influence of friends and/or family ☐ To improve my academic achievement ☐ Influence of friends and/or family ☐ To improve my academic achievement ☐ Influence of friends and/or family ☐ To improve my academic achievement

☐ Excessive drowsiness

☐ Gastrointestinal disorders

☐ By medical prescription

☐ Allergies ☐ Buzzing

☐ Symptoms of emotional exhaustion

Symptoms of emotional exhaustion

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

☐ Symptoms of emotional exhaustion ☐ Influence of friends and/or family ☐ To improve my academic achievement

☐ Symptoms of emotional exhaustion

☐ Symptoms of emotional exhaustion ☐ Influence of friends and/or family ☐ To improve my academic achievement

**Check all that apply**  ☐ Social pressure

**Check all that apply**  ☐ Social pressure

**34.** Have you ever experienced side effects like: \*

**34.** Have you ever experienced side effects like: \*

☐ By medical prescription ☐ I don't take any medication Other: \_\_\_\_\_\_\_\_\_\_\_

☐ Gastrointestinal disorders

☐ Gastrointestinal disorders

☐ Allergies ☐ Buzzing

☐ Other: \_\_\_\_\_\_\_\_\_\_\_

Social pressure

**Check all that apply** 

**Check all that apply**  Social pressure

**Check all that apply** 

**Check all that apply**  ☐ Social pressure

☐ Other: \_\_\_\_\_\_\_\_\_\_\_

☐ By medical prescription

☐ By medical prescription ☐ I don't take any medication

**Check all that apply**  ☐ Social pressure

**Check all that apply**  ☐ Social pressure

☐ By medical prescription

☐ By medical prescription ☐ I don't take any medication

☐ Other: \_\_\_\_\_\_\_\_\_\_\_

Social pressure

**Check all that apply** 

**Check all that apply**  Social pressure

☐ Other: \_\_\_\_\_\_\_\_\_\_\_

**34.** Have you ever experienced side effects like: \*

**Check all that apply** 

☐ Allergies

Symptoms of emotional exhaustion

☐ To improve my academic achievement

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

☐ Symptoms of emotional exhaustion ☐ Influence of friends and/or family ☐ To improve my academic achievement

☐ Symptoms of emotional exhaustion

☐ Symptoms of emotional exhaustion ☐ Influence of friends and/or family ☐ To improve my academic achievement

**Check all that apply**  ☐ Social pressure

**Check all that apply**  ☐ Social pressure

**34.** Have you ever experienced side effects like: \*

☐ Gastrointestinal disorders

**34.** Have you ever experienced side effects like: \*

☐ By medical prescription ☐ I don't take any medication Other: \_\_\_\_\_\_\_\_\_\_\_

☐ Gastrointestinal disorders

☐ By medical prescription ☐ I don't take any medication

☐ By medical prescription ☐ I don't take any medication

☐ Other: \_\_\_\_\_\_\_\_\_\_\_

**Check all that apply**  Social pressure

**Check all that apply** 

**Check all that apply**  Social pressure

**Check all that apply** 

☐ Symptoms of emotional exhaustion

Symptoms of emotional exhaustion Influence of friends and/or family To improve my academic achievement

**Check all that apply**  ☐ Social pressure

☐ Other: \_\_\_\_\_\_\_\_\_\_\_

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

☐ Symptoms of emotional exhaustion ☐ Influence of friends and/or family ☐ To improve my academic achievement

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

☐ Symptoms of emotional exhaustion ☐ Influence of friends and/or family ☐ To improve my academic achievement

☐ Symptoms of emotional exhaustion ☐ Influence of friends and/or family ☐ To improve my academic achievement

Symptoms of emotional exhaustion Influence of friends and/or family To improve my academic achievement

☐ Symptoms of emotional exhaustion

**34.** Have you ever experienced side effects like: \*

**34.** Have you ever experienced side effects like: \*

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

 Decreased attention, reaction time and performance speed ☐ Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpi-☐ Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpi-☐ Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpi-☐ Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpi-☐ Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpi-☐ Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpi-☐ Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpi-☐ Palpitations ☐ Palpitations ☐ Palpitations ☐ Palpitations ☐ Palpitations ☐ Palpitations Gastrointestinal disorders Gastrointestinal disorders Gastrointestinal disorders Gastrointestinal disorders Gastrointestinal disorders Gastrointestinal disorders Gastrointestinal disorders ☐ Other: \_\_\_\_\_\_\_\_\_\_\_ ☐ Other: \_\_\_\_\_\_\_\_\_\_\_ ☐ Other: \_\_\_\_\_\_\_\_\_\_\_ ☐ Other: \_\_\_\_\_\_\_\_\_\_\_ ☐ Other: \_\_\_\_\_\_\_\_\_\_\_ ☐ Other: \_\_\_\_\_\_\_\_\_\_\_ ☐ By medical prescription ☐ By medical prescription ☐ By medical prescription ☐ By medical prescription ☐ By medical prescription ☐ By medical prescription ☐ By medical prescription

tations, nausea and vomiting, disorientation, hallucinations and even seizures

 Confusion and motor incoordination tations, nausea and vomiting, disorientation, hallucinations and even seizures tations, nausea and vomiting, disorientation, hallucinations and even seizures tations, nausea and vomiting, disorientation, hallucinations and even seizures tations, nausea and vomiting, disorientation, hallucinations and even seizures tations, nausea and vomiting, disorientation, hallucinations and even seizures ☐ Convulsions or confusional states ☐ Convulsions or confusional states ☐ Convulsions or confusional states ☐ Convulsions or confusional states ☐ Convulsions or confusional states ☐ Allergies ☐ Allergies ☐ Allergies ☐ Allergies **34.** Have you ever experienced side effects like: \* **34.** Have you ever experienced side effects like: \* **34.** Have you ever experienced side effects like: \* **34.** Have you ever experienced side effects like: \* **34.** Have you ever experienced side effects like: \* ☐ I don't take any medication ☐ I don't take any medication ☐ I don't take any medication ☐ I don't take any medication ☐ I don't take any medication

☐ Yes

☐ Buzzing ☐ Palpitations

Ritalin? \* ☐ Yes

**36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or

and, consequently, improve their academic results? \*

and, consequently, improve their academic results? \*

☐ No

☐ Yes ☐ No

☐ Yes ☐ No

☐ No

Estazolam, Alprazolam or Zolpidem? \*

Ritalin? \* ☐ Yes

 I didn't feel any side effects ☐ Decreased attention, reaction time and performance speed ☐ Decreased attention, reaction time and performance speed ☐ Decreased attention, reaction time and performance speed ☐ Decreased attention, reaction time and performance speed ☐ Excessive drowsiness ☐ Excessive drowsiness ☐ Excessive drowsiness ☐ Excessive drowsiness Buzzing Buzzing Buzzing **Check all that apply Check all that apply Check all that apply**  ☐ Other: \_\_\_\_\_\_\_\_\_\_\_ ☐ Other: \_\_\_\_\_\_\_\_\_\_\_ ☐ Other: \_\_\_\_\_\_\_\_\_\_\_

 I don't take any medication ☐ Confusion and motor incoordination ☐ Confusion and motor incoordination ☐ Confusion and motor incoordination ☐ Confusion and motor incoordination ☐ Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpitations, nausea and vomiting, disorientation, hallucinations and even seizures ☐ Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpi-☐ Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpi-☐ Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpi-Palpitations Palpitations Palpitations Gastrointestinal disorders Gastrointestinal disorders **34.** Have you ever experienced side effects like: \* **34.** Have you ever experienced side effects like: \* **34.** Have you ever experienced side effects like: \* **34.** Have you ever experienced side effects like: \*

**35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins to improve their mental performance (memory and concentration), increase their energy and, consequently, improve their academic results? \* ☐ I didn't feel any side effects ☐ I don't take any medication **35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins ☐ I didn't feel any side effects ☐ I don't take any medication **35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins ☐ I didn't feel any side effects ☐ I don't take any medication **35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins ☐ I didn't feel any side effects ☐ I don't take any medication **35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins ☐ I didn't feel any side effects ☐ I don't take any medication **35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins ☐ I didn't feel any side effects ☐ I don't take any medication **35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins ☐ I didn't feel any side effects ☐ I don't take any medication **35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins ☐ Decreased attention, reaction time and performance speed ☐ Confusion and motor incoordination ☐ I didn't feel any side effects ☐ Decreased attention, reaction time and performance speed ☐ Confusion and motor incoordination ☐ I didn't feel any side effects ☐ Decreased attention, reaction time and performance speed ☐ Confusion and motor incoordination ☐ I didn't feel any side effects ☐ Decreased attention, reaction time and performance speed ☐ Confusion and motor incoordination ☐ I didn't feel any side effects ☐ Decreased attention, reaction time and performance speed ☐ Confusion and motor incoordination ☐ I didn't feel any side effects ☐ Decreased attention, reaction time and performance speed ☐ Confusion and motor incoordination ☐ I didn't feel any side effects ☐ Decreased attention, reaction time and performance speed ☐ Confusion and motor incoordination ☐ I didn't feel any side effects Convulsions or confusional states ☐Excessive drowsiness ☐ Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpi-Convulsions or confusional states ☐Excessive drowsiness ☐ Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpi-Convulsions or confusional states ☐Excessive drowsiness ☐ Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpi-Convulsions or confusional states ☐Excessive drowsiness ☐ Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpi-Convulsions or confusional states ☐Excessive drowsiness ☐ Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpi-Convulsions or confusional states ☐Excessive drowsiness ☐ Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpi-Convulsions or confusional states ☐Excessive drowsiness ☐ Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpi-Gastrointestinal disorders Allergies Palpitations ☐ Convulsions or confusional states Gastrointestinal disorders Allergies Buzzing Palpitations ☐ Convulsions or confusional states Allergies Buzzing Palpitations ☐ Convulsions or confusional states Allergies Buzzing Palpitations ☐ Convulsions or confusional states Allergies Buzzing Palpitations ☐ Convulsions or confusional states Allergies Buzzing Palpitations ☐ Convulsions or confusional states ☐ Gastrointestinal disorders **Check all that apply**  ☐ Gastrointestinal disorders ☐ Allergies **Check all that apply**  ☐ Gastrointestinal disorders ☐ Allergies **Check all that apply**  ☐ Gastrointestinal disorders ☐ Allergies **Check all that apply**  ☐ Gastrointestinal disorders ☐ Allergies **Check all that apply**  ☐ Gastrointestinal disorders ☐ Allergies **Check all that apply**  ☐ Gastrointestinal disorders ☐ Allergies

Excessive drowsiness

tations, nausea and vomiting, disorientation, hallucinations and even seizures

**Check all that apply** 

☐ Other: \_\_\_\_\_\_\_\_\_\_\_

Buzzing Palpitations

☐ Confusion and motor incoordination

☐ Allergies Buzzing Palpitations

☐ Confusion and motor incoordination

☐ Convulsions or confusional states

☐ Buzzing ☐ Palpitations

to improve their mental performance (memory and concentration), increase their energy

☐ Decreased attention, reaction time and performance speed

Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpi-

☐ Convulsions or confusional states

☐ Confusion and motor incoordination

**35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins

**36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or

**35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins to improve their mental performance (memory and concentration), increase their energy

☐ Decreased attention, reaction time and performance speed

**37.** Do you know any colleagues who have taken/take sleeping pills, such as Diazepam,

**36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or

**35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins to improve their mental performance (memory and concentration), increase their energy

Estazolam, Alprazolam or Zolpidem? \*

☐ Confusion and motor incoordination

Estazolam, Alprazolam or Zolpidem? \*

**Emotional exhaustion \*** 

and, consequently, improve their academic results? \*

Excessive drowsiness

☐ I don't take any medication

☐ Buzzing ☐ Palpitations

Ritalin? \* ☐ Yes

 I didn't feel any side effects ☐ I don't take any medication

and, consequently, improve their academic results? \*

☐ No

tations, nausea and vomiting, disorientation, hallucinations and even seizures

☐ Yes

tations, nausea and vomiting, disorientation, hallucinations and even seizures

**34.** Have you ever experienced side effects like: \*

tations, nausea and vomiting, disorientation, hallucinations and even seizures

☐ Excessive drowsiness

**Check all that apply** 

☐ Other: \_\_\_\_\_\_\_\_\_\_\_

to improve their mental performance (memory and concentration), increase their energy

☐ Decreased attention, reaction time and performance speed

☐ Confusion and motor incoordination

☐ Convulsions or confusional states

**35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins

Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpi-

**36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or

**35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins to improve their mental performance (memory and concentration), increase their energy

☐ Decreased attention, reaction time and performance speed

☐ Confusion and motor incoordination

**37.** Do you know any colleagues who have taken/take sleeping pills, such as Diazepam,

**36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or

**36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or

**35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins to improve their mental performance (memory and concentration), increase their energy

and, consequently, improve their academic results? \*

and, consequently, improve their academic results? \*

and, consequently, improve their academic results? \*

and, consequently, improve their academic results? \*

and, consequently, improve their academic results? \*

and, consequently, improve their academic results? \*

 I didn't feel any side effects ☐ I don't take any medication

tations, nausea and vomiting, disorientation, hallucinations and even seizures

and, consequently, improve their academic results? \*

and, consequently, improve their academic results? \*

tations, nausea and vomiting, disorientation, hallucinations and even seizures

☐ I don't take any medication

Excessive drowsiness

tations, nausea and vomiting, disorientation, hallucinations and even seizures

Gastrointestinal disorders

**34.** Have you ever experienced side effects like: \*

Estazolam, Alprazolam or Zolpidem? \*

Estazolam, Alprazolam or Zolpidem? \*

☐ Decreased attention, reaction time and performance speed

☐ Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpi-

**34.** Have you ever experienced side effects like: \*

**34.** Have you ever experienced side effects like: \*

☐ Confusion and motor incoordination

☐ Convulsions or confusional states

tations, nausea and vomiting, disorientation, hallucinations and even seizures

tations, nausea and vomiting, disorientation, hallucinations and even seizures

tations, nausea and vomiting, disorientation, hallucinations and even seizures

tations, nausea and vomiting, disorientation, hallucinations and even seizures

☐ Decreased attention, reaction time and performance speed

☐ Decreased attention, reaction time and performance speed

☐ Decreased attention, reaction time and performance speed

to improve their mental performance (memory and concentration), increase their energy

**35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins

Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpi-

**36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or

**35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins to improve their mental performance (memory and concentration), increase their energy

**37.** Do you know any colleagues who have taken/take sleeping pills, such as Diazepam,

**36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or

**36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or

**35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins to improve their mental performance (memory and concentration), increase their energy

☐ Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpi-

tations, nausea and vomiting, disorientation, hallucinations and even seizures

tations, nausea and vomiting, disorientation, hallucinations and even seizures

tations, nausea and vomiting, disorientation, hallucinations and even seizures

tations, nausea and vomiting, disorientation, hallucinations and even seizures

to improve mental performance (memory and concentration), increase their energy

**35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins

Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpi-

**36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or

**35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins to improve mental performance (memory and concentration), increase their energy

**37.** Do you know any colleagues who have taken/take sleeping pills, such as Diazepam,

**36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or

**36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or

**35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins to improve mental performance (memory and concentration), increase their energy

☐ Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpi-

☐ Decreased attention, reaction time and performance speed

☐ Convulsions or confusional states

☐ Allergies Buzzing Palpitations

☐ I don't take any medication

Gastrointestinal disorders

☐ Confusion and motor incoordination

**Check all that apply** 

☐ Other: \_\_\_\_\_\_\_\_\_\_\_

☐ Excessive drowsiness

**34.** Have you ever experienced side effects like: \*

☐ I don't take any medication

 Yes to improve their mental performance (memory and concentration), increase their energy to improve their mental performance (memory and concentration), increase their energy to improve their mental performance (memory and concentration), increase their energy ☐ I don't take any medication ☐ I don't take any medication ☐ I don't take any medication tations, nausea and vomiting, disorientation, hallucinations and even seizures ☐ Decreased attention, reaction time and performance speed tations, nausea and vomiting, disorientation, hallucinations and even seizures ☐ Decreased attention, reaction time and performance speed tations, nausea and vomiting, disorientation, hallucinations and even seizures ☐ Decreased attention, reaction time and performance speed ☐ Convulsions or confusional states Excessive drowsiness Excessive drowsiness Excessive drowsiness ☐ Buzzing ☐ Palpitations ☐ Buzzing ☐ Palpitations

 No and, consequently, improve their academic results? \* and, consequently, improve their academic results? \* and, consequently, improve their academic results? \* ☐ Yes **35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins **35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins **35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins ☐ Confusion and motor incoordination ☐ Confusion and motor incoordination ☐ Confusion and motor incoordination Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpi-Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpi-Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpi-☐ Convulsions or confusional states ☐ Convulsions or confusional states ☐ Convulsions or confusional states

**36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or Ritalin? \* **36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or **36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or ☐ No **36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or ☐ No **36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or ☐ No **36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or ☐ No **36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or ☐ No **36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or to improve their mental performance (memory and concentration), increase their energy and, consequently, improve their academic results? \* to improve their mental performance (memory and concentration), increase their energy and, consequently, improve their academic results? \* to improve their mental performance (memory and concentration), increase their energy and, consequently, improve their academic results? \* ☐ Yes to improve their mental performance (memory and concentration), increase their energy and, consequently, improve their academic results? \* ☐ Yes to improve their mental performance (memory and concentration), increase their energy and, consequently, improve their academic results? \* ☐ Yes to improve their mental performance (memory and concentration), increase their energy and, consequently, improve their academic results? \* ☐ Yes to improve mental performance (memory and concentration), increase their energy and, consequently, improve their academic results? \* ☐ Yes ☐ I didn't feel any side effects ☐ I don't take any medication ☐ I didn't feel any side effects ☐ I don't take any medication ☐ I didn't feel any side effects ☐ I don't take any medication ☐ I didn't feel any side effects ☐ I don't take any medication ☐ I didn't feel any side effects ☐ I don't take any medication ☐ I didn't feel any side effects ☐ I don't take any medication ☐ I didn't feel any side effects ☐ I don't take any medication tations, nausea and vomiting, disorientation, hallucinations and even seizures Decreased attention, reaction time and performance speed tations, nausea and vomiting, disorientation, hallucinations and even seizures Decreased attention, reaction time and performance speed ☐ Confusion and motor incoordination tations, nausea and vomiting, disorientation, hallucinations and even seizures Decreased attention, reaction time and performance speed ☐ Confusion and motor incoordination tations, nausea and vomiting, disorientation, hallucinations and even seizures Decreased attention, reaction time and performance speed ☐ Confusion and motor incoordination tations, nausea and vomiting, disorientation, hallucinations and even seizures Decreased attention, reaction time and performance speed ☐ Confusion and motor incoordination tations, nausea and vomiting, disorientation, hallucinations and even seizures Decreased attention, reaction time and performance speed ☐ Confusion and motor incoordination tations, nausea and vomiting, disorientation, hallucinations and even seizures Decreased attention, reaction time and performance speed ☐ Confusion and motor incoordination ☐ Excessive drowsiness ☐ Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpi-☐ Excessive drowsiness ☐ Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpi-☐ Excessive drowsiness ☐ Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpi-☐ Excessive drowsiness ☐ Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpi-☐ Excessive drowsiness ☐ Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpi-☐ Excessive drowsiness ☐ Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpi-☐ Excessive drowsiness ☐ Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpi-

☐ Yes

☐ I don't take any medication

☐ Decreased attention, reaction time and performance speed

Gastrointestinal disorders

Ritalin? \* ☐ Yes

**36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or

☐ No

 I didn't feel any side effects ☐ I don't take any medication

☐ Decreased attention, reaction time and performance speed

and, consequently, improve their academic results? \*

 Yes No Ritalin? \* ☐ Yes ☐ No **35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins to improve their mental performance (memory and concentration), increase their energy **35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins to improve their mental performance (memory and concentration), increase their energy **35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins to improve their mental performance (memory and concentration), increase their energy ☐ Confusion and motor incoordination I didn't feel any side effects ☐ I don't take any medication I didn't feel any side effects ☐ I don't take any medication I didn't feel any side effects ☐ I don't take any medication tations, nausea and vomiting, disorientation, hallucinations and even seizures ☐ Decreased attention, reaction time and performance speed ☐ Confusion and motor incoordination tations, nausea and vomiting, disorientation, hallucinations and even seizures ☐ Decreased attention, reaction time and performance speed ☐ Confusion and motor incoordination tations, nausea and vomiting, disorientation, hallucinations and even seizures ☐ Decreased attention, reaction time and performance speed ☐ Confusion and motor incoordination

and, consequently, improve their academic results? \*

**37.** Do you know any colleagues who have taken/take sleeping pills, such as Diazepam, Estazolam, Alprazolam or Zolpidem? \* **37.** Do you know any colleagues who have taken/take sleeping pills, such as Diazepam, **37.** Do you know any colleagues who have taken/take sleeping pills, such as Diazepam, ☐ No **37.** Do you know any colleagues who have taken/take sleeping pills, such as Diazepam, ☐ No **37.** Do you know any colleagues who have taken/take sleeping pills, such as Diazepam, ☐ No **37.** Do you know any colleagues who have taken/take sleeping pills, such as Diazepam, ☐ No **37.** Do you know any colleagues who have taken/take sleeping pills, such as Diazepam, ☐ No **37.** Do you know any colleagues who have taken/take sleeping pills, such as Diazepam, Ritalin? \* ☐ Yes Ritalin? \* ☐ Yes Ritalin? \* ☐ Yes Ritalin? \* ☐ Yes Ritalin? \* ☐ Yes ☐ Yes ☐ No ☐ Yes ☐ No ☐ Yes ☐ No ☐ Yes ☐ No ☐ Yes ☐ No **35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins to improve their mental performance (memory and concentration), increase their energy **35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins to improve their mental performance (memory and concentration), increase their energy **35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins to improve their mental performance (memory and concentration), increase their energy **35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins to improve their mental performance (memory and concentration), increase their energy **35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins to improve their mental performance (memory and concentration), increase their energy **35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins to improve their mental performance (memory and concentration), increase their energy **35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins to improve mental performance (memory and concentration), increase their energy ☐ I didn't feel any side effects ☐ I don't take any medication ☐ I didn't feel any side effects ☐ I don't take any medication ☐ I didn't feel any side effects ☐ I don't take any medication ☐ I didn't feel any side effects ☐ I don't take any medication ☐ I didn't feel any side effects ☐ I don't take any medication ☐ I didn't feel any side effects ☐ I don't take any medication ☐ I didn't feel any side effects ☐ I don't take any medication

and, consequently, improve their academic results? \*

and, consequently, improve their academic results? \*

☐ Yes ☐ No

☐ Yes ☐ No

Ritalin? \* ☐ Yes No

☐ Yes

☐ Yes

☐ No

Estazolam, Alprazolam or Zolpidem? \*

☐ Confusion and motor incoordination

Estazolam, Alprazolam or Zolpidem? \*

**Emotional exhaustion \*** 

☐ Yes

☐ Yes

☐ Yes ☐ No

☐ Yes ☐ No

Ritalin? \* ☐ Yes No

and, consequently, improve their academic results? \*

and, consequently, improve their academic results? \*

☐ No

Estazolam, Alprazolam or Zolpidem? \*

Estazolam, Alprazolam or Zolpidem? \*

Ritalin? \* ☐ Yes No

☐ Yes ☐ No

☐ Yes ☐ No

☐ Yes

☐ Yes

☐ No

**36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or

 I didn't feel any side effects ☐ I don't take any medication

and, consequently, improve their academic results? \*

and, consequently, improve their academic results? \*

Ritalin? \* ☐ Yes

☐ Confusion and motor incoordination

☐ No

☐ Yes

☐ Convulsions or confusional states

☐ Confusion and motor incoordination

☐ Buzzing ☐ Palpitations

☐ I don't take any medication

Excessive drowsiness

 Yes No Estazolam, Alprazolam or Zolpidem? \* Estazolam, Alprazolam or Zolpidem? \* Estazolam, Alprazolam or Zolpidem? \* ☐ Yes ☐ No **37.** Do you know any colleagues who have taken/take sleeping pills, such as Diazepam, **37.** Do you know any colleagues who have taken/take sleeping pills, such as Diazepam, ☐ No **37.** Do you know any colleagues who have taken/take sleeping pills, such as Diazepam, **36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or **36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or **36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or Ritalin? \* and, consequently, improve their academic results? \* and, consequently, improve their academic results? \* and, consequently, improve their academic results? \* ☐ Yes ☐ No **35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins to improve their mental performance (memory and concentration), increase their energy and, consequently, improve their academic results? \* **35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins to improve their mental performance (memory and concentration), increase their energy **35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins to improve their mental performance (memory and concentration), increase their energy

#### **Emotional exhaustion \* Emotional exhaustion \* Emotional exhaustion \* Emotional exhaustion \***  ☐ Yes ☐ Yes No No **36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or **36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or **36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or **36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or ☐ Yes ☐ Yes

Estazolam, Alprazolam or Zolpidem? \*

**41.** Attending a class and/or studying makes

**41.** Attending a class and/or studying makes

**40.** I feel tired when I get up in the morning and think I have to face another day at the

me tense.

me tense.

me tense.

university.

university.

**39.**exhausted at the end of a day of

☐ Yes

In the next questions, choose the frequency with which, throughout your academic career, you encounter/have encountered the feelings described. In the next questions, choose the frequency with which, throughout your academic career, you encounter/have encountered the feelings described. In the next questions, choose the frequency with which, throughout your academic career, you encounter/have encountered the feelings described. In the next questions, choose the frequency with which, throughout your academic career, you encounter/have encountered the feelings described. In the next questions, choose the frequency with which, throughout your academic career, you encounter/have encountered the feelings described. In the next questions, choose the frequency with which, throughout your academic career, you encounter/have encountered the feelings described. In the next questions, choose the frequency with which, throughout your academic career, you encounter/have encountered the feelings described. In the next questions, choose the frequency with which, throughout your academic career, you encounter/have encountered the feelings described. **Emotional exhaustion \***  ☐ No **Emotional exhaustion \***  ☐ No **Emotional exhaustion \***  ☐ No **Emotional exhaustion \***  ☐ No **Emotional exhaustion \***  ☐ No **Emotional exhaustion \* 37.** Do you know any colleagues who have taken/take sleeping pills, such as Diazepam, Estazolam, Alprazolam or Zolpidem? \* **37.** Do you know any colleagues who have taken/take sleeping pills, such as Diazepam, Estazolam, Alprazolam or Zolpidem? \* **37.** Do you know any colleagues who have taken/take sleeping pills, such as Diazepam, Estazolam, Alprazolam or Zolpidem? \* **37.** Do you know any colleagues who have taken/take sleeping pills, such as Diazepam, Estazolam, Alprazolam or Zolpidem? \* **37.** Do you know any colleagues who have taken/take sleeping pills, such as Diazepam, Estazolam, Alprazolam or Zolpidem? \* **37.** Do you know any colleagues who have taken/take sleeping pills, such as Diazepam, Estazolam, Alprazolam or Zolpidem? \* **37.** Do you know any colleagues who have taken/take sleeping pills, such as Diazepam, Estazolam, Alprazolam or Zolpidem? \* Ritalin? \* ☐ Yes Ritalin? \* ☐ Yes Ritalin? \* ☐ Yes Ritalin? \* ☐ Yes Ritalin? \* ☐ Yes **36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or **36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or ☐ No **36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or ☐ No **36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or ☐ No **36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or ☐ No **36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or ☐ No **36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or

**Emotional exhaustion \*** 

and, consequently, improve their academic results? \*


me tense.

me tense.

classes at the university.

**41.** Attending a class and/or studying makes

classes at the university.

**41.** Attending a class and/or studying makes

**40.** I feel tired when I get up in the morning and think I have to face another day at the

**39.**exhausted at the end of a day of

**41.** Attending a class and/or studying makes

**41.** Attending a class and/or studying makes

**40.** I feel tired when I get up in the morning and think I have to face another day at the

**39.**exhausted at the end of a day of

**41.** Attending a class and/or studying makes

**41.** Attending a class and/or studying makes

**40.** I feel tired when I get up in the morning and think I have to face another day at the

**39.**exhausted at the end of a day of

me tense.

classes at the university.

me tense.

me tense.

university.

university.

☐ Yes

☐ No

☐ Yes ☐ No

me tense.

classes at the university.

me tense.

me tense.

me tense.

classes at the university.

me tense.

me tense.

university.

university.

**41.** Attending a class and/or studying makes

classes at the university.

**41.** Attending a class and/or studying makes

**40.** I feel tired when I get up in the morning and think I have to face another day at the

university.

university.

**41.** Attending a class and/or studying makes

classes at the university.

**41.** Attending a class and/or studying makes

**41.** Attending a class and/or studying makes

**40.** I feel tired when I get up in the morning and think I have to face another day at the

and think I have to face another day at the

**39.** I feel exhausted at the end of a day of

me tense.

me tense.

me tense.

☐ Yes ☐ No

☐ Yes ☐ No

☐ Yes ☐ No

☐ Yes ☐ No

☐ Yes

me tense.

me tense.

me tense.

university.

**41.** Attending a class and/or studying makes

classes at the university.

university.

university.

☐ Yes ☐ No

Ritalin? \* ☐ Yes ☐ No

Ritalin? \* ☐ Yes ☐ No

Ritalin? \* ☐ Yes ☐ No

university.

☐ Yes ☐ No

**39.** I feel exhausted at the end of a day of

Ritalin? \* ☐ Yes ☐ No

classes at the university.

me tense.

me tense.

me tense.

me tense.

me tense.

me tense.

university.

university.

university.

university.

**Emotional exhaustion \*** 

☐ By medical prescription

☐ Other: \_\_\_\_\_\_\_\_\_\_\_

**Check all that apply**  ☐ Social pressure

**Check all that apply** 

**Check all that apply**  ☐ Social pressure

**Check all that apply** 

**Check all that apply** 

☐ Other: \_\_\_\_\_\_\_\_\_\_\_

☐ Other: \_\_\_\_\_\_\_\_\_\_\_

☐ Yes ☐ No

☐ Yes ☐ No

Ritalin? \* ☐ Yes ☐ No

Ritalin? \*

Ritalin? \* ☐ Yes ☐ No

Ritalin? \* ☐ Yes ☐ No

☐ Allergies ☐ Buzzing ☐ Palpitations

#### **Disbelief \*** ☐ No Estazolam, Alprazolam or Zolpidem? \* Estazolam, Alprazolam or Zolpidem? \* Estazolam, Alprazolam or Zolpidem? \* ☐ Yes ☐ No **36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or **36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or **36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or Ritalin? \* ☐ Allergies **34.** Have you ever experienced side effects like: \* **34.** Have you ever experienced side effects like: \* **34.** Have you ever experienced side effects like: \* ☐ I don't take any medication ☐ I don't take any medication ☐ I don't take any medication ☐ Influence of friends and/or family ☐ To improve my academic achievement ☐ Influence of friends and/or family ☐ To improve my academic achievement ☐ Influence of friends and/or family ☐ Social pressure ☐ Social pressure

☐ Yes

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

Gastrointestinal disorders

☐ Yes ☐ No

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

Estazolam, Alprazolam or Zolpidem? \*

Ritalin? \*

☐ Symptoms of emotional exhaustion

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

☐ Symptoms of emotional exhaustion ☐ Influence of friends and/or family ☐ To improve my academic achievement

☐ Symptoms of emotional exhaustion ☐ Influence of friends and/or family ☐ To improve my academic achievement

☐ Symptoms of emotional exhaustion ☐ Influence of friends and/or family ☐ To improve my academic achievement

☐ Symptoms of emotional exhaustion ☐ Influence of friends and/or family ☐ To improve my academic achievement

**Check all that apply**  ☐ Social pressure

**Check all that apply**  ☐ Social pressure

**Check all that apply**  ☐ Social pressure

☐ By medical prescription ☐ I don't take any medication

☐ By medical prescription ☐ I don't take any medication

☐ Gastrointestinal disorders

☐ By medical prescription ☐ I don't take any medication

☐ Gastrointestinal disorders

☐ By medical prescription ☐ I don't take any medication

☐ Gastrointestinal disorders

☐ Gastrointestinal disorders

☐ Convulsions or confusional states

☐ Allergies ☐ Buzzing ☐ Palpitations

☐ Convulsions or confusional states

☐ Allergies ☐ Buzzing ☐ Palpitations

☐ Convulsions or confusional states

☐ Convulsions or confusional states

☐ Allergies ☐ Buzzing ☐ Palpitations

☐ Symptoms of emotional exhaustion ☐ Influence of friends and/or family ☐ To improve my academic achievement

☐ Confusion and motor incoordination

☐ Allergies ☐ Buzzing ☐ Palpitations

☐ Confusion and motor incoordination

☐ Confusion and motor incoordination

☐ Confusion and motor incoordination

Symptoms of emotional exhaustion Influence of friends and/or family To improve my academic achievement

☐ Symptoms of emotional exhaustion ☐ Influence of friends and/or family ☐ To improve my academic achievement

☐ I didn't feel any side effects ☐ I don't take any medication

☐ I didn't feel any side effects ☐ I don't take any medication

> ☐ Yes ☐ No

☐ I didn't feel any side effects ☐ I don't take any medication

> ☐ Yes ☐ No

☐ I didn't feel any side effects ☐ I don't take any medication

By medical prescription I don't take any medication

☐ Yes ☐ No

Ritalin? \* ☐ Yes ☐ No

Ritalin? \* ☐ Yes ☐ No

Ritalin? \* ☐ Yes ☐ No

Ritalin? \* ☐ Yes ☐ No

☐ Allergies ☐ Buzzing

☐ Symptoms of emotional exhaustion

☐ To improve my academic achievement

☐ Yes ☐ No

Gastrointestinal disorders

☐ I don't take any medication

☐ Yes ☐ No

☐ Yes ☐ No

me tense.

☐ Yes ☐ No

Excessive drowsiness

me tense.

☐ Yes ☐ No

me tense.

☐ Yes ☐ No

☐ Yes ☐ No

☐ Yes ☐ No

☐ Yes ☐ No

☐ Yes

me tense.

me tense.

me tense.

university.

**41.** Attending a class and/or studying makes

university.

university.

Ritalin? \* ☐ Yes ☐ No

Ritalin? \* ☐ Yes ☐ No

Ritalin? \* ☐ Yes ☐ No

university.

Ritalin? \* ☐ Yes ☐ No

**39.** I feel exhausted at the end of a day of

☐ Convulsions or confusional states

☐ Other: \_\_\_\_\_\_\_\_\_\_\_

**Check all that apply**  ☐ Social pressure

**Check all that apply** 

**Check all that apply** 

**Check all that apply** 

**Check all that apply** 

☐ Other: \_\_\_\_\_\_\_\_\_\_\_

☐ Other: \_\_\_\_\_\_\_\_\_\_\_

☐ Other: \_\_\_\_\_\_\_\_\_\_\_

☐ Excessive drowsiness

☐ Excessive drowsiness

☐ Excessive drowsiness

☐ Excessive drowsiness

**Check all that apply**  ☐ Social pressure

**Check all that apply**  Social pressure

☐ Allergies ☐ Buzzing ☐ Palpitations

☐ Allergies ☐ Buzzing ☐ Palpitations

☐ Allergies ☐ Buzzing ☐ Palpitations

☐ Allergies ☐ Buzzing ☐ Palpitations

☐ Symptoms of emotional exhaustion ☐ Influence of friends and/or family ☐ To improve my academic achievement

☐ Symptoms of emotional exhaustion ☐ Influence of friends and/or family ☐ To improve my academic achievement

☐ Symptoms of emotional exhaustion ☐ Influence of friends and/or family ☐ To improve my academic achievement

**34.** Have you ever experienced side effects like: \*

**34.** Have you ever experienced side effects like: \*

**34.** Have you ever experienced side effects like: \*

**Check all that apply** 

☐ Other: \_\_\_\_\_\_\_\_\_\_\_

**Check all that apply** 

☐ Other: \_\_\_\_\_\_\_\_\_\_\_

☐ Excessive drowsiness

**34.** Have you ever experienced side effects like: \*

**Check all that apply** 

☐ Excessive drowsiness

☐ Excessive drowsiness

☐ Excessive drowsiness

**Check all that apply**  ☐ Social pressure

**Check all that apply**  Social pressure

☐ Gastrointestinal disorders

☐ By medical prescription ☐ I don't take any medication

☐ Gastrointestinal disorders

☐ By medical prescription ☐ I don't take any medication

☐ Gastrointestinal disorders

☐ Gastrointestinal disorders

**Check all that apply** 

☐ Other: \_\_\_\_\_\_\_\_\_\_\_

☐ Other: \_\_\_\_\_\_\_\_\_\_\_

**Check all that apply**  ☐ Social pressure

☐ By medical prescription ☐ I don't take any medication

☐ By medical prescription ☐ I don't take any medication

**Check all that apply**  ☐ Social pressure

**Check all that apply**  ☐ Social pressure

**Check all that apply**  ☐ Social pressure

☐ Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpi-

☐ Convulsions or confusional states

☐ Convulsions or confusional states

☐ Allergies ☐ Buzzing ☐ Palpitations

☐ Allergies ☐ Buzzing ☐ Palpitations

☐ Allergies ☐ Buzzing ☐ Palpitations

☐ Decreased attention, reaction time and performance speed

☐ Decreased attention, reaction time and performance speed

☐ Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpi-

☐ Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpi-

☐ Convulsions or confusional states

☐ I didn't feel any side effects ☐ I don't take any medication

☐ I didn't feel any side effects ☐ I don't take any medication

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

☐ I didn't feel any side effects ☐ I don't take any medication

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

☐ I didn't feel any side effects ☐ I don't take any medication

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

By medical prescription I don't take any medication

☐ Confusion and motor incoordination

☐ Allergies ☐ Buzzing ☐ Palpitations

☐ Confusion and motor incoordination

☐ Confusion and motor incoordination

☐ Confusion and motor incoordination

☐ Symptoms of emotional exhaustion ☐ Influence of friends and/or family ☐ To improve my academic achievement

☐ Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpi-

☐ Convulsions or confusional states

☐ Convulsions or confusional states

☐ Convulsions or confusional states

☐ Convulsions or confusional states

☐ Allergies ☐ Buzzing ☐ Palpitations

☐ Allergies ☐ Buzzing ☐ Palpitations

☐ Allergies ☐ Buzzing ☐ Palpitations

☐ Decreased attention, reaction time and performance speed

☐ Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpi-

☐ I didn't feel any side effects ☐ I don't take any medication

☐ I didn't feel any side effects ☐ I don't take any medication

☐ I didn't feel any side effects ☐ I don't take any medication

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

☐ I didn't feel any side effects ☐ I don't take any medication

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

By medical prescription I don't take any medication

☐ Confusion and motor incoordination

☐ Allergies ☐ Buzzing ☐ Palpitations

☐ Confusion and motor incoordination

☐ Confusion and motor incoordination

☐ Confusion and motor incoordination

☐ Symptoms of emotional exhaustion ☐ Influence of friends and/or family ☐ To improve my academic achievement

☐ Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpi-

☐ Convulsions or confusional states

☐ Allergies ☐ Buzzing ☐ Palpitations

☐ Convulsions or confusional states

☐ Allergies ☐ Buzzing ☐ Palpitations

☐ Convulsions or confusional states

☐ Allergies ☐ Buzzing ☐ Palpitations

☐ Convulsions or confusional states

☐ Allergies ☐ Buzzing ☐ Palpitations

☐ Confusion and motor incoordination

☐ Confusion and motor incoordination

☐ Confusion and motor incoordination

☐ Confusion and motor incoordination

Symptoms of emotional exhaustion Influence of friends and/or family To improve my academic achievement

☐ Symptoms of emotional exhaustion ☐ Influence of friends and/or family ☐ To improve my academic achievement

☐ Decreased attention, reaction time and performance speed

☐ Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpi-

☐ I didn't feel any side effects ☐ I don't take any medication

☐ Decreased attention, reaction time and performance speed

☐ Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpi-

☐ Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpi-

☐ Allergies ☐ Buzzing ☐ Palpitations

☐ Convulsions or confusional states

☐ Excessive drowsiness

☐ Excessive drowsiness

☐ Excessive drowsiness

☐ Excessive drowsiness

☐ Decreased attention, reaction time and performance speed

☐ Convulsions or confusional states

☐ Convulsions or confusional states

☐ Decreased attention, reaction time and performance speed

☐ Convulsions or confusional states

☐ Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpi-

☐ Allergies ☐ Buzzing ☐ Palpitations

☐ Confusion and motor incoordination

☐ Confusion and motor incoordination

☐ Confusion and motor incoordination

**Check all that apply**  ☐ Social pressure

☐ Confusion and motor incoordination

☐ Symptoms of emotional exhaustion ☐ Influence of friends and/or family ☐ To improve my academic achievement

☐ Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpi-

☐ Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpi-

**34.** Have you ever experienced side effects like: \*

**34.** Have you ever experienced side effects like: \*

**34.** Have you ever experienced side effects like: \*

**34.** Have you ever experienced side effects like: \*

☐ Decreased attention, reaction time and performance speed

☐ Decreased attention, reaction time and performance speed

☐ Decreased attention, reaction time and performance speed

☐ Decreased attention, reaction time and performance speed

☐ Confusion and motor incoordination

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

☐ Symptoms of emotional exhaustion ☐ Influence of friends and/or family ☐ To improve my academic achievement

☐ Confusion and motor incoordination

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

☐ Confusion and motor incoordination

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

☐ Confusion and motor incoordination

☐ Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpi-

☐ Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpi-

tations, nausea and vomiting, disorientation, hallucinations and even seizures

tations, nausea and vomiting, disorientation, hallucinations and even seizures

tations, nausea and vomiting, disorientation, hallucinations and even seizures

tations, nausea and vomiting, disorientation, hallucinations and even seizures

☐ Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpi-

☐ Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpi-

**35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins to improve mental performance (memory and concentration), increase their energy

☐ Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpi-

☐ Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpi-

**35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins to improve mental performance (memory and concentration), increase their energy

**36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or

**35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins to improve mental performance (memory and concentration), increase their energy

**37.** Do you know any colleagues who have taken/take sleeping pills, such as Diazepam,

**36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or

**37.** Do you know any colleagues who have taken/take sleeping pills, such as Diazepam,

**36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or

**36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or

**35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins to improve mental performance (memory and concentration), increase their energy

**35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins to improve their mental performance (memory and concentration), increase their energy

☐ Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpi-

☐ Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpi-

**35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins to improve their mental performance (memory and concentration), increase their energy

**36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or

**35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins to improve their mental performance (memory and concentration), increase their energy

**37.** Do you know any colleagues who have taken/take sleeping pills, such as Diazepam,

**36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or

**37.** Do you know any colleagues who have taken/take sleeping pills, such as Diazepam,

**36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or

**36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or

Decreased attention, reaction time and performance speed

**35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins to improve their mental performance (memory and concentration), increase their energy

to improve their mental performance (memory and concentration), increase their energy

Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpi-

tations, nausea and vomiting, disorientation, hallucinations and even seizures

tations, nausea and vomiting, disorientation, hallucinations and even seizures

**36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or

**35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins to improve mental performance (memory and concentration), increase their energy

to improve mental performance (memory and concentration), increase their energy

Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpi-

**37.** Do you know any colleagues who have taken/take sleeping pills, such as Diazepam,

**36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or

**35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins

**36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or

to improve mental performance (memory and concentration), increase their energy

**37.** Do you know any colleagues who have taken/take sleeping pills, such as Diazepam,

**36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or

**35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins

**36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or

to improve their mental performance (memory and concentration), increase their energy

**36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or

**35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins to improve their mental performance (memory and concentration), increase their energy

tations, nausea and vomiting, disorientation, hallucinations and even seizures

tations, nausea and vomiting, disorientation, hallucinations and even seizures

tations, nausea and vomiting, disorientation, hallucinations and even seizures

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

tations, nausea and vomiting, disorientation, hallucinations and even seizures

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

☐ Decreased attention, reaction time and performance speed

☐ Decreased attention, reaction time and performance speed

☐ Decreased attention, reaction time and performance speed

☐ Decreased attention, reaction time and performance speed

**35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins to improve their mental performance (memory and concentration), increase their energy

☐ Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpi-

☐ Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpi-

**35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins to improve their mental performance (memory and concentration), increase their energy

and, consequently, improve their academic results? \*

and, consequently, improve their academic results? \*

and, consequently, improve their academic results? \*

and, consequently, improve their academic results? \*

**36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or

**35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins to improve their mental performance (memory and concentration), increase their energy

**37.** Do you know any colleagues who have taken/take sleeping pills, such as Diazepam,

**36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or

**34.** Have you ever experienced side effects like: \*

**34.** Have you ever experienced side effects like: \*

**37.** Do you know any colleagues who have taken/take sleeping pills, such as Diazepam,

**36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or

**36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or

**35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins to improve their mental performance (memory and concentration), increase their energy

Symptoms of emotional exhaustion Influence of friends and/or family To improve my academic achievement

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

Estazolam, Alprazolam or Zolpidem? \*

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

☐ Symptoms of emotional exhaustion ☐ Influence of friends and/or family ☐ To improve my academic achievement

☐ Symptoms of emotional exhaustion ☐ Influence of friends and/or family

Estazolam, Alprazolam or Zolpidem? \*

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

tations, nausea and vomiting, disorientation, hallucinations and even seizures

tations, nausea and vomiting, disorientation, hallucinations and even seizures

tations, nausea and vomiting, disorientation, hallucinations and even seizures

☐ I didn't feel any side effects ☐ I don't take any medication

☐ I didn't feel any side effects ☐ I don't take any medication

☐ I didn't feel any side effects ☐ I don't take any medication

☐ I didn't feel any side effects ☐ I don't take any medication

By medical prescription I don't take any medication

☐ Convulsions or confusional states

☐ Convulsions or confusional states

☐ Convulsions or confusional states

☐ Convulsions or confusional states

and, consequently, improve their academic results? \*

tations, nausea and vomiting, disorientation, hallucinations and even seizures

and, consequently, improve their academic results? \*

and, consequently, improve their academic results? \*

and, consequently, improve their academic results? \*

**34.** Have you ever experienced side effects like: \*

☐ By medical prescription ☐ I don't take any medication

☐ By medical prescription

Gastrointestinal disorders

**34.** Have you ever experienced side effects like: \*

**34.** Have you ever experienced side effects like: \*

☐ Gastrointestinal disorders

☐ By medical prescription ☐ I don't take any medication

**34.** Have you ever experienced side effects like: \*

☐ Gastrointestinal disorders

☐ By medical prescription ☐ I don't take any medication

**34.** Have you ever experienced side effects like: \*

☐ Gastrointestinal disorders

**34.** Have you ever experienced side effects like: \*

☐ Gastrointestinal disorders

☐ By medical prescription ☐ I don't take any medication

☐ By medical prescription ☐ I don't take any medication

☐ Other: \_\_\_\_\_\_\_\_\_\_\_

**Check all that apply**  ☐ Social pressure

**Check all that apply** 

☐ Symptoms of emotional exhaustion ☐ Influence of friends and/or family ☐ To improve my academic achievement

**Check all that apply** 

**Check all that apply** 

**Check all that apply** 

☐ Other: \_\_\_\_\_\_\_\_\_\_\_

☐ Other: \_\_\_\_\_\_\_\_\_\_\_

☐ Other: \_\_\_\_\_\_\_\_\_\_\_

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

☐ Symptoms of emotional exhaustion ☐ Influence of friends and/or family ☐ To improve my academic achievement

☐ Symptoms of emotional exhaustion ☐ Influence of friends and/or family ☐ To improve my academic achievement

☐ Symptoms of emotional exhaustion ☐ Influence of friends and/or family ☐ To improve my academic achievement

**Check all that apply**  ☐ Social pressure

**Check all that apply**  ☐ Social pressure

**Check all that apply**  ☐ Social pressure

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

☐ Symptoms of emotional exhaustion ☐ Influence of friends and/or family ☐ To improve my academic achievement

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

☐ Symptoms of emotional exhaustion ☐ Influence of friends and/or family ☐ To improve my academic achievement

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

☐ Symptoms of emotional exhaustion ☐ Influence of friends and/or family ☐ To improve my academic achievement

☐ Symptoms of emotional exhaustion ☐ Influence of friends and/or family ☐ To improve my academic achievement

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

**35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins to improve their mental performance (memory and concentration), increase their energy

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

☐ Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpi-

☐ Decreased attention, reaction time and performance speed

☐ Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpi-

**35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins to improve their mental performance (memory and concentration), increase their energy

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

☐ Decreased attention, reaction time and performance speed

and, consequently, improve their academic results? \*

☐ I didn't feel any side effects ☐ I don't take any medication

and, consequently, improve their academic results? \*

Ritalin? \* ☐ Yes ☐ No

☐ I didn't feel any side effects ☐ I don't take any medication

and, consequently, improve their academic results? \*

By medical prescription I don't take any medication

Ritalin? \* ☐ Yes ☐ No

and, consequently, improve their academic results? \*

Ritalin? \* ☐ Yes ☐ No

Gastrointestinal disorders

☐ Allergies

☐ Yes ☐ No

☐ By medical prescription

☐ Yes ☐ No

☐ By medical prescription ☐ I don't take any medication

**34.** Have you ever experienced side effects like: \*

**34.** Have you ever experienced side effects like: \*

☐ Other: \_\_\_\_\_\_\_\_\_\_\_

**Check all that apply**  ☐ Social pressure

**Check all that apply** ☐ Social pressure

**Check all that apply** 

**Check all that apply**  ☐ Social pressure

☐ Other: \_\_\_\_\_\_\_\_\_\_\_

☐ Yes ☐ No

☐ Yes ☐ No

☐ Yes ☐ No

☐ I didn't feel any side effects ☐ I don't take any medication

☐ I didn't feel any side effects ☐ I don't take any medication

**36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or

**35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins to improve their mental performance (memory and concentration), increase their energy

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

**37.** Do you know any colleagues who have taken/take sleeping pills, such as Diazepam,

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

**36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or

**37.** Do you know any colleagues who have taken/take sleeping pills, such as Diazepam,

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

**36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or

**36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or

**35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins to improve their mental performance (memory and concentration), increase their energy

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

Estazolam, Alprazolam or Zolpidem? \*

☐ Symptoms of emotional exhaustion ☐ Influence of friends and/or family ☐ To improve my academic achievement

☐ Symptoms of emotional exhaustion ☐ Influence of friends and/or family

**Check all that apply** 

**Check all that apply**  ☐ Social pressure

☐ Other: \_\_\_\_\_\_\_\_\_\_\_

☐ Other: \_\_\_\_\_\_\_\_\_\_\_

**Check all that apply**  ☐ Social pressure

Symptoms of emotional exhaustion Influence of friends and/or family To improve my academic achievement

**Check all that apply**  ☐ Social pressure

**Check all that apply**  Social pressure

Estazolam, Alprazolam or Zolpidem? \*

tations, nausea and vomiting, disorientation, hallucinations and even seizures

☐ Excessive drowsiness

tations, nausea and vomiting, disorientation, hallucinations and even seizures

☐ Excessive drowsiness

tations, nausea and vomiting, disorientation, hallucinations and even seizures

☐ Excessive drowsiness

tations, nausea and vomiting, disorientation, hallucinations and even seizures

**Check all that apply**  ☐ Social pressure

**Check all that apply** Social pressure

☐ Excessive drowsiness

**34.** Have you ever experienced side effects like: \*

**34.** Have you ever experienced side effects like: \*

**34.** Have you ever experienced side effects like: \*

**34.** Have you ever experienced side effects like: \*

☐ Gastrointestinal disorders

☐ By medical prescription ☐ I don't take any medication

☐ Gastrointestinal disorders

☐ By medical prescription ☐ I don't take any medication

☐ Gastrointestinal disorders

☐ Gastrointestinal disorders

☐ Allergies ☐ Buzzing ☐ Palpitations

☐ Allergies ☐ Buzzing ☐ Palpitations

☐ Other: \_\_\_\_\_\_\_\_\_\_\_

**Check all that apply**  ☐ Social pressure

**Check all that apply** 

**Check all that apply** 

**Check all that apply** 

**Check all that apply** 

☐ Other: \_\_\_\_\_\_\_\_\_\_\_

☐ Other: \_\_\_\_\_\_\_\_\_\_\_

☐ Other: \_\_\_\_\_\_\_\_\_\_\_

☐ By medical prescription ☐ I don't take any medication

☐ By medical prescription ☐ I don't take any medication

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

☐ Symptoms of emotional exhaustion ☐ Influence of friends and/or family ☐ To improve my academic achievement

☐ Symptoms of emotional exhaustion ☐ Influence of friends and/or family ☐ To improve my academic achievement

☐ Symptoms of emotional exhaustion ☐ Influence of friends and/or family ☐ To improve my academic achievement

☐ Symptoms of emotional exhaustion ☐ Influence of friends and/or family ☐ To improve my academic achievement

**Check all that apply**  ☐ Social pressure

**Check all that apply**  ☐ Social pressure

**Check all that apply**  ☐ Social pressure

**35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins to improve their mental performance (memory and concentration), increase their energy

☐ Decreased attention, reaction time and performance speed

☐ Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpi-

☐ I didn't feel any side effects ☐ I don't take any medication

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

and, consequently, improve their academic results? \*

☐ Excessive drowsiness

and, consequently, improve their academic results? \*

**Check all that apply**  Social pressure

and, consequently, improve their academic results? \*

and, consequently, improve their academic results? \*

☐ By medical prescription ☐ I don't take any medication

☐ Other: \_\_\_\_\_\_\_\_\_\_\_

**Check all that apply**  ☐ Social pressure

**Check all that apply**  ☐ Social pressure

**34.** Have you ever experienced side effects like: \*

**34.** Have you ever experienced side effects like: \*

**Check all that apply** 

**Check all that apply**  ☐ Social pressure

☐ Other: \_\_\_\_\_\_\_\_\_\_\_

**36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or

**35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins to improve their mental performance (memory and concentration), increase their energy

☐ Yes ☐ No

☐ I didn't feel any side effects ☐ I don't take any medication

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

Ritalin? \* ☐ Yes ☐ No

Ritalin? \* ☐ Yes ☐ No

Ritalin? \* ☐ Yes ☐ No

☐ Yes ☐ No

☐ Decreased attention, reaction time and performance speed

☐ I didn't feel any side effects ☐ I don't take any medication

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

**35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins to improve their mental performance (memory and concentration), increase their energy

**37.** Do you know any colleagues who have taken/take sleeping pills, such as Diazepam,

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

**36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or

☐ Yes ☐ No

**36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or

**35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins to improve their mental performance (memory and concentration), increase their energy

☐ Yes ☐ No

By medical prescription I don't take any medication

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

☐ Yes ☐ No

**37.** Do you know any colleagues who have taken/take sleeping pills, such as Diazepam,

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

Gastrointestinal disorders

Estazolam, Alprazolam or Zolpidem? \*

☐ Symptoms of emotional exhaustion ☐ Influence of friends and/or family ☐ To improve my academic achievement

☐ Symptoms of emotional exhaustion ☐ Influence of friends and/or family

Estazolam, Alprazolam or Zolpidem? \*

☐ Allergies

tations, nausea and vomiting, disorientation, hallucinations and even seizures

**Check all that apply** 

**34.** Have you ever experienced side effects like: \*

tations, nausea and vomiting, disorientation, hallucinations and even seizures

tations, nausea and vomiting, disorientation, hallucinations and even seizures

tations, nausea and vomiting, disorientation, hallucinations and even seizures

**Check all that apply**  ☐ Social pressure

☐ Excessive drowsiness

☐ Excessive drowsiness

☐ Excessive drowsiness

**34.** Have you ever experienced side effects like: \*

**34.** Have you ever experienced side effects like: \*

**34.** Have you ever experienced side effects like: \*

**Check all that apply** 

☐ Other: \_\_\_\_\_\_\_\_\_\_\_

**Check all that apply** 

☐ Other: \_\_\_\_\_\_\_\_\_\_\_

**Check all that apply** 

☐ Other: \_\_\_\_\_\_\_\_\_\_\_

☐ Other: \_\_\_\_\_\_\_\_\_\_\_

**Check all that apply**  ☐ Social pressure

**Check all that apply**  ☐ Social pressure

**Check all that apply**  ☐ Social pressure

**Check all that apply**  ☐ Social pressure

☐ By medical prescription ☐ I don't take any medication

☐ By medical prescription ☐ I don't take any medication

☐ Gastrointestinal disorders

☐ By medical prescription ☐ I don't take any medication

☐ Gastrointestinal disorders

☐ By medical prescription ☐ I don't take any medication

☐ Gastrointestinal disorders

☐ Gastrointestinal disorders

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

☐ Symptoms of emotional exhaustion ☐ Influence of friends and/or family ☐ To improve my academic achievement

☐ Symptoms of emotional exhaustion ☐ Influence of friends and/or family ☐ To improve my academic achievement

☐ Symptoms of emotional exhaustion ☐ Influence of friends and/or family ☐ To improve my academic achievement

☐ Symptoms of emotional exhaustion ☐ Influence of friends and/or family ☐ To improve my academic achievement

**35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins to improve their mental performance (memory and concentration), increase their energy

☐ Decreased attention, reaction time and performance speed

☐ Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpi-

☐ Decreased attention, reaction time and performance speed

☐ Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpi-

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

**35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins to improve their mental performance (memory and concentration), increase their energy

☐ Decreased attention, reaction time and performance speed

and, consequently, improve their academic results? \*

and, consequently, improve their academic results? \*

and, consequently, improve their academic results? \*

and, consequently, improve their academic results? \*

☐ By medical prescription ☐ I don't take any medication

☐ By medical prescription

**34.** Have you ever experienced side effects like: \*

**Check all that apply**  ☐ Social pressure

**Check all that apply** 

**Check all that apply**  ☐ Social pressure

☐ Other: \_\_\_\_\_\_\_\_\_\_\_

☐ Other: \_\_\_\_\_\_\_\_\_\_\_

**Check all that apply**  ☐ Social pressure

**36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or

**35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins to improve their mental performance (memory and concentration), increase their energy

☐ Yes ☐ No

☐ Yes ☐ No

**37.** Do you know any colleagues who have taken/take sleeping pills, such as Diazepam,

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

**36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or

**37.** Do you know any colleagues who have taken/take sleeping pills, such as Diazepam,

Gastrointestinal disorders

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

**36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or

**35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins to improve their mental performance (memory and concentration), increase their energy

☐ Yes ☐ No

Estazolam, Alprazolam or Zolpidem? \*

Ritalin? \* ☐ Yes ☐ No

☐ Symptoms of emotional exhaustion ☐ Influence of friends and/or family

Ritalin? \* ☐ Yes ☐ No

☐ Symptoms of emotional exhaustion ☐ Influence of friends and/or family ☐ To improve my academic achievement

Estazolam, Alprazolam or Zolpidem? \*

☐ Allergies

☐ Yes ☐ No

☐ Yes ☐ No

Ritalin? \* ☐ Yes ☐ No

Symptoms of emotional exhaustion Influence of friends and/or family To improve my academic achievement

tations, nausea and vomiting, disorientation, hallucinations and even seizures

☐ Excessive drowsiness

tations, nausea and vomiting, disorientation, hallucinations and even seizures

☐ Excessive drowsiness

tations, nausea and vomiting, disorientation, hallucinations and even seizures

☐ Excessive drowsiness

tations, nausea and vomiting, disorientation, hallucinations and even seizures

**Check all that apply**  ☐ Social pressure

**Check all that apply**  Social pressure

☐ Excessive drowsiness

**34.** Have you ever experienced side effects like: \*

**Check all that apply**  ☐ Social pressure

**Check all that apply** 

☐ Other: \_\_\_\_\_\_\_\_\_\_\_

**34.** Have you ever experienced side effects like: \*

**Check all that apply** 

☐ Other: \_\_\_\_\_\_\_\_\_\_\_

**34.** Have you ever experienced side effects like: \*

**Check all that apply** 

☐ Other: \_\_\_\_\_\_\_\_\_\_\_

**34.** Have you ever experienced side effects like: \*

**Check all that apply** 

☐ Other: \_\_\_\_\_\_\_\_\_\_\_

☐ By medical prescription ☐ I don't take any medication

☐ By medical prescription ☐ I don't take any medication

☐ Gastrointestinal disorders

☐ By medical prescription ☐ I don't take any medication

☐ Gastrointestinal disorders

☐ By medical prescription ☐ I don't take any medication

☐ Gastrointestinal disorders

☐ Gastrointestinal disorders

**Check all that apply**  ☐ Social pressure

**Check all that apply**  ☐ Social pressure

**Check all that apply**  ☐ Social pressure

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

☐ Symptoms of emotional exhaustion ☐ Influence of friends and/or family ☐ To improve my academic achievement

☐ Symptoms of emotional exhaustion ☐ Influence of friends and/or family ☐ To improve my academic achievement

☐ Symptoms of emotional exhaustion ☐ Influence of friends and/or family ☐ To improve my academic achievement

☐ Symptoms of emotional exhaustion ☐ Influence of friends and/or family ☐ To improve my academic achievement

**35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins to improve their mental performance (memory and concentration), increase their energy

☐ Convulsions or confusional states

☐ Decreased attention, reaction time and performance speed

☐ Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpi-

**36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or

**35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins to improve their mental performance (memory and concentration), increase their energy

☐ Yes ☐ No

**36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or

**35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins to improve their mental performance (memory and concentration), increase their energy

☐ Yes ☐ No

Ritalin? \* ☐ Yes ☐ No

Symptoms of emotional exhaustion Influence of friends and/or family To improve my academic achievement

Ritalin? \* ☐ Yes ☐ No

☐ Symptoms of emotional exhaustion ☐ Influence of friends and/or family ☐ To improve my academic achievement

☐ Yes ☐ No

☐ Decreased attention, reaction time and performance speed

**35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins to improve their mental performance (memory and concentration), increase their energy

Estazolam, Alprazolam or Zolpidem? \*

☐ Allergies

Ritalin? \* ☐ Yes ☐ No

☐ Symptoms of emotional exhaustion

☐ Yes ☐ No

☐ Yes ☐ No

**37.** Do you know any colleagues who have taken/take sleeping pills, such as Diazepam,

Gastrointestinal disorders

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

tations, nausea and vomiting, disorientation, hallucinations and even seizures

☐ Excessive drowsiness

tations, nausea and vomiting, disorientation, hallucinations and even seizures

☐ Excessive drowsiness

tations, nausea and vomiting, disorientation, hallucinations and even seizures

☐ Excessive drowsiness

☐ Excessive drowsiness

and, consequently, improve their academic results? \*

tations, nausea and vomiting, disorientation, hallucinations and even seizures

**Check all that apply**  ☐ Social pressure

**Check all that apply**  Social pressure

and, consequently, improve their academic results? \*

and, consequently, improve their academic results? \*

and, consequently, improve their academic results? \*

☐ By medical prescription ☐ I don't take any medication

☐ By medical prescription

**34.** Have you ever experienced side effects like: \*

**Check all that apply**  ☐ Social pressure

**Check all that apply** 

**Check all that apply** 

☐ Other: \_\_\_\_\_\_\_\_\_\_\_

☐ Other: \_\_\_\_\_\_\_\_\_\_\_

**Check all that apply**  ☐ Social pressure

**34.** Have you ever experienced side effects like: \*

**Check all that apply**  ☐ Social pressure

**Check all that apply** 

☐ Other: \_\_\_\_\_\_\_\_\_\_\_

**34.** Have you ever experienced side effects like: \*

**Check all that apply** 

☐ Other: \_\_\_\_\_\_\_\_\_\_\_

**34.** Have you ever experienced side effects like: \*

**Check all that apply** 

☐ Other: \_\_\_\_\_\_\_\_\_\_\_

**34.** Have you ever experienced side effects like: \*

**Check all that apply** 

☐ Other: \_\_\_\_\_\_\_\_\_\_\_

☐ By medical prescription ☐ I don't take any medication

☐ By medical prescription ☐ I don't take any medication

☐ Gastrointestinal disorders

☐ By medical prescription ☐ I don't take any medication

☐ Gastrointestinal disorders

☐ By medical prescription ☐ I don't take any medication

☐ Gastrointestinal disorders

☐ Gastrointestinal disorders

**Check all that apply**  ☐ Social pressure

**Check all that apply**  ☐ Social pressure

**Check all that apply**  ☐ Social pressure

☐ Decreased attention, reaction time and performance speed

☐ Decreased attention, reaction time and performance speed

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

☐ Decreased attention, reaction time and performance speed

☐ I didn't feel any side effects ☐ I don't take any medication

☐ I didn't feel any side effects ☐ I don't take any medication

☐ I didn't feel any side effects ☐ I don't take any medication

☐ I didn't feel any side effects ☐ I don't take any medication

☐ Confusion and motor incoordination

☐ Allergies ☐ Buzzing ☐ Palpitations

☐ Confusion and motor incoordination

☐ Confusion and motor incoordination

☐ Confusion and motor incoordination

☐ Symptoms of emotional exhaustion ☐ Influence of friends and/or family ☐ To improve my academic achievement

☐ Convulsions or confusional states

☐ Convulsions or confusional states

☐ Convulsions or confusional states

☐ Convulsions or confusional states

☐ Allergies ☐ Buzzing ☐ Palpitations

☐ Allergies ☐ Buzzing ☐ Palpitations

☐ Allergies ☐ Buzzing ☐ Palpitations

and, consequently, improve their academic results? \*

and, consequently, improve their academic results? \*

and, consequently, improve their academic results? \*

☐ By medical prescription ☐ I don't take any medication

☐ By medical prescription

☐ Other: \_\_\_\_\_\_\_\_\_\_\_

**Check all that apply**  ☐ Social pressure

**Check all that apply**  ☐ Social pressure

**Check all that apply** 

**Check all that apply** 

☐ Other: \_\_\_\_\_\_\_\_\_\_\_

By medical prescription I don't take any medication

☐ Yes ☐ No

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

☐ Decreased attention, reaction time and performance speed

☐ Yes ☐ No

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

*Educ. Sci.* **2021**, *11*, x FOR PEER REVIEW 30 of 35

☐ Yes ☐ No

Ritalin? \* ☐ Yes ☐ No

Symptoms of emotional exhaustion Influence of friends and/or family To improve my academic achievement

Ritalin? \* ☐ Yes ☐ No

☐ Symptoms of emotional exhaustion ☐ Influence of friends and/or family ☐ To improve my academic achievement

In the next questions, choose the frequency with which, throughout your academic career, you encounter/have encountered the feelings described. In the next questions, choose the frequency with which, throughout your academic career, you encounter/have encountered the feelings described. **Emotional exhaustion \***  In the next questions, choose the frequency with which, throughout your academic career, you encounter/have encountered the feelings described. **Emotional exhaustion \***  In the next questions, choose the frequency with which, throughout your academic career, you encounter/have encountered the feelings described. **Emotional exhaustion \***  In the next questions, choose the frequency with which, throughout your academic career, you encounter/have encountered the feelings described. **Emotional exhaustion \***  In the next questions, choose the frequency with which, throughout your academic career, you encounter/have encountered the feelings described. **Emotional exhaustion \***  In the next questions, choose the frequency with which, throughout your academic career, you encounter/have encountered the feelings described. **Emotional exhaustion \***  In the next questions, choose the frequency with which, throughout your academic career, you encounter/have encountered the feelings described. ☐ Yes ☐ No ☐ Yes ☐ No ☐ Yes ☐ No ☐ Yes ☐ No ☐ Yes ☐ No **37.** Do you know any colleagues who have taken/take sleeping pills, such as Diazepam, Estazolam, Alprazolam or Zolpidem? \* **37.** Do you know any colleagues who have taken/take sleeping pills, such as Diazepam, Estazolam, Alprazolam or Zolpidem? \* **37.** Do you know any colleagues who have taken/take sleeping pills, such as Diazepam, Estazolam, Alprazolam or Zolpidem? \* **37.** Do you know any colleagues who have taken/take sleeping pills, such as Diazepam, Estazolam, Alprazolam or Zolpidem? \* **37.** Do you know any colleagues who have taken/take sleeping pills, such as Diazepam, Estazolam, Alprazolam or Zolpidem? \* **37.** Do you know any colleagues who have taken/take sleeping pills, such as Diazepam, Estazolam, Alprazolam or Zolpidem? \* **37.** Do you know any colleagues who have taken/take sleeping pills, such as Diazepam, Estazolam, Alprazolam or Zolpidem? \* ☐ Yes ☐ No Ritalin? \* ☐ Yes ☐ No Ritalin? \* ☐ Yes ☐ No Ritalin? \* ☐ Yes ☐ No Ritalin? \* ☐ Yes ☐ No ☐ Palpitations ☐ Buzzing ☐ Palpitations ☐ Buzzing ☐ Palpitations ☐ Buzzing ☐ Palpitations ☐ Buzzing ☐ Palpitations ☐ Buzzing ☐ Palpitations Gastrointestinal disorders **Check all that apply**  Gastrointestinal disorders Allergies **Check all that apply**  Gastrointestinal disorders Allergies **Check all that apply**  Gastrointestinal disorders **Check all that apply**  Gastrointestinal disorders **Check all that apply** Gastrointestinal disorders **Check all that apply**  Gastrointestinal disorders **34.** Have you ever experienced side effects like: \* ☐ Other: \_\_\_\_\_\_\_\_\_\_\_ **34.** Have you ever experienced side effects like: \* ☐ Other: \_\_\_\_\_\_\_\_\_\_\_ **34.** Have you ever experienced side effects like: \* ☐ I don't take any medication ☐ Other: \_\_\_\_\_\_\_\_\_\_\_ **34.** Have you ever experienced side effects like: \* ☐ I don't take any medication ☐ Other: \_\_\_\_\_\_\_\_\_\_\_ **34.** Have you ever experienced side effects like: \* ☐ I don't take any medication ☐ Other: \_\_\_\_\_\_\_\_\_\_\_ **34.** Have you ever experienced side effects like: \* ☐ I don't take any medication ☐ Other: \_\_\_\_\_\_\_\_\_\_\_ **34.** Have you ever experienced side effects like: \* ☐ By medical prescription I don't take any medication ☐ By medical prescription I don't take any medication ☐ To improve my academic achievement ☐ By medical prescription I don't take any medication ☐ To improve my academic achievement ☐ By medical prescription I don't take any medication ☐ To improve my academic achievement ☐ By medical prescription I don't take any medication ☐ To improve my academic achievement ☐ By medical prescription I don't take any medication ☐ To improve my academic achievement ☐ By medical prescription I don't take any medication ☐ Symptoms of emotional exhaustion ☐ Influence of friends and/or family ☐ Symptoms of emotional exhaustion ☐ Influence of friends and/or family ☐ Symptoms of emotional exhaustion ☐ Influence of friends and/or family ☐ Symptoms of emotional exhaustion ☐ Influence of friends and/or family ☐ Symptoms of emotional exhaustion ☐ Influence of friends and/or family ☐ Symptoms of emotional exhaustion ☐ Influence of friends and/or family ☐ Symptoms of emotional exhaustion ☐ Influence of friends and/or family **Check all that apply Check all that apply Check all that apply Check all that apply Check all that apply Check all that apply** 

☐ By medical prescription


**41.** Attending a class and/or studying makes **41.** Attending a class and/or studying makes **41.** Attending a class and/or studying makes **41.** Attending a class and/or studying makes me tense. **41.** Attending a class and/or studying makes me tense. **41.** Attending a class and/or studying makes me tense. and think I have to face another day at the and think I have to face another day at the and think I have to face another day at the and think I have to face another day at the and think I have to face another day at the and think I have to face another day at the classes at the university. classes at the university. **39.** I feel exhausted at the end of a day of classes at the university. **40.** I feel tired when I get up in the morning classes at the university. **47.** Have you thought about giving up the course you are attending? \* **36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or **36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or **36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or **36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or Ritalin? \* **36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or Ritalin? \* **36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or Ritalin? \* **36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or Ritalin? \* and, consequently, improve their academic results? \* and, consequently, improve their academic results? \* and, consequently, improve their academic results? \* ☐ Yes and, consequently, improve their academic results? \* ☐ Yes and, consequently, improve their academic results? \* ☐ Yes ☐ No and, consequently, improve their academic results? \* ☐ Yes and, consequently, improve their academic results? \* ☐ Yes **35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins **35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins **35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins **35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins **35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins **35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins **35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins ☐ I didn't feel any side effects ☐ I don't take any medication ☐ I didn't feel any side effects ☐ I don't take any medication ☐ I didn't feel any side effects ☐ I don't take any medication ☐ I didn't feel any side effects ☐ I don't take any medication ☐ I didn't feel any side effects ☐ I don't take any medication ☐ I didn't feel any side effects ☐ I don't take any medication ☐ I didn't feel any side effects ☐ I don't take any medication ☐ Decreased attention, reaction time and performance speed ☐ Decreased attention, reaction time and performance speed ☐ Decreased attention, reaction time and performance speed ☐ Decreased attention, reaction time and performance speed ☐ Confusion and motor incoordination ☐ Decreased attention, reaction time and performance speed ☐ Decreased attention, reaction time and performance speed ☐ Decreased attention, reaction time and performance speed Excessive drowsiness Excessive drowsiness ☐ Convulsions or confusional states Excessive drowsiness Excessive drowsiness Excessive drowsiness Excessive drowsiness

**41.** Attending a class and/or studying makes

Estazolam, Alprazolam or Zolpidem? \*

Ritalin? \* ☐ Yes ☐ No

☐ Yes ☐ No

☐ Yes

☐ No

☐ Yes ☐ No

and think I have to face another day at the

☐ Yes ☐ No

☐ Yes

Decreased attention, reaction time and performance speed

**41.** Attending a class and/or studying makes **41.** Attending a class and/or studying makes **41.** Attending a class and/or studying makes university. **41.** Attending a class and/or studying makes **40.** I feel tired when I get up in the morning **40.** I feel tired when I get up in the morning **40.** I feel tired when I get up in the morning **40.** I feel tired when I get up in the morning Yes Ritalin? \* ☐ Yes ☐ No to improve their mental performance (memory and concentration), increase their energy and, consequently, improve their academic results? \* to improve their mental performance (memory and concentration), increase their energy and, consequently, improve their academic results? \* to improve their mental performance (memory and concentration), increase their energy ☐ Confusion and motor incoordination ☐ Confusion and motor incoordination ☐ Confusion and motor incoordination Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpi-Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpi-Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpi-

me tense. and think I have to face another day at the and think I have to face another day at the and think I have to face another day at the and think I have to face another day at the No ☐ No **36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or **36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or **36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or and, consequently, improve their academic results? \* ☐ Yes **35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins to improve their mental performance (memory and concentration), increase their energy **35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins to improve their mental performance (memory and concentration), increase their energy **35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins to improve their mental performance (memory and concentration), increase their energy ☐ I didn't feel any side effects ☐ I don't take any medication ☐ I didn't feel any side effects ☐ I don't take any medication ☐ I didn't feel any side effects ☐ I don't take any medication tations, nausea and vomiting, disorientation, hallucinations and even seizures tations, nausea and vomiting, disorientation, hallucinations and even seizures tations, nausea and vomiting, disorientation, hallucinations and even seizures

**39.** I feel exhausted at the end of a day of

☐ Convulsions or confusional states

Decreased attention, reaction time and performance speed

**41.** Attending a class and/or studying makes **41.** Attending a class and/or studying makes **41.** Attending a class and/or studying makes university. **41.** Attending a class and/or studying makes university. **41.** Attending a class and/or studying makes university. **41.** Attending a class and/or studying makes **48.** Have you thought about giving up your studies? \* **37.** Do you know any colleagues who have taken/take sleeping pills, such as Diazepam, **37.** Do you know any colleagues who have taken/take sleeping pills, such as Diazepam, **37.** Do you know any colleagues who have taken/take sleeping pills, such as Diazepam, **37.** Do you know any colleagues who have taken/take sleeping pills, such as Diazepam, **37.** Do you know any colleagues who have taken/take sleeping pills, such as Diazepam, **37.** Do you know any colleagues who have taken/take sleeping pills, such as Diazepam, **37.** Do you know any colleagues who have taken/take sleeping pills, such as Diazepam, Ritalin? \* ☐ Yes Ritalin? \* ☐ Yes Ritalin? \* ☐ Yes Ritalin? \* ☐ Yes Ritalin? \* ☐ Yes ☐ No ☐ No ☐ No ☐ No ☐ No and, consequently, improve their academic results? \* and, consequently, improve their academic results? \* and, consequently, improve their academic results? \* and, consequently, improve their academic results? \* and, consequently, improve their academic results? \* and, consequently, improve their academic results? \* and, consequently, improve their academic results? \* **35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins **35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins **35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins **35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins **35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins **35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins **35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins ☐ Confusion and motor incoordination ☐ Confusion and motor incoordination ☐ Confusion and motor incoordination ☐ Confusion and motor incoordination ☐ Confusion and motor incoordination ☐ Confusion and motor incoordination ☐ Confusion and motor incoordination

university.

☐ I didn't feel any side effects ☐ I don't take any medication

☐ I didn't feel any side effects ☐ I don't take any medication

tations, nausea and vomiting, disorientation, hallucinations and even seizures

me tense.

me tense.

university.

☐ Yes ☐ No

☐ Yes

Decreased attention, reaction time and performance speed

☐ No

**39.** I feel exhausted at the end of a day of

☐ Convulsions or confusional states

me tense.

me tense.

☐ Yes ☐ No

Ritalin? \* ☐ Yes

☐ No

☐ Yes ☐ No

me tense. Yes Estazolam, Alprazolam or Zolpidem? \* Estazolam, Alprazolam or Zolpidem? \* Estazolam, Alprazolam or Zolpidem? \* ☐ No **36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or **36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or **36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or ☐ Yes ☐ No to improve their mental performance (memory and concentration), increase their energy to improve their mental performance (memory and concentration), increase their energy to improve their mental performance (memory and concentration), increase their energy ☐ I didn't feel any side effects ☐ I didn't feel any side effects ☐ I didn't feel any side effects

 No ☐ Yes ☐ No **37.** Do you know any colleagues who have taken/take sleeping pills, such as Diazepam, **37.** Do you know any colleagues who have taken/take sleeping pills, such as Diazepam, **37.** Do you know any colleagues who have taken/take sleeping pills, such as Diazepam, Ritalin? \* ☐ Yes and, consequently, improve their academic results? \* and, consequently, improve their academic results? \* and, consequently, improve their academic results? \* ☐ I don't take any medication ☐ I don't take any medication ☐ I don't take any medication

#### **Professional effectiveness \* Emotional exhaustion \* Emotional exhaustion \* Emotional exhaustion \***  Estazolam, Alprazolam or Zolpidem? \* Estazolam, Alprazolam or Zolpidem? \* Estazolam, Alprazolam or Zolpidem? \* Estazolam, Alprazolam or Zolpidem? \* ☐ No ☐ No **36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or **36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or **36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or Ritalin? \* **36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or Ritalin? \* ☐ Yes ☐ No ☐ Yes ☐ No **35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins **35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins **35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins **35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins

tations, nausea and vomiting, disorientation, hallucinations and even seizures

In the next questions, choose the frequency with which, throughout your academic career, you encounter/have encountered the feelings described. In the next questions, choose the frequency with which, throughout your academic career, you encounter/have encountered the feelings described. In the next questions, choose the frequency with which, throughout your academic career, you encounter/have encountered the feelings described. In the next questions, choose the frequency with which, throughout your academic career, you encounter/have encountered the feelings described. **Emotional exhaustion \***  In the next questions, choose the frequency with which, throughout your academic career, you encounter/have encountered the feelings described. **Emotional exhaustion \***  In the next questions, choose the frequency with which, throughout your academic career, you encounter/have encountered the feelings described. **Emotional exhaustion \***  In the next questions, choose the frequency with which, throughout your academic career, you encounter/have encountered the feelings described. **Emotional exhaustion \***  In the next questions, choose the frequency with which, throughout your academic career, you encounter/have encountered the feelings described. ☐ Yes ☐ No ☐ Yes ☐ No ☐ Yes ☐ No ☐ Yes ☐ No ☐ Yes ☐ No **37.** Do you know any colleagues who have taken/take sleeping pills, such as Diazepam, Estazolam, Alprazolam or Zolpidem? \* **37.** Do you know any colleagues who have taken/take sleeping pills, such as Diazepam, Estazolam, Alprazolam or Zolpidem? \* **37.** Do you know any colleagues who have taken/take sleeping pills, such as Diazepam, Estazolam, Alprazolam or Zolpidem? \* **37.** Do you know any colleagues who have taken/take sleeping pills, such as Diazepam, Estazolam, Alprazolam or Zolpidem? \* **37.** Do you know any colleagues who have taken/take sleeping pills, such as Diazepam, Estazolam, Alprazolam or Zolpidem? \* **37.** Do you know any colleagues who have taken/take sleeping pills, such as Diazepam, Estazolam, Alprazolam or Zolpidem? \* **37.** Do you know any colleagues who have taken/take sleeping pills, such as Diazepam, Estazolam, Alprazolam or Zolpidem? \* ☐ Yes ☐ No ☐ Yes ☐ No Ritalin? \* ☐ Yes ☐ No Ritalin? \* ☐ Yes ☐ No Ritalin? \* ☐ Yes ☐ No **36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or **36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or **36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or **36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or ☐ No **36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or ☐ No **36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or ☐ No **36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or to improve their mental performance (memory and concentration), increase their energy and, consequently, improve their academic results? \* to improve their mental performance (memory and concentration), increase their energy and, consequently, improve their academic results? \* to improve their mental performance (memory and concentration), increase their energy and, consequently, improve their academic results? \* ☐ Yes to improve their mental performance (memory and concentration), increase their energy and, consequently, improve their academic results? \* ☐ Yes to improve their mental performance (memory and concentration), increase their energy and, consequently, improve their academic results? \* ☐ Yes to improve their mental performance (memory and concentration), increase their energy and, consequently, improve their academic results? \* ☐ Yes to improve mental performance (memory and concentration), increase their energy and, consequently, improve their academic results? \* ☐ Yes

**41.** Attending a class and/or studying makes

☐ I didn't feel any side effects ☐ I don't take any medication

☐ Yes ☐ No

☐ Yes

Decreased attention, reaction time and performance speed

☐ No

☐ Yes ☐ No

☐ Yes

Ritalin? \* ☐ Yes ☐ No

☐ No

☐ Yes ☐ No

☐ I didn't feel any side effects ☐ I don't take any medication

Estazolam, Alprazolam or Zolpidem? \*

☐ Confusion and motor incoordination

☐ Convulsions or confusional states

Estazolam, Alprazolam or Zolpidem? \*

☐ Yes ☐ No

tations, nausea and vomiting, disorientation, hallucinations and even seizures

☐ Yes

and, consequently, improve their academic results? \*

☐ I didn't feel any side effects ☐ I don't take any medication

☐ No

tations, nausea and vomiting, disorientation, hallucinations and even seizures

☐ Yes ☐ No

☐ Yes

Ritalin? \* ☐ Yes ☐ No

☐ I didn't feel any side effects ☐ I don't take any medication

and, consequently, improve their academic results? \*

☐ No

☐ Yes ☐ No

Estazolam, Alprazolam or Zolpidem? \*

Decreased attention, reaction time and performance speed

Estazolam, Alprazolam or Zolpidem? \*

**36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or

**35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins to improve their mental performance (memory and concentration), increase their energy

to improve their mental performance (memory and concentration), increase their energy

Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpi-

☐ Convulsions or confusional states

☐ Confusion and motor incoordination

**37.** Do you know any colleagues who have taken/take sleeping pills, such as Diazepam,

**36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or

to improve their mental performance (memory and concentration), increase their energy

Estazolam, Alprazolam or Zolpidem? \*

☐ Confusion and motor incoordination

and, consequently, improve their academic results? \*

tations, nausea and vomiting, disorientation, hallucinations and even seizures

☐ I didn't feel any side effects ☐ I don't take any medication

☐ I didn't feel any side effects ☐ I don't take any medication

and, consequently, improve their academic results? \*

tations, nausea and vomiting, disorientation, hallucinations and even seizures

☐ Convulsions or confusional states

Estazolam, Alprazolam or Zolpidem? \*

**36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or

Decreased attention, reaction time and performance speed

**35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins to improve their mental performance (memory and concentration), increase their energy

and, consequently, improve their academic results? \*

and, consequently, improve their academic results? \*

tations, nausea and vomiting, disorientation, hallucinations and even seizures

tations, nausea and vomiting, disorientation, hallucinations and even seizures

to improve their mental performance (memory and concentration), increase their energy

Withdrawal syndrome, which causes tremors, psychomotor agitation, sweating, palpi-

**37.** Do you know any colleagues who have taken/take sleeping pills, such as Diazepam,

**36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or

**35.** Do you know any colleagues who have taken/take dietary supplements/multivitamins

**36.** Do you know any colleagues who have taken/take stimulants, such as Adderall or

to improve their mental performance (memory and concentration), increase their energy

**40.** I feel tired when I get up in the morning and think I have to face another day at the

**39.** I feel exhausted at the end of a day of

tations, nausea and vomiting, disorientation, hallucinations and even seizures

and, consequently, improve their academic results? \*

and, consequently, improve their academic results? \*


#### me tense. me tense. university. university. classes at the university. classes at the university. classes at the university. **Global Assessment \***

**41.** Attending a class and/or studying makes

me tense.

**41.** Attending a class and/or studying makes **41.** Attending a class and/or studying makes **41.** Attending a class and/or studying makes **41.** Attending a class and/or studying makes me tense. **41.** Attending a class and/or studying makes me tense. **41.** Attending a class and/or studying makes me tense. **40.** I feel tired when I get up in the morning and think I have to face another day at the **40.** I feel tired when I get up in the morning and think I have to face another day at the **40.** I feel tired when I get up in the morning and think I have to face another day at the **40.** I feel tired when I get up in the morning and think I have to face another day at the university. **40.** I feel tired when I get up in the morning and think I have to face another day at the university. **40.** I feel tired when I get up in the morning and think I have to face another day at the university. **55.** From 1 to 5, how do you rate your posture in relation to the following statement: "I can handle the pressures I feel very well in my study cycle." \* Strongly disagree Strongly agree

**41.** Attending a class and/or studying makes

**41.** Attending a class and/or studying makes

classes at the university.

**41.** Attending a class and/or studying makes

me tense.

me tense.

university.

**56.** How do you rate your performance on the course? \*

Bad

Fair

Good

Excellent

**57.** Which of the following do you think would help to improve your level of psychological well-being at the university? **\***

#### **Check all that apply**

Reduction in the number of hours of daily and weekly classes

Reformulation of teaching methods

 More opportunities (resource or second "calls") so that students can pass each of the course units

A wider range of curriculum options

Organization of more workshops to teach ways to reduce stress and anxiety

Provision of other means to provide psychological support to students

Other: \_\_\_\_\_\_\_\_\_\_\_

**58.** Describe what, in your opinion, could help improve your level of psychological and physical well-being at the university. **\*** \_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_

\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_ \_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_

**59.** E-mail for contact (in case of winning the draw). \_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_

#### **References**


### *Article* **A Study of the Emotional Intelligence and Personality Traits of University Finance Students**

**Ana M. Rosales-Pérez 1,\* , Manuel A. Fernández-Gámez <sup>2</sup> , Macarena Torroba-Díaz <sup>1</sup> and Jesús Molina-Gómez <sup>3</sup>**


**Abstract:** Studies on financial behavior indicate that emotional intelligence (EI) and personality traits (PTs) explain much of the bias in financial activity. This study aims to identify in which dimensions of theEI and PTs of university students in finance further training is needed to avoid financial behavior bias. To this end, the EI and PT levels of a sample of university finance students and financial industry professionals were compared using the Trait Emotional Intelligence Questionnaire (TEIQue) and Big Five Inventory questionnaire. Subsequently, the dimensions of EI and PTs in which students have a deficit compared to professionals were identified, and the impacts that this deficit causes on the financial behavior of students were determined. The results indicate that students are deficient in the EI competencies related to empathy, emotion regulation, self-motivation, stress management, optimism, and self-esteem. Furthermore, PTs are related to kindness, awareness, openness, and extraversion. This deficit makes students more likely to have financial behavior biases such as risk tolerance, endowment, optimism, self-control, and loss aversion. These findings suggest that universities should be aware of providing financial students with full training in EI and PTs to help them successfully address their professional future.

**Keywords:** financial education; emotional intelligence; personality traits; financial behavior; university students

#### **1. Introduction**

Studies on financial behavior address a combination of psychological, sociological, and economic concepts to explain what happens in the real economy, in which economic agents present limitations to the exercise of rationality [1,2]. In this regard, various criteria have been used to try to explain the factors that cause biases in financial behavior, highlighting among the most recent ones emotional intelligence (EI) and personality traits (PTs). On the one hand, high levels of EI have been shown to lead to superior employee performance [3–5] and the better financial performance of employees [6]. On the other hand, investor decisions and portfolio performance have also been found to be directly related to PTs [7,8].

For university students, the previous literature on financial behavior shows incipient and controversial results. For example, Haigh and List [9] found that professionals may have stronger financial behavior biases than non-professionals. However, other studies indicate that students have lower EI levels than professionals. Scott-Halsell, Shumate, and Blum [10] concluded that students did not have a sufficient level of EI to be successful leaders, and Scott-Halsell, Blum, and Huffman [11] concluded that there are significant differences between professionals and university students in all areas of EI.

The conclusions obtained from the previous literature indicate that the levels of EI and PTs can influence the financial decisions of both students and professionals, but that these conclusions are still initial and remain controversial regarding the differences

**Citation:** Rosales-Pérez, A.M.; Fernández-Gámez, M.A.; Torroba-Díaz, M.; Molina-Gómez, J. A Study of the Emotional Intelligence and Personality Traits of University Finance Students. *Educ. Sci.* **2021**, *11*, 25. https://doi.org/10.3390/educsci 11010025

Received: 30 October 2020 Accepted: 5 January 2021 Published: 11 January 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

that students and professionals can present. It may be that this controversy is due to data on the psychological influences on financial behavior not being well addressed in student populations [12]. Therefore, it is necessary to understand the abilities and skills of finance students because of their effects on financial behavior will have an important impact on their future personal well-being [13]. Likewise, EI and PTs explain a person's professional achievements, as high reflective skills coupled with high levels of EI are a decisive comparative advantage. However, the understanding of individual determinants of financial success may still be incomplete [14]. The research questions are, then, the following: Is there a difference in EI and PT levels between university finance students and financial industry professionals? If there is, could it cause students to have more financial behavior biases? If differences appear and are related to financial behavior biases, the curricula could incorporate training strategies in EI and PTs with the aim that students begin their careers in the financial industry with the highest levels of non-cognitive skills required to be successful.

To resolve the research questions, EI and PTs were measured from a sample of university students and professionals from the financial industry, confirming the differences between the two samples. These differences between students and professionals were then associated with different biases in financial behavior, which allowed us to determine which aspects of the EI and PTs of students should be used to improve their professional training. From the above, the present work is structured as follows. After this Introduction, a review of the literature is made that relates EI and PTs to the financial behavior of university students. The sample and analysis tools used for the investigation are detailed below. Finally, the results and main conclusions are presented.

#### **2. Literature Review**

The previous literature on financial behavior has developed different lines of research over time. Initially, financial behavior has been approached from a global perspective, with a descriptive analysis of the different biases in financial behavior [15]. Subsequently, it has tried to find solutions to these biases through two scientific criteria, one of them around the theory of emotions [16–18] and another according to the PTs [2,19–23]. The following is an analysis of the main conclusions obtained by the previous literature on financial behavior forEI, PTs, and university students.

#### *2.1. Emotions and Financial Behavior*

So far, only a limited number of studies have linked personality psychology and financial behavior. Typically, the main source of inspiration for financial behavior has been cognitive psychology, which offers a broad set of ideas about human decision-making and about the biases that tend to influence decision-making processes [24]. Cognitive psychology proposes that people act based on situations they already know and the interpretation they give them [25]. In the context of this cognitive theory, cognitive biases are defined as those rules that help the individual to simplify complex situations to be able to adopt models of decisions, and convenient actions are identified [26]. Pompian [27] classified biases into two categories: cognitive biases, which involve how people think, and emotional biases, which involve how they feel. For their part, cognitive biases arise from errors in memory and information processing from faulty reasoning, while emotional biases lead to reasoning influenced by feelings. In the area of financial behavior, Pompian [17] points out that investors are subconsciously influenced by past experiences and personal beliefs to the extent that even smart investors can deviate from logic and reason. He considers that EI is related to different biases such as trust in their gut, status quo, loss aversion, self-control, endowment, regret aversion, and affinity. He also identified anchoring bias, associating it with unsophisticated investors who are especially driven by emotional biases that lead them to be especially risk averse.

Munir et al. [18] concluded that people who have control over their emotions can make better investment decisions than those with lower EI, finding a positive and significant

relationship between EI and financial behavior. Ezadinea, Fathi, and Salami [28] also demonstrated the effects of EI on financial behavior biases and investment performance. Pirayesh [29] carried out an empirical study from which it was concluded that there was a relationship between global EI and the investors' decision, pointing out that investors with a high level of EI had a greater riskaversion, understood as mistrust that the individual manifests to accept an offer with an uncertain result compared to another offer with a less profitable but safer result [30]. On the other hand, Tanvir, Sufyan, and Ahsan [31] used Goleman's (1998) five dimensions of EI and demonstrated that there was an influence of global EI and some emotional competencies (emotion regulation, self-motivation, and trait empathy) in the investor decision-making process. Likewise, Dhiman and Raheja [32] stated that individuals who have high values of emotional competencies, emotion regulation, stress management, self-esteem, self-motivation, relationship, and sociability, have a higher level of risk tolerance. In this sense, risk tolerance refers to the amount of risk that the individual is willing to take to obtain a reward [17,33].

#### *2.2. PT and Financial Behavior*

Van Witteloostuijn and Muehlfeld [34], based on personal psychology and cognitive psychology, concluded that the personality of a human being is a key determinant of his/her performance and behavior, and that contributes to explaining the differences between individuals in terms of susceptibility to financial behavioral biases. From this perspective, an individual's financial decision process is based on a complex combination of demographics and personal characteristics [35]. Demographics include aspects such as age, gender, income, and level of education [36,37], and personal characteristics refer to personality traits, values, and emotions [38,39]. Furthermore, Tauni, Fang, and Yousaf [22] investigated the association between information acquisition and financial behavior by analyzing the influence of the investor's personality. The results of their study indicated that the acquisition of information is directly proportional to the frequency of trading.

Salovey [40] found that there are some PTs, such as extraversion, agreeableness, neuroticism, openness, and responsibility, that can systematically influence investment decisions. Durand, Newby, and Sanghani [7] also demonstrated that investment decisions and portfolio performance are directly related to extraversion, agreeableness, neuroticism, openness, and conscientiousness. Hopfensitz and Wranik [41] associated a greater tendency to yield to risk aversion with high levels of neuroticism. Dhiman and Raheja [32] affirmed that individuals who have high values of extraversion, agreeableness, and openness also have a higher level of risk tolerance. For their part, Durand et al. [21] proposed that investor confidence is associated with PT in two heuristics used to model market movements, that of availability (strategy of estimating the probability of an event occurring through associations that come to mind) and disposition (a tendency to disproportionately hold assets with monetary loss while simultaneously liquidating assets and accrued gains). Other studies have corroborated that financial advice is more likely to increase the frequency of trading when the adviser's personality tends to be frank, conscientious, and personable and that information obtained from financial advisors causes less adjustment in investor portfolios when her/his personality is extraverted and neurotic [42,43]. Furthermore, higher levels of openness could favor investment behaviors in changing markets because new information is more likely to be integrated into decision-making, that is investors with a high level of openness invest less in bad markets because they have greater control over risk averse bias [41].

#### *2.3. The Financial Behavior of University Students*

Previous studies suggested that EI is an important aspect of academic achievement and student retention [44]. Furthermore, there is a positive relationship between self-control and the employability potential of students [45]. The financial profile of young people of the millennial generation is often characterized by low financial literacy, risky financial behavior, and poor financial satisfaction [46,47]. Due to these generational changes, many experts consider that a change in the financial education system that affects interpersonal skills such as planning capacity, the confidence to be proactive, and the willingness to take investment risks is also necessary [48,49]. Falahati et al. [13] pointed out that the emotional training of students is important because it influences their financial behavior. Sjöberg and Engelberg [50] indicated that students have a strong tendency towards economic risktaking, achievement orientation, the search for emotions, and a relatively high level of EI. However, they detected a low trend in terms of the value of money, compared to other university students from different branches. Felton, Gibson, and Sanbonmatsu [51] found a relationship between students' optimism and the risk of their investments. Similarly, Saurilin et al. [52], after their research with university students from Portugal and Brazil, concluded that finance students showed less bias toward the status quo in their financial decisions than students without previous studies on finance. Oehler et al. [19] found that extraversion and neuroticism significantly influenced the financial behavior of university students. They pointed out that most extraverted students buy more financial assets when the assets are too expensive and that the more neurotic students have fewer risks in their financial portfolios.

Some studies have also detected the existence of a deficit in the EI of university students with respect to professionals that extends to a large part of the emotional competencies, highlighting among them self-motivation, the ability for emotional expression, and empathy [11,12]. Rzeszutek, Szyszka, and Czerwonka [53] demonstrated that susceptibility to financial behavior biases depends on the level of experience in the financial market and that some PTs such as impulsiveness and empathy are closely related to these biases.

#### **3. Methods**

#### *3.1. Samples*

The present study uses two different samples to solve the established research questions. On the one hand, there was a sample made up of a total of 240 students from the degrees of Economics, Business Administration and Management, and Finance and Accounting from a Spanish public university in the academic year 2019/2020, selected from those who had passed at least 80% of the total of the subjects. For this, a request for participation was sent to the population of students who met the characteristics indicated, obtaining a response of 45%. On the other hand, there was a sample made up of a total of 150 professionals selected randomly from among those who carry out their professional activities within the financial industry in the Spanish market in April 2020. Both samples completed EI and PT measurement questionnaires, with the specifications shown in the next section. These questionnaires were previously subjected to a methodological test to detect possible difficulties in their application. In this process, financial experts intervened and provided feedback that modified some questions and the addition of other elements to the final questionnaire. Of the total sample of students, forty-eight-point-zero-four percent were men, and fifty-one-point-nine-six percent were women. The mean age of the students was24.03 years, and fifty-three-point-nine-two percent of them had the intention of starting postgraduate studies at the end of their undergraduate studies. The socio-demographic characteristics of students in the sample appear in Table 1.

**Table 1.** Socio-demographic characteristics of the student sample.


In the sample of professionals, sixty percent were men, and forty percent were women, the average age of the participants being 44 years. Furthermore, one-hundred percent of the sample had a university degree, 10% a Ph.D. degree, and 30% a university Master's degree. Likewise, ten percent of the sample had experience in the financial industry of fewer than 10 years, 35% between 11 and 15 years, and 40% of the sample more than 16 years. The socio-demographic characteristics of the professionals in the sample appear in Table 2.


**Table 2.** Socio-demographic characteristics of the professional sample.

#### *3.2. Instruments*

Both the students and the professionals in the sample completed the EI measurement Trait Emotional Intelligence Questionnaire (TEIQue) [54] and the PT measurement Big Five Inventory (BFI) [55]. EI refers to a person's set of competencies to monitor and understand one's own emotions and those of others and use this affective information to guide one's thoughts and actions. In this study, the emotional competencies that were of particular significance due to their influence on the behavior of students and professionals in previous research were selected [11,56,57]. Furthermore, four dimensions of EI were considered (sociability, emotionality, self-control, and well-being) that grouped 15 emotional competencies [58–60]. The TEIQue version used in this study is the long version, which comprises 153 items, using a Likert-type measurement scale from 1 to 7 (1 = not at all agree and 7 = completely agree). A synthesis of the selected emotional dimensions and competencies appears in Table 3.

**Table 3.** Dimensions and emotional competencies in the emotional intelligence (EI) questionnaire.


For its part, PTs area set of qualities that describe the individual and that can be considered as key drivers of human behavior. In this regard, the BFI questionnaire used consists of 44 items grouped into five traits (extraversion, agreeableness, neuroticism, openness, and conscientiousness) selected from those that were of particular significance due to their influence on financial behavior [7,61]. This questionnaire uses a Likert-type scale of measurement from 1 to 5 (1 = not agree at all and 5 = completely agree). A synthesis of the BFI questionnaire used in the present study appears in Table 4.


**Table 4.** Personality traits (PTs) in the Big Five Inventory (BFI)questionnaire.

#### **4. Results**

#### *4.1. Descriptive Analysis*

Table 5 shows the levels of the emotional competencies of both the sample of students and of professionals. The differences between samples were analyzed using the Mann–Whitney test since the variables related to EI do not follow a normal distribution. The students presented lower levels than professionals in a large part of the emotional competencies, specifically in emotion regulation, relationships, self-esteem, self-motivation, stress management, empathy, optimism, and emotion perception (*p*-value < 0.05). However, in social awareness, students outperformed professionals (*p*-value < 0.05). Likewise, Table 5 offers the results of the descriptive analysis of the dimensions and the global score of EI. Students presented lower levels than professionals in three of the four EI dimensions (sociability, emotionality, self-control, and well-being) and globally. These results confirm that students have a deficit in EI compared to professionals.

Table 6 presents the results of the descriptive analysis of the PT variables corresponding to the sample of students and professionals. The differences between both samples were analyzed using the *t*-test since the variables related to personality follow a normal distribution. Students presented a lower average in agreeableness, openness, conscientiousness, and extraversion (*p*-value < 0.05), but in neuroticism, the average corresponding to students (2.66) was higher than that of professionals (2.39) (*p*-value < 0.05). These results also confirm that students presented significant differences with respect to professionals in all PTs, which may explain different behaviors in making financial decisions.


**Table 5.** EI of students and professionals in the sample.

S: students; P: professionals; M-W: Mann–Whitney test; \*\*\*: Sig. at the 0.001 level; \*\*: Sig. at the 0.05 level.



S: students; P: professionals; \*\*: Sig. at the 0.05 level.

#### *4.2. Impact Analysis*

The results obtained in the descriptive analysis indicated that there are significant differences between the EI and PTs of the university students and professionals in the financial industry. Besides, these differences represent a deficit for students compared to professionals in certain emotional skills and PTs. Therefore, the objective of this impact analysis is twofold. Firstly, we tried to identify if the student deficit was related to financial behavioral biases.Secondly, we determined the impact (positive or negative) on the different biases associated with the student deficit. The results of the impact analysis for EI appear in Table 7. Students presented a deficit in self-control compared to professionals. Taking into account the conclusions of previous literature [16], this should lead them to have endowment bias to a greater extent, that is to overvalue their assets and to make incorrect decisions in asset sales. Likewise, students presented a deficit in trait empathy, so they would have more optimism than professionals, reacting in an exaggerated manner to obtain superior performance [17]. Furthermore, the results show that students have deficits in emotion regulation, stress management, self-esteem, self-motivation, and relationships. According to the conclusions obtained by Dhiman and Raheja [32], this indicates that students are less risk tolerant than professionals, which may prevent them from making

aggressive investments to achieve higher returns in the medium to long term. Finally, and referring to global EI, we were also able to observe a deficit of students with respect to professionals. This aspect can lead them to have greater endowment bias, self-control, optimism bias, risk aversion, and regret aversion [29]. On the contrary, and due to this deficit in global EI, students were more risk averse than professionals, so they may tend not to analyze investment risk well.

**Table 7.** Students' EI gaps and impacts.


Table 8 shows the results of the impact analysis of the deficit in PTs. Students had lower levels of openness and higher levels of neuroticism than professionals. Then, and as proposed by Hopfensitz and Wranik [41], they can be more risk averse than professionals, resulting in being more likely to make unfavorable investment decisions. Likewise, the results suggest that students have lower levels of openness, extraversion, and agreeableness, which implies that they are less risk tolerant than professionals and, therefore, that they have greater difficulty in building an aggressive portfolio that provides high returns [32].

**Table 8.** Students' PT gaps and impacts.


#### *4.3. Discussion*

The results of this study are comparable to other findings on students in the United States of America [11] about the deficit in EI presented by the students regarding professionals, in addition to the results of the study by [53], in which they found that experience in the financial market is related to different PTs presented by students and professionals.

The results obtained also indicate that the students in the sample had a deficit in emotion regulation, stress management, self-esteem, self-motivation, and relationships, so they must be less risk tolerant than the professionals. However, these results differ from those obtained by Lin et al. [46] and Felton, Gibson, and Sanbonmatsu [51] when concluding that students have a strong tendency towards economic risk tolerance. Other aspects related to the deficit in student EI and its effects on financial behavior biases have also been highlighted in this study. Such is the case of self-control, empathy, and global EI levels, which are associated with higher endowment bias, optimism bias, self-control bias, loss aversion, and regret aversion, respectively. However, there are no precedents for these results in the previous literature.

On the other hand, regarding the PTs, previous studies showed that impulsiveness, empathy, extraversion, and neuroticism are closely related to the biases of the financial behavior that students present [19,53]. Our results coincide with those previous studies only about the deficit in empathy, but they were not concerned with other PTs such as openness and extraversion, which have not been found in the previous literature, nor the higher levels of neuroticism that students present compared to professionals.

Differences in the levels of EI and PTs can condition the biases of the financial behavior of students with respect to professionals in the financial industry. Despite the significant differences observed, the results obtained show interesting findings for discussion. For example, there were no significant differences in adaptability, assertiveness, emotion expression, emotion management, impulsiveness (low), and happiness. Perhaps this is because a greater personal and work experience in the financial industry does not have the effect of developing certain emotional skills. This is also because university training programs succeed only in certain aspects of EI, pending significant advances in other emotional skills necessary for the better future financial behavior of students in the labor market.

#### **5. Conclusions and Implications**

The results of the present study confirmed that there are significant differences in EI and PT levels between students and professionals in the financial industry and that such differences are associated with biases in financial behavior. Students present a gap with respect to professionals about relationships, empathy, emotion perception, emotion regulation, self-motivation, stress management, optimism, and self-esteem, as well as in three of the EI dimensions and the global EI scores. Gaps in EI, therefore, indicate that students tend to have optimism bias, risk tolerance, self-control bias, loss aversion, and regret aversion to a greater extent. On the other hand, and as for PTs, students present a gap in agreeableness, conscientiousness, openness, and extraversion and higher scores in neuroticism, confirming that students are more likely to have risk tolerance bias.

Our study presents several contributions to the literature on financial behavior and financial education. First, and from a theoretical perspective, there are significant differences in the levels of EI and PT between university students and professionals in the financial industry that are associated with financial behavioral biases. Previous research has shown that EI and PT influence financial behavioral biases [6] and that university students present a gap in EI compared to professionals from other industries [11]. However, this is the first study to show that university students present a gap in EI and PTs compared to professionals in the financial industry and that this gap is related to some biases of the students' financial behavior. These conclusions open up new research perspectives on EI and PT in the framework of financial behavior and financial education. Likewise, the results obtained confirmed that university students present a gap compared to professionals only in certain emotional competencies and that these are related to optimism bias, risk tolerance, self-control bias, loss aversion, and regret aversion. The literature indicates the existence of other financial behavior biases [15], although there is no empirical evidence that relates them to either EI or PTs.

This study also presents important practical implications for the management of university education and financial companies. Universities should be aware of providing students with full EI and PT training to help them successfully tackle their professional future in the financial industry. Similarly, financial companies should be aware that young professionals have low-risk tolerance and more likely to have endowment bias, optimism bias, self-control bias, and loss aversion. In this way, they should design the management of their investment portfolios taking into account the EI and PT levels of their employees.

Finally, the results of this study suggest future research on financial behavior. First, given that theories of emotions and PTs have only explained some of the financial behavior biases, other studies could check if other theoretical frameworks could explain them. Second, future research should empirically check the financial behavior bias of students

about the deficits of EI and PTs discovered in the present study. Third, future research should address the effects of gender, age, and educational level on the EI and PTs of students and professionals related to the financial industry. Fourth, future research should study what training techniques could serve to correct the EI and PT differences between students and professionals. Last, although some previous work has pointed out that experiential learning techniques have corrected differences in EI, there are still very few that refer to improvements in PTs.

**Author Contributions:** Conceptualization and writing, original draft preparation, A.M.R.-P. and M.T.-D.; investigation, A.M.R.-P. and M.A.F.-G.; methodology and funding acquisition, M.A.F.-G.; supervision, J.M.-G. All authors have read and agreed to the published version of the manuscript.

**Funding:** The authors gratefully acknowledge the financial support of the Chair of Sustainables Economy and Finance (University of Malaga).

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Data Availability Statement:** Not applicable.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**

