Analysis of Creative Thinking Skills Development under Active Learning Strategies

Abstract

Educational systems are now focusing on skills enhancement, such as creative thinking skills (CTS), as a means of long-lasting, significant learning. To this end, some universities and higher education institutions incentivize active learning (AL) strategies as CTS developers. Indeed, a positive link among creative results, time availability, and the educational environment has been reported; however, it is mainly based on qualitative and perceptual results. For this reason, we present this comparative, quantitative study in the context of a Mexican high school, weighing the effectiveness of the flipped learning and gamification teaching strategies against a conventional approach. The study revealed no differences in the learning environment; instead, the type of activity and the teamwork interaction affected CTS the most. However, those who participated in the learning Strategies (LS) evaluated themselves higher than their peers in the traditional classes. These results highlight the independence of CTS toward the referred LS and set a departing point for further research addressing the course activities’ qualities seemingly related to CTS enhancement.

1. Introduction

In high school, students prepare to enter a university program that will require them to solve problems increasingly closer to the reality of their future professional life. In a business environment propelled by innovation, the workforce within organizations is expected to achieve their productivity goals and generate new ideas for improving products and processes. Creativity, problem-solving, and the generation of new ideas are the expectations that the labor market has of individuals [1]. Therefore, the education system should seek, among others, the development of creative thinking skills (CTS).

Recent efforts at higher education centers aim to the development of competencies. However, measuring intangibles is not an easy task, and the decisions taken by such centers are usually driven by qualitative and perceptual results that favor active learning (AL) techniques. While it is true that the teaching methods must motivate the exercise of those skills and give opportunities to practice [2], it is also true that without measuring their effectiveness, a helpful comparison is hindered. Moreover, their benefits may be case-specific. Thus, the measurement of the learning techniques’ benefits is as necessary as the referred to skills are.

1.1. The Concept of Creativity

The establishment of what is considered creative has not been straightforward. It is usual to see widespread myths and misconceptions around the concept of CTS [3]. However, researchers agree that creative ideas are generally novel and helpful [4]. Other researchers affirm that these should be contingent on a process of interactions and discussion among the participants [5]. Creativity is also associated with producing original, high-quality, and “elegant” solutions to complex, novel, or ill-defined problems [6].

While working in teams, creativity can be understood as a skill developed in workgroups, emerging from its inner interactions; however, there is no single or precise way to measure it [7]. There have also been discussions about the difference between creative thinking and original thought in searching for precision, the former requiring the idea to be helpful [8]. While previous works have tried to go beyond describing creativity using natural language, defining novelty or usefulness remains subjective and linked to the viewer’s context.

One unsolved problem of creativity research is to measure if the individuals’ CTS are being developed. Some researchers have used controlled scenarios to measure creative achievements based on tests requiring the participant to “perform creatively” by associating three disparate words with a fourth one [8]. Others separate the creative process from the creative outcome and quantify the latter in terms of “implausible utility” [9]. However, extrapolating to real-life scenarios or assessing how plausible or useful a solution is might be seldom attainable.

Another proposed method allows the participant to rate his/her creative achievements on a four-point scale over several different variables [10,11]. However, questionnaires may be biased due to the participants’ perception of creativity or divergent thinking [12]. These ways of approaching creativity commonly involve short time limits that do not allow the participants to see their performance in a task associated with a real problem. Sometimes, only the mere perception of the participant is measured. Data obtained from students’ perceptions may not be precise indicators of their outcomes; as Mwalumbwe and Mtebe stated, it is unlikely that perceptions reveal causality [13].

1.2. The Time Available for the Student’s School Activity

Exercising CTS requires the student to allocate his/her time among all academic and extracurricular activities [14]. In addition, the theoretical content and the CTS activities may overlap. Furthermore, curricular, time, and space limitations often discourage creativity in schools [15], so the teacher must seek a way to overcome these limitations. In a systematic review study published in 2012, different factors were associated with CTS development, including flexibility in using time enabled by self-paced learning. Other aspects were the learning environment, the role of the professor, and the availability of resources [16].

In Mexico, the private high school curriculum has at least six demanding subjects. In addition, participation in extracurricular activities, such as dance groups, theater, and athleticism, is encouraged. Thus, the student must invest a great deal of time, possibly impairing his creativity. Therefore, in seeking to boost CTS, one may significantly increase the student’s perceived workload: a somewhat subjective issue discussed next.

A student does not necessarily perceive the time burden of his workload proportionally to the amount of actual time invested [17], generally influenced by the learning environment and his/her interests [18]. The perceived excessive workload may increase anxiety and stress, leading to a shallower approach to the content [19,20,21]. On the other hand, the time everyone requires to achieve the same learning objectives may be different [22].

There are case studies where the pressure caused by time constraints affect CTS differently, depending on the characteristics of the environment and how the individual perceives urgency: there are both high and low levels of CTS. Such studies conclude that the lack of pressure caused by time does not guarantee that the individual can think creatively [23,24]. Active learning strategies are commonly recommended to develop creativity and innovation in the students because they boost engagement and time availability. However, their associated advantages are only indirect and have not been thoroughly evaluated. In the light of such uncertainty, it is then relevant to measure the expression of CTS depending on the time enabled by the much-recommended AL strategies.

1.3. Flipped Learning and Gamification for the Development of Creative Thinking Skills

A 2018 study reported discordant results regarding the use of FL and the lack of studies focused on assessing its impact on developing specific skills, such as creative thinking [25]. On the other hand, for a gamification-based experiment, a correlation between game dynamics and creativity was shown in one study whose results were supported by the participants’ surveyed perception, lacking quantitative metrics [26].

Few studies present a combination of FL and gamification [27,28,29], none focusing on developing CTS. Others stress the need to study both techniques compared with a traditional classroom [30]. However, it was found that FL and gamification increase interest in completing pre-class activities and making better quality products [31]. Other factors in the environment that influence CTS are managing the physical spaces to allow barrier-free communication, plurality, variety [32], and the teacher’s leadership [33].

1.4. Hosting Creativity and Optimizing Time through Active Learning Strategies

A relationship has been found between the student’s perception of his/her learning environment and how he/she will approach the task to be performed [17]. Therefore, it is suggested in the literature to optimize the time in front of the class and avoid excessive workload to develop CTS.

According to that, flipped learning (FL) enables such optimization: the events that traditionally take place inside the classroom will now take place outside and vice versa [34]. The intention is to create an environment of collaboration under the teacher’s supervision, allowing, in turn, a greater involvement of the students [35]. In a study by Serene Chan and Mantak Yuen in 2014, it was found that the atmosphere of a classroom that fosters creativity is such that it gives students time to reflect and think without having to cover material in a hurry [36].

One of the critical elements in reducing time through FL has to do with pre-class learning. Content addressed outside the classroom needs to be reduced to focus only on the essential concepts [37]. However, the lack of motivation to review the material before class can hinder students’ active role in their learning process [38], a requirement for known AL methodologies [39]. As a result, small tasks are typically assigned, such as worksheets, mind maps [40], and quizzes [29]. Indeed, using other didactic strategies can be just as helpful for this purpose [31].

FL is not directly associated with CTS development throughout the literature, but an indirect link between time availability and creativity is assumed. However, available time does not prevent the students from feeling stressed or addressing the course’s material with apathy. In addition, the enhancement of skills is expected to happen when the students work on their own, relying on unknown personal circumstances and predispositions. Briefly, associating CTS with mostly unknown individual environments is complex, and students’ perceptions about their progress would be equally hard to associate as a generic asset of FL.

Another strategy called gamification has proven helpful in keeping the students motivated and involved with their learning [41]. The term gamification “refers to the process of integrating and using game design elements in a non-gaming context” [42]. Some of the most common gamification elements are leaderboards, badges, a points system, and specific milestones, visible in an aesthetic interface that presents everyday actions as if they happened in a game [43]. In general, the gamification elements seem to provide clear objectives, goals, and incentives suitable for a course in the FL model while keeping the active students’ role [29].

Then, this study aims to (a) identify the relationship between AL strategies and CTS development within the limits of the school curricula in higher education and (b) to define whether, in the scheme used, the implementation of the selected AL strategies influences the development of CTS as compared to a traditional classroom. The answers to the following questions are sought: What is the relation between AL and a creative solution when working with problem-solving projects? How does an environment influenced by AL affect the development of CTS? The following hypotheses are stated for answering these: (H0a) An environment empowered by AL strategies will not incite enhancements in CTS among students. (H1a) An environment empowered by active learning strategies will drive the development of CTS among students. (H0b) CTS are not enhanced by reducing the lecture time in favor of the students’ time to provide a creative solution. (H1b) The enhancement of CTS is enabled by reducing the lecture time in favor of the time the students devote to providing a creative solution.

2. Materials and Methods

The following issues are covered in what follows: (i) the study characteristics and demographics are introduced; (ii) the aims, structure, and resources used for enabling AL in the studied course are outlined; then, (iii) the rubric designed to assess CTS is presented and (iv) its statistical significance is verified.

A quasi-experimental, comparative, quantitative study [44,45] was conducted at PrepaTec [46], a private high school in Mexico, over four groups of the subject “Creativity and Digital Design.” Two groups were part of the “Bicultural” academic program, and the other two of the “Multicultural” comprising 109 students in total (the characteristics of the academic programs are listed in

Table 1. The differences between the two academic programs.

	Bicultural	Multicultural
Entry requirements	Pass the admission test.	- Pass the admission test. - Have a B1 level on the Cambridge English exam.
General differences	Mastery of English as a second language. Up to 16 subjects in English.	- Mastery of the English language and study a third one. - The development of multicultural competencies is added to meet the demands generated by different cultures. - More foreign teachers. - 18 subjects taught in English. - 6 subjects taught in a third language. - History subjects with a global focus.

). One group from each academic program was set as a control group, taking the class traditionally, while the other group used active learning.

The same teacher taught all groups during the same semester, and the same eight projects were used to perform the CTS assessment. One semester after completing the courses, the evaluation was reconducted using a specific rubric, a blinded review approach, and two expert evaluators: the teacher who taught the class and another alien to the experiment. In this way, the CTSs were evaluated statistically, reducing biases derived from the academic program, the professor in charge of the subject, the type of activities, and the judges.

The students enrolled in this course ranged between 14 and 16 years, of whom 48 were male and 61 were female. In addition, the teacher in charge of the course had professional design experience.

2.1. Class’s Overview

The course focuses on developing creativity and innovation in students. It addressed the following topics: intellectual property, creativity awareness, idea generation techniques (brainstorming, SCAMPER, six thinking hats), design process, and prototyping. In addition, eight projects were designed around a problem from their group, school, or local community that addressed the class topic studied at the time.

The control groups were under a traditional class scheme where the teacher explained the topic, and then the students tried to solve the related project in the remaining time. Instead, the students in the experimental group watched a short video of the class topic at home (FL pre-class activity) and solved doubts with their group and teacher at the beginning of the face-to-face class. After that, they answered a quiz of five random questions (the students were given 10 min to finish, but it was designed to be solved in five minutes) and proceeded to solve the class project. Within the course’s virtual platform, they had a board of badges and players that measured their progress on topics such as teamwork, group recognition of their project’s creativity, elimination of distractors, and grades.

All groups had time to present their projects’ results to receive feedback from the teacher and their classmates. Furthermore, every five weeks, the teacher had an individual talk with each student regarding their performance during the class. Additionally, the teacher recognized those who obtained a new badge related to creativity performance in the experimental groups.

2.2. Tools to Enable Active Learning

Pre-Class material: Eight videos were prepared with essential class contents. A quiz associated with each video was used to incentivize students into watching them pre-class. The quiz was taken before starting the class, comprising five randomly and individually selected questions from a bank of 10 questions.

Game design elements: A “goals and badges” board and a panel with the players’ avatars and data were designed (see Figure 1). The goals board was divided into three sections: badges, goods, and leaderboard. The badges were classified into four types: daily, partial, possible to reach multiple times, and hidden or time-limited badges. Each goal showed a brief description and the points earned upon completion, whereas the goods showed a description of the benefit and the points needed to obtain it. Finally, the leaderboard displayed the avatar of the 10 students with the highest number of points.

Class projects: Eight projects were set up for the measurement of CTS. Such projects were, for instance, developing a campaign to avoid plagiarism directed to a target audience, crafting a story having random elements and characters, redesigning the packaging of a product to stand out from the competitors, and composing a poster that expressed a feeling. The students’ final project involved solving a specific need of a civic association that served children with cerebral palsy or children and young people with different capabilities. These exercises were evaluated based on the students’ work on solving a problem rather than the theoretical content. The differentiating characteristics of each project are listed in

Table 2. Projects’ characteristics.

Project	Team Members	Notes
1 Anti-Plagiarism campaign	3
2 Story with random elements	3
3 Packaging creative redesign	5
4 Create a GIF with emotions	1	Usage of specialized software
5 Magazine cover design	2	Usage of specialized software
6 Movie poster	2-3	Usage of specialized software
7 Poster of an emotion	2
8 Prototype of a new product	3-4	Real problem of a civil association

.

The projects were developed in teams comprising different students every time, enabling their individual evaluation. Although there is no standard way to measure the groups’ creativity [7], the teams’ variation enabled statistical significance.

2.3. Evaluating CTS: Rubric and Judges

An evaluation rubric was designed for measuring the CTS expressed in each project. The rubric considered six different parameters. Four of them are principal elements of the creative process, namely, Originality (Or), Impact (Im), Divergence (Di), and Flexibility (Fl) [3,47,48,49,50,51]. Originality and Divergence are part of what other works name novelty, related to the idea’s infrequency of being generated, and impact is part of usefulness [52]. In addition, Technique (Te) and Resolution (Re) were also considered, focusing on the technical quality of the result. These elements are ranked in order of importance (as shown in

Table 3. Rubric for the evaluation of CTS.

Relevance	Characteristic	Description
6	Divergence	The degree to which the solution departs from the most common ones; proposes a clear differentiator.
5	Impact	An attractive result: surprising, unusual, or eye-catching. The solution stands out, considering their level of experience.
4	Originality	Exhibits unique features related to the requested characteristics in the activity instructions and according to their level of experience with the topic.
3	Flexibility	How are several ideas from different fields of study pulled together? For example, the student links different situations within a particular job or combines tools to achieve the expected result.
2	Technique	The work’s technical quality.
1	Resolution	The accomplishment of the project’s specific objectives.

) to evaluate CTS. As shown in the sequel, this rubric was statistically validated to align with CTS measurement.

Although the rubric employed relevant information found in the literature, the criteria used may appear subjective or interchangeable to a judge, even if he/she is contextualized in the area or topic. Thus, it is essential to maximize the effectiveness of the evaluation mechanism. Furthermore, we started from the premise that both judges knew they were evaluating CTS, even if its comprising elements were distinctly understood. That is, the overall given grade would exhibit a high correlation among judges, even if the rubric elements’ definitions were not entirely shared.

To identify the ambiguous or interchangeable criteria of the rubric, we calculated a correlation matrix out of every registered rubric. A high correlation indicates that the tested parameters evaluate the same precept; otherwise, the parameters are independent.

Table 4. Correlation matrices for each judge.

Professor 1								Professor 2
	Or	Im	Di	Fl	Te	Re			Or	Im	Di	Fl	Te	Re
Or	1						3.12	Or	1						3.56
Im	0.77	1					2.8	Im	0.71	1					3.23
Di	0.54	0.48	1				3.3	Di	0.74	0.55	1				3.21
Fl	0.57	0.49	0.82	1			3.35	Fl	0.7	0.54	0.79	1			3.31
Te	0.59	0.53	0.72	0.79	1		3.31	Te	0.67	0.73	0.49	0.61	1		3.22
Re	0.64	0.53	0.74	0.68	0.68	1	3.27	Re	0.74	0.69	0.64	0.67	0.72	1	3.45

shows the resulting matrices, including every parameter for each judge. Additionally, the last column of each matrix shows the sum of the correlations to enable the parameters’ sorting.

It can be seen that Professor 1 found little difference between “Flexibility” and “Divergence,” as well as between “Technique” and “Flexibility.” In turn, “Technique” and “Flexibility” were the parameters showing the least independence; therefore, they were removed. Similarly, Professor 2 had high correlations, except for the intersections of “Technique” and “Divergence,” whereas “Impact,” “Originality,” and “Resolution” are those showing less independence. Thus, “Originality” and “Resolution” were removed. The resulting models are as follows:

HC1 = Or + Im + Div + Res

(1)

HC2 = Im + Div + Fle + Tec

(2)

A correlation increase between both judges was expected by removing the redundant parameters if the overall CTS concept was shared. It is important to note that “Technique” and “Resolution” are the lowest priority parameters in Table 3, while “Impact” and “Divergence” have the highest. Hence, despite having different results, both teachers identified a scale of similar priorities, and their resulting models were not discordant, despite using different parameters.

The correlation was analyzed between HC1 and HC2 to find a concordance of 92.27% between both judges. However, using all the global parameters (even the redundant ones) produced 81%. It indicates that both judges perceived and evaluated CTS equivalently but that the generated models significantly improved the numerical accuracy of the CTS scale.

2.4. Tracking CTS Enhancement among Students

Various factors could explain the existence and development of CTS outside the experiment. For instance, these could be autogenous of the student. In addition, they could be related to each student’s academic performance under the premise that maintaining a better grade requires a more significant effort to achieve favorable results. To this end, the students’ academic records were analyzed to weigh their CTS outcomes in their first year’s academic performance. Specifically, we calculated the correlation between the students’ cumulative averages and the HC1,2 model outcomes on a scale of 0 to 100. A high correlation would imply that the CTS model is ineffective as it only reflects the students’ academic profile.

Even though the class “Creativity and Digital Design” is taught in the first semester, we also analyzed the second semester to form a more solid profile of their academic skills in general. In addition, we analyzed the correlation between the results of the HC1,2 models with all the subjects on the students’ records, including Creativity and Digital Design. The purpose was to study the independence of the models over other styles of course content, class, and assessment.

Finally, the results of the experimental and control classes were compared, removing the biases that the academic profiles of the students could carry. Thus, it can be seen that, by following the above procedures, the proposed CTS evaluation can be effectively disassociated from (1) judges’ biases, (2) students’ academic profile, (3) academic program, and (4) learning technique.

It is essential to mention that the students’ data was not stored by replacing their names and other identifying information with an automated ID to evade individual tracking. The captured information was only used to benefit the present study, and no further direct or indirect use was given to it.

3. Results

Using the HC model, the evaluations of each student’s activities were weighted. Figure 2 shows the percentage of evaluations for the experimental (E) and control (C) groups, separated into quintiles. The trend lines for each group have also been added to observe the scores’ distribution qualitatively.

Figure 2 shows the very similar distributions between GE1 and GC1 and between GE2 and GC2. Then, there is no difference in the development of CTS when considering the application of the learning strategies used in this study. However, there is a correlation between the pairs GE1/GC1 and GE2/GC2, associated with the Bicultural and Multicultural programs, respectively. In turn, Figure 3 exhibits no noticeable chronological change in the evaluation of CTS for any group. In other words, there are no differences between the two learning styles; there is also no improvement or detriment to the students’ CTS. However, regarding students’ perception within the experimental groups, they expressed that they had improved their skills due to the AL strategies.

It can be noted that the evaluation trend lines have the same shape and that the observed difference between the Multicultural and Bicultural groups remained. This result verifies that the activities posed the same “challenge” to all the groups and that the shown differences should respond to similar factors other than the learning strategies.

Finally, Figure 4 shows the scatter analysis of each class for the average of all activities per student. Again, the above statements hold, highlighting the difference according to the academic program to which each class belongs. As an additional parameter, it is possible to see a slight change in the dispersion between the experimental and control groups, finding less variability in the experimental groups than their control counterparts. To complement these observations,

Table 5. Numerical data from Figure 4.

	GE1	GC1	GE2	GC2
MAX	7.69	7.61	9.29	10.04
MED	6.09	6.25	8.16	8.06
MIN	5.09	5.18	6.63	6.29
Q1	5.78	5.69	7.67	7.4
Q3	6.44	6.75	8.63	8.83
IQR	0.66	1.06	0.96	1.43
VAR	0.39	0.43	0.43	0.86

provides details of some metrics from Figure 4.

Table 5 enables the following observations: the experimental groups found relative variations from the median of about 1.8%; from their maximums, about 4.3%; and from their minimums, approximately 3.5%. Similarly, their first and third quartile relative variations were 2.6% and 3.5%, respectively. These results indicate that there was no statistical distinction between the experimental and control groups.

However, the IQR of the experimental groups presented a relative variation of 35.6%, whereas the variance was 29.5%. In other words, the experimental strategy had a slight effect on the students’ grades, which became more homogeneous. So then, a decrease in the dispersion of students’ outcomes can be associated with the experimental strategy.

According to the medians calculated, the classes in the Multicultural program obtained higher overall results than the bicultural groups by 31%. However, their dispersion was more significant by 41% and 55% in the IQR and variance, respectively. Indeed, despite the increased dispersion, it can be observed that there is no overlap between both programs’ dispersion metrics.

It is noteworthy that most students in the experimental classes expressed wanting to continue with courses employing AL, perceiving the strategies as valuable and motivating. The students’ interest was evidenced by their greater participation, better discipline, and willingness to give up distractions such as phones. On the other hand, individuals in the traditional classes expressed wanting to experience the teaching techniques that their classmates had received. The teacher emphasized that students in the experimental groups perceived their results as “better” than their peers taking the conventional courses. As shown above, they were convinced that the class structure benefitted their CTS without effectively benefiting their performance. This issue is in line with the perceptual studies commented on in the Introduction.

4. Discussion

The study findings revealed a correlation of approximately 41% between the CTS model and the final evaluation of the control groups, a correlation that was not reflected in the experimental groups. Possibly, the teacher could focus on developing these skills by not having to create the FL and the gamification of the course. It should be noted that the inverted classroom and gamification techniques also intervened numerically in the final grade, possibly downplaying the weighting of CTS. Therefore, a study that reveals the correlations among these factors would have to be carried out to confirm their effect.

Although the control and experimental groups had students from both academic programs, a more significant correlation of CTS was observed in the Multicultural program. The admission requirements for these two programs do not differ drastically, only requiring a better English level for the Multicultural program. By design, this program aims to strengthen skills associated with developing in diverse cultural environments and interacting with people from different countries. Therefore, most of the courses in the program exercise such a different focus, and an academic trip abroad is mandatory. However, all the participants in this study belonged to the first semester, where the strategies targeting skills development are incipient. Then, the differences found between the two programs seem to be unrelated to the said design. Indeed, applicants and enrolled students subjectively associate the Multicultural program with a higher academic level so that the results appear to be more related to the student’s predisposition.

It is noteworthy that activities 4, 5, and 6 showed the lowest grades. These activities required the usage of specific software over which the students did not have mastery; their lack of experience probably explains the notable differences relative to other activities. As Baer (2015) [53] and Aleksić et al. [54] state, creativity requires a certain level of expertise. On the other hand, CTS took a noticeable leap in their final project, associated with a civic association. The work teams were larger than those in the other activities, leading to discussion spaces; also, an increased commitment was evidenced because they worked on an actual project to benefit an association. Both the discussion spaces [1] and the students’ interest in the project [18] were factors that seem to have positively influenced their expression of CTS.

It has been reported that the use of the FL has shown indications of helping favor creativity. However, it is recommended to prepare the students beforehand to take on a class using AL because some are reluctant to change [55]. In this case, although the students were not prepared previously, their comments over the results depict engagement and a high self-perception, contrary to the discussed reluctance.

The results suggest that incentivizing CTS does not require a change in the course structure, unlike H1. Instead, their enhancement is more linked to the type of activities, how a student feels about his work environment, his experience with the tools used [6], and the spaces for the discussion of ideas [56]. Finally, the commitment to the expected result, related in this study to implement the project for a civic association, is also relevant in favoring CTS. Hence, the different sources comprising the students’ “engagement” must be precisely defined and identified as they may be perceptually similar, but their effect on CTS is different.

5. Conclusions

The objective of this research was to evaluate the impact of active learning strategies over the enhancement of CTS and tried to answer the questions: What is the relation between active learning and a creative solution in solving a problem? How does an engaging environment affect the development of CTS? The study was conducted with middle-school students from two academic programs at a private school in Mexico. The study worked with two control groups and two experimental groups. The results suggest no influence of the learning technique in expressing CTS, except for a slight homologation of student outcomes, not supporting H1a or H1b. In contrast, CTS seems to be related to factors external to the learning strategy, such as the type of activity, the tools needed to perform the activities, the actual relevance of the results, peer discussions, expectations, and self-appreciation. This finding aligns with those presented by Kember & Leung [18] and Törnqvist [32].

In addition, it was found that the type of activity and the teamwork interaction affected CTS the most. Moreover, those who participated in the AL evaluated themselves higher than their peers in the traditional classes. These results highlight the independence of CTS toward the referred AL and set a departing point for further research addressing the course activities’ qualities seemingly related to CTS enhancement instead of the learning strategy. Results favoring AL for CTS enhancement supported by students’ perceptions might be biased, as shown above.

In qualitative studies with a similar aim, the investment of time in developing creative products for the class results in enhanced expressions of CTS; this study did not see such impact, rejecting H1b. In addition, whereas the learning environment did not influence CTS, it contributed to the students’ engagement and favored an unsupported high self-assessment of their CTS. Thus, it seems advisable in future studies to evaluate the specific activities that enhance CTS and those factors that discourage them and the potential biases that arise from the perceptions of the teachers and students.