The Impact of a Sustainable Progressive STEAM Program on Primary School Students’ Critical Thinking Dispositions and Mathematics Achievements

Abstract

This study, which was supported by a quantitative research paradigm with two experiments and two control groups of 4 × 4 quasi-experimental design, aimed to find out the impact of a sustainable progressive STEAM (SP-STEAM model) application model on 5th-grade primary school students’ critical thinking dispositions and mathematics achievements in North Cyprus. The treatment model was applied to two independent experimental groups for 14 weeks. The split-plot multiple group analysis of variance (split-plot ANOVA) statistical techniques was used to calculate between- and within-group significances regarding exogenous variables. The SPSS-24 software package was used for the analysis. The Pre-test and post-test results deriving from the experiment and control groups revealed a significant effect of the SP-STEAM model upon the 5th-grade primary school students’ critical thinking dispositions, as measured using the CCTDI, mathematic achievements, as well as measured using a dedicated exam. The results were discussed in detail in light of the related literature, with suggestions for further studies proposed.

1. Introduction

In recent years, the demand for improving the critical thinking dispositions (CTDs) of primary school students led educators, researchers, and program developers to consider issues related to developing and adapting effective intervention strategies [1,2,3,4]. The basic premise behind this movement is related to the notion that the earlier students encounter opportunities to gain critical thinking dispositions, the more effective and successful they will be in critical thinking and academic achievement in their future careers [1,5]. For this reason, studies have proposed different strategies to nurture or enhance the CTDs of students at different levels of education. The most intriguing strategy to foster CTDs and mathematics achievements of primary school students was the application of STEAM (Science, Technology, Engineering, Arts, Mathematics)-oriented educational programs [6,7,8], which has brought a paradigm shift in education. STEAM-oriented educational programs are considered an inevitable part of helping primary school students gain CTDs and improve their skills in science, technology engineering, arts and mathematics [9]. Nonetheless, a paradigm shift in education requires important educational policies, and numerous countries have tried different strategies to adapt their educational settings to STEAM education as a new paradigm. A very recent and comprehensive book titled ‘Status and Trends of STEM Education in Highly Competitive Countries: Country Reports and International Comparison’ summarizes the issues and solutions that several countries have experienced and tested [10]. As Lee and Lee summarize, the common issues among many countries such as Canada, Finland, Germany, Hong Kong, Ireland, Singapore, Sweden and Taiwan, were teacher training, materials, and methods of STEAM education. For that reason, the paradigm shift began with teacher education and training and continued with material development and methodology of teaching in those countries [10]. Teachers have been considered as the most crucial part of STEAM education. A very recent study which systematically reviewed experimental studies regarding STEAM applications worldwide reported that 95% of experiments have first concentrated on training teachers prior to any treatment application, and summarized that any application without training of teachers would not be effective [11]. While studies report that experimental studies regarding STEAM education will mostly fail without in-service teacher training, the question arises as to why teachers are found to be inadequate in application of STEAM in classrooms. The answer to this question is based on the pre-service teacher education curriculum within the scope of many studies [12,13,14,15]. In STEAM applications, the question is actually how it is taught rather than what is taught. A study carried out in Turkey suggested changing the policy and curriculum in Teacher Education Programs within Education Faculties throughout Turkey because graduate teachers see students as passive receivers of information rather than active interpreters of knowledge and experience [16]. Therefore, STEAM and its applications for the betterment of education and for expected transitions in societies should be understood well in order to know where to start the paradigm shift.

STEAM, by many researchers, is defined as an educational approach that aims to provide students with interdisciplinary cooperation, openness to communication, ethical values, research, production, and problem-solving skills, using creativity by focusing on the engineering design of knowledge and skills, in the fields of science, technology, engineering, and mathematics [1,6,9,17,18,19,20,21,22,23,24,25,26,27]. There are many different studies that reveal the strong link between STEAM education, critical thinking, and mathematic achievement, yet most of them are theoretical and need to be empirically tested [25,28]. Lately, some studies conducted on the phenomenon of critical thinking and STEAM education display some empirical support for the strong tie between the application of STEAM education and improvement in CTDs and mathematic skills [2,19,29,30]. These studies, however, carried out correlational effects, which are valuable in supporting the proposed theory, yet are insufficient in supporting the true causal differences that STEAM education programs possibly made on students’ CTDs and their academic achievements. With respect to the specified lack of related literature on the possible casual comparative effects of STEAM applications on primary school students’ CTDs and academic achievements, the current research aimed to test the impacts of a sustainable progressive STEAM model on primary school students’ CTDs and their mathematics achievements.

2. Statement of Problem and Aim of This Study

North Cyprus (NC) is one of those countries that is ripe for an upcoming drastic change in its educational system. Although it is known that STEAM education has some rigid principles in application, it also does maintain some flexibility to be embedded in an existing educational system [7]. In addition to the need for STEAM education, researchers question the way it is administered and practiced [7,25]. This is crucial because every society has its own dynamics and cultural realms. The extraneous factors that might possibly affect the process of administration of this approach are unknown, and there are no experimental results supporting the success of such an administration. This is quite important, because not every practice gives the same results and not every culture holds and responds to a new design the same way. For that very reason, it has long been an urgent need to design a STEAM education approach to be embedded in the educational context of NC and to elicit empirical evidence regarding its effects on the targeted audiences’ academic achievements and thinking qualities. The vision of NC education for 2030 strategic aims, which were determined by the Ministry of National Education, underlines the importance of appropriate application models of STEAM education [31].

If a country wants to have a say in scientific, economic, or technological fields, STEAM education must be considered in its education systems [26]. However, in-depth research carried out found that studies on STEAM applications for primary school students are quite limited in NC. Therefore, the STEAM application within the classroom at the primary school level was deemed necessary. One of the results of the Vision 2030 Education Strategic Plan Workshop Report, as announced by the Ministry of National Education of North Cyprus, was the call for studies on various adaptable applications of STEAM education models.

Based on a developed framework and as part of a larger study, the current research, which is supported by a quantitative empirical paradigm, aims to figure out the possible effects of a sustainable progressive STEAM education model on groups of 5th-grade primary school students’ CTDs and their academic achievements in NC, asking the following research questions:

Research Question 1: Will groups of 5th-grade primary school students who are instructed by a SP-STEAM education model, show statistically significant differences in comparison to similar groups of 5th-grade primary school students who are instructed with a traditional education model in terms of their critical thinking dispositions?

Research Question 2: Will groups of 5th-grade primary school students who are instructed by a SP-STEAM education model show statistically significant differences in comparison to similar groups of 5th-grade primary school students, who are instructed by a traditional education model in terms of their mathematics achievements?

3. The Experiment

3.1. The Sustainable Progressive STEAM Model

When the Ministry of Education of North Cyprus clearly explained the long-term objectives of the national education, a special quotation opened for the sustainability of problem solving skills among children. For this reason, the SP-STEAM model was developed especially for primary school students to help and guide them, nurture their critical thinking dispositions, and develop their academic achievements in mathematics.

Starting with the term sustainability, we not only expect our children to develop positive attitudes toward critical thinking and elicit high academic grades but also want them to maintain these skills throughout their academic careers and succeed in life in the long term. The term sustainability in our research has two levels. Sustainability is considered at the macro and micro levels. At the micro level is the sustainability of a systemic approach at the schooling level, whereas; sustainability at the macro level refers to the successes of individuals who have graduated from an educational system. For the latter, it is too early to make inferences; but for the former, this research will create a basis for further negotiations.

The sustainability of any gained positive experience is understood to be crucial for further educational and professional careers of children and youth [18,29]. That is to say, a treatment model that is not sustainable will fail in the following stages of life, and thus, children will fail to think critically and solve problems. The core element for the sustainability of any given positive experience is characterized by a paradigm shift in the way children think [9,30]. This is also known as habit of mind. If a certain way of thinking has become a habit of mind, then it becomes an inevitable part of a person’s thinking, which is where a person begins to establish unique characterological profile to question the universe around them in a unique way [3,23]. This is especially true if high-quality education is provided to children as early as possible, and this establishes the core of sustainability. In our application of the SP-STEAM model, great importance was given to sustainability. Specifically, a dedicated STEAM instruction approach was developed according to the needs of a particular group of students. We believe that we cannot avoid rote learning by developing a specific approach for a specific group of children.

The SP-STEAM Model also has a progressive dimension, which refers to the gradual tracking of each person’s performance upon a dedicated scale. For each child, a detailed 10-point dichotomous scale performance tracker was developed. As discussed in length across the following sections of this manuscript, a detailed in-service education was provided to teachers in utilizing this assessment tool. As detailed further, teachers, the applicators of any given school of thought, need to be in a constant state of development in order to keep themselves fresh [32]. For each subject matter and unit in mathematics, this tool was used as supplementary material to guide teachers in the process. Teachers obtained opportunities to guide children’s learning, especially where they noticed any obstacles, misconceptions, and struggles, that could possibly be experienced by the children. With these means, the children were expected to self-screen their improvements while simultaneously gaining confidence in scientific thinking. On the other hand, it is believed that the teachers would better monitor children’s improvements, and provide better feedback in turn. This cyclic progressive approach would enable children to build experience upon experience, finding opportunities to practice and gain a scientific thinking culture. The classroom atmosphere was designed for the experiment groups; thus, student-student, student-teacher, group-group, and group-teacher interactions were enhanced. In the promotion of problem-solving, research-based collaborative techniques were expected to be used by the teachers, in turn establishing an expectancy for the students to practice critical thinking. The detailed schematic expression of such a classroom atmosphere is mimicked in Figure 1 below.

Based on the research context explained above, detailed are some expected outcomes and assumptions, regarding the reasons why the treatment groups are expected to have achieved better than the control groups. The treatment was considered to be done in a classroom atmosphere, where students concentrate on solving life-oriented mathematic problems rather than being passive receivers of information. As explained by the related literature, students, who are encouraged to solve meaningful life problems, are expected to become more successful in mathematics [5,8,17,33]. Moreover, it was assumed that the students in the treatment groups were provided with opportunities by the classroom teachers, to clarify and redefine the objectives of the problem, and deal with issues, add context, experiment with materials, and build collaborative group works. It is additionally assumed that they would be reinforced in generating ideas via brainstorming in group discussions. This process was believed to help and guide them in using their potential to formulate, discuss, use different strategies, design, interpret, and evaluate ideas for possible solutions of problems. Such a classroom environment where students were encouraged to develop their own ideas in solving mathematic problems would establish the students as not only being successful in mathematics, but would also display high performance in critical thinking [8,17,33]. The bulk of the literature highlighted the strong tie between analytic thinking, which is an inevitable facet of critical thinking, and mathematics [9,10,12,13,14,27,34,35].

A school that is denoted with hi-tech science laboratories, technology laboratories, mathematics practice tools, robotic applications, and updated content for subject matters is not enough alone to build a decent critical thinking culture for children and youth [26]. Highly trained teachers with philosophical perspectives are required, positioning them as good role models for children [32]. Besides the scientific part, the artistic dimension of becoming an effective teacher plays a significant role in this experiment [3,9]; hence, the detailed and dedicated training program developed for teachers prior to the application of this dedicated SP-STEAM Model.

3.2. Teacher Training Process

The teacher training program was developed and carried out by two independent teacher trainers, who were appointed by the North Cyprus Ministry of Education for a European Union project, and have more than 10 years experience in providing in-service teacher training for primary school teachers. After obtaining their consent to participate in this study as teacher trainers, the researchers met with the trainers several times, negotiating the expected outcomes of the experiment. As a result of the meetings, a consensus upon the aim and expected outcomes of the teacher training program were reached. Accordingly, the teachers within the experiment groups would be trained in terms of methodology, the materials, and technology they would use during their instructions. The experimental group teachers were informed about the application of problem-solving-based teaching methods and collaboration in teaching mathematics. The training lasted for 5 weeks and included workshops, where teachers practiced linking every subject matter to real life situations, and produced materials for learners to solve mathematic problems. As asserted by several researchers, learning mathematics is more meaningful and interesting if problems are linked to children’s living quarters, establishing more enjoyable problem solving [12,14]. For this reason, the first and an important facet of the teacher training program, was that teachers were trained to produce examples and activities enjoyable for the learners, so that the problem solving process would be more interesting and appealing. The second facet of the teacher training program was to help teachers create lesson plans to cover the expected outcomes. The lessons were expected to be designed in a way that would enable both the teachers and the students to practice their thinking skills and subject maters in a systematic way. A five phase systematic teaching approach has been developed and introduced to teachers via training and workshops. This five phase systematic teaching approach developed for embedding the SP-STEAM model, is explained in detail in Section 3.3 below.

3.3. Lessons in Experiment and Control Groups

The lessons within the control groups were delivered as usual. The teachers in these classrooms were not enrolled in any teacher training programs. Both groups followed the same curriculum, however, students in the experimental groups followed a totally different procedure of teaching learning situations. Conventionally, in a majority of the schools in North Cyprus, lessons were held by utilizing direct teaching methods and materials, where students were passive receivers of information, and the teachers were active transmitters, constituting as a motivating point which triggered the conduct of this and similar studies. Within the experimental groups, teachers utilized the SP-STEAM model, where students were provided with opportunities to use their potential, form ideas for problem solutions, and enjoy the practicing of subject matters. The schematic expression of instructions can be seen in Figure 2 below.

The first phase of instruction was establishing a basis for the upcoming phases of instruction, and was thought to be the phase of preparation. Students were given a sample situation where they can generate a problem out of a given situation. The situation was chosen as a real life situation and an attractive one, for enjoyment while working on it. Students, in this phase, were allowed to collect data, discuss, and find a problem to be solved, under the guide of teachers. Students were instructed to work in groups, as can be seen in Figure 1. They were permitted to interact with group members, and with other groups as well. Teachers acted like moderators and guiders. Here, the important point was allowing the students to work together in a collaborative classroom environment, as per the suggestions of many researches, in order to establish a decent SP-STEAM education model [2,6,9,17,19,21,32,33]. The second phase characteristically shared the same approach as the first phase; however, each group was expected to come up with possible solutions to the problem defined in the earlier phase. They could come up with one solution or more. During the third phase, the students again worked in groups and decided on one solution to a given problem, however; the important element here was to explain the rationale behind the solution they saw to the problem. Choosing a solution means to eliminate other solutions; thus, the point here was to allow students the experience of giving up on something in the knowledge of other existing options and their consequences if they are chosen. As students practice mathematics, they also practice their reasoning abilities via a systematic method of thinking. In this way, they were also expected to practice critical thinking as well. Studies conducted worldwide underline an important dimension of the SP-STEAM education models, especially in teaching mathematics, that unless students find opportunities to practice mathematics without developing a rationale behind any solution to any given problem, students only learn in schools but do not solve problems in life [8,17]. In the fourth phase, students presented and displayed how their solutions worked to solve the problem they defined. Here, students could redefine a solution or adapt it to make the solution work better. Finally, at the fifth phase, students summarized their results and made some inferences out of their evaluation of the whole process.

The experimental group lessons generally followed these five phases. Teachers were trained (detailed teacher training process explained in the previous section) to prepare their lessons considering the problem solving sequence as explained in Figure 2. While the experiment groups lesson design was in this innovative format, both the control and experiment groups followed the same curriculum content, with the control groups still using the conventional teaching approaches. The curriculum content, subject matters, and objectives of the primary school mathematics education were prepared by the North Cyprus Ministry of Education and sent to all public and private schools prior to the beginning of each new semester in North Cyprus. The current and renewed curriculum content and objectives can be seen in

Table 1. Program Outcomes and Content Domains of the Primary Mathematics Education of North Cyprus for 2022–2023 Spring and Fall Semesters.

General Program Outcomes	Content Domains
Defining the Problem Students will be able to draw a figure expressing the problem create a number of sentences related to the problem detect missing or excess information about the problem examine the logic of the information given about the problem recognize hidden information about the problem	Operations with Natural Numbers Operations with Fractions Operations with Percentages Decimal Notation Geometric concepts and illustrations Data collection and evaluation Measurement Measuring time Measuring areas
Strategy Determination and Control Students will be able to collect, organize, and interpret information simplify information use number sentences and select operations develop abilities to guess develop their mathematical reasoning apply modal solutions break problems down into smaller parts use models to solve problems express the result and explain why it is true
Communication, Presentation, and Expression Students will be able to explain each step of the problem in detail use appropriate symbols, concepts, and terminology apply grammar rules
Writing a Problem Students will be able to write a problem in accordance with the given operation(s) write a suitable problem based on a given result write a problem for a given theme/topic complete an unfinished problem

below.

No restrictions in the distinctive application of these phases were applied, and teachers were permitted to merge some phases according to the flow of their lectures. For better understanding of how these phases generally functioned in real life situations, the following example in

Table 2. Example lesson designed and applied according to the SP-STEAM model.

Phases	Events
1	1. Students are divided into groups of 3–4 people. 2. Groups are asked to design nests for birds on the trees in the school garden, and the following explanations are made for this purpose: Scientists estimate that there are 10,000 different bird species worldwide. You do not have to go far to see some of the different birds found in nature. You may see them around your home as well. How can these birds survive in difficult conditions? What do they need to live and grow? It is also very important that birds have food, water, and shelter and can breathe. 3. A group discussion environment is created about what to do, and they are asked to write their ideas. They are asked if they watch birds in nature (have you ever watched birds in your garden or playground? What do birds do? etc.) 4. Students are asked to clarify the features of the bird nest they will design. 5. They are asked to draw a sketch on a worksheet. 6. They are asked to create a material list by considering which materials are required. 7. They are asked to plan the design process and report everything they will do step-by-step. 8. They are asked to continue researching until the second stage, clarify their thoughts on the design, and bring the necessary materials to the class.
2 and 3	9. The design plans of the groups and the materials they bring accordingly are examined, and their explanations are obtained. 10. Design processes are monitored and guided. 11. Probing questions (sir, I meant deepening questions) are asked to help them think and look from a broader perspective. For example: How do you think your design meets a bird's basic needs? What parts of your design provide food, water, or shelter for the bird? 12. Groups explain and report what they did and why they did it. 13. They are asked to give the final version of the design and report the features of the design, the materials used, and the points they paid attention to while using it. 14. Until the next stage, they are asked to question their designs, gain opinions from experts, and report.
4 and 5	15. They are asked to explain what they did in the previous stages. 16. They are asked to place the nest on the trees (teacher supported). 17. Groups are observed as they place their designs and asked to examine what they did and why. 18. They are asked to explain their designs to other children in the garden. 19. They are asked to evaluate/discuss the similarities and differences among the designs.

can be considered.

4. Methodology

4.1. Research Design

Based on the cause and effect and causal-comparative nature of the study, quasi-experimental research with experiment and control groups was designed to test the effects of the defined exogenous variable (A Sustainable Progressive STEAM Model) on the defined endogenous variables (CTDs and Mathematics Achievement). As an exogenous variable, a sustainable progressive STEAM education model was taken as the main treatment strategy. Since a relatively newly designed approach was being tested, applicators, who were teachers of 5th graders, were enrolled in an in-service teacher training program prior to any application of pre-test and post-test applications. The ingredients of the so-called in-service teacher training program aimed to prepare applicator teachers for the experiment by denoting them with all the necessary knowledge and experience regarding the SP-STEAM education model, preparation of lessons, measurement, and evaluation. This research was designed in a way that ensures the control of most of the variables that might possibly be affecting the results of the experiment. For this reason, most of the extraneous variables were extracted and evaluated carefully. For an experiment to be truly valid and reliable, the selection of participants and research settings played crucial roles [36]. In order not to cause any deficiencies regarding the reliability and validity of the experiment, all precautions suggested by the pioneers of the field were followed inclusively [36].

4.2. Population and Sample

The population for which the results of this study would be generalized is composed of all 5th-grade primary government school students in NC (N = 1200–1500). With respect to the mainframe of this population, the research setting might have been any one of the government schools, which could have been randomly selected from the list of schools. In this scenario, all the schools in the list had an equal chance to be selected as a research setting, with every student in that randomly selected school having an equal chance to be selected as a participant for both experiment and control groups. This is also valid for the applicators, who were the teachers in this case. However, for the current research, pre-existing groups of 5th grade students were selected for both the control and treatment groups, because of the limited research conditions, which turned the experiment into a quasi-experiment research design.

The sampling procedure for this experiment yielded two experimental groups and two control groups (4 × 4 pre-post experiment design). Participants were randomly selected to establish the experimental and control groups from all the 5th graders in a public school in NC, with 26 students in each group, making a total number of 104 students. The groups were coded as experiment group 1 (n = 26 with 15 girls and 11 boys), experiment group 2 (n = 26 with 19 girls and 7 boys), control group 1 (n = 26 with 16 girls and 10 boys) and control group 2 (n = 26 with 16 girls and 10 boys).

4.3. Data Collection Tools

The California Critical Thinking Disposition Inventory (CCTDI) [37] was used to measure 5th-grade primary school students’ critical thinking dispositions. This tool was selected because of its grounding in the APA Delphi Report (American Philosophical Association), which has achieved cross-consensus on the conceptualization of critical thinking, its conceptual clarity [38], and its design in measuring different aspects of critical thinking disposition [39]. As the CCTDI originated in the United States, care was taken to ensure its suitability as an instrument in assessing the same dimensions for Turkish speaking students. A Turkish translation of the CCTDI was tested in the TRNC by İskifoğlu [40] and was found to be equivalent to the English original in its validity and reliability. The Turkish version of the CCTDI was also adapted in a second form to be suitable for primary school students by İskifoğlu [40]. The CCTDI is composed of 75 items rated on a 6 point, forced-choice scale (1 = totally disagree, 2 = disagree, 3 = partially disagree, 4 = partially agree, 5 = agree, 6 = totally agree) and intends to measure 7 dimensions of critical thinking dispositions with 7 sub-scales. The scores for each of the seven subscales range from a possible minimum of 10, to a possible maximum of 60. Scores of 30 or below indicates a negative tendency toward that subscale; scores of 31–39 suggest ambivalence; scores of 40 or higher are evidence of a positive inclination, and scores between 50 and 60 indicate a strong positive tendency. The CCTDI total score is the sum of the seven subscale scores, and can range from 70 to 420; a total score of 280 or higher indicates a positive disposition toward critical thinking in samples [39,40]. The translated Turkish version and the original English version of the CCTDI displayed positive psychometric properties, thus supporting the applicability of the CCTDI in a Turkish educational context, with alpha coefficients ranging from 0.81 and 0.90 for the sub-scales of the CCTDI in Turkish. Results also supported high content validity indicates of 0.81 and 0.97 [40] and high reliability scores for the sub-scales of the CCTDI; (1) Truth-seeking (12 items, α = 0.72), (2) Open mindedness (12 items, α = 0.73), (3) Analyticity (11 items, α = 0.72), (4) systematicity (11 items, α = 0.74), (5) Critical thinking self-confidence (9 items, α = 0.78), (6) Inquisitiveness (10 items, α = 0.80), and (7) Maturity of judgment (10 items, α = 0.75).

The second measurement tool utilized was the Mathematics achievement test, composed of 25 items specifically developed for this experiment. The development of the achievement test included several distinct processes, such as the preparation of the item pool from the related content domains, content validity check with the subject matter experts, pilot study, and reliability analysis. The initial item pool included 75 items. 28 items were then removed by two independent experts consistently, as they were considered either too difficult or too easy for the targeted audience. More importantly, every single item was checked against the related content domain, in terms of their relevance. The content validity indices (CVIs) for the rest of the 47 items ranged from 0.83 to 1, which symbolizes high acceptability for content validity [36]. Following the content validity check, items were first used to collect some initial data from a similar group of students (n = 200). The collected data were processed to make inferences for the reliability of items and the entire test in general. Since there was only one correct answer for each multiple-choice item, the Kuder-Richardson 20 formula was utilized, as shown below.

KR20 [k/k − 1] × [1 − (Σp_j × q_j)/σ²]

The KR20 was calculated using the Excel software version 2023, and applied to assess the degree of reliability for each item of the mathematics achievement test. When the results of analysis were interpreted, 20 items were found to be lower in terms of their reliability. Therefore, items with a KR20 score of 0.85 and above, were selected to establish the final form of the test. Finally, 25 items with high content validity indices ranging from 0.9 to 1, and with high KR20 scores ranging from 0.85 to 0.89, were chosen as the items to be used in the experiment, in assessment of the mathematics achievements of 5th grade primary school students. The detailed item content validity indices and KR20 values of each question in the mathematics achievement test, are displayed below in

Table 3. Item content validity indices and KR20 scores of mathematics achievement test.

Question Number	I-CVI 1	KR20
Question 1	1	0.85
Question 2	0.9	0.89
Question 3	0.9	0.85
Question 4	0.9	0.88
Question 5	0.9	0.87
Question 6	1	0.85
Question 7	0.9	0.85
Question 8	1	0.85
Question 9	1	0.88
Question 10	0.9	0.85
Question 11	1	0.86
Question 12	1	0.85
Question 13	0.9	0.86
Question 14	1	0.85
Question 15	0.9	0.89
Question 16	0.9	0.85
Question 17	0.9	0.85
Question 18	0.9	0.88
Question 19	1	0.85
Question 20	0.9	0.86
Question 21	0.9	0.85
Question 22	0.9	0.86
Question 23	0.9	0.88
Question 24	0.9	0.85
Question 25	0.9	0.88

¹ I-CVI: Item content validity index.

.

The curriculum content domains and their correlated subject matters were considered the main source for the generated questions in the mathematics achievement test. The content domains and subject matters can be seen in Table 1. The final version of the mathematics test included 25 items, corresponding with the subject matters covered during the 14 weeks throughout the experiment. Sample questions included in the test can be seen in Figure 3 below. Because the medium of instruction at the schools is Turkish, the question statements were translated to English, to be included in this paper.

4.4. Data Collection Procedures

Prior to any data collection attempt, all the necessary permissions were collected from the Ministry of Education in North Cyprus (NC), from the parents of students and the students themselves, administrators, and applicator teachers. They were eventually informed about the aim of the research and informed about their rights in the study. They were also given the information that they could withdraw from the study at any time they desired without showing any reason for their withdrawal.

After eliciting acceptance from all parties, the next procedure was to invite the teachers for an in-service training program in order to inform them in advance regarding the SP-STEAM education model. The training included STEAM procedures (explained in detail in a specially prepared booklet), materials, and technology and assessment procedures. Following the completion of the teacher training program, data collection tools were administered to both defined control and experiment groups (pre-tests), in two distinct sessions. The first session included the demographic data collection and the administration of the CCTDI, and the second session included the administration of the mathematics achievement test. The former and the second sessions were administered on different days. In addition, participants were re-informed of their rights within the study, and informed on how they were going to take the tests and respond to each item in the inventory. After eliciting the initial data from the groups, the application of the treatment to the experiment groups followed as soon as the settings were ready. Meanwhile, the pre-existing procedures were maintained for the control groups.

An SP-STEAM education model was administered to 5th-grade primary school students within a primary school setting in two separate experimental groups for 14 weeks, with an expectation of positive changes both in their mathematic achievements and their dispositions toward seven defined critical thinking facets. Following the completion of the treatment and the achievement test, the CCTDI was re-given to both the control and experiment groups, in order to check on any positive increments and/or improvements in their scores. The most important part of the data collection procedure was to maintain the consistency between and among the administration of the test and inventory, in order not to cause any bias with relation to the data collection.

4.5. Data Analysis Procedures

To conduct any analysis and to answer the first and the second research questions, a preliminary analysis was conducted to ensure the homogeneity of the data set distributions for both the control and experiment groups. The mean scores, standard deviations, lower and upper boundaries in the 95 percent of a confidence interval, minimum and maximum observations, skewness levels, kurtosis levels, standard errors for mean and deviation scores, and normality statistics were all conducted in order to make further decisions regarding the type of inferential statistics required to be considered for the each of the variables and data sets. The first and the second research questions are the ones that directly question the effect of the treatment model. In such group-wise comparisons, the most important facets to be focused upon are those at the entry-level of the groups in terms of the so-called endogenous variables. Therefore, as an initial analysis, the entry levels of the experiment and control groups were analysed and were compared to each other, in terms of the defined endogenous variables. This comparison was done by independent samples from a t-test, rather than the Mann–Whitney U test, because the Kolmogorov–Smirnov test and the Shapiro–Wilk tests showed no significant difference between the distributions of research data and the hypothetical normal distribution. Our expectation here was to elicit a non-significant difference, which is a sign that no covariate is needed for further group-wise analysis. Once the prerequisite analyses were satisfied, the effect of the treatment model on the defined endogenous variables were tested via multiple-group split-plot ANOVA. This analysis was safely done with the SPSS 24 software. The logic behind selecting split-plot ANOVA was because the analysis involved two sets of scores within two independent groups for several endogenous variables (pre-test and post-test scores for two control and two experiment groups). In such a complex design, where control over variables is hard to manage, a dedicated approach for analysis is essential [36], as is for this case.

5. Results

The one-sample Kolmogorov–Smirnov test was used to determine if the data for the pre-test and post-test across experimental and control groups were distributed normally. The results gave non-significant sig (For 2-tailed bi-nominal distribution) values. For experimental groups, the alpha values showed to be 0.563 and 0.198 for each, and for control groups, the results also showed 0.110 and 0.174 for each variable. Based on the evidence that the results derived from, these data sets came from a normally distributed population, and parametric difference tests were preferred to test the hypothesis, because the research satisfied the preconditions of the F-test, suggested for experimental researches in social sciences [36].

5.1. Split-Plot ANOVA Results

5.1.1. Impact of SP-STEAM Model on Critical Thinking Dispositions

Research Question 1: Will groups of 5th-grade primary school students who are instructed by an SP-STEAM education model show statistically significant differences in comparison to similar groups of 5th-grade primary school students who are instructed by a traditional education model in terms of their critical thinking dispositions?

Results Regarding Research Question 1: Before using split-plot ANOVA to compare pre-post test results across experiment and control groups, one-way ANOVA was conducted to compare only the pre-test results between the experiment and control groups. The results showed no statistically significant difference between groups with Sig. scores ranging from 0.741 to 0.117, which meant that all groups were equal in terms of their CTDs.

Using split-plot ANOVA, cross-sectional analysis differences were tested to figure out if there were any statistically significant differences between groups across the pre-post-test administrations. The results of the analysis showed that the experiment groups where the SP-STEAM program was applied performed better (see

Table 4. Descriptive statistics regarding seven facets of critical thinking dispositions across the pre-posttest design of control and experiment groups.

Variables	Design	Groups	Mean	Std. Deviation	n
Truth-seeking	Pre-Test	Experiment 1	24.1795	4.09035	26
		Experiment 2	22.3077	4.02760	26
		Control 1	20.8077	2.19124	26
		Control 2	24.2692	3.51634	26
		Total	22.8910	3.77422	104
	Post-Test	Experiment 1	33.7179	7.12285	26
		Experiment 2	32.9167	5.43778	26
		Control 1	20.5769	2.11987	26
		Control 2	23.6923	3.56392	26
		Total	27.7260	7.51916	104
Open-mindedness	Pre-Test	Experiment 1	22.1538	2.94879	26
		Experiment 2	25.5000	2.59615	26
		Control 1	24.9615	3.97473	26
		Control 2	22.6538	4.89034	26
		Total	23.8173	3.93334	104
	Post-Test	Experiment 1	36.3141	3.48148	26
		Experiment 2	30.9295	4.25785	26
		Control 1	23.3846	4.34582	26
		Control 2	21.6923	5.34991	26
		Total	28.0801	7.34182	104
Inquisitiveness	Pre-Test	Experiment 1	23.1154	2.86115	26
		Experiment 2	23.0000	2.28035	26
		Control 1	21.0385	2.40800	26
		Control 2	22.5000	3.40881	26
		Total	22.4135	2.85783	104
	Post-Test	Experiment 1	33.7308	4.37774	26
		Experiment 2	31.2692	5.43734	26
		Control 1	20.1923	2.65359	26
		Control 2	21.2692	4.38687	26
		Total	26.6154	7.35416	104
Systematicity	Pre-Test	Experiment 1	22.3077	3.51874	26
		Experiment 2	23.6154	2.57801	26
		Control 1	23.6923	3.72848	26
		Control 2	20.4231	2.53256	26
		Total	22.5096	3.36456	104
	Post-Test	Experiment 1	35.1748	4.30083	26
		Experiment 2	35.0699	4.99370	26
		Control 1	20.4615	3.30128	26
		Control 2	19.6923	2.60414	26
		Total	27.5997	8.48825	104
Analyticity	Pre-Test	Experiment 1	24.1795	4.09035	26
		Experiment 2	22.3077	4.02760	26
		Control 1	20.8077	2.19124	26
		Control 2	24.2692	3.51634	26
		Total	22.8910	3.77422	104
	Post-Test	Experiment 1	33.7179	7.12285	26
		Experiment 2	32.9167	5.43778	26
		Control 1	20.5769	2.11987	26
		Control 2	23.6923	3.56392	26
		Total	27.7260	7.51916	104
Maturity of Judgement	Pre-Test	Experiment 1	23.0385	5.86843	26
		Experiment 2	19.9615	3.91388	26
		Control 1	23.4231	3.63509	26
		Control 2	23.3846	3.71028	26
		Total	22.4519	4.55363	104
	Post-Test	Experiment 1	29.6538	7.00253	26
		Experiment 2	32.1154	6.59289	26
		Control 1	20.6154	5.26176	26
		Control 2	22.8462	3.84388	26
		Total	26.3077	7.43656	104
CT-Self Confidence	Pre-Test	Experiment 1	26.1154	3.79818	26
		Experiment 2	23.0000	3.40588	26
		Control 1	23.8462	2.52495	26
		Control 2	23.3077	2.42931	26
		Total	24.0673	3.28653	104
	Post-Test	Experiment 1	37.8205	6.17027	26
		Experiment 2	36.5812	6.97029	26
		Control 1	23.3462	3.74104	26
		Control 2	23.2308	2.61240	26
		Total	30.2447	8.66837	104
Overall Disposition	Pre-Test	Experiment 1	161.5641	10.28171	26
		Experiment 2	159.0385	10.86455	26
		Control 1	159.8077	8.37147	26
		Control 2	156.4231	6.57372	26
		Total	159.2083	9.23389	104
	Post-Test	Experiment 1	243.0554	14.79129	26
		Experiment 2	233.7770	14.92111	26
		Control 1	148.1538	6.86843	26
		Control 2	151.2308	7.69575	26
		Total	194.0542	46.17483	104

for means and standard deviations) across all sub-dimensions of the CCTDI, in comparison to the control groups where the progressive STEAM program was not applied.

Specifically, the empirical evidence supported that experiment group 1 and experiment group 2 performed significantly better in post-test results in comparison to the post-test results of control group 1 and control group 2 in all sub-scales of the CCTDI (see Table 3 for means and standard deviations; see

Table 5. Tukey F Post hoc results regarding mean differences for experiment and control groups across the pre-post-test design.

Variables	Groups		df	F	Mean Difference	Std. Error	Sig.
Truth-seeking	Experiment 1	Experiment 2	3100	32.541	1.3365	0.92407	0.474
		Control 1			8.2564	0.92407	0.000
		Control 2			4.9679	0.92407	0.000
	Experiment 2	Experiment 1			−1.3365	0.92407	0.474
		Control 1			6.9199	0.92407	0.000
		Control 2			3.6314	0.92407	0.001
	Control 1	Experiment 1			−8.2564	0.92407	0.000
		Experiment 2			−6.9199	0.92407	0.000
		Control 2			−3.2885	0.92407	0.003
	Control 2	Experiment 1			−4.9679	0.92407	0.000
		Experiment 2			−3.6314	0.92407	0.001
		Control 1			3.2885	0.92407	0.003
Open-mindedness	Experiment 1	Experiment 2	3100	90.636	1.0192	0.99117	0.733
		Control 1			5.0609 *	0.99117	0.000
		Control 2			7.0609 *	0.99117	0.000
	Experiment 2	Experiment 1			−1.0192	0.99117	0.733
		Control 1			4.0417 *	0.99117	0.001
		Control 2			6.0417 *	0.99117	0.000
	Control 1	Experiment 1			−5.0609 *	0.99117	0.000
		Experiment 2			−4.0417 *	0.99117	0.001
		Control 2			2.0000	0.99117	0.188
	Control 2	Experiment 1			−7.0609 *	0.99117	0.000
		Experiment 2			−6.0417 *	0.99117	0.000
		Control 1			−2.0000	0.99117	0.188
Inquisitiveness	Experiment 1	Experiment 2	3100	42.618	1.2885	0.75952	0.331
		Control 1			7.8077 *	0.75952	0.000
		Control 2			6.5385 *	0.75952	0.000
	Experiment 2	Experiment 1			−1.2885	0.75952	0.331
		Control 1			6.5192 *	0.75952	0.000
		Control 2			5.2500 *	0.75952	0.000
	Control 1	Experiment 1			−7.8077 *	0.75952	0.000
		Experiment 2			−6.5192 *	0.75952	0.000
		Control 2			−1.2692	0.75952	0.344
	Control 2	Experiment 1			−6.5385 *	0.75952	0.000
		Experiment 2			−5.2500 *	0.75952	0.000
		Control 1			1.2692	0.75952	0.344
Systematicity	Experiment 1	Experiment 2	3100	95.069	-0.6014	0.77982	0.867
		Control 1			6.6643 *	0.77982	0.000
		Control 2			8.6836 *	0.77982	0.000
	Experiment 2	Experiment 1			0.6014	0.77982	0.867
		Control 1			7.2657 *	0.77982	0.000
		Control 2			9.2850 *	0.77982	0.000
	Control 1	Experiment 1			−6.6643 *	0.77982	0.000
		Experiment 2			−7.2657 *	0.77982	0.000
		Control 2			2.0192	0.77982	0.053
	Control 2	Experiment 1			−8.6836 *	0.77982	0.000
		Experiment 2			−9.2850 *	0.77982	0.000
		Control 1			−2.0192	0.77982	0.053
Analyticity	Experiment 1	Experiment 2	3100	32.541	1.3365	0.92407	0.474
		Control 1			8.2564 *	0.92407	0.000
		Control 2			4.9679 *	0.92407	0.000
	Experiment 2	Experiment 1			−1.3365	0.92407	0.474
		Control 1			6.9199 *	0.92407	0.000
		Control 2			3.6314 *	0.92407	0.001
	Control 1	Experiment 1			−8.2564 *	0.92407	0.000
		Experiment 2			−6.9199 *	0.92407	0.000
		Control 2			−3.2885 *	0.92407	0.003
	Control 2	Experiment 1			−4.9679 *	0.92407	0.000
		Experiment 2			−3.6314 *	0.92407	0.001
		Control 1			3.2885 *	0.92407	0.003
Maturity of Judgement	Experiment 1	Experiment 2	3100	28.019	0.3077	1.09623	0.992
		Control 1			4.3269 *	1.09623	0.001
		Control 2			3.2308 *	1.09623	0.020
	Experiment 2	Experiment 1			−0.3077	1.09623	0.992
		Control 1			4.0192 *	1.09623	0.002
		Control 2			2.9231 *	1.09623	0.044
	Control 1	Experiment 1			−4.3269 *	1.09623	0.001
		Experiment 2			−4.0192 *	1.09623	0.002
		Control 2			−1.0962	1.09623	0.750
	Control 2	Experiment 1			−3.2308 *	1.09623	0.020
		Experiment 2			−2.9231 *	1.09623	0.044
		Control 1			1.0962	1.09623	0.750
CT-Self Confidence	Experiment 1	Experiment 2	3100	43.891	2.1774	0.87161	0.066
		Control 1			8.3718 *	0.87161	0.000
		Control 2			8.6987 *	0.87161	0.000
	Experiment 2	Experiment 1			−2.1774	0.87161	0.066
		Control 1			6.1944 *	0.87161	0.000
		Control 2			6.5214 *	0.87161	0.000
	Control 1	Experiment 1			−8.3718 *	0.87161	0.000
		Experiment 2			−6.1944 *	0.87161	0.000
		Control 2			0.3269	0.87161	0.982
	Control 2	Experiment 1			−8.6987 *	0.87161	0.000
		Experiment 2			−6.5214 *	0.87161	0.000
		Control 1			−0.3269	0.87161	0.982
Overall Disposition	Experiment 1	Experiment 2	3100	378.754	5.9020	2.27898	0.053
		Control 1			48.3290 *	2.27898	0.000
		Control 2			48.4828 *	2.27898	0.000
	Experiment 2	Experiment 1			−5.9020	2.27898	0.053
		Control 1			42.4270 *	2.27898	0.000
		Control 2			42.5808 *	2.27898	0.000
	Control 1	Experiment 1			−48.3290 *	2.27898	0.000
		Experiment 2			−42.4270 *	2.27898	0.000
		Control 2			0.1538	2.27898	1.000
	Control 2	Experiment 1			−48.4828 *	2.27898	0.000
		Experiment 2			−42.5808 *	2.27898	0.000
		Control 1			−0.1538	2.27898	1.000

*: p < 0.01.

for post hoc Tukey F-test results and significance levels).

When the F test results were interpreted with Tukey group-wise comparison results, it is clear that the SP-STEAM program made a considerable difference in the CTDs of 5th-grade primary school students. Profile plots also showed the intercepts for better visual interpretation of the results since interpreting split-plot ANOVA results is complicated (see Figure 4 for profile plots for each facet of the CCTDI across pre-post-test design for experiment and control groups below).

5.1.2. Impact of SP-STEAM Model on Mathematics Achievements

Research Question 2: Will groups of 5th-grade primary school students who are instructed by an SP-STEAM education model show statistically significant differences in comparison to similar groups of 5th-grade primary school students who are instructed by a traditional education model in terms of their mathematic achievements?

Results Regarding Research Question 2: Cross-sectional split-plot ANOVA results yielded that the experiment group 1 and experiment group 2 displayed statistically significant differences F(3100) 40.581, p < 0.001 in comparison to control group 1 and control group 2, across pre-post test results, in terms of mathematics achievement (see

Table 6. Descriptive statistics regarding mathematics achievement test across a pre-post-test design of control and experiment groups.

Variable	Design	Groups	Mean	Std. Deviation	n
Mathematics Achievement	Pre-Test	Experiment 1	71.1538	8.16182	26
		Experiment 2	74.6154	10.09189	26
		Control 1	72.1154	8.38726	26
		Control 2	71.7308	8.93782	26
		Total	72.4038	8.89477	104
	Post-Test	Experiment 1	89.2308	6.58670	26
		Experiment 2	81.1538	9.72704	26
		Control 1	71.7308	6.62455	26
		Control 2	74.6154	8.35740	26
		Total	79.1827	10.33682	104

for descriptive data). No significant difference was observed in favor of control groups.

Group-wise comparison was carried out, along with the Tukey post hoc test, in an effort to observe where significant differences existed between the groups. The test results yielded that significant differences only existed between the experiment groups and control groups in favor of the experiment groups, as shown in

Table 7. Tukey F Post hoc results regarding mean differences of mathematics achievement for experiment and control groups across the pre-post-test design.

Groups		df	F	Mean Difference	Std. Error	Sig.
Experiment 1	Experiment 2	1.3	7.649	0.3077	1.09623	0.992
	Control 1			4.3269	1.09623	0.001
	Control 2			3.2308	1.09623	0.020
Experiment 2	Experiment 1			−0.3077	1.09623	0.992
	Control 1			4.0192	1.09623	0.002
	Control 2			2.9231	1.09623	0.044
Control 1	Experiment 1			−4.3269	1.09623	0.001
	Experiment 2			−4.0192	1.09623	0.002
	Control 2			−1.0962	1.09623	0.750
Control 2	Experiment 1			−3.2308	1.09623	0.020
	Experiment 2			−2.9231	1.09623	0.044
	Control 1			1.0962	1.09623	0.750

below.

6. Discussion

It is a well-known fact that it is a difficult task to graduate individuals who are keen to question, solve problems, be a world citizen, and be a good person, all at the same time [21,25,40,41]. The far aims of education determine the aims of the schools, lessons, and students. Educational policies are the determinants of the quality of education [34]. Education, on the other hand, within the era of globalization, is trying to achieve its goals. Various different strategies are being tested and used to create a culture of critical thinking [17,19,28,33]. However, very few of them succeed and very few of them are sustainable. The excitement that motivated this research is rooted in the call for research in the 2030 vision of North Cyprus, which was announced by the North Cyprus Ministry of Education [31]. After longitudinal seminars and workshops, with the inclusion of many academicians, researchers, teachers, and educators, some critical decisions were made at the governmental level in North Cyprus. One of the decisions was to embed SP-STEAM applications to graduate individuals who can think critically and solve problems.

This quasi-experimental research aimed to test the impact of a sustainable progressive STEAM education design on 5th-grade primary school students’ CTDs and mathematic achievements. When the results were evaluated, it was vivid that the SP-STEAM application had significant effects upon students’ CTDs and mathematic achievements. As measured with CCTDI, both the experimental groups scored significantly high in post-test results, in comparison to pre-test results, and in comparison to the post-test results of the control groups, whereas the control groups either remained the same or displayed non-significant changes in terms of the seven facets of CCTDI. Similar effects were recorded for mathematic achievements, too.

When the experimental studies carried out in Turkey were evaluated, the results signified that if teaching-learning environments are redesigned according to the STEAM education model, then the students display increased performances in mathematics and in the use of their cognitive abilities [13,15,16,35,42], which is a supportive finding for the current research. This experiment also presented that the control groups maintained the same performance, without any significant changes in their mathematic achievements and CTDs, even though the control groups followed the same curriculum content. Thus, the results of this experiment support the use of the SP-STEAM education model for mathematics achievement and for CTDs. Other countries, as well as Turkey, have seen researchers producing similar results in the application of the STEAM education. Significantly, numerous studies carried out across different countries appropriating the application of SP-STEAM education have all displayed improved academic achievements of the students in every respect, reporting positive increments in science, technology, and the mathematic achievements of primary school learners in comparison to non-STEAM education forms of teaching and learning [12,14,43,44].

For the last decade, there has been an increase in experimental researches in relation to the influences and impacts of the SP-STEAM oriented education, except for the period encompassing the pandemic. Consideration in this context reveals that the findings of many studies displayed parallelism with this study [11,24,26,27], which emphasizes the validity and reliability of the current research.

The bulk of the literature had theoretically supported the possible effects of STEAM education on critical thinking and academic achievement [1,2,17,18,19,21,26,27,30,34,38,39,41]; however, no empirical evidence existed to prove this hypothesis, especially in the context of North Cyprus. Recent research on enhancing mathematics critical thinking skills has revealed that STEAM plays a crucial role in the development of CTDs [34,41]. However, this study only mentioned the spirit of mathematics. The current research deliberately intended to figure out the sole impact of the SP-STEAM on mathematics achievement.

One other important dimension that requires focus relates to the applicator teachers, who were provided with in-service training regarding the correct application of STEAM in experiment groups. This opens up another discussion point, which is where applicators begin to be subjected to the sole contribution role regarding the effects of the SP-STEAM. As many researches explained, any successful result would be obtained from applications where the applicators were well-trained [17,18,19,33]. Obeying the suggestions of the related literature, it was found that the results were parallel to what has been proposed by the pioneers of the field [1,2,3,5,6,17,18,19,20,25,26,29,33].

What we have learned from the results of this research is that children can achieve well if they are provided with opportunities to use their potential. However, this requires background experience, and achieving goals in STEAM applications is very difficult without establishing a baseline with teachers who are well-trained for such applications.

7. Conclusions

Specifically, the current research proposed that primary school students can achieve better when they are provided with better opportunities to use their potential. Additionally, experiments conducted with primary school students with an average age of 10 to 12 are rarely set upon in the literature [14,16,42]. Many studies have pointed out that STEAM education should be tested with primary school students to establish a STEAM culture at the middle school and high school levels.

SP-STEAM, which forms an integrated model with a combination of different disciplines, enables students to look at the problems they face from a wider perspective. STEAM also includes the 21st-century skills that education systems aim to transfer to students [28]. For this reason, using the activities prepared within the scope of the STEAM education approach will enable students to develop the skills of the science-technology-mathematics and engineering disciplines [1] and increase their readiness [30]. For countries to have a voice in the international arena to compete and grow economically, the STEAM approach should be included in education systems [21]. In North Cyprus, the Ministry of National Education published “Science, Engineering and Entrepreneurship applications”, which says the subject field is added [31]. The results of the current research can be considered invaluable in terms of underlining the positive impact of STEAM integration and the importance of in-service teacher training.

8. Suggestions for Further Studies

This experiment had some limitations that can be taken into consideration for further implications. Empirical studies, especially experimental ones, are valuable due to the possibilities of their repetition in different settings with similar groups of students. This experiment took mathematic achievements into consideration only. However, SP-STEAM includes engineering, science, and technology applications as well. A further study may include these endogenous variables to test the complete effect of SP-STEAM on other subject matters.

In addition, conducting further experiments with 4th-, 6th-, and 7th-grade students could be fruitful to see the effects of SP-STEAM against different educational levels. Moreover, this research design can be turned into a true experimental design with a completely random sampling procedure and more participants enrolled in the experiment. In our research, to increase the validity of the results, two experimental groups were used against two control groups. A follow-up study may include more than two experimental groups, making the results more generalizable. Finally, factors associated with the application of SP-STEAM, perceptions of students, and the applicators' views on possible obstacles can also be studied with a survey. The results of such studies have the potential to bring deeper insights into STEAM applications and their effects on various exogenous variables.