Artificial Intelligence Applications in Primary Education: A Quantitatively Complemented Mixed-Meta-Method Study

Abstract

In recent years, rapidly advancing technology has reshaped our world, holding the potential to transform social and economic structures. The United Nations’ Sustainable Development Goals (SDGs) provide a comprehensive roadmap that promotes not only economic growth but also social, environmental, and global sustainability. Meanwhile, artificial intelligence (AI) has emerged as a critical technology contributing to sustainable development by offering solutions to both social and economic challenges. One of the fundamental ideas is that education should always maintain a dynamic structure that supports sustainable development and fosters individuals equipped with sustainability skills. In this study, the impact of various variables related to AI applications in primary education at the elementary school level, in line with sustainable development goals, was evaluated using a mixed meta-method complemented with quantitative analyses. Within the framework of the mixed meta-method, a meta-analysis of data obtained from studies conducted between 2005 and 2025 was performed using the CMA program. The analysis determined a medium effect size of g = 0.51. To validate the meta-analysis results and enhance their content validity, a meta-thematic analysis was conducted, applying content analysis to identify themes and codes. In the final stage of this research, to further support the data obtained through the mixed meta-method, a set of evaluation form questions prepared within the Rasch measurement model framework was administered to primary school teachers. The collected data were analyzed using the FACETS program. The findings from the meta-analysis document review indicated that AI studies in primary education were most commonly applied in mathematics courses. During the meta-thematic analysis process, themes related to the impact of AI applications on learning environments, challenges encountered during implementation, and proposed solutions were identified. The Rasch measurement model process revealed that AI applications were widely used in science and mathematics curricula (FBP-4 and MP-2). Among the evaluators (raters), J2 was identified as the most lenient rater, while J11 was the strictest. When analyzing the AI-related items, the statement “I can help students prepare a presentation describing their surroundings using AI tools” (I17) was identified as the most challenging item, whereas “I understand how to effectively use AI applications in classroom activities” (I14) was found to be the easiest. The results of the analyses indicate that the obtained data are complementary and mutually supportive. The findings of this research are expected to serve as a guide for future studies and applications related to the topic, making significant contributions to the field.

1. Introduction

By the late 20th century, technological and technical advancements had led to profound changes in societies. Today, information technologies are at the core of continuously evolving education systems [1]. Among the notable technologies of this era is artificial intelligence (AI), which can be defined as a technology that enables computer systems to mimic certain human capabilities [2]. AI encompasses autonomous machines capable of analyzing large datasets, recognizing patterns, learning, and solving problems [3,4]. Currently, AI has shaped the future of computer systems and has become an indispensable part of daily life. With advancements in AI-driven computing, continuous innovations in software and hardware have emerged, integrating AI-powered applications such as robots, smart home systems, and autonomous vehicles into everyday life [5].

AI has a wide range of applications across various sectors, from healthcare to finance and manufacturing to education. For instance, in the healthcare sector, AI can analyze hospital data to assist in diagnosis and treatment planning, as well as provide preventive healthcare services [6]. In financial services, AI can be used to prevent fraud and enhance customer service [7]. In the industrial sector, AI enables more cost-effective and widespread production, while in transportation, it contributes to safer travel [8,9].

The rapid advancement of AI is profoundly transforming our lifestyles, learning methods, and ways of conducting business [10]. Today, the greatest challenge in sustainable education for AI-integrated smart manufacturing capabilities is the ability of the education system to adapt to changing conditions and to focus on the continuous development of individuals’ competencies [11]. Therefore, AI education should evolve from being limited to experts to being accessible to the general public [12]. Traditionally covered in higher education, AI topics have now been strategically introduced into secondary and primary school curricula worldwide [13]. Introducing AI education at an early age provides opportunities to build a sustainable world by helping students understand AI-based solutions to global issues such as environmental challenges, food security, poverty, inequality, and violence [14,15]. Through such educational initiatives, children not only become aware of emerging technologies and their functionalities but also gain inspiration to become future AI users, designers, software developers, and researchers. By maintaining a balance between society, the economy, and the environment, AI education can help achieve the goals of sustainable development.

1.1. Artificial Intelligence in Education

As in many fields, artificial intelligence (AI) is being utilized in education. The significance of applying AI in the education sector became evident during the COVID-19 pandemic, which resulted in a shift from traditional classroom teaching to digital learning methods [16,17,18]. In fact, studies on the use of AI in educational settings began as early as the 1980s [19]. Over the past five decades, research in the literature has focused on AI-supported educational applications [20], language-teaching processes in educational environments, systems for evaluating and supporting individual performance [21], and intelligent teaching systems and environments [22].

These studies indicate that using AI in education can provide more effective learning experiences by personalizing learning, helping students discover their abilities, fostering creativity, and reducing teachers’ workload [23,24]. Additionally, the literature highlights that AI can support students with special needs, offer content tailored to different learning styles, analyze students’ learning processes to focus on their individual needs, and guide them in career planning [25]. These findings align with the three paradigms defined by [26] for the use of AI in education: guiding, supporting, and integrating students into enhanced practices.

While such findings showcase the benefits of AI for students, studies examining teachers’ perceptions of AI reveal that teachers often see AI as a professional threat that could replace their jobs [27]. However, recent research indicates that these perceptions are changing as teachers now expect AI to bring significant advancements to education in various settings [28]. Teachers’ perceptions of AI systems depend on factors such as their pedagogical beliefs, teaching experience, and history of using educational technology. These factors can influence their willingness to adopt new educational technologies [29].

Studies exploring teachers’ perceptions of AI in education (AIE) reveal that teachers see AI as a tool that can contribute to the teaching–learning process by providing digitized learning materials and facilitating human–computer interaction [30]. They also perceive AI as a means to address learning difficulties faced by students in overcrowded classrooms [31]. Moreover, the research indicates that teachers believe AIE can reduce their administrative workload by taking over repetitive and simple tasks [32].

1.2. Artificial Intelligence in Elementary Schools

Artificial intelligence (AI) is becoming increasingly significant in our daily lives, with applications in disaster prevention, agriculture, civil engineering, and healthcare, among others [33,34,35,36]. Educating all age groups, from elementary school students to the elderly, about AI and how to use it is crucial for fostering a society where AI is effectively utilized. AI education should not be limited to higher education but should be integrated into all levels, starting from elementary school through secondary education [37].

In 2019, the United Nations Educational, Scientific, and Cultural Organization (UNESCO) evaluated educational programs provided by governments and institutions, adopting a policy to support research aimed at developing standardized AI curricula for K-12 education [13]. This initiative has led to an increase in research on K-12 education and the emergence of conferences and journals dedicated to the topic [38].

As children increasingly interact with new technological tools and engage with learning processes through AI systems, the importance of AI education tailored to children is growing. For instance, Ref. [39] found significant improvements in elementary students’ spatial reasoning skills after participating in short- or long-term robotic interventions.

Ref. [40] developed a Model-Tracing Intelligent Tutor (MIT) to interpret students’ mathematical problem-solving behaviors, diagnose learning difficulties, and provide individual feedback. Their findings showed that the model-tracing-based learning diagnosis system significantly outperformed traditional web-based tests in helping students learn mathematics.

Ref. [41] investigated the impact of robotics-based education on students’ mental rotation abilities. Their study revealed that robotics-based education significantly improved boys’ mental rotation skills compared to the control group, while no significant difference was observed for girls. Although the study provides evidence of the potential of robotics-based education, it also highlights the need for future research to deeply analyze gender differences in learning outcomes achieved through educational robotics.

Ref. [42] emphasized the importance of introducing children to AI concepts at an early age and proposed a curriculum design focusing on why, what, and how to teach AI. Similarly, Ref. [31] echoed this perspective, underscoring the value of early AI education for children.

1.3. Purpose and Significance of This Study

In the 21st century, technology has led to revolutionary changes in the field of education. With these transformations, knowledge is no longer acquired solely through individual memory or within classroom settings but also through interactions with online networks and communities. As a result, the learning process is conceptualized as a network structure in which knowledge is continuously updated by establishing connections between different information nodes. In this context, connectivism theory asserts that knowledge changes rapidly and is particularly effective in fields requiring constant updates, such as technology and digital literacy. The connectivist paradigm views learning as a multilayered networked process [43]. In other words, connectivism aims to redefine teaching and learning processes in knowledge societies using emerging technologies. Additionally, connectivism explains learning in e-learning environments [44] and contributes to modernizing traditional education by integrating e-learning scenarios into face-to-face learning settings [45].

Within this framework, artificial intelligence (AI) applications offer significant contributions to educators who seek to integrate digital technology into learning environments. AI is defined as the ability of a digital device to perform tasks typically attributed to humans [46]. In today’s world, innovative technologies such as AI are increasingly playing a pivotal role in classrooms and student learning processes [47,48]. Consequently, AI research has focused on student learning, teacher instruction, student performance evaluation, and school administration [49]. Studies related to student learning primarily explore academic performance, 21st-century skills, student motivation, and engagement [50,51,52].

When considering these aspects as a whole, no previous study has been found that integrates the impact of AI applications on different variables at the elementary education level using a mixed-meta method combined with quantitative analyses. In this research, the effect of different variables on AI applications is examined in three dimensions, aiming to provide an original perspective to the literature by addressing the following questions:

• What are teachers’ perspectives within the scope of this meta-analysis (quantitative dimension)?
• What are teachers’ perspectives within the scope of this meta-thematic analysis (qualitative dimension)?
• What are teachers’ perspectives within the scope of the Rasch measurement model (quantitative dimension)?

In this context, following a document review methodology within a mixed-meta method complemented with quantitative analyses, this study focuses on the points below:

Within the scope of the meta-analysis, the following was performed:

• Determining the overall effect size of different variables on AI applications;
• Assessing the effect size of different variables on AI use based on subject area, the duration of implementation, and sample size.
  Within the scope of the meta-thematic analysis, the outlined was performed:
• Identifying the impact of AI applications on learning environments and determining potential challenges and solutions in AI implementation;

Within the scope of the Rasch measurement model (teacher perspectives), the following was performed:

• Conducting a general analysis of teachers’ opinions on AI applications;
• Analyzing the leniency or strictness of evaluators (jury members);
• Performing a difficulty analysis of AI-related assessment items (criteria).

This study aims to contribute to the field by providing a comprehensive evaluation of AI applications in elementary education through a systematic, data-driven approach.

2. Methods

The methodology of a scientific study encompasses every stage of the research process, from its justification and content to the concepts used, measurement techniques applied, the analysis of collected data, and the interpretation of results [53]. Methodology can be classified into qualitative, quantitative, and mixed-methods approaches [54]. Among these, methodological pluralism—which combines quantitative and qualitative methods—has gained significant traction in fields such as education and sociology [55].

Methodological pluralism involves the holistic analysis of quantitative and qualitative data through document analysis [56]. Mixed-methods research, as defined by [57], aligns with this concept of methodological pluralism. It involves the integration of qualitative and quantitative approaches, either concurrently or sequentially, to create a unified dataset.

This study adopts a methodological pluralism framework to evaluate the use of AI in primary schools, encompassing both qualitative and quantitative analyses. It employs a mixed-meta approach, which integrates quantitative and qualitative methods within a single research process. Specifically, the methodological process includes three methods, outlined below:

• Meta-analysis: a quantitative synthesis of data to determine the effect size of AI applications;
• Meta-thematic analysis: a qualitative examination of recurring themes in the literature, focusing on the effects of AI applications in educational contexts;
• The Rasch measurement model: a quantitative analysis of participant opinions, providing insights into teacher perspectives and evaluating response consistency.

This combination of methods ensures a scientifically robust and holistic research process. The research integrates findings from meta-analysis, meta-thematic analysis, and the Rasch model to comprehensively analyze AI applications in education. A visual representation of this methodological framework is presented in Figure 1.

The mixed-meta method involves the holistic analysis of quantitative and qualitative data based on document examination. In other words, a mixed-meta method allows for the analysis of quantitative data using statistical programs such as CMA V3/MetaWin 2.1, and qualitative data using software like Nvivo/Maxqda 2020, thereby enabling the integration of both datasets into a single study framework. This makes the method a comprehensive and rich approach to research [59]. The Multiple Complementary Approach (McA), a design within this method, integrates meta-thematic analysis with the quantitative research process to address the shortcomings of meta-thematic findings, support the obtained results, and provide a more holistic perspective [60]. The analysis of scientifically valuable quantitative and qualitative data is crucial in this research process. In this context, the mixed-meta method integrates meta-analysis, meta-thematic analysis, and the Rasch analysis based on participant views in the quantitative dimension. This three-phase approach enhances the depth of this research and ensures the generation of more comprehensive and valid results.

Method Phases:

Meta-analysis: statistical analysis of quantitative data, determining the impact of different variables on AI applications;

Meta-thematic analysis: thematic analysis of qualitative data, exploring the impact of artificial intelligence applications on education.

2.1. Meta-Analysis Process

In the first dimension of the research, meta-analysis was used. Ref. [61] defined the concept of a meta-analysis as a statistical analysis process aimed at integrating research findings by combining the analysis results obtained from individual studies. A meta-analysis is a statistical method that, instead of examining individual studies one by one, aims to gather various studies effectively and reliably to produce broader and more meaningful results [62]. In this dimension of the research, the meta-analysis process combines the general effect size of studies involving artificial intelligence applications at the elementary school level as well as the effect sizes related to lessons, application duration, and sample size. By doing so, the results of various studies conducted at the elementary school level are effectively and validly integrated, leading to a holistic conclusion [62].

2.1.1. Data Collection and Analysis

Research on the use of artificial intelligence in elementary schools was conducted in both English and Turkish using the keywords “impact/effectiveness of artificial intelligence use/in elementary schools” in the literature. During this study, databases such as YÖK, Google Scholar, Web of Science, Taylor & Francis Online, Science Direct, and ProQuest Dissertations & Theses Global were searched. The search was conducted according to the inclusion criteria listed below. These criteria are shown in

Table 1. Inclusion criteria for meta-analysis process.

Criteria	Description
Time Period	2005–2025
Publication Language	English and Turkish
Appropriateness of Teaching Method	Experimental and/or quasi-experimental designed studies with pre-test–post-test control groups using artificial intelligence applications
Statistical Data	Sample size (n), arithmetic mean (X), and standard deviation (ss) for effect size calculation

.

Table 1 outlines the inclusion criteria used to select studies for the meta-analysis. Studies that did not meet these criteria were excluded from the analysis. In this context, studies that lacked access permission, did not contain quantitative data, lacked the necessary data for analysis, were found in multiple databases, or did not involve an experimental process were excluded from the analysis. The number of studies included and excluded, along with the reasons for exclusion, are shown in the PRISMA flow diagram [63] in Figure 2.

As shown in Figure 2, as a result of the screenings, in the first phase, N = 1475 studies examining the impact of various variables on the use of artificial intelligence were identified. Out of these studies, 182 were excluded due to duplication, 486 due to irrelevant topics, 437 due to failure to meet the inclusion criteria—identified through reading the abstracts (NMA = 437)—and 64 due to insufficient digital data (NMA = 64). As a result, as presented in Figure 2, 12 studies were included in the meta-analysis. Additionally, inter-rater reliability was calculated using the formula [agreement/(agreement + disagreement) × 100] proposed by [64], and the reliability level of the research was determined to be 0.90. The data were analyzed using the CMA 2.0 program.

2.1.2. Effect Size and Model Selection

Meta-analysis procedures were carried out using the CMA 2.0 software. The effect size (g) obtained from the analysis was interpreted according to the effect levels defined by [65]. The data were assessed and interpreted within the framework of the random-effects model (REM). Ref. [66] pointed out that the use of the fixed-effects model (SEM) is limited in most cases, emphasizing that the REM is a more appropriate option. Therefore, in this research, the REM was preferred.

2.1.3. Moderator Analysis

In the meta-analysis of studies on artificial intelligence applications, a heterogeneity test was performed and an I² value of 85.90 was found. This value indicates that the overall effect size of artificial intelligence applications, as well as different variables, could be evaluated. Values with heterogeneity rates of 75% or higher are considered high heterogeneity so it is recommended to perform moderator analysis [67]. Therefore, in this research, to deepen the meta-analysis, various factors influencing artificial intelligence applications were examined, and conducting a moderator analysis was deemed necessary.

2.1.4. Publication Bias

In the meta-analyses, reliability is a key aspect. In studies where effect size is calculated, the inclusion of published studies or those with significant differences can raise concerns about publication bias. For this reason, certain calculations were made to assess publication bias in the meta-analyses. Figure 3 shows the reliability calculations related to this in a funnel plot.

Figure 3 shows a funnel plot that presents a visual summary of the meta-analysis dataset and also illustrates the potential for publication bias [67]. Funnel plots are used to detect publication bias in meta-analyses [68]. In this plot, the horizontal axis represents effect size, and the vertical axis represents sample size, variance, or standard error values. The funnel plot highlights the relationship between effect size and sample size. As sample size increases, studies tend to cluster closer to the average effect size at the top of the plot [69]. If there is no publication bias, the plot should form a symmetrical, inverted funnel shape, as seen in Figure 3 [70]. Upon examining Figure 3, it can be observed that there is a balanced distribution of studies on both the left and right sides of the symmetry axis. This indicates that no publication bias was found in this research.

To test for publication bias, the calculation of Rosenthal’s fail-safe N is a method used for this purpose [71]. The N value indicates how many unpublished null studies would be required to invalidate the existing effect. A high N value suggests that the results are valid [72]. In this research, the safe N value is 841. Comparing this with the number of studies included in the analysis, it can be said that this number is quite high, meaning no bias was detected.

2.2. Meta-Thematic Analysis Process

As part of the mixed-meta method, the second dimension of this study applied the meta-thematic analysis process. Meta-thematic analysis can be defined as a type of analysis where participants’ views (raw data) from qualitative research on a particular topic are re-evaluated, and themes and codes are extracted [59]. In this study, qualitative research on the use of artificial intelligence at the primary school level was reviewed and the meta-thematic analysis process was evaluated. In meta-thematic analysis, it is not the quantity of data that is important but rather working with enough data to reach saturation. This type of analysis involves developing new codes and themes from the data obtained through document analysis. In other words, meta-thematic analysis is a process based on text or verbal data that combines qualitative findings to generate new themes and codes. The themes and codes obtained in this process are included in this study to provide broader and more comprehensive results. In the qualitative dimension of this study, themes and codes related to the use of artificial intelligence at the primary school level were created using the meta-thematic analysis process [60].

2.2.1. Data Collection and Review

Qualitative studies on the use of artificial intelligence at the primary school level were gathered using the document review method. These studies were reviewed after a preliminary reading, and the studies that were considered appropriate were selected through four stages: initial reading, full reading, and a review of the findings section. The review phase involved reading the title, a short summary, and an expanded summary before moving to the findings section. After examining the findings section, it was decided whether the study would be included in the current research [58]. As indicated in the PRISMA diagram, the studies were selected after a four-stage process. Ultimately, four studies were reviewed in the meta-thematic analysis process. Document analysis is defined as the process of collecting, examining, evaluating, and analyzing different documents as primary sources of research data [73]. In other words, document analysis includes a series of processes involving the examination and evaluation of printed and digital (computer-based and internet-accessible) data [74]. In the processing of the meta-thematic data obtained through qualitative research using document analysis, the Maxqda program was used. The data were transferred into the program, and content analysis—a commonly used method [74]—was applied to analyze indicators, comments, and discourses [75]. In this context, the participant opinions obtained in this research were re-analyzed to generate different codes and themes.

2.2.2. Coding Process

An important phase of the meta-thematic analysis process is coding. The opinions of participants were re-formulated and codes were created, grouping similar codes under common themes. In this phase, the findings of the meta-thematic analysis process—which were based on the opinions of participants from the reviewed studies—included the themes and codes determined for the study’s main objectives. The reliability of the coding process is demonstrated by ensuring that the generated codes are consistent with each other [76]. In this study, two themes were created at the end of the meta-thematic analysis process: “the impact of AI applications on learning environments” and “problems and solutions encountered in AI applications”. Two coders independently created the themes and codes. One of the coders was the researcher, and the other was an academic expert in the field. After completing the coding process, the themes and codes created by both coders were compared to check for consistency and agreement. Similar themes and codes were recorded jointly, while discrepancies were discussed until agreement was reached between the coders. The reliability between coders was calculated using Cohen’s Kappa coefficient [77]. The Kappa value for the theme “the impact of AI applications on learning environments” was 0.79, and for the theme “problems and solutions encountered in AI applications” it was 0.86 (Appendix A). The Kappa values ranging from 0.79 to 0.86 indicate a good to very good level of agreement between the coders.

2.2.3. Reliability in the Meta-Thematic Analysis Process

Reliability methods used in qualitative research were also applied to the meta-thematic analysis in this study. During this phase, the concept of researcher triangulation [78] was used, where two researchers collaborated throughout the meta-thematic analysis process. Furthermore, direct quotes from participant expressions were included to provide the raw data source while creating the themes and codes. The related quotes were expressed with codes indicating the study and page number from which the quotes were taken. For example, the numeric expression M12-s.15 refers to M (article), 12 (study number), and “s.15” (page number). The codes obtained from the YÖK National Thesis Center were also abbreviated using thesis numbers. Through the meta-analysis of studies conducted on primary school AI use, effect sizes were identified, and codes and themes were created through the meta-thematic analysis. To support these findings and provide a holistic table, the opinions of class teachers were analyzed using the Rasch measurement model.

2.3. Rasch Measurement Model Analysis Process

In the third dimension of the research, quantitative data related to teacher opinions on the use of AI in primary schools were analyzed using the Rasch measurement model, developed by [79,80]. This model is based on objectivity [81]. It also evaluates other variable sources that may affect the test results, such as item difficulty levels, raters, scoring keys, conditions, tasks, and scoring criteria [82,83]. The multi-faceted Rasch model establishes necessary linear relationships between different facets (such as AI application usage levels, evaluation item characteristics, and rater behaviors) and forms these connections [84]. In the current research, the Rasch model includes three facets: 21 primary school teachers, 18 items related to AI use in education, and 4 educational programs (Turkish, Mathematics, Life Science, and Science). To collect the data, the researchers created the “Primary School Teachers’ AI Application Usage Evaluation Form”.

2.3.1. Study Group

The study group consisted of 21 classroom teachers working in public primary schools during the 2024–2025 academic year (Appendix D). The raters independently assessed the levels of artificial intelligence application usage using an 18-item evaluation form. Since studies based on the Rasch measurement model do not assume generalizing the results of the sample data to the population [79] (pp. 2–15), no generalization issues were encountered.

2.3.2. Research Data and Analysis

To collect data for this study, the “Primary School Teachers’ AI Application Usage Evaluation Form” was used (Appendix B). The evaluation form—developed within the framework of the Rasch measurement model and consisting of 18 items—assesses teachers’ knowledge acquisition, knowledge deepening, and knowledge creation under the headings of curriculum and assessment, pedagogy, and the application of digital skills. The form includes evaluations of the Turkish, Mathematics, Science, and Life Science curricula. The evaluation form was created based on a literature review [85,86] and expert opinions. Afterward, the form was re-examined by experts for verification, and necessary additions and deletions were made. This form contains a total of 18 criteria. The content validity index (CVI) of the items was calculated using the Content Validity Ratio (CVR) formula developed by [87] and found to be 0.81 (Appendix C). According to [88], this value is statistically significant at the 0.05 level. The analysis of the AI application evaluation form was conducted using the FACETS analysis program developed by [79] (pp. 2–15). FACETS is an analysis program frequently used in the multi-faceted Rasch measurement model. This program generally includes three main facets: rater, ability, and task [89].

3. Findings

This section interprets the findings obtained through the meta-analysis, meta-thematic analysis, and participant opinions on various variables related to the use of AI applications in both classroom and out-of-class teaching environments. In the first phase of this study, meta-analytic findings based on document analysis are presented; in the second phase, results of the meta-thematic analysis are provided; and in the third phase, participant opinions are supported by the quantitative data.

3.1. Meta-Analysis Findings on AI Applications

When examining the meta-analysis findings in

Table 2. Meta-analysis data.

Test Type	Model			95% + Confidence Intervals		Heterogeneity
		n	g	Lower	Upper	Q	p	I2
AA	FEM	24	0.59	0.47	0.64	163.11	0.00	85.90
	REM	24	0.51	0.28	0.74

, it was determined that the effect size of academic achievement scores (AA) for AI-supported applications—calculated using the REM method—was g = 0.51 [0.28; 0.74]. This effect size is classified as moderate according to the classification in [65], indicating that AI applications have a positive and beneficial effect on these variables. Additionally, a significant difference was found between the scores based on test types (p < 0.05).

When examining the results of the heterogeneity test in Table 2, it is seen that the effect sizes of the scores for AI applications exhibit a heterogeneous distribution (QB = 163.11; p < 0.05). The I² value was calculated as 85.90%, indicating that the observed 86% variance originates from the true variance among the studies. According to [67], an I² value of 25% is considered low, around 50% is moderate, and 75% or higher is high heterogeneity. The calculated I² value of 85.90% in this study points to a high level of heterogeneity [90]. This suggests the presence of moderator variables affecting the total effect size. In other words, the detection of high heterogeneity indicates the need for a moderator analysis [72].

As presented in

Table 3. General effect sizes of studies included in the analysis according to moderator analysis.

Items	Groups	Effect Size and 95% Confidence Intervals				Null Test		Heterogeneity
		n	g	Lower Limit	Upper Limit	Z-Value	p-Value	Q-Value	df	p-Value
Application duration	1–4	0.59	0.59	0.30	0.88	4.01	0.00
	5+	0.09	0.09	−0.15	0.33	0.75	0.45
	Unspecified	0.58	0.58	−0.02	1.19	1.89	0.06
	Total	0.32	0.32	0.14	0.50	3.54	0.00	7.69	2	0.02
Subjects	Maths	19	0.44	0.18	0.71	3.25	0.01
	AI	3	0.80	0.07	1.53	2.15	0.03
	Others	2	0.81	0.19	1.44	2.54	0.01
	Total	24	0.53	0.30	0.76	4.47	0.00	1.7	2	0.43
Sample size	Small	6	0.50	0.09	0.90	2.40	0.02
	Medium	9	0.37	0.18	0.55	3.93	0.00
	Large	6	0.63	0.18	1.08	2.75	0.01
	Total	24	0.42	0.26	0.57	5.24	0.00	1.29	2	0.52

, the duration of implementation and sample size have been selected as moderator analyses. In the significance test, a significant difference was found in the duration of implementation (QB = 7.69; p < 0.05); however, no significant differences were observed in terms of lessons (QB = 1.7; p > 0.05) and sample size (QB = 1.29; p > 0.05). Nevertheless, when the results of this analysis are generally evaluated, it can be stated that various variables have a moderate effect on artificial intelligence applications.

3.2. Meta-Thematic Findings Regarding Artificial Intelligence Applications

In this part of the study, the themes and codes obtained through the meta-thematic analysis of the studies accessed regarding the effects of various variables on artificial intelligence applications are presented. The data related to the effects of different variables on artificial intelligence applications were analyzed, and two themes were formed: the impact of artificial intelligence applications on educational settings, and challenges encountered and solutions proposed. The themes and codes are illustrated in models. Additionally, direct quotations are provided as reference sentences within the comments.

When examining Figure 4, the contributions of artificial intelligence applications to learning environments can be seen in the model as codes. Some of the codes in this model can be explained as providing support for challenging subjects, presenting topics through visualization, enhancing problem-solving skills, fostering intrinsic and extrinsic motivation, supporting socio-emotional development, and improving presentation skills. Expressions and quotations that can serve as references to these codes are stated in the studies as follows:

“I like to learn English with it (the AI coach) as it helps improve my English competence”, and

“You can get (virtual) flowers and awards if you practice English with the AI coach every day and achieve good performance” (M1-p. 6).

In another study, the following is conveyed: “He also started to improve his oral skills, and finally gave presentations in front of large audiences” (M2-p. 1890).

Additionally, in the study coded (M5-p. 12), it is stated as follows: “I thought it was interesting because I was able to actually control the AI to play rock-paper-scissors with only two fingers”.

Furthermore, it can be stated that artificial intelligence applications help students develop critical thinking skills, promote learning through exploration, enhance problem-solving skills, and foster the acquisition of various other competencies. However, some challenges may be encountered during the implementation of these applications. The problems faced by practitioners and the proposed solutions are presented in the figure below.

When examining Figure 5, the problems encountered in learning environments during the artificial intelligence (AI) application process and the corresponding solutions are visualized. Some of these issues are indicated with the following codes: AI applications being deemed insufficient, difficulty in understanding questions asked, a lack of knowledge about AI, inability to produce adequate outcomes, concerns about personal data being compromised, lack of emotional support from AI applications, and anxiety related to AI. Expressions related to these codes include those outlined below:

“In my opinion, artificial intelligence is software that humans create, that we decide on the different things it learns and, after that, the computer adds them to other things it learned, like when we trained it with pictures, we also showed some pictures, and not always the computer was right, so we tried to give it more pictures and teach it things” (M3, p. 187).

In the study (M1, p. 10), the limitations of AI in sentence formation and vocabulary memorization were stated as follows: “For instance, one student indicated that ‘… it (the AI coach) improves our oral English through several ways, such as English shadowing, mimicking picture books, and memorizing vocabulary…’ Another student wrote, ‘it (the AI coach) contains many resources linked to our textbooks, picture books, and movie clips for budding practice’”.

Additionally, concerns about AI were expressed as follows: “I had concerns about the potential for AI to dominate the world, given its ability to complete tasks in just a few steps” (M5, p. 12).

During the analysis process, solution suggestions for potential problems in AI applications were identified. Some of the proposed solutions include the following: students should actively engage in learning efforts with AI; AI applications should be used through teamwork; AI application activities should be conducted interactively and actively; security measures should be prioritized in AI applications.

Quotations referencing these suggestions include the following:

In the study (M2, p. 1822), “Students building basic robotic models benefit when they are working individually; meanwhile, students might work better in teams (ideally, two to three members per team) when working on advanced robotic models that include writing code (programming)”.

Another example states the following: “In order to reduce such fraud, I think it is more convenient and accurate to use AI than humans” (M4, p. 12).

In conclusion, while certain issues may arise during AI applications, these problems can be resolved by utilizing the proposed solutions.

Findings derived from classroom teachers’ opinions have been tabulated and expressed using the Rasch measurement model. This approach provided quantitative support to the data obtained through the mixed-meta process, presenting a detailed interaction of all the dynamics of this study.

3.3. Findings Related to the Rasch Measurement Model for Artificial Intelligence Applications

Since this research was conducted using a quantitatively supported mixed-meta method, this section presents the findings related to AI applications in primary school education using the multi-faceted Rasch measurement model. When analyzing the AI applications, the surfaces used in this study (teachers’ AI applications, judges’ strictness/generosity, and the appropriateness of the items used) and general information about these surfaces are provided in the calibration map shown in Figure 6. The measurement on the left side of Figure 6 falls between (−) and (+) and is the same for all three surfaces. On the measurement scale, rankings are made based on the levels of educational programs, the strictness/leniency of raters (jury members), and the difficulty levels of items. Educational programs are arranged such that the program with the highest use of artificial intelligence applications is at the top, while the program with the least application is at the bottom. Similarly, raters are ordered from the most lenient evaluator (J2) to the stricter ones. Additionally, more difficult items are positioned at the top, while easier items are placed at the bottom.

In the findings, four curricula (Turkish, mathematics, life sciences, and science education curricula), 21 judges, and 18 evaluation items related to the program content were taken into consideration. In this calibration map, it was found that the use of AI applications was at a high level in the science and mathematics curricula (science education program: FBP-4 and mathematics program: MP-2), while it was observed to be at a low level in the life sciences and Turkish education programs (life sciences program: HBP-3 and Turkish program: TP-1).

Among the judges (raters), J2 was identified as the most generous rater, while J11 was determined to be the strictest rater.

When the column containing the items related to the use of AI applications was examined, the item “I can enable students to prepare a presentation describing their environment using AI tools” (I17) was found to be the most difficult item, whereas the item “I know how to efficiently use AI applications in classroom activities” (I14) was identified as the easiest item.

After evaluating the findings from the calibration map, the analysis report prepared for the curricula is presented in Figure 7.

As a result of the Rasch analysis, the reliability coefficient was determined to be 0.91. This coefficient value indicates the reliability with which the curricula were ranked. Furthermore, when the data in Figure 7 were analyzed, it was observed that there were statistically significant differences between the curricula (X² = 44.4; df = 3; p = 0.00). Additionally, the standard error (RMSE) of the logit values related to the curricula was found to be 0.07, indicating a very low level of error.

This error rate, with its adjusted standard deviation, is below the critical value of 1.0, which is considered acceptable. In this context, the order of AI usage rates in the curricula from high quality to low quality can be listed as follows:

FBP-4 (science education curriculum);

MP-2 (mathematics education curriculum);

HBP-3 (life sciences curriculum);

TP-1 (Turkish education curriculum).

In the Rasch analysis, the quality control limits for “infit” (internal consistency) and “outfit” (outlier-sensitive consistency) values are accepted to be between 0.6 and 1.4 [91]. During the decision-making process, the “infit” values are used for unexpected responses from judges, while “outfit” values are used for unexpected distant responses [92,93]. Upon examining the values in Figure 7, it is understood that these limits are appropriate.

In Figure 8, information regarding the strictness/leniency of the judges in relation to their use of AI in secondary education is presented. In Figure 7, it is observed that the judges’ scores are ranked from the strictest to the most lenient. The judge identified with the code J2 is seen to be the most lenient, while the judge identified with the code J11 is the strictest. Additionally, the judge separation index was observed to be 6.44, and the reliability coefficient was noted as 0.98. A statistically significant difference was also found between the strictness/leniency of the judges’ scores (X² = 872.8; df = 20; p = 0.00).

As a result, it was determined that the score observed in J2, with 283, was the most lenient, while the score observed in J11, with 150, was the strictest.

When the infit (internal consistency) and outfit (outlier-sensitive consistency) values of the surfaces in Figure 8 were evaluated, it was found that the judge identified with the code J15 did not meet the infit and outfit criteria (range of 0.6 to 1.4). In this case, it can be stated that the mean of the infit and outfit squares for the judge coded as J15 fell outside the defined limits. This can be interpreted as the lenient judge (J15) not exhibiting consistent scoring behavior while evaluating AI applications.

On the other hand, it was noted that the other judges (17 judges) demonstrated consistency in scoring among themselves as their values fell within the expected quality control range, and thus, they can be considered as appropriate.

Upon examining the results of the data analysis presented in Figure 9, it was observed that the separation index is 3.04 and the reliability coefficient is 0.90. This reliability coefficient indicates that the items used in this study to determine teachers’ levels of using AI tools are reliable. Additionally, there were statistically significant differences among the item difficulty levels used to evaluate teachers’ opinions regarding their use of AI tools (X² = 181.7; df = 17; p = 0.00).

The most difficult items related to the use of AI applications were identified as outlined below:

“I can guide students to prepare a presentation describing their environment using AI tools”.

“I can enable students to create simple patterns (e.g., drawing, storytelling) using AI tools”.

“I can integrate AI applications into pedagogical methods and techniques to contribute to students’ learning processes”.

On the other hand, the easiest items were as outlined below:

“I know how to use AI applications effectively in classroom activities”.

“I know how AI applications can support teaching activities in a classroom setting”.

“I can encourage students to collaborate and improve their skills using AI tools”.

The standard error (RMSE) for the analysis of criteria prepared by the researchers to determine the level of teachers’ use of AI applications was found to be 0.15, indicating that the standard error related to identifying the levels of AI application use is quite low.

Furthermore, when the infit (internal consistency) and outfit (outlier-sensitive consistency) statistical values for the research surfaces in Figure 9 were examined, it was noted that only the item coded I16 exceeded the outfit threshold. In this context, it can be stated that the mentioned item demonstrates inconsistency in evaluating the levels of AI use.

All other items, however, were within the specified consistency value limits, indicating their reliability in assessing AI application usage levels.

4. Discussion and Conclusions

In this study, a mixed-meta method, complemented with a quantitative research process, was employed to examine the effects of studies conducted at the primary school level on various variables and to reveal the current state of the literature in comparison with other studies.

In line with the purpose of the research, the following methodological steps were followed:

A meta-analysis was conducted first;

This was followed by meta-thematic analysis;

Finally, the participants’ opinions were collected in the quantitative dimension and evaluated using the Rasch measurement model.

4.1. Results of the Meta-Analysis Process

This study includes 37 quantitative studies for the meta-analysis, while 4 qualitative studies were included in the meta-thematic analysis process. In the quantitative scope, the teachers’ views on the relevant variables were collected using the prepared evaluation form. The findings based on the data obtained from the analysis are presented in this section. According to the meta-analysis, the results of the included studies based on the REM showed that the effect of various variables on the use of artificial intelligence was moderate (g = 0.51 [0.28; 0.74]), based on the classification in [65]. The results of other studies in the literature, which were not included in this study, support the research results [94,95]. Based on these results, it can be stated that different variables have an impact on the use of artificial intelligence applications.

During the analysis, a moderator analysis was also conducted regarding the effects of various variables on the use of artificial intelligence. The first moderator analysis, related to the duration of implementation, indicated that a duration of 1–4 weeks had an effect size of g = 0.59. Secondly, when subjects were used as moderators, the highest effect size (g = 0.81) was found for other subjects. Thirdly, regarding sample size as a moderator, large sample groups showed the highest effect size (g = 0.63). When analyzing the effect sizes of moderators, a significant difference was observed for the duration of implementation (QB = 7.69; p < 0.05), while no significant difference was found for subjects (QB = 1.7; p > 0.05) and sample size (QB = 1.29; p > 0.05). The findings of studies in the literature, such as those by [96,97], support the findings of this study in terms of subjects and sample size. In contrast to the studies in the literature, no effect on academic achievement was observed with time durations but this research found a significant difference in the moderator related to implementation time. When considering the results of the moderator analyses as a whole, it can be stated that artificial intelligence applications have a moderate impact across all groups with similar effect sizes. During the meta-analysis process, it was observed that the majority of studies regarding artificial intelligence applications were conducted in mathematics classes. In this context, further research on the use of artificial intelligence applications in other subjects is recommended.

4.2. Results of the Meta-Thematic Analysis Process

In the second phase, a document-analysis-based meta-thematic analysis was conducted, integrating the findings from the meta-analysis to validate them and expand the scope of the results. The themes and codes regarding the effects of artificial intelligence applications on learning environments and the problems and solutions encountered during their use were formed. The analysis revealed that students were in an interactive learning environment while using artificial intelligence applications. It can be stated that being highly interactive in the constructivist learning process helps make artificial intelligence applications more comprehensible [94]. One of the codes reached in the research was that artificial intelligence applications support individuals who learn at different paces and paths; Refs. [98,99] have reached similar findings in their studies. It can be said that an individual’s self-confidence in any task contributes to their success in that task. Our research concluded that artificial intelligence applications increase individuals’ self-confidence and enhance their creativity. This result is supported by findings from other studies using digital technology to support student learning [100,101].

In our research, it was also found that artificial intelligence applications improve problem-solving skills, creative thinking, and higher-order thinking abilities, aligning with findings in the literature indicating that technology-supported lesson content enhances students’ cognitive capacities [102,103].

Another theme created in our study is the problems and solutions encountered during the use of artificial intelligence applications. Studies indicating that individuals with low self-efficacy experience high computer anxiety [104] align with the findings in this research. The research findings show that students’ anxiety while using artificial intelligence applications may be related to their low self-efficacy in this area. One of the codes found was that students were disappointed by the responses provided by artificial intelligence during the process. Similar to this finding, the research of [105] suggests that poor performance of the applications does not meet students’ expectations. Another issue identified in this study is that artificial intelligence applications do not provide emotional support. The literature review shows that a lack of emotional support in artificial intelligence applications can negatively affect the targeted lesson outcomes [106,107]. Another identified issue is that students’ feedback on artificial intelligence usage is not understood by the application. Findings from studies in the literature [108,109,110] support this finding in our research.

It is suggested that artificial intelligence applications should have an attractive design as this will encourage students to develop a positive attitude toward using them [111]. Based on this, it can be concluded that artificial intelligence applications should be produced with engaging designs. Another problem encountered in the process is that educators do not have the necessary training, which prevents them from obtaining the desired results from the application. Our research found that teachers need sufficient knowledge and skills regarding artificial intelligence applications to overcome this issue. Ref. [98] study also aligns with our findings, indicating that teachers should receive technical training on artificial intelligence.

Data privacy and ethical concerns in artificial intelligence applications are among the other findings of this study. These concerns may cause anxiety among users during the implementation process. Ref. [112] stated in his research that how data are stored, who can access it, and how it is protected are significant factors limiting the integration of artificial intelligence in education. In another study, Ref. [113] emphasized that artificial intelligence has the potential to deepen existing inequalities and exacerbate privacy-related issues. The Rasch measurement model has been used to evaluate the artificial intelligence applications used by classroom teachers in their lessons.

4.3. Results Related to the Rasch Measurement Model Process

The quantitative section of our research presents the findings related to teachers’ views on their level of use of artificial intelligence applications, derived through analyses using the Rasch measurement model. In this way, the findings obtained through the mixed-meta processes were supported quantitatively to ensure the alignment of this study’s results. The Rasch measurement model adopts a fundamental approach that relates the probability of answering a question correctly to an individual’s ability [114]. The multi-faceted Rasch measurement model, developed by John M. Linacre, not only examines the relationship between individuals’ ability levels and the difficulty levels of the items on the measurement tool but also allows the evaluation of other variable sources that might affect test results, such as scorers, scoring keys, conditions, tasks, and scoring criteria [83]. In this context, simultaneous surfaces (the level of use of artificial intelligence applications, the rigidity/generosity of the jury, and the characteristics of the evaluation questions) that were prioritized and analyzed in the multi-faceted Rasch measurement model have been ranked among themselves. Awareness of artificial intelligence applications—a concept related to Industry 4.0—is crucial as these applications play a significant role in shaping the future of technology [115]. In the current study, teachers’ views were examined in the context of curricula by associating dimensions such as curriculum and assessment, pedagogy, and the application of digital skills with sub-dimensions like knowledge acquisition, knowledge creation, and knowledge deepening.

The analysis revealed that the science and mathematics curricula had the highest levels of use in the teaching programs. Artificial intelligence, which has made significant advancements in the past 50 years, has become an important research area [116]. AI encompasses many cognitive areas of human intelligence, including learning, reasoning, planning, problem-solving, perception, natural language processing, deep learning, expert systems, image processing, sentiment analysis, speech recognition, and more [117]. Therefore, it can be said that the primary use of AI in science and mathematics is due to the functionalities of artificial intelligence itself.

When the scorers’ rigidity/generosity information was evaluated in relation to the assessment of AI applications, it was found that the jury defined by the J2 code was the most generous, while the jury defined by the J11 code was the strictest. Furthermore, it was determined that the scorers were reliably ranked in terms of rigidity/generosity and differed from each other. Studies using the multi-faceted Rasch measurement model in the literature also indicate that scorers (juries) can be both objective and biased at times [93,114,118,119]. In addition, the analysis concluded that the jury separation index was 6.44, which is above the desired level. This value indicates that there are differences in scoring among the juries, that juries vary according to their generosity/rigidity levels, and that scoring errors related to generosity/rigidity exist in the scores given by the juries [120].

The items in the teacher evaluation form regarding the level of use of AI applications were found to serve the purpose of measuring teachers’ competence levels. When the items prepared to determine teachers’ level of use of AI were analyzed, it was found that the most difficult items were “I can make students prepare a presentation describing their environment using AI tools”, “I can make students prepare simple designs (like drawing pictures, creating stories) using AI tools”, and “I can integrate AI applications into pedagogical methods and techniques to support students’ learning processes”. Therefore, it can be said that teachers may face difficulties in using AI. Based on these results, it can be expressed that it is an area that needs further research on how AI can be used effectively in educational environments to achieve the desired outcomes and create designs based on the characteristics of the gains. On the other hand, the easiest items were “I know how to use AI applications effectively in classroom activities”, “I know how AI applications can support teaching activities in the classroom”, and “I can encourage students to collaborate and develop using AI tools”. Although the participants possess basic knowledge about AI, it can be said that teachers need to receive necessary training to stay up-to-date with developing technologies [121].

4.4. Integrative Results of This Study

When considering the results of the entire research process as a whole, it is evident that the obtained data support each other. In the meta-analysis process, it was found that AI studies in elementary schools were most frequently conducted in mathematics lessons [122]. In the Rasch measurement model section of this study, it was observed that classroom teachers most commonly used AI applications in science and mathematics lessons. In this regard, it can be said that the results of these two processes overlap. Furthermore, during the meta-thematic analysis process, the theme of training practitioners in AI applications emerged as an issue and solution. In the Rasch measurement model in the jury opinions section, it was found that the most difficult items were related to teachers preparing presentations with AI, integrating AI applications into the curriculum, and preparing simple designs related to teaching practices, indicating that educators need training in AI applications. Thus, the results of this study’s processes are compatible. Moreover, during the meta-thematic analysis, the theme related to the effect of AI on educational environments was “providing interactive teaching opportunities”, which aligns with the item in the Rasch measurement model “I can encourage students to collaborate and develop using AI tools” being among the easiest items. In this way, it can be said that the results from the two processes are integrated as both contribute to the creation of an interactive classroom environment where teachers facilitate collaboration among students using AI tools [123]. The use of artificial intelligence applications offers several advantages, such as providing personalized learning opportunities for students, giving immediate constructive feedback, creating collaborative learning environments, ensuring access to rich educational resources, identifying gaps in students’ learning areas through smart learning analytics, and enhancing students’ educational performance through continuous learning support [124]. In today’s technological age, the findings of this study—which involves both qualitative and quantitative research within the methodological pluralism framework—reflect how artificial intelligence, as the most prominent tool of the connectivity paradigm in modern educational environments, encapsulates and supports many of the findings.

4.5. Limitations and Future Research

Although this study presents a mixed-methods approach supported by quantitative analyses within the framework of methodological pluralism, there are several limitations to consider. The data collected for the meta-analysis and meta-thematic analysis were limited to certain databases. The meta-analysis focused solely on the impact of AI applications on academic achievement. The effects of the studies on attitudes and retention can also be included in the meta-analysis process. Moderator variables in this study included the duration of the intervention, the subject areas, and the sample size. In the Rasch measurement model, data were collected regarding the Turkish, mathematics, science, and life-sciences curricula. Research on artificial intelligence applications in the art, music, and physical education curricula in primary schools could also be conducted. Additionally, survey questions were developed based on teachers’ digital competencies [86], covering aspects such as curriculum and assessment, pedagogy, and digital skills. Studies could be conducted on other professional practice areas within the framework of teachers’ knowledge and technology competencies [86]. This study focused on gathering perspectives from classroom teachers. In the quantitative aspect of this study, opinions were gathered from classroom teachers. The scope of this study could be expanded by including raters with different levels of expertise in the research.

4.6. Recommendations

• The application duration, subject areas, and sample sizes in AI-related research have significant effects on academic success and the impact of AI on educational environments. The use of the mixed-meta method, supported by the Rasch measurement model, has provided a more holistic perspective, allowing for a deeper exploration of the topic. Based on the limitations and findings of this study, the following points are recommended:
• Research on AI applications in primary school subject areas such as art, music, and physical education can be conducted. In addition to quantitative methods, qualitative methods could be employed to explore the effectiveness and applicability of survey questions;
• The meta-analysis phase of this study could include investigations into the impact of AI applications on attitudes and long-term retention;
• Studies could explore teachers’ information and technology competencies [86] within other professional practice areas;
• This study focused on perspectives from classroom teachers. Including evaluators from different expertise levels could broaden the scope of the study;
• Despite teachers’ positive expectations regarding AI, it is essential that they first familiarize themselves with the technology and learn how to integrate it into their classrooms. Many teachers may regard AI as an advanced technological product without prior experience. In this regard, in-service training could increase teachers’ knowledge about AI and improve their integration of this technology, significantly enhancing student success and the learning experience [86];
• Given the methodological diversity, the use of a mixed-meta method combined with quantitative analyses has allowed for a comprehensive examination of the findings, with detailed insights into how various variables affect the use of AI applications. Therefore, it is recommended to apply the mixed-meta method integrated with either qualitative or quantitative analyses in other areas to achieve comprehensive research findings;
• Policymakers should take necessary measures to address concerns related to ethics, data security, and human rights as AI becomes more integrated into education;
• Artificial-intelligence-supported assessment tools are highly effective in monitoring student performance and providing immediate feedback. Educational institutions can make these systems more widespread to reduce teachers’ workload and track students’ progress in more detail;
• For students to succeed in AI-supported learning environments, they need to possess critical thinking, problem-solving, and digital literacy skills. Curriculum adjustments should be made to equip students with these skills.