Intelligent Tutoring Systems in Mathematics Education: A Systematic Literature Review Using the Substitution, Augmentation, Modification, Redefinition Model

Abstract

Scholars have claimed that artificial intelligence can be used in education to transform learning. However, there is insufficient evidence on whether intelligent tutoring systems (ITSs), a representative form of artificial intelligence in education, has transformed the teaching and learning of mathematics. To fill this gap, this systematic review was conducted to examine empirical studies from 2003 to 2023 that used ITSs in mathematics education. Technology integration was coded using the substitution, augmentation, modification, redefinition (SAMR) model, which was extended to suit ITSs in a mathematics education context. How different contexts and teacher roles are intertwined with SAMR levels were examined. The results show that while ITSs in mathematics education primarily augmented existing learning, recent ITS studies have transformed students’ learning experiences. ITSs were most commonly applied at the elementary school level, and most ITS studies focused on the areas of number and arithmetic, algebra, and geometry. The level of SAMR varied depending on the research purpose, and ITS studies in mathematics education were mainly conducted in a way that minimized teacher intervention. The results of this study suggest that the affordance of an ITS, the educational context, and the teacher’s role should be considered simultaneously to demonstrate the transformative power of ITSs in mathematics education.

1. Introduction

The use of technology in mathematics education is recommended to form a problem-solving and inquiry-based educational environment in terms of constructivism and epistemological approaches [1]. To address this purpose, mathematics educators should use technology in a transformative way rather than simply assimilated into traditional instruction practices [2,3,4,5]. Artificial Intelligence (AI) technology has developed rapidly, AI in Education (AIED) can support student learning through customized learning, realize an educational design that is impossible in existing educational modes, and provide new opportunities, potential, and challenges for educational innovation [6,7,8]. Intelligent tutoring systems (ITSs) are a representative form of transformative application in AIED [9]. An ITS is an AI application that aims to provide immediate, personalized instruction or feedback to learners without the intervention of a human teacher [10]. An ITS has the potential to play a pivotal role in supporting and supplementing the individualized educational needs of learners in mathematics education, such as personalized learning paths [11], problem-solving steps [12], and scaffolding [13].

However, despite abundant empirical evidence that ITSs are effective in mathematics learning, several scholars have pointed out that ITSs have not reached their full perceived potential in mathematics education [7,14,15]. For example, some ITSs do not provide constructivist opportunities for learners to construct their own knowledge but simply replace or augment traditional practices [15]. Therefore, it not only matters to integrate ITSs in mathematics learning, but it matters how instructors integrate them, because the use of ITSs may otherwise continue to reflect traditional teaching methods [16]. Scholars have also postulated the use of AIED can transform learning (e.g., [6,7,8]), but there is insufficient evidence to determine whether the ITSs at the heart of AIED are transforming mathematics learning or are being used to replicate past practices. A literature review that systematically synthesizes existing empirical evidence can help mathematics education stakeholders make more informed decisions. Existing literature reviews and meta-analyses have addressed ITSs in education, including the effect of ITSs [9], characteristics, application, and evaluation of ITSs [17], and the effectiveness of K–12 mathematics learning [18]. However, little research has addressed explicitly how ITSs are integrated into mathematics education and what challenges impede the integration process.

To address this gap, this study systematically analyzed 20 years of empirical studies on the use of ITSs in mathematics education using the substitution, augmentation, modification, redefinition (SAMR) model [19], which is a taxonomy-based approach for selecting, using, and evaluating technology in educational settings. This model is suitable for describing the level of technology integration in a specific educational context [1]. The present study used the SAMR model to investigate whether ITSs were used to change learning in mathematics education or to replicate traditional learning. The findings provide educators and researchers with insight into why the potential of ITSs has not yet been fully realized in mathematics education and guide future research directions.

2. Background

2.1. Technology Integration in Mathematics Education

Integrating technology into mathematics education has the potential to support student learning and fundamentally transform teaching and learning [1,20]. Cheung and Slavin [21] conducted a meta-analysis of the effects of educational technology applications on mathematics achievement in K–12 classrooms over 30 years. Cheung and Slavin found that technology had a positive effect on student achievement compared to traditional methods, with differential impacts depending on the type of educational technology used. Advances in various information, communication, and computing technologies (e.g., mobile technology; [22]), information and communication technology (ICT) [23], and augmented reality [24] have provided new opportunities for learning mathematics. However, a recent literature review of empirical studies reported that technology use has not lived up to its perceived potential to transform mathematics learning experiences [1]. Students most often experience digital technologies to enhance traditional practices in the context of mathematics education. When technology is used primarily as training and practice in mathematics learning, its impact on student achievement may be minimal or even negative [25,26].

Several scholars have recommended that it is better to use technology for tasks that change significantly using technology rather than tasks that can be performed without technology [1,2,3,5]. In these environments, students have the opportunity to use mathematics for productive purposes rather than following learning procedures according to predefined scenarios [5]. However, technology does not fulfill its potential on its own, instead requiring instructive, careful implementation to realize its potential and avoid negative impacts [27]. Drijvers [28] found the affordances of the technology, the educational context, and the teacher’s role while using technology are important factors in integrating technology into mathematics classrooms successfully. Drijvers emphasized that three elements should be interrelated to make technology work in mathematics education; to do this, teachers should orchestrate learning by (a) synthesizing the results of technology-rich activities, (b) helping students develop ways to use tools effectively, and (c) relating experiences within the technology environment to mathematical activities [29]. Learners will not use the technology independently in ways that result in positive learning gains [30].

2.2. ITSs in Mathematics Education

An ITS is a type of educational technology designed to support one-on-one adaptive learning by providing feedback and hints [31]. Typically, an ITS features a user interface that presents students with instructional material and allows them to answer structured questions, often breaking each question into several steps to prevent student failure [17]. Since the introduction of the first ITS by Jaime R. Carbonell in 1970, ITSs have been used in mathematics education to provide personalized learning activities tailored to learners’ characteristics and needs. The growing interest in ITSs within mathematics education is largely driven by the limitations of human resources and time, as it is often difficult for a single teacher to meet the educational needs of all students [32]. From an economic perspective, ITSs hold the potential to address this issue.

With recent advancements in AI, ITSs have evolved into adaptive learning systems that incorporate AI technologies. These new AI-based systems guide learners through various stages of learning by generating hints and feedback from expert knowledge databases as needed [33]. As AI continues to be integrated into educational contexts, ITSs have emerged as one of the most prominent fields where AI is applied in mathematics education [34]. The primary objective and strength of AI-driven ITSs lie in their ability to assess learners, define models, and provide appropriate feedback. These unique functional competencies can be combined with using games [35], fostering exploratory learning environments [15], and working with multiple representations [36] to form the basis of opportunities for driving curriculum changes, assessment changes, and pedagogical changes [37]. ITSs may offer a chance to reshape how mathematics is assessed, taught, and learned.

However, applying ITSs in mathematics education does not guarantee positive educational outcomes. It is well known that students do not always use technology in ways that lead to positive learning benefits [30]. Despite the unique affordances of ITSs and their potential to transform learning experiences, the innovative potential of ITSs is not always fully realized in mathematics learning [33]. Several scholars have noted that mathematics activities are often not designed to take full advantage of ITSs, with the technology primarily replacing traditional instructional practices [7]. For instance, ITSs typically diagnose learners and provide feedback while structuring the learning process into multiple steps to prevent failure. This design feature minimizes failure and adheres to standardized instructional methods, potentially overlooking the value of productive failure and alternative instructional approaches, such as collaborative learning, inquiry-based learning, and group discussions [7]. Therefore, while ITSs offer unique potential and limitations and the opportunity to fundamentally transform learning, fully realizing these innovative functions requires a deeper understanding of how ITSs can be effectively utilized in educational settings (e.g., [7,14,15]).

2.3. SAMR Model

Several perspectives describe technology integration, and one representative model that classifies technology integration in educational contexts is the SAMR model [19]. The SAMR model was developed based on the technological, pedagogical, content knowledge (TPACK) framework [38] and the replacement, amplification, and transformation (RAT) framework [39].

The SAMR model categorizes the degree of technology integration into two ranges and four levels within each range (see Figure 1). This classification reflects the view that technology integration in education falls into one exclusive category [40]. The substitution level is a situation in which technology is used to directly substitute for traditional methods without functional or conceptual changes, such as transferring the learning content to be taught through technology. In the augmentation level, technology substitutes for traditional teaching methods, but with functional improvements. It is possible to change small-scale improvement, which does not imply strong changes to teaching and learning at this level. In the modification level, technology allows for the redesign of typical lesson goals, activities, and tasks. Physical barriers in the classroom are eliminated as students’ expressions of their positions become more flexible and closer to learning that occurs in a variety of contexts. Finally, the redefinition level is the use of technology for learning in ways that could not be implemented without technology. Two levels at the bottom represent using technology to enhance learning, and two levels at the top represent using technology to transform learning.

The SAMR model reflects the idea that when teaching and learning mathematics, technology should not simply replicate classroom practices but provide opportunities to use mathematics for more productive purposes [1,5]. Therefore, several scholars recommended that it is better to use technology for tasks that are significantly changed by the use of technology rather than tasks that can be performed without technology [1,2,3,5]. In these environments, students have the opportunity to use mathematics for real productive purposes rather than following learning procedures according to predefined scenarios [5]. Based on this perspective, the SAMR model provided a classification standard for integrating technology into mathematics education, such as a literature review of technology [1], interactive whiteboard [41], and digital technology [42]. Researchers have found that the use of technology in mathematics education has been primarily at the replacement and augmentation level (e.g., [1,43]). These results suggest that technology does not fulfill its potential in mathematics education and that educational and careful implementation is needed to fulfill its potential and avoid negative impacts [27].

The SAMR model can lead to misunderstanding and confusion in explaining or understanding how to understand, interpret, and apply the levels due to subjective interpretation [40]. Accordingly, some researchers have expanded the SAMR model to fit specific research contexts, such as learning processes [44], Bloom’s taxonomy [45], and teaching and learning in AIED [46]. In this study, the SAMR model was extended in the context of ITS use in mathematics education. The extended SAMR model for ITSs of mathematics education is described in the methods section.

2.4. Review of Previous Research

As the publication of ITS research papers in the field of education increases, it is necessary to organize and review related issues systematically.

Table 1. Previous studies on ITSs in education.

Author	Number of Studies	Timeline	Focus of Review
Wang et al. [9]	40	2011–2022	Effectiveness on studies that applied social experiment methods
Mousavinasab et al. [17]	53	2007–2017	Characteristics, applications, and evaluation
Kulik and Fletcher [33]	50	1990–2013	Effectiveness in education
Steenbergen-Hu and Cooper [47]	35	1990–2011	Effectiveness for college students
Steenbergen-Hu and Cooper [18]	26	1997–2010	Effectiveness on K–12 students’ mathematical learning
VanLehn [48]	28	1975–2010	Comparing the effectiveness of human tutoring, computer tutoring, and no tutoring

outlines the existing literature review and meta-analysis studies that analyzed ITS-related research papers in the field of education and provides comprehensive results.

Some studies have focused on examining the effectiveness of ITSs across the education field, and the results demonstrated the value of ITS integration in education. For example, VanLehn [48] conducted a meta-analysis of 28 articles and compared the effectiveness of human tutoring, computer tutoring, and no tutoring. VanLehn reported that ITSs are nearly as effective as human tutoring in increasing learning outcomes in STEM subjects, but ITS should not be used to replace the full classroom experience. Steenbergen-Hu and Cooper [47] conducted a meta-analysis of 35 studies in higher education. They reported that ITSs had a moderate positive effect on college students’ academic learning, and although less effective than human tutoring, ITSs outperformed other teaching methods, such as traditional classroom instruction, computer-assisted instruction, and homework.

Some studies have addressed the overall trend in ITSs in education. Mousavinasab et al. [17] reviewed the characteristics, application, and evaluation of ITSs in 53 articles. Wang et al. [9] synthesized 40 empirical studies that examined the effects of ITSs on teaching and learning through social experiments. Other studies have focused on examining the effectiveness of ITSs in mathematics education. Steenbergen-Hu and Cooper [18] conducted a meta-analysis of studies of 34 ITSs on K–12 mathematics learning published from 1997 to 2010, concluding that ITSs did not have a negative effect on mathematics learning and had a small positive effect. Kulik and Fletcher [33] conducted a meta-analysis of 50 articles on K–12 mathematics learning published from 1990 to 2013 and reported that ITSs had stronger effects than the typical outcomes of another private tutoring source.

As a result, existing studies have mainly focused on the effectiveness and overall trends in ITSs in the field of education, particularly the effectiveness of ITSs in the mathematics subject. The findings of these existing studies suggest that the integration of ITSs in the field of education and mathematics education is effective. However, none of the mentioned studies focused on the integration of ITSs in mathematics education. Research examining the degree of integration of ITSs in mathematics education can help researchers understand how ITSs inform the design of mathematics instruction and guide future research that can transform mathematics teaching and learning. Moreover, research focusing on ITSs in mathematics education is very limited. Most studies have addressed the use of ITSs across educational sectors; some studies focused on mathematics education and ITSs (e.g., [18,33]), but they conducted meta-analyses examining effectiveness rather than using a literature review approach. Furthermore, no ITS studies in mathematics education were systematically reviewed after 2013. With the recent rapid advancements and innovations in ITS studies, these reviews may subsequently be outdated. Because the number of ITS studies in education has increased significantly over the past decade, it is timely to conduct a comprehensive and critical review of the extent to which ITSs have been integrated into mathematics education, including the latest research.

2.5. Research Questions

The purpose of this study was to present a systematic review of ITS studies in mathematics education from 2003 to 2023 and reveal how ITSs are used in mathematics education based on the SAMR model. The relevance of the SAMR model to contextualize the teacher’s role of implementing ITSs is emphasized. Three research questions guided this study:

RQ1. What levels of the SAMR levels are ITS studies integrated into mathematics education?

RQ2. What are the trends in ITS studies in mathematics education regarding the SAMR levels across contexts (e.g., publication year, educational level, mathematics domain, research purpose)?

RQ3. What are the trends in ITS studies in mathematics education regarding the SAMR levels across teachers’ roles?

3. Methods

The purpose of a systematic review is to answer specific questions based on an explicit, systematic, and replicable search strategy using inclusion and exclusion criteria that can identify studies that should be included or excluded [49]. This systematic review drew on recently published literature and included empirical studies on the use of ITSs in mathematics education between 2003 and 2023. The literature screening process followed the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) proposed by Page et al. [50].

3.1. Search Strategy

The study search was conducted using the databases of Education Resources Information Center (ERIC), Web of Science, and EBSCO Education Source, which are widely used in the field of educational technology. There are other databases, such as IEEE Xplore, Scopus, and ProQuest, but they were not included in the initial literature search because their scope overlaps with the three selected databases [51]. To obtain comprehensive data, search terms were created by referring to the related search terms used in other studies [9,18]. The Boolean search is listed in

Table 2. Searching strings.

A	B	C	D
1. intelligent * 2. adaptive 3. customized	1. learning 2. instruction 3. tutoring 4. mentoring	1. system 2. software 3. application	1. math 2. maths 3. mathematics 4. mathematics education

Note. The asterisk (*) was used to broaden a search.

. The search was conducted on March 1, 2024, and the initial search yielded 4219 results. The search string was formulated as follows to include as many ITS-related articles in mathematics education as possible: (intelligent * OR adaptive OR customized) AND (learning OR instruction OR tutoring OR mentoring) AND (system OR software OR application) AND (math OR maths OR mathematics OR mathematics education). This text string encompasses all studies addressing ITS in mathematics education in this systematic review. Search terms were separated by commas, and the asterisk character (*) was used to capture variations of intelligent, intelligence, etc.

3.2. Study Selection

The obtained articles were screened according the inclusion and exclusion criteria presented in

Table 3. Inclusion and exclusion criteria.

Inclusion Criteria	Exclusion Criteria
Published 2003–2023	Published before 2003
English language	Not in English
Empirical research with evaluation results	Not empirical research (e.g., commentary) or only proposing ITS design solutions or ITSs without offering evaluation results
Article has been peer reviewed	Article has not been peer reviewed
Journal article	Not a journal article
ITS application in mathematics educational setting	No mathematics educational setting
Articles in which the full text was available.	Articles in which the full text was not available.

after removing 519 duplicate articles and 1389 articles published before 2003 using EndNote reference management software. The period was set from 2003–2023, as ITSs have been very actively researched over the past 20 years and have produced many publications. Only empirical studies in which ITSs were applied in a mathematics education setting and the evaluation results were revealed were included. Articles that were not published in journals were excluded because it was unclear whether they had undergone peer review, an acceptable standard for ensuring scholarly quality.

According to the established inclusion and exclusion criteria, the title and abstract of the paper were reviewed, and the full text was secondarily reviewed. In the first screening, two separate reviewers read the titles and abstracts of all papers and coded them into three groups: (a) included, (b) ambiguous, and (c) excluded. Both reviewers had extensive publishing experience in performing coding. Coding agreement between reviewers was found to be 92%, and papers that did not receive the same coding reached 100% agreement through discussion. An initial review of 2311 studies resulted in the removal of 2007 articles. The remaining 304 articles were retrieved in full text for Stage 2 screening and evaluated by two reviewers. In this process, 21 articles in which the full text was not freely retrievable were excluded. In addition, 225 articles were excluded because the contents of the full text referred superficially to mathematics education or focused on the development process of ITSs rather than the context of mathematics education. Ultimately, 58 articles were identified as meeting the current study goals. The literature search and review procedure based on PRISMA is presented in Figure 2.

3.3. Analysis

An electronic data extraction form based on a coding scheme was designed to reduce the potential for bias [52]. The 58 articles included in the literature review were coded based on the research questions and the background section: (a) contexts (i.e., publication year, educational level, mathematics domain, and research purpose), (b) SAMR levels, and (c) teachers’ roles.

3.3.1. Contexts

In “contexts,” data on the publication year, education level, mathematics domains, and research purpose were collected by reviewing 58 articles. Most of the contexts are self-explanatory; however, “research purpose” included five purposes for conducting research (see

Table 4. Classification of research purpose.

Categories	Description	Example
Development of ITS	Focus on developing and validating new ITSs	Pai et al. [53]
Improvement of ITS	Focus on improving learning productivity by integrating other ITSs or adding new functions to an existing ITS.	Nye et al. [54]
Application of existing ITS	Focus on applying existing ITSs to a specific educational context or mathematics domain and confirming its effectiveness.	Huang et al. [55]
Investigation of factors	Focus on examining factors affecting learning mathematics using ITSs.	San Pedro et al. [56]
Exploring teaching methods	Focus on exploring effective teaching methods in mathematics learning using ITSs.	Cung et al. [57]

).

3.3.2. SAMR Levels

The present study developed the SAMR framework for the technology integration classification of ITSs in mathematics education.

Table 5. SAMR model for ITSs.

Level	Existing Definition	Mathematics Education and ITS Context
Redefinition	Tech allows for the creation of new tasks, previously inconceivable.	Teachers and students use ITSs to implement the teaching and learning of mathematics that cannot be inconceivable without ITSs.
Modification	Tech allows for significant task redesign.	Teachers and students can use ITSs to redesign the goals and tasks of teaching and learning of mathematics.
Augmentation	Tech acts as a direct tool substitute with functional improvement.	The ITS acts the same as teaching and learning of mathematics, in that teachers and students can implement it in classrooms, but with functional improvements.
Substitution	Tech acts as a direct tool substitute, with no functional change.	ITSs act the same in the teaching and learning of mathematics, in that teachers and students can implement it in classrooms.

shows the extended SAMR model for ITSs in mathematics education by referring to Crompton and Burke [44]. This framework was used to code the level of ITS integration in mathematics education.

At the substitution level, an ITS directly substitutes for traditional classes without functional or conceptual changes. For example, the ITS can be used simply to provide practice problems or content to be taught. At the augmentation level, the ITS substitutes for the teacher’s role in a typical classroom, but with functional improvements. For example, researchers can use ITSs to analyze learner performance and provide appropriately tailored content. In a real classroom, it is difficult for a single teacher to respond individually to the individual needs of multiple students, so an ITS is a functional improvement over the role played by traditional teachers. At the modification level, ITSs can redesign the objectives, activities, and assignments of mathematics classes. Learners can use the ITS to design their own tasks or conduct learning in a game-based ITS. In this type of technology integration, the goals and tasks of mathematics classes are redesigned based on learner autonomy. Finally, at the redefinition level, ITS can create new tasks that are inconceivable in traditional mathematics classes. For example, an ITS can act as a virtual student seeking learning, and a student can learn by teaching the virtual student. This type of learning is impossible to achieve without ITSs, as students with the same level of knowledge and background context cannot exist in a real-context classroom. Two levels at the bottom are using technology to enhance learning (i.e., substitution and augmentation levels), and two levels at the top are using technology to transform learning.

3.3.3. Teachers’ Roles

Kessler et al. [58] found four types of roles in which teachers intervene in interactions between students and ITSs in an educational environment using Cognitive Tutor. In this study, three out of four types of teachers’ roles were identified, with the exception of the role where the teacher directly interacted with the ITS on behalf of the student: (a) the ITS environment as designed, (b) teacher facilitating the ITS environment, and (c) teacher facilitating mathematics. In an ITS environment as the designed role, teachers either have no involvement in students’ mathematics learning or only technically supervise it, and pedagogical interaction between students and teachers is minimal. All mathematics teaching and learning processes occur between ICTs and students. In the teacher facilitating the ITS environment role, the teacher assists the student using the ITS through the process and provides feedback upon the student’s help. Although the mathematical teaching and learning process still occurs between students and the ITS, teachers intervene in this process as mediators and play a role in facilitating students’ ITS learning. In the teacher facilitating mathematics role, teachers lead mathematics learning through ITSs from lesson design to lesson conducting. Teachers determine what learning content is provided to students in the ITS or connect ITS learning with meaningful mathematical activities outside of the ITS learning environment. Teachers interact with students about their specific mathematical thinking, and students can physically move away from the ITS.

3.3.4. Coding

Table 6. Coding framework.

Category	Code	Example
Context	Publication year	2013
	Educational level	Elementary School
	Mathematics domain	Algebra
	Research Purpose	Development of ITS
SAMR level	Substitution	Substitution
	Augmentation
	Modification
	Redefinition
Teachers’ roles	ITS environment as designed	ITS environment as designed
	Teacher facilitating the ITS environment
	Teacher facilitating mathematics

summarizes the coding framework in three categories (i.e., contexts, SAMR levels, and teachers’ roles). Two coders coded 58 articles independently based on the coding framework and compared the results. The agreement rate for the initial coding was 92%. The two coders met online to review articles with inconsistent coding and reached 100% agreement through discussion.

4. Results

A total of 58 studies were finally reviewed and coded based on the coding scheme. Coding results are presented in the Appendix A. This section presents overall trends in the studies reviewed and findings for each of the three research questions that guided the systematic review.

4.1. RQ1. What Level of the SAMR Levels Are ITS Studies Integrated into Mathematics Education?

All four levels of the SAMR framework were identified in the reviewed 58 studies (see Figure 3). The substitution level comprised 1.7% (n = 1) of the total article. The ITS substituted for the teacher’s role without functional improvements at this level, such as presenting problems to students and providing feedback. For example, del Olmo-Muñoz et al. [14] used an ITS called HINTS (Hypergraph-based INtelligent Tutoring System) to address the gap in low-income households. Students solved practice problems provided by the system, read solutions, or solved similar additional problems. To prevent students from relying on messages from the ITS, aids on-demand was prevented. This did not seem to have any added value compared to teachers giving students worksheets with practice problems and providing additional feedback.

The augmentation level comprised the largest proportion of reviewed articles at 67.3% (n = 39). This result indicates that the ITS has been used to replace traditional approaches in mathematics education with some functional or conceptual improvements directly. For example, Walkington and Bernacki [59] applied Cognitive Tutor to learning algebra, which provides customized problems tailored to students’ individual interests. Students learned mathematics according to the learning path provided by the ITS, which did not involve lesson redesign or self-inquiry learning.

The modification level comprised 24.1% (n = 14) of total articles. For example, Çetin et al. [35] developed ArtiBos, which allows students to perform problem-posing activities based on the information provided by the system. Students were able to select appropriate questions and problems and solve and evaluate them following Polya’s problem-solving steps. Students are given autonomy to redesign course objectives and tasks rather than learning according to a rigid ITS structure at this level.

The redefinition level comprised 6.9% (n = 4) of all articles. Matsuda et al. [60] used SimStudent, which enables learning by teaching, to provide students with an artificial peer learning environment. The ITS acted as a student seeking learning, and the student learned by teaching the ITS. Implementing such learning is impossible without technology, as real-world educational contexts cannot produce students with the same level of knowledge and background.

In summary, most of the reviewed literature used ITSs for the substitution and augmentation of mathematics learning. It is desirable to use the technology for tasks that fit the two higher levels of the SAMR hierarchy [1,5]. However, only 31% of the reviewed literature used ITSs in this way, and the redefinition level included only 6.9%. Despite the smaller number of the transformation range compared to the enhancement range, studies classified as modification and redefinition indicate ongoing efforts to promote change in mathematics learning.

4.2. RQ2. What Are the Trends of ITS Studies in Mathematics Education Regarding the SAMR Levels across Contexts?

4.2.1. Publication Year

Figure 4 illustrates the change in the number of published articles from 2003 to 2023. There was a peak in 2019 in terms of the number of published studies on ITSs in mathematics education. The ITS studies were categorized into two time periods per decade unit, that is, 2003–2012 and 2013–2023. The number of publications in the latter period (n = 44, 75.9%) was over 3 times that of the previous period. This shows that ITS research in mathematics education has grown rapidly over the past 10 years.

Figure 5 shows trends in changes in SAMR levels across published studies by decade. In the first decade, ITSs were integrated into mathematics education mainly at the augmentation level (n = 12, 85.7%). There were two studies (14.3%) on the modification level, and no study on the transformation range was conducted. On the other hand, in the latter decade, several studies were conducted at the level of modification 14.3% (n = 12) and redefinition 8.5% (n = 4). These results suggest most ITS research in the transformation range in mathematics teaching and learning has been conducted in the last decade.

4.2.2. Educational Levels

Figure 6 shows the distribution of SAMR levels across educational levels covered in the reviewed literature. Some studies ([61,62]) applied ITSs to multiple educational levels, so 60 educational levels were included. The most ITS research was conducted in elementary school at 41.7% (n = 25), followed by middle school at 25% (n = 15), high school at 18.3% (n = 11), colleges at 13.3% (n = 8), and teacher at 1.6% (n = 1). Most studies were conducted in the context of K–12 mathematics learning; however, there was no ITS study conducted at the Pre–K educational level, and the fewest ITS studies were conducted for teacher education.

Across all educational levels, researchers used ITSs primarily at the augmentation level (elementary school 61.5%, middle school 73.3%, high school 50%, college 75%, teachers 100%). This shows that mathematics teaching and learning at educational levels are not fundamentally changed by ITSs but are mainly enhanced. However, studies at the modification level or redefinition level were also conducted at diverse educational levels. Specifically, at the high school education level, studies at the transformation range (modification 41.7%, redefinition 8.3%) were conducted similarly to the augmentation level (50%). For example, Walker et al. [63] applied adaptive collaborative learning support, which provides collaborative learning environments to promote collaborative interactions among individual students. Students became tutors and provided feedback on the problem-solving process of their tutee peers and received feedback from the system on their own reflective behavior. These findings provide evidence for multiple examples in which an ITS has been transformative for mathematics learning. These researchers did not substitute for traditional practices but rather focused on pedagogical practices that go beyond existing teaching methods.

4.2.3. Mathematics Domain

Figure 7 shows the distribution of SAMR levels across the mathematics domains. The reviewed studies covered six content domains of mathematics education. Most studies were conducted around the domains of numbers and arithmetic (38.7%), algebra (32.3%), and geometry (11.3%). Seven studies classified as ‘Not specified’ did not explicitly describe to which mathematical domain the ITS was applied. In some studies, the ITS applied to multiple domain contexts (e.g., [12,64]), and the total number of mathematical domains was 62. For example, Chang et al. [12] applied ITSs to multiple domains, such as the operation of fractions, the area of triangles, and unit cost.

In numbers and arithmetic, ITSs augmented mathematics learning primarily through problems solutions, and feedback to practice basic arithmetic and fraction operations (e.g., [65,66,67]). This type of ITS presented students with mathematics problems step by step, taking individual differences into account and providing immediate feedback. On the other hand, some studies have transformed mathematics learning in the domain of numbers and arithmetic using ITSs. For example, Kong and Kwok [68] applied ITSs to learn fraction equivalence. Students were able to engage in exploratory activities that allowed them to formulate and test hypotheses, and the ITS provided a profitable space for interaction. In algebra, the ITS was most frequently used at the modification level (n = 7; 35%) and redefinition level (n = 3; 15%) compared to other domains. The ITS at these levels provided an opportunity to redesign modeling lessons [69], support self-regulated learning by applying an open-learner model [70], and implement artificial teachable agents that would be difficult to inconceivable without an ITS [60]. In geometry, ITSs were most often used at the augmentation (n = 6; 85.7%) level, and there was only one study that applied ITSs at the modification level. Butcher and Aleven [71] applied ITSs to promote rule-diagram mapping where students connect knowledge to diagrams. Students first made solvability decisions through a reasoning process about a given problem, specifying the geometric rules and diagram features that they would use to arrive at the correct answer. Statistics, logic, and calculus had fewer studies than other domains. Although it is difficult to draw generalizable conclusions about SAMR levels in the three domains due to the limited number of studies, all three domains have been studied at the augmentation level.

4.2.4. Research Purpose

Figure 8 shows that the reviewed literature conducted various studies with five research purposes. “Development of ITS” was the research purpose of most studies (n = 17; 28.3%). This finding means researchers have paid the most attention to developing new ITSs in mathematics education for two decades. Although there are slight differences in the number of studies conducted with each research purpose set, they are generally evenly distributed. This result indicates that research on ITSs in mathematics education has been conducted for various purposes without being biased.

4.3. RQ3. What Are the Trends of ITS Studies in Mathematics Education Regarding the SAMR Levels across Teachers’ Roles?

Figure 9 shows the distribution of SAMR levels across teachers’ roles. One ITS study for teachers was excluded from the analysis [72]. Some studies reported diverse roles for teachers [66,73]. For example, Phillips et al. [73] reported that although teachers were encouraged to consider different models for blended learning, most teachers did not intervene in ITS learning, and only some teachers integrated ITSs into mathematics learning. As a result, a total of 59 papers were analyzed. The teacher’s role is predominantly one of managing or not intervening in learning between students and ITSs (72.9%). There were 16.9% (n = 10) of studies in which teachers played a role in facilitating learning between students and the ITS, and the role of teachers in leading ITS learning and connecting ITS learning with external mathematics activities accounted for the lowest percentage of all studies (n = 6; 10.2%). These results implicitly reveal the dominant perspective that ITSs can replace the role of teachers in mathematics education.

In the ITS environment as the designed role, studies at the augmentation level were the most dominant, with all levels of SAMR presented in this teacher’s role. This finding could be interpreted to suggest that teacher intervention did not have a significant impact on the task design of ITSs. These studies mainly attempted to prove that ITSs can replace teachers by comparing the effects of ITS learning and teacher-led learning (e.g., [74,75,76]). Because mathematics learning takes place between the ITS and the student and the teacher’s role is minimal, the effectiveness of mathematics learning is difficult to exceed the affordances of the ITS.

In the teacher facilitating the ITS environment role, most studies were conducted at the augmentation level (n = 8; 80%), and some studies were conducted at the modification level (n = 2; 20%). Teachers intervene and provide assistance with students’ ITS learning, but mathematics learning occurs between the ITS and students, and it is difficult for teachers’ intervention to change SAMR levels. For example, Stillson and Alsup [77] found that although teachers supported ITSs by providing additional help in understanding explanations, students either gave up participating or experienced confusion and frustration due to the difficulty of using the ITS.

In the teacher-facilitating-mathematics role, studies were conducted at 50% each at the augmentation level and 50% at the modification level. In these studies, students engaged in self-directed learning, engaging in ITS and off-ITS learning. For example, Kong and Kwok [68] designed a learning cycle for ITS learning. Students participated individually and sometimes in groups, with support from the ITS in the early stages and without ITS support in the later stages. Students were encouraged to freely explore the ITS according to their interests and mainly used the ITS as a communication medium or to conduct group discussions to consolidate the objectives of the learning activities. In these processes, the ITS was one part, not the whole of learning, and the teacher played a role in designing and leading the entire mathematics lesson.

5. Discussion

The purpose of this study was to provide an updated synthesis of the level at which ITS studies have been integrated into mathematics education. The multiple contexts and teachers’ roles that determine successful technology integration in mathematics education were examined simultaneously at the SAMR levels. Although the reviewed literature does not provide a clear picture of the actual integration of ITSs in the classroom, it does provide a clear guide to research trends.

The results of the study showed that ITSs have been mainly used (67.2%) at the augmentation level in mathematics education over the past 20 years. The use of ITSs in mathematics education at lower levels of the SAMR framework supports the argument of several scholars that the externally recognized potential of technology generally lags behind in improving teaching and learning [1,5,14]. However, 31% of total studies implemented using ITS were found to be at the level of modification and redefinition. Although this number is not sufficient to drive transformation in mathematics education, recent ITS studies in mathematics education provide evidence that they are transforming experiences of students’ mathematics learning. The SAMR classification results for publication year support this evidence. In recent decades, researchers have been using ITSs to redesign the mathematics task and realize teaching and learning of mathematics that would be unimaginable without ITSs (see Figure 5). Some examples in this range include game-based learning [78], exploratory environments [36], and teachable learning [60]. These approaches demonstrate that researchers recognize the affordances of ITSs in allowing learning beyond what learning activities performed without technology can achieve.

ITS research on mathematics education was conducted in the context of K–12 mathematics learning, focusing primarily on elementary schools (41.7%). This result is inconsistent with existing literature studies [9], which revealed that ITS studies in education mainly focus on secondary education. Elementary school students may have more opportunities to use ITSs because they are relatively free from the constraints of the scholastic aptitude test. There is little empirical ITS research with teachers and preschoolers. This is consistent with Wang et al. [9], which reported that ITS research on adult education and early childhood education in the education field is limited. Teacher professional development related to technology can lead to greater confidence in the benefits of technology to support mathematics learning [79]. Early mathematics is a strong predictor of later learning, so the use of technology in mathematics is encouraged from early childhood [80,81]. Therefore, more research is needed to better understand the affordances of ITS-supported learning in mathematics education at the teacher level and preschool level.

Studies using ITSs at the educational level have mainly been conducted at the augmentation level (see Figure 6). These results show that the affordances offered by ITS learning have not been fully explored at the educational level. However, it is interesting to note that research on transformation range has primarily been conducted at the high school education level compared to other education levels. One example is research that promotes positive collaborative learning and encourages reflective processes among colleagues [63]. Walker et al. [63] used an ITS to focus on pedagogical practices that go beyond traditional classroom practices rather than replacing them. These types of learning provide opportunities for students to apply, evaluate, and analyze knowledge regardless of the educational content (Bloom’s taxonomy; [45]) and need to be expanded to other educational levels.

This study found that the three domains of number and arithmetic, algebra, and geometry were the most studied domains of ITSs in mathematics education over the past 20 years. On the other hand, there has been little empirical research on reasoning, statistics, and calculus. Most studies have been conducted at the augmentation level, and this tendency may be due to the characteristics of the domain. Since most ITSs guide learning in a step-by-step manner toward a mastery goal [7], the process of calculating the operations of natural numbers and fractions and measuring the area of shapes in the step-by-step design method is intuitively appropriate for ITS development. However, the potential of technology to transform mathematics education goes beyond simply memorizing and mastering knowledge. If mathematics is characterized as a subject that simply uses rote learning methods, it will be almost impossible to redefine ITSs in mathematics education [44]. Technology integration in mathematics education should be based on realistic contexts, inquiry-based approaches, and collaborative approaches [1]. This gap between research findings and recommendations in mathematics education presents a challenging domain and potential opportunity for applying ITSs and suggests that further research is needed.

This study also found that SAMR levels differ depending on the research purposes. Studies conducted for the purpose of developing an ITS, applying an existing ITS, or exploring factors affecting ITS performance have mainly been conducted at the augmented level. On the other hand, studies aimed at improving the performance of ITSs or exploring effective teaching methods have primarily been conducted on the level of technology integration that can transform mathematics learning. These findings may stem from differences in the research purpose. Studies conducted with the purpose of applying existing ITSs to a specific educational context or exploring factors that influence ITS learning may be difficult to learn mathematics to exceed the limits of the affordances of the predesigned ITS. On the other hand, studies conducted for the purpose of improving the functionality of ITSs and exploring effective teaching methods generally start from considering the limitations of existing ITSs or teaching methods and are therefore more likely to change from ITS-led learning to learner-centered learning. Interestingly, studies conducted for the purpose of the “Development of ITS” mainly focus on enhancing mathematics learning. Considering the cost of designing and developing a new ITS, it may be more effective to improve the functionality of an existing ITS. However, these results do not mean that ITS research in mathematics education should be conducted for specific research purposes. Rather, it is important to recognize the limitations of existing ITSs and teaching methods and to make challenging attempts that consider the affordances of ITSs, educational context, and the teachers’ roles [28]. These attempts can lay the foundation to help learners redesign their lesson goals and realize mathematics lessons that would be unimaginable without ITSs.

This study found that ITS studies in mathematics education primarily were conducted in a way that minimizes teacher intervention. Almost 72% of ITS studies excluded or minimized teacher involvement, reflecting an implicit belief that ITSs could replace teachers to teach students. This finding counters the dominant perspective that the success of technology use in mathematics education depends on the teacher (e.g., [28,29]). del Olmo-Muñoz et al. [14] criticized research on the application of technology in mathematics education as having a very limited vision that prioritizes the use of teacher intervention, and most studies in this study have seemed to emphasize the standalone use of ITSs. However, several scholars have reported that students face various challenges in learning ITSs with minimal teacher intervention. For example, Shih et al. [75] noted that teaching using ITSs is well designed only for specific course topics, requiring teacher intervention to flexibly adjust the lesson content and handle unexpected student responses. Matsuda et al. [82] reported that teachers’ intervention is essential because students using teachable ITSs do not accurately recognize their own errors, unconsciously make inappropriate instructional decisions, and provide incorrect feedback and hints. Bartelet et al. [66] pointed out that teachers’ explanations and interventions are essential for the positive effects of ITSs on students. These findings show that even though an ITS is designed to minimize the teacher’s role, teacher intervention is essential to increase student engagement and minimize errors. Therefore, the present study contributes to guiding teachers on how to use ITSs to promote productive student learning and create exploratory learning environments. For example, teachers can use ITSs to promote student engagement in exploratory activities, develop learning tasks, or test student hypotheses to innovate mathematics lessons using ITSs (e.g., [68,71]). Furthermore, teachers can use the extended SAMR model as a guide when teaching mathematics using ITSs to determine whether ITS use is transforming the mathematics classroom or simply replicating existing practices and to reflect on their own teaching practices.

6. Conclusions

The present study offers a new perspective for investigating the integration of ITSs in mathematics education using the SAMR framework as a lens. This study extends existing research on the level of technology integration by including explicit definitions of mathematics education and ITSs at the SAMR level. The results of this study suggest that ITSs were mainly integrated into mathematics education at the augmentation level, but recent changes are taking place to transform mathematics learning. Given the affordances of AI for educational enrichment, the reason for using ITSs may not be limited to replicating and improving teaching methods that can be implemented in the classroom. The most powerful use of ITSs is in using it to transform learning. When an ITS implements teaching and learning that cannot be realized in traditional mathematics classes, ITSs will become an infrastructure for specific teaching and learning, and their use will become essential [83]. However, an ITS itself does not guarantee success in mathematics education. When applying ITSs to education, priority should be given to examining what is pedagogically feasible rather than pursuing what is technologically possible [20]. It is recommended that teachers and researchers provide opportunities to engage students in new teaching and learning activities that would be inconceivable without using ITSs. For ITSs to be useful in mathematics education, the affordances of ITSs, educational context, and teachers’ roles in lesson design should be considered together.

7. Limitations and Future Research Agenda

This study had limitations. The keywords, databases, search periods, and inclusion criteria used in the literature search may not provide a description of all studies that have integrated ITSs into mathematics education. Despite this limitation, the findings of this review offer several practical research directions for stakeholders in mathematics education. First, it may not be desirable to expand the use of ITSs across all educational levels and content areas in mathematics education. The reviewed ITS studies minimized teacher involvement and focused on teaching-specific content areas at the elementary level. Therefore, decisions about teacher adoption of ITSs should be made in ways that contribute to effective mathematics learning. For example, if mathematical learning is better supported by other activities, such as inquiry-based learning, discussions, or group projects, ITSs could be utilized as a tool for certain instructional activities rather than as a standalone method. Therefore, future research should explore why teachers utilize ITSs in mathematics education and how they utilize ITSs in their lessons.

Second, although an ITS is considered a technology for implementing personalized learning in mathematics education, only a limited number of ITS interventions provided fully productive and inquiry-based learning experiences. Moreover, students using ITSs continue to face persistent challenges, such as technical malfunctions and high costs. These limitations may hinder the broader adoption of ITSs in more general curriculum instruction. Thus, to generate collective evidence on the effectiveness of ITSs in mathematics education, a shift from anecdotal findings within specific topics to a broader academic context is necessary. Such a shift would enhance the generalizability of research outcomes and uncover the complex relationships between mathematical content and technological affordances.

Third, researchers should shift their focus from ITS technology itself to ITS-based pedagogy, emphasizing the effective integration of technology and educational theory. The effectiveness of an ITS, once it becomes an integral part of mathematics curricula, will largely depend on whether it can support instructional design and implementation. Therefore, future studies need to investigate the theoretical foundations of ITS interventions and their intricate connections to educational theory.