Evaluating Recent Advances in Affective Intelligent Tutoring Systems: A Scoping Review of Educational Impacts and Future Prospects

Abstract

Affective intelligent tutoring systems (ATSs) are gaining recognition for their role in personalized learning through adaptive automated education based on students’ affective states. This scoping review evaluates recent advancements and the educational impact of ATSs, following PRISMA guidelines for article selection and analysis. A structured search of the Web of Science (WoS) and Scopus databases resulted in 30 studies covering 27 distinct ATSs. These studies assess the effectiveness of ATSs in meeting learners’ emotional and cognitive needs. This review examines the technical and pedagogical aspects of ATSs, focusing on how emotional recognition technologies are used to customize educational content and feedback, enhancing learning experiences. The primary characteristics of the selected studies are described, emphasizing key technical features and their implications for educational outcomes. The discussion highlights the importance of emotional intelligence in educational environments and the potential of ATSs to improve learning processes. This review identifies gaps in the current research and suggests future directions, including broader implementation across diverse educational settings and deeper integration of affective data to refine system responsiveness. Future research should investigate the integration of advanced natural dialogue modules and generative AI to create more sophisticated interfaces, underscoring the role of affective adaptation in educational technology.

1. Introduction

An intelligent tutoring system (ITS) is an advanced piece of educational software that integrates artificial intelligence technologies to personalize learning. It adapts to the individual needs, styles, and knowledge levels of each student through integrated models of the domain, student, tutoring, and interface, aiming ultimately to improve educational outcomes with an adaptive and flexible learning experience [1,2].

Computational intelligence approaches in ITSs use techniques, such as neural networks and data mining, to customize teaching and enhance educational interaction. Teaching strategies in intelligent tutoring systems, such as scaffolding [3], Socratic questioning [4], and game-based learning [5], are designed to increase interactivity, motivate students, and facilitate a deeper understanding of the educational material by adapting to their individual needs [6].

Formative feedback in an ITS involves adaptive and personalized feedback that continuously guides and enhances student learning [7]. Educational approaches in intelligent tutoring systems adapt the content and teaching methods to the individual visual, auditory, kinesthetic, or reflective preferences of the students. This adaptation is based on established learning style models, such as Fleming’s VARK model [8], the Felder–Silverman model [9], or the MBTI model [10]. These frameworks help personalize instruction to meet each student’s unique learning needs [11,12]. According to [13], these adaptations can be further categorized within a taxonomy that considers both cognitive and affective dimensions, providing a more nuanced understanding of how ITSs can be designed to respond to the emotional states of learners.

Educational Recommender Systems (ERSs), a type of ITS, tailor their educational recommendations to each student, considering aspects such as learning pace, content comprehension, and skills mastery. In addition to cognitive profiles, these systems also take into account students’ affective experiences, recognizing the significant impact of emotional states on educational performance [14,15,16,17]. According to [18], adaptive learning technologies not only tailor the instructions based on learners’ current knowledge states, but also dynamically adjust to their affective and motivational states, thus enhancing the overall learning experience.

In particular, Context-Sensitive Affective Educational Recommender Systems (CSAERSs) personalize pedagogical recommendations by considering the context, emotional state, and personality traits of the user [19]. In this vein, an affective tutoring system (ATS) is an advanced tutoring system that, in addition to considering cognitive aspects, integrates technologies for recognizing and responding to students’ emotions. It utilizes artificial intelligence and computational models to dynamically adapt educational content, teaching strategies, and feedback, thereby enhancing the learning experience, motivation, and educational outcomes, and supporting self-regulated learning [20,21,22].

According to [6], the architecture of an ATS is structured around three main modules: the affect perception module, which captures and processes emotional signals from the student; the student subsystem, which infers the emotional state and adjusts the learning strategy; and the tutoring subsystem, which leverages both emotional and cognitive information from the student to personalize teaching and optimize the educational experience. The emotional models in ATSs are based on conceptual frameworks that view emotions as basic and universal [23], or on ecological valence and activation models [24,25,26], which are essential for tailoring the system’s educational responses to the students’ affective states [27].

The interface of an ATS can respond to different designs and functionalities. It may present itself as smart software in a traditional 2D design, adapted for learning via traditional computer screens or mobile devices [28], but also for three-dimensional virtual learning environments, whether immersive or not, depending on its incorporation of various types of cutting-edge virtual reality, augmented reality, or mixed reality [29]. Figure 1 shows a basic architectural diagram of an ATS, distinguishing the modules intrinsic to such a system, from the interface and capabilities of the dialogue system to the incorporation of virtual agents.

Additionally, communication between the ATS and the user may be mediated by educational virtual agents, which may also present various appearances [30] and dialogic capabilities, influencing how students perceive them [31,32]. The ability to process and understand natural language (NLP and NLU) can be decisive in the form, perception, and effectiveness of the interaction with the user, impacting the outcome of the learning process [33,34]. The role that the virtual pedagogical agent (VPA) adopts is also key [35].

Emotion recognition in ATSs involves the collection of diverse data types, such as facial recognition, posture analysis, text analysis, voice recognition, and/or the monitoring of biometric data [36,37]. These data, through various technical approaches, are interpreted and linked to the emotional states of students, allowing the system to adapt its teaching strategies and feedback in real-time [38]. These procedures, which require recording devices like cameras and sensors, are subject to the ethical considerations inherent to sustainable education [39].

Ref. [40] compiled some of the most relevant ATSs from the past decade. Affective AutoTutor [41,42], for example, is a comprehensive multimodal system featuring an array of sensors for gaze, posture, and text recognition. It incorporates virtual agents and natural language processing (NLP), providing cognitive and affective feedback in predefined short phrases. Guru Tutor [43,44] includes gaze and text recognition and tracking, incorporating interactive virtual agents (IVAs) in non-immersive 3D environments and NLP. MetaTutor [45] is a tool for learning about the circulatory system, with a carefully designed interface, facial recognition, gaze tracking, and biometric data measurement, featuring IVAs and NLP. FERMAT, on the other hand, is an ATS oriented towards social networks and mobile devices, including IVAs [46,47].

Although ATSs have evolved since their inception and some have established long histories, the majority still remain theoretical concepts within the realm of research, with no significant impact on current education [1]. Recent advances in generative artificial intelligence, in areas such as content creation [48], natural language processing (NLP), and natural language understanding (NLU), along with developments in spatial computing and mixed reality, could be crucial for developing immersive interfaces and flexible, versatile virtual pedagogical agents. These agents facilitate self-regulated learning (SRL) from an emotionally sensitive perspective, promoting meta-affective capabilities within sustainable education [17].

Thus, the primary goal of this study is to evaluate the impact and potential of affective intelligent tutoring systems (ATSs) in education by reviewing the latest trends, strategies, and designs proposed in educational research, to identify future developmental directions. From this main objective, the following specific objectives are derived:

• Identify whether these systems employ traditional 2D interfaces, leverage virtual or mixed reality environments, or even utilize robot tutors, considering how these aspects of their design affect their integration and functionality;
• Explore which educational levels (elementary, secondary, university) and areas of study (sciences, humanities, etc.) these systems primarily target, and identify the specific contexts (online learning, hybrid classrooms, etc.) where they offer the most benefits;
• Categorize the technical features related to the emotional recognition of students, as well as the pedagogical features, such as content adaptation, adjustments in task difficulty, and personalization of feedback based on the emotional state of the student;
• Distinguish between systems that use natural language processing (NLP) and conversational agents and those that do not, evaluating whether this feature constitutes a differential factor in terms of pedagogical effectiveness and user satisfaction;
• Investigate whether adjusting assessment strategies and feedback based on emotions improves understanding and knowledge retention, and assess its impact on academic outcomes;
• Analyze the effect of these systems on student satisfaction and motivation, as well as the attitudes, opinions, and acceptance of students and teachers in regard to these systems, including potential concerns or resistance;
• Identify the technical, pedagogical, and contextual challenges that affect the effectiveness of these systems;
• Explore future directions for research and technological development in this field, including the integration with other emerging technologies.

Based on this general objective and the specific objectives derived from it, the following research questions are stated:

Q1.

What type of interface do ATSs use, and what implications does this design have for the system?

Q2.

What are the main educational levels and fields of knowledge that ATSs are applied to in educational contexts?

Q3.

What specific functionalities do ATSs provide to identify the emotional state of students and support personalized learning?

Q4.

To what extent do ATSs integrate NLP and conversational agents to create a stronger connection and more natural and effective interaction with users?

Q5.

How does emotional recognition in ATSs impact the assessment and feedback provided to students?

Q6.

How do ATSs influence academic performance and the learning experience of students?

Q7.

What are the perceptions of students and teachers regarding the implementation of ATSs?

Q8.

What factors contribute to the success or failure of ATSs?

Q9.

What future is projected for ATSs? What are the emerging trends and potential future developments?

To address these questions comprehensively, we will conduct a scoping review, analyzing research from the last five years. This method will allow us to identify trends, advancements, and key challenges in intelligent tutoring systems with emotional recognition, thus establishing a solid foundation for future inquiries in this field.

2. Methods

For the execution of this scoping review, we followed the guidelines from the PRISMA for Scoping Reviews (PRISMA-ScR) framework [49,50]. The primary databases searched were the Web of Science (WoS) and Scopus, utilizing the concepts, connectors, and Booleans as described in

Table 1. Search strategy: terms, connectors, and Booleans.

Category	Connector	Search Terms
Systems		“intelligent tutoring system*” OR “intelligent educational system*” OR “smart tutor*” OR “adaptive learning system*” OR “personalized learning environment*” OR “Intelligent Virtual Agent *” OR “Animated Pedagogical Agent*” OR “Intelligent Computer-Assisted Learning” OR “ICAL” OR “Intelligent Tutor*” OR “Intelligent Pedagogical Agent” OR “Intelligent Learning Environment” OR “Affective Tutoring System”
AI Technologies	AND	“artificial intelligence” OR “AI” OR “machine learning” OR “deep learning” OR “neural networks” OR “predictive modeling” OR “generative artificial intelligence” OR “GAI”
Emotional Aspects	AND	“emotion*” OR “affect*” OR “sentiment*” OR “emotion detection” OR “emotional analytics” OR “affective computing” OR “sentiment analysis” OR “emotional intelligence” OR “emotional adaptation” OR “empathetic computing” OR “mood detection” OR “emotional response analysis” OR “affective state modeling” OR “emotional state recognition” OR “emotional adaptation” OR “emotionally aware tutoring” OR “emotion-based adaptation” OR “personalized learning paths based on emotional state” OR “adaptive content delivery based on emotional cues”

. The search was conducted on 4 April 2024.

The inclusion and exclusion criteria were established based on the type of publication, type of study, technologies used in intelligent tutoring systems (ITS) with emotional recognition, educational context, studies written in English, publication within the past five years, research quality, and accessibility, as detailed in

Table 2. Inclusion and exclusion criteria.

Criterion	Inclusion	Exclusion
Publication Type	- Peer-reviewed journal articles. - Relevant conference chapters. - Chapter books.	- Editorials, commentaries. - Peer-unreviewed conferences that do not provide original data. - Preprints.
Time Period	- Published within the last five years.	- Publications older than the last five years (2019–April 2024).
Study Topic	- Specific studies on ITS with automatic emotional recognition. - Educational applications, impact, and ethical considerations.	- Studies not focused on ITS with emotional recognition.
Employed Technologies	- Use of computational intelligence, automatic affect recognition, instructional approach and feedback management, NLP, and conversational agents.	- Works not specifying technologies applied in regard to ATS.
Educational Context	- Applications across different educational levels and study areas.	- Research outside the educational context. Simulators or training systems are not specifically excluded if they present an empirical component.
Language	- Primarily in English. - Other languages, if reliable translation is guaranteed.	- Documents in inaccessible languages without the possibility of reliable translation.
Research Quality	- Empirical ATS testing in real educational environments. - Rigorous evaluation of methodological quality.	- Studies only descriptive, not empirical. - Peer-unreviewed studies with low methodological quality.
Incomplete Data	- N/A	- Studies that do not provide enough data for evaluation.
Repeated Studies	- N/A	- Redundant publications.
Accessibility	- Completely accessible documents.	- Inaccessible documents.

.

The inclusion criteria were applied by the sole author of this review. An initial screening of titles, abstracts, and keywords of the automatically selected works was conducted to filter out those that did not meet the specified preferences. Following this initial selection process, a detailed reading of each chosen work was carried out, and studies that did not strictly comply with the established criteria were removed from the sample. Automation tools used in this process included database filters for publication year, type of article, and language.

The flow diagram shown in Figure 2 illustrates the filtering process used to obtain the final sample, in accordance with PRISMA guidelines. It is important to emphasize that the process was not completely linear. During the in-depth analysis of the initial sample, obtained by applying automatic filters, such as publication type, time period, and article language, a few studies were identified outside the initial database searches. Consequently, a small portion of the outcomes can be attributed to a snowballing process, which involved systematically identifying additional studies through the references found in the initially selected works.

From each article selected for the final sample, relevant information was extracted according to the topics chosen to help answer the research questions posed, resulting in several distinct review tables and figures.

The outcome domains extracted from the selected articles include study design, ITS (intelligent tutoring system), educational level/area, affect system functions, outcomes, ATS (affective tutoring system) interface design, facilitators, barriers, ATS knowledge domain, ATS educational level, ATS affect recognition system, ATS pedagogical features, ATS dialogue system, evidence, and future work and implications. Each of these domains was meticulously analyzed to ensure comprehensive coverage of the research questions posed in this scoping review. The detailed extraction of these outcome domains facilitated a robust synthesis of the findings, providing a clear framework for understanding the various aspects and impacts of ITSs with emotional recognition in educational contexts.

Risk of Bias Assessment

The application of the ROBIS tool [51] to assess the bias risk in the present scoping review on affective intelligent tutoring systems demonstrated that the methodology used is robust and well-founded. The eligibility criteria were clear and appropriate, the search strategies were comprehensive, and the processes of study selection and data collection were rigorously conducted to minimize potential errors. Since this is a scoping review, we did not conduct a quality appraisal of the included studies, which is consistent with the framework proposed by [52]. All these aspects led to the conclusion that there is a low risk of bias in this review, reinforcing the reliability and validity of the findings, ensuring that the results reflect an unbiased interpretation of the available literature.

3. Results

While various criteria could be used to categorize the selected studies, we have chosen to initially divide them into two main groups based on the modality of their emotional recognition systems. Therefore, the first group includes studies that describe ATSs with multimodal systems capable of the affective recognition of users. Conversely, the remaining selected studies focus on ATSs with unimodal systems, highlighting this key aspect of the intelligent tutoring systems under review.

3.1. Multimodal Affect Recognition Systems

This subsection presents the ATSs with emotional identification systems that integrate at least two distinct data sources or strategies. A total of 11 ATSs, employed across 14 studies, satisfy this criterion. Their main characteristics are summarized in

Table 3. Multimodal affective tutoring systems.

Citation	ITS	Educational Level/Area	Affect System Functions
[53]	ALS	6–18 yrs/Intell. Disabilities	Multimodal affect em. recognition
[54]	EAITS	FLL at University Level	Facial/head pose em. recog., adaptive FB
[55]	Unnamed	HE/Digital Art	Facial/Eye tracking em. recog.
[56]	ATTENDEE	E-learning, HE/GAI	Facial/head pose em. recog.
[57]	SimInClass	University Teacher Education	EEG, GSR, facial recog., affective FB
[58]	Unnamed	HE/Technology and Art	Facial and semantic em. recog., em. FB
[59]	MetaTutor	HE/Circulatory System	Multimodal data. Classify, monitor, scaffold emotions
[60]			Detect co-occurring emotions via eye tracking
[61]			Scaffolding SRL
[62]		HE/Learning Disabilities
[63]	InIv ATS	HE/HCI	Facial/hand gestures/body postures E.R.
[64]	PING	HE/Digital Logic Design	Facial/mouse tracking, interactive feedback
[65]	VR-based ATS	ASD Adolescents/STEM	VR em. recog. speech and text analysis
[66]	MaTHiSiS	Ages 6–18, Special Education	Multimodal affect em. recognition

.

Table 4. Multimodal ATSs: detected affects, system adjustments, and outcomes.

ATS	Detected Affect	Adjusted Behaviors	Adjustment Method	Study Type	Participants	Control Condition	Measures	Results
ALS [53]	Engagement, frustration, boredom	Tailored content	Multimodal data, ML	Within-subjects/quant.	Eighty-five, ages 6–18	Achievement-based content	Emotional states, achievement	Increased engagement, reduced boredom, no sig. achievement
EAITS [54]	Language anxiety	Adaptive feedback	ML (random forest)	Quasi-experimental/quant.	Eighty (adaptive: 20, ctrl: 60)	Fixed feedback	Anxiety reduction, learning gain	Reduced anxiety, improved learning gains
Unnamed [55]	Negative, positive emotions	Emotional feedback	Semantic emotion recognition	Quasi-experimental/mixed	Thirty (exp: 15, ctrl: 15), ages 20–30	Traditional tutoring	Usability, eye movement	Higher satisfaction, longer fixation for exp. group
ATTENDEE [56]	Arousal, valence, head pose	Supplementary feedback	Real-time analysis	Quasi-experimental/mixed	Fourteen (7 test, 7 control), ages 19–28	No ATS feedback	Test scores, attempts, time, satisfaction	Fewer attempts to pass, high satisfaction, no significant mean score difference
SimInClass [57]	Happiness, sadness, neutral, etc.	Affective recommendations	Teaching performance report	Within-subjects/mixed	Fifteen prospective teachers, ages 20–22	No recommendations	Emotional state frequencies	Reduced disgust, changes in happiness, sadness, neutral states
Unnamed [58]	Emotions (facial, semantic)	Feedback from affective agents	Adaptive feedback, presentation	Quasi-experimental/quant.	Seventy (exp: 35, ctrl: 35)	No adaptive feedback	Learning achievement, AEQ, anxiety, etc.	Improved learning effectiveness, lower anxiety, higher engagement, positive attitude
MetaTutor [59,60,61,62]	Negative activating, deactivating emotions	Interaction with avatars	Microsoft Emotion API, eye tracking	Within-subjects/quant.	Forty (mean age: 23.58; SD: 8.18; 17 m., 23 f.)	No specific control condition	Learning strategies, emotions, fixations	Best results with REPTree algorithm, 87.50% accuracy, 0.88 AUC
		Prompts for SRL strategies	Pedagogical agents	Quasi-experimental/quant.	One hundred and seventy-four (mean age: 20.43; 80 exp./94 control)	No SRL prompts, self-initiated only	Performance, emotions, cognitive strategies	Stability of negative emotions -> performance, changes in negative emotions -> strategies
		Adaptive feedback	Eye tracking, interaction data	Within-subjects/quant.	Seventy-nine, ages 18–25	No intervention control group	Emotion prediction accuracy, learning gains	Co-occurrence of boredom and frustration, improved detection accuracy
		Prompts, feedback	Adaptive rules	Quasi-experimental/mixed	One hundred and nineteen (59 exp., 60 ctrl.; m. age: 23.35; 65.5% f.; 9 with LD)	No adaptive feedback	SRL strategy use, pre/post assessments	More SRL strategies in exp. group, significant in agent-initiated strategies
InIv ATS [63]	Attentive, in-attentive, neutral	Immediate feedback	Feature fusion, Atkinson’s index, etc.	Quasi-experimental/mixed	Three hundred and fifty, ages 20–26, 30–130 per class	Non-inquiry-based, non-intervention	Affective states, engagement, marks, accuracy	A 43–53% reduction in in-attentive, 0.77 accuracy, 4% error rate
PING [64]	Emotions	Personalized support	Random-forest feedback loop	Within-subjects/mixed	Twenty-six (7 initial, 19 pilot)	No specific control conditions	Classif. accuracy, performance	Improved accuracy from 88.57% to 91.73%
VR-based ATS [65]	Cognitive, emotional states	Adaptive learning support	NLP, ML	Within-subjects/mixed	Four adolescents with ASD, ages 13–19	Baseline sessions	Frequency of behaviors, cognitive and emotional states	Positive correlation between cognitive and emotional states, challenges with negative states
MaTHiSiS [66]	Engagement, frustration, boredom	Adjusts learning material	Multimodal affect recognition, ML	Within-subjects/quant.	Sixty-seven (ages 6–18, disabilities, autism)	Achievement-based content	Engagement, frustration, boredom, learning	Higher engagement, lower boredom in intervention, no significant difference in achievement

elaborates on the aspects of the affects that the selected studies respond to, the system feature behaviors that are adjusted, and the methods of adjustment.

Ref. [53] proposes an ATS with a multimodal interface that adapts educational content based on emotional states, such as engagement, frustration, and boredom, to optimize learning. Applicable in various educational areas, from navigation to mathematics, the system includes voice recognition, although without specifying conversational capability. It adjusts system behaviors through the personalization of educational content, modifying the difficulty, type of activities, and presentation of material based on the detected emotional states to keep students in an optimal flow state. In the control condition, only achievement data were used to select content. The participants were 85 students aged 6 to 18, and the dependent measures included emotional states and learning progress. Personalization according to emotional state encourages engagement and reduces boredom but does not show a significant impact on academic performance. The study does not detail specific user perceptions nor address ethical considerations. Future research could focus on improving machine learning techniques and diversifying content.

Ref. [54] proposes an emotionally adaptive intelligent tutoring system (EITS) that uses a multimodal interface with text, voice, and an animated agent to provide feedback, manage foreign language anxiety (FLA), and enhance educational performance. It utilizes a machine learning model based on the scikit-learn library, with a random forest chain regression algorithm to detect and predict FLA levels. It adjusts motivational and explanatory feedback behaviors based on the detected anxiety levels. The control condition includes fixed feedback strategies. The dependent measures are anxiety reduction and learning gains. The study involved 80 participants (20 adaptive, 60 control) with varied ages, English levels, and backgrounds. Adaptive feedback has proven to be more effective in reducing FLA compared to fixed methods and has also improved learning gains. Furthermore, the presence of emotional support agents was beneficial, regardless of the system’s adaptability. However, the study does not address the perceptions of students and teachers about the system, nor does it discuss ethical or privacy considerations.

Ref. [55] developed an ATS in the field of digital art, featuring an interface that includes an intelligent virtual agent and adaptive course modules to enhance student interaction and engagement. The affective tutoring system detects negative and positive emotions, adjusting its behavior through semantic recognition and emotional feedback. The system provides emotional feedback and guidance, dynamically adjusting the virtual agent’s responses based on the emotions detected in the user’s input text. The control group used a traditional tutoring system. The dependent measures included a usability scale and eye movement analysis. The system is highly regarded by students for its ability to adapt to their emotional states, thereby enhancing their engagement and understanding of digital art. The success of the ATS relies on its precise emotional recognition and adaptability to individual learning styles. It does not address ethical or privacy considerations. Future projections for the ATS involve integrating more advanced AI capabilities including a chatbot, to enrich learning experiences with more immersive and personalized interactions.

ATTENDEE [56] uses a web interface that personalizes educational content through facial emotion recognition and head pose estimation, adapting the experience to students’ cognitive styles. While it does not explicitly integrate NLP, or conversational agents, it has been implemented in artificial intelligence courses, enhancing academic performance and student satisfaction by adjusting the feedback and supplementary content based on the student’s emotional state. It reacts to arousal, valence, and head pose, adjusting the content and feedback based on real-time emotion and behavior analysis. It customizes learning by providing supplementary material and personalized feedback when learners show signs of confusion or disengagement. In the control condition, participants watched videos without intelligent analyzer feedback. The dependent measures included test scores, number of attempts to pass, time spent watching content, and educational satisfaction. Student and teacher perceptions are positive, highlighting the customization of learning. It emphasizes that its success depends on the accuracy of emotion recognition, while maintaining ethical standards in data management. ATTENDEE promises to expand and improve in regard to the field of adaptive educational technologies.

SimInClass [57] is a virtual classroom interface designed to train future teachers. Focused on the development of classroom management skills, this system utilizes an affective recommendation system to enhance users’ emotional regulation. The system reacts to emotions such as happiness, sadness, neutrality, disgust, anger, surprise, and fear, adjusting its behavior through affective recommendations based on teaching performance. In the control condition, the participants did not receive affective recommendations in the first session. The dependent measures included the frequency of emotional states before and after the recommendations. The study involved a single-group design, where recommendations were applied in the second session only. The system adjusts its feature behavior by providing detailed reports on detected emotions and their contexts after each teaching session. Users demonstrated changes in their affective adaptation upon receiving emotional feedback. Sensitive to ethical considerations, future work highlights the importance of accounting for individual cognitive differences and the use of a control group to better assess the effects of the affective recommendation system.

Ref. [58] describes an affective tutoring system that uses a web-based interface, employing facial and semantic emotional recognition technologies to tailor feedback to the student’s emotions through affective agents, contributing to improved learning effectiveness, increasing engagement, reducing anxiety, promoting self-directed learning, and enhancing outcomes. It dynamically modifies content with interactive games or videos to counteract negative emotions. The control group did not receive adaptive feedback, and the dependent measures included learning achievement, AEQ, anxiety, engagement, attitude, and self-directed learning. The system is primarily applied in online educational contexts in the fields of technology and art. However, it does not detail the specific perceptions of students or teachers, nor does it discuss factors of success or failure, ethical considerations, or future projections in terms of the system.

MetaTutor [67] is an ATS that boasts over 10 years of research and uses a 2D hypermedia interface designed to support students’ self-regulated learning. Specifically, a learning unit on the human circulatory system has been developed. The interface incorporates virtual pedagogical agents, which [62] enhanced with 3D versions in a non-immersive environment to encourage student engagement. This system provides adaptive feedback and suggestions based on the continuous monitoring of students’ cognitive, affective, metacognitive, and motivational processes, promoting self-regulated learning, aiding in the understanding of complex content, and improving learning effectiveness. It adjusts behaviors, such as avatar interaction, content focus, and time management, using the Microsoft Emotion API and eye-tracking data. The system personalizes learning by responding to detected emotions with appropriate feedback and prompts [59].

Ref. [62] found that MetaTutor promoted more self-regulation strategies among students who used it compared to those who did not. This suggests a positive impact on their academic performance and learning experience, including those with learning difficulties, although the studies analyzed do not explicitly mention this aspect. On the other hand, Ref. [60] measured the performance, emotions across six time points, and time spent on cognitive strategies after detecting negative (de-)activating emotions and adjusting its behaviors by prompting learners to use self-regulated learning (SRL) strategies through pedagogical agents, indicating that the stability of activating negative emotions over time is negatively related to performance, suggesting a significant impact of negative emotional dynamics on learning. Random forest classification showed that previous scores, changes in deactivating negative emotions, and time spent on cognitive strategies are key predictors of performance.

Ref. [61] investigates the presence and prediction in terms of coexisting emotions during learning with MetaTutor, utilizing eye-tracking and interaction data. The system personalized the support provided, such as shifting activities to reduce boredom or offering encouragement to alleviate anxiety. This work broadens the evidence that emotions can coexist in educational settings and introduces a novel approach to predicting these coexistences. They found that boredom combined with frustration and curiosity combined with anxiety frequently coexist in terms of students’ engagement with MetaTutor. These emotional coexistences were predicted with significantly greater accuracy than a standard baseline. Key factors determining success or failure include the system’s capacity to accurately recognize and respond to students’ emotional and cognitive states, as well as the impact of these responses on their motivation and engagement.

These findings have important implications for the design and construction of affective tutoring systems (ATSs) that can adapt to complex and coexisting affective states. The future of MetaTutor involves optimizing scaffolding for self-regulated learning through personalized feedback and adaptive structures; integrating real-time analysis of multimodal data for more effective interventions; researching metacognitive strategies and their systemic support; developing capabilities to recognize and act on students’ emotions and motivations; adapting to individual differences to maximize learning; ensuring validity in real and long-term contexts; incorporation into classroom practices; and building comprehensive theoretical models that unite cognitive sciences, educational psychology, and computing [67].

The InIv ATS [63] incorporates an interface that analyzes students’ affective states in real-time through computer vision technologies that evaluate facial expressions, hand gestures, and body postures. This system is applied in various educational settings, such as e-learning, flipped classrooms, traditional classrooms, and webinars. The system detects attentive, in-attentive, and neutral affective states, adjusting behaviors through immediate feedback, conceptual questions, and tasks using feature fusion, Atkinson’s index, CNN, and decision trees. It adapts in real-time by classifying affective states and triggering inquiry interventions, when needed. These interventions are designed to improve interactions and personalize the educational process, facilitating deeper and more autonomous learning by adapting teaching strategies to the emotional and cognitive needs of the student, thus enhancing their engagement and academic performance.

PING [64] is an ATS implemented through a narrative game that uses a random forest model to adapt teaching to a digital logic design. It reacts to emotions, adjusting personalized learning support through a random forest-based feedback loop, which tailors the instructions based on real-time student performance and affective states. Without a specific control condition, the study measured classification accuracy and student performance. This system not only adjusts the difficulty and assistance in real-time based on the detected emotions and behaviors of the student, but also improves with the collection of real data, which is demonstrated in the alignment between the game ratings and the academic grades of the students. It does not include explicit ethical and privacy considerations. Future work will focus on expanding comparative studies and retraining the system using new data to optimize its effectiveness.

Ref. [65] describes an ATS with a VR interface designed to enhance representational flexibility in students with autism spectrum disorder (ASD), applicable in STEM education. This system uses natural language processing (NLP) and machine learning techniques to assess cognitive and emotional states, providing adaptive support based on student engagement and needs. For example, it promptly provides in situ feedback based on real-time speech data analysis. The control condition included baseline sessions, and the dependent measures were the frequency of behaviors and correlations between cognitive and emotional states. Although its direct impact on academic performance is not detailed, it is suggested that its focus on adaptive assessment and feedback could significantly enrich the learning experience. The ATS also raises important ethical considerations regarding data privacy, highlighting the need to adhere to human subject protection policies, and presents a promising future for expanding its accuracy and applicability in various educational settings.

Ref. [66] describes MaTHiSiS, an ATS with a multimodal affective recognition interface that uses sensor data and machine learning to detect states like engagement, frustration, and boredom. It increases the challenge to reduce boredom and decreases it to alleviate frustration. Designed for special education, MaTHiSiS lacks a dialogue system with NLP or conversational agents, but optimizes the educational content to enhance learning and affective states. While it increases engagement and decreases boredom, no significant academic performance improvements were noted. Future improvements will focus on refining the machine learning methods and expanding the learning materials. Ethical approval was granted by the University of Nottingham, and while it does not detail user perceptions or critical success factors, it marks progress toward an adaptive system that supports teachers in aiding students with intellectual disabilities.

3.2. Unimodal Affect Recognition Systems

This subsection presents the ATSs with emotional identification systems that integrate just one data source or strategy. A total of 15 ATSs, employed across 15 studies, satisfy this criterion. Their main characteristics are summarized in

Table 5. Unimodal affective tutoring systems.

Citation	ITS Name	Educational Level/Area	Technical Functionalities
[68]	EMITS-F	HE B.Tech/Programming	PANAS: self-evaluation
[69]	ILOs ALS	Undergraduate/Multimedia	Facial em. recog.
[70]	MathSpring-COVES	Middle school/Mathematics	Visual affective analysis, log data integration
[71]	KNN ATS	Undergraduate/Computer Science	EEG em. recog., KNN, affective content personalization
[72]	SeisTutor	HE/Seismic Data Int.	Facial emo. recog., adaptive response
[73]	NAO Robot	Kids 11–14/Eco-awareness	DeepFace for basic emotion recognition
[74]	AMLTS	HE/Japanese Language	Emojis/text analysis, adaptive response
[75]	GEA2: A New Earth	High School/STEM	Semantic analysis, emotional FB
[76]	AELTS	HE/Interactive Design	Semantic analysis, emotional FB
[77]	IESAC	ASD children/Basic Skills	Facial E.R., AR for interactive learning
[78]	ACALS	Elementary/Mathematics	Response time analysis, adaptive response
[79]	Personalized Brain-Based Quiz Game	Undergraduate/Computer Science	Semantic analysis, adaptive response
[80]	EBALS	HE/Programming	Facial em recog., personalized content
[81]	Hypocrates+	HE/Medical Education	EEG-based emotion analysis, real-time help
[82]	EasyLogic	HE/Programming	Facial em. recog., adaptive response

.

Table 6. Unimodal ATSs: detected affects, system adjustments, and outcomes.

ATS	Detected Affect	Adjusted Behaviors	Adjustment Method	Study Type	Participants	Control Condition	Measures	Results
EMITS-F [68]	Emotional dimension (pos. aspect/neg. aspect)	Motivational quotes, games, personalized tutoring	Fuzzy inference system	Within-subjects/quant.	Ten slow learners (specific selection)	N/A	Pre-assessment and post-assessment scores	Six learners showed significant improvement, system accuracy of 90%
ILOs ALS [69]	Emotions, engagement	Learning materials, quizzes, features	Machine learning, fuzzy C-means, gamification	Within-subjects/mixed	Three hundred and ten university students	Traditional, non-gamified	Engagement, quiz scores, completion rates	Highest engagement in adaptive gamified system, best grades for visual learners
MathSpring-COVES [70]	Facial expressions, gestures	Personalized problem difficulty selection	Fusion model of visual and non-visual data	Quasi-experimental/quant.	Twenty-two 7th grade students (13 exp., 9 control)	Legacy algorithm (EBT)	Test scores, time, attempts, mastery level	Treatment group improved scores and engagement
KNN ATS [71]	Emotional states (EEG)	Personalized content delivery (videos, text/audio)	KNN, reinforcement learning	Quasi-experimental/mixed	Thirty students (24 m., 6 f., age 21–24) 15 exp./15 control	Standard web-based English lecture	Learning (pre/post-tests), engagement, satisfaction	No significant impact on learning and engagement, increased satisfaction
SeisTutor [72]	Facial expressions	Personalized curriculum, state tracking	Pre-test results, learning style, emotions	Quasi-experimental/mixed	Sixty learners (32 exp./28 control)	Standard curriculum, no tracking	Learning, concept underst., emotions	A 22% gain with customization, 12% without
NAO Robot [73]	Facial DeepFace basic emotions	Tailored feedback	DeepFace for emotion recognition	Quasi-experimental/mixed	Fifty (28 m, 22 f), ages 11–14. 25 exp./25 control	Web app without affective support	Enjoyment, environmental awareness	NAO: higher enjoyment and awareness vs. control
AMLTS [74]	Emotions (positive, negative, neutral)	Feedback from virtual agents	Emotion recognition, feedback	Within-subjects/mixed	Sixty-seven (prot.), 63 (formal)	N/A	Pre-test and post-test, emotion analysis, usability	Learning gains, high usability, increased engagement/pos. emotions
GEA2: A New Earth [75]	Emotions (positive, negative, neutral)	Unsolicited hints, autonomous interventions	Bayesian network, sentiment analysis	Within-subjects (pilot/exp.)/mixed	Twenty high school students (age 15)	N/A	Engagement, game progress, emotion	Increased engagement and trust, positive feedback
AELTS [76]	Positive and negative emotions	Emotional feedback via agent	Affective computing, semantic recognition	Within-subjects/mixed	Sixty-two HE students (33 m., 29 f.)	N/A	SUS, QUIS, emotional data	High usability and satisfaction, feedback turned 19% negative to positive
IESAC [77]	Facial expressions	Displayed objects and audio	ML model, facial emotion detection	Within-subjects/mixed	Six (4 m., 5–9; 2 f., 4–8), autism diag. (CARS): 27–37	N/A	Interaction, comfort time, responses	Four of 6 rapidly comfortable, improved learning/engagement/interaction
ACALS [78]	A: affective and cognitive, B: cognitive, C: none	A: based on affect. and cogn., B: based on cogn., C: standard	A: fuz. inf. (affect. and cogn.), B: fuz. inf. (cogn.), C: none	Quasi-experimental/mixed	A: 53 (26 M, 27 F, age 10); B: 53 (26 M, 27 F, age 10); C: 56 (26 M, 30 F, age 10)	Conventional system	Learning achievement, anxiety, cognitive load	A: higher achiev., lower anx.; B: moderate achiev.; C: moderate achiev., higher anx.
Brain-Based Quiz Game [79]	Basic emotions	Quiz content, difficulty, hints, time	Marzano taxonomy, student traits	Quasi-experimental/mixed	One hundred CS undergrads (50 exp./50 control)	Standard e-assessment	Pre/post-test scores on cognition levels	Greater improvement in cognition for experimental group
EBALS [80]	Basic emotions	Personalized content, adaptive timetable	Neural net. facial emotion detection	Within-subjects/quant.	Seventeen students (no programming background)	Non-adaptive learning	Emotion detection accuracy, test performance	Improved emotion detection, higher adaptive test scores
Hypocrates+ [81]	Frustration	Real-time help information	EEG emotion analysis	Within-subjects/quant.	Five medical students (4 females)	Not mentioned	Mean frustration before/after help, performance	Frustration reduced from 0.53 to 0.50, performance range 72.77–55%
EasyLogic [82]	Engagement, frustration, excitement, boredom	Motivational/informative interventions, gamification	ML facial emotion recognition, gamification	Quasi-experimental/mixed	Forty-two students (21 exp., 21 control), ages 18–20	Traditional learning	Pre-test/post-test scores, time spent on exercises	Improved post-test scores with system use

elaborates on the affective aspects that the selected studies respond to, the system feature behaviors that are adjusted, and the methods of adjustment.

EMITS-F [68] is an ATS that uses a user interface designed in Python to facilitate personalized interactions between the learner and the educational agent, Rachel. Specifically supporting slow learners in programming concepts, it integrates NLP and conversational agents to enhance interaction and emotional connection. The EMITS-F system reacts to affective aspects by measuring the students’ levels of well-being (PA) or discomfort (NA). It adjusts its behavior by providing motivational quotes, games, and personalized tutoring through the agent. This adjustment is performed using a fuzzy inference system (FIS) that evaluates the performance in pre- and post-intervention tests. There is no explicit control condition mentioned, and the dependent measure is the test scores. The system has been shown to significantly improve academic performance. Although the article does not specify the perceptions of students and teachers about the ATS nor discusses ethical and privacy considerations, it highlights the system’s effectiveness in improving knowledge retention and understanding in slow learners.

The ATS described by [69], uses a facial emotional recognition interface to adapt educational content to students’ intended learning outcomes (ILOs), applied in an operations research course. This system improved student grades and student satisfaction compared to traditional methods. The ATS adjusts the materials and assessments according to visual, kinesthetic, and auditory learning styles, enhancing interaction and educational effectiveness. The learning materials and quizzes are adapted using machine learning and fuzzy C-means techniques, personalizing the student experience based on the detected learning styles. For future improvements, integrating facial emotion tracking during the study and comparing student performance in online and in-class environments is being considered, seeking to perfect the system’s adaptability and accuracy.

Ref. [70] describes MathSpring-COVES, an ATS that uses facial recognition to adjust the difficulty of math problems in real-time and enhance student engagement, although it does not integrate NLP or conversational agents. It adjusts the difficulty level of math problems in real-time through a deep fusion model of visual and non-visual data. The control condition used the legacy effort-based tutoring (EBT) algorithm, while the dependent measures included the test scores, time spent, number of attempts, and mastery level. The study involved 22 seventh-grade students, all from low-income families. Designed for middle school level math education, it enhances personalization through facial analysis, but does not explicitly discuss user perceptions, success or failure factors. This system explicitly considers ethical and privacy considerations. The results indicate that MathSpring-COVES significantly improves academic performance and the learning experience by keeping students within their zone of proximal development and increasing their mastery of math concepts.

Ref. [71] describes an affective tutoring system (ATS) that uses a web interface and an electroencephalogram (EEG) device to interpret emotions through the KNN machine learning algorithm. It reacts to the emotional states detected, adjusting the content delivery by providing videos for negative emotions and additional materials for positive emotions, using KNN classification and reinforcement learning. The control condition involved a standard web-based English lecture, with learning, engagement, and satisfaction as the dependent measures. This system adapts the educational content to emotional states, focusing on English learning, especially through reading and listening, although it does not mention the integration of NLP or conversational agents. While it increased student satisfaction, it did not significantly impact learning or participation. The data are ethically handled, collected anonymously, and with ethical approval. The future of the system could be to improve emotional detection and expand the educational content using more physiological data.

The SeisTutor system [72] utilizes a 2D adaptive interface for seismic data interpretation, with affective identification through facial expression recognition using CNNs. SeisTutor employs scaffolding techniques to support learning, providing affective formative feedback. It adjusts the personalized curriculum and state tracking based on the pre-test results, learning style, and the detected emotions. The system customizes the learning paths and content according to the learner’s psychological state and performance. The dependent measures included learning gain, concept understanding, and emotions. The study involved 60 participants: 30% undergraduates, 17% graduates, 35% postgraduates, and the remainder were Ph.D. candidates. While lacking a dialogue system with NLP or a virtual pedagogical agent, the article emphasizes future improvements in terms of tutoring strategies and adaptive content design. Positive student perceptions are noted, with success factors including the effective personalization of learning and adaptation to student emotional states. The document points to future areas of improvement in terms of tutoring strategies, adaptive content design, and the identification of student psychological states, but lacks specific discussions of the ethical and privacy considerations.

Ref. [73] uses the interface of the humanoid robot NAO, along with the DeepFace facial recognition system, to adjust the interactions in real-time based on the detected emotions in educational contexts. The control group used a web app without emotional support. The dependent measures were user enjoyment and environmental awareness. This approach enhances the learning experience. With limited natural language processing (NLP), the system offers functionalities, such as reviewing responses based on negative emotions to promote personalized learning, thereby improving student engagement and performance. Users show positive reactions, particularly valuing the personalization and engagement generated. Although the system does not explicitly address ethical or privacy considerations, it proposes future improvements in terms of emotional recognition and natural interaction.

AMLTS [74] is a mobile application for learning Japanese, integrating affective computing and asynchronous discussion forums. It employs emotion recognition to personalize the feedback and assessments via agents attuned to students’ emotional states, improving interaction and learning outcomes. This feedback includes motivational messages and learning suggestions. The dependent measures included learning outcomes, emotional event analysis, and system usability. Despite high praise for its usability and personalization, user suggestions include improving dynamic interactions and technical elements like download speed and audio options. Future plans involve advancing techniques, such as recognizing complex emotions, and employing gamification to enhance engagement and learning effectiveness. Ethical and privacy considerations remain unaddressed.

Gea2 [75] is an ATS in the form of an educational game designed to teach STEM subjects in high school. Its interface includes intelligent pedagogical agents (IPAs) capable of interacting in natural language and providing unsolicited hints and autonomous interventions based on the player’s emotional and progress state using a Bayesian network, sentiment analysis, and emoticon use. The ATS adapts the learning tasks to the personal traits and current state of the learner, potentially improving both academic performance and the learning experience through individualized interventions. The article notes that both students and teachers found the IPAs to be engaging and useful, suggesting positive perceptions of the system. Although the system integrates NLP to facilitate more natural interactions, it does not specifically address the success or failure factors, ethical and privacy considerations, or future projections in terms of the system.

The ATS proposed by [76] features a three-column web interface for mobile devices, integrating technology to recognize the emotions expressed textually by users. It employs emotional agents that respond graphically to influence the user’s emotional state. The system maintains positive emotions and converts negative ones by recognizing user input and responding with supportive messages. The dependent measures included the system usability scale (SUS) and the questionnaire for user interaction satisfaction (QUIS), showing high usability and satisfaction, although there is no evidence of improved learning, and it does not address ethical and privacy considerations. Future research will focus on optimizing the user experience.

The Intelligent Educational System for Autistic Children (IESAC) [77] is an ATS designed for children with autism spectrum disorder that leverages an augmented reality interface and a Kinect 3D camera to provide an interactive and personalized educational experience. The system reacts to emotions like happiness, sadness, and fear by adjusting the displayed objects and audio. It uses a machine learning model to recognize facial expressions and tailor the responses, projecting objects of interest and repeating them based on the child’s emotional state. The dependent measures included interaction, comfort time, and verbal responses. Although it does not integrate NLP or conversational agents, it enhances engagement by adapting the content to facially recognized emotions. The positive impact on learning and the student experience is notable, with feedback reflecting the enthusiasm of students and teachers. The system’s success lies in its ability to engage children through personalization based on the emotion detection, without detailing specific barriers, ethical considerations, or future lines of work.

ACALS [78] is an ATS featuring a web interface developed in HTML5, CSS3, JavaScript, and MySQL, tailored to students’ cognitive and affective states. Deployed in fifth-grade mathematics, it is compared with conventional and other adaptive systems in terms of its effectiveness. The system reacts to affective factors such as concentration, patience, and learning willingness, adjusting the level of detail in educational materials through fuzzy inference based on affective and cognitive analysis. System behaviors are adjusted by providing standard, detailed, or advanced materials, according to student performance. The dependent measures included academic achievement, math anxiety, and cognitive load. While it does not mention NPL or conversational agents, it evaluates aspects such as concentration and patience, improving academic performance and reducing mathematics anxiety. The study advocates expanding this approach to higher educational levels and considering personal factors to optimize adaptive learning environments, ensuring participants’ personal data protection.

Ref. [79] presents a Personalized Brain-Based Quiz Game, an ATS with a quiz game interface, applied in computer science education and based on Marzano’s taxonomy [83]. It adjusts the quiz content, difficulty level, hints, and time based on Marzano’s taxonomy and the student’s characteristics. Specifically, the system customizes quiz items to match the student’s knowledge level, emotional state, and learning goals. It does not use NLP or conversational agents, but it adapts the assessments and feedback based on student emotions, enhancing learning outcomes and higher cognitive functions compared to traditional methods. The document does not address user perceptions, success or failure factors, ethical and privacy considerations, nor future projections of the ATS.

The affective tutoring system (ATS) by [80] advances adaptive learning by merging a web-adaptable 2D interface, emotional recognition, prior knowledge evaluations, and neural networks to tailor educational content and schedules, highlighted in a C# programming case study. The system adjusts personalized content and study timetables using neural network predictions, based on user performance and emotions. The dependent measures were emotion detection accuracy and test performance. The study included 17 students with no programming background, tested in both adaptive and non-adaptive scenarios. It does not address the use of natural language processing (NLP) or feedback from students/teachers, but suggests enhancing academic performance by adapting to students’ emotions. Ethical and privacy considerations in emotional data handling are emphasized as crucial for future acceptance and development. Anticipated upgrades include integrating gesture recognition and NLP for a more comprehensive learning experience.

Hypocrates+ [81] employs a virtual reality interface in medical education to simulate clinical environments safely. The Hypocrates+ system detects frustration through EEG analysis and adjusts its behavior by providing real-time help information. This adjustment aims to reduce the user’s frustration and improve performance. The study measured the mean frustration before and after help, along with the performance scores. Although it does not integrate NLP or conversational agents, a neural agent assesses the emotional state and manages the necessary assistance. The experiments suggest it improves emotional stability and reduces frustration, enhancing performance and learning experience. However, the article lacks details on students’ and teachers’ perceptions, as well as ethical and privacy considerations regarding EEG data collection.

EasyLogic [82] is an ATS with an advanced interface that uses facial emotional recognition and gamification to teach computer programming. The system reacts to engagement, frustration, excitement, and boredom by adjusting the motivational and informative interventions and incorporating gamification elements. Specifically, it uses machine learning and emotion recognition to tailor feedback and support. The control condition involved traditional classroom learning. The dependent measures included pre-test and post-test scores, and the time needed to solve the exercises. Although it does not use NLP or conversational agents, it automatically adjusts the support provided based on emotional responses and has been proven to improve the academic performance of its users. Student feedback is generally positive, emphasizing enjoyment and motivation, although some question its efficacy in regard to boosting academic results. The participants were informed about the details of the study and signed a consent form. The future plans aim to expand the exercises and adapt the content to a broader educational level and age range.

3.3. ATS Interface

As Figure 3 shows, from the collected sample, a large majority of ATSs feature a 2D interface design. Those that incorporate three-dimensional interfaces are limited to non-immersive experiences. Only one of the proposals uses a robot as a key element in the interaction, affective feedback, and emotional recognition of the system.

3.4. ATS Knowledge Domain and Educational Level

A high percentage of the identified ATSs are dedicated to teaching content specific to the STEM area (science, technology, engineering, and mathematics), with language learning being the other predominant field of knowledge. Moreover, a majority of ATSs are intended for students in higher education. Figure 4 and Figure 5 detail the specifics in terms of the findings.

3.5. ATS Affect Recognition Features

Figure 6 indicates that emotional identification is primarily carried out through the analysis of participants’ facial expressions, followed by text analysis. Additionally, approximately 60% of the identified ATSs opt for emotion classification based on the theory of basic emotions [23], as opposed to other valence and arousal options [84].

3.6. ATS Pedagogical Features

Most systems include audiovisual material and a menu with different options. Only about 18% perform simulations, and there is no reported incorporation of generative AI beyond text in systems with advanced chat features. On the other hand, more than 70% of adaptive systems use scaffolding as an instructional strategy, with 37% opting for game-based learning, while only about 7% utilize Socratic questioning. The formative feedback they provide is predominantly affective in nature, as Figure 7 depicts.

3.7. ATS Dialogue System

Figure 8 shows that only about a quarter of the systems declare capabilities for processing or understanding natural language. However, approximately half include avatars to assist in communicating with and guiding students. Of these, nearly half serve as tutors, who guide, explain, and set guidelines. In almost the remaining half, they take on the role of a motivator, providing emotional feedback. Only one of the studies features an avatar that acts as a tutee of the student, reversing the roles to increase motivation and responsibility. None of the cases opt for a pedagogical agent that participates as if it were a peer.

3.8. Evidence

Approximately 40% of the studies show evidence that the affective tutoring developed by the system impacts student assessments. About 22.2% of the research compares the effectiveness of affective tutoring by the ATS against conventional tutoring, and only 10% investigate how these systems promote self-regulated learning. On the other hand, nearly 65% of the studies identify an effect of using these systems on learning outcomes, while about 55% do so regarding the student experience. While approximately a third of the studies gather student perceptions about the system, only 19.4% of the documents collect teachers’ perspectives in this regard. Figure 9 outlines these particular findings.

3.9. Succes or Failure Factors

From the analysis of the documents, 10 key factors emerge as either facilitators or barriers for adaptive affective intelligent tutors. The facilitators include technological and pedagogical support, real-time feedback, adaptability, emotion detection, integration and use of diverse data sources, and content personalization. The barriers consist of technological and integration complexity, design and technical limitations, emotion detection challenges, implementation difficulties, and data collection limits.

Table 7. Success or failure factors.

	ATS	Study	Facilitators					Barriers
			Technological and Pedagogical Support	Real-Time Feedback and Adaptability	Emotion Detection and Response	Integration and Use of Diverse Data Sources	Content Personalization and Suitability	Technological and Integration Complexity	Design Issues and Technical Limitations	Challenges in Emotion Detection and Response	Implementation and Adaptation Difficulties	Data Collection Limitations and Small Sample Size
Multimodal ATS	1	[53]	X	X	X	X	X	X	X	X	X	X
	2	[54]	X	X	X			X	X	X
	3	[55]	X	X	X	X	X	X	X	X	X	X
	4	[56]	X	X	X		X	X		X	X
	5	[57]	X	X	X	X	X	X		X
	6	[58]	X	X	X	X	X	X	X	X	X
	7	[67]	X	X	X	X	X	X	X	X	X	X
		[59]	X		X	X		X		X		X
		[60]	X	X	X	X	X			X	X	X
		[61]	X	X	X	X	X	X		X	X	X
		[62]	X		X	X		X	X	X	X
	8	[63]	X	X			X	X	X			X
	9	[64]	X	X	X	X	X		X		X
	10	[65]		X	X	X	X	X	X			X
	11	[66]	X	X	X	X	X		X	X		X
Unimodal ATS	12	[68]	X	X	X	X	X	X	X	X	X
	13	[69]	X	X	X	X	X	X	X	X		X
	14	[70]	X	X	X	X	X	X	X		X
	15	[71]	X	X	X	X	X			X		X
	16	[72]	X	X	X	X	X	X	X	X	X	X
	17	[73]	X	X	X		X	X		X	X
	18	[74]	X	X	X		X	X	X	X	X	X
	19	[75]	X	X	X	X	X
	20	[76]	X	X	X			X			X
	21	[77]	X	X	X	X	X	X	X	X	X	X
	22	[78]	X	X	X	X	X	X		X	X	X
	23	[79]	X	X	X	X	X
	24	[80]	X	X	X	X	X		X		X
	25	[81]	X	X	X	X	X	X
	26	[82]	X	X	X	X	X

compiles the studies analyzed, marking the dimensions each document explicitly considers, although these factors are potentially common to all of them.

3.10. Perceptions On the Educational Use of ATSs

Table 8. Perceptions on the educational use of ATSs.

Category	Studies	Student Perceptions	Teacher Perceptions
Engagement	[57,58,63,64,71,72,73,74,75,76,77,78,79,82]	Increased engagement, reduced boredom; improved emotional awareness; enhanced learning effectiveness, engagement, reduced anxiety; support for learning strategies; generally positive with some frustration; requests for varied support; increased engagement and satisfaction; effective and adaptable systems; engaging learning; positive usability and motivation; high usability and emotion handling; engagement and happiness in autistic children; positive impact on motivation and interest; enhanced performance and engagement; mixed on learning effectiveness	Positive perception of SimInClass for improving emotional awareness
Reduced anxiety	[58]	Reduced anxiety; moderate learning gains; improved learning effectiveness
Usability and satisfaction	[55,76]	High usability and satisfaction; effective emotion handling
Frustration, distraction	[56,63,82]	Generally positive but some frustration with frequent interventions; initial feedback distraction; mixed on learning effectiveness

summarizes the main perceptions reported by students and teachers regarding the use of the ATSs, identifying four different categories.

3.11. Future Work and Implications

Similarly, from the analysis of the future research lines contemplated in the studies reviewed, these have been grouped into six dimensions: technological and methodological improvements; expansion and scalability; enhancement and refinement of emotional and behavioral factors; generalization and applicability; content and educational tools innovation; and interface and learning platform improvements. Figure 10 illustrates the weight of each dimension, in terms of the improvements and future implications described.

4. Discussion

4.1. RQ1. What Type of Interface Do ATSs Use, and What Implications Does This Design Have for the System?

The results indicate that a large majority of ATSs use 2D interfaces, many of which are web-oriented, focusing on optimizing efficiency and accessibility. This finding aligns with the taxonomy proposed by [13], which emphasizes the importance of designing adaptive learning technologies that can cater to both the cognitive and emotional needs of learners. For instance, MetaTutor [67] proposes a perfectly balanced design resulting from a decade of development, aligned with other long-standing ATS research, such as the AutoTutor family [41,42]. While several systems include certain three-dimensional resources for content representation or features, only four of the 26 identified systems opt for purely three-dimensional interfaces, three of them being non-immersive VR [57,65,81] and one incorporating augmented reality [77].

Despite the intense development of mixed reality display technologies over the last decade [85], it continues to be considered an immature emerging technology. Its popularization appears to be concentrated in leisure environments, while its use in educational settings remains very limited. This limitation may be attributed to several factors, including insufficient skills, limited instructional design, a lack of focused attention, time constraints, and limited environmental resources [86]. Economic considerations, especially concerning the latest generation of immersive equipment, can also play a significant role.

In any case, this study focuses on a relatively narrow field: intelligent affective tutoring systems. The fact that more than 15% of the identified designs utilize three-dimensional interfaces suggests that researchers recognize the potential contribution of these technologies to the teaching–learning process. This recognition directly influences pedagogical approaches [87]. Notably, two of the studies featuring three-dimensional interfaces are specifically aimed at teaching students with ASD. Extensive literature explores the use of these interfaces to recreate audiovisual contexts adapted to the visuospatial needs of this demographic and to enhance their learning [88,89].

It is worth highlighting the study conducted by [73], which customizes the tutoring strategy using an NAO robot. The implications and pedagogical processes involved in human–machine interactions when incorporating an affective robot tutor are extensive [90]. This area of research, focusing on individuals with ASD [91], also explores the inclusion of anthropomorphic pedagogical agents and the capabilities of natural language processing (NLP) and natural language understanding (NLU) in these systems.

4.2. RQ2. What Are the Main Educational Levels and Fields of Knowledge Where ATSs Are Applied in Educational Contexts?

The predominance of fields such as science and technology, and higher education, as the areas in which the reviewed systems are allocated, indicates a strong bias in developing empirical research to verify their efficiency and functionality. Following the STEM domain, which comprises more than 90% of the identified ATSs, language learning ranks next, accounting for 18.5%. This aligns with findings by researchers [92], who observe a similar trend when analyzing ITSs, in general, from 2011 to 2022. However, they note that [93] found that studies using ITSs in the humanities and social sciences had a significantly higher weighted mean effect size. It appears evident that, at least in the realm of affective ITSs, there is a need to balance research across different knowledge domains and target student populations.

We identified only four cases where the content or target audience was more comprehensive. The VR-based ATS developed by [65] combines STEM content to enhance flexibility for students with ASD; MaTHiSiS [66] integrates language learning, mathematics, vocabulary, attention, navigation, and social skills sequencing for students with learning disabilities; and the ATS proposed by [53] is also aimed at students with intellectual disabilities, teaches vocabulary, an understanding of cause and effect, attention, language, mathematics, and social skills. MetaTutor [62], the only ATS on which several recent studies have been focuses, shows variability in its target audiences, catering to both students with special needs and those with typical development. Interestingly, all the described proposals are geared towards special education.

4.3. RQ3. What Specific Functionalities Do ATSs Provide to Identify the Emotional State of Students and Support Personalized Learning?

The results obtained show a broad predominance of emotional recognition from facial images and the use of classification frameworks aligned with the theory of basic emotions. More than half of the selected ATSs feature only one mode of emotional identification, and a third of the total employ text analysis for this function. However, while strategies based on deep learning, particularly through convolutional neural networks, offer promising laboratory results, their accuracy in real environments is called into question. Even in controlled online settings, where students must remain in front of their screens, these systems sometimes fail to deliver as expected [94].

The complexity of pedagogical processes demands multimodal emotional identification techniques that also consider affective frameworks adapted to these contexts [25]. In this regard, examples such as MetaTutor [67], the adaptive learning system proposed by [53], and MaTHiSiS [66], implement at least three different sources for their emotional recognition systems. Additionally, they employ activation valences tailored to the learning context. This approach aligns with the control-value theory (CVT) of achievement emotions [95], which is contemplated in the taxonomy by [13].

Additionally, scaffolding is the most commonly used instructional strategy, while game-based learning is employed by just over a third of the assembled ATSs, and Socratic questioning is rarely used. These results are consistent with [6], which notes that the first option is the easiest to implement, the second still presents significant challenges, and the third requires an advanced dialogue system.

On the other hand, the most common types of resources include audiovisual materials and menus of options, aligned with the design preferences previously discussed. However, it is noteworthy that there is no use of generative artificial intelligence, which could significantly enhance these systems’ capabilities in terms of personalized learning [48]. Additionally, as might be expected in such a system, the feedback provided is predominantly affective; however, this can also be considered a form of cognitive adaptive feedback. In any case, combining different types of adaptive feedback, consistent with the design, content, and user interaction, should improve the experience and learning outcomes, although some studies challenge this assertion [96].

4.4. RQ4. To What Extent Do ATSs Integrate NLP and Conversational Agents to Create a Stronger Connection and More Natural and Effective Interaction with Users?

Approximately 20% of the identified systems feature NLP (natural language processing) and NLU (natural language understanding). However, these systems do not appear to integrate the features of large language models with large-scale transformers, which would allow them to maintain conversations with generative, contextually tailored responses that are not predefined. The emergence of these generative conversational models represents a potential transformation in how users interact with ATSs and could lead to the development of a completely new adaptive learning experience [97,98]. Ref. [18] emphasizes the transformative potential of integrating advanced natural language processing and natural language understanding techniques to create more interactive and effective educational systems.

Half of the ATSs reviewed feature virtual pedagogical agents, evenly divided between those acting as tutors and those serving as motivators. The VPAs in MetaTutor and VR-based ATSs can fulfill both roles, while SimInClass is the only system in which the agent acts as a tutee. The studies indicate that the inclusion of these avatars, regardless of their role, positively affects the student experience [99]. Their anthropomorphic appearance, capabilities in terms of interaction and communication in the learning process, the complexity of the associated tasks, and their expressiveness and affective abilities are key factors in the development of empathy toward them [30].

4.5. RQ5. How Does Emotional Recognition in ATSs Impact the Assessment and Feedback Provided to Students?

Several of the systems analyzed focus on adapting content based on the recognition of emotional states, suggesting a positive impact on the efficiency of assessments. This is evident in the observed improvements in grades and the reduction of attempts needed to pass tests [56,59].

While we have mentioned that [96] found no significant differences between the effects of adaptive and non-adaptive feedback, the study by [54] presents a contrasting finding. It identified that adaptive emotional feedback significantly reduces anxiety when learning a foreign language compared to fixed feedback strategies and also moderately improves learning achievements. Furthermore, ref. [62] observed that students who received support from intelligent tutoring systems (ITSs) used more self-regulated learning (SRL) strategies compared to the control group.

Additionally, the study by [57], utilizing a repeated measures design, provided evidence that affective recommendation systems can significantly impact the emotional regulation of future teachers in simulated educational settings, reducing the persistence of low-intensity negative emotions. On a related note, ref. [78] compared a cognitive–emotional adaptive system with a purely cognitive one and a conventional system, among students. The results showed that the system integrating both affective and cognitive aspects not only improved the performance in mathematics but also reduced anxiety, without any differences in cognitive load between the groups.

Furthermore, research by [59,62] demonstrates that systems, such as MetaTutorES and MetaTutor, support SRL by providing scaffolds through animated pedagogical agents. This suggests that affective tutoring feedback can foster deeper and more effective SRL strategies.

These results indicate that affective tutoring can be beneficial in both self-regulation and performance, fostering more effective self-regulated learning strategies. However, it is important to note that, except for studies focused on MetaTutor, most research employs empirical case studies with convenience samples that are relatively small. Therefore, the data obtained should be interpreted with caution.

4.6. RQ6. How Do ATSs Influence Academic Performance and the Learning Experience of Students?

While refs. [53,66,71] acknowledge that significant improvements in performance and learning strategies were not found, other studies such as those by [58,62,67] suggest improvements in the application of didactic strategies and in self-regulated learning. Furthermore, refs. [59,61,63] found a relationship between the identification of emotional states and the prediction of student performance. Additionally, studies by [56,72,80,81,82] identify increases in pass rates, understanding, learning effectiveness, and post-test outcomes.

Also, when considering aspects such as understanding and performance in specific areas, refs. [64,70] detected improvements in mathematical problem-solving. Meanwhile, ref. [65] reported an increase in representational flexibility in adolescents with ASD. Further, refs. [53,54,55,66,69,74] identified increases in student engagement and participation, as well as reductions in boredom and anxiety.

Regarding the influence on student experiences, refs. [71,73,75,77,78] report improvements in satisfaction, engagement, and the personalization of experiences. Meanwhile, refs. [57,79] identified enhancements in emotional awareness and regulation, thereby improving the educational experience.

4.7. RQ7. What Are the Perceptions of Students and Teachers Regarding the Implementation of ATSs?

Several studies report that students have positive perceptions of ATSs, based on their participation in the research. In terms of instructional effectiveness, ref. [67] note that MetaTutor is highly valued for supporting learning strategies on human biology. Similarly, ref. [72] finds that SeisTutor received positive feedback for its effectiveness and adaptability in aiding the interpretation of seismic data.

Other studies highlight favorable perceptions regarding usability and user experience. For example, ref. [74] discusses the ATS for beginners learning Japanese, while ref. [76] focuses on the affective tutoring system for mobile learning. Similarly, ref. [73] report that their NAO robot is positively perceived by participants from various educational stages. Ref. [79] suggests that their game based on a personalized brain model is viewed positively. Likewise, ref. [39] reports high satisfaction and perceived ease of use among university students who tested their ATS.

Refs. [65,77,78] identify positive perceptions among participating students related to the motivation and engagement generated by their ATS. On the other hand, perceptions from teachers are rarely collected, although [57] reveals that future university faculty participants perceived SimInClass positively for improving emotional awareness.

However, mixed or negative perceptions are also reported. Ref. [56] highlights that in a university e-learning context, ATTENDEE is perceived as enhancing learning, although the initial feedback can be distracting. Ref. [63] reports that their ATS, aimed at e-learning and classroom environments, generally received positive perceptions, but it also causes some frustration due to its frequent interventions. Meanwhile, ref. [82] reports that EasyLogic is positively received for engagement in higher education, but there are mixed opinions about its effectiveness in teaching computer programming.

4.8. RQ8. What Factors Contribute to the Success or Failure of ATSs?

As refs. [59,67] indicate, one key factor contributing to the success of these systems is the pedagogical support they provide. This support involves organizing and presenting information in a structured manner, based on reliable instructional strategies. Additionally, according to refs. [39,70,74,80], this is complemented by real-time feedback and adaptability. The efficiency of these features logically depends on the accuracy of the emotional detection and response, which enables these automated resources to act as intelligent tutors sensitive to affects [65,66,71].

This accuracy, which largely depends on the integration and use of diverse data sources [60], makes it surprising that a high percentage of ATS proposals rely on single-dimensional approaches to identifying students’ emotional states. Without accurate and reliable affective identification, it is not possible to effectively address another key factor: the personalization and tailoring of content, which requires a balance between emotional support and resource delivery [56].

This faces a series of challenges in terms of content integration and the complexity of the technology involved [63], which can result in a high dropout rate in terms of system usage [54]. Therefore, inadequate design or technical limitations can impact its acceptance and usability, especially when targeting students with special needs [62], with content diversity and customization being crucial [53].

4.9. RQ9. What Future Is Projected for ATSs? What Are the Emerging Trends and Potential Future Developments?

The future work contemplated by the reviewed research focuses on several key aspects. On one hand, the technological and methodological improvements are in focus, such as the need to consolidate robust models for the integration of multimodal affective data [14,59,67,75], including non-verbal behavior [65], fine-tuning emotional responses [66], or the integration of a chatbot and deep learning for emotional recognition [55].

On the other hand, expansion and scalability are a focus, recognizing the need to control the effect of these systems in regard to larger samples [56,71,72] or their extension to a wider variety of knowledge areas for the generalization of the results and confidence in their applicability [53,77,78,82].

Additionally, improvements and refinements of emotional and behavioral factors are foreseen, such as bias minimization and real-time detection [70], expanding the datasets used for better applications in real-world contexts [64], and conducting future research that delves into the impact of emotional factors on learning through ATSs [74].

The innovation in educational content and tools involves a more comprehensive application of intelligent pedagogical agents (IPAs) in game-based learning contexts [75], as well as the improvement of adaptive technology for personalized learning [68]. This also relates to the enhancement of interfaces and learning platforms, proposing more ambitious and advanced solutions involving virtual or mixed reality, which in the reviewed sample are limited to special or medical education [65,77,81,88].

As we mentioned earlier, challenges in real-time emotional detection and response need to be addressed [58], along with interface design and the user interaction experience. In the last 5 years, there have been few studies on ATSs proposing unconventional, three-dimensional, immersive interfaces. While virtual pedagogical agents play a significant role, advanced dialogue systems for deeper and more natural affective feedback are lacking, incorporating generative AI to enhance learning variety and personalization. Despite technical challenges, further research is needed to explore the transformative potential of affective tutoring systems incorporating emerging technologies poised to revolutionize human–machine interactions in education.

5. Conclusions

Following our discussion, we respond to the posed research questions based on these conclusions:

RQ1 A: The majority of ATSs employ 2D interfaces, predominantly web-based, to optimize efficiency and accessibility. Despite the availability of three-dimensional and augmented reality systems, their application remains confined to specialized contexts such as special education for students with autism spectrum disorder (ASD). This scenario illustrates that despite technological progress, the adoption of immersive technologies in education is hindered by factors including cost, the complexity of instructional design, and resource limitations.

RQ2 A: ATSs are primarily utilized in STEM fields and higher education, demonstrating a pronounced bias towards these disciplines. Notwithstanding a smaller presence in the humanities and social sciences, these areas significantly influence research outcomes, pointing to the necessity for enhanced equity in ATS research and development across a wider spectrum of domains and educational levels.

RQ3 A: ATSs predominantly focus on emotional recognition, utilizing facial analysis and basic emotion classification. The effectiveness of these systems in authentic settings remains questionable. The adoption of multimodal strategies and adaptive pedagogical techniques, such as scaffolding, alongside, to a lesser extent, game-based learning and Socratic questioning, indicates a shift towards learning experiences more tailored to students’ emotional and cognitive needs.

RQ4 A: Approximately 20% of ATSs incorporate natural language processing (NLP) and natural language understanding (NLU), yet they lack sophisticated language models capable of generating contextual responses. Integrating large-scale language models could markedly enhance ATS interactions, offering more adaptive and intuitive educational experiences. Notably, half of these systems utilize virtual pedagogical agents, whose anthropomorphic design and emotional capabilities significantly enrich educational outcomes.

RQ5 A: In ATSs, the incorporation of emotional recognition significantly enhances the efficacy of assessment and feedback mechanisms, reducing student anxiety, improving academic performance, and promoting more effective self-regulated learning strategies.

RQ6 A: ATSs exhibit a variable, yet generally positive, impact on students’ academic performance and learning experiences, enhancing engagement, learning strategies, satisfaction, and emotional regulation.

RQ7 A: Both students and teachers predominantly report positive perceptions of ATS implementation, noting benefits such as enhanced learning support, usability, and increased motivation and engagement. Despite some mixed or negative views, these do not overshadow the overall positive assessments that underscore ATSs’ significant contributions to emotional and pedagogical development.

RQ8 A: The efficacy of ATSs is contingent upon robust pedagogical support, adaptive feedback, and precise emotional detection. Effective and reliable customization depends on the utilization of diverse datasets. Technical limitations and inadequate design can precipitate failures, particularly in settings requiring high personalization for students with special needs.

RQ9 A: Looking forward, ATSs development is projected to emphasize technological enhancements, including the integration of multimodal affective data and diversification into various areas of knowledge. Prospects include an increase in personalized learning facilitated by emerging technologies, such as virtual reality and intelligent pedagogical agents. Further, there is a concerted effort to optimize real-time emotional detection and response and to advance the development of more immersive interfaces that bolster educational interactions. Notably, the current and anticipated projects reflect limited engagement with large language models (LLM) and generative artificial intelligence.

Limitations

This study has several limitations that should be noted. First, the review was limited to articles in English and to the Web of Science (WoS) and Scopus databases, which may have excluded relevant research published in other languages or in less well-known databases, potentially adding bias in terms of the breadth of perspectives analyzed. Second, the diversity of the affective intelligent tutoring systems (ATSs) evaluated, which vary considerably in terms of design and functionality, makes it difficult to generalize the results to all types of ATSs. Finally, most of the empirical studies included use small and convenience-based samples, which limits the generalizability of the findings to broader and more diverse contexts. These limitations underline the need for future research with more inclusive and representative methodologies that allow the validation and extension of the results obtained.