AI Chatbots in Education: Challenges and Opportunities

Abstract

With the emergence of artificial intelligence (AI), machine-learning (ML), and chatbot technologies, the field of education has been transformed drastically. The latest advancements in AI chatbots (such as ChatGPT) have proven to offer several benefits for students and educators. However, these benefits also come with inherent challenges, that can impede students’ learning and create hurdles for educators. The study aims to explore the benefits and challenges of AI chatbots in educational settings, with the goal of identifying how they can address existing barriers to learning. The paper begins by outlining the historical evolution of chatbots along with key elements that encompass the architecture of an AI chatbot. The paper then delves into the challenges and limitations associated with the integration of AI chatbots into education. The research findings from this narrative review reveal several benefits of using AI chatbots in education. AI chatbots like ChatGPT can function as virtual tutoring assistants, fostering an adaptive learning environment by aiding students with various learning activities, such as learning programming languages and foreign languages, understanding complex concepts, assisting with research activities, and providing real-time feedback. Educators can leverage such chatbots to create course content, generate assessments, evaluate student performance, and utilize them for data analysis and research. However, this technology presents significant challenges concerning data security and privacy. Additionally, ethical concerns regarding academic integrity and reliance on technology are some of the key challenges. Ultimately, AI chatbots offer endless opportunities by fostering a dynamic and interactive learning environment. However, to help students and teachers maximize the potential of this robust technology, it is essential to understand the risks, benefits, and ethical use of AI chatbots in education.

1. Introduction

Advancements in machine-learning (ML) and artificial intelligence (AI) have revolutionized and transformed the landscapes of various industries, including education. Natural language processing (NLP) and ML algorithms have enhanced and empowered AI chatbot technologies to act as human conversational agents, reshaping the traditional communication channels for industries. ChatGPT is an emerging and leading AI chatbot innovation that is based on the large language model (LLM), which uses the transformer architecture of generative pre-trained transformers (GPT) [1,2]. Different industries are trying to find ways to modernize and streamline their business operation through the adoption of AI chatbots. With the capabilities of producing natural human-like conversation, ChatGPT has immense potential for applying to a diverse array of industries such as education, healthcare, finance, e-marketing, and e-commerce [3,4].

AI chatbots have the potential to address many gaps in traditional as well as online educational systems, particularly in student engagement, instructor workload, and personalized learning. Students often struggle with delayed responses to their queries, especially in large classes or online courses. A significant number of students disengage from learning due to a lack of interactive, just-in-time, and adaptive support. Educators spend a significant amount of time on administrative tasks such as answering repetitive queries, grading assignments, and providing feedback. Managing student inquiries, scheduling, and course-related communication can be time-consuming. Traditional teaching methods often fail to cater to individual learning needs. Students, particularly those in remote or underprivileged areas, may struggle to access learning materials and instructor support. Non-native speakers and students with disabilities often face challenges in understanding course materials. Educators may struggle to monitor student performance in real time and identify at-risk students. Recent research indicates that 42% of students had difficulty staying motivated during online courses, highlighting significant engagement challenges in virtual learning environments [5]. Additionally, a survey conducted during the 2022–2023 academic year found that 64% of faculty and instructors reported feeling burned out due to work, underscoring the substantial workload pressures faced by educators [6]. These statistics underscore the critical need for effective interventions.

AI chatbots have the potential to fill these gaps and modernize the education system by timely providing an interactive and dynamic learning experience to students. Online education has not only opened doors to numerous opportunities but also presented many challenges for educators and students. The concerns related to the low retention rate seen in online education due to a lack of student engagement were addressed in an online chemistry course by an AI chatbot called Chem Quest at the Singapore Institute of Technology (SIT) [7]. Educators also often burn out by responding to numerous repetitive types of queries or providing scalable support to students, which can be alleviated by using AI chatbots [8]. AI chatbots can help to bridge such barriers by serving as virtual teaching assistants and providing real-time feedback. AI chatbots can enhance students’ engagement in blended learning, such as flipped classroom environments, by serving as virtual tutors [9]. Integrating such technologies into learning management systems (LMS) can open an array of opportunities, such as serving as a research assistant tool and aiding in understanding complex concepts [10]. AI chatbots also have a growing interest in teaching programming languages and coding concepts by creating interactive environments [11].

Despite the benefits, using AI chatbots in education comes with its own set of challenges and limitations. These technologies pose ethical concerns regarding plagiarism, impacting overall academic integrity [12]. Obtaining data from various sources, AI chatbots are prone to produce incorrect and biased information [13]. The tool also has limitations in handling complex tasks [14], and this is reflected in the case study conducted on nursing diagnoses where results were not aligned with the North American Nursing Diagnosis Association’s international standards [15]. Integrating AI chatbots in educational platforms has also raised concerns regarding data security and privacy. Such tools can be used to plagiarize and aid in obtaining answers to assessments, quizzes, etc. [12]. As with any technology, AI chatbots are also prone to increasing the learning gaps amongst students who do not have the means to avail themselves of such technology [16].

The education industry needs to adapt to emerging technological advancements, such as AI chatbots, to offer innovative solutions and techniques to empower learners and educators. However, it is crucial to understand the limitations and complexities involved in such integration. This research study holds significant importance as AI chatbots such as ChatGPT are on the verge of reshaping various industries and gradually making their way into the education sector. This paper presents a narrative review of AI chatbots and aims to identify the advantages and challenges of adopting AI chatbots in educational platforms. This study also presents findings from the review to highlight the benefits and the educational gaps that can be accomplished through the integration of AI chatbots, such as ChatGPT. The objectives of this research are achieved by discussing the following three research questions (RQs):

• RQ 1: How can AI chatbots aid in improving the learning experience in higher education?
• RQ 2: How can the integration of AI chatbots into learning management systems help to enhance student engagement and support services?
• RQ 3: What are the key challenges associated with the adoption of AI chatbots in online learning, and how can these be addressed?

The rest of the paper is organized as follows. Section 2 presents the background of AI chatbots, starting from the emergence of initial chatbots leading to the evolution of AI chatbots. This section also sheds light on the technologies, programming languages, AI concepts, and ML models associated with designing AI chatbots. Section 3 discusses the methodology, techniques, and approach taken to conduct this review. Section 4 discusses the findings of the review on AI chatbots in education with a key focus on ChatGPT. Section 5 discusses the overall findings along with personal recommendations. Lastly, Section 6 discusses the future recommendations, gaps, and the conclusion.

2. Background

2.1. History

The evolution of AI chatbots can be traced back to the 1960s when computer scientists became eager to discover the possibilities of programming when it came to developing applications that could mimic human behavior. The first AI chatbot, Eliza, was developed in the AI laboratory of MIT in 1966 by Joseph Weizenbaum. Eliza’s purpose was to simulate the psychotherapy techniques created by Carl Rogers, which is referred to as Rogerian Psychotherapist [17]. Eliza simulated the role of a therapist by asking open-ended questions along with follow-ups, sparking the interest of many developers who strived to create such a technology [18]. Eliza became an inspiration for the AI chatbot industry using NLP. The emergence of Eliza gave way to the development of a more advanced chatbot known as Parry, which was created by Kenneth Colby in 1972. Parry used complex models of assumptions and emotions to imitate the behavior of a person with paranoid schizophrenia. Unlike Eliza, Parry had an aggressive and unfriendly tone [19], which earned it the name of “Eliza with attitude” [17]. Eliza and Parry demonstrated that human-like interaction with AI was possible. This massive breakthrough with these two chatbots proved that further research in this area was necessary and warranted [19]. Figure 1 shows the significant evolution of chatbots over time.

In 1982, Rollo Carpenter created a chatbot called Jabberwacky, giving the illusion of human-like conversation by using a technique called contextual pattern matching that involves collecting a group of keywords from previous and existing conversations, automatically adding it to the database as opposed to using a static database [17]. Jabberwocky was later redesigned and launched in 2008 and is now known as Cleverbot [17].

In 1995, Richard Wallace developed a chatbot named Alice [17]. Alice’s design consisted of pattern-matching algorithms that analyzed and compared the user’s text with the knowledgebase, which contained categories of modules with questions and answers orchestrated in a language called Artificial Intelligence Markup Language (AIML) [20]. Alice’s knowledge base consisted of AIML tags containing standard categories, allowing the bot to have human-like conversations with the user [20]. In 1997, AOL developed a chatbot called AOL Instant Messenger (AIM) that allowed users to interact with each other [21]. Released in 2001, SmarterChild was an advanced and widely popular AI chatbot that was designed and built by ActiveBuddy using the AIM platform; it provided users with access to news, weather, transit schedules, etc. [17].

Evolving developments and the unveiling potential of AI have led tech giants such as IBM to intensify their development efforts and delve deeper into exploring the possibilities such technology has to offer. Researchers at IBM developed a complex question-answering system called IBM Watson in 2006, consisting of components of various technologies such as machine-learning, query transformation techniques, semi-structured databases such as DBpedia, and matching algorithms such as skipping bigrams, etc. [22]. Watson’s extreme computing power allows the processing of 200 million records along with providing access to a range of databases, making it an ideal question–answer system that has been widely adopted in weather forecasting, the health industry, and the educational industry, etc. [17]. Mitsuku, created by Steve Worswick in 2012, is an example of another chatbot built using AIML and was considered to be a highly intelligent system [17]. The emergence of AI and newer, more advanced technologies have resulted in the remarkable evolution of chatbots. One very notable and widely used chatbot today is ChatGPT, which was created by OpenAI (https://openai.com/index/openai-codex/, accessed on 16 January 2025) in 2022 and is based on GPT3.5 architecture using NLP models [23].

2.2. Framework and Technology of AI Chatbots

The design and implementation of a chatbot can vary based on the technology stack and the complexity of the use case. AI chatbots are designed and intended to act as human conversational agents due to their ability to converse in a natural human-like language. NLP is used in chatbots to extract and understand user input by breaking down the data into smaller fragments, along with the task of formulating the response and responding to the user in the form of natural language [24]. The key components of chatbots fall under two subsets of NLP, which are Natural Language Understanding (NLU) and Natural Language Generation (NLG) [24]. NLU is the process that interprets and comprehends the human language input received from the user [25]. The response segment is processed by NLG, which generates a computer-based response that can be understood and interpreted by humans [25]. Named Entity Recognition (NER) and Intent Extraction (IE) are two sub-tasks of NLU where NER is used to detect and identify text that can be labeled as named entities such as person, time, and currency. [25]. IE is the technique in NLU that aims to classify and understand the action and the objective being conveyed in a sentence [25]. The structure of a conversational agent consists of four stages. The first stage of the process involves classifying and processing the input received from the user through the extraction of named entities and intent discovery [24]. The second stage involves understanding the input through complex functions like Sentence Boundary Detection, Chunking, and Sentence Parsing. [24]. The third and fourth stages of the process involve generating responses from datasets that have been trained using machine-learning models and then selecting the most suitable response [24].

There are three models classified as part of NLP: pattern-based, retrieval-based, and deep-learning-based [26]. In the pattern-based model, the chatbot generates a response using a question–answer pattern that matches the most relevant response to the user’s query [26]. However, pattern-based models have limitations, such as the fact that a response cannot be generated if the patterns or the questions do not exist in the database [24]. AIML, Rivescript, and Chatscript are widely used programming languages for the implementation of pattern-based chatbots [27].

The retrieval-based model works on the principles of retrieving a response from a set of predefined responses using various techniques, such as a straightforward rule base or ML ensemble techniques, to provide the best possible response given the context [28]. While training such models on a large dataset can make them less prone to errors, they are still prone to errors due to other limitations. One notable limitation of retrieval-based models is that they can only generate a response from the available options rather than generating a new text [28]. Additionally, due to its limitations, the model’s responses do not sound human-like, thus making the model unsuitable for chatbots that require complex queries [28].

In deep-learning-based models, the sequence-to-sequence (Seq2Seq) model was introduced in 2014 and is a class of the Recurrent Neural Network and is now one of the most widely researched models for implementing AI chatbots [29]. The model was initially developed for neural machine translation but is now frequently used in conversational modeling. Based on neural networks, a key advantage of using the Seq2Seq model in chatbots is that it is a learning algorithm; the model works by the encoder capturing the input data and converting them into hidden vectors, which are then consumed by the decoder to generate the results and produce the output [24]. Another popular model, Long Short-Term Memory (LSTM), is a deep-learning algorithm based on a Recurrent Neural Network that can be used for sequential data such as speech and videos [24]. LSTM models are used in areas of speech recognition, but due to their complex computational processing, they fall short in areas where a detailed explanation of prediction results is required [24]. The Hybrid Emotion Interference Model (HEIM) is another advanced algorithm that uses NLP to understand human emotions, expressions, voices, etc. [24]. The algorithm uses deep, sparse neural networks to understand and analyze various kinds of data patterns containing acoustic, textual, and geo-social [24]. Figure 2 provides a holistic overview of the technologies and framework underlying chatbots.

3. Research Methodology

This research study investigates the benefits and drawbacks of using AI chatbots in education. We conducted a narrative review and answered the three research questions (RQs 1, 2, and 3) regarding the investigation of AI chatbots in the educational setting, as stated earlier in the Introduction (Section 1). The data collection was conducted using scholarly journal articles from reputable sources such as ACM Digital Library, Scopus, IEEE Xplore Digital Library, MDPI, Springer, ScienceDirect, and Google Scholar. Figure 3 illustrates the inclusion and exclusion criteria applied to select papers to conduct this narrative review. The criteria were determined to ensure the studies chosen are relevant to the research questions and maintain a certain level of quality and consistency.

In the first step, to select the papers, we conducted a search through a combination of key terms “ChatGPT”, “chatbots”, and “AI chatbots” with the key terms “higher education”, “education”, “online learning”, “teaching”, “learners”, “students”, “teachers”, and “learning management systems”. Multiple searches were performed using different combinations of keywords to find a range of articles covering different aspects of chatbots in education. Articles that were published beyond 10 years were not considered, as AI chatbots in education have become more popular in recent years, and older articles would lack relevant information on the topic. Initially, the search yielded a total of 976 papers from all sources. In the second step, we reviewed the studies’ title, abstract, and introduction and eliminated the articles that were not relevant to the research questions. This reduced the number of studies to 117 articles. In the third step, we evaluated the quality of the studies by assessing the research methodology, research design, evaluation strategies, and clarity of objectives, which refined the selection to 45 articles. The articles were further divided into two separate groups according to their titles. The first group of articles was mainly geared toward AI chatbots in education, and the second group of articles focused on ChatGPT in education, as discussed in Section 4.

4. Chatbots in Education

4.1. AI Chatbots in Education

AI chatbots are becoming increasingly popular in education. AI chatbots built for education have been shown to provide significant benefits, such as the fact that they can act as conversational tools and can aid in answering education-related queries through text or speech using Artificial Intelligence Markup Language (AIML) [30]. They can also be accessible round the clock, serving as continuous aid to students for their inquiries even if they are in a different time zone. Some potential benefits of using AI chatbots in education are summarized in Figure 4.

Technology has helped overcome communication barriers. Ayedoun et al. [31] proposed a conversational agent to help learners learn English as a Foreign Language (EFL) based on a model of willingness to communicate (WTC). The results showed a boost in learners’ confidence and an increase in the frequency of communicating in English [31]. AI chatbots can provide a personalized learning environment, and due to their simplistic UI characteristics, they are more intuitive than other web-based and mobile applications [32]. Additionally, AI chatbots can aid visually impaired students in their learning and educational activities [30].

The Hong Kong University of Science and Technology (HKUST) implemented a chatbot using Google Dialog Flow for training teaching assistants [8]. The training program consisted of 550 postgraduate students who were trained as Graduate Teaching Assistants (GTAs) to assist in the teaching activities of undergraduate students. This case study highlights some critical longstanding challenges in educational systems, such as the fact that instructor-to-student ratios are often unbalanced, thereby leading to delayed feedback from instructors and the instructors being bombarded with multiple repetitive queries. The GTAs would rely on receiving feedback from instructors through discussion forums, email, or an automatic feedback system from LMS [8]. However, this would result in significant delays and would be a cumbersome process just to acquire clarifications on some basic questions such as grading criteria, submission, etc. The chatbot solution was designed to address key challenges encountered by GTAs, provide dynamic support, overcome barriers of diversity, and offer real-time feedback for repetitive task [8].

The lack of functionality or a mechanism where students can communicate and receive feedback from educators in real time can negatively impact a student’s performance and retention rate. Singapore Institute of Technology (SIT) experienced a similar issue with their Online Chemistry course, where the overall completion rate was relatively low at around 15%, the key reason being that students were unable to obtain an immediate response from their instructors while completing the course [7]. To overcome this problem, they developed an AI chatbot using a deep-learning AI platform called Chatlayer [7]. The chatbot design was geared more toward providing guidance and clarifications to inquiries rather than giving definite answers to the assessments. The beta version of the chatbot gave promising results, and students appreciated the chatbot being available 24/7, leading to the deployment of the chatbot to the production servers [7].

Traditional LMSs often provide structured and static learning content for all users. They lack features that can provide customized content based on individuals’ learning needs. To address this issue, Subramanian et al. [33] designed an AI chatbot called TutorBot using the Google Dialog flow interface and leveraging techniques using ML and NLP. The chatbot was able to give flexibility to accessing external resources within the platform according to the students’ needs in addition to the integrated static contents in LMS. The study was conducted among 20 participants, and the participants were divided into Group A and Group B. Group A used TutorBot, and Group B used a traditional LMS to complete the training program. After two weeks of running the program, the results reflected a 90% improvement rate in obtaining content and relevant information on the topic while saving 60% of their time by participants from group A compared to participants from group B. The authors also suggested that integrating features to support multilingual conversations and refine the accuracy of automated speech recognition components can further improve the TutorBot [33].

Chatbots are also gaining popularity in streamlining various administrative tasks in educational institutions. A study aimed to evaluate the potential of AI chatbots in assisting students with administrative queries for seeking admission into college programs [34]. The chatbot was developed using BotPress and leveraged NLP and NLU techniques. The web integration was performed using a website based on WordPress and a WAMP server through WordPress’s Header and Footer scripts plugin. The responses generated from the chatbot had an accuracy of 90.6% percent and a CSAT score of 84.7% on various inquiries related to admission, accommodation, fees, etc. The research shows the potential of integrating AI chatbots into educational platforms to serve as virtual assistants in the administrative domain. It can serve as a tool and a resource for providing real-time information and can assist with repetitive administrative tasks from multiple sources.

Flipped education provides students with the opportunity to familiarize themselves with the course content via videos or research before interacting with teachers. This model has proven to be greatly effective where students’ learning does not start from scratch [35]. To justify the effectiveness of this model, research was conducted on 32 student participants at a private Mexican institution by implementing Bing Chat in a Mathematics for Decision-Making course [36]. In the first stage of the research, the participants were asked to explore a theoretical presentation and solve two math problems without the aid of any instructor. The second stage involved discussing the solutions and clarifying their doubts with an instructor. In the final stage, the students were asked to provide their reflections and thoughts on the effectiveness of using Bing Chat throughout the study. The overall results were positive, showing Bing Chat to be a helpful tool for students in regard to their critical thinking processes [36].

Recent developments in AI and ML have greatly improved AI chatbots [30]. The need for AI chatbots is rising in higher education due to the administrative and educational support they provide to instructors, enabling them with resources to create a personalized learning environment and enhance learning experiences for students while simultaneously reducing the overall cost [9]. Online education has unlocked a spectrum of opportunities and has provided the benefit of learning in a flexible and globally accessible environment. However, a key challenge is that despite having an abundance of features and resources available in online platforms, such learning platforms still lack functionalities that can provide students with an immediate response to their queries. AI chatbots have great potential to solve these issues [8].

4.2. ChatGPT in Education

ChatGPT was initially designed to handle language translation tasks but is now an AI-generative tool used for carrying conversations and generating text in real time [37]. It poses several advantages over other language models, such as BERT and XLNet, by achieving a higher accuracy model through access to enormous datasets with billions of parameters [10]. Compared to other models developed by Open AI, such as DALL-E, which is designed for text-to-image, ChatGPT’s framework is based on a text-to-text AI generative model with the ability to have a human-like conversation with natural dialogs, making it suitable for a wide range of applications [37].

ChatGPT and AI models can be an asset to researchers in the education industry [38]. Jalil et al. [39] conducted a study to assess the accuracy and practicality of using ChatGPT to answer questions for an undergraduate-level software testing course offered at George Mason University. The experiment was based on evaluating multiple questions from five chapters of the book, and the study focused on a couple of research questions that determined the degree of correctness of the answers provided by ChatGPT [39]. Findings from this study showed that ChatGPT was able to respond to 77.5% of the questions, from which 55.6% of the questions answered were correct or partially correct, and the explanations provided for the answers were 53% correct or partially correct [39]. The research also reflected that the accuracy level and the degree of correctness could further be improved through follow-up questions [39]. Lund et al. [10] researched and analyzed the scholarly writing capabilities of ChatGPT, and the results showed that the software could craft professional papers and essays with human-like language that could even exceed the expectations of a doctoral-level student.

Laato et al. [40] carried out extensive research that was divided into three phases to demonstrate the capabilities of ChatGPT in higher education. The first part of the research involved the authors familiarizing themselves with the application for two months, followed by the second stage, which involved formulating use cases for ChatGPT to be used in higher education. The final part involved evaluating how ChatGPT could assist students in completing courses for a Bachelor of Computer Science degree from a university based in Finland. The research spanned several weeks and included vigorous testing of ChatGPT’s response on computer science-related topics such as machine-learning, algorithms, microprocessors, and different programming languages [40]. The results revealed that ChatGPT was able to provide answers to many of the questions with a high level of accuracy and was also able to provide extensive answers for essay-based questions. However, it was noted that some answers lacked critical details and contained information that pertained to common definitions rather than applied concepts. Some inaccuracies were also found in responses to questions requiring practical implementation [40].

Research was conducted on understanding public sentiments in adopting ChatGPT in the education industry by extracting data from Twitter API using Python [41]. The study specifically chose tweets as the source of input data for this research as they provide valuable, unfiltered views of the public and also because they occur in real time and are accessible [41]. The dataset consisted of a total of 11,830 tweets, collected over twelve days by using a unique set of keywords [41]. The TextBlob library was used after cleaning the data to categorize the tweets into positive, negative, and neutral sentiments [41]. The results were a good reflection of the public opinions towards AI in education, where 6179 tweets were in the positive category, 1688 in the negative, and 3963 in the neutral category of sentiments [41]. The positive opinions of the public far exceeded the other sentiments, praising ChatGPT’s computing power and the consistency of delivering correct answers to various inquiries [41].

4.2.1. ChatGPT for LMS

Integrating ChatGPT with Learning Management Systems (LMS) can enhance the learner’s interaction through instant feedback and support [42]. Moreover, it can be tailored to accommodate personalized learning and foster an interactive learning environment that enhances learners’ engagement and retention rates, specifically in higher education, where educators face challenges in providing individual assistance due to the low student–teacher ratio [42]. Learning management platforms (LMS) can be enhanced by integrating ChatGPT via APIs to provide personalized support for students in answering questions, suggesting resources, and generating quizzes and practice exams [42].

Blended learning systems are hybrid academic models that comprise elements of online learning along with traditional face-to-face methods [42]. Blended models, such as flipped classrooms and hybrid courses, are becoming increasingly popular due to their flexibility and sustainability [42]. ChatGPT can increase the level of engagement in blended models and can aid in understanding complex concepts, creating an enjoyable, dynamic, and interactive learning experience [42]. AI chatbots such as ChatGPT that use Natural Language Processing (NLP) can also act as virtual instructors, changing the dynamics of education by introducing flipped educational models where students can learn before the class and use the actual class time for group discussions [9]. Integrating ChatGPT with educational platforms can also empower learners with disabilities to overcome barriers by tailoring course content according to their specific needs [16].

4.2.2. ChatGPT for Educators

ChatGPT’s capacity to conduct conversational-style exchanges with users makes it an ideal virtual tutoring system where learners can ask questions and receive feedback in real time [37]. Integrating it as a virtual teaching assistant can reduce the work of instructors by helping them respond to students’ queries and providing daily support [43]. Instructors can use it to create intelligent tutoring programs that can be customized for students, offering them a personalized learning experience [44]. It can also aid instructors in creating content such as quizzes, presentations, course outlines, etc. [37].

Furthermore, ChatGPT can serve as an online training tool for instructors to help them advance their skills through professional development resources [43]. Instructors can use it as a tool for analyzing students’ performance and identifying patterns and trends to improve teaching strategies [16]. It can be a cost-effective solution for designing educational programs along with recommended resources based on the academic program [16]. ChatGPT can aid research assistants with research ideas and use methodologies that have been applied in previous studies [14]. It can also be used as a tool for statistically analyzing data, finding relationships between data, interpreting data, and providing suggestions for future research [14].

Dai et al. [45] conducted a study to determine the practicality of using ChatGPT as an assessment tool for providing feedback on student’s assignments. A dataset was taken from a postgraduate data science course from an Australian university, in which students were tasked with proposing a data science project based on a business scenario [45]. The feedback to the assessments was given by instructors and ChatGPT through a series of prompts based on the following rubric points: the clarity of the project goals, the relevance of the topic to data science, details on the business benefits, the creativity of the topic and the overall clarity of the proposed solution [45]. The feedback on each criterion received from ChatGPT and instructors was further graded by three experts on a five-point scale, determining fluency and coherency [45]. The results were promising, with ChatGPT being able to analyze students’ performance and provide detailed feedback alongside suggested learning strategies that students could adopt in the future [45].

Another study was conducted to analyze the gaps in incorporating lesson plans into ChatGPT for learning activities. The lesson plans were created by 29 pre-service elementary teachers, who comprised 11 males and 18 females [46]. Most of the participants were sophomores from the field of science at the Korean teacher’s university [46]. They went through a three-week on-site teacher’s training school program, enabling them to experience a classroom-like environment [46]. The study plans were created for various science domains such as Physics, Chemistry, Biology, Earth Science, and Environmental Science [46]. The results of the study and the performance of teacher’s lesson planning were evaluated using GenAI-TPACK, which looked into four categories such as the correct way to use ChatGPT in creating lesson plans, how well the curriculum goals can be supported by ChatGPT, how seamlessly ChatGPT integrates with teaching strategies, and lastly, how effectively curriculum goals, methods, and techniques can be designed using ChatGPT [46]. To gain deeper insight into pre-service teachers’ GenAI-TPACK, the participants were also asked to complete a survey that comprised three open-ended questions [46]. The research results were examined by two researchers using two sets of data: (1) lesson plans using a scoring rubric and (2) the responses received from the survey [46]. The overall results showed ChatGPT being effectively integrated into lesson plans of science subjects, covering methods such as group learning and predict–observe–explain (POE) [46]. Qualitative findings revealed that leveraging such strategies through ChatGPT increased student engagement and promoted thinking skills [46].

4.2.3. ChatGPT for Learners

Using ChatGPT in education can foster an active learning experience by responding to questions and offering spontaneous suggestions, hence creating a framework for an interactive learning model [9]. This style of learning approach is attributed to the Two Sigma Effect, where the learning is enhanced by two standard deviations compared to traditional methods [37]. This method of learning also aligns with the concept of mastery learning, in which students learn to strengthen their foundational concepts at their own pace before moving on to advanced materials [37]. Learning a programming language would be an ideal use-case where students can gradually strengthen their foundational concepts of data structures and algorithms through interactive learning [11]. Another example could be taken of understanding complex concepts found in the field of biomedical sciences, where ChatGPT can provide a simplified explanation of a difficult concept [44].

Integrating artificial intelligence systems along with virtual reality technologies can break the traditional barriers to learning and can also enhance student engagement [11]. Students can leverage the ChatGPT interface as a system to help narrow down their research and delve deeper into a topic through a series of queries [10]. Researchers have been exploring the benefits that AI chatbots can bring to the software development industry. They have found that ChatGPT can be leveraged to generate code, translate software development requirements, and formulate use-case diagrams, class diagrams, and sequence diagrams for illustrating the flow between objects [47].

Speth et al. [48] conducted a study to examine the quality of exercises generated by ChatGPT for a Java programming course. The research consisted of generating twelve exercise sheets ranging from beginner to intermediate level, covering various concepts following the framework of the object-first method [48]. The experiment was aimed at demonstrating ChatGPT’s capabilities to serve as an instructor for a Java programming course. The experiment was conducted by telling ChatGPT to play the role of a professor and the instructions specified, containing the list of topics and the expected learning goals to be achieved [48]. To evaluate the accuracy and relevance of the exercises generated by ChatGPT, eight participating students were asked to fill out a questionnaire using a Likert scale at the end of the programming course [48]. The results showed that the quality of content created for some topics, such as flow structures, APIs, and inheritance, was good [48]. The questionnaire responses also showed interesting results, as many of the students could not detect that the exercise sheets were generated by ChatGPT. Other observations showed that AI chatbots needed precise and detailed instructions to accurately generate the content [48]. It was also seen during the experiment that if slight changes were required in the exercises generated, ChatGPT would modify a substantial amount [48]. Certain exercises, e.g., creating UML diagrams—could not be generated, and the response was provided in the form of textual data [48]. Some other challenges were seen in generating exercises relating to finding erroneous codes. The overall results showed positive signs of ChatGPT being used as a powerful and useful tool in universities for generating content [48]. Effectiveness in generating Java programming exercises has also been analyzed with other tools, such as Codex (OpenAI), Replit AI, Codecademy, and HackerRank. Vaithilingam et al. [49] showed that ChatGPT can produce diverse coding tasks for different skill levels. However, compared to dedicated platforms, ChatGPT lacks built-in testing environments or immediate feedback mechanisms. Chen et al. [50] demonstrated that Codex can generate functional code with high accuracy but struggles with nuanced requirements. Similarly, systems like LeetCode provide curated exercises with test cases, which ChatGPT can complement by explaining solutions or generating problem variations.

Sudirman and Rahmatillah [51] conducted research in Bandung to explore the benefits of ChatGPT to students in an Entrepreneurship program. The research consisted of 213 students who were divided into five classes, and each class was further divided into groups of five students [51]. The goal of the study was to analyze if students can use ChatGPT as a tool to generate information and use it to formulate their ideas. The participants were directed to use ChatGPT for gathering information related to designing a business mobile app for solving problems experienced by consumers or finding solutions for issues encountered due to existing technology; however, the participants were instructed not to directly ask ChatGPT to generate such an idea. The participants’ experience and results were evaluated through a questionnaire. The results showed that many of the students found ChatGPT to be a useful tool for brainstorming ideas and acquiring data insights; they were keen on using it in other courses. The participants indicated they had an enjoyable experience, and the overall research findings leaned towards showcasing ChatGPT as a tool for enhancing creativity and modernizing education [51].

AI chatbots are also gaining popularity in the areas of language development by creating an adaptive learning environment that can help learners strengthen their communication skills and learn foreign languages [52]. They can be geared to offer a personalized learning program to help learners improve vocabulary and grammar for English for specific purposes (ESP) students [52]. The model can be trained using a dataset from English as a Second Language (ESL), and the generated exercises can be customized to meet students’ proficiency levels and learning needs [52].

4.3. Limitations and Drawbacks of AI Chatbots in Education

ChatGPT poses a range of benefits in the educational sector. However, it also has its shortcomings. The following subsections delve deeper into the limitations and drawbacks associated with such technology in educational settings.

4.3.1. Ethical Implications

With the ability to generate text in natural language, academic integrity, and plagiarism are key concerns in adopting ChatGPT in educational platforms. Plagiarism plays a critical role in protecting academic integrity and the ethical foundation of education. Investigations have shown that ChatGPT can reflect the information they produce is unique by surpassing well-known plagiarism detection tools such as Turnitin [9]. This poses a critical concern in the education sector. ChatGPT can generate plausible data, but it does not reference the source of the data, which leads to concerns about the integrity of the information [12]. Students can impact learning integrity by using ChatGPT to generate essays and research papers and submit them as their content [16]. They can also misuse the advanced AI capabilities of ChatGPT to cheat on exams and quizzes [16]. Ever since COVID-19, online assessments and learning have boomed at a rapid rate and have shown to offer a range of benefits. However, with the emergence of ChatGPT, academic integrity might be at stake since students can use it to cheat during online exams or gain an unfair advantage against students who do not have access to such tools [13].

Accidental plagiarism is another growing concern faced by researchers when using AI chatbots such as ChatGPT, where the data generated do not contain links or references to the original source. This can result in unintentional plagiarism by not giving credit to the original author [13].

Detecting AI-written content can be quite challenging, even for large organizations such as OpenAI that are actively engaged in researching and developing solutions to provide transparency between AI and human-written text. Advanced technology containing high computational power that can detect plagiarism in research and other sectors will be a much-needed necessity for educational organizations in the future [53]. OpenAI had developed a classifier to detect AI-generated text. However, with a low accuracy rate of only 26%, it was discontinued in July 2023 with an aim to develop a more robust and accurate model [54].

The classifier had limited capabilities of producing inaccurate results with text containing less than 1000 characters and was only recommended for text written in English [54]. LLMs often misinterpret the data, which could lead to inaccurate results [40]. Current LLM models such as ChatGPT are available commercially, and uploading copyrighted material may violate intellectual property rights and copyright laws that could possibly lead to legal action [40]. Responses received from a poll of 1000 college students in the US showed that 60% of students claimed to use AI chatbots for completing more than half of the coursework, and 33% of the students used such tools to complete their written assignments [11]. Mosaiyebzadeh et al. [13] proposed a solution to overcoming such a problem by reverting to invigilating or conducting oral exams, reducing the risk of cheating during online exams. Wagholikar et al. [55] suggested another approach where instructors design exams and assignments that involve supporting their answers with diagrams and graphical representations.

Limitations and drawbacks along with ethical implications of using ChatGPT in education have been summarized in Figure 5. Addressing ethical concerns has been challenging for educational institutions. Educational institutions can promote responsible AI use by developing comprehensive policies. Institutions can establish AI ethics committees and ethical guidelines to help monitor compliance, address ethical concerns, and mitigate potential biases and societal harm. Institutions can document decision-making processes, use interpretable models, or provide clear explanations for complex systems to ensure transparency and explainability. Training and awareness programs would be useful among different stakeholders and users to understand ethical AI practices and foster a culture of accountability. Lastly, policies should be dynamic, with continuous evaluations to adapt to technological advancements and evolving regulations.

4.3.2. Limitations in Understanding Human Emotions

AI chatbots can provide guidance and feedback in the form of human-like conversations; however, they lack the capabilities to understand human emotions [13]. Educators play a vital role in the education system, and their core responsibilities and skills go far beyond just sharing knowledge. They serve as key pillars in building a supportive learning environment that is based on trust and understanding a student’s needs. Over the years, educational technology has evolved drastically, helping overcome many barriers, but one aspect of education that cannot be changed by the doings of technology is the human touch of understanding emotions and building relationships.

4.3.3. Programming Challenges with ChatGPT

While ChatGPT can enhance learning in the software development industry, it also comes with its own set of limitations. Berrezueta-Guzman and Krusche [56] conducted an experiment to test the capabilities of ChatGPT in solving programming assignments. The experiment analyzed 22 homework exercises from a first-year programming course offered at the Technical University of Munich (TUM) [56]. The results highlighted both the positive outcomes and drawbacks of using ChatGPT in the educational system, more specifically, in programming courses. The results revealed that ChatGPT showed inaccuracies while trying to solve problems related to JSON jobs, generating the code of a basic calculator, and was unable to solve two exercises related to ULM diagrams and implementing GUI features [56]. The experiment also highlighted the long-term negative impacts ChatGPT could have on students’ abilities to think critically and independently in solving problems [56]. There should be awareness amongst learners to use ChatGPT as a supplemental tool and not as a replacement for traditional learning techniques [51].

4.3.4. Misinformation and Source Reliability Issues

The dependency, accuracy, and biased responses of ChatGPT have raised alarming concerns among researchers about adopting and using it in educational platforms [57]. ChatGPT can produce incorrect information that may hamper the learning process and the integrity of education for users who frequently rely on such platforms [9]. Producing incorrect or false information is a key concern in AI technology and is often referred to as AI hallucinations [18]. ChatGPT being trained on vast amounts of data could lead to ambiguous and inaccurate results, hindering the overall learning process [57]. The inaccuracies are due to the wide range of data available on the internet that is being used to train such models [13]. OpenAI also acknowledges that there may be discrepancies in the results produced by ChatGPT [58].

Researchers using ChatGPT could also be prone to factual inaccuracies and may unintentionally plagiarize data or fail to provide credit to the original author, as the responses generated do not cite the authors nor the source of origin [57]. ChatGPT uses data from various sources available on the internet for training its models and generating results, and this also poses a risk of producing incorrect or false information and lacks the functionality of verifying factual information [13]. Generating incorrect factual information can severely impact the integrity of education by developing students’ learning and knowledge bases with inaccurate information.

Zuccon et al. [59] conducted research to test the accuracy and the quality of data, along with testing the correctness of the references produced by ChatGPT for the data it generates. The research pertained to the topics based on the agriculture domain and was taken from the Ag-valuate collection. The 160 topics used for this research were created by agriculture scientists and crop growers [59]. The reason for using this collection of topics as part of this study was that all the questions and answers were readily available to the public, and the information was backed up with references [59]. Also, domain experts were available to validate the information provided by ChatGPT. The methodology involved prompting ChatGPT with questions on agricultural topics and instructing the tool to provide answers along with references to the source [59]. The authors of the research, along with a research assistant, validated if the answers contained references and if the sources indicated by ChatGPT were accessible, then answers were further validated by an agricultural scientist and an expert in crop science [59]. The results showed that 49.4% of the answers were incorrect, 37.5% of the answers were partially correct, and only 13.1% percent of the answers were completely correct [59]. The incorrect answers also contained misleading information, and some answers contained very general information. It was also found that out of 160 answers, 14 answers were not supported by references despite instructing ChatGPT to include references for each answer [59]. The majority (87%) of the references provided by ChatGPT were from Wikipedia pages, and some sources cited gave the impression that articles taken from scientific journals did not exist, which relates to the challenges of hallucination faced by the tool [59]. Two scientific articles referenced by ChatGPT did not have the correct metadata, such as the authors and the years [59]. The study revealed that only 18% of the answers had the correct content that correlated with the references cited by ChatGPT [59].

4.3.5. Reliance on ChatGPT

While the many advantages of ChatGPT for students are evident, it is crucial to keep in mind that student dependency on ChatGPT may also lead to negative effects on them as well as instructors. Using ChatGPT daily can limit the ability to solve problems and generate ideas independently [16]. Relying heavily on such technology may disrupt the learning process and lead to frustration if system failures or malfunctions occur [16]. Also, the heavy use of chatbots such as ChatGPT can greatly impact students’ critical thinking process and may hinder overall learning by providing them with tools that have the capability of generating answers without engaging in the learning process [13]. Though ChatGPT can offer consistent support and guidance, it cannot surpass the level of interaction through traditional teaching methods provided by humans [16]. The constant use of ChatGPT impacts the ability to create social groups and foster an environment of collaborative learning [16]. Overreliance on ChatGPT can also hinder the principles of research as it enables students to become accustomed to the tool and refrain from seeking other sources to conduct research [14].

4.3.6. Security and Privacy Concerns

Security is crucial for any organization when it comes to integrating new technology into its existing infrastructure. Privacy and data security are key concerns in integrating AI chatbots into education systems [13]. Unauthorized access can lead to data breaches and can compromise students’ personal information [13]. Cyber security domains have raised concerns about attackers using ChatGPT to create phishing and using such AI tools to create code changes [55]. Data breaches would be another security concern due to storing sensitive data such as student grades and personal information [14]. Integrating ChatGPT into educational platforms is a complex and rigorous process that requires a thorough evaluation of workflows and procedures [57].

4.3.7. Language Limitations and Accessibility

ChatGPT may provide an unfair advantage to students who are well-versed in the languages supported by the tool, and such barriers may lead to discrimination because every individual may not be able to reap the benefits of the tool [53]. Another concern arises in regions where students do not have access to such tools; this leads to those students experiencing an unfair disadvantage compared to students who have the opportunity to access and utilize such technology as part of their learning journey [9]. Furthermore, not all institutions will have the resources and financial backing to integrate and test this technology in their learning platforms. This may impact the growth and reputation of educational institutions that aim to modernize their platforms.

4.3.8. Limitations with Complex Tasks

ChatGPT is also known to have limitations in handling complex tasks, which may limit its use in certain research areas [14]. A study was conducted to demonstrate the potential of ChatGPT to serve as a viable diagnostic tool in nursing education [15]. A case study on preventive care was selected from Train4Health, and based on it, a detailed description of a patient was entered into ChatGPT, prompting the tool for information regarding nursing diagnoses, interventions, and outcomes [15]. The results showed that ChatGPT was able to analyze the case study and present the nurses with appropriate diagnoses along with planning goals and interventions related to the patient’s health. Though the tool was able to provide detailed diagnoses, the results deviated from the requested standards of the North American Nursing Diagnosis Association—International (NANDA-I) [15]. The study revealed that ChatGPT has the capabilities of analyzing data and can be used as a guide in nursing education; however, the results also reflected that the diagnosis was not consistent with the NANDA-I standards and produced some data incorrectly [15].

5. Discussion

5.1. RQ1: How Can AI Chatbots Aid in Improving Learning Experiences in Higher Education?

Findings from research studies highlight the benefits and the potential of AI chatbots in transforming traditional learning practices by offering personalized learning that can be tailored to individual needs. Chempavathy et al. [30] shed light on the benefits of AI chatbots being used as conversational agents to aid students in answering educational-related queries and aid visually impaired students.

The solution proposed by Ayedoun et al. [31] to help learn English as a Foreign Language through a conversational agent demonstrated the effectiveness of such technology and the positive impact on the learner’s confidence. AI chatbots such as ChatGPT can be used to hold open-ended conversations about any subject and can be used as a conversational partner for completing text, creating dialogue, and translating languages [1]. Students who do not have any means of improving their language proficiency through speaking can utilize AI chatbots for role-playing as a conversation partner. NLP technology has revolutionized language learning, empowering students to learn at their own pace with lesson plans tailored to their personal needs.

Learning a programming language can be a daunting task that involves understanding complex concepts about data structures, algorithms, and programming syntax. Students embarking on this journey often face challenges in understanding the concepts and the structure of the language, which eventually leads them to discontinue their learning. ChatGPT can help students overcome such obstacles by providing a personalized and interactive learning plan where students can gradually learn and enhance their problem-solving skills with real-time feedback. Adaptive learning courses can be created via ChatGPT, where the software can analyze the learner’s knowledge and tailor the lessons accordingly. As discussed by Arista et al. [57], ChatGPT can help build the foundational concepts in computer programming on data structures and algorithms through interactive learning paths. This can be a great asset to institutions that do not have the resources or skills to teach such subjects. ChatGPT can generate adaptive learning paths containing quizzes and assessments that can measure the student’s understanding of the subject. This is the key differentiator of introducing techniques through technology as opposed to traditional learning practices. Adaptive learning via AI Chabot provides the opportunity and platform for students to strengthen their knowledge of concepts in areas where they encounter the most challenges. This reflects on the previously discussed viewpoints of Zaabi et al. [44], where students can use ChatGPT to understand complex concepts in the field of biomedical sciences by breaking them down into smaller fragments. AI technologies such as ChatGPT can help break educational gaps and overcome challenges faced by third-world countries that prevent them from indulging in fields related to technical studies. ChatGPT can help overcome obstacles and bridge gaps for institutions that lack qualified instructors to teach technical courses and advanced studies.

Pedagogical models that integrate AI chatbots into education enhance personalized, adaptive, and student-centered learning experiences. Constructivist models, for instance, leverage chatbots to provide real-time feedback, guide inquiry-based learning, and support problem-solving through interactive dialogues. In flipped classrooms, AI chatbots can help students with pre-class content, answer questions, and reinforce concepts, freeing up class time for deeper discussions and collaborative activities. Socio-constructivist approaches benefit from chatbots as virtual peers, promoting collaborative learning by facilitating group discussions or suggesting resources. Mastery learning models use chatbots to track student progress, offer tailored exercises, and recommend learning paths based on individual needs. Additionally, chatbots can support metacognition by encouraging students to reflect on their learning strategies through guided prompts.

5.2. RQ2: How Can the Integration of AI Chatbots into Learning Management Systems Help to Enhance Student Engagement and Support Services?

Integrating AI chatbots into learning management systems can empower institutions to create a dynamic, personalized learning environment with a support system that increases their engagement and is available 24/7. As highlighted by [16], AI chatbots can serve as a tool to help aid students with disabilities. LMS platforms are often integrated with assistive technologies, and AI chatbots can further enhance their experience by offering a personal support system tailored to the individual’s disability needs. Additionally, they can act as virtual assistants by providing students with guidance and resources to help create an inclusive environment where they can contribute equally.

LMS platforms offer a bundle of resources. However, they lack features that allow students to seek information from external sources within the platform. Revisiting the viewpoints of Subramanian et al. [33], where LMS platforms can benefit through features that allow API calls to external websites, leveraging ChatGPT in LMS platforms can serve as a research tool, generating information from various sources. A positive reflection of such integration can be seen by revisiting the research by Subramanian et al. [33] on the TutorBot chatbot that showed a significant improvement in students by 90% in obtaining data and resources along with a 60% boost in efficiency compared to students who used traditional LMS and web sources.

Drawbacks in online education include the limited interaction of students with educators, resulting in delayed feedback, hindering their performance, and causing demotivation. However, educators in the traditional or online learning space are also bombarded with multiple repeated queries that can be difficult to address promptly. Integration of AI chatbots can help students receive timely feedback and ease the burden on educators in answering repeated queries. Returning to the findings discussed earlier on the implementation of an AI chatbot at HKUST to aid GTAs in answering repetitive student inquiries [8]. The research is an ideal example of how AI chatbots can help institutions automate daily tasks and create a mechanism for providing students with timely support. Furthermore, the analysis of the Chatlayer chatbot at the Singapore Institute of Technology also demonstrates the effectiveness and the benefits AI chatbots can offer to educational platforms through real-time assistance [7].

Leveraging AI chatbots in LMS can serve as a cost-effective model for managing administrative tasks. The integration can be enhanced, allowing chatbots to access academic data to generate responses tailored to users’ needs. Reflecting on Bhharathee et al. [34]’s study on AI chatbots as administrative tools, the effectiveness and accuracy of using such models in the education platform are highlighted.

ChatGPT within LMS platforms can enhance the flipped education model. Students can make use of ChatGPT to create customized learning plans and build their foundation on the course material before the class.

Instructors can use ChatGPT as a skills assessment tool to evaluate the proficiency of students in a wide range of subjects. The assessments should be designed to be interactive, where the level of difficulty is automatically adjusted as students’ progress through the test. An ideal scenario would be to conduct the assessment twice in a four-month semester course. The first round of assessment should be conducted after the first five weeks of the course have passed, allowing instructors to assess students’ skills and their ability to understand the initial concepts of the course. The data from the assessments can be used by instructors to restructure the remainder of the course content by revisiting the fundamental concepts where students failed to demonstrate the required proficiency level. Identifying and addressing learning gaps in the initial stages can assist in building a strong foundation, enabling them to understand deeper and more complex topics as the course progresses. The next phase of assessment should be four weeks before the end of the course. Feedback from this phase can highlight the areas that students most appreciate or understand, as well as their ability to grasp complex concepts. Scores and data from the assessments can be used to analyze students’ performance, areas of improvement, and learning outcomes and summarize their overall progress. Using such techniques and strategies can also help in predicting future learning outcomes based on specific course content.

Educators can use such techniques to address the learning gaps and design content that enhances overall learning. Instructors can also use these data for professional development activities where they can self-reflect on their teaching practices and identify areas for improvement. Figure 6 illustrates how ChatGPT can be used as an assessment tool by both students and educators

5.3. RQ3: What Are the Key Challenges Associated with the Adoption of AI Chatbots in Online Learning and How Can These Be Addressed?

The education system is built on key principles and foundations of academic integrity, which fosters a culture of trust, honesty, and fairness. AI technologies have become advanced now to the point where they can emulate human behavior and generate text in natural language. Research and case studies have shown that AI chatbots can revolutionize traditional learning practices and open doors to innovations; however, the adoption of such technology comes with an array of challenges. Academic integrity is a fundamental challenge faced by educational systems when it comes to adopting AI chatbots. Plagiarism is one of the core problems associated with such technology, where the information generated by AI chatbots could resemble or be the ideas and work of authors without acknowledging or giving them credit. Additionally, the content generated resembles the writing style of human-like text, making it harder to distinguish the authorship of the output [60].

As discussed earlier, one approach suggested by Wagholikar et al. [55] is to design assignments that require drawing graphical diagrams. Another solution proposed to reduce the risk of plagiarism was to conduct oral exams where students can demonstrate their understanding of the subject. Another feature that can be implemented to help in the investigation of plagiarism is the ability to disable the option of deleting the chat history.

Educators can use a hybrid model system for grading assignments where students can complete some portion of the assignment as a take-home exam and then finish the rest during class sessions. The latter part of the assignment should be revealed during class sessions and should be constructed around demonstrating the knowledge of the prior take-home exam questions. This will ensure that students reinforce their concepts and use their critical thinking skills to solve problems and not rely solely on technology.

Technical and educational organizations are making ongoing efforts to develop technology and tools for detecting AI-generated text. However, the text generated by AI chatbots such as ChatGPT is well crafted to even surpass widely known plagiarism detection tools such as Turnitin [9]. AI chatbots can also be used by students to rephrase the generated data or ideas and claim them as their own. A drawback for many educational institutions lies in the lack of funding and resources to integrate plagiarism detection tools into their platforms. This can have a devastating impact on educational systems in third-world countries, where students who have access to online resources such as ChatGPT can use them to complete their coursework and gain an unfair advantage over their peers. Submitting physical paper copies of assignments and assessments using ChatGPT is another unethical tactic that students can use to avoid being detected by plagiarism detection tools.

The accuracy and trustworthiness of data are other significant concerns when using AI chatbots in education. Producing false information can have detrimental impacts on students’ learning, leading to misconceptions of factual information. Findings from the literature review showed that ChatGPT could produce inaccuracies and misleading information, along with the possibility of generating incorrect citations. The study [59] revealed that 49.4% of the answers provided on agricultural content were incorrect, with 87% of references mainly from sources such as Wikipedia and non-scholarly articles. The research also showed that some of the scientific journals referenced had incorrect metadata [59]. The study is a good reflection of the disruption AI chatbots can bring to the education system and their severe consequences on students’ learning.

Security concerns highlighted by Wagholikar et al. [55] showed the possibility of attackers using AI chatbots such as ChatGPT to create phishing attacks and exploit the platform through code changes. Institutions that lack security policies and infrastructure can compromise their platform to vulnerable attacks through the integration of AI chatbots within their system. To mitigate and prevent security threats, institutions can enforce stringent security and data access protocols. This can include the integration of ChatGPT as part of a LMS software that uses single sign-on for user authentication. Educational institutions can adopt security practices from the corporate sector, such as auditing and monitoring instant messaging tools to ensure that employees are compliant with the firm’s policies and guidelines. Educating students on data privacy issues, security attacks, policies, and best practices is crucial in safeguarding the platform against security threats.

Integrating AI chatbots with LMS platforms can bring on another aspect of complexity, as many institutions use vendor-based software. In addition to designing complex architecture, educational organizations may also not have the resources and financial funds to support such projects. Integrating AI chatbots with LMS platforms exposes the infrastructure to a range of threats as AI chatbots generate data using models to retrieve data from external sources. Encryption techniques can be implemented to secure and protect the data; however, this may involve further complexities and challenges.

While efforts are still underway to advance NLP technology to detect human emotions through tones and expressions, technology still has limitations and cannot serve as a replacement for humans. Mosaiyebzadeh et al. [13] highlighted the lack of capabilities in AI chatbots to understand human emotions. These are the key elements in developing the basic building blocks of a teacher–student relationship that fosters trust, communication, and mentorship and builds a supportive environment.

Some other drawbacks of AI chatbots have been seen in the software development industry. Findings from the Berrezueta-Guzman and Krusche [56] experiment showed the inaccuracies and limitations of ChatGPT in problem-solving questions and also reflected the long-term impacts it could have on students’ critical thinking with prolonged use. The convenience of using tools to generate code and solve problems can negatively impact on a learner’s thought process and the ability to think dynamically in solving complicated problems.

6. Conclusions and Future Recommendations

Integration of AI-driven chatbots in educational platforms can unveil a spectrum of opportunities for educators and learners. However, adoption also comes with significant risks and challenges. AI chatbots such as ChatGPT can serve as virtual assistants and can open doors for new learning techniques, offering an interactive and personalized learning experience. Conversational agents can also reduce communication barriers and help strengthen learner’s communication skills. AI chatbots can serve as a cost-effective model for streamlining administrative tasks and offering educational support around the clock. ChatGPT can empower learners to understand complex concepts, learn programming languages, promote engagement levels in flipped education, and serve as a research assistant tool. ChatGPT can also be leveraged as an assistive tool for students with disabilities. Leveraging ChatGPT in LMS can help educators create a collaborative and dynamic learning environment. ChatGPT can open pathways to many new possibilities and techniques, such as conducting assessments to evaluate students’ performance and progress throughout the course curriculum. Educators can proactively use the data for future predictions, allowing them to plan and strategically design the course contents to meet the student’s needs and learning objectives.

Students from different demographics interact with AI chatbots in varied ways, influenced by factors such as age, academic level, digital literacy, field of study, and cultural background. Younger students and those with lower digital fluency tend to seek simple, step-by-step guidance, while university students and tech-savvy individuals engage in more complex, nuanced interactions. STEM students often use chatbots for problem-solving, while humanities students turn to them for brainstorming and writing support. Multilingual and culturally diverse students leverage chatbots for language assistance, and those with accessibility need to benefit from personalized, structured help. Additionally, socioeconomic status plays a role, as resource-limited students may rely heavily on chatbots for free educational support, while wealthier students often integrate them with other learning tools.

Though AI chatbots encompass a range of benefits, the integration with LMS platforms needs a thorough assessment to evaluate the security risks and privacy concerns. AI hallucinations and dependency on technology are other factors of concern when adopting such technology. Educators must also critically evaluate and implement strategies to maintain academic integrity and tackle ethical concerns. ChatGPT is relatively new to the education industry, and while there are ongoing research efforts to explore its full potential, there are still significant research gaps in exploring the technical complexities of integrating it with LMS platforms. There is also a lack of scholarly articles examining the details of security procedures and policies associated with AI chatbots in educational settings. To secure an ethically sustainable future of AI in education, case studies and research must be conducted to ensure best practices, awareness must be raised regarding the implications of AI chatbots on students and others alike, and lastly, training must be provided to educational administrations to ensure that they know how to wield such powerful technology.