Chatbots in Education and Research: A Critical Examination of Ethical Implications and Solutions

Abstract

A new era of education and research based on chatbots and artificial intelligence is quickly growing. However, the application of these new systems is associated with several challenges and limitations, mainly related to ethics. This paper explores the potential use of AI systems and chatbots in the academic field and their impact on research and education from an ethical perspective. Through a qualitative methodology, the researcher perform exploratory research and data collection based on expert analysis and interpretation. The researcher conducted a comprehensive review of the main potential challenges associated with the use of chatbots in education and research to identify current practices, challenges, and opportunities. This explorative work provides a foundational understanding of the studied topic. It also helps us to better understand the subjective experiences and perspectives of the observed phenomenon, and uncovers their meanings and proposes potential solutions to the observed issues. This study examines the advantages and limitations of AI systems and chatbots, as well as their role in supporting human expertise and judgment. The paper also discusses the ethical challenges related to the use of AI systems and chatbots in research, as well as the potential for misuse and exploitation. It also proposes effective solutions to the observed ethical dilemmas. The research admits that we live in a new era of AI-based education and research. The observed technological advancements will definitely shift research processes and transform educative systems, especially in term of assessments. Digital assessments are going to disappear and assessment methods need to be more creative and innovative. The paper highlights the necessity of adaptation to the new reality of AI systems and chatbots. Co-living, sustainability and continuous adaptation to the development of these systems will become a matter of emergency. Raising awareness, adopting appropriate legislations and solidifying ethical values will strengthen research and protect educational systems. The presence of AI systems and chatbots in education needs to be considered as an opportunity for development rather than a threat.

1. Introduction

The integration of AI systems and chatbots into the academic field has gained significant attention in recent years. AI technologies have the potential to transform the way research and education are conducted by automating tedious and repetitive tasks, assisting in data analysis, and enabling new forms of learning and assessment [1]. However, the adoption of AI in the academic field is not without challenges and controversies.

There have been numerous studies and articles exploring the potential benefits of using AI systems and chatbots in the academic field [2,3]. One of the main benefits is the improvement of the efficiency and accuracy of research. AI systems can quickly process vast amounts of data and identify patterns and relationships that might be difficult for humans to detect. This can lead to more efficient and effective research, enabling researchers to focus on more complex and creative tasks [4]. Another benefit of AI systems and chatbots in education is the ability to personalize learning pathways [5,6,7,8]. AI systems can analyze students’ learning styles and abilities, and provide individualized recommendations and support to help students achieve their goals. Additionally, AI systems can facilitate online learning and make education accessible to students in remote areas, leading to more equitable and inclusive education [9,10].

However, there are also concerns and challenges related to the use of AI in the academic field. One of the main concerns is the potential for AI systems to perpetuate existing biases and discrimination in research and education. Additionally, there is a risk that AI systems may be misused or manipulated to produce unreliable or biased results. At the end of the year 2022, we observed the emergence of a new AI system called ChatGPT. It is considered a language-based AI system that is part of a larger family of AI technologies known as transformers. Transformers are deep neural networks designed to process and generate sequences of data, such as text [11]. As a chatbot, ChatGPT was trained on a massive dataset of text from the internet, allowing it to generate human-like responses to a wide range of questions and prompts [12]. Artificial intelligence (AI) and chatbots have been rapidly advancing in recent years and are becoming increasingly prevalent in various fields, including the academic field. With the rise of big data and the need for efficient and fast data processing, AI systems and chatbots are being seen as a valuable tool for researchers and academics. Recent research conducted by [13] admitted that little work has been conducted studying the application of AI in the educational field. These researchers also assumed that there is a continuously increasing interest in studying the impact of AI in education and research. It is also interesting to mention that to date, only two studies have been conducted on the ethical challenges associated with the use of chatbots in the educational field. The first one was conducted Akgun and Greenhow [14] (2021), and was applied to K-12 students. The second one was conducted by Okonkwo and AdeIbijola [15] and consisted of evaluating the ethical implications of using chatbot systems in higher education. As of today, no research paper has yet studied the ethical challenges associated with the use of chatbots in education and research. In this research, the educational field includes all levels of education, from primary to higher education levels.

This paper aims to explore the potential use of AI systems and chatbots in the academic field and their impact on research and education from an ethical perspective. Consequently, the first research question (RQ1) explores how AI and chatbots could impact the education field, and their impact on the integrity of assessments. The second research question (RQ2) investigates whether chatbots are going to transform the reality of academic research. The third research question (RQ3) asks what are the potential ethical challenges associated with the use of AI and chatbots in education and research.

Overall, this study will examine the advantages and limitations of AI systems and chatbots, as well as their role in supporting human expertise and judgment. The paper will also discuss the ethical concerns related to the use of AI systems and chatbots in research, as well as the potential for misuse and exploitation. The study will utilize an exploratory research approach, with a qualitative research methodology, to collect data through expert analysis and interpretation. The research findings will provide insights, future perspectives, and possible developments related to the use of chatbots in education and research. The study intends to contribute to the ongoing discussion on the ethical challenges of education and research transformation through the use of AI systems and chatbots.

2. Literature Review

The integration of artificial intelligence (AI) and chatbots into education and research has become more prevalent in recent years, especially as of the end of 2022. Chatbots are automated conversational agents that use natural language processing and machine learning algorithms to interact with users in a human-like manner. However, the increasing use of AI and chatbots in these fields also raises ethical challenges that need to be addressed [14]. This literature review aims to explore the ethical challenges of using AI and chatbots in education and research, with a focus on the major observed issues.

Ref. [1] provided an overview of the current state of AI in education and its potential benefits, including personalized learning, increased accessibility, and improved efficiency. The authors also discussed some ethical challenges associated with the adoption of AI in education. Zhang and Aslan [3] made a comprehensive review of AI in education and its potential benefits, including improved student engagement and motivation, enhanced assessment and feedback, and increased efficiency and cost-effectiveness. Pedro et al. [16] highlighted the advantages of the integration of AI in the educational field. The researchers admitted that AI in education can automate repetitive tasks, free up more time for complex cases, and upskill the workforce to become AI-ready.

Pedro et al. [16] also discussed the challenges and policy implications of introducing AI into education and preparing students for an AI-powered context. The challenges include developing comprehensive public policies for sustainable development, ensuring inclusion and equity, preparing teachers for AI-powered education, developing quality and inclusive data systems, making research on AI in education significant, and addressing ethical concerns related to data collection, use, and dissemination. These challenges require international and national partnerships and public discussions on ethics, accountability, transparency, and security. The authors also addressed some of the ethical and technical challenges associated with the adoption of AI in education. The authors mentioned the sixth challenge of introducing AI in education. Pedro et al. [16] admitted that the concerns surrounding AI and its impact on education include access to educational systems through machine learning algorithms, potential discrimination, inadequate recommendations for certain groups of students, personal data concentration, liability, and the impact of AI automation on teacher jobs. The challenge lies in protecting personally identifiable information and privacy preferences, particularly for young learners who cannot provide express consent. Pedro et al. [16] consequently mentioned the need to address the ethical concerns related to data collection, use, and dissemination.

An exploratory study conducted by Tlili et al. [17] investigated the use of conversational agents, including ChatGPT, as a tool for enhancing online learning experiences. They found that students preferred using conversational agents for learning activities, as they provided a more engaging and interactive experience. Further, Kuhail et al. [18] found that chatbots can provide students with instant feedback and support, as well as personalized learning experiences. The authors also found that chatbots have the potential to increase student engagement and motivation in learning. Similarly, another study [19] explored the potential use of chatbots in higher education. The research showed that the use of chatbots in the first year of the university studies eases the transition of students into their first year of university, and increases their study engagement. The findings showed a positive correlation between the use of chatbots, study engagement and engagement with peers. Students reported that the chatbot helped them obtain support and connect with their program leader.

A recent newspaper article [20] admitted that some school districts and a college in Hamilton (Canada) and its neighboring areas are being vigilant against any attempts made by students to cheat using ChatGPT. The article also confirmed that students use ChatGPT to produce essays or answer assessments.

The observed advantages and disadvantages of the use of AI in education need further investigation, especially with the emergence of the latest powerful AI based chatbot, named ChatGPT. Thus, the (RQ1) explores how AI and chatbots could impact the education field.

To successfully achieve his mission, a researcher needs to be supported by a research assistant that plays a crucial role in supporting their work and ensuring the smooth operation of a research project. Stevano and Deane [21] described the role of a research assistant as assisting in the design, execution, and analysis of research projects, as well as in the preparation of research reports and presentations. Turner [22] recognized the vital role played by research assistants. She admits that they play an integral role in the process of producing knowledge. Consequently, it is important to acknowledge their contributions to our understanding of field experiences and the outcomes we produce. From their side, Johnson and Harris [23] highlighted the contributions of research assistants in various stages of the research process, including data collection, data analysis, and manuscript preparation. Briefly, we maintain that research assistants play a crucial role in the success of academic research by performing tasks that contribute to the design, execution, and analysis of research projects, as well as in the preparation of research reports and presentations. Among the scarce papers that discuss the role that chatbots could play as research assistants, we highlight the one written by Araujo [24]. That study introduces the Conversational Agent Research Toolkit (CART), a tool designed to assist researchers in building conversational agents for experimental studies. The paper provides an overview of the tool and a step-by-step tutorial for designing an experiment with a chatbot. Another research paper [24] published on the 11th of March 2023 highlighted the future collaboration between researchers and chatbots. This research admitted that in the short term, chatbots are going to serve as research assistants for desk research and support the idea of hybrid work. So, again, in the face of a scarcity of research in the discussed area, the second research question (RQ2) investigates whether chatbots are going to transform the reality of academic research.

The use of chatbots by students has become increasingly prevalent in recent years, as technology continues to advance and education moves towards online and hybrid models. While chatbots can provide students with quick and convenient access to information, they also present a number of dangers and ethical concerns. Recent research [25,26] analyzed the development and implementation of a chatbot designed to support student–teacher interaction. The chatbot was integrated into an online platform used in a university course and provided assistance to students regarding course content and assignments. The results show that the chatbot was positively evaluated by the majority of the students and was seen as a useful tool for facilitating communication with their teachers. The paper concludes that chatbots have the potential to enhance student–teacher interaction and improve the overall learning experience.

A study by King on ChatGPT [27] explored the ethical challenges associated with the use of chatbots in educational assessments. The authors highlighted the potential for cheating and the need to ensure the integrity of assessments when using chatbots. They also noted the importance of considering the ethical implications of AI systems and the need to ensure that chatbots are unbiased and fair. Thus, the use of chatbots in educational assessments has a potential for cheating. Students may use chatbots to cheat during exams or other assessments by inputting questions and receiving answers in real time. This undermines the integrity of assessments and can result in unfair advantages for students who use chatbots.

Furthermore, the use of chatbots in assessments and exams raises questions about the role of technology in education. The research of Alam [28] argues that the use of chatbots in assessments and exams prioritizes technological solutions over traditional pedagogical methods, potentially leading to a devaluation of the role of human teachers and educators.

One ethical challenge associated with the use of chatbots in education is the potential for the technology to replace human interaction and expertise. This is particularly concerning in fields such as counseling and mental health, where students may seek emotional support from chatbots instead of trained professionals. A study by Pesonen [29] found that although chatbots were perceived as useful and helpful by college students, they were not seen as a substitute for human support.

Another ethical challenge is the potential for bias in chatbots. AI systems are only as unbiased as the data they are trained on. If the data used to train chatbots are biased, then the chatbot’s responses may also be biased. This could result in unfair assessment outcomes and could potentially perpetuate discrimination and inequality in the education field [16].

Chatbots have emerged as a promising educational tool, with the potential to enhance the learning experience by providing personalized and immediate feedback to students. However, the use of chatbots in the educational field also raises ethical challenges that need to be addressed.

3. Materials and Methods

This study was exploratory and interpretivist in nature. It relies on the interpretivism philosophy that refers to the idea that human behavior and meaning are socially constructed and subjective, and that knowledge and understanding can only be gained through interpretation and meaning-making. In this approach, the researcher strives to understand the subjective experiences and perspectives of the observed phenomena and seeks to uncover their meanings and interpretations. The focus is on the social construction of reality, and the researcher aims to understand the world through the observed phenomena.

The study required extensive, in-depth, and subjective information to address its objectives, so it relied on qualitative information. The collected secondary and qualitative data were analyzed using a thematic analytical framework to construct themes that aligned with the study’s goals and questions. The study has tried to discuss in-depth the ethical dilemmas cropping up in the education sector due to the ongoing developments of artificial intelligence systems and chatbots. So, mainly it follows an exploratory research method that aims to explore the new phenomenon of the use of chatbots in education and research. The goal is to gain a better understanding of the topic and generate new ideas and hypotheses. The paper aims also to explore the phenomenon in-depth and identify any relevant factors that may need to be investigated further.

An exploratory research approach could be useful in this scenario, because it would allow researchers to gather preliminary information and insights on the potential use of AI systems and chatbots in the academic field, and the ethical considerations associated with their use. According to Creswell [30], exploratory research is used to gain familiarity with a phenomenon or to develop new insights and hypotheses about it. In a previous research, Følstad and Taylor [31] performed a qualitative study based on an exploratory research approach and methodology that sought to analyze the use of chatbots in customer service. Similarly, the actual research also provides valuable insights into the potential benefits and drawbacks of AI systems and chatbots in education and research, as well as the ethical implications of their use.

As an expert in higher education, the researcher here analyzed this new phenomenon and presents his insights, future perspectives and possible developments as regards the use of chatbots in education and research. The researcher used a qualitative research methodology to collect data through expert analysis and interpretation. The opinions of leading expert are considered in this study, following Foerster’s [32] initial suggestion and several subsequent studies that adopted an expert-based approach to various topics related to information systems (e.g., [24,33,34]). The author analyzes the significant viewpoints of the challenges and impact of ChatGPT and related generative AI technologies.

The researcher here conducts a comprehensive review of the main potential challenges associated with the use of chatbots in education and research to identify current practices, challenges, and opportunities. This explorative work will provide a foundational understanding of the topic and inform subsequent research designs. We analyze the significant perspectives regarding the impact and essential challenges arising from the widespread usage of ChatGPT and other generative AI technologies. The research paper includes two main steps. The first one consists of identifying the challenges associated with the application of ChatGPT and chatbots in education and research. The second step relies on expert knowledge to address the avoidance of the misuse of chatbots in education and research following an ethical focus.

4. Results

4.1. Artificial Intelligence, Chatbot and Education

The first research question (RQ1) explores how AI and chatbots could impact the education field, and what their impact is on the integrity of assessments. As a development in AI, the concept of chatbots can be traced back to the early days of computing, with early computer programs being capable of performing simple, repetitive tasks. However, it was not until the advent of artificial intelligence (AI) and natural language processing (NLP) technologies that chatbots became more advanced and capable of human-like interactions.

In the 1990s, researchers began developing conversational agents, or chatbots, that could understand and respond to human inputs in natural language. These early chatbots were limited in their capabilities, but they laid the foundation for the development of more sophisticated chatbots in the following decades.

With the growth of AI and NLP technologies, chatbots have evolved significantly, becoming more capable of handling complex tasks and undertaking more human-like interactions. Today, chatbots are used in a wide range of industries, including customer service, healthcare, finance, and education, among others. Overall, the development of chatbots is a result of advancements in AI and NLP technologies, and the increasing demand for more efficient and convenient ways to interact with technology. One of these NLP technologies that is frequently used in the education sector is ChatGPT. As a variant of the GPT (Generative Pretrained Transformer), it constitutes a language model developed by OpenAI. It was trained on a massive amount of text data, allowing it to generate human-like text and answer questions. The original GPT model was introduced in 2018 by researchers at OpenAI and was based on the transformer architecture, which is a neural network designed for processing sequential data such as text. The model was trained on a large corpus of text data, which allowed it to learn patterns and relationships between words and sentences. The latest version of ChatGPT was launched at the end of 2022, and was trained on conversational data and fine-tuned for specific tasks, such as answering questions or generating chatbot responses. ChatGPT was trained on a massive dataset of text from the internet, allowing it to generate human-like responses to a wide range of questions and prompts. The development of ChatGPT and other GPT models has been a significant step forward in the field of NLP (Natural Language Processing) and AI, allowing for the creation of more advanced and human-like conversational AI systems.

Personally, as an educator and researcher, I made an attempt to check the performance of ChatGPT. I was surprised to see that the AI chatbot was able to perfectly answer the multiple-choice questions of one of my university assessments. It is important to note that relying on a chatbot or any other tool to answer exam questions goes against the fundamental aims and principles of education, and can lead to a superficial understanding of the material and hinder the development of these essential skills.

Using chatbots or any other artificial intelligence tools to answer exam questions is a form of cheating and academic misconduct, and goes against the fundamental principles of learning and academic integrity. Therefore, if misused by students, chatbots could generate serious ethical concerns in education. It could also severely affect the student’s academic progress and knowledge-acquiring processes by hindering their critical thinking skills, creativity, and ability to apply the concepts learned to real-world situations.

Additionally, using chatbots during exams and assessments could affect the reliability of assessments. The reliability of the assessment process refers to the consistency and stability of the results obtained from the assessment. A reliable assessment process should yield consistent results that reflect the real qualifications and competencies of the students. In the context of exploring the potential of AI systems and chatbots in the academic field and their impact on research and education from an ethical perspective, it is important to ensure the reliability of the assessment process in order to ensure that the results obtained are valid and useful for decision-making. However, students may use chatbots to cheat on exams or assignments, which could compromise the reliability and validity of the assessment results. Receiving help from chatbots leads to unfair advantages and inaccurate evaluations of the students’ knowledge and skills.

One of the main dangers of chatbots is the potential for cheating during assessments, exams, and projects. Chatbots can provide students with immediate answers to questions, which can lead to academic dishonesty and a lack of learning. In addition, the use of chatbots can create an uneven playing field, as some rich students may have access to better or more advanced chatbots than others. So, it is essential that students put in the necessary effort and time to learn the course material and acquire the necessary knowledge and skills through legitimate means. Additionally, it is important to uphold academic honesty and ethics in all learning environments.

Furthermore, in the future, we could observe raised concerns about the impact of chatbots on the education levels of future generations. We could argue that the use of chatbots could lead to a decrease in critical thinking skills and a lack of independent problem-solving abilities. This could result in a generation of students who are overly reliant on technology and unable to think critically or creatively. Future longitudinal qualitative and quantitative research needs to better explore these aspects.

4.2. Chatbot as Research Assistant

The second research question (RQ2) investigates whether chatbots are going to transform the reality of academic research. On the research side, basically, ChatGPT can aid researchers in writing academic papers by providing writing assistance and suggestions for phrasing, tone, and style. It can also help with fact-checking and provide quick access to relevant information and data. Additionally, ChatGPT can automate repetitive tasks, such as data entry and formatting, freeing up more time for researchers to focus on higher-level tasks, such as analysis and interpretation. However, it is important to note that while ChatGPT can assist in the writing process, the ultimate responsibility for the content and quality of academic papers remains with the human researcher. Thus, it becomes important to explore the mission of a researcher and how they act to develop knowledge.

A scientific researcher aims mainly to contribute to the advancement of knowledge and the improvement of society through research. They formulate research questions and hypotheses, and design and conduct experiments or studies by collecting, analyzing, and interpreting data. Finally, a researcher writes up results and publishes findings in academic journals or other outlets.

Right now, the task of the research assistant is performed by humans. In the execution of their mission, they may recourse to technology in order to quicken and facilitate the research mission. Through the emergence of AI systems, such as chatbots, it will become interesting to explore the idea of using these intelligent systems as research assistants.

AI-powered chatbots can serve as an excellent research assistant in the academic field. First, they are considered as efficient data collection tools. In fact, chatbots can efficiently collect and process large amounts of data and provide researchers with relevant information for their studies. Second, unlike human research assistants, AI research assistants have an unlimited availability and capacity of production. Thus, chatbots can be accessible 24/7, allowing researchers to collect data and receive information at any time, regardless of their location. Third, they provide personalized services. Consequently, chatbots can personalize the information they provide based on the researcher’s preferences, making their research experience more objective, efficient and personalized. On the other hand, human research assistants may introduce subjectivity and bias into the research process, which chatbots can help to mitigate. Fourthly, by using artificial intelligence we could generate an improved data quality. So, chatbots can assist researchers in collecting high-quality data by providing consistent and accurate information, reducing the risk of human error. Fifth, AI systems are characterized by automation. In fact, chatbots can automate repetitive and time-consuming tasks, freeing up researchers to focus on more complex and important aspects of their research. Finally, these systems show improved collaboration. Chatbots can support researchers in collaborating with colleagues and sharing information, improving the overall quality of their research. Chatbots can improve collaboration among researchers by enabling them to share information and work together more efficiently. For example, a chatbot could be programmed to provide quick access to research databases, helping researchers quickly find relevant studies and articles. A chatbot could also facilitate communication between team members by sending automated reminders and updates, reducing the likelihood of missed deadlines and miscommunication. Additionally, chatbots can be used to automate certain research tasks, freeing up researchers’ time and allowing them to focus on more complex, creative aspects of their work. Chatbots could also help to reduce inequalities between researchers. Mainly in developed countries, researchers have difficulties with financing their research and hiring research assistants. They also may not be able to translate their research or publish in other useful languages. So, chatbots could offer an opportunity for these underserved researchers to benefit from the previously mentioned advantages.

Ultimately, the use of chatbots in research can lead to improved collaboration, better-coordinated efforts, and higher-quality research outcomes. However, these chatbots need to fulfill support needs and cannot replace the essential role of the researcher. Like human research assistants, these chatbots need close and continuous supervision in order to avoid derivations.

The previous analysis helps to answer the second research question by explaining how chatbots could transform the reality of academic research. However, despite all the advantages provided by the artificial intelligence systems in the research field, several voices are raised to rationalize the use of these machines, as they raise several ethical concerns. Several reasons could be advanced against the delegation of the research assistance mission to chatbots. AI systems can be trained on biased data, leading to biased results. This can compromise the validity of research findings and harm communities. Additionally, these machines could lack the expertise and knowledge of human researchers, and may produce results that are inaccurate, incomplete, or irrelevant.

Again, chatbots lack the ability to understand context and nuance, which is essential in many areas of research. This can result in errors and the misinterpretations of data. Sometimes, chatbots lack the creativity and critical thinking skills of human researchers. This can limit the scope of research and the potential for new discoveries. Another major concern is linked to the lack of empathy. Chatbots lack the ability to understand emotions, empathy, and human behavior, which are important aspects of many areas of research. On another side, the results produced by chatbots may not be trusted by participants, stakeholders, and other researchers. This can harm the credibility of research and limit its impact. Finally, the use of chatbots in research raises ethical concerns, such as those related to data privacy, data security, and exploitation of participants. Therefore, chatbots should not be used as a substitute for human researchers, but as a complementary tool to aid in research, under the supervision and control of human experts.

4.3. Artificial Intelligence and Ethics in Education and Research

The third research question (RQ3) asks about the potential ethical challenges associated with the use of AI and chatbots in education and research. It is important to use chatbots as an aid to human researchers, but not as a substitute, and to critically evaluate and verify the information provided by chatbots before using them in research. This can help to ensure that research is accurate, reliable, and ethical.

As they rely on pre-programmed algorithms, chatbots could show serious limitations in understanding. These systems lack the ability to understand context and nuances in the same way humans do. Additionally, the data that chatbots use to generate answers and recommendations may contain biases and errors that can affect their responses and generate bias in data.

Again, chatbots lack creativity and the ability to generate new ideas, which are essential for advancing scientific and academic research. Additionally, the results generated by chatbots often require human interpretation and evaluation to be meaningful and actionable. Finally, the use of chatbots in research raises ethical concerns about the accuracy and reliability of the data collected, and the potential for chatbots to perpetuate harmful biases and discrimination.

Once we compare between human research assistants and AI research assistants, we can conclude that both of them require close observation and outcome verification from the main researcher. However, the machines’ performance could be much better than the human’s. The degree of innovation and creativity could also be higher in machines compared to humans, as they could show higher capacities in terms of data analysis, experience and expertise. On the other side, human research assistants could show a better awareness and understanding of the context and nuances.

With the fast technological development characterizing the 21st century, the reliance on AI systems is becoming an obligation and not a choice. Consequently, we need to carefully pay attention to the limitations of these systems, especially in regard to the ethical challenges. Their early detection and better understanding could help us to mitigate their potential harms. Nowadays, the use of chatbots in research raises several ethical concerns as regards data privacy, bias in data, responsibility for accuracy, lack of transparency and potential for misuse.

In regard to data privacy, chatbots collect and store large amounts of personal data, which raises concerns about data privacy and security. Additionally, the data used to train chatbots may contain biases and errors that can affect the accuracy of their responses, leading to unintended consequences and discriminatory outcomes. Again, it is unclear who is responsible for the accuracy of the data generated by chatbots, which can affect the validity and reliability of research results. Moreover, chatbots often operate behind the scenes, making it difficult for researchers to understand how they make decisions and generate answers, which raises concerns about the transparency and accountability of their results.

Finally, chatbots can be used to manipulate or exploit individuals and communities, and their results can be used to make decisions that harm or disadvantage specific groups. As an example, through frequent questions that could be asked by users about political conflicts, the answers provided by chatbots could mislead general opinion. As a concrete example, in the case of the Russian/Ukrainian conflict, the programmers and developers could train chatbots to provide positive or negative answers to the public based on their own political interests.

This potential for the misuse of chatbots is considered as the most dangerous and harmful one. Chatbots have the potential to be misused in a number of ways, which can harm individuals and communities, including spreading misinformation, manipulating public opinion, exploiting vulnerable populations, reinforcing biases and discrimination, and impacting the accuracy of research. As an example, chatbots could be programmed to spread misinformation and propaganda, which can have a harmful impact on public opinion and decision-making. Chatbots can also be used to manipulate public opinion by suppressing certain viewpoints and amplifying others, leading to biased and misleading results. Chatbots can be used to exploit vulnerable populations, such as the elderly or individuals with limited digital literacy, by presenting false or misleading information.

From another point of view, the data used to train chatbots may contain biases and errors that can perpetuate discrimination and harm marginalized groups. Finally, chatbots can produce inaccurate results that are used to make decisions that harm or disadvantage specific groups, such as in the context of medical research or public policy.

A few concrete examples are necessary to better understand the potential misuse of chatbot in research and public policy. In medical research, chatbots trained on biased data can produce inaccurate results, leading to harm to patients or communities. For example, a chatbot trained on a biased dataset may incorrectly identify a certain medical condition in a patient and suggest inappropriate treatment, causing harm to the patient. As a concrete example, a study published in the Journal of the American Medical Association (JAMA) [35] found that an AI system used to diagnose skin lesions had a high rate of false positive results, leading to unnecessary biopsies and patient anxiety.

In regard to public policy, chatbots can be used to manipulate public opinion and influence policy decisions, potentially leading to harm to specific communities. For example, a chatbot programmed to spread false information about the benefits of a certain policy may lead to the adoption of a harmful policy.

Chatbots can also be used to manipulate market research by amplifying certain viewpoints and suppressing others, leading to biased and misleading results. This can have significant impacts on the economy, for example, a chatbot programmed to spread false information about a company’s financial performance could lead to a drop in the company’s stock price.

In academic research, chatbots can be used to produce inaccurate or biased results in academic research, leading to the dissemination of false or misleading information. For example, a chatbot trained on biased data may produce inaccurate results in a social science study, leading to false conclusions that could harm marginalized communities. It can also over-exaggerate, citing certain scientific journals or researchers to inflate its citing index.

These examples help to answer the third research question by discussing potential ethical challenges associated with the use of AI and chatbots in education and research. Finally, in public opinion surveys, chatbots can be used to manipulate public opinion by amplifying certain viewpoints and suppressing others, leading to biased and misleading results. This can have significant impacts on the democratic process. For example, a chatbot programmed to spread false information about the popularity of a political candidate could lead to an incorrect outcome in an election. These are a few examples of the potential harm that can be caused by the misuse of chatbots in research.

4.4. How to Avoid the Misuse of Chatbots in Research

The second research question (RQ2) investigates whether chatbots are going to transform the reality of academic research. This section aims to provide advice on how to avoid the misuse of chatbots in research.

It is important to be aware of the previously presented concerns in regard to the misuse of chatbots in research, and to implement measures to mitigate the associated risks. Mainly, the results generated by chatbots often require human interpretation and evaluation to be meaningful and actionable. Additionally, the use of chatbots in research should be transparent and their presence should be disclosed to participants and stakeholders. Chatbots can be trained on biased data, leading to biased results. To mitigate this risk, researchers should be mindful of the data used to train chatbots, and implement methods for detecting and mitigating bias in the results produced by these AI systems. Researchers and their human assistants should show a high degree of awareness. The results produced by chatbots should be independently verified using human experts to ensure their accuracy and validity. On the legal side, there should be clear regulations in place to govern the use of chatbots in research, to ensure that they are used responsibly and ethically. Researchers should also work with experts in the field of artificial intelligence and ethics to develop best practices for the use of chatbots in research. Additionally, the public should be made aware of the potential dangers of chatbots in research and the need for measures to mitigate these risks. In regard to the ethical considerations, researchers should be aware of the ethical implications of using chatbots in research and ensure that their use does not violate the rights of participants or harm communities. By implementing these measures, researchers can help to ensure that chatbots are used responsibly and ethically in research, and that their results are accurate, reliable, and trustworthy. These recommendations can help address the second and third research questions, usher in a new era of academic research, and prevent potential and ethical challenges associated with the use of AI and chatbots in education and research.

4.5. How to Avoid the Misuse of Chatbots in Education

The first research question (RQ1) explores how AI and chatbots could impact the field of education and their potential impact on the integrity of assessments. Additionally, the third research question (RQ3) examines the potential ethical challenges associated with the use of AI and chatbots in education and research. This section aims to provide guidance on how to prevent the misuse of chatbots in education.

The use of AI by students in education can raise ethical concerns on several aspects. AI technology has the potential to be used by students to cheat on assessments and exams, which undermines the integrity of the education system and devalues the efforts of other students who have earned their grades honestly. AI systems can also be used to generate essays or other written assignments, which can lead to instances of plagiarism and devalue the learning experience for all students. Additionally, when students rely on AI to complete their work, they may not fully understand the material and may lack a sense of ownership over their learning. This can reduce the effectiveness of education and limit their ability to apply their knowledge in the future. Such situations can also provide some students with an unfair advantage over others, particularly if not all students have access to the technology or if it is used in an unequal manner. To prevent the potential negative effects of AI system misuse in education, professors need to assess students’ skills. They also need to discourage the use of AI systems, such as chatbots, by incorporating a variety of innovative assessments.

In this new era of chatbots, educational institutions need to incorporate assessments that require students to think critically, be creative, and apply their knowledge in novel and unpredictable ways that can be difficult for AI systems to replicate. Incorporating open-ended questions in assessments could encourage creativity as students are required to provide in-depth and original responses, which can be difficult for AI systems to replicate. Assessments need also to focus on problem-solving tasks. Assessments containing complex, real-world problems can help demonstrate a student’s ability to think critically and apply their knowledge in new and innovative ways. Educators are also encouraged to evaluate their students through projects that require original solutions or prototypes that can help demonstrate their creativity and problem-solving abilities. Requiring students to give presentations on a topic of their choice, in which they must incorporate original ideas and perspectives, can help demonstrate their creativity and ability to communicate effectively. Encouraging the use of interactive quizzes is also effective as these require students to actively engage with the material and respond to questions in real-time, rather than just passively reading or watching a lecture.

As a second strategy, students need to be assessed through hands-on tasks. Practical tasks can help discourage the use of AI systems, as these tasks are often difficult for AI to complete accurately. As an example, students could be assigned laboratory experiments in science or engineering courses where they need to record their observations and results. Students could also be evaluated via hands-on projects where they need to build or construct a physical product or model, such as a bridge, robot, or machine. Field works could also be incorporated into the work in subjects such as geography, biology, or environmental science, where students must gather data and make observations in the field.

As a third effective student assessment strategy, instructors need to encourage active learning through discussions, group work, and interactive assessments. This strategy can help discourage the use of AI systems and promote deeper learning. In class, students are encouraged to actively participate in discussions, listen to others, ask questions, and share their own thoughts and ideas. In group work such as projects or assignments, students must work together to complete a task or solve a problem. The use of interactive quizzes is also encouraged, as it requires students to engage with the material and respond to questions in real-time, rather than just passively reading or watching a lecture. Additionally, holding class debates where students must research and present arguments is highly encouraged. Assessing students through case studies is also a good example of an effective student assessment strategy. Using case studies that require students to analyze real-world scenarios and apply their knowledge to solve complex problems could help to mitigate effects of the excessive use of AI.

A fourth practical assessment strategy involves performing authentic assessments. To discourage the use of AI systems, students need to be assessed through techniques that are grounded in real-world scenarios and reflect the types of tasks and responsibilities students will face in their future careers. As an example, educators could assign performance tasks that require students to demonstrate their skills and knowledge in real-world situations, such as role-playing, a scenario, or presenting a project to a panel of experts. Internships or work-based learning opportunities could also push students to apply their skills and knowledge in real-world settings, and receive feedback from professionals. Another example could be conjugated in service-learning initiatives. Here, students need to engage in community service projects or volunteer work, where they must apply their knowledge and skills to address real-world problems and contribute to their communities. Portfolios and capstone projects are also good examples of authentic assessments.

As a fifth strategy that could discourage the use of AI systems, educational institutions and educators need to implement strong anti-cheating measures. This could include secure exam environments, monitored testing conditions, and plagiarism detection software. On the practical side, future exams need to be complex, multi-faceted, or based on open-ended questions that are conducted in a proctored environment. The return to paper-based exams also becomes a matter of emergency. Digital assessments, e-assessments, online assessments, and computer-based assessments may need to be reconsidered over paper-based exams. Further, future assessments need to include verbal or oral components, such as presentations or oral exams, which are difficult for chatbots to replicate. On another side, software developers and programmers are highly encouraged to develop and incorporate plagiarism detection software. Finally, professors need to educate students on the ethical implications of using AI in education and the importance of developing their own knowledge and skills, rather than relying on AI systems to do the work for them. This can help foster a culture of academic integrity and discourage the use of AI systems in assessments. All of these recommendations will help to answer the first and third research questions, while also facilitating a new era of education and research supported by AI and chatbots, and avoiding the potential ethical challenges associated with their use.

5. Conclusions

There are a variety of AI tools and techniques that can be used in the education field to help researchers, educators or students. These methods are constantly evolving as AI technologies advance. This paper has reviewed their utility as well as the ethical challenges that might emerge from their excessive use.

This study admits that the potential benefits of AI systems and chatbots in the academic field are substantial, and their use is likely to increase in the coming years. However, to fully realize the potential use of AI in research and education, it is important for researchers and educators to critically evaluate the ethical and technical implications of AI systems and ensure that they are used in a responsible and transparent manner. The actual study provides a solid foundation for exploring the potential use of AI systems and chatbots in the academic field and their impact on research and education. It supports the arguments that AI has the potential to improve efficiency and accuracy in research, personalize learning experiences, and make education more accessible and inclusive. However, it also highlights the need to critically evaluate the ethical and technical implications of AI and ensure that it is used in a responsible and transparent manner. This conclusion helps to answer RQ1, exploring how AI and chatbots could impact the education field and their impact on the integrity of assessments.

These mentioned ethical concerns highlight the importance of using AI in education in a responsible and equitable manner, and of ensuring that students are not solely reliant on AI to complete their work. It is also important to consider the potential consequences of using AI in education and to weigh the benefits with the potential risks. This conclusion helps to answer the third research question (RQ3) that asks about the potential ethical challenges associated with the use of AI and chatbots in education and research.

The paper suggests that AI systems and chatbots should be used as an aid and not a substitute for human expertise, judgment, and creativity. Once they are used in research, it is essential for researchers to critically evaluate the information provided by chatbots and verify it before using it. The paper highlights the innovative role of chatbots as machine research assistants that could complement the roles played by human research assistants. This conclusion helps to answer RQ2 that aimed to investigate whether chatbots are going to transform the reality of academic research.

The paper also suggests the recourse to innovative assessment methods to not only discourage the use of AI systems, but also provide valuable opportunities for students to showcase their creativity, critical thinking, problem-solving and collaboration skills, which are essential for success in many careers. This conclusion help to answer RQ3 and RQ1.

The paper suggests also a set of anti-cheating measures that can help ensure that students are demonstrating their own knowledge and skills, and discourage the use of chatbots or other unauthorized resources in the assessment process. These can also help maintain the integrity and reliability of the assessment process.

This paper could serve as a reference study that increases the awareness of education actors about the need to be aware of the potential for misuse of chatbots and to implement measures to sustainably address these concerns. It could also help educators and researchers to better understand the ethical challenges associated with the excessive use of chatbots and AI in education and research. The proposed set of innovative assessment methods could help educators to counteract the excessive use of AI by students. Social policy makers could use this research as an effective reference tool that guides them to elaborate new educational policies and guidelines.

Finally, this paper invites AI system developers to act in a more ethical, transparent and responsible manner once they develop such systems that could cause harm to researchers, educators and students. The potential for misuse of AI intelligence systems and chatbots could destroy the aim of education as a mechanism of knowledge and capacity-building processes.

Without doubt, we will soon experience a new era of education and research. Chatbots will invade the educational sector and will definitely shift the research processes. Ignoring or evading the presence of AI in our life is not realistic. We need to adapt our academic systems to the new AI systems and chatbots. Co-living, sustainability and continuous adaptation to the development of these systems becomes a matter of emergency. Raising awareness, adopting appropriate legislations and solidifying ethical values will strengthen research and protect educational systems. The presence of AI systems and chatbots in education needs to be considered as an opportunity of development rather than a threat.

As a qualitative study, this paper attempts to gain a more complete understanding of the ethical issues associated with the use of chatbots and AI systems in education and research. Future research needs to test the observed dilemmas and develop more effective solutions through quantitative research. The results of exploratory research can help inform the design of more focused research studies, and can also provide a foundation for developing hypotheses and research questions that can be tested through more rigorous research methods. Overall, an exploratory research approach can be an effective way to gain a better understanding of a new phenomenon, and to identify important questions and directions for future research.