Constructing a Socio-Legal Framework Proposal for Governing Large Language Model Usage and Application in Education

Abstract

Due to the fast-changing environments caused by artificial intelligence development, the socio-technical challenge in contemporary educational systems focuses on the need for more regulative measures guiding system stakeholders’ behavior. In fulfilling the present legal gap, enacted soft law regulation has been laid out, and a detailed systematic literature review was conducted in the paper presented. The specific methodological approach was selected to deal with two crucial research tasks: to reveal and recommend fundamental governing mechanisms regarding the use and application of generative artificial intelligence; more precisely, large language models in educational systems. Three systematically guided layers of quantitative and qualitative content analysis of central policy, legislation, and regulatory mechanisms in governing AI in education were extracted from the 142 Scopus Database and Web of Science research papers analyzed and presented. These research findings benefit policymakers, regulatory and legislative bodies, and agencies in constructing governing frames for using and applying generative artificial intelligence in education.

1. Introduction

In the context of the prevailing technological advancements in contemporary society (Tegmark 2018), individuals actively shape and cultivate their perception of reality (Berger and Luckmann 2016) in alignment with the prevailing social framework. The pre-norming phase of human social behavior is a social phenomenon that may be attributed to individuals’ inherent intellect and creativity (Durkheim and Durkheim 1982). Given that novelty and creativity are distinctive human traits in the present day, the fast development of technology necessitates a flexible and adaptive regulatory framework that can accommodate new models of socially influenced behaviors.

Isaac Asimov established the fundamental legal framework for artificial intelligence in his Laws on Robotics (Asimov 1950). Today, owing to the rapid changes and advancements in the area, an AI framework that can effectively use these assertions and progress towards a more advanced level of approximation is needed.

The term “artificial intelligence” (AI) refers to intelligence displayed by machines, especially computer systems. It is an area of computer science study that creates and examines techniques and software that allow machines to sense their surroundings and utilize intelligence and learning to make decisions that will increase the likelihood that they will reach predetermined objectives (Russell and Norvig 2021).

In the last ten years, artificial intelligence and generative artificial intelligence (Gen AI) have rapidly developed. Generative models can create text, images, sounds, video, and other content (Pinaya et al. 2023). An event that stands out globally in the development of generative models is the appearance of the ChatGPT service, which at the end of 2022 provided easy access to one of the best language models, GPT-3.5. From that moment, educational institutions faced a significant challenge: regulating generative artificial intelligence in general and large language models (LLM) in particular.

This paper provides a systematic review of the literature related to the problem of regulating large language models in education. Based on the authors’ analysis and guidelines, the recommendations given summarize the currently collected knowledge and ideas for solving this problem. With this paper, the authors will try to answer the following research question: What are the recent literature recommendations for governing large language models in education?

2. Framing Current Achievements in the Field: State-of-the-Art

The first part will provide an overview of the currently available guidelines related to artificial intelligence and education by supranational institutions. The positions of UNESCO and the European Union are described here. In addition, Stanford University’s “The AI Index 2024 Annual Report” is analyzed for its importance and impact.

The second part will provide an overview of approaches for discovering the use of text generated by large language models, as well as development guidelines.

2.1. Current Soft Law Regulative Enacted Overview

As a world leader in Education, UNESCO, due to the significant emergence of publicly available generative tools and the lack of national regulations, adopted guidelines with the goal: “to support countries to implement immediate actions, plan long-term policies and develop human capacity to ensure a human-centred vision of these new technologies” (Holmes and Miao 2023). The guidelines aim to provide GenAI with a tool to positively affect the research work of professors, teachers, students and pupils. Following the ethical guidelines for reliable artificial intelligence created by the High-level Expert Group on Artificial Intelligence established in 2018 by the European Commission, three frameworks are provided: (1) compliance with legal regulations; (2) respect for ethical principles and values; (3) act, both from the technical and societal frame, in good faith, and if possible, eliminate the potential effect of unintentional damage (European Commission 2019, 2022).

Following these ethical guidelines, the steps foreseen are:

(a)

Implementation of international or regional data protection regulations (GDPR) is necessary; otherwise, data protection could be questioned. Also, first of all, it is necessary to provide the legal framework for collecting and processing personal data.

(b)

Establish/revise the entire management system to implement the artificial intelligence strategy.

(c)

Implement special ethical regulations on artificial intelligence.

(d)

Adapt the laws regulating copyright. Only China, EU countries, and the United States have adapted their copyright laws.

(e)

Adapt national and local regulations to the emergence of artificial intelligence. This is a crucial issue in its use and application in different aspects of digitalized society.

(f)

Introduce the infrastructure for the proper use of GenAI in Education and research. All educational institutions should introduce an infrastructure that enables quality use of the system and its tools and avoids negative aspects.

(g)

Think about the long-term consequences of GenAI in Education and research.

A quality legal system is required. Only a few countries have introduced specific policies or plans for artificial intelligence. The starting point for regulating artificial intelligence in the education system is UNESCO’s guidelines for policymakers (Miao et al. 2021), and the following eight proposed measures supplement the existing guidelines.

The OECD document lists the advantages and disadvantages of artificial intelligence and digital technology (OECD 2019). The advantages of artificial intelligence include numerous tools that can help students and pupils learn and thus enable teachers to conduct personalized lessons. As another advantage, inclusive education and fairness are mentioned, where visually and hearing-impaired people can easily follow classes and master the material by converting speech into text or automatic subtitling. On the other hand, the risks of using digital technologies can create inequality in the education process, where more advanced students would have more significant advantages from using tools, in contrast to students who are less able to use technology due to poor digital literacy. In addition, the question of protection of private data, security, and use of personal data of students and teachers is raised. Also, the increased use of technology can lead to the atrophy of specific human skills that are essential for individual development. The absence of socialization due to excessive activity on computers should be emphasized.

At the level of the European Union, it is essential to mention the Action Plan for Digital Education (2021–2027), which has two strategic priorities:

• In order to achieve a thriving ecosystem of digital education, it is necessary to introduce quality infrastructure and establish mutual connectivity. Then, effective planning and development of digital capacities should be implemented to educate teaching staff who could impart quality knowledge and provide quality platforms with user-friendly tools.
• The achievement of the second strategic goal refers to the improvement of digital skills and competencies, and the following actions are foreseen: take care of digital literacy, with a particular emphasis on recognizing misinformation; education in the field of computing; good knowledge and understanding of artificial intelligence; increase the number of girls in digital studies and women in the field of digital careers (EU 2020).

Artificial intelligence is increasingly being used to support teaching, learning, and assessment. There are four use cases: teaching students (intelligent tutoring system, dialogue-based teaching systems, language learning applications), supporting students (exploratory learning environments, formative writing assessment, collaborative learning supported by artificial intelligence), supporting teachers (summary writing assessment and essay, student forum monitoring, AI-based teaching assistants, pedagogical resource recommendation), system support (mining educational data for resource allocation, diagnosing learning disabilities, guidance services) (European Commission 2022).

There are critical requirements for reliable artificial intelligence: human action and supervision; transparency; diversity, non-discrimination and fairness; the welfare of society and the environment; privacy and data management; technical stability and security; and responsibility. In each of the above, specific questions appear, which entail consequences if there is no adequate answer. Specific questions are asked in each area: Are teachers and principals sufficiently trained to handle artificial intelligence? Are quality monitoring systems in place? Are the application instructions presented to both teachers and students? Does the system provide specific interactions for students with special needs in education? How does the system affect students’ and teachers’ social and emotional well-being? Are there specific mechanisms that ensure data security? Are artificial intelligence systems compliant with the General Data Protection Regulation? How is the effectiveness and impact of the artificial intelligence system evaluated, and how does this evaluation consider the critical values of education? (European Commission 2022)

According to the 2024 Artificial Intelligence Index Report, it was realized that the introduction of generative AI tools, such as ChatGPT, into the education system has led to numerous discussions. It is believed that the mentioned tool could lead to the plagiarism of documents. In January 2023, China introduced legal regulations to solve security problems and the situation of creating realistic virtual entities and multimodal media. In addition to the above, US lawmakers passed the National Security Act in March 2022, which aims to enable the Department of Defense to use artificial intelligence to automatically detect and mitigate threats to its networks, digital infrastructure, and public finance. However, in 2023, the application areas have expanded to include the following policies: armed forces and national security, education, labor and employment, civil rights and liberties, minority issues, trade, science, technology and communication (Maslej et al. 2024).

The United Nations General Assembly adopted the Resolution on Artificial Intelligence, which advocates protecting human rights and personal data. The same aims to monitor artificial intelligence due to potential risks. The Resolution encourages all member states to implement reliable artificial intelligence systems according to their national priorities inclusively and fairly while achieving sustainable development in three areas: economic, social, and environmental. In addition, the aim is to encourage the development of internationally interoperable technical tools, including reliable mechanisms for verifying the authenticity of content and origin (e.g., watermarking, tagging) to recognize manipulation of information and distinguish or determine authentic digital content and digital content generated or manipulated by artificial intelligence. Trusted artificial intelligence preserves and promotes linguistic and cultural diversity (Mishra 2024).

On the territory of the European Union, in April 2021, the European Commission proposed a draft law on artificial intelligence, which represents the legal framework for the procurement and use of artificial intelligence. In recent years, the central political issue at the European Union level has been the regulation of artificial intelligence. The goal is to create mutual benefit, i.e., not to limit technology and enable society to use all its benefits, but to protect basic human rights, all based on the fundamental values and principles of the European Union (EU 2024). In its impact assessment, the Commission noted several problems: opacity (the limited ability of the human mind to understand AI models), complexity, constant adaptation and unpredictability, autonomous behavior, functional dependence on data, and data quality. AI systems that present “unacceptable risks” are banned. Accordingly, these are artificial intelligence systems that use harmful manipulative “subliminal techniques”, systems that exploit vulnerable groups (people with special needs), artificial intelligence systems used by public bodies, and “real-time” remote biometric identification systems in a publicly accessible space for law enforcement. Limited risk systems are mentioned as the third item: systems that communicate with people, i.e., chatbots; emotion recognition systems; biometric categorization systems; and AI systems that generate or manipulate image, audio or video content, i.e., deepfakes. All other low- or minimal-risk systems can be used in the European Union without additional harmonization. State bodies in individual member states would be appointed to implement the regulation. Some experts already emphasize that the preliminary assessment of risk (and even high risk) is left to the self-assessment of service providers. Another legal gap to be filled by legal regulations is unique coordination mechanisms between authorities, especially regarding cross-border offences (EU 2024).

The concepts and regulatory frames presented belong to the area of soft law. It creates issues among legal experts because it is also at the border between law, society, economy, and political science. “We could define soft law as a non-binding normative framework whose implementation is conditioned solely by the will of the addressee of the norm.” (Vuletić 2011). Analogous to that, the concept of quasi-law is problematized and can be consequently created. Nevertheless, in recent times, especially in international and European public law, attempts have been made to include soft law in legal science. Soft law aims not to implement formal, legally enforceable norms but to influence subjects’ behavior.

In the current social environment surrounding AI usage and application in education, legislators are guided by soft law instruments, mainly since techniques and technology cause frequent changes that the legal system can hardly keep up with. In this context, first of all, attention should be drawn to the principle of proportionality contained in the Charter of Fundamental Rights of the European Union: “Any limitation on the exercise of the rights and freedoms recognised by this Charter must be provided for by law and respect the essence of those rights and freedoms.” (EU 2000). Subject to the principle of proportionality, restrictions are possible only if necessary and correspond to the goals of general interest recognized by the Union or the need to protect the rights and freedoms of other persons. The principle of proportionality is a crucial link between technique and technology on the one hand and rights on the other, especially when protecting the already-mentioned rights and freedoms of man and citizen, which can be threatened if artificial intelligence is not used reliably. The principle of proportionality is also defined in Article 5, Paragraph 4 of the Treaty on European Union, where the measures of the European Union: “must be suitable for achieving the desired goal, must be necessary to achieve the desired goal and, they must not impose a burden on individuals that is excessively heavy about the goal to be achieved (proportionality in the narrower sense).” (EU 2012).

2.2. Overview of LLM Usage Detection Technology

In addition to analyzing the influence of large language models on the educational system from the perspective of regulation, it is necessary to look at the situation from the point of view of technology. Regardless of the different attitudes related to sanctioning large language models used for content creation, it is essential to determine whether the use of large language models can be detected and with what certainty. In order to determine this, several papers dealing with the problem of detecting the use of LLMs were reviewed, and the key conclusions and opinions of the authors are presented below.

Wu et al. reviewed the literature related to the detection of content generated by large language models, and the first thing they noticed was a significant increase in published papers in 2023 covering the topic. From 2019 to 2022, the total number of published papers covering that topic was less than the number of published papers in 2023. The appearance of the ChatGPT service influenced this increase in the number of publications. The authors are optimistic about detection, although they indicate problems that still need to be solved. First, they state the need for robust detectors that are resistant to attacks, optimization of detectors so they can be used even with few resources, and adaptation of detectors for text classification in which human-generated and LLM content is mixed. In addition, they state the problem of the reliability of the content about the source, because it will be increasingly challenging to be sure who generated a text. Therefore, the classifier’s quality decreases if trained with incorrectly classified data (Wu et al. 2023).

Orenstrakh et al. reviewed existing text detectors generated by large language models. The authors report that CopyLeaks is the most accurate LLM-generated text detector, GLTR is the most robust LLM-generated text detector, and GPTKit is the best LLM-generated text detector for reducing false positives. The authors report that GPTZero reported nearly 50% false positives in 114 text samples tested. In addition, the detectors worked best with texts in English, and their accuracy decreased with paraphrasing tools. The detectors still require improvements to offer a complete solution for maintaining academic integrity (Orenstrakh et al. 2023).

In their paper, Tang et al. state two approaches for detecting content generated by large language models: black-box and white-box detection methods. Black-box detection methods are based on training a classifier that can be trained to classify new content into one of those two categories based on a large amount of human-generated content and large language models. The authors state the shortcomings of the method, which come down to the problem of bias in categorizing training content. However, they state that with the improvement of large language models, it will be increasingly more work to differentiate this content shortly. Eventually, the black-box approach will be unusable (Tang et al. 2024).

On the other hand, white-box detection methods require the cooperation of people who offer to implement the model using watermarking. These are strings of content that mark the generated text in some way. This approach introduces new problems: first of all, the necessity of cooperation of the model owner and the problem with the decrease in text quality by inserting watermarks. A particular problem beyond these frameworks is that, when using large open-source language models, it is difficult to achieve the level of control requiring watermarks. The authors summarize numerous challenges in content detection generated by large language models, which suggests that creating a rigid repressive regulation with reliable detection methods would be the right direction (Tang et al. 2024).

In previous decades, tools were developed to detect plagiarism by comparing new texts with the papers published so far. Their functionality is unquestionable, and therefore, their application is defined in the regulations of educational institutions. Unlike those tools, which are based on determinism, the detection of content created by LLMs is more based on probability and stochasticity. Authors describing tools for the detection of LLM applications agree that detection is a major challenge, and there is a possibility that the implementation of a completely reliable detector will never be possible. Perhaps the biggest problem is that open-source LLMs can be customized to such an extent that it will be impossible to confidently claim that the text generated by a custom model was created by an LLM.

The analysis of papers related to the possibility of detecting the application of LLMs indicates that in this field, the battle is likely already lost if we want to implement restrictive policies and sanction the application of LLMs. A similar situation took place a few decades ago with the appearance of electronic calculators, which became a daily used tool in mathematics in elementary schools, while at one time, there was a debate about their advantages and disadvantages in the educational system. It seems that the authors of the analyzed papers from the primary and secondary categories are also aware of this and are far more inclined to use LLMs in the educational system, as sanctioning them is rarely mentioned as an option.

3. Methodology

The authors employed a systematic literature review (SLR) as a research methodology focusing on content analysis to extract inductive data. As suggested, in this context, SLR can be utilized for diverse objectives, including gathering, integrating, and charting literature in the field. This study conducted a bibliometric analysis to examine the publication patterns of the papers in terms of their frequency of publication by year, author, and country. Furthermore, this study conducted a bibliometric analysis of the final sample, providing a detailed description of the papers based on their properties.

The authors followed the three processes. Tranfield et al. (2003) outlined planning, conducting, and disseminating. In the planning phase, the authors defined the terms and elements of the search. In addition, the databases that would be used (SCOPUS and WOS) were defined. This phase required special attention, because we would obtain papers that did not match our content by choosing the wrong terms. The choice of “ChatGPT” instead of “Large language models” should be emphasized here. Since it became available, ChatGPT and its influence have been analyzed by many authors, but as a rule, authors entered the term ChatGPT in the keywords and not the superordinate term LLM to which ChatGPT belongs. This anomaly, quickly discovered, influenced the search for “ChatGPT”. In addition, the papers had to include the term “education” and one of the following three terms: “policy”, “regulation”, and “legislation”. The authors applied these criteria to filter and establish the final research sample. In the third phase, known as disclosure, the authors performed a comprehensive and detailed analysis of the thematic categories that emerged through inductive categorization.

Table 1. Research protocol.

Research Protocol	Detailed Description
Research various databases	Scopus Database and Web of Science
Publication type	Peer-review journals and conference papers
Language	All
Date range	2000–2024
Search fields	Title, abstract, and keywords
Search terms (Scopus)	(TITLE-ABS-KEY (“ChatGPT”) AND TITLE-ABS-KEY (“education”) OR TITLE-ABS-KEY (“Policy”) OR TITLE-ABS-KEY (“regulation”) OR TITLE-ABS-KEY (“legislation”)
Search terms (Web of Science)	ChatGPT (All Fields) AND Education (All Fields) AND Policy OR Regulation OR Legislation (All Fields)

shows the research protocol.

Conducting, as the second part of the process, starts with citation database queries. After querying using the specified criteria, the obtained results are stored in files with the extension “.bib”. The next step was to remove the papers that appeared in both databases, and the pyBibX library was used for this. It is a bibliometric and scientometric Python library that uses raw files from scientific databases including Scopus, WOS, and PubMed. Also, it is powered by advanced AI technologies to analyze bibliometric and scientometric results and textual data. The “Scopus + WOS” script was used to merge the results obtained from the mentioned databases. The results obtained by merging are given below in

Table 2. Results after merging.

Original Database

A total of 104 documents were found (104 documents and 0 duplicates) Articles = 76 Conference papers = 6 Editorials = 3 Errata = 1 Notes = 2 Reviews = 16

Added Database

A total of 101 documents were found (101 documents and 0 duplicates) Article = 65 Articles in press = 14 Editorial materials = 6 Editorial materials; early access = 1 Proceedings papers = 5 Reviews = 9 Reviews; early access = 1

Merging Information

A total of 142 documents were found (38 new documents from the added database) Articles = 105 Articles in press = 1 Conference papers = 6 Editorials = 3 Editorial materials = 3 Editorial materials; early access = 1 Errata = 1 Notes = 2 Proceedings papers = 2 Reviews = 18

.

Bibliometric Description of the Articles

Table 3. Exploratory data analysis.

Timespan	2023–2024
Total number of countries	34
Total number of institutions	140
Total number of sources	110
Total number of references	0
Total number of languages	3
--English (# of docs)	37
--Norwegian (# of docs)	1
--Unknown (# of docs)	104
Total number of documents	142
--Articles	105
--Articles in press	1
--Conference papers	6
--Editorials	3
--Editorial materials	3
--Editorial materials; early access	1
--Errata	1
--Notes	2
--Proceedings papers	2
--Reviews	18
Average documents per author	1.03
Average documents per institution	5.69
Average documents per source	1.27
Average documents per year	71.0
Total number of authors	710
Total number of authors’ keywords	148
Total number of authors keywords plus	109
Total single-authored documents	19
Total multi-authored documents	123
Average collaboration index	4.42
Max H-index	2
Total number of citations	1922
Average citations per author	2.71
Average citations per institution	13.73
Average citations per document	13.54
Average citations per source	17.45

shows the exploratory data analysis, which shows a series of characteristics of the analyzed set of papers.

Below is a list of countries where the authors of the paper are from: Finland, Singapore, Denmark, New Zealand, Mali, Cyprus, Philippines, Switzerland, India, Canada, United States of America, France, Austria, Australia, Belgium, Qatar, Russian Federation, China, Israel, Taiwan, Netherlands, Brazil, Niger, Germany, Malaysia, United Kingdom, Spain, Japan, Ukraine, Oman, Mexico, Argentina, and Italy.

The Litmaps service was used to perform two further analyses. In the first, the paper’s publication period is shown on the horizontal axis, while the vertical axis shows how often the paper was cited in other papers. As expected, all the papers were published in 2023 and 2024 because we searched for the keyword “ChatGPT”, and it is a service that is available from the end of 2022. The Figure 1 showed us which papers had a more significant impact on the scientific community; thus, we gave them more attention in the analysis.

The Figure 2 again shows the publication period of the paper on the horizontal axis, while the number of references in the paper is on the vertical axis. This graph clearly shows which authors performed extensive literature reviews, and most often, they were review papers. As a rule, papers with fewer references were original scientific papers.

After the second phase of collection, removal of duplicates and analysis of the obtained results, the third phase, which includes a qualitative analysis of the paper, was carried out. At this stage, out of 142 papers, 93 remained, the content of which was mentioned in this analysis and was listed in the literature. Papers that were not open-access or did not cover the main topic were excluded.

The third and final level of analysis included quantitative and qualitative analyses to obtain critical papers to help us recommend regulating generative artificial intelligence in education. The quantitative analysis consisted of counting the occurrences of the words “policy”, “regulation”, and “legislation”, which indicated to us whether the terms were often mentioned in the paper or were mentioned only once. These data indicated papers that should be studied in detail, while papers in which the mentioned terms were rarely mentioned were analyzed more briefly. There were 29 papers left in the category of primary papers, while 64 papers remained in the category of secondary papers.

4. Fundamental Governing Principles: Recommendations for Implementation of AI in Education

The final stage of qualitative analysis encompassed detailed content analysis of 29 primary papers, aiming to extract and reveal fundamental governing principles for AI regulation in education.

Table 4. Fundamental governing principles for AI regulation in education.

Author	Main Governing Recommendations
Abdaljaleel (Abdaljaleel et al. 2024)	Details on policy and strategies for AI integration in education whose key elements focus on: Adoption of flexible and adaptable curricula that blend technical knowledge with higher-level cognitive skills Prioritization of ethical foreground establishment through curriculum embeddings and encouraging and supporting inclusivity and diversity Mandatory teacher training programs on AI technologies and innovative teaching method induction Building and promotion of continuous learning culture, equipping students with lifelong learning skills Encouraging and supporting broad academic–industry cooperation partnerships throughout different forms of joint research, training, and other practical skills training and development Innovative assessment methods, critical thinking, creativity, problem-solving implementation, and development as essential skills Resources allocation in support of new educational technology implementation, new programs, and infrastructure development Stakeholder collaboration and feedback empowerment Ethical and legal implications of AI integration in education
Bauer (Bauer et al. 2023)	Focuses on enhancing peer-feedback scenarios in higher education, encompassing key points through: Support measures, adaptation targets, leverage points, and automation goals Digital tool (based on modern NLP models) implementation in learning management systems (LMS) Translating automation goals into prediction targets, utilizing relevant data, and evaluating predictions
Bearman (Bearman and Ajjawi 2023)	Reveals AI learning strategies through: Focusing on quality standards in understanding AI’s social boundaries Promoting learner interactions with AI, aiming towards evaluative judgement Encouraging user understanding of AI evaluative, ethical, and practical aspects
Bukar (Bukar et al. 2024)	Constructs policy-making framework for generative AI through risk, reward and resilience categories: Proposes a policy-making framework for generative AI Key concerns in AI applications are categorized as risk, reward, and resilience Legislation can have both positive and negative effects, potentially decreasing rewards regarding the limitations or sanctions imposed Providing a comprehensive decision-making model for policymakers and higher-education institutions to navigate the complexities of AI use in education
Chan (Chan and Hu 2023)	Outlines AI training, ethical use, and risk management as crucial components in AI implementation in higher education through the following requirements: Student and teacher training on the use of generative AI technologies Developing policies for ethical use and risk management AI technologies Incorporating AI as supplementary, not replacement, tools for human interaction Enhancing and fostering holistic competencies (digital competencies and time management) through AI implementation in education Fostering a transparent and open AI environment through broad discussions of benefits and concerns Ensuring data privacy and security in AI technology implementation
Chang (Chang et al. 2023)	Suggests pedagogical principles for AI chatbot integration in education. The main observations deal with: Concerns about unethical use of generative AI technologies by students Need for an upgrade of educational and pedagogical principles to promote self-regulated learning Proposal for collaboration among different stakeholders (including educators, instructional designers, AI researchers, software developers) Incorporation of individualization elements into the process (goal setting, self-assessment, feedback, personalization) as essential educational principles Development of students’ self-regulated learning Data-driven mechanism development for learning analytics Guidelines for implementing AI in teaching and learning contexts
Chauncey (Chauncey and McKenna 2023)	Emphasizes the importance of ethical AI chatbots in education. The main contributions include the following: Formulation of a conceptual framework for the responsible use of AI chatbots in education, which supports cognitive flexibility in AI to enrich teaching and learning environments Developed exemplars for math, English language, and arts in AI-supplemented learning
Chiu (Chiu 2024)	Focuses on implications for policy assessment and development in educational institutions, emphasizing: Guidelines and policy development on future workforce competence and skills required AI literacy and generative AI integration into teacher and student professional development programs Generative AI-enriched educational institutions should construct their disciplinary knowledge Institutional-level workshops or courses on AI literacy and use for educational purposes should be provided It is essential to enhance students critical thinking abilities
Chiu (Chiu 2023)	Deals with AI educational aspects in school implementation, highlighting the importance of: Prerequisite knowledge (critical thinking, disciplinary knowledge, inquiry-based questioning skills, AI literacy) that is crucial for teachers to assess students’ readiness for Gen AI tool implementation Frequent assessment of generic skills. The generic skill-assessment platform should be available for students to assess and self-evaluate their skills independently AI should be one of the core subjects in school education (in line with traditional ones like language, mathematics, and science) AI and media literacy should be a mandatory part of professional training standards for teachers and educational institutions management AI knowledge, a foundation course related to AI and education, should support learning and teaching development throughout the educational lifecycle
Cowling (Cowling et al. 2023)	Prospects on ChatGPT’s potential in higher-degree research amending Research practice enhancement Student psychological needs, autonomy, competence, and relatedness support systems Formative feedback for researchers and doctoral students
Dai (Dai et al. 2023)	Proposes a model for AI-enhanced postgraduate research, highlighting: ChatGPT-facilitated self-directed research for enhanced independence Supervisor’s usage of AI-enhanced supervision for student personalized feedback Generative AI tools can be used for research supervision Universities should develop AI literacy curricula for responsible AI tool usage
Galent (Gallent-Torres et al. 2023)	Explains pro and cons of AI use in education through three main categories: Pedagogical focusing on ethical and responsible AI usage for improved teaching and learning Governance focusing on clear policies, guidelines, and regulations for privacy, security and responsibility Operational focus towards addressing infrastructure and operational aspects of AI application in education
Hung (Hung and Chen 2023)	Aims to regulate ChatGPT’s use and application in academic settings through: Oral examinations for students’ essays to maintain academic integrity Deploying traditional assessment methods (short in-class essays) to evaluate student learning and retention Prioritizing grading students throughout the whole period of a learning process rather than at the far end-of-semester on individual academic performance Conducting more tutorial discussions during the semester to allow students to express their views Identifying jointly vast purposes of education and learning to update educational regulations The future of education is inevitably integrating technology with humans
Kayali (Kayalı et al. 2023)	Highlights AI in education through two dimensions, risks and precautions, stressing the importance of: Privacy, ethics, confidentiality, and security decision-makers’ responsibility
Khanal (Khan 2023)	Reveals and problematizes big tech’s impact on public policy theory through: Its dominance in technology and GenAI advancements in transforming public policy Becoming a central player in domestic policy domains and the global sector Traditional policy frameworks undergoing seismic shifts in big tech are becoming a “super policy entrepreneurs” Calling for a joint initiative of scholars, policymakers, and civil society members to examine these evolving dynamics critically The need for re-evaluation of traditional policy theories and adaptation of existing governance (policy, regulation, and legislation) frameworks
Li (H.-F. Li 2023)	Outlines implications for practice and policy, leaning towards: AI-supported student learning and problem-solving AI student-centered pedagogical transformation (using open online courses or flipped classrooms) Self-regulated learning online self-efficacy while enforcing learning motivation and creative thinking
Lo (Lo 2023)	Brings anti-plagiarism guidelines urging for: Reshaping codes of conduct and regulating language model use Providing student education on academic integrity policies and consequences Training on new elements of academic integrity for both teachers and students
Luo (J. Luo 2024)	Recommends adapting higher education for generative AI through: The holistic approach to AI in education Importance of identifying essential epistemic skills Judgment, understanding knowledge boundaries, recognizing unknowns, and assessing information sources’ credibility Self-regulation in decision-making based on empirical data, ethical considerations, and sociocultural contexts Skill set that balances technological fluency with nuanced judgment and contextual awareness
Mathews (Matthews and Volpe 2023)	Is open to implementing generative AI in education, overcoming challenges, and finding aolutions in: Academics’ struggle to differentiate between AI and human texts Uncertainty about AI’s capabilities necessitates assessment redesign Institutions need to improve assessment designs, AI policies, and procedures
Michel (Michel-Villarreal et al. 2023)	Debates on AI use in higher education, urging for: Clear policies and establishments for responsible AI use Academic integrity, data privacy, algorithmic bias, and ethical considerations solutions
Ng (Ng et al. 2024)	Study on AI technologies for student-Centered learning and self-regulated learning while Advocating for the use of ChatGPT and other generative AI technologies Stressing the importance of understanding AI limitations and appropriate usage Underlining the necessity for equipping teachers and students with AI competencies
Perkins (Perkins and Roe 2023)	Focuses on academic integrity policies in higher-education institutions to: Clarify how students may use LLMs-based tools Ensure these tools are appropriately regulated in academic integrity policies to make their usage fully clear to students and staff State clear limits to LLM use and provide specific examples for students and staff for assured understanding Communicate the final policy widely to students and staff, with full training provided Avoid a blanket ban due to LLMs’ lack of enforceability and potential benefits Advocate for a nuanced approach that acknowledges potential benefits, evolving social understanding of plagiarism, and the changing nature of digital writing and future human–AI co-creation development
Pham (Pham et al. 2023)	Examines AI-assisted learning in engineering technology courses based on the following: Review of ChatGPT usage by educators and administrators Study of generative AI tools’ implications in higher education Academic and student adaptation of AI-assisted learning flow Follow-up on ChatGPT’s applications in teaching, learning, and assessment design
Polyportis (Polyportis and Pahos 2024)	Extracts institutional policy and ChatGPT adoption to: Ensure alignment with the institution’s values while protecting practical pedagogical standards Conduct an empirical investigation to determine the effects of policy on student adoption and overall system effects
Rahman (Rahman et al. 2023)	Promotes ethical ChatGPT usage in education, empowering Policymakers to design clear strategies Encourage sustainability of AI use in education
Rejeb (Rejeb et al. 2024)	Develops educational I-institutions’ AI usage guidelines urging for: Establishment of policies on responsible AI use, data privacy, and academic integrity Leverages the benefits of AI application in education while mitigating misuse, misconduct, and potential risks
Tarisayi (Tarisayi 2024)	Aligns innovation with integrity in education through: Policies that should allow for transparent AI usage Delegating repetitive tasks while maintaining human agency in creativity Advocating responsible assimilation of emerging technologies Updating policies and pedagogies for human oversight and surveillance
Thanh (Thanh et al. 2023)	The proposed framework for generative AI assessments pointing toward key topics needs to: Systematically evaluate the capabilities of AI tools in assessments Human assistance in the assessment design process Re-evaluation of tertiary institutions’ programs and learning outcomes Focus on a centered approach to higher educational goals of Bloom’s taxonomy
Yan (Yan et al. 2024)	Updates innovations for educational technology, undelaying Reduction in manual effort in adapting models Improving reporting standards for significant language model-based research Adopting a human-centered approach to address practical and ethical challenges

shows the details of these findings. The listed author contributions to this discussion are presented in alphabetical order.

The secondary paper qualitative content analysis encompassed 64 papers and showed additional vital topics and issues in the author’s focus. It is relevant to point out how the authors mostly debated student’s roles, attitudes, perceptions, perspectives, and experiences in using generative AI in education (Alkhaaldi et al. 2023; Crawford et al. 2024; Farhi et al. 2023; Habibi et al. 2023; Hasanein and Sobaih 2023; Johnston et al. 2024; Luo et al. 2023; Polyportis and Pahos 2024; Sakib et al. 2023; Vignesh et al. 2023). The importance of students’ position in these debates is extracted from their position in the process, being in focus as the key stakeholder of the educational process.

Additionally, teachers’ roles and attitudes towards generative AI usage and implementation in the educational process are significant. Revealing teachers’ attitudes, opinions, and experiences in using generative AI in education is present in the works of (Farhi et al. 2023; Fütterer et al. 2023; Habibi et al. 2023; and Robledo et al. 2023).

Overall, the systematic literature review confirmed that most authors undoubtedly accept the idea of adopting generative AI tools in the educational process, pointing both towards its positive (Chan and Hu 2023; Fütterer et al. 2023; Luo et al. 2023) and negative aspects (Chan and Hu 2023; Dwivedi et al. 2023; Hasanein and Sobaih 2023; Fütterer et al. 2023; Luo et al. 2023; Šedlbauer et al. 2024).

It is significant to note that the most prominent concern and interest in generative AI-supported educational processes are found in medical student education, as researched and exposed by (Alkhaaldi et al. 2023; Barrington et al. 2023; Haverkamp et al. 2023; Karabacak et al. 2023; and Vignesh et al. 2023).

A systematic literature review helps to summarize recent literature developments and trends in the field, as seen by (Dempere et al. 2023; Imran and Almusharraf 2023; and Sarker and Ullah 2023).

While technology is exhausting societies’ regulatory capacities, it is a reasonable request to slow down and pause AI development to ensure it is used for the benefit of all and to give society a chance to adapt at its own pace (Dempere et al. 2023).

Self-regulated learning practices also significantly contribute to understanding the novelty generated by AI enacting educational systems, as noted by (Lee et al. 2024; Luo et al. 2023; Ng et al. 2024; Prasad and Sane 2024; and Xu et al. 2024). A few novel considerations have also been observed, such as AI-related technostress (Wach et al. 2023) and autopoiesis as a frameworks of the technological system under consideration (Watson and Romic 2024).

5. Discussion and Conclusions

A systematic literature review primary analysis extracted 29 papers based on content analysis, vigorously contributing to guiding suggestions regarding governing AI implementation in education. Key findings are attributed to ethical considerations and responsible use while promoting critical thinking, cognitive flexibility, problem-solving, student-centered learning, and courses adapted to self-regulation.

Furthermore, the analysis revealed a necessity for promoting a continuous learning culture, inclusivity, and diversity. An urge for clear and inviolable data privacy, security, and transparency policies takes on a significant role and proportion of research works.

Crucial assets in overbridging the existing gap in AI governing education are represented throughout a pledge for an unavoidable shift in traditional assessment methods, curricula flexibilization, comprehensive teacher training programs, and a necessary inclination towards innovative teaching methods and assessment design.

AI coaching and learning strategies must focus on quality standards, evaluative judgment, understanding, transparency, and explainability of AI systems. A more comprehensive, interdisciplinary, and internationalized policy-making framework for generative AI in educational usage is to be proposed to help policymakers navigate emerging complexities. Institutions should adopt the wide, generative referencing frame to develop their guidelines and policies highlighting future workforce competence, integrating AI into teacher–trainer–educator professional development programs, and providing AI literacy for students and teachers. A human-centered approach is a cornerstone in dealing with new challenges in the interplay of AI with education system upgrading.

Finally, a comprehensive overview of guidance recommendations in policy, legislation, and regulation considering LLM in education emerged based on the foundation provided by the soft law overview presented and the comprehensive systematic literature review, which should encompass and focus on the following aspects:

• Ethical guidelines and standards. Values compose and construct all communities, so they are foreseen as the founding principle. Existing codes of ethics in education should be supplemented by crucial directions guiding new frontiers of fairness in AI implementation in education (Abdaljaleel et al. 2024; Bearman and Ajjawi 2023; Chan and Hu 2023; Chang et al. 2023; Chauncey and McKenna 2023; Gallent-Torres et al. 2023; J. Luo 2024; Michel-Villarreal et al. 2023; Rahman et al. 2023; Yan et al. 2024).
• Compliance upgrade. Overall, the vast existing system of regulation and standards across different aspects and functional parts of education need to be harmonized and adjusted to the AI framework. Most importantly, licensing, certification, quality, and safety standards must be reviewed and debated within a vast interdisciplinary expert network to ensure a comprehensive approach (Abdaljaleel et al. 2024; Bauer et al. 2023; Bearman and Ajjawi 2023; Dempere et al. 2023; Khan 2023; J. Luo 2024; Pham et al. 2023; Yan et al. 2024).
• Data governance. In our right to protection, security, and privacy, transparent, robust, and unbreachable systems to collect, store, and govern sensitive data are necessary for an AI-driven education system. Data anonymization, protection, and consent are critical factors in rethinking these aspects in AI-supplemented education (Bauer et al. 2023; Chan and Hu 2023; Chang et al. 2023; J. Luo 2024; Michel-Villarreal et al. 2023; Rejeb et al. 2024).
• AI systems’ transparency, understandability, and explainability are crucial in AI-involved decision-making processes that reflect different aspects of the educational system (e.g., student and teacher evaluation, hiring, and admission procedures) (Bearman and Ajjawi 2023; Chan and Hu 2023; Gallent-Torres, J. Luo 2024; Ng et al. 2024; Perkins and Roe 2023; Gallent-Torres et al. 2023; Tarisayi 2024; Lee et al. 2024; Prasad and Sane 2024; Xu et al. 2024).
• Algorithm accountability and transparency. Mutual (social) trust between developers, producers, maintenance services, and users is crucial in building confidence and openness towards system bias and error elimination (Michel-Villarreal et al. 2023; Chan and Hu 2023; Tarisayi 2024; Matthews and Volpe 2023).
• Quality control and auditing mechanisms are already well-established in continuous circles of accreditations, certifications, evaluations, revisions, and reviews embedded in educational settings. An amendment in existing procedures is needed to ensure AI-generated educational content fits the required users’ standards, needs, and expectations (Bearman and Ajjawi 2023; Tang et al. 2024; Bauer et al. 2023; Bearman and Ajjawi 2023; Hung and Chen 2023; Khan 2023; Thanh et al. 2023).
• Accessibility standards. Embedding AI educational content novelties cannot leave anyone behind. Inclusion and diversity standards must be considered, as must alternative formats and compatibility with assistive technologies intended for special-needs social groups (Abdaljaleel et al. 2024; Cowling et al. 2023; Dai et al. 2023; Matthews and Volpe 2023).
• Teacher training. The generational gap between digital nomads and “old school” teachers must be closed through mandated extensive teacher training and young researchers’ involvement reinforcement (Abdaljaleel et al. 2024; Chan and Hu 2023; Chiu 2023, 2024; Lo 2023; Ng et al. 2024).
• Research and innovation. Significant research is essential to determine the impact of generative AI on teaching, training, and learning outcomes. Longitudinal and comparative analysis, case studies, and best and past practice examples must precede the starting point or testing phase when introducing AI in education (Abdaljaleel et al. 2024; Chang et al. 2023; Cowling et al. 2023; Dai et al. 2023; Yan et al. 2024).
• Regulatory bodies and enforcement mechanisms need to be equipped and enforced to oversee the implementation of AI in education and regularly ensure ethical, regulatory, and legislative compliance (Chang et al. 2023; Gallent-Torres et al. 2023; Hung and Chen 2023; Khan 2023; H.-F. Li 2023; J. Luo 2024; Perkins and Roe 2023).
• Stakeholder initiative. As a unique asset that multiplies when shared, knowledge requires different perspectives to obtain the best performance. In aspiring to build a network of stakeholders with a unified goal of achieving excellence in AI deployment in education, stakeholder initiatives and perspectives can overcome new challenges (Abdaljaleel et al. 2024; Chang et al. 2023).
• Internationalization, interdisciplinarity, and harmonization of best practices and collaboration can strengthen resilience, bring new solutions, help overcome emerging issues and challenges, accomplish consistency, and improve alignment across diverse jurisdictions (Hung and Chen 2023; Khan 2023; Abdaljaleel et al. 2024; Chang et al. 2023; J. Luo 2024).

By following these regulative recommendations, educational institutions with their wide stakeholder networks can harness the potential of GenAI in education while simultaneously mitigating risks and ensuring ethical and responsible use. By implementing these recommendations, policymakers, regulators, and legislators can create a conducive environment for the responsible adoption of GenAI in education. This approach ensures that it benefits students, educators, and society, transcending towards future socio-technologically aligned and human-centered educational systems.

Conclusively, while establishing a suitable normative framework will be a notable accomplishment, we are still far from embedding a fully functional and socially parenthetic AI-supplemented educational system. It is necessary to create norms that require complete internalization (Etzioni 2000) to fit successfully into different cultural and societal frames. Hence, the subsequent stage of integrating AI systems into education raises concerns about the legal and societal regulation of AI conduct, particularly criminal liability and accountability (Hallevy 2010).

Regulation is defined after the emergence of the technology that needs to be regulated, and future research could go in the direction of monitoring the compliance of these two processes. It is clear that a particular technology for detection exists, for which the authors state that there is room for improvement, so it will be interesting to monitor the extent to which certain educational institutions or institutions at the state level will follow the trends in developing detection methods.