Impacts of Generative Artificial Intelligence in Higher Education: Research Trends and Students’ Perceptions

Abstract

In this paper, the effects of the rapid advancement of generative artificial intelligence (Gen AI) in higher education (HE) are discussed. A mixed exploratory research approach was employed to understand these impacts, combining analysis of current research trends and students’ perceptions of the effects of Gen AI tools in academia. Through bibliometric analysis and systematic literature review, 64 publications (indexed in the SCOPUS and Web of Science databases) were examined, highlighting Gen AI’s disruptive effect on the pedagogical aspects of HE. The impacts identified by the literature were compared with the perceptions held by computer science students of two different HE institutions (HEIs) on the topic. An exploratory study was developed based on the application of a questionnaire to a group of 112 students. The results suggest that while Gen AI can enhance academic work and learning feedback, it requires appropriate pedagogical support to foster critical, ethical, and digital literacy competencies. Students demonstrate awareness of both the risks and benefits associated with Gen AI in academic settings. The research concludes that failing to recognize and effectively use Gen AI in HE impedes educational progress and the adequate preparation of citizens and workers to think and act in an AI-mediated world.

1. Introduction

Generative artificial intelligence (Gen AI) is a category of artificial intelligence that utilizes deep-learning models to create new, original content. These models are trained on extensive amounts of existing data, such as text, images, music, or other forms of media, and learn to generate new content that resembles but is not identical to the training data. Based on a user’s prompt or input, Gen AI tools can produce statistically probable outputs consistent with the patterns and structures learned from the training data (Martineau 2023).

The widespread availability of ChatGPT and other Gen AI tools has led to a variety of disruptive effects across numerous human activities. Gen AI holds the promise of enhancing productivity and nurturing creativity while also facilitating more robust collaboration between humans and machines (Limna et al. 2023a). Its overall potential poses substantial challenges to the higher education (HE) system, particularly to teaching and learning approaches, including the assessment of students and the role of educators (Kohnke et al. 2023). These challenges must be addressed to ensure that education remains relevant and effective in the rapidly changing digital landscape (Bearman and Ajjawi 2023).

In 2023, there was a significant increase in scientific research focusing on the effects of Gen AI on HE. This research aimed to uncover the potential advantages and drawbacks associated with Gen AI, as well as the transformations it could be expected to bring about. A key concern surrounding the integration of Gen AI in education is identifying the most efficient methods and strategies for its implementation to enhance the teaching and learning process. Furthermore, there was a concerted effort to investigate the viewpoints of educators and students regarding the utilization of Gen AI in an academic environment, along with an analysis of their respective usage patterns. The studies conducted thus far have demonstrated a keen interest in deriving insights and recommendations for the development of an optimal pedagogical framework, considering both the potential benefits and risks posed by these new digital resources.

Given the existing scientific literature on the relationship between Gen AI and HE, the study presented below took as its research question: “What impacts is Gen AI having on HE?”. To address this, we developed a research path guided by two objectives: (O1) to identify the effects generated by Gen AI in HE highlighted in the literature, and (O2) to gather students’ perceptions on the risks and benefits of using Gen AI in academic contexts and compare it with what is identified in the literature.

In the first phase of the study, we conducted a bibliometric analysis complemented with a systematic literature review (SLR). The main objectives were to identify crucial insights from current knowledge and pinpoint existing research gaps. Additionally, based on a mixed-profile questionnaire applied to an intentional sample of first and second-year computer science students from two different higher education institutions (HEIs), the patterns of opinion shared by students about the main positive and negative consequences of using Gen AI were identified.

2. Materials and Methods

To identify the current research trends regarding the impacts of Gen AI in HE, we developed a bibliometric analysis using VOSviewer software (version 1.6.19), combined with an SLR that followed the Preferred Reporting Items for Systematic Reviews (PRISMA 2020) guidelines. The PRISMA approach allowed the application of a standardized sequence and peer acceptance of article selection criteria, search strategy, data extraction, and data analysis procedures (Page et al. 2021).

For the analysis, and due to their access to relevant, high-quality journals and publications, the scientific production indexed in the SCOPUS and Web of Science databases that met the following search criteria (logical operators) was selected:

• “Generative Artificial Intelligence” or “Generative AI” or “Gen AI”, AND;
• “Higher Education” or “University” or “College” or “Post-secondary”, AND;
• “Impact” or “Effect” or “Influence”.

A total of 148 documents were identified, which, after removing 69 duplicate items (first exclusion criteria), allowed a set of 79 documents to be identified for analysis, as shown in Figure 1. Following that, we screened the titles and abstracts of the publications and excluded the ones that focus on the effects of AI instead of Gen AI; after applying this second exclusion criterion (which led to the exclusion of 15 documents), we were left with a final list of 64 publications to be considered for inclusion in the study (Figure 1). The final list includes 52 articles, eight conference papers, two notes, one book chapter, and one editorial. All documents were published in 2023; the search was carried out on 7 January 2024. Figure 1 shows the steps followed.

After identification of the final selected group of publications, the metadata were exported into a CSV file for bibliometric analysis. The VOSviewer software was used to identify the predominant thematic clusters based on the most repeated keywords and significant correlations in the selected literature. Next, based on the detailed reading and exploration of the full texts, a thematic analysis of the content of the deductive profile was developed based on the overarching themes/clusters that emerged in the bibliometric review. The combined analysis allowed us to organize and interpret the main impacts of Gen AI found in HE by categories and subcategories. The key risk of bias in the findings is associated with the fact that only documents indexed in the two identified databases were considered in the analysis, and there may be other documents in other sources.

To characterize the students’ perceptions about the effects of Gen AI, an online mixed profile questionnaire was applied to an intentional sample of students in the first and second year of the computer science degree from two different HEIs. One is a public institution within the polytechnic system situated in a city with low socioeconomic activity, while the other is a private institution within the university system located in a city with a high level of socioeconomic activity.

Computer science students were selected for two main reasons. Firstly, given the integral role of Gen AI in the future of computing, their unique position at the forefront of technological advancements makes their perceptions and understanding of this technology particularly relevant. Additionally, their academic background provides them with the technical knowledge to engage with the nuances and implications of Gen AI in a way that students from other disciplines might not. Secondly, computer science students generally exhibit a higher motivation and interest in using computers and technology than students from other fields. This intrinsic motivation makes them more likely to engage with tools and applications that use Gen AI, providing valuable insights into its real-world impact. However, this also implied a significant deficit in gender diversity, which is common in computer science courses.

These two HEIs were deliberately selected to ensure a diverse range of student profiles, not only in sociodemographic terms but also in terms of access to HE, which directly influences the variety of opinions gathered. The public polytechnic institution in the area with a low socioeconomic level is more likely to attract students from the local community who may face financial or other barriers to accessing education. In contrast, a private university in an area with a high socioeconomic level is more likely to attract students from more privileged backgrounds with previous, excellent educational opportunities. This diversity is crucial to capture a wide range of opinions and experiences regarding Gen AI, reflecting the multifaceted nature of its impact on different segments of the student population.

The choice to focus on first- and second-year students was made to capture the perspectives of those who are new to higher education and are just beginning their exploration of Gen AI (first-year students), as well as those who have some experience in the field (second-year students).

The questionnaire was applied to a total of 170 students from two classes in the first and second year of HEI 1, as well as two classes in the first and second year of HEI 2, as indicated in

Table 1. Profile of the surveyed group of students (by HEI, curricular year, and gender).

	Selected Group of Students		Students Who Answered the Questionnaire
	M	F	M	F
HE1
1st year	59	5	34	2
2nd year	36	5	29	4
HE2
1st year	39	3	24	2
2nd year	21	2	15	2
Total	155	15	102	10
	170		112

Source: Own elaboration based on the applied questionnaire, November 2023.

.

A total of 112 students responded to the questionnaire, which corresponds to a response rate (global average) of 65.88% (corresponding to 68.37% in HEI 1 and 67.91% in HEI 2). The distribution by curricular year of attendance and gender of students who responded to the survey is statistically representative of that in the classes selected in both HEIs (HEI 1: distribution by curricular year, chi-square test, 3.567; p value: 0.586 (>0.05—margin of error); distribution by gender, chi-square test, 8.234; p value: 0.345 (>0.05—margin of error). HEI 2: distribution by curricular year, chi-square test, 2.435; p value: 0.789 (>0.05—margin of error); distribution by gender, chi-square test, 1.589; p value: 0.473 (>0.05—margin of error).

The questionnaire included three questions (Q), such as the following:

According to your opinion and experience of use:

• Q1—Does GenAI have more positive or negative effects on higher education? Options (to choose one): 1. It has more negative effects than positives; 2. It has more positive effects than negative; 3. There is a balance between positive and negative effects; 4. Don’t know.
• Q2—Identify the main positive effect of Gen AI in an academic context. Open-ended question.
• Q3—Identify the main negative effect of Gen AI in an academic context. Open-ended question.

The construction of the questionnaire was carried out following the following steps:

(1)

Identification of questions (objectives, content, and format) in accordance with the defined research objectives. The process benefited from the experience of building this type of data collection instrument shared by the study authors and from the suggestions and evaluations made by a specialist in social research tools.

(2)

Pre-test of the questionnaire with a class of first-year students from an area of study other than computer sciences at one of the HEIs involved in the study. This intentional selection allowed for a thorough assessment of the clarity and comprehensibility of the questions.

The data protection regulation was duly followed, with the participants confirming their participation in the study and consenting to the use of the data made available for scientific purposes, safeguarding anonymity. The questionnaire was structured in Office Forms and applied online between November and December 2023. The data collected are stored in the first author’s institutional Microsoft 365 account.

The data analysis was based on (i) descriptive statistics analysis for the results gathered in Q1 and (ii) content thematic categorical analysis with an inductive profile of the answers given to the open-ended questions in Q2 and Q3. The study is exploratory and collects the first evidence on the issue.

The types of effects generated by Gen AI in HE highlighted by the students were compared with those identified by the literature to explore the existing similarities and differences and conclude about the main lines of action to ensure the correct use of Gen AI in academic contexts. The use of a mixed method approach allowed for the combination of quantitative and qualitative data collection and analysis procedures. This combination enriched the research path and the results achieved.

3. Results

The following sections present the current research trends on analysing the impacts of Gen AI in HE, as well as the main points of students’ perceptions regarding the risks and benefits of using Gen AI in academic contexts.

3.1. Impacts of Gen AI in HE: Research Trends

Bibliometric analysis was carried out to identify the current research trends regarding the impacts of Gen AI in HE (O1) based on the metadata of the 64 publications selected. Three thematic clusters (Figure 2) were identified—marked in red, green, and blue.

As shown in the illustration, the biggest cluster (red) is represented by publications whose focus of analysis is the educational and pedagogical effects of Gen AI and large language models (LLM) in the HE context. How Gen AI can be exploited by students and teachers, particularly in computing education, is a vector of analysis that stands out.

The interrelations between ChatGPT, HE, AI, Gen AI, and assessment form the structure of the green cluster. Topics of analysis that are extensively explored in various publications include the implications of Gen AI for assessment methods, as well as the ability of Gen AI tools, such as ChatGPT in particular, to perform assignments that students then present as their own without referencing the use of these tools, and which can pass plagiarism detectors. These are grouped in the green cluster. Finally, the blue cluster grouped scientific production on how HE deals with the challenges that Gen AI imposes on academic integrity.

In terms of the country where the authors’ affiliated institutions are located, there is great diversity: 35 countries were identified (

Table 2. Country to which the institution of affiliation of the author(s) belongs.

Country	N.	Country	N.	Country	N.	Country	N.
Australia	16	Italy	2	Egypt	1	South Korea	1
United States	7	Saudi Arabia	2	Ghana	1	Sweden	1
Singapore	5	South Africa	2	Greece	1	Turkey	1
Hong Kong	4	Thailand	2	India	1	United Arab Emirates	1
Spain	4	Viet Nam	2	Iraq	1	Yemen	1
United Kingdom	4	Bulgaria	1	Jordan	1
Canada	3	Chile	1	Malaysia	1
Philippines	3	China	1	Mexico	1
Germany	2	Czech Republic	1	New Zealand	1
Ireland	2	Denmark	1	Poland	1

Source: Own elaboration based on VOSviewer (version 1.6.19) results.

). Note that each publication can be associated with more than one country.

Table 3. Distribution of citations by country to which the institution of affiliation of the author(s) belongs.

Country	N.	Country	N.	Country	N.	Country	N.
Singapore	271	United States	15	India	2	Iraq	0
Australia	187	Italy	11	Turkey	2	Jordan	0
Hong Kong	37	United Kingdom	6	Denmark	1	Poland	0
Thailand	33	Canada	6	Greece	1	United Arab Emirates	0
Philippines	31	Ireland	6	Sweden	1	Yemen	0
Viet Nam	29	Spain	6	Saudi Arabia	1
Malaysia	29	South Africa	6	Bulgaria	1
South Korea	29	Mexico	3	Czech Republic	0
China	17	Chile	3	Egypt	0
New Zealand	17	Germany	2	Ghana	0

Source: Own elaboration based on VOSviewer (version 1.6.19) results.

displays the distribution of citations in publications by country to which the institution of affiliation of the author(s) belongs. It is worth highlighting the significant number of citations registered by texts authored by researchers affiliated with institutions located in Singapore and Australia.

An inductive profile content analysis of the selected publications complemented the results of the bibliometric analysis (results are presented in

Table 4. Scientific production by categories and subcategories of effects generated by Gen AI in HE—results of the content analysis.

Categories	Subcategories	Nr. of Documents	References
HE with Gen AI	The key role that the pedagogy must have	15	Bearman and Ajjawi (2023); Chan (2023); Cowling et al. (2023); Crawford et al. (2023a); Currie (2023a); Currie et al. (2023); Eager and Brunton (2023); Farrokhnia et al. (2023); Ilieva et al. (2023); Lopezosa et al. (2023); Mondal et al. (2023); Solopova et al. (2023); Sridhar et al. (2023); Sullivan et al. (2023); van den Berg and du Plessis (2023); Watermeyer et al. (2023); Wolf et al. (2023); Zawiah et al. (2023).
	New ways to enhance the design and implementation of teaching and learning activities	15	Akakpo (2023); AlAfnan et al. (2023); Chang et al. (2023); Chiu (2023); Chun and Elkins (2023); Dai et al. (2023); Dogru et al. (2023); Farrelly and Baker (2023); Gong (2023); Hernández-Leo (2023); Kaplan-Rakowski et al. (2023); Pitso (2023); Rudolph et al. (2023a); Walczak and Cellary (2023); Wang et al. (2023); Yilmaz and Karaoglan Yilmaz (2023).
	The student’s role in the learning experience	14	Almaraz-López et al. (2023); Chan and Hu (2023); Chan and Zhou (2023); Duong et al. (2023); Elkhodr et al. (2023); Han et al. (2023); Javaid et al. (2023); Karunaratne and Adesina (2023); Kelly et al. (2023); Laker and Sena (2023); Lemke et al. (2023); Limna et al. (2023a); Tominc and Rožman (2023); Zawiah et al. (2023).
	The key teacher’s role in the teaching and learning experience	8	Álvarez-Álvarez and Falcon (2023); Al-Zahrani (2023); Athilingam and He (2023); Hidayat-ur-Rehman and Ibrahim (2023); Kohnke et al. (2023); Limna et al. (2023b); Ryall and Abblitt (2023); Santiago et al. (2023).
Assessment in Gen AI/ChatGPT times	The need for new assessment procedures	8	Boháček (2023); Currie (2023b); Currie and Barry (2023); Hassoulas et al. (2023); Moorhouse et al. (2023); Overono and Ditta (2023); Perkins et al. (2023); Rudolph et al. (2023b).
New challenges to academic integrity policies	New meanings and frontiers of misconduct, personal data usurpation and cheating	4	Bannister et al. (2023); Crawford et al. (2023b); Pechenkina (2023); Plata et al. (2023).

Source: own elaboration based on bibliometric and content thematic analysis.

). The detailed reading and analysis of the corpus allowed us to typify the main effects generated by Gen AI in HE highlighted in the literature.

The following sections outline the key impacts of Gen AI in HE according to the revised literature.

3.1.1. HE with Gen AI

The Key Role That Pedagogy Must Play

As argued by Crawford et al. (2023a), disruptive technologies only have a major impact when they positively transform practice and are informed by pedagogic models. A solid understanding of pedagogy is crucial because it enables the application of more effective teaching strategies and methods embedded with Gen AI systems.

The more creative and intelligent the dialogue building and the learning partnership developed with the intelligent assistants, the better the results will be. According to Eager and Brunton (2023), the best way to interact with intelligent assistants is to have a solid culture and a clear notion of how to explore the added value that the existing resources can provide. From a pedagogical point of view, it is essential to ensure the incorporation of Gen AI as systems that broaden the possibilities of cognitive cooperation, resulting in more personalized and effective educational experiences (Chan 2023).

According to the literature, Gen AI has been criticized for its tendency to generate inferior results. These results are typically described as uninformed, incorrect, and biased responses. However, Crawford et al. (2023a) highlighted that this limitation can be seen as a valuable chance to create teachable moments. It allows educators to help students improve their critical analysis skills in an AI-driven society.

When confronted with change, significant challenges and skepticism always present invaluable opportunities for personal growth and advancement. In this context, Eager and Brunton (2023) emphasize the importance of adopting a well-balanced approach that effectively tackles the challenges and opportunities associated with integrating Gen AI into academic settings. It is crucial to have a thorough understanding of the implications of this integration, ensuring that all stakeholders, including teachers, students, and HEI governance, are fully aware of their responsibilities and can take appropriate actions accordingly. As argued by Chang et al. (2023), this calls for the adoption of pedagogical and practical guidelines for implementing Gen AI in teaching and learning contexts, focusing on promoting students’ ethical, self-regulated, and instructed digital use.

New Ways to Enhance the Design and Implementation of Teaching and Learning Activities

Based on the examination of the effects produced by Gen AI in the creation of learning activities within various specialized teaching domains, including computer education, medicine education, teaching education, and foreign language teaching, among others (a total of 15 of the 64 publications yielded relevant outcomes), it can be inferred that modifications in the curriculum and teaching and learning methodologies are of utmost importance.

There are new resources that can and should be exploited in a pedagogical context to facilitate the appropriation of the defined knowledge and competencies (Chun and Elkins 2023). Rather than banning the use of Gen AI tools, as argued in some HEIs, educators should adopt them as learning assistants (Hernández-Leo 2023).

Several authors argue that Gen AI can be explored as a student-driven innovation with immense potential to empower students and enrich their educational experiences and resources (Dai et al. 2023; Sridhar et al. 2023; Hernández-Leo 2023; Wang et al. 2023). However, for this empowerment to be effective, collaborative efforts are required among the educational stakeholders to address the new and emerging challenges regarding student training, higher education curricula, and assessment (Dai et al. 2023).

According to Rudolph et al. (2023a), the current conceptualization of Gen AI chatbots in education is deemed inadequate. Therefore, it is imperative to incorporate three essential educational principles: goal setting (prompting), self-assessment and feedback, and personalization. To ensure this, Rudolph et al. (2023a) propose the following measures:

• Firstly, prompting in teaching should be prioritized as it plays a crucial role in developing students’ abilities. By providing appropriate prompts, educators can effectively guide students toward achieving their learning objectives.
• Secondly, configuring reverse prompting within the capabilities of Gen AI chatbots can greatly assist students in monitoring their learning progress. This feature empowers students to take ownership of their education and fosters a sense of responsibility.
• Furthermore, it is essential to embed digital literacy in all teaching and learning activities that aim to leverage the potential of the new Gen AI assistants. By equipping students with the necessary skills to navigate and critically evaluate digital resources, educators can ensure that they are prepared for the digital age.

The Student’s Role in the Learning Experience

Several studies examine the correlation between students’ knowledge profile and their utilization of Gen AI tools, as well as their perceptions of the associated risks and benefits in teaching and learning. A total of 14 studies were identified in this subarea of research. The results highlight that students generally have a positive perception towards the use of Gen AI in education (Almaraz-López et al. 2023; Han et al. 2023; Karunaratne and Adesina 2023; Laker and Sena 2023; Lemke et al. 2023; Zawiah et al. 2023). In the research by Lemke et al. (2023), optimism toward the new technology is found to positively influence technology acceptance, while discomfort with the technology negatively influences the perceived impacts of its use. Additionally, Han et al. (2023) conclude that low expectations regarding the use of Gen AI are linked to the possible generation of dependence on its use, while high expectations are linked to the possibility of making it easier to solve specific academic needs. This raises the need to balance the benefits and harms of Gen AI use, considering students’ particular needs (Han et al. 2023).

For students to benefit most effectively from Gen AI tools and environments, it is important to provide students with critical thinking skills. The use of these tools could potentially lead to plagiarism if not used responsibly. To prevent such occurrences, it is crucial to establish clear guidelines, enhance classroom assessments, and require the reporting of Gen AI usage, as recommended by Farrelly and Baker (2023).

Gen AI has the power to fortify and amplify the cognitive presence of the student (Kelly et al. 2023). Therefore, it is crucial to encourage the exploitation of Gen AI as a co-piloting resource for the learning journey (Elkhodr et al. 2023).

The Key Teacher’s Role in the Teaching and Learning Experience

Three studies have been conducted to explore teachers’ perceptions regarding the educational use of ChatGPT. These studies have identified that the chatbot is widely regarded as a valuable tool for delivering prompt feedback, addressing inquiries, and offering support to students. Educators also noted that ChatGPT could reduce their workload by answering routine questions and enabling them to focus on higher-order tasks (Hidayat-ur-Rehman and Ibrahim 2023; Kohnke et al. 2023). However, the findings also showed some concerns regarding using ChatGPT in education. Participants were worried about the accuracy of information provided by the chatbot and the potential loss of personal interaction with teachers. The need for privacy and data security was also raised as a significant concern (Hidayat-ur-Rehman and Ibrahim 2023).

According to Kohnke et al. (2023), perceived unfair evaluation of students, perceived overreliance of students on it, and perceived bias/inaccuracies were shown to have significant impacts on educators’ resistance to using the Gen AI tools. For Al-Zahrani (2023), teachers should not treat Gen AI as “an enemy”; instead, they should find ways to work with it for the betterment of learning outcomes for students. Working with Gen AI has the potential to revolutionize teaching and reshape the student–teacher relationship, leading to mutually beneficial outcomes and enriching learning experiences for all parties involved (Limna et al. 2023b).

According to Álvarez-Álvarez and Falcon (2023), to ensure improvements in educational practices and the learning process, it is crucial for teachers to have a comprehensive understanding of available resources and their suitability within the defined model and pedagogical path. It is fundamental to be explicit with students about how Gen AI can scaffold their learning and how they can work together with Gen AI, with the teacher maintaining the role of main mediator and facilitator of learning (Kohnke et al. 2023).

3.1.2. Assessment in Gen AI/ChatGPT Times

The Need for New Assessment Procedures

In the age of “postplagiarism”, where hybrid human–AI generated writing will become standard, as stated by Moorhouse et al. (2023), assessment tasks need to be redesigned. This should include reducing the overall reliance on assessments where Gen AI may be used to mimic human writing or using AI-inclusive assessments (Currie 2023b). With the potential for Gen AI to serve as a co-orientator and learning regulator, it is crucial to leverage its feedback capabilities, its access to a wide range of information sources, and its ability to offer diverse solutions to problems during the evaluation process (Currie and Barry 2023). Instead of asking students to simply reproduce knowledge in essays or exams—tasks that are easily accomplishable for a Gen AI tool like ChatGPT—students should engage in class-specific, guided self-reflection and take part in assessing their own work (Overono and Ditta 2023).

For Perkins et al. (2023), “the future of academia should focus on striking a balance between harnessing AI technology responsibly and maintaining the sanctity of its academic integrity” (p. 108).

3.1.3. New Challenges to Academic Integrity Policies

New Meanings and Frontiers of Misconduct, Personal Data Usurpation and Cheating

According to Plata et al. (2023), ethical principles and proper education of the educational community are key to using Gen AI as a tool in knowledge generation. HEIs must ensure that Gen AI is used in a way that aligns with their values, mission, and pedagogical framework and that the whole educative community is informed about how it can or cannot be used.

It is imperative to update the academic integrity policies and/or honor codes to incorporate the utilization of Gen AI tools. This step is paramount in maintaining a professional and ethical academic environment (Bannister et al. 2023).

The expansion of the definition of academic misconduct, along with the potential for data appropriation and misuse, particularly by deepfakes, are significant concerns that demand deliberate and proactive measures. These issues require our utmost attention and a proactive approach to address them effectively (Crawford et al. 2023b).

Next, we present the HE students’ perceptions about the impacts of Gen AI use in the HE context.

3.2. Students’ Perceptions about the Impacts of Gen AI in HE

To identify and characterize HE students’ perceptions about the risks and benefits of Gen AI for HE, an online questionnaire was applied to an intentional sample of 112 students.

Regarding their experience using Gen AI tools, 52 of the 112 students (46.4%) had already used it, particularly in an academic context (

Table 5. Student sample—experience using Gen AI tools.

Have You Tried Using a Gen AI Tool?	Nr.	%
Yes	52	46.4%
No	60	53.6%
Total	112	100.0%

Source: Own elaboration based on the applied questionnaire, November 2023.

).

The 52 students with experience using Gen AI in an academic context then responded to questions about their opinions on the effects generated by Gen AI in HE. When questioned about whether Gen AI has more positive or negative effects on HE, 41.1% believe there is a balance (Figure 3). The group that believes that there are more positive effects than negative effects is more pronounced (27.8%) compared to those who assume that the negative effects are more significant (only 10% of respondents chose this opinion).

When prompted to indicate the main positive effect of the use of Gen AI in an academic context, the results of the content analysis of the answers obtained in the open profile question allowed us to identify the most expressive thematic category and subcategories (

Table 6. Main positive effect of Gen AI in academic context.

Categories and Subcategories	%	Unit of Analysis (Some Examples)
1. Learning support:
1.1. Helpful to solve doubts, to correct errors	34.6%	“to help to study better” “to help in clarifying questions asked in practical work/practical sheets” “to clarify and simplify doubts or questions that we know less about” “to help students to solve doubts during their studies”
1.2. Helpful for more autonomous and self-regulated learning	19.2%	“help students to be more independent” “helps us to better manage what we learn” “it helps us not to have to ask so many questions/put doubts to teachers”
2. Helpful to carry out the academic assignments/individual or group activities	17.3%	“optimizer of work completion time and productivity” “to give a starting point in carrying out academic work” “to produce texts easier”
3. Facilitates research/search processes
3.1. Reduces the time spent with research	13.5%	“act as a search tool” “reduces research time”
3.2. Makes access to information easier	9.6%	“information is more accessible” “information becomes more accessible to everyone”
4. Reduction in teachers’ workload	3.9%	“helps the teacher in preparing the lesson” “eases the workload of teachers given the help of AI”
5. Enables new teaching methods	1.9%	“enables new teaching methods”

Source: Own elaboration based on the applied questionnaire, November 2023.

), which are as follows:

Gen AI is a “learning support”:

▪

“It is helpful to solve doubts or correct errors”, namely “to help to study better”; “to help in clarifying questions asked in practical work/practical works”; “to clarify and simplify doubts or questions that we know less about”; “to help students to solve doubts during their studies”;

▪

“It is helpful for guaranteeing more autonomous and self-regulated learning”, namely “to help students to be more independent”; “to help us better manage what we learn”; “to help us not to have to ask so many questions or put doubts to teachers”.

Students consider that Gen AI tools can support their learning paths and highlight their potential use to self-regulate learning and facilitate the resolution of doubts. The help that Gen AI tools can give in developing learning assignments, the fact that they can speed up the process, and the time devoted to research are also highlighted. The possibility of dynamizing new teaching methods and reducing teachers’ workload in class planning are also positive effects identified by some students, although with a lower number of responses (Table 6).

From the results achieved (Table 6), it can be inferred that, as proven by the scientific literature analysed, Gen AI supports these students and helps to simplify the learning process for them. It is, therefore, essential to ensure that this appreciation is contextualized and complemented by using new teaching and learning methodologies duly supported by pedagogical guidelines that guarantee the implementation of reflective and critical learning, as highlighted by scientific analysis. Also, students recognize the potential of Gen AI for the research process and the quality of the teacher’s teaching/learning work; therefore, it is essential to capitalize on this recognition by exploring it coherently and sustainably.

At the level of the main negative effect of Gen AI (

Table 7. Main negative effect of Gen AI in academic context.

Categories and Subcategories	%	Unit of Analysis (Some Examples)
1. Harms the learning process:
1.1. What is generated by Gen AI has errors	13.5%	“AI generates many errors and is not 100% reliable” “because what is generated has to be worked on and not used as it is generated because there are many errors” “leads to error”
1.2. Generates dependence and encourages laziness	15.4%	“total dependence on the use of tools to solve a problem presented” “lack of trying to do things and asking the AI to do things for us”
1.3. Decreases active effort and involvement in the learning/critical thinking process	28.8%	“bad learning” “reduces the knowledge needed to do the course” “laziness about learning” “many use it to do work without understanding what they are doing and without learning anything”
2. Encourages plagiarism and incorrect assessment procedures	17.3%	“the use of these resources for evaluations” “students use artificial intelligence for illicit purposes”
3. Reduces relationships with teachers and interpersonal relationships	9.6%	“helps students to lose contact with teachers, believing that the solution to their doubts is there” “socially isolates”
4. No negative effect—as it will be necessary to have knowledge for its correct use	7.7%	“as long as they are used judiciously and appropriately, they can help and not harm” “correct use helps… only if used incorrectly”
5. Don’t know	7.7%	“I don’t know”

Source: own elaboration based on the applied questionnaire, November 2023.

), students highlight the following:

• “It harms the learning process”:

  ▪

  “What is generated by Gen AI has errors”;

  ▪

  “Generates dependence and encourages laziness”;

  ▪

  “Decreases active effort and involvement in the learning/critical thinking process”.

The encouragement of plagiarism is also highlighted as a potential adverse effect.

Seven percent of students believe that Gen AI has no adverse effects if used properly, and seven percent do not know any negative effects.

For the students, the most concerning consequence is the potential of Gen AI to decrease students’ active effort and involvement in the learning/critical thinking process (Table 7). As in the studies developed by Akakpo (2023), Cowling et al. (2023), and Yilmaz and Karaoglan Yilmaz (2023), students show awareness of the potential unethical or incorrect effects of using Gen AI, as well as the errors and hallucinations generated by the systems. Plagiarism, the adulteration of the evaluation process, and the artificialization of the relationship between the teacher and the learning context are also significant adverse effects. However, the majority think that the effects generated by its use in an academic context will be more positive than negative.

The findings from the survey conducted among a group of students underscore the significant impact that the integration of Gen AI in an academic setting is currently having. It is evident that Gen AI is playing a crucial role in streamlining academic work and research processes. The majority of students have reported experiencing more positive than negative outcomes as a result of using Gen AI. To sustain these positive effects, it is imperative that all stakeholders in HE take conscious and critical steps to maximize the benefits of Gen AI while mitigating its potential risks. This calls for a concerted effort to enhance awareness and understanding of the capabilities and limitations of Gen AI in the context of learning.

4. Discussion

One of the first conclusions drawn from the analysis conducted in this study is the essential role that pedagogy must play in guiding the integration of Gen AI into the educational path. Critical and constructivist pedagogy is essential because it emphasizes the importance of analyzing and questioning (Crawford et al. 2023a). It is imperative to develop students’ critical thinking skills to sift through copious amounts of data effectively and to make well-informed decisions. This need is reinforced by the evidence gathered in this study, that most students use Gen AI to resolve doubts, correct errors when carrying out academic work, or self-regulate learning.

Another critical aspect highlighted in the literature and induced from the survey is the crucial role that teachers must play in this “new” learning environment. It is up to the teacher to reinforce the foundational importance of human relationships in the pedagogical process in the face of the seductive temptation of automating processes, greatly facilitated by Gen AI (Hidayat-ur-Rehman and Ibrahim 2023; Moorhouse et al. 2023).

The widespread appeal of the benefits often leads to overlooking the detrimental consequences associated with Gen AI systems. These systems rely on algorithmic-based correlational models that generate data based on predominant numerical trends, which can be biased and fail to account for unique cases and profiles that deviate from the norm (Han et al. 2023). Despite the awareness that the Gen AI system generates errors and hallucinations, the immediate benefit generated tends to lead to an underestimation of the short-, medium-, and long-term negative effects induced. In the specific case of the learning process, overestimation of the potential of Gen AI is very associated with a decrease in effort and time dedicated to the learning process, as well as with the devaluation of critical and active involvement, as some studies report (Elkhodr et al. 2023; Javaid et al. 2023; Kelly et al. 2023).

Despite recognizing the potential negative effects associated with Gen AI in an academic context, it is also assumed that prohibiting its use, as some HEIs have done, is not an option. As reported in different teaching and learning experiences, the overall outcomes of Gen AI tools can and should be explored pedagogically (Athilingam and He 2023; Mondal et al. 2023). This can be performed to promote innovation in teaching methods, to improve pedagogical content creation, or to enhance the exploration of learning paths that are more focused on the students’ active participation via doubt-based learning or even prompt-based learning (Farrokhnia et al. 2023). Gen AI can be used to enhance student learning by providing personalized, real-time feedback and adapting to individual learning styles (Chan and Hu 2023; Sullivan et al. 2023). Several publications also argue for the pedagogical exploration of the errors and biases generated by Gen AI tools (namely ChatGPT) for the benefit of a more critical, participative, and ethical learning path (Walczak and Cellary 2023; Sridhar et al. 2023).

These findings underline the need for a comprehensive AI policy in HE that addresses the potential risks and opportunities associated with Gen AI technologies. According to Chan (2023, p. 12), a Gen AI HE teaching and learning policy should include:

• Training: providing training for both students and teachers on effectively using and integrating Gen AI technologies into teaching and learning practices.
• Ethical use and risk management: developing policies and guidelines for ethical use and risk management associated with Gen AI technologies.
• Incorporating AI without replacing humans: incorporating AI technologies as supplementary tools to assist teachers and students rather than replacements for human interaction.
• Continuously enhancing holistic competencies: encouraging the use of AI technologies to enhance specific skills, such as digital competence and time management, while ensuring that students continue to develop vital transferable skills.
• Fostering a transparent AI environment: promoting an environment in which students and teachers can openly discuss the benefits and concerns associated with using AI technologies.
• Data privacy and security: ensuring data privacy and security using AI technologies.

The impacts of Gen AI on the foundations of the educational experience require diverse kinds of responses involving not only educators and learners but also the institutional and policy stakeholders (Wang et al. 2023).

At the level of the HEIs’ governance body, it is essential to recognize the need for a new pedagogical framework. This framework should aim to harness the potential added value of Gen AI to improve the educational mission while mitigating any potentially harmful effects on academic integrity, security, and data protection. This strategic shift will enable HEIs to adapt to the evolving technology and education landscape, ensuring a balanced approach that maximizes benefits and minimizes risks. It is crucial to establish organizational identity and institutional direction that outlines the principles and guidelines for the appropriate integration of Gen AI within an academic setting (Chan 2023).

In addition to adapting pedagogical practices, it is also crucial to adapt and update organizational practices. This includes redesigning the following:

• The dynamics of technological support to align with the most suitable Gen AI resources;
• The training policy to ensure that teachers, students, and academic staff are properly trained to utilize the potential of Gen AI and its tools;
• Security and data protection policies;
• Quality and ethical action policies.

These organizational practices must be adjusted to incorporate Gen AI technology effectively and promote a safe and ethical learning environment.

HEIs must acknowledge the necessity of transitioning to a new organizational, technical, and pedagogical framework to effectively adjust to the various influences of Gen AI and successfully carry out the goal of educating individuals. This shift is crucial as everyone will be involved in an environment where the interaction between humans and technology will be appreciated in various ways and for different reasons.

5. Conclusions

In a rapidly evolving world where AI algorithms play a significant role in shaping content, students’ widespread use of these algorithms is evident. The results of the survey conducted in this study underscore the importance of HE in equipping students with the skills to critically evaluate sources of information, assess the evidence presented, and identify any potential biases. It is imperative for an educative system to nurture capable citizens who can navigate the complexities of information in the digital age. Knowing how to recognize the harmful effects of Gen AI misuse is also fundamental. Thinking critically allows us to make informed choices and avoid falling prey to misinformation or manipulation.

Ensuring adequate pedagogical guidance and resources to equip students (and citizens) with skills to think and act critically in a world driven by AI is a mandatory path that the HE system must follow. By opening new approaches to teaching and learning, AI could serve as a means to help students become creators of knowledge rather than merely consumers, and teachers could truly become facilitators of learning rather than transmitters of knowledge. While some are largely fixated on the potential problems created by the possibilities of plagiarism and academic integrity, others (the majority in the literature revised) prefer to focus on possibilities for using Gen AI technologies to generate new teaching and learning activities and new ideas to engage and personalize the educative experience.

The results of this study emphasize the importance of HE and its key stakeholders (teachers, students, and governance bodies) taking a proactive approach to defining and implementing the use and extent of Gen AI in the educational setting. It is crucial to thoroughly assess the advantages and potential drawbacks to ensure that Gen AI is utilized to its fullest potential and effectively addresses the unique requirements and obstacles faced in the teaching and learning processes. This issue is also the responsibility of policy stakeholders and is a crucial aspect missing in the revised literature. It is imperative to examine how national governmental HE policies address this topic. Further research is necessary to fully understand the implications.

The changes induced by Gen AI in the HE system will be increasing and disruptive. While this study provides a valuable snapshot of the current state of knowledge, some limitations should be noted:

• Database constraints: the analysis is based on existing publications in SCOPUS and the Web of Science, potentially omitting relevant research from other sources.
• Inclusion criteria: due to the early stage of scientific production on this topic, all publications were included in the analysis, rather than focusing solely on articles from highly indexed journals and/or with a high number of citations as recommended by bibliometric and systematic review best practices.
• Dynamic landscape: the rate of publications on Gen AI has been rapidly increasing and diversifying in 2024, highlighting the need for ongoing analysis to track trends and changes in scientific thinking.

To fulfill the research objective of exploring the usage patterns of Gen AI among students in an academic setting and gather their opinions on its effects, an intentional group of students was selected for data collection. The study cohort exhibits a significant gender imbalance, potentially skewing the findings and limiting the generalizability of the results. The results obtained are only applicable to this specific sample. To enhance the validity of the findings, future studies should consider both increasing the sample size and including students from a diverse range of training areas.

Another fundamental area of future research is the implications of incorporating Gen AI into pedagogy. It is imperative to analyze the impact of Gen AI on the learning process, including its integration into teaching and learning activities, its influence on student engagement, the implementation of pedagogical support for Gen AI in HEIs, and the incorporation of Gen AI skills into national educational frameworks by educational policymakers. Overall, as Gen AI will deeply shape the landscape of HE, it is crucial to stay up to date with emerging trends and perspectives in order to adapt and respond effectively.