ChatGPT: Challenges and Benefits in Software Programming for Higher Education

Abstract

ChatGPT is a substantial language model developed by OpenAI, rooted in the GPT-3.5 architecture, with the capacity to generate human-like responses to text-based inputs. ChatGPT serves various purposes, encompassing chatbots, customer service, and personal assistants, which can significantly contribute to sustainability initiatives. Its applications range from language translation and content creation to text summarization. Utilizing ChatGPT offers several advantages, notably its rapid response generation, high accuracy, and its capacity to evolve and improve over time, aligning with sustainability goals for efficiency and innovation. In an educational context, ChatGPT can provide invaluable support to students and educators, aiding in tasks such as generating summaries for extensive texts and addressing subject-related queries. For programming education, ChatGPT can assist students with coding assignments by offering suggestions, hints, and even generating code snippets, fostering sustainable coding practices. Nevertheless, employing ChatGPT in coding education presents challenges, particularly the risk of students becoming overly dependent on AI-generated code and failing to grasp fundamental concepts, which can hinder long-term sustainability in the field. To gauge the viability of ChatGPT in programming education and sustainability, we conducted a Likert scale questionnaire with a group of 40 Brazilian students from March to April 2023. Our primary goal was to assess students’ interest in utilizing ChatGPT as a tool to face programming challenges and problems. Specifically, we aimed to determine their level of inclination towards relying exclusively on ChatGPT during programming classes. In addition to these objectives, we sought to discern not only the positive and beneficial perceptions of using ChatGPT in the classroom but also to investigate its potential impact on learning outcomes and student engagement. Furthermore, we aimed to explore whether participants would consider transitioning to exclusive reliance on ChatGPT in the context of their programming education. Our study revealed that students recognized ChatGPT as an innovative set of AI tools applicable to various classroom contexts, including programming and computer languages, thereby fostering sustainability in the adoption of AI technology for educational purposes. Notably, a majority of students participating in the study expressed a keen interest in employing this tool as a supplementary educational resource in the classroom, promoting sustainable and enhanced learning experiences.

1. Introduction

Artificial Intelligence (AI) represents a branch of computer science dedicated to the development of intelligent machines proficient in tasks typically necessitating human intelligence, including learning, problem-solving, perception, and decision making. This advancement in AI technology has profound implications for sustainability as it can enhance efficiency, resource management, and eco-friendly decision-making processes across various sectors. AI plays a pivotal role in task automation, data analysis, and the provision of insights and recommendations that human capabilities might overlook. Common AI applications encompass natural language processing, image and speech recognition, and predictive analytics, as well as robotics and autonomous systems. This AI-driven transformation holds great promise for sustainability, as it can optimize productivity, efficiency, and accuracy across diverse industries, ranging from healthcare and finance to manufacturing and transportation. AI’s potential to enhance sustainability is particularly evident in its capacity to streamline operations and resource management, reducing waste and energy consumption while bolstering eco-friendly practices. Moreover, AI’s impact extends to the interface of human interaction with AI-based chatbots [1,2], offering a sustainable means of improving customer service, reducing response times, and providing valuable insights that contribute to overall eco-conscious business practices. AI-powered chatbots [3], such as ChatGPT, can significantly impact human interaction interfaces, have been attracting headlines, and have become the center of an ongoing debate regarding the potential negative effects that they may have on education. They offer personalized support and can handle routine tasks, freeing up human resources for more complex activities. With machine learning capabilities, chatbots can continuously improve and enhance customer experiences.

One reason for ChatGPT’s popularity is its ability to generate highly coherent and contextually appropriate responses [4]. Since its release, it has established itself as a cultural sensation around the world [5]. Unlike earlier chatbot models, which often produced nonsensical or irrelevant responses, ChatGPT can generate highly human-like conversations that are difficult to distinguish from those between real humans. Additionally, ChatGPT is highly flexible and can be fine-tuned for specific applications, making it a versatile tool for businesses and developers. It can also be trained on large datasets, allowing it to improve its performance over time and adapt to changing user needs. Overall, ChatGPT represents a major advance in the field of natural language processing (NLP) and is poised to become a key tool for businesses, developers, and individuals looking to build intelligent conversational agents. The field of NLP is now gradually turning over a new leaf by leveraging ChatGPT’s ability to continue a discussion over multiple questions and create software code [6]. ChatGPT has been widely used in various fields for natural language processing tasks. Its integration has been observed in customer service chatbots [7], educational chatbots [8], healthcare chatbots [9,10], financial chatbots [11], gaming chatbots [12], marketing chatbots [13], news and media chatbots [14], personal productivity chatbots [15], social media chatbots [16], and language translation chatbots [17]. In most cases, chatbots have been presented as a technologically innovative solution, as they combine customer service, emerging technologies, rapid response to real-time events, corporate decision making, a low cost of implementation, and flexible interaction customization in a single environment with users [18,19,20].

ChatGPT has been widely recognized as a powerful tool that has the potential to revolutionize the field of education. It can be used in a variety of courses, including language learning, literature, and history, to provide instant feedback, clarify concepts, and assist with personalized learning [21]. ChatGPT’s natural language processing capabilities make it an ideal tool for improving writing skills and facilitating communication between teachers and students, especially in higher education [22]. Chatbots powered by ChatGPT can be integrated into learning management systems to provide additional assistance with assignments and answer questions. Moreover, ChatGPT can be used to generate content for online courses, such as quizzes and exams, which can be tailored to individual learning styles and abilities. Its ability to analyze large amounts of data can also be used to create personalized learning plans and track student progress. Overall, the use of ChatGPT in education has the potential to enhance student learning outcomes [23], improve the quality of instruction, and make education more accessible and engaging for all learners, including primary school students [24]. ChatGPT presents significant threats and weaknesses in the educational context, especially concerning privacy [25]. The tool’s misleading privacy practices, as highlighted in the OpenAI website, where conversations are stored despite previous denials, raise critical concerns, particularly for users with limited technology and privacy knowledge, such as young learners. Ethical and regulatory concerns include the necessity for users to fact-check answers generated by ChatGPT to avoid the dissemination of misinformation in educational materials. Plagiarism is a major concern, with the potential for students using ChatGPT to engage in AI-assisted plagiarism, prompting diverse responses from institutions. The risk of social inequality emerges as a concern, as only those with sufficient resources may access advanced versions of ChatGPT, creating an unfair advantage. Additionally, the prospect of ChatGPT becoming the sole information source raises important questions about media literacy and the potential loss of diversity in information sources [25].

In the realm of software education, chatbots have emerged as a valuable asset in teaching coding programs [26,27] or disseminating best practices in software engineering [28]. Students engage with ChatGPT, receiving immediate responses, thereby enabling self-paced learning and fostering a deeper grasp of intricate coding concepts. This approach not only supports individualized education tailored to the unique needs of each student and preferences but also contributes to sustainability by optimizing educational resource utilization. ChatGPT serves as a potent tool for addressing coding software challenges within the educational sphere. Its capacity to offer personalized feedback encourages critical thinking and active learning, thereby diminishing the temptation to engage in passive copying and pasting. Furthermore, ChatGPT’s adaptability to various levels of coding proficiency ensures tailored assistance for each student. By incorporating ChatGPT into coding education, we facilitate a profound understanding of programming concepts and cultivate more effective learning practices, thereby promoting sustainability through the development of well-rounded and eco-conscious skill sets. However, it is essential to address the concern of code plagiarism, as it can hinder students’ progress and ultimately compromise their coding skills, underscoring the need for ethical and sustainable educational practices.

In this paper, we present an analysis of the integration of ChatGPT into educational settings, with a particular emphasis on courses related to coding and software programming. Our examination delves into the benefits of incorporating ChatGPT into the learning environment, along with an exploration of the potential drawbacks and challenges for educators, including the sustainability aspect of code copying and pasting by students. To initiate our study, we introduced first-semester students to the concept of ChatGPT and its diverse applications within a classroom setting. Subsequently, after a few weeks, the same group of students received a more focused introduction to ChatGPT, specifically in the context of computer coding and programming. Building upon the practical experience gained from utilizing this chatbot as an educational tool, we administered a questionnaire to gauge individual perceptions and interests among students regarding the utilization of ChatGPT in computer coding and programming. This assessment encompasses not only academic interests but also sustainable educational practices and their implications for fostering well-rounded, eco-conscious skill development. Our main research questions in this project are 1. How does the integration of ChatGPT in coding and programming courses impact student perceptions of educational support, sustainability, and their individual learning experiences? 2. To what extent does the increased reliance on ChatGPT in programming education pose challenges related to ethical considerations, code plagiarism, and potential drawbacks in fostering independent problem-solving skills among students?

As a significant contribution of this paper, we delve into an extensive exploration of the boundaries within which ChatGPT can assist both educators and learners in computer programming classrooms. Our objective is to enhance the educational process by embracing sustainable practices, ensuring that ChatGPT serves as a valuable tool for advancing coding and software knowledge rather than a mere shortcut for problem-solving. Furthermore, we tackle concerns related to the integration of ChatGPT in educational contexts and its potential to inadvertently dissuade students from pursuing in-depth theoretical and practical knowledge. The sustainability aspect pertains to maintaining a balanced approach that nurtures self-reliance and critical thinking, averting overdependence on chatbots for research and problem-solving in software coding. Concluding this endeavor, we engaged in applied research with students enrolled in an introductory programming course, embarking on hands-on experimentation and the practical utilization of ChatGPT within the classroom setting. Subsequently, we scrutinized the students’ perceptions, examining how they perceive this tool: as an educational support resource that encourages sustainability in learning and teaching practices, or as a quick-fix solution for programming challenges, potentially undermining their growth as independent problem solvers in the field of computer programming.

It is known that the use of AI tools in education has been widely explored over the years. However, it is important to highlight that, in 2023, we are witnessing a significant change in the educational scenario, marked by a notable increase in the use of AI tools, such as ChatGPT, by students. To illustrate this scenario, Vision Super [29] shows, through Google search terms, a dramatic increase in interest in AI in the last months of 2023. Our study aims to capture and analyze this contemporary transformation, questioning not only students’ interest but also their inclination to rely exclusively on ChatGPT during programming classes. Thus, our research contributes to understanding the current implications of the growing use of AI tools in education, especially in the field of programming.

In addition to this introductory section, we present, in Section 2, a background on ChatGPT and its application in educational environments and computer programming teaching environments. Section 3 presents the methodology of the research work carried out with students and teachers through the perception questionnaire. The results are presented in Section 4. Finally, the conclusions of the work and suggestions for future work are presented in Section 5.

2. Background

2.1. ChatGPT Fundamental Topics

ChatGPT is a language model developed by OpenAI that uses deep learning techniques to generate human-like responses to text input. Specifically, ChatGPT is a variant of the GPT (Generative Pretrained Transformer) series of language models that has been fine-tuned on a large corpus of conversational data to enable it to respond to open-ended natural language prompts in a way that simulates human conversation [30]. ChatGPT has a wide range of potential applications, including chatbots, language translation, text summarization, and content generation, among others.

GPT is a family of language models developed by OpenAI that uses deep learning techniques to generate human-like text. These models are based on the Transformer architecture, which is a type of neural network that excels at capturing long-range dependencies in sequential data, such as natural language text. The first version of GPT, GPT-1, was introduced in 2018, followed by GPT-2 in 2019 and GPT-3 in 2020 [31,32]. Each new version of GPT has been larger and more powerful than the previous one, with GPT-3 currently being the largest and most advanced model in the series. GPT models are typically pretrained on large amounts of text data using unsupervised learning techniques, such as masked language modeling and next-sentence prediction. Once pretrained, these models can be fine-tuned on specific tasks, such as language translation, text summarization, and question answering, among others. GPT has become one of the most widely studied and influential language models in the field of natural language processing (NLP) [33] and has led to significant advances in a variety of NLP tasks.

GPT is a type of language model that is based on the Transformer architecture, which is a type of neural network that excels at capturing long-range dependencies in sequential data, such as natural language text. There are several other technologies that are related to or have been influenced by GPT, including (i) BERT (Bidirectional Encoder Representations from Transformers) [34,35], which is designed to generate bidirectional representations of text by training on large amounts of unlabeled data; (ii) Transformer-XL [36], a variant of the Transformer architecture that is designed to handle longer sequences of text by introducing new positional encoding techniques and memory mechanisms; (iii) XLNet [37], a language model that combines the ideas behind autoregressive models like GPT with the bidirectional training of BERT, resulting in a model that can generate high-quality text in a variety of contexts; (iv) T5 (Text-to-Text Transfer Transformer) [38], a language model that was specifically designed for text-to-text tasks, such as translation, summarization, and question answering, among others; and finally (v) GShard, a distributed training framework for large-scale language models like GPT that allows for training on multiple accelerators and enables the use of even larger models. These technologies are all related to GPT in various ways and have all contributed to the rapid progress that has been made in the field of natural language processing in recent years.

ChatGPT can be an ethical [39,40,41,42] problem in education and research publications for several reasons, as shown in

Table 1. Ethical issues using ChatGPT in education or research publications.

Problems/Issues	Description
Plagiarism	Since ChatGPT can generate text that closely resembles human writing, there is a risk that students or researchers may use it to generate academic work without properly citing or attributing the source of the text [39,43,44].
Bias	ChatGPT is trained on a massive corpus of text data, which may contain biases and inaccuracies that can be perpetuated in its generated text. This can have a negative impact on research studies and educational materials that rely on ChatGPT-generated content [45,46,47].
Authenticity	As ChatGPT can generate realistic text, there is a risk that it may be used to create fake news [48] or disinformation [47,49,50], which can have serious implications for research studies and educational materials that rely on accurate and truthful information.
Dependence	There is a risk that students or researchers may become too reliant on ChatGPT-generated content, potentially leading to a lack of critical thinking and independent research skills [51,52,53].
Ownership	There may be questions about the ownership of the content generated by ChatGPT, which could lead to legal and ethical issues regarding intellectual property rights and plagiarism [5,43,54,55,56]

.

Previously, just a few of the ethical concerns that arise from the use of ChatGPT in education and research publications were shown. It is important to carefully consider these issues and take steps to ensure that ChatGPT-generated content is used in an appropriate and ethical manner.

2.2. ChatGPT for Education

The use of ChatGPT in education is still in its exploration stage, with limited related research available due to its novelty [22]. A set of benefits and opportunities for teaching are presented in Figure 1, including learning, planning, development, writing, and evaluation process [57].

Recognizing the diverse learning preferences, capabilities, and needs of individuals, large language models offer a unique opportunity to foster personalized and sustainable learning experiences. By leveraging natural language processing and machine learning algorithms, these models can seamlessly adapt to the distinctive requirements and learning styles of each student, offering customized feedback that augments their overall educational outcomes. Furthermore, in the realm of higher education, these models hold the potential to serve as invaluable resources for enhancing writing skills, research proficiency, and comprehensive understanding [58]. Through their capacity to generate text, summarize information, and identify grammatical and stylistic errors, tools like ChatGPT not only save time but also enhance the quality of written assignments. Moreover, ChatGPT can introduce students to previously unexplored dimensions of their topics, thereby facilitating a more profound understanding and critical evaluation of the subject matter, contributing to sustainability in education [59].

For instance, Zhai [60] discusses the potential impacts of ChatGPT and similar AI tools on education, drawing upon user experience. Zhai suggests adjusting learning goals to focus on improving students’ creativity and critical thinking and emphasizes the importance of designing AI-involved learning tasks to engage students in solving real-world problems, rather than just developing general skills. Although ChatGPT has the potential to revolutionize the way we learn, it has also been a topic of controversy in education [61]. Some argue that reliance on AI tools like ChatGPT could lead to a decline in critical thinking, innovation, and creativity, while others see it as a valuable resource for enhancing students’ learning experiences [62]. Additionally, there may be questions about the accuracy and reliability of the information generated by ChatGPT, particularly in fields where precision and accuracy are crucial. Some educators may also worry about the potential for ChatGPT to perpetuate biases or reinforce stereotypes if not properly monitored and trained [63].

ChatGPT can be a valuable tool for detecting grammar and style errors in student writing, improving the overall quality of their work. By analyzing texts and offering suggestions for improvement, ChatGPT can help students identify and correct mistakes [64] they may have missed on their own. This can ultimately lead to better grades and a greater understanding of proper writing techniques. However, it is important to remember that ChatGPT should not be relied on exclusively, as it may not catch every mistake and cannot replace the value of human feedback and editing. For Baidoo-Anu and Ansah [46], beyond wrong information, ChatGPT can present other limitations. It can bring users bad experiences, such as biases in data training, which may accentuate existing biases, privacy issues, etc.

For students, beyond wrong data, they could create the false idea that the chatbot can provide one hundred percent of the information on specific topics or resolutions for problems, e.g., physics, math, or statistics.

The incorporation of AI language models such as ChatGPT in academic paper writing can offer notable advantages, fostering creativity and aiding in the writing process [65]. However, within this context, sustainability takes on a crucial role. Meanwhile, academic journal editors are increasingly grappling with the responsibility of identifying and rejecting papers that have been authored with the assistance of AI technology, reflecting concerns related to the maintenance of ethical and sustainable practices in academia [66]. Nonetheless, it remains imperative for researchers to bear the responsibility of ensuring the authenticity and proper attribution of the content generated by AI models, as AI-generated text carries the potential risk of being flagged for plagiarism. To safeguard sustainability, researchers should diligently scrutinize and revise any material generated by AI models, ensuring accuracy and clarity in communication, or they should avoid relying solely on ChatGPT as a sole source for papers, thus preserving the integrity and authenticity of their academic work [67].

ChatGPT is a versatile tool that is being increasingly used in education across a variety of fields, including dentistry [68], medicine [10,52,69,70,71], chemistry [72,73], education science [74], journalism and media [75], and many others. It can assist in teaching and learning [76] by providing information on various concepts, explaining procedures, answering questions, and generating examples and practice problems. However, it is important to note that ChatGPT should not be seen as a replacement for human teachers or mentors. Instead, it should be used in conjunction with other teaching methods to supplement learning and provide additional information and context. By using ChatGPT, students can develop their critical thinking and problem-solving skills, ask open-ended questions, and explore different perspectives. In summary, ChatGPT has the potential to be a valuable tool in education across many different areas, but it is important to use it responsibly and in conjunction with human guidance and support.

2.3. Coding Software Education Using ChatGPT

As technology advances, students have become increasingly dependent on it for their educational needs. From online research to virtual classrooms, technology has transformed the way how students learn. However, this dependence can also lead to a lack of critical thinking skills and the ability to learn independently. Using Python code snippets, ref. [27] looks at how chatbots (non-ChatGPT) can be used to teach students the foundations of programming concepts as mandated in the high school curriculum. Savelka et al. [77], evaluated GPT (non-ChatGPT) on three Python courses that employ assessments ranging from complex programming projects with code bases distributed into multiple files, with 599 exercises overall. Despite showing good results (>55%) for solving programming problems in C, GPT still presents challenges and limitations, such as the inadequate handling of exercises that require complex chains of logical reasoning steps.

For educators, ChatGPT became a good/bad tool to support the learning and teaching process, but it is worrying that students use this technology to write essays [78]. According to Biswas [79], ChatGPT in software engineering can conduct several tasks that include features such as those presented in Figure 2.

Surameery and Shakor [80] suggest that ChatGPT can contribute toward resolving programming bugs by facilitating debugging assistance, bug prediction, and bug explanation. Due to its proficiency in comprehending and scrutinizing code snippets, coupled with its aptitude for knowledge representation and natural language generation, it is highly proficient in executing these functions. Still, for fixing bugs, Sobania et al. [81] provide a comparison with different standard bug fixing benchmark sets, including GhatGPT. According to the author “The bug fixing performance of ChatGPT, …, is so far unclear”, and the results show that the bug-fixing performance of ChatGPT is competitive to the common deep learning approaches, solving 31 of 40 Phyton language bugs. On the other hand, programming codes can be a challenge for many students. Piccolo et al. [82] carried out an experiment with 184 programming exercises in Phyton from an introductory bioinformatics course. Results show that ChatGPT solved about 75.5% of the exercises correctly. In order to improve the number of correctly solved exercises, after each one, users provide natural language feedback to ChatGPT for better results. After a few attempts, ChatGPT presented a correct ratio of 97.3%. Even though it is a tool that optimizes the program coding process, researchers are already concerned that the code is decoupled from third-party libraries and simulates a web application API before its implementation [83]. An online behavioral task in HTML, CSS, and JavaScript code was developed using ChatGPT by [84], where the authors affirm that AI cannot completely replace programmers; however, bear in mind that using ChatGPT allows for obtaining detailed programming solutions which, at the same time, could reduce the time associated with programming.

However, it is not a problem to use AI tools to solve practical and real programming problems, as it optimizes the work and effort of programmers. On the other hand, we have educational environments such as schools and universities that need to adapt to this new technology. For students and teachers, while ChatGPT may initially appear as a sensational coding tool, there are teachers questioning whether students are actually learning or just copying and pasting code from automatic programs, or even submitting these educational exercises and projects that are evaluative.

On the topic of engineering education, Qadir [50] outlined the potential benefits of incorporating ChatGPT into classrooms. According to him, some researchers argue that the impact of AI is so profound that it will fundamentally transform the way programming is currently practiced. Jalil et al. suggest there is a potential promise or danger that ChatGPT might pose to traditional forms of instruction, especially to solve exams for software testing education [85].

It is notorious that the existence of these tools leads students to have a predisposition to use such technologies to solve problems, ranging from the simplest to the most difficult. This culture causes more and more students to lose the incentive to solve programming problems, for example, by conducting research using books, virtual programming groups, trial and error, analytical/logical thinking, and the frustration of spending hours trying to solve a highly complex problem. This trend can be detrimental to the development of critical thinking and problem-solving skills among students. Therefore, it is important to find a balance between the use of technological tools and traditional problem-solving approaches. To solve this problem, teachers are preferring to move online exams to paper formats after concerns that students have been using ChatGPT to complete recitation assignments in COSC 10, “Problem-Solving via Object-Oriented Programming” [86]. In the next section, we present the methodology of work where we investigate how students are using ChatGPT as a positive tool in order to increase their skills and abilities.

According to OpenAI [87], user-provided information is carefully safeguarded for ethical and privacy reasons, regardless of its sensitivity, ensuring the highest level of confidentiality. ChatGPT, a leading component of this commitment, does not retain memory or store specific information, such as the source code of programs shared by users. OpenAI’s architecture has been intricately designed with a primary emphasis on privacy and security, guaranteeing the absence of sensitive personal data or the specific details of past interactions. This underscores OpenAI’s dedication to preserving privacy and cultivating user trust in their interactions with ChatGPT. However, account information, including names, contact information, account credentials, payment card information, and transaction history, may be collected by OpenAI [87].

3. Methods

The methodology of this work is carried out in three distinct stages, as shown in Figure 3. In Step 1, we introduce ChatGPT to students so that they can interact with the chatbot and become familiar with its functionalities. Next, in Step 2, we conduct the experimentation with the students using ChatGPT on specific programming questions in the C language, suggesting specific and open-ended questions for the students. Finally, in Step 3, we seek to identify the perceptions and feelings of the students after using ChatGPT to help solve C programming problems where we try to understand how students will or pretend to use it in classrooms.

Firstly, we focused on collecting information related to the perception and initial understanding of students in the first two months of 2023. This research was conducted on forty students completing bachelor’s degrees in computer science and technology. Our research was carried out with students aged 17 to 27, with an average age of 19.51 years. Among the 40 participants, 31 were male and 9 were female. It is worth mentioning that more than 90% of the participants come from public schools in Brazil and that they have had almost no contact with computer technology, in particular with concepts of AI, logic, or programming languages. Although these characteristics increase the relevance of our findings to this specific demographic group, they limit generalizability to a broader population. Therefore, a significant limitation arises from the relatively small sample size, as our focus specifically targets first-year students in computer science and software courses. These incoming students had only recently been introduced to concepts of logic and programming languages within the last few weeks of February 2023, so their knowledge is at the level of beginners. Our primary goal was to assess their current knowledge level concerning programming languages and automated coding tools like chatbots, without any mention of ChatGPT to avoid bias in the results.

3.1. Step 1: First Interactions with ChatGPT

The first step of our scientific research involves collaborative exploration and hands-on interaction with the ChatGPT tool. The primary objective is to introduce ChatGPT (https://chat.openai.com/chat, accessed on 1 June 2023) to the students and allow them the freedom to ask their own questions, fostering an environment of autonomy and engagement.

In this first part of the step, all students participate together in the introductory session. They collectively posed a series of questions directly to the visual web interface of ChatGPT. The questions are designed to help the students understand the capabilities and functionalities of the AI-powered tool. The student groups worked collaboratively to explore the following questions using ChatGPT:

• “What are you, ChatGPT?”—The students will seek to understand the nature and identity of the AI tool.
• “How did you come into existence?”—The origin and development of ChatGPT will be explored.
• “What is your purpose?”—The students will inquire about the applications and uses of ChatGPT.
• “How do you function?”—The workings and mechanisms behind ChatGPT’s operations will be explored.
• “What are your available resources?”—The students will learn about the features and resources accessible through ChatGPT.
• “What are your limitations?” Understanding the boundaries and constraints of ChatGPT’s abilities.
• “Do you have knowledge in the field of computer science?”—The students will inquire about ChatGPT’s knowledge of computer science topics.
• “Is it possible for you to know programming languages?”—Exploring ChatGPT’s familiarity with programming languages.
• “Which programming languages do you know?”—Identifying the range of programming languages ChatGPT is acquainted with.
• “Can you solve a specific C language programming problem for me?”—The students will test ChatGPT’s capability to solve a given programming problem.

The objective of this second part is to delve deeper into the functionalities of ChatGPT and have an interactive session with the tool to understand its capabilities better. During this session, the students have the opportunity to ask more specific questions directly to the ChatGPT tool. The questions aim to explore programming concepts and the usage of specific tools like DEV C++ in programming in the C language. The students collectively asked the following questions to ChatGPT:

• “How can I start a program in the C programming language using DEV C++?” —Understanding the process of initializing a C program.
• “What are the purposes of the mentioned libraries?”—Exploring the functionalities of the mentioned libraries in the context of C programming.
• “What is the role of int main() function?”—Understanding the significance of the int main() function in C programs.
• “Why did you open and close the curly braces?”—Exploring the importance of curly braces in defining program blocks.
• “What happens if I do not include the curly braces?”—Understanding the consequences of omitting curly braces in the code.
• “What is a compiler?”—Exploring the role and functionality of a compiler in the context of programming.
• “You mentioned syntax errors. What are they?”—Understanding the concept of syntax errors in programming.
• “Why do some lines in C programming need a semicolon at the end? What could happen if I forget?”—Understanding the significance of semicolons in C programming and the potential consequences of their omission.
• “In the next two minutes, students are free to ask any questions related to the previous topic!”—Allowing the students to ask any additional questions they may have about the previous topics.
• “Based on the responses from ChatGPT, write a text that reflects your perception of this brief interaction with the Artificial Intelligence (AI) tool.”—Encouraging the students to summarize their experience and insights gained during the session.

Finally, in this final part of Step 1, students are encouraged to conduct independent research on different applications of AI in software development that can assist programmers. This segment offers the students the freedom to explore diverse topics related to AI implementation in software development. They had the opportunity to investigate and learn about innovative AI-driven tools and techniques that aid programmers in their work. This research part allows students to explore cutting-edge advancements in the field of AI and its practical applications. By delving into various AI-powered software development solutions, the students gained valuable insights into the potential of AI in revolutionizing the programming landscape and enhancing software development practices.

Throughout Step 1, we aimed to provide students with an enriched learning experience, fostering curiosity, critical thinking, and the hands-on exploration of AI technology. By the end of this phase, we assumed that the students had gained a deeper understanding of the capabilities and limitations of AI tools like ChatGPT, as well as the wider implications of AI in the field of software development.

3.2. Step 2: Training ChatGPT for Computer Science

The second step of our scientific research marks a more complex phase and happens exactly a week after the firststep, where students delve into the practical application of ChatGPT in the teaching and learning process of programming in the C language. This step aims to explore ChatGPT’s educational capabilities, encouraging students to interact with the tool and use it as a teaching aid. The students are given the freedom to ask their own questions and engage with ChatGPT in an exploratory and subjective manner.

During this part, students collectively explored various programming concepts with ChatGPT, seeking explanations, examples, and step-by-step procedures. The following topics are covered using ChatGPT: (1) What are variables? Please explain and provide an example in a C programming language program. Additionally, provide a detailed explanation. (2) What are integer, float, and character variables? Can you provide other examples? (3) Are there any rules or recommendations for declaring variable names? (4) Ask a question about variables related to the previous topic that you still have doubts about. (5) I want to create a C program that reads two grades, A and B, and calculates their average. Can you explain step-by-step how to write and execute the program in DEV C++? (6) Can you explain the purpose of the printf and scanf functions in the provided program? (7) Can the data type being read or written influence the way I write the printf and scanf commands? Can you provide an example to illustrate this? (8) Ask a question about data input and output related to the previous topic that you still have doubts about. (9) What are conditional structures? Please explain and provide an example in a C programming language program with comments throughout, followed by an explanation. (10) It is still not clear to me; can you provide another example with explanations? (11) Ask a question about conditionals related to the previous topic that you still have doubts about. (12) What is a compound conditional structure? Please explain and provide an example in a C programming language program with comments throughout, followed by an explanation. (13) It is still not clear to me; can you provide another example with explanations? (14) Ask a question about compound conditionals related to the previous topic that you still have doubts about.

Also, students are provided with a code challenge and they seek assistance from ChatGPT to identify errors and find solutions. This crucial part of the research involves providing students with a text-based exercise to solve in the DEV C++ IDE. The exercise involves programming a C program that calculates a person’s Body Mass Index (BMI) based on their weight and height. Students are encouraged to complete the task independently and they can ask ChatGPT for guidance or support whenever needed.

Additionally, the students are asked to extend the program based on a provided BMI table to determine a person’s obesity level. They could further inquire about the code implementation or any aspects they found unclear.

By the end of Step 2, students had experienced a direct and interactive approach to learning, where they actively asked questions, sought explanations, requested examples, inquired about doubts, asked for code debugging, and even requested complete codes for specific problems.

3.3. Step 3: Evaluation of Students’ Perception

The third and final step of our scientific research occurred one month after Step 2. During this period, the students had likely become more familiar with and started using the ChatGPT tool, either for coding or other related subjects. The objective of this step was to assess the overall perception of students regarding the existence of the tool and its various possibilities in solving programming problems in the C language.

This step is considered the most important as it aimed to address the critical question of how to query the students to determine if they would use the tool responsibly. It focused on understanding whether the students would rely on ChatGPT merely for answers (copy and paste), without attempting to solve problems on their own.

The data collection in this step involved an exploration of students’ perceptions and subjective feedback on the use of ChatGPT. A well-designed Likert scale questionnaire was employed, categorizing the questions into two groups, Group C and Group D, each with specific perceptions to assess. Group C questions aimed to gauge the general opinion of the students, while Group D questions focused on students’ conscious use of ChatGPT in the classroom or during their studies.

The Likert scale published in [88] is a widely employed tool in research to gauge respondents’ attitudes, opinions, and feelings toward a specific topic. It comprises a series of statements related to the subject, with response options ranging from agreement to disagreement. In the context of coding the interests and desires of a group of interviewees, the Likert scale provides a quantitative framework that facilitates the analysis and comprehension of responses. Through this scale, researchers can capture nuanced perspectives among participants. A Likert scale serves as a valuable tool for evaluating attitudes and opinions in international business research, showcasing its applicability across diverse contexts.

The questionnaire included the following questions, shown in

Table 2. Questionnaire.

Group/Question	Affirmatives
C/Q1:	ChatGPT is an interesting tool that can help with doubts in the classroom.
C/Q2:	ChatGPT can be used in conjunction with theoretical and practical classroom activities.
C/Q3:	In some situations, ChatGPT can replace the role of the teacher.
C/Q4:	ChatGPT can completely replace the teacher in any programming-related situations in the classroom.
C/Q5:	ChatGPT has a friendly interaction with me.
C/Q6:	ChatGPT promptly clarified my doubts when requested.
C/Q7:	The examples provided by ChatGPT are educational and enlightening.
C/Q8:	ChatGPT can quickly identify errors and bugs.
C/Q9:	ChatGPT seems to respond faster to doubts compared to the teacher in the classroom.
C/Q10:	In case of doubt and in the absence of the teacher, I will consult ChatGPT instead of using books or Google tutorials.
C/Q11:	Compared to YouTube videos, ChatGPT does not seem to be better than listening to someone explaining.
C/Q12:	In a fully online course, ChatGPT seems sufficient to solve the problem of lack of interaction with a teacher.
D/Q13:	When given an exercise by the teacher in the classroom, I first try to solve it on my own.
D/Q14:	When given an exercise by the teacher in the classroom, after attempting to solve it on my own without success, I seek the teacher, books, or videos for help.
D/Q15:	When given an exercise by the teacher in the classroom, I may seek help from ChatGPT.
D/Q16:	Without asking for the answer directly, I try to ask ChatGPT for initial explanations on how to approach the problem given by the teacher.
D/Q17:	Depending on the complexity of the code, I will always seek help from ChatGPT.
D/Q18:	It is easier for me to ask for the answer and study the provided solution by ChatGPT.
D/Q19:	I believe I can learn better if ChatGPT provides a ready-made code, even if it comes without an explanation.
D/Q20:	I believe my classmates will start using ChatGPT to help them with C programming.
D/Q21:	I believe my classmates will always use ChatGPT to solve C programming problems without trying on their own.
D/Q22:	I think ChatGPT might be harmful to me as I might be tempted to use it whenever necessary.
D/Q23:	I feel that ChatGPT might create a culture where “If I can’t do it, ChatGPT will always rescue me.”
D/Q24:	ChatGPT might prevent me from attempting to solve problems on my own before seeking its help.
D/Q25:	I believe in-person teachers should challenge me with code defenses as I might have used ChatGPT.
D/Q26:	I believe teachers should demand code defenses, as my classmates may have a higher chance of copying code from ChatGPT.

. We separated questions into two groups. Group C primarily focuses on understanding how students perceive ChatGPT as a supportive tool across various activities. The questions in this section aim to delve into students’ perspectives on the utility of ChatGPT in different contexts, providing insights into its perceived effectiveness and versatility. On the other hand, Group D concentrates on exploring students’ perceptions of ChatGPT in the context of teaching and learning programming. The questions in this section aim to uncover students’ views on the application of ChatGPT in educational settings, examining how students consider its benefits and drawbacks, specifically in the realm of programming education.

The Likert scale responses, ranging from “1—Completely Disagree” to “5—Completely Agree”, aimed to provide objective data regarding the students’ perspectives on ChatGPT usage. This data collection allowed for an understanding of potential biases and tendencies in students’ views on responsible usage, which could impact their education and technical training. By exploring the students’ perceptions and insights in Step 3, we gained valuable insights into the extent to which ChatGPT positively influenced their learning experiences and fostered a culture of responsible usage, critical thinking, and problem-solving skills. This final part played a pivotal role in assessing the overall effectiveness and impact of ChatGPT as an educational tool in the field of programming in the C language.

The results of the questionnaires are presented in the next section.

4. Results

In summary, the initial data underscore that students generally perceive their knowledge in the realms of AI, chatbots, automatic programming, and ChatGPT to be on the lower end of the spectrum. The prevalence of “Very Low” and “Low” ratings across the questionnaire implies that these emerging technologies are areas where students often feel less confident. This awareness of their limitations serves as a valuable starting point, encouraging students to actively seek out learning opportunities to bridge these gaps in understanding.

The responses to the given statements shed light on the perceptions of first-year students regarding ChatGPT’s role and impact in the classroom, as evaluated on a five-point Likert scale. Analyzing these responses provides valuable insights into the students’ perspectives, as shown in

Table 3. Results of the first group of questions: perception of students about using ChatGPT.

Affirmative	Strongly Disagree	Disagree	Neutral	Agree	Strongly Agree
C/Q1:	0.0%	2.7%	5.4%	37.8%	54.1%
C/Q2:	0.0%	5.4%	2.7%	59.5%	32.4%
C/Q3:	24.3%	32.4%	24.3%	16.2%	2.0%
C/Q4:	48.6%	29.7%	13.5%	5.4%	2.7%
C/Q9:	2.7%	29.7%	40.5%	13.5%	13.5%
C/Q12:	27.0%	32.4%	13.5%	13.5%	13.5%

. Descriptive statistics, including measures of central tendency and dispersion, are thoughtfully provided in Appendix A for a comprehensive examination of the data presented. This supplementary information increases the depth of the results, ensuring a thorough understanding of the statistical characteristics underlying the research findings.

The results from the affirmative Q1 reveal a notable consensus among participants, with an overwhelming 91.9% expressing agreement or strong agreement with the statement. This high level of positive response indicates a widespread recognition among students regarding the potential of ChatGPT as a valuable tool for resolving queries within the classroom setting. The remarkably low percentage of “Disagree” responses (2.7% in total) further strengthens the argument for the perceived utility of ChatGPT in addressing doubts during academic activities. Similarly, affirmative Q2 demonstrates a substantial agreement among participants, with a significant 92% responding positively with “Agree” or “Strongly Agree.” This robust consensus emphasizes the belief that ChatGPT has the capacity to complement both theoretical and practical aspects of classroom activities. The minimal occurrence of “Disagree” responses (5.4% in total) further consolidates the prevalent positive sentiment surrounding ChatGPT’s potential contributions to educational endeavors.

Affirmatives Q3 and Q4 offer insights into the perspective of using ChatGPT as a teaching tool during classes, yielding a more nuanced view. Responses to the Q3 statement present a diverse range, with 27.9% agreeing or strongly agreeing that ChatGPT could partially replace teachers in certain scenarios. However, a substantial 56.7% disagreed with this idea, showcasing a varied perception regarding the potential substitution of teachers by ChatGPT. Q4 results, in particular, are striking. A noteworthy 78.3% of students disagreed with the notion that ChatGPT could entirely replace teachers in programming-related situations. Conversely, only a small percentage (8.1%) agreed or strongly agreed with this concept. This underscores a robust conviction among students that human instructors remain indispensable for topics related to programming. In the context of affirmative Q9, where 40% of responses were neutral, it becomes evident that students lack a clear idea about whether ChatGPT is faster than teachers in providing feedback for student queries. This neutrality suggests uncertainty or a lack of consensus among students on the efficiency of ChatGPT in comparison with human instructors when it comes to addressing student doubts. The diversity of responses to Q3 highlights the complexity of views regarding the potential role of ChatGPT in the teaching environment. The strong resistance expressed in Q4 reinforces the irreplaceable value attributed to human instructors, particularly in the domain of programming education. The neutrality in Q9 signals an area of ambiguity, prompting further investigation into students’ perceptions and expectations regarding the speed and efficacy of ChatGPT in delivering feedback. This multifaceted analysis contributes to a more comprehensive understanding of the nuanced attitudes and reservations students may have towards integrating ChatGPT into the educational process.

From Q1 to Q4, the students’ responses reflect an overall positive perception of ChatGPT’s potential contributions to classroom activities and addressing doubts. However, there is a cautious approach to the extent of its role in replacing teachers, especially in programming-related scenarios. The students’ recognition of ChatGPT’s value as a supportive tool while acknowledging the irreplaceable role of human educators underscores a balanced perspective on the integration of technology in education.

Table 4. Results of group of questions: interactions of students using ChatGPT in programming statements.

Affirmative	Strongly Disagree	Disagree	Neutral	Agree	Strongly Agree
C/Q5:	2.7%	2.7%	24.3%	40.5%	29.7%
C/Q6:	5.4%	2.7%	16.2%	54.1%	21.6%
C/Q7:	0.0%	13.5%	32.4%	37.8%	16.2%
C/Q8:	0.0%	8.1%	35.1%	37.8%	18.9%
C/Q11:	5.4%	21.6%	24.3%	29.7%	18.9%

presents the perceptions of students about the facility and usability of ChatGPT for programming challenges.

The outcomes derived from the provided table illuminate the students’ perceptions and attitudes toward the integration of ChatGPT in an educational setting. Notably, a significant portion of students express agreement and strong agreement with various facets of ChatGPT’s interaction. In the case of C/Q5, approximately 70% of students perceive ChatGPT’s interaction as positive, suggesting that the tool is deemed to have a user-friendly interface. Furthermore, in C/Q6, a substantial 75.7% majority of students agree that ChatGPT is effective in promptly clarifying doubts, indicating its potential to offer swift assistance in the learning process. Of particular interest is C/Q7, where a noteworthy 70.2% of students find the examples provided by ChatGPT to be educational and enlightening. This outcome implies that the tool’s generated content positively contributes to students’ comprehension. Similarly, C/Q8 indicates that over 56% of students believe ChatGPT can identify errors and bugs, suggesting its potential as a valuable aid in debugging programming-related challenges. Turning our attention to C/Q11, approximately 48.1% of students lean towards agreement or strong agreement that ChatGPT may not surpass traditional methods, such as listening to explanations in YouTube videos. This perspective highlights students’ recognition of ChatGPT’s limitations compared with real-time visual and auditory learning methods. While the results underscore the positive impact of ChatGPT on students’ learning experiences, particularly in terms of clarifying doubts, providing educational examples, and assisting in identifying programming errors, it is crucial to acknowledge potential limitations. One limitation may be the reliance on self-reported perceptions, which might be influenced by individual biases or preconceptions. Additionally, the survey format may not capture the nuances of certain aspects, such as the multifaceted nature of learning through YouTube videos. Hence, while ChatGPT appears to offer valuable support, students seem to acknowledge its place alongside other learning methods rather than viewing it as a complete replacement. This nuanced perspective underscores the importance of considering ChatGPT as a supplementary tool in the educational landscape, complementing traditional methods and enhancing programming language acquisition.

The responses presented in

Table 5. Results of questions: integrations of ChatGPT into the learning process.

Affirmative	Strongly Disagree	Disagree	Neutral	Agree	Strongly Agree
C/Q10:	8.1%	13.5%	27.0%	27.0%	24.3%
D/Q13:	0.0%	2.7%	8.1%	35.1%	54.1%
D/Q14:	0.0%	10.8%	8.1%	48.6%	32.4%
D/Q15:	5.4%	10.8%	8.1%	51.4%	24.3%
D/Q16:	2.7%	8.1%	10.8%	45.9%	32.4%
D/Q17:	18.9%	32.4%	24.3%	18.9%	5.4%
D/Q18:	16.2%	16.2%	29.7%	24.3%	13.5%
D/Q19:	43.2%	18.9%	21.6%	5.4%	10.8%
D/Q20:	2.7%	2.7%	10.8%	37.8%	45.9%
D/Q21:	13.5%	29.7%	37.8%	10.8%	8.1%
D/Q22:	16.2%	29.7%	10.8%	24.3%	18.9%
D/Q23:	2.7%	21.6%	13.5%	40.5%	21.6%
D/Q24:	29.7%	13.5%	18.9%	24.3%	13.5%
D/Q25:	10.8%	16.2%	21.6%	32.4%	18.9%
D/Q26:	10.8%	13.5%	24.3%	27.0%	24.3%

reflect a spectrum of attitudes and perceptions towards ChatGPT’s integration into the learning process.

Several statements, such as C/Q10 and C/Q15, reveal a significant portion of students (ranging from 27.0% to 51.4%) leaning towards relying on ChatGPT when encountering difficulties in the absence of a teacher. This inclination might stem from the tool’s immediacy and accessibility, suggesting a trend towards embracing AI assistance as a viable option in educational settings. However, it is noteworthy that a balanced approach is essential to ensure that students still engage with other learning resources and cultivate problem-solving skills. A contrasting viewpoint emerges from statements like C/Q18 and C/Q24, where a notable proportion of students (ranging from 24.3% to 32.4%) acknowledge the value of personal effort and comprehension. They express reservations about solely relying on ChatGPT’s solutions without understanding the underlying concepts. These responses underscore the importance of integrating AI tools in ways that complement active learning rather than replacing it.

The most compelling insights arise from statements such as C/Q19 and C/Q20, revealing divided opinions about ChatGPT’s impact on the learning process. A substantial percentage of students (ranging from 21.6% to 45.9%) express a belief in the tool’s potential to aid their understanding, even if solutions lack explanations. Simultaneously, a significant number (ranging from 18.9% to 43.2%) indicate concern about potential over-reliance and the risk of hindering their own learning experiences.

Addressing the challenges in using ChatGPT for coding education, particularly the risk of students becoming too reliant on AI-generated code, is a crucial consideration based on findings from questions D/17, D/18, and D/19. The concern is significant, as the convenience provided by ChatGPT may lead students to overlook the development of fundamental programming skills. Ethical issues (D/17) arise when contemplating the origin and reliability of the generated code, while the need to foster learners autonomy (D/18) is pivotal in preventing detrimental dependency. The relevance of this concern is underscored by the importance of cultivating a deep and independent understanding of programming, as indicated in question D/19. Therefore, it is essential to balance the benefits of ChatGPT with educational strategies that encourage students’ autonomy and a profound comprehension of programming concepts.

Statement D/Q13 highlights that a significant majority of students (54.1%) initially attempted to solve exercises on their own before seeking external help. This demonstrates a positive inclination towards independent problem-solving and active engagement with the material, a crucial aspect of effective learning. In contrast, D/Q14 presents an intriguing trend, as a notable percentage (48.6%) express their willingness to seek assistance from various sources when their initial attempts fail. This balanced approach signifies an acknowledgment of the value of teacher guidance, resources, and collaborative learning, indicating a healthy learning mindset.

Students’ receptiveness to AI assistance is evident in D/Q16, with 45.9% indicating their willingness to ask ChatGPT for initial problem-solving strategies. This illustrates an open attitude towards integrating AI tools into the learning process, potentially aiding students in formulating structured approaches to complex challenges. Notably, D/Q17 showcases divided opinions, as 32.4% of students express a higher likelihood of seeking help from ChatGPT depending on the complexity of the code. This indicates that while AI can provide immediate solutions, there is recognition that its effectiveness is contingent upon the intricacy of the problem.

Statements D/Q21 and D/Q22 underscore divergent viewpoints on the potential implications of ChatGPT’s integration. While 29.7% of students believe their peers might consistently rely on AI for problem-solving (D/Q21), a similar percentage (29.7%) express concerns that overuse of AI could be detrimental to their own learning (D/Q22). This presents an interesting tension between the perceived benefits and potential drawbacks of AI assistance. Likewise, D/Q23 showcases mixed sentiments, with 40.5% of students agreeing that ChatGPT could foster a culture of dependency, where immediate answers might replace the pursuit of understanding. Conversely, 21.6% disagree with this notion, suggesting that they recognize the tool’s potential to provide valuable insights and explanations. Regarding assessment, statements D/Q25 and D/Q26 underline students’ perspectives on maintaining the integrity of evaluations in the presence of AI tools. While 32.4% of students agree that in-person teachers should challenge them with code defenses due to the possibility of AI use (D/Q25), an overlapping 27.0% agree that educators should demand code defenses to counter potential code copying facilitated by AI (D/Q26).

Patterns and trends in responses from D/15 to D/26 reveal nuanced perspectives. Students with prior programming experience showed a predominant inclination (51.4%) toward seeking ChatGPT’s help when given an exercise by the teacher. However, when complexity increased, 32.4% disagreed, indicating a cautious approach. Students expressed a desire for balanced usage, with 40.5% acknowledging ChatGPT’s potential to prevent independent problem-solving. Notably, familiarity did not uniformly impact views on exclusive reliance, as 29.7% with prior knowledge still considered it harmful. These insights suggest that while experience influences engagement, varied attitudes prevail, emphasizing the importance of tailoring educational strategies to individual learning styles.

The students’ perceptions of ChatGPT in solving programming challenges revealed a mixed response. While some recognized its potential benefits for collaborative learning, concerns about excessive reliance and potential harm were evident. Additionally, there was apprehension about ChatGPT fostering a culture of dependency and hindering individual problem-solving initiative. Despite these concerns, a notable percentage believed in the continued importance of traditional evaluation methods, such as code defenses by in-person teachers. These nuanced views highlight the need for a balanced approach to integrate ChatGPT in programming education, ensuring it enhances learning without compromising essential aspects of independent problem-solving. These varied responses collectively underscore the complex interplay between AI integration and traditional learning methods. While AI can offer immediate solutions and insights, it is essential for educators to guide students toward a balanced approach that leverages AI’s strengths while cultivating essential skills for independent thinking, problem-solving, and comprehensive understanding.

4.1. Strategies to Use ChatGPT in Classrooms

Based on our study, educators can effectively integrate ChatGPT into programming courses by strategically incorporating it into specific modules that enhance collaborative problem-solving without overshadowing traditional methods. Clear guidelines should be provided, emphasizing ChatGPT as a supplementary tool rather than a substitute for independent thinking. It is essential to preserve traditional assessment methods, such as code defenses and practical exams, to evaluate students’ understanding comprehensively and prevent ChatGPT from replacing critical evaluation. Emphasis should be placed on skill development, encouraging students to develop problem-solving skills and critical thinking alongside ChatGPT usage, promoting a deeper understanding of programming principles. The creation of interactive learning environments, where students engage with ChatGPT collaboratively and validate solutions collectively, fosters a more enriching educational experience. Establishing a system for continuous feedback on ChatGPT usage and regularly assessing its impact ensure the ongoing improvement and adjustment of instructional strategies. Finally, recognizing the dynamic nature of technology and staying adaptable to new tools and AI advancements is important for successful integration into programming curricula.

4.2. Research Limitation

One notable limitation of our study stems from its relatively modest sample size, which may be further constrained if the number of students exceeds 40. While our research focuses on enhancing specificity by exclusively targeting first-year students in computer science and software courses, this intentional focus may compromise the generalizability of our findings to a broader student population.

It is important to acknowledge that our study is specifically tailored for students in their first year of computer science courses, excluding individuals from other academic disciplines. Consequently, the applicability of our results is confined to this particular academic context, limiting the broader implications for students in different fields of study.

Moreover, the prerequisite for students to undergo a preintroduction to ChatGPT usage before participating in the research introduces a potential limitation. This precondition may lead to a selection bias, as participants are likely to possess a certain level of familiarity with the technology, impacting the external validity of our study.

Additionally, this research is not optimal for students who already possess advanced knowledge in chatbots. This study assumes a baseline level of understanding, potentially excluding those with extensive prior experience in the field. As such, the outcomes may not accurately represent the perspectives and challenges faced by students with advanced knowledge in chatbot technologies. These limitations collectively emphasize the need for caution in generalizing our findings and underscore potential areas for future research refinement.

5. Conclusions

In conclusion, ChatGPT, a potent AI tool rooted in OpenAI’s GPT-3.5 architecture, has emerged as a remarkable innovation with vast implications for sustainability. Its adaptability and human-like text generation capabilities have propelled it to prominence across various domains, including chatbots, customer service, and personal assistants. Within the educational realm, ChatGPT holds the promise of providing invaluable support to both students and educators in the teaching and learning process. Its applications span from generating concise summaries and addressing subject-specific queries to assisting with programming tasks. Nonetheless, as this transformative tool finds its way into education, challenges come to the forefront. One of the pivotal concerns is the risk of students becoming excessively reliant on code generated by ChatGPT, potentially hindering their comprehensive understanding of the underlying concepts. Therefore, while we explore the extensive potential of ChatGPT, it is imperative to address these issues and continually refine its utilization to ensure a positive and sustainable impact in the field of education. This entails fostering ethical practices, nurturing a deeper comprehension of the subjects at hand, and ensuring that technology aligns with the principles of education that prioritize sustainability and lasting knowledge.

According to the two research questions presented in the introduction section, we conclude (RQ1) that the integration of ChatGPT in coding and programming courses significantly impacts students’ perceptions of educational support, sustainability, and individual learning experiences. The survey results indicate that over 90% of students view ChatGPT as a valuable tool for addressing both theoretical and practical concerns. This suggests a positive impact on educational support, providing students with an additional resource for understanding coding concepts. Moreover, the findings highlight the sustainability of traditional educational roles, as 70% of students recognize the continued importance of teacher–student interaction in the learning process. Overall, the integration of ChatGPT appears to enhance educational support and sustainability while influencing positive learning experiences among students. (RQ2) The increased reliance on ChatGPT in programming education poses challenges related to ethical considerations, code plagiarism, and potential drawbacks in fostering independent problem-solving skills among students. While around 45% of students were receptive to AI assistance, concerns arose, as 29.7% expressed worries about excessive AI use potentially hindering their learning, especially in assessment activities. This suggests ethical considerations regarding the reliance on AI-generated solutions and the potential impact on students’ independent problem-solving skills. Additionally, the findings indicate the need for a balanced integration of AI, emphasizing the role of educators in guiding students to use ChatGPT responsibly and maximize its benefits without compromising the development of crucial skills like independent problem-solving.

As a direction for future work, we intend to further exploit how ChatGPT can be integrated into students’ teaching–learning process. For example, it was found that there was a percentage of just over 20% of students who did not agree with the answers presented by ChatGPT, so further study is warranted to understand how to better structure the questions to obtain greater assertiveness in the answers. Furthermore, practically half of the students were unable to effectively use ChatGPT’s resources, indicating the need to better understand these difficulties so that the AI tool can help them. Furthermore, we identified the need to explore comparative analyses with results from other relevant studies with a similar approach to this work, with the aim of improving the contextual understanding of the research results.