Pre-Service Teachers’ Assessment of ChatGPT’s Utility in Higher Education: SWOT and Content Analysis

Abstract

ChatGPT (GPT-3.5), an intelligent Web-based tool capable of conducting text-based conversations akin to human interaction across various subjects, has recently gained significant popularity. This surge in interest has led researchers to examine its impact on numerous fields, including education. The aim of this paper is to investigate the perceptions of undergraduate students regarding ChatGPT’s utility in academic environments, focusing on its strengths, weaknesses, opportunities, and threats. It responds to emerging challenges in educational technology, such as the integration of artificial intelligence in teaching and learning processes. The study involved 257 students from two university departments in Greece—namely primary and early childhood education pre-service teachers. Data were collected using a structured questionnaire. Various methods were employed for data analysis, including descriptive statistics, inferential analysis, K-means clustering, and decision trees. Additional insights were obtained from a subset of students who undertook a project in an elective course, detailing the types of inquiries made to ChatGPT and their reasons for recommending (or not recommending) it to their peers. The findings offer valuable insights for tutors, researchers, educational policymakers, and ChatGPT developers. To the best of the authors’ knowledge, these issues have not been dealt with by other researchers.

1. Introduction

An interesting development in recent decades is the use of technology in higher education. Learning and teaching have been affected by the incorporation of technology, and they have changed compared to previous time periods. A notable technological advancement that has played an important role in higher education concerns the widespread use of Web-based tools [1]. A further technological advancement in higher education is the use of artificial intelligence (AI) tools. This advancement is in progress.

Web-based tools support a variety of needs in the specific educational sector. Blended learning (i.e., a combination of face-to-face and Internet-based learning), distance, and lifelong learning models have been supported in higher education institutions providing opportunities to diverse learners and tutors. Web-based tools may be used for learning and teaching activities anywhere, anyplace, and with any device connected to the Internet. It is also notable that the functionality of a wide range of Web-based tools is accessible at no cost, facilitating their use.

AI tools have been integrated into education for decades. Various survey papers in the field of AI in education (AIED) have been published attempting to highlight the main trends. One may note different viewpoints in the main trends discussed in relevant surveys. For instance, in [2], the main trends highlighted are based on the implemented functionality (i.e., teaching students, providing support to students, and providing support to teachers). Intelligent Tutoring Systems and Dialogue-based Tutoring Systems are representatives of the first trend. Representatives of the second trend are Exploratory Learning Environments, learning companions, and collaborative learning approaches. Teaching assistants, automated evaluation and monitoring of students, and the use of AI as a research tool are examples of approaches supporting teachers. In [3], work in AIED spanning five decades (1970–2020) is surveyed, highlighting the main research themes. These concern adaptive learning and personalization, deep learning, and machine learning in online educational processes and data mining, human-AI interaction, and the educational use of AI-generated data and AI in higher education. A general trend that one may note in AIED is the incorporation of AI in Web-based tools, which offers advantages because the strong points of AI and the Web are combined.

Higher education institutions are at the forefront of AIED. This is due to the existence of faculty members, researchers, and students doing research in AIED. Trends in AIED in higher education can be discerned based on general AIED surveys such as [2,3]. However, due to the large amount of research work concerning higher education, surveys specifically involving AIED in higher education have been published (e.g., [4,5,6]). One may note that research within institutions has yielded various AI-based tools implemented and supported by resources of the institutions (e.g., [7,8]). Very recently, widespread Web-based tools that incorporate AI methods have been used by higher education students and staff members. The consequences of these recent advances are significant due to the large number of academic community members that use them around the world.

The continuous advancements in technology create challenges. New tools are continuously made available, but the functionality of existing tools also evolves. The evolution in the availability of tools and their functionality means that effort is needed in order to analyze the corresponding consequences for learners and tutors. One may determine the strengths and weaknesses of the tools, as well as opportunities and threats concerning learners, tutors, and academic institutions. A useful task is also to assess the tools in terms of the requirements that need to be satisfied in learning settings. Feedback from learners may play an important role in determining the aforementioned strengths, weaknesses, opportunities, and threats. Therefore, it would be practical to learn the perceptions of higher education students about technological tools used in learning settings. It could be specifically useful to learn the perceptions of higher education students who are pre-service teachers because they will become the ones who will serve in education in the future. One may note that the perceptions and familiarity of teachers with technological tools are factors affecting the use of these tools in their teaching [9].

An intelligent Web-based tool that has recently become very popular around the world is ChatGPT. The main reason for its popularity is its ability to provide human-like text-based responses to human queries in real time and in any subject. In most cases, the responses are quite accurate and time-efficient. Therefore, this tool can effectively engage in real-time text-based conversations with humans. In 1950, Alan Turing published his seminal paper in which he introduced the imitation game (i.e., the Turing test) as a way of assessing the ability of a machine to think [10]. A machine that could pass the test would be regarded as able to think. Until recently, this was not possible, but a tool such as ChatGPT constitutes a development to derive tools that could pass the Turing test.

ChatGPT affects higher education in various ways. Taking into consideration the three main trends discerned in [2], ChatGPT certainly involves two of them (i.e., support of students and teachers). To a certain degree, ChatGPT may be used to teach students. Taking into consideration the main research themes discerned in [3], ChatGPT certainly concerns two themes, i.e., human-AI interaction and the educational use of AI-generated data.

In [11,12], the main uses of ChatGPT in higher education are discerned. One may discern four main types of ChatGPT uses in a university department according to the users they concern: (i) uses addressed to faculty members, (ii) uses addressed to students, (iii) uses addressed to other staff members (e.g., administrative staff members), and (iv) uses that concern all members of the departments’ communities [12]. Indicative general uses of ChatGPT concern assistance in preparing announcements, notes, letters and guidelines, and translation of content. Indicative uses addressed to students are personalized assistance while studying and doing assignments (e.g., the retrieval of useful information, explanations of ambiguous concepts, and answers to questions), guidelines about how to generally structure academic work and presentations, and the preparation of text involving requests to staff members. Indicative uses addressed to faculty members concern assistance in retrieving resources about their courses and assistance in conducting research (e.g., the retrieval and summarization of relevant work and the indication of promising research directions). Further details are available in [11,12].

However, questions are raised about how ChatGPT affects higher education, taking into consideration the potential negative aspects [11]. The introduction of AI tools like ChatGPT in educational settings presents both opportunities and challenges [11]. These challenges include addressing the educational needs with AI, ensuring content validity, and managing ethical and privacy concerns.

In this context, it is insightful to examine the perceptions of higher education students, especially pre-service teachers, on ChatGPT’s impact on higher education. This work utilizes a SWOT analysis framework [13,14], a method commonly applied in market research, strategy development within organizations, project planning, and process assessment. Additionally, SWOT analysis has found applications in software engineering, demonstrating its versatility and effectiveness across various domains. In SWOT analysis, strengths and weaknesses are the internal factors, whereas opportunities and threats are external factors.

This paper is structured as follows. Section 2 presents related work. Section 3 presents the aim and research questions of the study. Section 4 and Section 5 present the applied methods and the results, respectively. Section 6 discusses aspects of robotics in educational contexts and ChatGPT. Section 7 provides a discussion of the results, and Section 8 outlines the limitations of the research.

2. Related Work

This section establishes the context, identifies gaps or limitations in current knowledge, and highlights the significance of the current study within the broader scholarly conversation in two directions, a broader direction and a more focused one. The scholarly conversation is formed on a broader direction that includes the discussion of the significance of ChatGPT in education [15,16], the impact of ChatGPT in education [17], and the application of ChatGPT in higher education [18,19,20].

A second, more focused direction that has been formed concerns the use of SWOT analysis in order to understand the use of ChatGPT in teaching and learning. A look at the related work will help to understand how the current research contributes to the existing literature in the field. A search was conducted for the application of SWOT analysis in assessing ChatGPT within educational contexts. The result of this search showed that no empirical study utilizing SWOT analysis with the use of questionnaires was found. The studies that were identified primarily employed SWOT analysis for theoretical evaluations or assessments.

In a previous study [11], a SWOT analysis was carried out based on student perceptions to evaluate the tool’s alignment with educational principles and to identify areas requiring further attention or enhancement. Although this initial study was limited in scope, it represents the first effort to employ SWOT analysis to illuminate student perspectives on incorporating ChatGPT into teaching and learning contexts.

A recent systematic review [21] examines the strengths, weaknesses, opportunities, and threats of using ChatGPT in teaching and learning contexts. This review collates findings from various studies to facilitate discussions about the strengths and weaknesses of using ChatGPT in teaching and learning, as well as the opportunities and threats associated with its use in teaching and learning. Using thematic analysis to investigate relevant topics within related articles, they apply the 3P (Presage, Process, and Product) model of teaching and learning, as originally proposed in [22]. Their SWOT analysis has revealed thirteen strengths, ten weaknesses, five opportunities, and four threats. This analysis can further explain the four paradoxes of ChatGPT: “ChatGPT is ‘friend’ yet a ‘foe’, ChatGPT is ‘capable’ yet ‘dependent’, ChatGPT is ‘accessible’ yet ‘restrictive’, ChatGPT is ‘popular’ even when ‘banned’” [23].

In [24], a qualitative approach with a SWOT design was employed. Guided by the SWOT framework and based on available literature, their work provides an overview of ChatGPT’s strengths, which can help identify its various opportunities for education. The review also discusses ChatGPT’s weaknesses, which may highlight potential threats. The paper highlights ChatGPT’s self-improvement or self-learning capabilities and its ability to provide personalized and real-time feedback as significant strengths. Opportunities identified include the facilitation of complex learning processes and the reduction in teaching workloads, among others. However, they also point out weaknesses, such as ChatGPT’s inability to verify the credibility of information and its potential to perpetuate biases and discrimination. The paper notes significant threats, including the “democratization of plagiarism” within education and research. They emphasize that ChatGPT’s tendency to amalgamate text from multiple sources can, if used uncritically, lead to plagiarism in academic and student projects [24] (p. 9).

Similar to [24], the theoretical study in [25] also employs a qualitative methodology within a SWOT framework. Their findings underscore ChatGPT’s strengths, notably its advanced natural language processing capabilities, ability for self-improvement, and capacity to deliver personalized and real-time feedback. Nevertheless, they identify weaknesses, including the system’s shallow comprehension and the difficulties in evaluating the quality of its responses. Furthermore, they highlight threats such as risks to academic integrity and the reinforcement of discrimination. Addressing these issues is critical for ChatGPT’s successful adoption in educational contexts.

A SWOT analysis of ChatGPT strategic management and the utilization of technology in education was conducted in [26]. The author claims that SWOT analysis provides a structured framework to assess ChatGPT from different perspectives, including its technical capabilities, how accepted its use is in education, and how prepared the members of the educational setting are to use it. It is mentioned that data items were collected by surveying relevant literature and interviewing AI experts and industry executives. The SWOT analysis findings highlighted various strengths, weaknesses and opportunities for ChatGPT in the education sector. Personalization emerged as a key strength, facilitating individualized learning through customized materials, personalized lesson plans, and targeted feedback for both teachers and students. Accessibility also stands out as a strength, enabling self-paced learning and enhancing access to educational resources, particularly for students with disabilities. Additionally, ChatGPT offers cost-effectiveness by providing valuable insights and predictive analytics for informed decision-making and targeted interventions in educational institutions. Integration with learning technologies further enhances the experience by offering personalized support, resource recommendations, and automated grading. The main weaknesses mentioned concern the lack of human interaction, ethical issues (e.g., plagiarism, cheating), and the dependence on technology. The main threats highlighted are concerns about data privacy (e.g., users’ personal data and conversations) and response quality.

A SWOT analysis of ChatGPT was also conducted in [27]. The notable strengths mentioned are the provision of expert solutions and guidance in complex tasks and the ability to assess students’ work using rubrics and checklists it generates to provide relevant feedback. These are possible by utilizing expert knowledge it incorporates into various domains. A notable weakness mentioned is the inability to fully comprehend the meaning of the generated text. Opportunities pointed out are the popularity for distance learning and personalized learning support. A threat mentioned is the prohibition of generative AI tools in the education sector. Based on the SWOT analysis, the authors propose how to integrate ChatGPT into teaching and learning practice.

The corresponding work in [28] aims to conduct an examination of the aspects of ChatGPT in relation to the potential utilization of ChatGPT within educational contexts. Specifically, its goals include promoting the integration of ChatGPT in educational settings and providing educators with various methodologies and approaches to ensure the thoughtful and effective integration of ChatGPT into pedagogical or research activities. To this end, a limited-scale SWOT analysis has been performed in order to highlight possible ways that ChatGPT could enhance pedagogical and learning efforts.

Although previous studies have shed light on the integration of ChatGPT into educational contexts, their analyses are primarily based on literature reviews without incorporating firsthand user experiences from the education sector for the derivation of SWOT content. This gap highlights an underexplored research avenue concerning student perspectives. Apart from the study in [11], there has been little investigation into how higher education students view the strengths, weaknesses, opportunities, and threats (SWOT) related to ChatGPT. This new focus constitutes a substantial original contribution of our research. As pointed out in [24], while SWOT analysis provides a comprehensive understanding of ChatGPT’s role in education, it falls short in ranking the significance of issues within each category. Our study seeks to build upon and broaden the scope of existing literature by integrating SWOT analysis with a quantitative approach and insights from user experiences, specifically those of student users. An additional novelty of our research is the use of diverse data analysis methods, including descriptive statistics, inferential analysis, clustering, and decision trees. This methodological blend, incorporating both statistical and AI techniques, aims to offer valuable insights into ChatGPT’s educational utility.

3. Study Aim and Research Questions

The aim of this study is to investigate the perceptions of pre-service teachers regarding the utility of ChatGPT (GPT-3.5) in academic environments, focusing on its strengths, weaknesses, opportunities, and threats (SWOT). This exploration is guided by the following research questions:

• How do undergraduate students evaluate the application of ChatGPT in academic settings, specifically assessing its strengths in assisting with tasks like text correction, comprehensive task responses, and paraphrasing; its weaknesses concerning the validity, originality, and potential biases of the information provided; the opportunities it presents for enhancing academic experiences through research experimentation, collaborative projects, and creative expression; and the perceived threats it may have to traditional teaching methods, critical thinking, and human interaction?
• How does familiarity with ChatGPT influence students’ perceptions of its strengths, weaknesses, opportunities, and threats?
• Are there identifiable groups among undergraduate students characterized by their familiarity and interaction levels with ChatGPT that exhibit distinct perspectives in a SWOT analysis of ChatGPT’s role in academic environments?
• How do undergraduate students’ perceptions of ChatGPT’s strengths, weaknesses, opportunities, and threats in academic settings vary according to different factors, and what are the key determinants influencing their overall assessment of its utility in educational contexts?
• What are the primary topics and queries directed to ChatGPT by undergraduate students (pre-service teachers)?
• What are the main reasons why undergraduate students would or would not recommend ChatGPT to their peers?

4. Methods

4.1. Participants

The study involved the participation of 257 undergraduate students from the School of Education at Democritus University of Thrace, Greece, distributed across the Department of Primary Education and the Department of Education Sciences in Early Childhood. Each department requires the completion of a minimum of four years for graduation. The selection process was non-random and voluntary. An invitation to participate in the study was posted as an announcement via the eClass platform, which is used by students and faculty members for course management and communication. Interested students voluntarily signed up through a link provided in the invitation posted on the platform. The selection was intended to mirror the demographic makeup of the student populations in these departments, focusing on ensuring a representative mix of gender and year of study. No specific eligibility criteria or exclusion factors were applied beyond being a currently enrolled student in the relevant departments. The majority of respondents were female, comprising 91.1% of the sample, with males representing only 6%. Three percent chose not to disclose their gender. This demographic distribution was anticipated, as the majority of undergraduate students in these departments are typically female. Regarding the year of studies, the majority of the sample comprised third-year students, making up approximately 34% of respondents. These are followed by second-year students at 32%, fourth-year students at 29%, and students extending beyond the typical study period, accounting for 5%. Regarding familiarity with ChatGPT, a significant portion of the respondents, 47%, have never used ChatGPT before, closely followed by 46% who have used it but not extensively, and a minority of 7% who use it frequently.

4.2. Instrumentation

A structured questionnaire was designed to capture a comprehensive view of students’ perceptions regarding ChatGPT’s application in academic settings. It was structured into five main sections: strengths (5 items), weaknesses (5 items), opportunities (5 items), threats (5 items), and general sentiments (2 items). The items were formulated to assess diverse aspects of ChatGPT’s use, such as its utility in academic tasks, concerns about the validity and originality of information provided, potential enhancements to educational experiences, and risks like ethical and privacy issues. Each item was presented as a statement, with participants responding on a Likert scale ranging from 1 (Strongly Disagree) to 5 (Strongly Agree). The questionnaire content was based on the research conducted in [11]. The structure of the questionnaire was based on SWOT analysis. Strengths and weaknesses were the internal factors regarding ChatGPT, whereas opportunities and threats were the external factors concerning the academic setting.

The reliability of the questionnaire was assessed using Cronbach’s alpha, a measure of internal consistency. The values obtained were 0.81 for strengths, 0.77 for weaknesses, 0.83 for opportunities, and 0.75 for threats. These values indicate that the questionnaire sections range from acceptable to good in terms of internal consistency, suggesting that the items within each section reliably measure a single underlying construct. The construct validity of the questionnaire was evaluated through Exploratory Factor Analysis (EFA) using oblique promax rotation. This analysis was chosen due to the expected correlations among factors reflecting different aspects of perceptions toward ChatGPT. The EFA identified four distinct factors corresponding to the main sections of the questionnaire, with all items showing factor loadings greater than 0.3. This indicates that each item adequately contributes to its respective factor, supporting the instrument’s construct validity. All statistical analyses for reliability and validity testing were performed using Jamovi software version 2.4.12.

Prior to the main study, the questionnaire was pre-tested with a small group of students from the same academic context but not included in the main sample. Feedback from this pre-test was used to refine the wording of items to ensure clarity and to adjust the scale as necessary to better capture the range of responses. Adjustments made based on pre-testing results were aimed at enhancing the questionnaire’s face validity and ensuring that the questions were interpreted as intended.

4.3. Procedure

The main questionnaire was administered to the 257 participants via Google Forms, a digital platform chosen for its accessibility and ease of use, ensuring a broad reach among the target population.

Additionally, as part of an elective course taught in the Department of Education Sciences in Early Childhood, a project involving ChatGPT was assigned to 33 students. The project was performed individually or in groups of two to three students. More specifically, students were asked to submit ten queries to ChatGPT that they deemed the most interesting. They were asked to record the queries and the corresponding replies of ChatGPT. The subjects of five of these queries would have to concern their academic studies. The subjects of the other five queries would have to be of general interest; that is, their subjects would have to be beyond their academic studies. Students were also asked to prepare a brief report explaining the following: (i) the reasons why they would recommend the use of ChatGPT to other students and (ii) the reasons for not recommending the use of ChatGPT to other students. It should be mentioned that the elective course consists of lab sessions involving three main aspects: (i) 3D digital storytelling, (ii) robotics, and (iii) AI concepts. The 3D digital storytelling section concerns the implementation of educational digital stories using a cost-free 3D visual programming tool. The course section about robotics concerns robots used in ECE and acquaintance with the various types of robots used in real-world applications. The section about AI concepts concerns AI in ECE and generative AI.

The study was conducted from February to March 2024, a time frame selected to accommodate the academic schedules of the students while maximizing response rates. The recruitment of participants was facilitated through various channels, including email notifications, announcements on the university’s learning management system, and posts on departmental bulletin boards. Participation in the study was voluntary, with an emphasis on confidentiality and the anonymous processing of responses to encourage honest and uninhibited feedback. Approval for this study was granted by the Ethics and Deontology Committee in Research of the Department of Education Sciences in Early Childhood, and its endorsement was further confirmed by the General Assembly of the Department.

4.4. Data Analysis

To address the research questions effectively, a combination of statistical and machine learning methods was chosen based on the nature of the data and the specific objectives of each research question. Descriptive statistics, including means and standard deviations, were employed to summarize the questionnaire responses related to perceptions of ChatGPT’s strengths, weaknesses, opportunities, and threats (research question 1), setting a foundation for more complex analyses. Inferential analysis (one-way ANOVA) was employed to explore differences in perceptions based on students’ familiarity with ChatGPT and other demographic variables (research question 2). This method is appropriate for comparing means across more than two groups, making it ideal for assessing the impact of categorical predictors (demographic characteristics) on continuous outcome variables (students’ perceptions). K-means clustering was selected to uncover groups within the data, effectively segmenting a large volume of responses into distinct clusters based on shared characteristics. This method was chosen to identify common themes and categories in perceptions of ChatGPT, enhancing our understanding of how these perceptions are structured and interrelated. K-means clustering was performed for 2 to 8 clusters with 100 repetitions for each cluster number to ensure the stability and reliability of the clustering outcomes. Decision tree analysis was employed to delve deeper into the factors that influence students’ willingness to integrate ChatGPT into their educational practices by examining the relationships between their perceptions and demographic or academic characteristics (research question 4). Students’ overall assessment of ChatGPT was treated as the dependent variable, and responses to the twenty SWOT items acted as predictors. The analysis was conducted using the rpart() function in the rpart package v4.1.23 [29]. Initially, a fully grown decision tree was generated (setting the complexity parameter to 0). To fine-tune the model, repeated cross-validation was applied to identify the complexity parameter that minimizes cross-validation error. This method involves partitioning the data into k subsets and running the analysis multiple times to ensure reliability. The model’s accuracy was evaluated through a 10-fold cross-validation, repeated 10 times. Subsequently, the initial tree was pruned using a complexity parameter of 0.002, leading to the construction of the final decision tree. This last step was executed using the rpart.plot function from the rpart.plot package [30], providing a visual representation of the analysis results. Content analysis was employed to explore the primary topics and queries undergraduates direct towards ChatGPT (research question 6), and their main reasons for recommending or not recommending it to peers (research question 7). This method was chosen to distill responses into meaningful themes and patterns, revealing students’ perceptions of ChatGPT’s utility and limitations in academic contexts.

5. Results

A detailed analysis of perceptions and concerns regarding the use of ChatGPT among the study participants is shown in

Table 1. Descriptive statistics on student perceptions of ChatGPT in academic settings.

Strengths	Mean	SD
Q1. Students’ assessment of the tool’s functionality to correct and improve texts written by themselves or others	2.70	1.13
Q2. Students’ assessment of the tool’s functionality to give a complete answer to a task	2.54	1.21
Q3. Students’ assessment of the tool’s functionality to provide paraphrasing of a text typed by themselves or others	2.80	1.19
Q4. Students’ assessment of the tool’s ability to adapt itself to their needs (Do you consider that the tool is able to adapt itself to your needs?)	3.33	1.09
Q5. Students’ assessment of the tool’s ability to improve their overall learning experience (Do you consider that the tool is able to improve the student’s overall learning experience?)	3.07	1.12
Weaknesses
Q6. Students’ assessment about their feeling of uncertainty concerning the validity of the information it provides them (Do you feel uncertain about the validity of the information it provides?)	2.85	1.12
Q7. Students’ assessment about their feeling of uncertainty concerning the originality of the texts that the tool produces (Do you feel uncertain about the originality of the texts that the tool produces?)	2.89	1.14
Q8. Students’ concern about the possible biases and inaccuracies that may arise since the resulting texts are influenced by algorithms (Are you concerned about the possible biases and inaccuracies that may arise since the resulting texts are influenced by algorithms?)	3.06	1.06
Q9. Students’ assessment if they are affected given that no sources or references are provided by the tool (Are you affected by the fact that no sources or references are provided by the tool?)	3.49	1.18
Q10. Do you think that ChatGPT can affect the privacy and security of student data in an academic environment?	2.90	1.19
Opportunities
Q11. Students’ assessment if the tool can provide opportunities for experimentation in academic research (Does the tool provide opportunities for experimentation in academic research?)	3.27	1.07
Q12. Students’ assessment if the tool can be used to improve collaborative projects and teamwork among students (Can the tool be used to improve collaborative projects and teamwork among students?)	3.14	1.15
Q13. Students’ assessment if the tool promotes interdisciplinary and innovative research in the academic community (Does the tool promote interdisciplinary and innovative research in the academic community?)	2.88	1.10
Q14. Students’ assessment if the tool enhances accessibility to educational resources for students with different learning needs (Does the tool enhance the accessibility to educational resources for students with different learning needs?)	3.18	1.10
Q15. Students’ assessment if the tool enriches language, expression and imagination (Does the tool enrich language, expression and imagination?)	3.06	1.22
Threats
Q16. Students’ concern about possible abuse of ChatGPT, such as creating fake academic content or concerns about plagiarism (Are you concerned about possible abuse of ChatGPT, such as creating fake academic content or concerns about plagiarism)	3.53	1.15
Q17. Students’ ethical concerns about privacy and data security in using ChatGPT in academic settings (Do you have ethical concerns about privacy and data security when using ChatGPT in academic settings?)	3.05	1.18
Q18. Students’ assessment if the integration of ChatGPT can affect traditional teaching methods (Do you believe that the integration of ChatGPT can affect traditional teaching methods?)	3.39	1.21
Q19. Students’ assessment if critical thinking can gradually be weakened by using ChatGPT (Do you believe that critical thinking can gradually be weakened by using ChatGPT?)	3.66	1.18
Q20. Students’ concern about a possible weakening of the human dimension of communication/contact (Are you concerned about a possible weakening of the human dimension of communication/contact?)	3.33	1.20
General Questions
F1. Are you comfortable with the idea of ChatGPT being part of your academic experience?	3.07	1.05
F2. Would you like to receive additional training to better understand and use the ChatGPT tools in your studies?	3.49	1.07

, focusing on its strengths, weaknesses, opportunities, and threats, as well as general sentiments about incorporating ChatGPT into the academic experience. The strengths section reveals a moderate appreciation for ChatGPT’s capabilities in correcting and improving texts, giving comprehensive answers to assignments and rephrasing texts, with mean scores ranging from 2.54 to 3.33. Notably, students recognize its potential to adapt to their needs as learners and to enhance the overall learning experience, indicating a positive perception of its utility in educational settings.

Conversely, the weaknesses highlight uncertainties regarding the validity of information provided by ChatGPT, originality concerns, potential biases, and inaccuracies due to algorithmic determinations, lack of sources or references, and data privacy and security in academic environments. These concerns are reflected in mean scores between 2.85 and 3.49, suggesting that while there is acknowledgment of ChatGPT’s helpful aspects, there remains a significant level of apprehension about its reliability and integrity.

Opportunities identified by the study suggest a positive outlook on the potential of ChatGPT in academic research experimentation, improving collaborative projects and teamwork, promoting interdisciplinary and innovative research, enhancing accessibility to educational resources for diverse learning needs, and enriching language, expression, and imagination. Mean scores in this category range from 2.88 to 3.27, illustrating optimism about the beneficial roles ChatGPT can play in educational advancement.

Threats, however, underscore concerns about the misuse of ChatGPT, including the generation of fake academic content, plagiarism, the impact on traditional teaching methods, the weakening of critical thinking, and the reduction in human interaction. These issues are marked with mean scores from 3.05 to 3.66, highlighting a critical awareness of the potential negative impacts of ChatGPT’s integration into academic environments.

General questions about comfort with ChatGPT as part of the academic experience and the desire for additional training to better understand and utilize ChatGPT tools in studies received mean scores of 3.07 and 3.49, respectively. This indicates a general willingness to engage with ChatGPT, coupled with a recognition of the need for more knowledge and skills to effectively leverage this tool in educational contexts.

Table 2. Influence of familiarity with ChatGPT on perceived strengths, weaknesses, opportunities, and threats.

Familiarity with ChatGPT	Strengths		Weaknesses		Opportunities Threats
	Mean	SD	Mean	SD	Mean	SD	Mean	SD
I have never used it	2.76	0.877	2.97	0.839	3.04	0.905	3.36	0.877
I have used it, but not much	2.91	0.834	3.13	0.809	3.12	0.804	3.47	0.814
I use it frequently	3.53	0.688	2.91	0.776	3.41	1.045	3.09	0.697

presents how familiarity with ChatGPT influences perceptions of its strengths, weaknesses, opportunities, and threats. Participants who frequently use ChatGPT rate its strengths highest (mean = 3.53) and perceive fewer weaknesses compared to less frequent users. Interestingly, those with no experience perceive more threats (mean = 3.36) than frequent users (mean = 3.09), suggesting that familiarity may reduce perceived risks. Opportunities are viewed more positively by frequent users (mean = 3.41), indicating that engagement with ChatGPT correlates with recognizing its potential benefits more strongly. Overall, the data suggest that increased use of ChatGPT leads to a more favorable assessment of its capabilities and less concern over its drawbacks.

K-means clustering has delineated three distinct clusters based on students’ utilization and perceptions of ChatGPT. Overall, these clusters reflect a spectrum of students’ interactions with and perceptions toward ChatGPT, ranging from enthusiasm to more cautious or less engaged, each with unique recognitions of its benefits and concerns. Cluster 1 (43%) represents a group of students with high engagement and a very positive perception towards ChatGPT. These students acknowledge the capabilities of ChatGPT for tasks such as improving texts, providing complete answers, and paraphrasing and believe strongly in its adaptability to their needs and its potential to enhance their learning experience. While they have moderate concerns about the validity and originality of information, as well as potential biases and privacy issues, they highly value the opportunities ChatGPT provides for academic experimentation, collaborative work, and accessibility to educational resources. This cluster shows a high level of comfort with ChatGPT being part of their academic experience and expresses a strong desire for additional training to better utilize these tools.

Cluster 2 (30%) includes students who engage with ChatGPT to a moderate extent and have some reservations about its use. This group’s assessment of ChatGPT for editing, task completion, and paraphrasing is notably lower compared to Cluster 1, and their perception of its adaptability and learning enhancement capabilities is moderate. Concerns about the validity and originality of the content generated by ChatGPT, as well as biases and inaccuracies, are more pronounced in this cluster. Despite these reservations, the students acknowledge the potential of ChatGPT for fostering academic experimentation and enhancing collaborative projects. However, their concerns extend significantly to data privacy and the impact on traditional teaching methods, though they still show a very high interest in receiving additional training.

Cluster 3 (27%) is characterized by students with lower engagement and more varied perceptions of ChatGPT. Their assessment of ChatGPT for specific functionalities is generally lower, and they exhibit cautious optimism about its adaptability to student needs and its ability to enrich the learning experience. Concerns in this cluster about the validity and originality of information, as well as algorithmic biases, are present but slightly lower than in Cluster 2. The acknowledgment of ChatGPT’s potential for promoting innovation and enhancing collaborative efforts is mixed, indicating a recognition of opportunities but with more reservations. This cluster’s concerns about the impact on traditional teaching and critical thinking are similar or slightly lower than those in other clusters, with a mixed level of comfort with ChatGPT in their academic lives and a noticeable interest in further training.

Figure 1 displays the decision tree used to assess the factors influencing students’ overall comfort with integrating ChatGPT into their academic experiences. The primary split in the tree is based on Q5, i.e., students’ assessment of ChatGPT’s ability to improve their overall learning experience. Responses to Q5 below the threshold of 4 (agree) suggest a path of less comfort, whereas responses at or above this threshold indicate a more favorable view of ChatGPT’s role in educational enhancement. Further subdivisions in the tree show that the assessment of ChatGPT’s adaptability to students’ needs (Q4) and its potential impact on privacy and data security (Q10) play significant roles in shaping students’ comfort levels. For instance, respondents who perceived ChatGPT as highly adaptable (Q4 ≥ 3) and had fewer concerns about data privacy (Q10 < 4) were more comfortable with its academic integration. Additionally, perceptions of the tool’s functionality in providing complete answers to tasks (Q2) also affected comfort levels, with lower ratings correlating with less comfort. The terminal nodes of the tree, representing the outcomes of these decision paths, are color-coded to denote varying levels of comfort with ChatGPT, from red (least comfortable) to green (most comfortable).

Table 3. Main categories of queries submitted by students regarding their academic studies and the corresponding number of students.

ID	Category of Queries	#Students
C.A.S.1	The future prospects of a person who holds an ECE degree	21
C.A.S.2	General issues involving education sciences (e.g., information about specific terms, information about specific university course outlines)	20
C.A.S.3	The subject of a diploma thesis they could do during their final year of studies	5
C.A.S.4	Preparation for face-to-face written examination of courses (e.g., how to organize study time, most important topics likely to be included in examination questions)	6
C.A.S.5	Preparation of bibliography	5
C.A.S.6	Rights of university students	3
C.A.S.7	Preparation of email message text addressed to staff members	3
C.A.S.8	Learning theories	6
C.A.S.9	Importance of a specific cognitive field in ECE	3
C.A.S.10	Management of an ECE class including students who are non-native speakers or from different cultures	5
C.A.S.11	Support of young students in classroom (maintaining children’s focus, support of children with learning difficulties, ways to calm children in the classroom, supporting isolated children to collaborate with other children)	7
C.A.S.12	Teaching of specific subjects in ECE (e.g., mathematics, environmental issues, music)	18
C.A.S.13	Integration of technology (in general) in ECE or integration of specific technological tools in ECE	12
C.A.S.14	Role of ChatGPT in education	8

presents the main categories of queries submitted by students regarding their academic studies and the number of students whose queries are included in each category. Most students submitted queries regarding the future prospects of a person who holds an early childhood education (ECE) degree and general issues involving education sciences. The initials C.A.S. in the table stand for categories of queries regarding academic studies. The table does not include categories of queries that were provided by only one or two students. One may note that the categories of queries included in this table may be organized into three broader categories. These broader categories are(i) categories of queries involving university students as pre-service teachers (i.e., the first nine categories of queries, C.A.S.1–C.A.S.9), (ii) categories of queries involving aspects that could mainly be considered interesting for in-service teachers or pre-service teachers performing their field practice and practicum in ECE settings (i.e., categories C.A.S.10 to C.A.S.13), and (iii) a discrete category concerning specifically ChatGPT in education (i.e., C.A.S.14).

Table 4. Main categories of queries submitted by students regarding subjects of general interest and the corresponding number of students.

ID	Category of Queries	#Students
C.G.I.1	Health, healthy eating, or weight loss	24
C.G.I.2	Tourist trips	18
C.G.I.3	Gifts or wishes given to other people (e.g., for birthdays)	9
C.G.I.4	Future predictions	9
C.G.I.5	Recommended literary books to read	5
C.G.I.6	Recipes	5
C.G.I.7	Sports (e.g., football)	3

presents the main categories of queries submitted by students regarding subjects of general interest beyond their academic studies and the number of students whose queries are included in each category. We note that most students submitted queries regarding two main categories: (i) health, healthy eating, or weight loss (C.G.I.1) and (ii) tourist trips (C.G.I.2). The initials C.G.I. in the table stand for categories of queries regarding subjects of general interest.

The categories of queries presented in Table 3 and Table 4 constitute information that may be useful in assisting the integration of ChatGPT in higher education settings besides Departments in Education Sciences. The categories of queries in Table 4 involve students in all scientific fields. As far as the categories of queries shown in Table 3 are concerned, one may note that a number of them are quite general (i.e., C.A.S.3, C.A.S.4, C.A.S.5, C.A.S.6, C.A.S.7, and C.A.S.14) involving all scientific fields. Several other categories in Table 3 may be slightly changed to adapt them to any scientific field (i.e., C.A.S.1, C.A.S.2, C.A.S.9, C.A.S.12, and C.A.S.13). Only three categories of queries (i.e., C.A.S.8, C.A.S.10, C.A.S.11) may not be adapted to any field.

Table 5. Main categories of queries that may be submitted by students in any field regarding their academic studies.

ID	Category of Queries
C.A.S.1′	The future prospects of a person who holds a degree in [the specific field of study]
C.A.S.2′	General issues involving a [specific field of study] (e.g., information about specific terms, information about specific university course outlines)
C.A.S.3′	The subject of a diploma thesis they could write during their final year of studies
C.A.S.4′	Preparation for face-to-face written examination of courses (e.g., how to organize study time, most important topics likely to be included in examination questions)
C.A.S.5′	Preparation of bibliography
C.A.S.6′	Rights of university students
C.A.S.7′	Preparation of email message text addressed to staff members
C.A.S.8′	Importance of a cognitive field in [the specific field of study]
C.A.S.9′	Teaching of subjects in [the specific field of study]
C.A.S.10′	Integration of technology (in general) in [the specific field] of study or integration of specific technological tools in [the specific field of study]
C.A.S.11′	Role of ChatGPT in higher education or in [the specific field of study]

presents six of the categories in Table 3 without any change and five others slightly changed to be adapted to any scientific field. Therefore, Table 5 contains eleven categories of queries regarding any scientific field with new IDs. Text in italics corresponds to adaptations of categories in Table 3. The term ‘[the specific field study]’ is a generic term that may be set to any field (e.g., chemical engineering, biology, medicine, computer science, etc.). The categories of queries shown in Table 5 may assist in the design of educational activities in any higher education major.

Students provided several reasons for recommending ChatGPT to their peers, illustrating a broad spectrum of benefits that span from academic support to personal development (

Table 6. Summary of reasons for recommending ChatGPT to peers.

Reason for Recommending ChatGPT to Peers	#Students
Students save time in the acquisition of information, with time-efficient responses on any topic	14
Specific aspects regarding the responses given by ChatGPT	10
Assistance provided concerning the courses	8
Assistance provided concerning projects	12
The ease of using the tool and the free access	3
Availability twenty-four hours per day throughout the week	2
Ability to practice in a foreign language	3
Assistance offered to persons that may have spelling difficulties	3
Ability to obtain well-written text with good structure assisting in the preparation of text	3
Assistance that may be provided with everyday issues	2
Assistance in preparing an email message in an academic context or any other context	3
Assistance provided to students who may have difficulties expressing their opinions in public media	3
Answers to various questions about the scientific field of education and the courses taught in such a department	5
Information about the opportunities given to them as members of an academic environment	5
Personalized suggestions if prior information about the user is given, personalization of responses based on previously asked questions	1
Learning how to prepare specific and comprehensive questions	1

). The primary reason, highlighted by fourteen students is the efficiency ChatGPT offers in completing various tasks. This includes searching for information, solving problems, and writing texts, which significantly saves time and enhances productivity. Ten students pointed out specific strengths in ChatGPT’s responses, such as their comprehensiveness and understandability. ChatGPT was praised for its ability to provide detailed information and advice on a wide range of topics, although it was noted that for specialized queries, the tool might not always deliver suitable responses. Regarding course-related assistance, eight students appreciated ChatGPT’s capability to enrich notes and educational materials and to facilitate the acquisition of new academic content. This aligns with examples shown during a lab session, where ChatGPT’s utility in retrieving immediate answers to course-related questions was demonstrated.

Project support was another significant advantage mentioned by twelve students, who valued ChatGPT for its ideation and organizational assistance and for offering innovative perspectives on project topics. This reflects ChatGPT’s role in fostering creativity and enhancing academic projects. Ease of use and free access were highlighted by three students as key factors contributing to ChatGPT’s appeal. These aspects underscore the importance of user-friendly and accessible tools in education. Additionally, the 24/7 availability of ChatGPT was noted by two students as particularly beneficial, accommodating students’ varied schedules and supporting distance learning. Three students mentioned the tool’s utility in foreign language practice, despite some inaccuracies, emphasizing its potential in language learning. Another three students discussed how ChatGPT helps overcome spelling challenges and assists in generating well-structured texts, thus contributing to improved writing skills.

ChatGPT’s versatility was also recognized in its ability to provide solutions for everyday issues, with two students discussing how it offers practical advice for non-academic queries. Similarly, three students valued ChatGPT’s assistance in drafting emails, particularly in academic settings, which supports effective communication. In the realm of public expression, ChatGPT was seen as beneficial for those who struggle with articulating their opinions in online forums and social media. This aspect, alongside its educational benefits, such as answering queries related to the field of education and offering insights into maximizing academic opportunities, was appreciated by five students. One student highlighted ChatGPT’s capacity for personalization, suggesting that the tool can tailor its responses when provided with specific user information, adding a layer of individualized support. An emerging theme mentioned by one student relates to AI literacy. Interacting with ChatGPT not only aids in acquiring desired information but also in learning to formulate specific and comprehensive queries, a skill that will become increasingly important in the digital age.

Students also mentioned several reasons for not recommending ChatGPT to their peers, which are mainly aligned with the identified weaknesses and threats in the questionnaire (

Table 7. Summary of reasons for not recommending ChatGPT to peers.

Reason for Not Recommending ChatGPT to Peers	#Students
Provision of inaccurate, vague, or misleading information, especially on complex or controversial topics. Inability of ChatGPT to thoroughly comprehend the context of certain topics and the sensitivity to the input provided by users. The need for users to express their queries with clarity. The need to double-check the responses acquired from the tool. ChatGPT provides different responses about the same or similar queries when these are submitted by different persons.	12
The sources of the acquired information are not cited, creating doubts about the quality of the retrieved information. Students’ deliverables may be rejected if they rely on ChatGPT due to the inexistence of citations to sources and possible plagiarism issues. In diploma theses and semester projects, there should be impartiality and validation requiring the citation of a range of valid sources.	8
Reduction, hindrance, absence of critical thinking and problem-solving skills, as well as creativity. ChatGPT does not employ a discovery learning approach but readily lists all information items. Some persons are probably (or could become) too lazy to look for or to reach answers through thinking and discovery. Replacement to a certain degree of books and other learning items by the tool.	12
Interaction with ChatGPT reduces the fluency of writing. Inactivity and passivity of students’ thinking, in terms of writing, preparing an original text, avoiding mistakes, and personal speech development.	3
Users’ identity and personal data may not be totally safe.	2
Slight weakness in the formulation of texts in Greek and some mistakes may be spotted in the produced text.	2
It is important for students to prepare by themselves the email messages addressed to faculty members and gain relevant experience	1

). The most common concern, shared by ten students, revolves around ChatGPT’s tendency to provide inaccurate, vague, or misleading information, especially on complex or controversial topics. This issue stems from ChatGPT’s limited understanding of the context and the complexity of certain topics, which are exacerbated when users do not clearly articulate their queries. A related concern mentioned by two students is the need for clear expression of queries, highlighting that unclear queries might not yield the desired responses. Additionally, there’s a noted need to double-check ChatGPT’s responses for accuracy, particularly if these responses are to be used in academic projects, as ChatGPT might provide different answers for similar queries submitted by various users. Eight students expressed doubts about the quality of information due to the absence of cited sources in ChatGPT’s responses. This lack of citations raises questions about the validity, accuracy, and completeness of the information, which is critical in academic settings where source citation is paramount. This reliance on ChatGPT without proper citation can lead to academic integrity issues, including plagiarism, and is deemed unsuitable for significant academic endeavors like diploma theses or semester projects that require impartiality and a range of validated sources.

Twelve students raised concerns about the impact of ChatGPT on critical thinking and problem-solving skills. They argued that excessive reliance on ChatGPT for information retrieval might lead to an addiction, thereby reducing, hindering, or even completely eliminating critical thinking and creativity as individuals become accustomed to receiving ready information without engaging in the search and analysis process themselves. Three students pointed out that interaction with ChatGPT could diminish writing fluency, as reliance on the tool might bypass the learning process involved in mastering grammar, syntax, and vocabulary. Moreover, there are security concerns, with two students highlighting the potential risks to users’ identity and personal data online. Additionally, two students noted weaknesses in ChatGPT’s ability to formulate texts in Greek, spotting some errors in the produced text. One student specifically mentioned the importance of students learning to compose emails to faculty members independently, emphasizing the development of formal communication skills. Two groups of students sought reasons from ChatGPT itself for not recommending its use. Their responses included concerns over the accuracy of information, the tool’s inability to cater to individual needs, the importance of self-education, and the risks of plagiarism and privacy violations. These reasons highlight a cautious approach to using ChatGPT, suggesting a balance between leveraging technological tools and traditional learning methods for a comprehensive educational experience.

6. Robotics in Educational Contexts and ChatGPT

The curriculum in ECE and primary education incorporates various technological resources. Generally speaking, the integration of technological resources in ECE, primary education, and subsequent levels of education serves three main purposes: (i) acquaintance of students with technology and the use of specific technological resources, (ii) the use of technological resources in all parts of the curriculum as learning tools, and (iii) the comprehension of the role that technology plays in society [31].

Robots are among the technological resources used in ECE and primary education. Robots have certain unique features compared to other technological devices that attract the interest of students. These features involve, among other things, mobility, sensors, sounds, lights and physical interaction with their environment. Programming is also combined with robotics. Furthermore, the various parts of robotic devices may be assembled, enhancing the creativity of students. Additional components and other relevant materials may be manipulated, assembled, or constructed by students with the guidance of teachers [32].

Robots in ECE and primary education are used to introduce students to and raise their interest in various technological aspects, including engineering, electronics, and programming. Robotics may also provide motives to students to pursue relevant studies in higher education and to seek relevant jobs. Cross-curricular activities are implemented with robotics concerning language, mathematics, natural sciences, environmental education, arts, and other fields. The implementation of robotic activities promotes discussions and collaboration among students, creative thinking, and problem-solving skills.

In ECE, usual types of robots are devices programmed to move on the floor and other surfaces (e.g., desks or tables). In this context, a robot is programmed to follow a specific route, that is, to reach a destination, taking into consideration its starting point and its initial direction. A route may include intermediate destinations besides the final destination. The space in which the robot moves may also include obstacles that need to be avoided. The programming instructions involve symbols that define the main directions that a robot may follow and other necessary functions. A sequence of instructions is given to the robot in order to follow the desired route. The sequence of instructions may be considered a type of program the creation of which is based on discussion and collaboration. This program needs to be debugged in case the results of its execution are not the desired ones. According to the type of robot used, the sequence of instructions may be given in various ways such as the following: (a) by pressing a sequence of buttons on the robot’s surface, (b) by using a sequence of specialized cards that are scanned by robot sensors, (c) by using a sequence of tangible objects whose function is recognized by the robot wirelessly, (d) through a block-based programming application running on a device (e.g., tablet) with wireless connection to the physical robot, or (e) in a hybrid way that combines two or more of the previous types of interaction.

In the lower grades of primary education, robots similar to the ones used in ECE (or advanced models) may be employed. In subsequent grades, robots are usually combined with block-based or text-based programming (e.g., Scratch 3.0, Python 3.12.3). Typical examples are the robotic kits for the micro:bit circuit platform. The micro:bit circuit platform may be used to implement activities concerning electronics and programming. Its combination with robotic kits enables the implementation of activities combining robotics, electronics, and programming.

An aspect of interest is the use of social robots, especially AI-based ones, in ECE and primary education. Their form and functionality vary. Social robots are able to adapt to their environment and interact with individual students and groups of students. They may be utilized in typical and special education settings [31].

Robotic concepts are part of the curriculum in the two university departments involved in this study. In the Department of Education Sciences in Early Childhood, robotic concepts are taught as sections in three courses (two obligatory and one elective). The elective course for which results were given in Section 5 (based on a project about ChatGPT) involves a section about robotics. In their projects, certain students attending the course submitted queries to ChatGPT whose subject involved robotics. Some of the submitted queries and the retrieved results will be analyzed. One may note that the responses of ChatGPT are satisfactory.

A group of three students submitted a query about the positive and negative aspects of robots in society. One may consider that this query concerns the overall role of technology in society, which constitutes a main goal in the integration of technology in education. Positive aspects mentioned by ChatGPT involved improved productivity, advanced solutions in health and improved quality of life for persons with mobility difficulties or other disabilities. Negative aspects mentioned involved the loss of jobs for people due to automation and ethical issues about privacy and liability. The overall comment of ChatGPT is that the integration of robots and technology into our lives is a complex issue that requires attention to ethics, social consequences, and understanding how to manage them for the common good.

Another group of students submitted a query about how to use Bee-Bot (i.e., a type of robot addressed to young students) in ECE. This query concerns the use of a technological resource such as a robot in all parts of the curriculum. The corresponding reply of ChatGPT involved the following: (i) introduction to spatial directions and robot functionality, (ii) programming of the robot to follow a route, (iii) the use of the robot to introduce mathematical concepts (e.g., counting and geometrical figures), (iv) collaborative and problem-solving activities in combination with programming, (v) the use of the robot as a character in stories and games. The general assertion of ChatGPT is the use of the robot in a way that promotes creativity and experimentation.

A group of three other students submitted a query about the uses of robotics (in general) in ECE. ChatGPT provided a good summary of uses of robotics. More specifically, the uses mentioned were the following: (i) introduction to technology, (ii) problem solving activities, (iii) early coding concepts, (iv) development of fine motor skills, (v) spatial awareness, (vi) creativity and imagination, (vii) cross-curricular integration, (viii) sequencing, (ix) early exposure to engineering concepts, (x) engagement and motivation, and (xi) preparation for future learning.

Other groups of students submitted queries about the integration of technology (in general) in ECE. ChatGPT mentioned robots among the different types of technological resources that may be used in ECE. Another group of students submitted a query about how AI may be used to assist children in special education settings. ChatGPT provided several corresponding functions. Certain functions that AI may perform in this context according to ChatGPT may be incorporated in robots interacting with children (e.g., facial recognition, emotional recognition, speech recognition, emotional and behavioral support, personalized learning, and adaptive learning content displayed in screens on robots) [31].

Table 8. Indicative topics of queries about robotics in educational contexts that may be submitted to ChatGPT.

Indicative Topics of Queries
Role of robots in society
Main types of robots used in real-world applications
How a specific type of robot is used in real-world applications
How to follow a career in robotics
The subject of a diploma thesis about educational robotics that may be done during undergraduate or postgraduate studies
How to use a specific robot in learning activities (in general)
How to use a specific robot in learning activities involving specific subjects (e.g., mathematics, natural sciences, arts)
How to use a specific robot in cross-curricular activities
How to use a specific robot in learning activities involving specific goals (e.g., improving collaboration among students)
How to use a specific robot in learning activities in combination with other technological resources
How to introduce a specific robot to students for the first time
The robots that may be used in a specific educational level
Instructions about the functionality of a specific robot and additional components
Ideas about constructions that students may do in robotic activities
Ideas about programming activities associated with specific robots
Ideas about projects combining electronics, robotics and programming
Ideas about using robots in special education
Ideas about how to utilize social robots in classroom
Ideas about how to utilize social robots to assist students individually and in groups
Ideas about how to combine AI with robots in educational contexts

summarizes indicative topics of queries concerning robots in educational contexts that may be submitted to ChatGPT. These topics of queries correspond to the three main aforementioned purposes of the integration of technological resources in education (in the specific case of robots): (i) acquaintance of students with robots and the use of specific robots and components, (ii) use of robots in all parts of the curriculum as learning tools, and (iii) comprehension of the role that robots play in society.

7. Discussion

Overall, this study reflects a nuanced perspective among participants, acknowledging both the promising applications and the challenges of integrating ChatGPT into academic settings. This balanced view underscores the importance of addressing the limitations and concerns associated with ChatGPT while exploring its potential to enrich the educational landscape.

The results indicate a moderately positive view of ChatGPT’s benefits, with both strengths and opportunities being recognized. However, notable concerns about potential negatives, particularly threats—which received the highest average score—suggest significant apprehension regarding ChatGPT’s negative impacts.

In summary, the study shows that students’ comfort with ChatGPT is influenced by both its strengths—like adaptability and enhancing learning experiences—and concerns about weaknesses and threats, such as data privacy, originality of work, and the potential introduction of biases or inaccuracies. Opportunities for improving academic research and collaboration also emerged as significant factors.

The reasons for recommending ChatGPT span from its practical benefits in academic task efficiency, project support, and language practice to its broader impact on personal development, daily life assistance, and the cultivation of AI literacy. These insights, combined with specific examples and responses generated by ChatGPT itself, offer a comprehensive understanding of the tool’s multifaceted value to students. While presenting numerous advantages, students also recognize ChatGPT’s significant limitations and risks, particularly concerning information accuracy, critical thinking development, security concerns, and the importance of traditional skills. These insights suggest a nuanced perspective on the integration of AI tools in educational contexts, emphasizing the need for critical engagement and balanced use.

Other papers have utilized SWOT analysis to evaluate ChatGPT, presenting its strengths, weaknesses, opportunities, and threats. Our approach diverges in that it addresses certain issues that are not explored in these studies. Notably, unlike other research, our methodology included a ranking of issues, which was made possible by collecting quantitative data from students. Additionally, our use of a clustering approach allowed for a better understanding of how students perceive ChatGPT, and decision tree analysis helped identify key factors influencing their comfort with the tool. Our research also provided new insights into the types of queries students submit to ChatGPT, revealing their areas of interest—information that is absent in related works but could be invaluable for integrating ChatGPT into higher education settings. Moreover, analyzing the reasons students might recommend or not recommend ChatGPT offered valuable feedback for educational program design and further development of the tool. Another issue discussed is the use of ChatGPT to provide information about robotics in educational contexts.

This study employed ChatGPT (GPT-3.5) rather than the more advanced ChatGPT (GPT-4), which is currently accessible exclusively through a subscription fee. The choice to use the more accessible version of ChatGPT, which is free of charge, aligns with the practical constraints faced by students and educational institutions. Often, students are either unable or unwilling to pay for digital tools that may be required during their academic journey. Similarly, it is not always feasible for institutions or researchers to cover subscription costs for all potential digital tools that could be used by students and staff. This preference for cost-free tools addresses aspects of the digital divide, ensuring broader access. While this choice of tool version is a limitation, it does not appear to significantly influence the results presented in this study.

Future research could benefit from longitudinal studies that track changes in students’ perceptions and usage of AI tools like ChatGPT over time. Such studies would provide deeper insights into the evolving relationship students have with AI technologies and whether prolonged exposure impacts their trust and reliance. Time is required to adopt technological innovations in education, and this is achieved when each person achieves this at their own pace [33]. Additionally, expanding the demographic scope of the studies to include a wider range of educational institutions, disciplines, and cultural contexts would enhance the generalizability of the results and offer a broader understanding of AI’s role in diverse educational settings.

The integration of AI tools into educational practices also presents a promising area for further investigation. Future studies could explore effective strategies for embedding AI into different teaching and learning methodologies, assessing their impact on educational outcomes. Alongside this, there is a clear need for the development of comprehensive AI literacy programs that educate both students and educators about the potential and limitations of AI [34,35]. Such programs would support a more informed and effective use of AI technologies. Moreover, developing robust ethical guidelines and policy frameworks is essential for guiding the use of AI in education. This includes addressing concerns related to privacy, academic integrity, and the balance between AI and traditional educational methods. Detailed impact studies could also be conducted to examine how AI tools affect critical thinking, creativity, and interpersonal skills, which are pivotal in students’ educational development. Comparative studies between ChatGPT and other AI educational tools could provide additional insights into their respective efficiencies and help identify best practices for their implementation in educational settings.

In addition to the aforementioned directions, integrating qualitative methods such as interviews or focus groups would be highly beneficial for future studies. These approaches can offer deeper insights into the nuanced opinions and experiences of students using ChatGPT. By engaging directly with students through these methods, researchers can capture detailed responses and explore complex attitudes that are often not accessible through quantitative measures alone. Such qualitative data could greatly enrich our understanding of how students perceive the effectiveness, usability, and impact of AI tools in their academic lives, providing a more holistic view of the integration of technology in education.

AI tools like ChatGPT can potentially serve to significantly enhance student engagement. By providing interactive and personalized learning experiences, large language models (LLMs) can capture students’ interest and motivation. Drawing on their natural language processing capabilities, LLMs can adapt to individual learning styles and preferences, delivering tailored content to address the unique needs of each student. This customized approach fosters greater engagement with course materials, stimulating active involvement and improving knowledge retention. By maintaining ongoing interaction and providing feedback, LLMs can identify students’ strengths, weaknesses, and learning preferences, empowering educators to customize learning paths and resources accordingly. This personalized approach not only accommodates diverse learning styles but also addresses individual learning requirements, ultimately enhancing academic achievement and student satisfaction.

Furthermore, integrating AI tools such as ChatGPT can address various educational challenges, including accessibility and inclusivity. ChatGPT can serve as a valuable resource for students with disabilities, offering alternative means of accessing course materials and support. Additionally, by providing real-time assistance and guidance, ChatGPT can bridge learning gaps and ensure equitable learning opportunities for all students, regardless of their backgrounds or abilities. By situating ChatGPT within the broader landscape of educational innovation, our study illuminates the transformative role of technology in shaping academic environments. Through the utilization of AI-driven tools like ChatGPT, educators can revolutionize teaching and learning practices, fostering greater engagement, personalization, and inclusivity. Embracing innovative solutions like ChatGPT becomes essential for educators to meet the evolving needs of learners and create dynamic and enriching educational experiences as technology continues to evolve.

To ensure that the findings from this study are actionable and beneficial, we offer several specific recommendations for educational decision-makers aimed at optimizing the integration of AI tools such as ChatGPT in academic settings. Firstly, it is advisable for educational leaders to devise a structured strategy for adopting and integrating these technologies. This could include pilot programs to evaluate the tool’s effectiveness within particular educational contexts before wider deployment. Additionally, it is essential to provide comprehensive training for both students and faculty, which should not only cover how to use these tools effectively but also include discussions on ethical considerations, data security, and best practices for technology integration in educational processes.

Furthermore, establishing clear ethical guidelines and policies is critical. These policies should address data privacy, prevent academic dishonesty, and ensure the integrity of educational assessments [36]. In terms of support for diverse learning needs, decision-makers should utilize AI to enhance accessibility and inclusivity, tailoring technologies to provide personalized learning experiences that meet individual student requirements.

Regular evaluation and feedback mechanisms should also be implemented to continually assess the impact of these AI tools on educational outcomes. Such feedback, gathered from both students and faculty members, can drive iterative improvements to both the technology itself and its application in educational settings.

Moreover, fostering a culture of innovation within educational institutions is crucial. Encouraging faculty members and students to explore and experiment with AI technologies can be supported by establishing innovation labs or centers dedicated to educational technology research and development. Lastly, it is vital to address technological inequities to ensure that the deployment of AI tools like ChatGPT does not widen the digital divide. This includes providing necessary technological resources to all students to ensure that everyone can benefit equitably from AI integration [35,36,37,38].

8. Limitations of the Research

Our study primarily focused on pre-service teachers with the aim of understanding their perceptions of ChatGPT and its potential utility in educational settings. This targeted approach was chosen due to the crucial role pre-service teachers play as future educators and early adopters of new educational technologies. While this focus provided valuable insights into the perspectives of this specific group, it also limited the generalizability of our findings to a broader population of educators and students. Further investigation into the pedagogical integration of AI technologies like ChatGPT across different educational levels and disciplines could offer deeper insights into the specific needs and challenges associated with such implementations.

In addition, expanding the scope of the research to include students from various academic disciplines, cultural backgrounds, and educational levels would enhance our understanding of the broader implications and utility of ChatGPT across different contexts. Future studies could aim to involve a wider range of participants, which would help to identify specific needs and perceptions across disciplines, potentially leading to more tailored and effective integration strategies for AI tools in education. Additionally, comparative studies could be conducted to highlight any significant differences or similarities in perceptions between pre-service teachers and other student groups, thereby providing deeper insights into the pedagogical integration of AI technologies like ChatGPT.

Additionally, the reliance on a structured questionnaire for data collection, while efficient, may not capture the depth and nuances of students’ experiences and opinions regarding ChatGPT. Such a methodological approach might overlook the complexity of students’ interactions with AI tools and their impact on learning processes. Furthermore, the study’s cross-sectional design does not allow for the assessment of changes in students’ perceptions over time, particularly as they gain more experience with ChatGPT or as the technology itself evolves. Lastly, the analytical methods employed, including descriptive statistics and inferential analysis, provide a snapshot of current attitudes but might not fully elucidate the underlying reasons for these perceptions or explore potential long-term implications for teaching and learning.