Submit to this Journal Review for this Journal Propose a Special Issue

Article Menu

Share Help Cite

Open AccessArticle

Peer-Review Record

The Good and Bad of AI Tools in Novice Programming Education

Educ. Sci. 2024, 14(10), 1089; https://doi.org/10.3390/educsci14101089 (registering DOI)

by Rina Zviel-Girshin

Reviewer 1: Anonymous

Reviewer 2:

Muhammad Irwan Padli Nasution

Reviewer 3:

I Kadek Suartama

Educ. Sci. 2024, 14(10), 1089; https://doi.org/10.3390/educsci14101089 (registering DOI)

Submission received: 26 August 2024 / Revised: 29 September 2024 / Accepted: 30 September 2024 / Published: 6 October 2024

(This article belongs to the Special Issue How Artificial Intelligence Can Enhance Education: Current Practices and Challenges)

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

While the topic of the paper is indeed timely and engaging, it would benefit from a more diversified research design, including enhanced comparisons to traditional educational approaches. Additionally, expanding the reference list and incorporating more educational content would strengthen the statistical foundation of the results. The recommendations for future research should involve more technical approaches.

Comments on the Quality of English Language

The quality of the English is average. It could be better to make prrofreading.

Author Response

Comment 1: While the topic of the paper is indeed timely and engaging, it would benefit from a more diversified research design, including enhanced comparisons to traditional educational approaches.

Response 1:

Thank you for your insightful recommendation. We agree that incorporating a more diversified research design will strengthen the paper. We have added a section that enhances the comparisons to traditional educational approaches, providing a more comprehensive analysis.

Traditional teaching methods often suffer from low student participation, lack of personalized instruction, and insufficient motivation. Modern approaches in education enhance active learning approaches. Traditional education often relies on theory-based materials, but the integration of ChatGPT can foster personalized learning experiences tailored to individual needs and preferences. Students can leverage ChatGPT to outsource certain knowledge tasks, allowing them to concentrate on 'hands-on' learning and gain practical experience in their chosen fields. Additionally, ChatGPT could also create adaptive learning environments that respond to individual learner progress and performance (Baidoo-Anu, & Ansah, 2023; Pardos, & Bhandari, 2023; Ray, 2023).

Constructivist theory emphasizes that students construct knowledge through their own experience and participation. Integrating AI tools into a constructivist framework can significantly enhance educational experiences by encouraging students to take charge of their educational journeys through real-time assistance like clarification of concepts, answering questions, and offering tailored guidance based on individual learning needs and styles. With AI tools, educators can create environments that emphasize active learning. Students can engage in inquiry-based activities, utilizing AI as a resource for exploration and problem-solving (Chen, & Zhao, 2024; Fischer, et al., 2023; Huang et al., 2024; Lo, 2023; Mishra et al., 2023). AI integration may transform educators from content providers to facilitators, emphasizing mentorship and the development of soft skills (Baidoo-Anu, & Ansah, 2023; Firat, 2023; Rudolph et al. 2023; Tlili et al., 2023).

Cognitive load theory posits that learning is most effective when cognitive load is managed, helping students focus on essential concepts without being overwhelmed (Koć-Januchta et al., 2022; Sandoval-Medina et al., 2024). AI tools can reduce extraneous cognitive load by automating routine tasks, allowing students to concentrate on understanding core programming principles. This can be particularly useful for novice programmers who may struggle with complex concepts.

Mandai and co-authors (2024) in their opinion paper discuss the potential impacts of ChatGPT on higher education through the lens of educational theories by John Dewey’s Reflective Thought and Action model, and the revised Bloom’s taxonomy. The analysis is based on a review of existing literature on ChatGPT and educational theories. The key points mention positive expectations, like ChatGPT can enhance personalized learning, shift education towards more practical, hands-on learning by reducing the need for memorizing information, and lead to assessment reforms. They also mention negative expectations, like over-reliance on ChatGPT, failing to acquire necessary skills, diminished student creativity and originality, and raised concerns about the authenticity and accuracy of sources.

Lo (2023), in his literature review, provided an analysis of the implications of ChatGPT in educational contexts. He assessed its capabilities across various subject domains, its potential applications in education, and the challenges it poses. ChatGPT's performance varied significantly across different subjects, demonstrating outstanding results in critical thinking and economics but unsatisfactory performance in mathematics and medical education. Potential applications in education include acting as an assistant for instructors and/or a virtual tutor for students. He also identified several challenges, such as accuracy issues, plagiarism concerns, and the need for updated institutional policies to address these issues. He concludes that while ChatGPT holds significant promise for enhancing educational practices, careful consideration must be given to its limitations and the ethical implications of its use.

Comment 2: Additionally, expanding the reference list and incorporating more educational content would strengthen the statistical foundation of the results.

Response 2:

Additionally, we expanded the reference list:

Baidoo-Anu, D., & Ansah, L. O. (2023). Education in the era of generative artificial intelligence (AI): Understanding the potential benefits of ChatGPT in promoting teaching and learning. Journal of AI, 7(1), 52-62.

Biswas, S. (2023). Role of ChatGPT in Computer Programming. Mesopotamian Journal of Computer Science, 2023, 9-15.

Chen, R., & Zhao, H. (2024, March). ChatGPT in Creative Writing Courses in Chinese Universities: Application and Research. In 2024 12th International Conference on Information and Education Technology (ICIET) (pp. 243-247). IEEE.

Fischer, R., Luczak-Roesch, M., & Karl, J. A. (2023). What does chatgpt return about human values? exploring value bias in chatgpt using a descriptive value theory. arXiv preprint arXiv:2304.03612.

Firat, M. (2023). What ChatGPT means for universities: Perceptions of scholars and students. Journal of Applied Learning and Teaching, 6(1), 57-63.

Haleem, A., Javaid, M., & Singh, R. P. (2022). An era of ChatGPT as a significant futuristic support tool: A study on features, abilities, and challenges. BenchCouncil transactions on benchmarks, standards and evaluations, 2(4), 100089.

Huang, Z., Mao, Y., & Zhang, J. (2024). The Influence of Artificial Intelligence Technology on College Students' Learning Effectiveness from the Perspective of Constructivism—Taking ChatGPT as an Example. Journal of Education, Humanities and Social Sciences, 30, 40-46.

Jalil, S., Rafi, S., LaToza, T. D., Moran, K., & Lam, W. (2023, April). Chatgpt and software testing education: Promises & perils. In 2023 IEEE international conference on software testing, verification and validation workshops (ICSTW) (pp. 4130-4137). IEEE.

Lo, C. K. (2023). What is the impact of ChatGPT on education? A rapid review of the literature. Education Sciences, 13(4), 410.

Mandai, K., Tan, M. J. H., Padhi, S., & Pang, K. T. (2024). A Cross-Era Discourse on ChatGPT’s Influence in Higher Education through the Lens of John Dewey and Benjamin Bloom. Education Sciences, 14(6), 614.

Mishra, P., Warr, M., & Islam, R. (2023). TPACK in the age of ChatGPT and Generative AI. Journal of Digital Learning in Teacher Education, 39(4), 235-251.

Pardos, Z. A., & Bhandari, S. (2023). Learning gain differences between ChatGPT and human tutor generated algebra hints. arXiv preprint arXiv:2302.06871.

Ray, P. P. (2023). ChatGPT: A comprehensive review on background, applications, key challenges, bias, ethics, limitations and future scope. Internet of Things and Cyber-Physical Systems, 3, 121-154.

Sandoval-Medina, C., Arévalo-Mercado, C. A., Muñoz-Andrade, E. L., & Muñoz-Arteaga, J. (2024). Self-Explanation Effect of Cognitive Load Theory in Teaching Basic Programming. Journal of Information Systems Education, 35(3), 303-312.

Surameery, N. M. S., & Shakor, M. Y. (2023). Use chat gpt to solve programming bugs. International Journal of Information Technology and Computer Engineering, (31), 17-22.

Vukojičić, M., & Krstić, J. (2023). ChatGPT in programming education: ChatGPT as a programming assistant. InspirED Teachers' Voice, 2023(1), 7-13.

Also we added the following parts to the discussion section:

This study uniquely combines quantitative and qualitative methods to examine novice programmers’ interactions with AI tools. The findings illustrate a clear evolution in usage patterns, indicating that as students became more familiar, they were more likely to voluntarily use AI tools even in assignments where it wasn't required.

And later

... This suggests that prompt engineering is a critical skill that can be developed over time and should be an integral part of AI education. Therefore, teaching students how to effectively interact with AI tools can enhance their learning experiences, a point that has not been extensively addressed in prior literature.

And...

This means that AI tools can minimize extraneous cognitive load by automating routine tasks, thereby enabling students to focus more effectively on mastering core programming principles.

And These results are consistent with the findings of Biswas (2023), Haleem et al. (2022), Jalil et al. (2023), and Surameery & Shakor (2023), which indicate that ChatGPT can provide users with explanations, examples, and guidance, assist with debugging by analyzing data on the programming language, code structure, and error messages, code documentation and even assist with code review.

And later in the conclusion part

Incorporating AI tools into courses represents a new essential form of digital literacy in modern education. As technology becomes integral to various aspects of life and work, proficiency in AI tools is as crucial as traditional digital skills. This new literacy involves not only effectively using AI but also critically assessing its outputs and understanding its ethical implications. By integrating AI tools into curricula, educators equip students with the skills needed to navigate an increasingly automated future, fostering creativity and innovation. Ultimately, teaching students how to leverage AI thoughtfully prepares them to thrive in evolving job markets while enhancing their problem-solving abilities.

Comment 3: The recommendations for future research should involve more technical approaches.

Response 3:

Thank you for your valuable advice regarding the recommendations for future research. I’m pleased to share that we have already initiated a new experiment. We have added a section to the future research recommendations that outlines our upcoming comparative study between traditional teaching methods and AI-enhanced learning approaches across various courses:

We have received approval to conduct a comparative study between traditional teaching methods and AI-enhanced learning approaches in different courses: introduction, regular, and advanced level courses. In this study, one group of students will receive instruction through conventional methods, while the other group will participate in a variation of the flipped classroom model. In the flipped classroom, the instructor will provide a brief overview of the material, accompanied by a list of guiding questions for students to explore, check, or study using AI tools for research and problem-solving. Later, both groups will take a closed-book and closed-device test in class to assess their comprehension of the material. After this initial assessment, we will switch the groups, allowing the students who experienced traditional instruction to engage with AI tools, and vice versa. This crossover will help eliminate potential biases and ensure that any differences in test performance can be attributed to the teaching methods used rather than the individual capabilities of the students. We will analyze the results to determine the effectiveness of AI-enhanced learning on student comprehension compared to traditional methods. Additionally, we aim to gather qualitative feedback from students about their experiences with both instructional approaches, which will provide insights into their perceptions of AI tools in the learning process. This research could contribute valuable data to the ongoing discussion about the integration of AI in education and its potential to enhance student learning outcomes.

In addition we made English editing.

We appreciate your feedback and look forward to your thoughts on these additions.

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

Please describe clearly the indicators measurement, explore data interpretation and significant findings to show the novelty.

Author Response

Comment: Please describe clearly the indicators measurement, explore data interpretation and significant findings to show the novelty.

Thank you for your valuable suggestions. We really appreciate your feedback and have made revisions accordingly. We have added a detailed section to the methodology that clarifies the indicators used for measurement, including familiarity with AI tools.

The impact of AI tools was measured using several indicators: familiarity with AI tools was assessed through pre- and post-semester questionnaires that captured students' self-reported familiarity levels; the frequency of AI tool usage in assignments was evaluated with a dichotomous (yes/no) question, “I used AI tools during this assignment,” focusing on weeks where usage was not explicitly required to identify trends in voluntary adoption; students' comfort levels with AI tools were tracked during weeks 3, 7, and 10 using a 5-point Likert scale, allowing for the assessment of how comfort evolved as they became more accustomed to the tools;satisfaction with AI tools was measured through survey questions asking students to rate their satisfaction with the results provided by the tools on a Likert scale, offering insights into their perceptions of the quality of outcomes from AI tool usage.

Also the following part was added to the discussion

And later

And...

This means that AI tools can minimize extraneous cognitive load by automating routine tasks, thereby enabling students to focus more effectively on mastering core programming principles.

And also

These results are consistent with the findings of Biswas (2023), Haleem et al. (2022), Jalil et al. (2023), and Surameery & Shakor (2023), which indicate that ChatGPT can provide users with explanations, examples, and guidance, assist with debugging by analyzing data on the programming language, code structure, and error messages, code documentation and even assist with code review.

And later in the conclusion part

We also updated and added new references: