The Good and Bad of AI Tools in Novice Programming Education

Abstract: As AI coding tools become more prevalent in programming, it is essential to understand how they influence programming education. This study, conducted in a first-semester Introduction to Programming course, aimed to determine the positive and negative effects of these tools on students’ learning experiences and their ability to develop essential programming skills. Using a mixed-methods approach, we collected data from 73 teams of engineering students over a 12-week period. Students completed surveys and reported on their AI tool usage. We analyzed this data quantitatively to identify trends in tool familiarity, usage, and student satisfaction. Additionally, qualitative analysis of student reports provided insights into the specific ways AI tools were used and their perceived benefits and drawbacks. The findings revealed a significant increase in AI tool familiarity (from 28% to 100%) and usage among students. Students’ satisfaction with AI tools improved over time. The most prevalent tasks for which novice programmers used AI tools included creating comments (91.7%), identifying and correcting bugs (80.2%), and seeking information (68.5%), while other tasks were less common. While these tools offered benefits like assisting in learning and enhancing real-world relevance, they also raised concerns about cheating, over-reliance on AI tools, and a limited understanding of core programming concepts.