Fostering AI Literacy in Elementary Science, Technology, Engineering, Art, and Mathematics (STEAM) Education in the Age of Generative AI

Abstract

The advancement of generative AI technologies underscores the need for AI literacy, particularly in Southeast Asia’s elementary Science, Technology, Engineering, Art, and Mathematics (STEAM) education. This study explores the development of AI literacy principles for elementary students. Utilizing existing AI literacy models, a three-session classroom intervention was implemented in an Indonesian school, grounded in constructivist, constructionist, and transformative learning theories. Through design-based research (DBR) and network analysis of reflection papers (n = 77), the intervention was evaluated and redesigned. Findings revealed clusters of interdependent elements of learner experiences, categorized into successes, struggles, and alignments with learning theories. These were translated into design moves for future intervention iterations, forming design principles for AI literacy development. The study contributes insights into optimizing the positive effects and minimizing the negative impacts of AI in education.

1. Introduction and Background

The potential adoption of artificial intelligence (AI) in various sectors in Southeast Asia is promising, with significant implications for financial services, healthcare, the high-tech and telecommunication sector, and education. In ASEAN education, AI-based intelligent tutor systems have been adopted and studied by Singapore and Malaysia to provide personalized teaching [1]. The ubiquity of smartphones and internet connectivity has made AI an integral part of children’s lives in many countries. For instance, AI-powered applications such as language learning apps, personalized tutoring systems, and educational games are increasingly being used by children for learning purposes [2].

Moreover, children are also interacting with AI through voice assistants like Siri, Alexa, and Google Assistant, which are commonly found in smartphones and other smart devices. These AI assistants can answer questions, set reminders, play music, and perform a variety of other tasks, thereby familiarizing children with AI technology from an early age [3,4].

In the context of STEAM (Science, Technology, Engineering, Art, and Mathematics) education, numerous research studies have employed artificial intelligence (AI) to enhance learning. For instance, Hsu et al. [5] explored the use of AI image recognition in conjunction with a microcomputer control board. Kim and Kim [6] investigated teachers’ perspectives on the application of AI in scientific writing within STEM disciplines. A unique approach was adopted by Narahara and Kobayashi [7], who trained a toy robot car using TensorFlow, an open-source machine learning framework pre-loaded on the Raspberry Pi, to infuse more AI elements into robotics workshops for STEAM education. Additionally, Leonard [8] briefly discussed the potential of AI in art education, providing examples of Google AI tools such as Quick Draw and Auto Draw.

Despite the numerous benefits of integrating AI across various sectors, including education, ethical issues pose significant challenges to its adoption [1]. It is crucial to recognize that while experiences exposing children to AI are valuable, education alone does not equip them with the necessary knowledge and skills to understand, use, and evaluate AI technologies in an ethically appropriate manner. This highlights the importance of incorporating AI literacy into the formal school curriculum, a step that many Asian countries should consider.

To address the urgent need for AI literacy among children, this study adopted a design-based research approach. It applied key learning principles derived from the literature on AI literacy in an intervention grounded in cognitive constructivism, social constructivism, constructionism, and transformative learning theories. The intervention was implemented with grade five primary school students in Salatiga, Central Java, Indonesia. Consequently, the research question we sought to answer was: how do elementary students experience the application of learning theories and AI literacy principles in an intervention designed to facilitate their understanding of AI, their engagement in the learning process, and their ability to articulate the ethical or societal impacts of AI?

The study’s analysis drew on data from student reflective papers (n = 77) in the fourth design iteration of the AI literacy development intervention. This data enabled us to construct a network map using cluster analysis, which identified patterns in correlations among various aspects of their experience of the intervention. These nodes reflected the successes and struggles in students’ learning experiences, as well as the alignment of their experiences with principles from learning theories. Based on these findings, we developed design moves for the next iteration, providing solutions to the issues students faced. Further details regarding the interventions, the design moves, and design principles are elaborated upon in this study.

This study implemented a design-based research approach explained in the Methodology section to develop and iteratively refine an AI literacy intervention for elementary students in Indonesia. Grounded in learning theories described in the Literature Review section, the intervention (also described in the Methodology section) utilized student reflections and network analysis to derive design principles for AI literacy education suited to young learners. In the Findings section, we highlight the importance of social learning, hands-on activities, and appropriate scaffolding to make AI concepts accessible and meaningful. The Discussion section suggests that this research provides valuable direction for integrating AI literacy into elementary STEAM education through pedagogically sound methods tailored to children’s learning needs. Details of the study design, methods, and outcomes are presented in the sections that follow.

2. Literature Review

2.1. Definition of AI and Generative AI

Artificial intelligence (AI) is a branch of computer science focused on developing smart machines capable of mimicking human thought and behavior. It involves the creation of algorithms that enable machines to learn from data, make informed decisions, and solve problems. AI can be divided into two primary categories: narrow AI and general AI [9]. Narrow AI is programmed to carry out specific tasks such as facial recognition or online searches, while general AI has the potential capacity to execute any cognitive task a human can do. Generative AI specifically refers to a class of artificial intelligence models that use existing data to create new content that mirrors the underlying patterns of the real-world data [10]. These models have found significant applications across various domains, including natural language processing, computer vision, and brain imaging [10,11]. In the creative arts, generative AI tools have been instrumental in producing high-quality artistic media, including visual arts, music, literature, video, and animation [12]. We believe that the generative capabilities of these AI tools are fundamentally altering creative processes, leading to a reimagining of creativity across many sectors of society, including education.

2.2. Importance of AI Literacy in STEAM Education

AI literacy is increasingly important in STEAM (Science, Technology, Engineering, Art, and Mathematics) education. The rapid expansion of artificial intelligence (AI) in various sectors necessitates the integration of AI literacy in STEAM education. As Lee et al. [13] highlight, preparing students to become informed citizens and critical consumers of AI technology is crucial for their future endeavors as AI-empowered workers. This preparation involves not only the development of technical skills but also the cultivation of critical thinking. How and Hung [14] demonstrate how STEAM learners can use AI to predictively simulate various scenarios, thereby enhancing their problem-solving skills.

The importance of AI literacy in STEAM education extends beyond the development of technical and critical thinking skills. It also involves understanding the ethical and societal implications of AI. Zhang et al. [15] posit that students must learn about these implications alongside technical concepts and processes. This comprehensive understanding of AI as a sociotechnical system with socio-political implications is crucial for educating AI-literate citizens.

Moreover, the inclusion of AI literacy in STEAM education can foster creativity and innovation. Aguilera and Ortiz-Revilla [16] argue that the inclusion of the arts in STEM education can enhance student creativity, a key skill in the era of AI. Furthermore, How and Hung [14] suggest that AI analytics can serve as educational scaffolds in STEAM education, promoting AI-assisted human-centric reasoning and knowledge development.

2.3. Existing Models and Approaches for Developing AI Literacy in Children

As of July 2023, two articles that propose models of AI literacy have gained attention in the academic community, as evidenced by their citation counts in Google Scholar. The Long and Magerko [17] article has received 471 citations, while the article by Touretzky et al. [18] has been cited 348 times.

Long and Magerko [17] define AI literacy as a set of competencies that empower individuals to critically evaluate AI technologies, communicate and collaborate effectively with AI, and utilize AI as a tool in various settings like online platforms, home, and workplaces. Notably, they argue that programming ability is not a prerequisite for AI literacy. The authors constructed an AI literacy framework around five central questions. The first question, “What is AI?”, encompasses four competencies: Recognizing AI, Understanding Intelligence, Interdisciplinarity, and General vs. Narrow AI. The second question, “What can AI do?”, includes two competencies: AI Strengths & Weaknesses and Imagining Future AI. The third question, “How does AI work?”, incorporates nine competencies: Representations, Decision Making, Machine Learning Steps, The Human Role in AI, Data Literacy, Learning from Data, Critical Interpretation of Data, Action and Reaction, and Sensors. The fourth question, “How should AI be used?”, is associated with one competency: Ethics. The final question, “How do people perceive AI?”, is linked to one competency: Programmability.

On the other hand, Touretzky et al. [18] do not provide a direct definition of AI literacy, but they provide a conceptual framework that implies a definition. According to this framework, AI literacy involves understanding the fundamental concepts of AI, the ability to interact with AI technologies, and an awareness of the societal impacts of AI [18]. They argue that K-12 students should be equipped with the knowledge to understand how AI works and how it will shape their future. Furthermore, the authors emphasize the importance of hands-on experiences with AI technologies and the need for critical thinking regarding the impacts of AI applications. They suggest that students should be able to evaluate new AI technologies and articulate the ethical or societal impact questions they raise. This approach to AI literacy underscores the importance of not only understanding AI technology but also being able to engage with it in a meaningful and informed way [18].

2.4. Theoretical Foundations of Constructivist, Constructionist, and Transformative Learning

This study leveraged a design-based research (DBR) approach, the initial phase of which involves the identification of applicable learning theories that underscore the learning experience under design, and subsequently, translating these theories’ core concepts into a set of cogent design principles [19]. We started with constructivist learning theory, which has two strands. Cognitive constructivist learning theory conceptualizes learning as a process where individuals construct meaning and knowledge [20]. From cognitive constructivism we derived the design principles of (1) individual knowledge construction, (2) schema building or connecting ideas, and (3) knowledge consolidation or reinforcement. In social constructivist learning theory, learning is seen as collective knowledge construction mediated by artifacts, tools, language, and culture [21]. The design principles from social constructivism were (1) collaborative knowledge construction, (2) scaffolding, (3) working in the zone of proximal development, and (4) mediating artifacts, tools, and technologies. The constructionist theory advances the constructivist perspectives by positing that knowledge is active—that is, it should be treated as a verb rather than a noun—which suggests that learning occurs when students construct physical, digital, or representational artifacts [22]. As students construct artifacts that reflect the knowledge they are constructing, the construction of the artifact further informs the construction of knowledge in a positive feedback loop. From constructionist theory we derived the design principles of (1) making or generativity, (2) tinkering, exploration, and productive failure, (3) learner agency, authority, and autonomy, and (4) authentic audience or purpose with real-world impact. Transformative learning theory defines learning as a change in assumptions, beliefs, and frames [23]. The design principles from this theory were (1) questioning beliefs or assumptions, (2) developing new perspectives, and (3) changing beliefs or assumptions.

3. Methodology

This study employed a design-based research approach, and focused on the fourth iteration. Design-based research is an iterative, collaborative process conducted in real-world settings, driven by both theory and empirical evidence, with the ultimate goal of improving educational practices. It integrates the design of learning environments with their analysis, acknowledges the importance of context, and aims to produce contextual outcomes that can positively impact education [19,24,25,26]. In conducting our design-based research, we adapted the design framework proposed by Anderson and Shattuck [24]. The first step in DBR was identifying a practical problem in a real-world context of AI literacy. In this case, our research commenced with grounding the design in the principles derived from cognitive constructivism, social constructivism, constructionist, and transformative learning theories, which framed our understanding of how students learn. This step was crucial to ensure that our intervention was theoretically sound and informed by relevant pedagogical perspectives.

The second DBR step was developing a solution informed by existing learning theories and research about AI literacy. Following the establishment of our theoretical grounding, we moved to the design phase of our intervention. Guided by the principles derived from our selected theories, we crafted a learning environment that was both supportive and conducive for students to construct knowledge, explore and experiment, work collaboratively, and engage in transformative learning.

The third step in the DBR process was to iteratively test and refine the solution in the context of improving students’ AI literacy. Once the intervention was designed, we then implemented it in a real-world setting. The intervention was carefully redesigned based on findings from our analysis of the previous intervention in iteration 3. Throughout and following the implementation of the intervention, we collected data to evaluate its effectiveness. We used students’ self-reflections as the primary source of data. In the reflections, students were required to explain what worked, what didn’t work, and what new idea(s) they learned from the learning activities. This step was critical in understanding how students interacted with the intervention, what they learned, and how they felt about the learning experience.

The fourth step in DBR was to evaluate the findings from the previous step to produce changes to the design of the intervention. In this step, we analyzed data to craft design moves. After data collection, we undertook a thorough analysis of the data to identify patterns of relationships between challenges, successes, and alignment of aspects of learning experiences with principles from the theories in which the design was initially grounded. We developed a network map of significantly correlated aspects of learning experiences described in students’ reflections to identify patterns in students’ struggles, what worked particularly well for them, and principles from learning theories aligned with their experiences. These patterns allowed us to leverage strengths of the design and principles from theories to address struggles. The insights derived from this analysis informed our design moves as part of lesson modifications and refinements to improve the intervention. This step helped us understand what worked, what didn’t, and what needed to change to enhance the effectiveness of the intervention.

The final DBR stage was to derive design principles from the design moves that can guide future work, followed by sharing the results to contribute to theory and knowledge about AI literacy. This step involved sharing findings and the refined intervention with the broader research community, contributing to the knowledge about AI literacy.

The analysis in this study focuses primarily on the fourth iteration, with a brief description of prior iterations to provide the reader with sufficient context. In line with the DBR approach, this intervention did not happen as a one-off occurrence. Instead, it was a systematic and iterative process consisting of four iterations. Each iteration was carefully informed by data analysis from the preceding cycle. The aim of these four iterations was not merely to create, but to continually refine and improve the intervention, ensuring its effectiveness in supporting student learning.

3.1. Participants and Context

The study was conducted in Salatiga, Central Java Province, Indonesia, and involved two public schools centrally located in the city. The first school was the site for iterations 1-3, with a total of 55 fifth-grade primary school students participating. The second school was involved in iteration 4, where 25 fifth-grade primary school students participated. Both schools had parental consent for the students involved. According to teacher interviews, the students in both schools predominantly come from middle-low-income families. While almost all students have access to smartphones and the internet, the second school additionally provided one computer per student with internet access. It should be noted that the students’ computer literacy is at a beginner level, and although they had no prior knowledge of AI, they were all familiar with platforms like YouTube and the Google search engine.

3.2. Intervention Design and Implementation

The goal of the intervention was to refine best practice and strategies to improve primary school students’ AI literacy by using principles of AI literacy from the literature. The design principles were derived from learning theories mentioned in the literature review section. This study focuses on analysis of data from the fourth iteration. However, to help the reader better understand the fourth iteration, we will first describe each of the previous iterations and the design moves resulting from analysis of those iterations. The design moves were constructed to leverage the principles and strengths in order to systematically resolve the struggles, based on an analysis of the relationships between struggles, successes, and theories evident in the learning experience network map. This exemplifies an evidence-based approach to refining the intervention by allowing the data on student learning experiences to guide incremental improvements grounded in theory.

3.2.1. Iteration 1: Design and Design Moves

Iteration one was divided into three sessions. Each session consisted of a 60 min lesson. The following paragraphs explain each session in detail.

Session 1—Interaction

In this session, students used AI to solve problems where children collaborated with others using AI. This session consisted of three main activities.

Session 1 Activity 1—Categorizing and classifying jumbled pictures (unplugged): Students in the first activity developed a strategy to categorize images. Then they were expected to explain the process of classifying images. Lastly, they needed to recognize patterns and cognitive dissonance in the categorization processes. Students were given jumbled pictures. Then they worked together in groups of 4–5 to sort the pictures into at least 3 categories (see Figure 1). The students were not given guidance or hints about what categories are possible. Once they finished, another unidentified and uncategorized picture was added, enabling them to have a little debate among the members of the group to decide the appropriate categorization of the new picture. Then, each group briefly shared their chosen categories with the other groups.

This unplugged activity was designed to help the student build a basic understanding about data training in machine learning. In machine learning, categorization is a fundamental part of training a model. For example, in supervised learning, data points are categorized into different classes or labels, and the model is trained to predict these labels accurately. The task of grouping pictures into categories teaches students to identify common characteristics, features, or patterns within a set of data that can be used for classification. Recognizing cognitive dissonance in the categorization process can serve as a useful introduction to dealing with noisy or inconsistent data. In the real world, data is often imperfect and can be challenging to classify consistently. Cognitive dissonance in this context might come from images that could plausibly fit into multiple categories or from images that don’t clearly fit into any category. This mirrors the situations that a machine learning model often needs to deal with, and strategies to manage this uncertainty (such as probabilistic classification, or creating a separate “other” category) can be explored by the students.

Session 1 Activity 2—Collaborate with others and AI using Teachable Machine: In the second activity of the first session, students used Google’s Teachable Machine [27] to collaborate with an AI and with friends to solve a problem, which in this activity involved finding a way to make an AI recognize their faces, things, or gestures. In groups of 3 with one computer for each group, students were asked to build classifiers in Google’s Teachable Machine (see Figure 2). They were instructed to choose the following categories: “boy-girl”; “happy-sad”; or make your own category. Then the students built these classifiers by taking as many pictures as possible with their webcams. The pictures had to be representative of the 2 categories in the classification scheme they had selected. These pictures were input automatically in the Teachable Machine (they did not have to save the photos to their computer first). After the classifiers were built, they tested the algorithm with at least 3 new pictures correctly classified in a row. If it was not classifying correctly, they had to either re-train the classifier or figure out why it was not working and explain the problem.

Teachable Machine is a web-based tool that allows users to train machine learning models visually, without coding. There was a whole-class discussion after students finished their work. This activity was designed to help students learn about the concept of training data by using their webcams to take pictures that are then used to train the AI. They experience how these data are used to inform the AI’s understanding and categorization. This provides a concrete example of how AI systems learn from data. Students also learn about problem-solving and debugging. If the AI is not classifying correctly, students will need to troubleshoot and figure out why. This process can improve their understanding of how AI works and the types of issues that can arise, such as bias in the training data. This also enhances critical thinking skills and collaboration, which are crucial for AI literacy.

Session 1 Activity 3—Self-reflection: A reflection activity was used not only to help students consolidate their learning, but also as part of data collection. It was conducted with the students writing manually on paper. The questions students addressed in their reflections were (1) “What worked really well today? What part of it did you enjoy most?”; (2) “What did you struggle with today? Why did you struggle?”; (3) “What new ideas did you have today?” They were expected to explain as deeply as possible how they experienced the learning activities, including the challenges they faced. This data, along with data from reflections at the end of the next two sessions, was then used for analysis, resulting in a list of design moves for improving the next iteration.

Session 2—Interaction, Explaining, and Evaluating

In this session, students interacted with AI products. Then, grounded in the principles of constructionist learning theory, students utilized AI for art and creative expression, while also developing an understanding of the underlying logic and processes that the AI uses. This session consisted of 5 main activities.

Session 2 Activity 1—Interaction demo with AI: The instructor used a smart speaker and smart plug to introduce this new type of AI. Then we let students interact with the products by giving commands, asking questions, or otherwise communicating with the smart speaker in a game called Hey Google! In this game, the students needed to prompt Google to mention a word the instructor provided. This activity was designed because smart speakers and smart plugs are ubiquitous, tangible, and interactive examples of AI technologies. Interacting with these devices provides students with firsthand experience of AI in action, helping them understand its capabilities and limitations.

Session 2 Activity 2—Pictionary game (unplugged): In this activity, the instructor started a drawing, and the student guessed. The student who guessed correctly took the next turn to draw on the whiteboard. In this activity, the instructor needed to start with an easy example and a difficult example. The purpose of this activity was to demonstrate the limitations and potential ambiguity in AI. Just as an image drawn by a person may be ambiguous and lead to incorrect guesses, AI algorithms can also struggle with ambiguity, leading to misclassification or errors. It can also help students understand that the quality of the output (the guess) heavily depends on the quality of the input (the drawing).

Session 2 Activity 3—Guessing game using Google Quick Draw: The instructor had students volunteer to come to the front of the room and draw on the teacher’s computer (projected on the classroom overhead projector) using Google’s Quick Draw [28] and the other students guessed what they were drawing. Then students had a whole-class discussion about how they guessed and how the AI guessed. The goal of this activity was to deepen the students’ understanding of the crucial role quality data plays in training effective machine learning models. By using Quick Draw, they experience firsthand how both they and the AI make guesses based on the data (drawings) provided. The subsequent discussion aims to highlight the parallels between their human processes of interpreting drawings and the AI’s process of recognizing patterns. Through these experiences, students learn that the accuracy of AI predictions heavily relies on the quality and clarity of the training data.

Session 2 Activity 4—Making art using Google AutoDraw: In groups of 4, each student opened AutoDraw [29] on their computer. They started drawing for 15 s. When the timer stopped, they had to stop drawing and move to another student’s computer in their group. Then they added to their friend’s drawing for 15 s. Then they moved to another student’s computer. They repeated this until all students had contributed to all their group-mate’s drawings, which took approximately 3 min. Then everyone in each group looked at each of their member’s drawings and discussed the original idea versus the emergent idea (see Figure 3). Then the instructor had each team clear all their drawings and do 2 more rounds.

This learning activity was designed to help students recognize that each addition of data impacts the end result. This also helps them recognize some of the limitations of AI.

Session 2 Activity 5—Self-reflections: The self-reflection activity was similar to the reflection activity in the first session.

Session 3—Recognition and Ethics

The objectives of session 3 were to help students be able to recognize the product of AI, as well as increasing their ethical awareness about AI. Thus, after completing the session 3 activities, students should be able to understand the good and bad effects of AI on society, create a poster of how to identify AI products, and create a poster of how to protect yourself and others from the bad effects of AI. This session consisted of 4 activities.

Session 3 Activity 1—Observing AI products: This first activity began by watching a clip on YouTube from the American Got Talent show about the use of AI to create “hyper real content”, in this case a deepfake of Simon Cowell (one of the judges) singing on the stage [30]. Then students discussed briefly how it works and what the purposes may have been. By watching the video, students will have an example of how AI is being used in a real-world context. The purpose of this activity was to demonstrate how AI can be used in creative and entertaining ways, potentially inspiring students to consider how they could use AI for good purposes. Then, students watched the Indonesian YouTube video of news about how an AI was used to create deepfake content of the Indonesian president [31]. By watching the video, the goal was to open up a discussion about the ethical considerations of AI use. They are expected to learn that deepfakes can also be misused, such as for spreading misinformation or fraud. Understanding this helps students to consider the wider implications and responsibilities that come with developing and using AI technologies.

Session 3 Activity 2—Instagram AI filters: In this activity, the instructor connected a smartphone to the projector and had students volunteer to come up and mess up the video and image Instagram AI filters (see Figure 4). Instagram filters are a tangible demonstration of how AI technology is embedded in our everyday lives.

This activity was designed to help students realize that AI isn’t just about robots or complex algorithms, but also about the apps and tools they use daily. Using Instagram filters can also open a discussion about the ethical and social implications of AI, for instance, how altering images can create unrealistic beauty standards, or how image recognition might be used to spread misinformation.

Session 3 Activity 3—Evil genius plan group project: This activity engaged students in collaborative work in groups of 3 to develop an “evil genius” plan of using AI for bad things. Then students were asked to discuss in small groups about how we can protect ourselves (and people around us) from AI-produced harmful effects, deepfakes, or fake news. Then each group created a short presentation of their strategies for protecting ourselves (see Figure 5).

This activity was designed to help students become aware of the potential misuse of AI by developing “evil genius” plans. This includes deepfakes, targeted misinformation, manipulation of personal data, and more. This helps them understand the ethics and darker side of technology, promoting a more balanced view of AI’s pros and cons. The presentations could be in the form of a poster or PowerPoint presentation as a means through which students reify the knowledge they constructed.

Session 3 Activity 4—Self-reflection: The self-reflection included questions similar to the previous questions in session 1 and 2 but with the addition of 3 more questions: (4) “From all 3 days we worked together, what was the most wonderful thing?”; (5) “From all 3 days we worked together, what was the worst problem?”; and (6) “From all 3 days we worked together, what was the most important new idea you had?”

3.2.2. Iteration 1 Design Moves

Through analysis of the network map of significantly correlated struggles (the data analysis process will be described in the section on iteration 4 below in greater detail), what worked particularly well, and experiences aligned with principles from theory we constructed a set of design moves. The first design move involved modifying the session 1 activity to use different types of photographs that more closely resemble the types of categorizations they will do in activity 2. In this case, the instructor will need to use photos of a diversity of people doing a diversity of things wearing a diversity of clothes. This activity will also be modified to limit the number of categories each group makes to 2 categories. Another design move was that when students conduct data training with Teachable Machine, the instructor will give more time for students to tinker with training the AI. This tinkering time will be done collaboratively in groups and will allow for optimal learner agency. During the tinkering, the facilitator will encourage them to talk a lot about what they are thinking. The instructor will have to start with a discussion right after the teacher demonstration about what kinds of data the AI does not handle well. This activity will also be modified to limit the number of categories each group uses for data training to 2 categories. This will keep their work within their zone of proximal development.

The next design move was that in session 2 the teacher will guide the students in what kinds of things the AI (Google Home speaker) can understand, and will give them hints for better recognition during the activity. Moving to session 2 activity 2, in guessing the picture, it will be modified to be a whole-class activity where the teacher translates (elicit translations first) each prompt in Quickdraw before pressing the “Got it!” button. The time for discussion will be changed from 10 min to 5 min. In using AutoDraw, there will be 2 “rounds” of drawing in teams, but between the 1st and 2nd round, students will be asked to discuss as a whole class about what the AI needs from them, such as what types of data AI will use. In activity 2, the instructor will have to connect a smartphone to the projector and have students volunteer to come up and play with the video and image Instagram AI filters.

Moving to the design moves for session 3, students will have a whole-class discussion about protecting ourselves. Students may have a discussion while doing a new DALL·E activity. The instructor will use DALL·E for making deepfakes on the instructor’s computer using deepfake ideas from the evil plan activity. The activity will be modified also to be more collaborative—each team has to come up with only one evil genius plan and teams will compete for the title of “evil genius”—and the students will collaboratively create and present posters for their presentation.

3.2.3. Iteration 2: Design and Design Moves

The list of design moves from the analysis of iteration 1 described above were implemented in iteration 2. Analysis of the data resulted in a new list of design moves. First, all activities in the next iteration should end with knowledge consolidation and reinforcement discussions, especially focusing on categorizing and classifying data. In the first activity, the reason for the activity will more clearly be stated in the beginning and at the end of the activity. The instructor will emphasize that it is not about right and wrong, but instead, it is important to explain the logical reasons for each group decision in categorizing and classifying data.

Another design move was that in the second activity, the instructor should tell the student sitting at the computer to act like a “driver”, and that they cannot make any decisions. Rather, the teammates will take on the role of “navigators” and tell them what to do. If the “driver” has an idea, they have to ask their teammates, “can I do something like this?” The goal of this is to prevent the person sitting at the computer from being overly dominant. Students will also be asked to move away from the computer during instructions and discussion time. Students will be told that they can huddle up at the front of the classroom like a sports team before a game. Then, the instructor will have the students explain to their classmates what they were planning to do and check to make sure they understood. In the second activity, students also will be given a chance to present the result of their work. In their presentation, students will have to provide the reasons for why the data training failed or was successful.

In design moves for session 2, the Google Home speaker assignment was changed to emphasize that we are testing the limits of AI, rather than testing students’ ability, because some students in this iteration felt disappointed when Google Home could not give them the answer they wanted. During the interaction with Google Home speaker, the student volunteers will also be told to raise their hand right before they say “Ok Google”. The other students can give advice, but they have to become completely silent when the student speaking says “Ok Google”. At the end of each activity in session 2, students will engage in a knowledge consolidation discussion about what they learned. The design for the Quickdraw activity will be changed to emphasize that we are not testing the speed of or the student’s ability in drawing, but rather that we are testing the AI speed and accuracy. Thus, if they run out of time, that means that their tinkering is good, but the AI-human collaboration has failed. The AutoDraw assignment originally was planned to be in teams of 4 students, but during implementation, it was changed to be a whole-class activity in which students lined up to move like a snake. In the next iteration, the activity may be changed to be done in teams of 5 students, and the instructions also should emphasize adding ideas, but never erasing previous students’ ideas.

The major design move for session 3 was that the evil plan activity should be modified to be divided into two steps. In step 1—before the instructor forms the groups—each student will silently write two to three evil genius ideas individually on sticky notes with 1 idea on each note. Then, the instructor will divide the students into several groups, after which each group member will present all of their ideas. Finally, as a team, they should vote on which idea was the most interesting. In step 2 students will collaboratively create the evil plan poster. Also in session 3, related to the Instagram assignment, instead of using 1 phone, there will be 3 phones operated by the instructor and two other facilitators (university students) who will be recruited by the instructor. Then the instructor will divide the class into three large groups. Each group will take a phone and play with the video of the facilitator. At the end, each of the other 2 facilitators should send the screen recordings to the primary facilitator, who will show them on the overhead projector. During the knowledge consolidation at the end of the evil genius activity, there should be a greater focus on what new things they have learned. At the end of each activity in session 3, there should be a knowledge consolidation discussion. During the DALL·E deepfake activity, the instructor will continually refocus the discussion on questions such as “AI can trick us, but can we trick AI? How can the data we give the AI determine the quality of the AI? How can we fight back to have control over AIs that might be trying to control us or manipulate our data?”

3.2.4. Iteration 3: Design and Design Moves

All of the design moves derived from the analysis of patterns of learning experiences in iteration 2 were implemented in iteration 3. Data were collected and analyzed to construct a new list of design moves for the subsequent iteration. In terms of design moves for the first session, the instructor will need to be observing more and asking questions of students about their decisions to help them develop tinkering skills and encourage them to be confident in making decisions without being afraid of making mistakes. The purpose of this design move is to promote productive failure. Also, the instructor will tell the students again that the main goal of the activity is to find logical reasons for making decisions. This should be emphasized in the beginning, middle, and end of the activity. Students will also be reminded again that making mistakes is part of learning. In this case, when students fail, the instructor needs to ask the group why they failed and lead them to the process of tinkering and exploration to find the best solution. If there is enough time, the instructor will lead the students in discussion with the goal of guiding them towards arriving at their own conclusion that people learn through mistakes and AIs also learn through mistakes. When categorizing the pictures, the instructor and facilitators will need to ask leading questions in each group such as “what is the ‘big idea’ of this category?” and “what is the ‘story’ of this category?” They have to make sure the students are talking a lot about the meaning of the category instead of talking about unrelated things. Also, the instructor and facilitators will help them just a little bit so they stay on time. In the activity with the Teachable Machine, the instructor will ask the students to be as “silly” as possible when capturing data. Also, they will need to push each group to provide more data samples for AI training.

The design moves for the second session included changing the “Hey Google!” activity to be shortened to 6 min, as well as shortening the demo to 30 s. It was decided that a new set of keywords for the instructor to provide the students will need to be tested with various people outside the class before the session. During the second activity, the instructor should frequently remind them to slow down and not attempt to compete with the computer. Before students start doing the AutoDraw activity, the instructor will show students the functions of each button of the tool.

The major design move for the third session involved changing the evil genius activity such that the instructor will start by forming groups and have the groups discuss several evil genius ideas, then individual students will write on sticky notes more ideas individually. Then the activity will continue as planned. In the activity with Instagram, at the beginning of when the groups form, each facilitator will explain the navigation and functions of Instagram filters. Then, during the time students are tinkering with the Instagram filters, the facilitators will provide more guidance when they see students struggling or doing things that obviously won’t have the desired effect. Also, more facilitator talking and interaction should be encouraged such as asking students “what are you trying to do there?”.

3.2.5. Iteration 4 Design

The design moves from iteration 3 were implemented as planned with some additional changes of the design for iteration 4. In the design of iteration four, in session 1, before the categorizing and classifying activities (unplugged), the instructor adds an analogy of being a programmer. As a programmer, students will imagine that they are going to program a robot to decide which picture belongs to which category (in which they decide the category together as a team). The purpose of doing that is to build the schema for the students about how data training works. In session 2, before the class begins, the instructor will need to make sure that the Google Home speaker and the smart plug are working properly, as some technical issues may delay the learning process. In session 3, three projectors connected to smartphones are needed. Initial setup before the class should have been conducted. The purpose of doing that is to increase time efficiency and effectiveness for students to explore the use of Instagram AI filters in 3 big groups. In the last session, the instructor will need to provide both paper surveys and online surveys. Students in this case may choose to do the reflection offline or online.

3.3. Iteration 4 Data Collection and Analysis

This paper presents the analysis of iteration 4. The data collection and analysis for iteration 4 was conducted as follows.

3.3.1. Data Collection

Reflection papers were collected online using Google Forms at the end of each session. The digital format allowed students to formulate and rethink their responses over time, without the pressure of crafting a perfect response in one sitting. They could revise and resubmit their answers as necessary. Also, using the online form allowed automated collection and organization of responses. In these reflection papers, students were asked to write about what worked well, struggles they encountered, and new things they learned. In addition to producing data for this study, this reflective practice promotes a comprehensive understanding and self-assessment of their learning experiences, which is crucial for the ongoing success of our study. A total of 77 reflection papers were collected.

3.3.2. Qualitative Coding

After the reflection papers were collected, the next step was to code the data in MAXQDA Analytics Pro qualitative analysis software. The challenges and difficulties identified in the reflections were coded as “struggles”. The successes students experienced are coded as “what worked.” The student experiences were coded for alignment with principles from cognitive constructivist learning theory, social constructivist learning theory, constructionist learning theory, and transformative learning theory. Coding resulted in 38 codes covering 1057 coded segments.

3.3.3. Network Analysis

The rationale for conducting learning experience network [32] analysis was to leverage the relationships between struggles, what worked, and alignment of student experiences with principles from learning theories in order to leverage what worked, and alignment of experiences with theory to address struggles. Also, it is a holistic and expansive method that keeps the complexity of learning intact, rather than reducing learning to quantitative measures.

Once the reflection papers were coded in MAXQDA, correlation analysis was conducted in order to examine significantly correlated relationships between all possible pairs of codes. We used Pearson’s 1-tailed tests because the type of network analysis we conducted required that we only analyze positive correlations. The resulting Pearson correlation values were exported to MS Excel as a symmetrical correlation matrix. In preparation for network analysis, the matrix was edited to remove all values except those that were significant at the p < 0.05 level. The matrix was then transformed into a network database in the UCINET network analysis software.

Creating network maps allows us to analyze the complex interdependencies between multiple codes, representing the complexity of relationships between different aspects of learner experiences. Learning experience network maps are crucial in analyzing complex interdependencies among various codes, representing the intricate relationships between different facets of learner experiences [33].

In UCINET, we utilized the Girvan–Newman algorithm, a method for detecting clusters within a network based on the concept of betweenness centrality [34]. The algorithm identifies the edges in a network that have the highest betweenness centrality score, removes these edges, and continues this process iteratively. This results in the division of the network into distinct clusters or communities. The Girvan–Newman algorithm is particularly effective in detecting community structure where there is no prior information about the communities and provides a powerful tool for identifying and understanding the organization and relationships within complex networks. We used cluster analysis to identify distinct patterns in relationships between various aspects of learning experiences characterized as struggles, what worked, and principles from theory, and to do so in a way that honors the complexity of learning.

We also calculated the betweenness centrality of every node. Betweenness is a measure that captures the extent to which a node within a network lies on the shortest path between other nodes, highlighting the influence a specific node exerts over the interactions of other nodes in the network [35]. This quantifies the number of times a node acts as a bridge along the shortest path between other nodes, reflecting the amount of control a particular node (in our data, a node represents a type of learning experience) exerts over the interactions of other nodes in the network. We adjusted the node sizes such that the nodes that have the highest betweenness centrality were the biggest nodes to indicate visually the types of experience that were most critical and could serve as leverage points when making changes to the learning experience design.

After constructing the learning experience network map, we identified all the struggles, strengths, and theory principles within each cluster. Utilizing a table format (see

Table 1. Learning experience network analysis design moves table template.

Struggles	Strengths	Theory Principles	Design Move
(black triangle cluster) Some students struggled with …	For these students, … worked really well.	These students’ experiences aligned with the … theory principles of …	(Design move to be constructed—leverage the theory principles and strengths in this cluster to address the struggles)
(blue square cluster) Some students struggled with …	For these students, … worked really well.	These students’ experiences aligned with the … theory principles of …	(Design move to be constructed—leverage the theory principles and strengths in this cluster to address the struggles)
(red circle cluster) Some students struggled with …	For these students, … worked really well.	These students’ experiences aligned with the … theory principles of …	(Design move to be constructed—leverage the theory principles and strengths in this cluster to address the struggles)
(grey diamond cluster) Some students struggled with …	For these students, … worked really well.	These students’ experiences aligned with the … theory principles of …	(Design move to be constructed—leverage the theory principles and strengths in this cluster to address the struggles)

for an example) serves to organize the data more effectively, making the connections among various data points easier to visualize, and thereby facilitating the formulation of solutions. The design moves table consisted of rows representing each cluster and four columns, with the first column representing the struggles encountered by the students. The second column highlighted the strengths of the learning experience as represented by aspects that the students reported as working particularly well. The third column captured the principles from learning theories, showcasing which learning theory principles were most aligned with the students’ struggles and successes within each cluster.

The final column of the design moves table was where we proposed potential solutions that leveraged the principles from theory and strengths to address the related struggles.

4. Results

In our study, we utilized the Girvan–Newman cluster analysis method to identify four distinct clusters of aspects of learning experiences correlated at the p < 0.05 level. A modularity value (Q) of 0.697 was obtained, indicating a robust community structure. These clusters represented various facets of student learning, which included the learners’ successes, their struggles, as well as the alignment of their experiences with learning theory principles. The size of the nodes within each cluster was shaped by their betweenness centrality values; larger nodes signified greater importance within the network. We categorized the clusters as the black triangle cluster, blue square cluster, red circle cluster, and grey diamond cluster. Detailed descriptions of the learning network analyses for each cluster are presented as follows.

Through Girvan–Newman cluster analysis, four clusters were identified with a Q value of 0.697, where the size of the nodes signifies their betweenness centrality values (see Figure 6). Based on the network map presented with labels, the clusters allow us to visualize the connections between the struggles, effective practices, and learning theories. In order to create design moves for each struggle in the clusters, the analysis was conducted in four main steps:

• In-depth Node Analysis: We began by investigating each cluster in detail. We describe the nodes and the relationships between them within each cluster, including evidence from the raw data, such as direct quotations from students. This enables us to understand the struggles students faced, ensure an evidence-based solution, and identify the successful learning activities implemented.
• Relational Analysis: We examined the relationships between struggles, successful practices, and alignment with theoretical principles. This provides valuable insights into the learning theory principles that contribute to students’ learning success and helps identify the link between successful activities and supporting learning theories.
• Leveraging Success and Theoretical Principles: We used what worked well and the principles from learning theories to address the struggles in each cluster. The nodes with the highest betweenness values (i.e., struggles, successful practices, and principles from theories) can be expected to have the most impact when either successfully addressed or leveraged to handle related struggles.
• Addressing Struggles: Lastly, we carefully addressed the learning difficulties students experienced. If there was a cluster without any struggle nodes but connected to another cluster, the successful practices and principles from theory nodes in the first cluster could be used to address struggles in the connected cluster. The learning theories are utilized to provide solutions for the struggles found in each cluster.

4.1. Black Triangle Cluster Analysis

The black triangle cluster (see Figure 6) is connected to the blue square cluster, indicating that there is a relationship between both clusters. This cluster reflects students’ successes in learning AI when they worked together with their classmates, collaborated with AI and friends, and enjoyed fun activities with enthusiasm specifically in the activities where they collaborated with AI and friends to express their imagination. Students in this cluster did not report experiencing any struggles in their learning. Thus, in the connection with the blue square cluster, the black triangle cluster takes the role of providing leverage points in terms of principles from theories and aspects of the experience that worked particularly well. These leverage points were used to address struggles in the blue square cluster. The first principle from learning theories included in the black triangle cluster was individual knowledge construction from cognitive constructivist theory. One of the students said, “I found out that when I sketch, AI can identify what I want to draw”; another said, “I enjoy face training [training AI for face recognition] because it makes me laugh and I can learn new things”. This means students were engaging in an individual cognitive process when they expressed their ideas and were actively involved in the experience. The second principle was making (or generativity) from constructionist theory. One student said that “I like drawing and coloring because AI helps suggest images”, suggesting the presence of AI as a collaboration partner in the creative process enhanced the students’ engagement. Furthermore, AI’s ability to provide suggestions offered a form of co-constructed generativity, inspiring the students’ imagination and enabling them to explore new ideas or artistic directions. Through this collaborative interaction with AI, the students are actively constructing their knowledge and expressing their imaginations in a meaningful way. The last principle from theory in this black triangle cluster was the social constructivist theory principle of scaffolding. One student reported, “Nothing is difficult because I can do it with friends”, indicating that these students found the learning process was feasible because they could collaborate with their friends through discussion and sharing of knowledge. Through these interactions, the students received scaffolding support, which helped them understand and engage with difficult concepts or problems.

4.2. Black Triangle Cluster Design Moves

As mentioned before, the black triangle cluster had no struggles. Therefore, the strengths and the theory principles in this cluster can be leveraged to help address the struggles students had in the blue square cluster. However, this also means that there were no design moves specific to the black triangle cluster.

4.3. Blue Square Cluster Analysis

In the blue square cluster (Figure 6), there were 2 struggles that students experienced, 3 aspects of the learning experience that worked particularly well, and 2 principles from theories aligned with student experiences. The two struggles were categorizing and classifying data and students’ failure in training the AI. For instance, one student said, “The challenging part is when training AI to recognize faces because my face is almost similar to [my classmate’s]”. Other students said, “I cannot provide data to AI” and “I have difficulty to use webcam”. Yet another student reported that “The difficult part is when grouping images because I am confused about which group to place them in”. Based on the data, we can see that students tried to build their understanding, but problems arose when conducting data training with Teachable Machine such as having similar data, having difficulty operating the camera, and experiencing difficulty in providing good data to the AI. Some students also had difficulty in the unplugged activity of categorizing and classifying images. At this level, students gained an understanding about the importance of the data for the effective result of AI data training. On the other hand, in the same cluster, these students achieved some successes in recognizing AI and the products of AI, categorizing and classifying data, and data training (input datasets, training, prediction). For instance, a student said that “I just learned that training AI to recognize faces is done in such a way that during the training, there should be no person next to or behind the face being recognized. If the face is covered or if there is someone next to or behind it, the AI cannot recognize it”. The students’ experiences in this case aligned with the constructionist principle of tinkering, exploration, and productive failure. The student in this case isn’t just learning from what works but also from what doesn’t work, which is a central tenet of the constructionist approach. These failures become valuable learning opportunities, fostering deeper understanding and problem-solving abilities. Another student said, “I like when categorizing images successfully because I am discussing (with friends)”. In this context, the students’ learning experience aligns with the social constructivist principle of collaborative knowledge construction. This suggests that students co-constructed understandings of how AI works, which they developed through collaborative discussions or activities. These students did not passively receive information, but actively applied it in a real situation and created a product (a trained AI model).

4.4. Blue Square Cluster Design Moves

Based on the interdependent relationships between what worked, principles from theory, and struggles in session 1, there are several solutions and changes needed for the next iteration. The first design move will be adding a “think-aloud protocol” as part of the first activity, a practice rooted in social constructivist theory. This theory emphasizes that learning happens through social interactions and the active co-construction of knowledge. By instructing students to vocalize their thoughts while classifying and categorizing images, they will be encouraged to articulate their thought processes and decision-making criteria. This not only helps individual students clarify their own understanding for their own benefit and the benefit of peers, but also allows peers and instructors to provide immediate feedback and guidance, fostering a richer collaborative learning environment.

The second design move is that the instructor needs to remove overly difficult images from the categorization activity and replace them with simpler ones. This relates to the principle of scaffolding in social constructivist learning theory. By starting with easier tasks, students can build foundational skills and understanding before moving on to more complex problems. This step-by-step approach with gradual release of structure allows learners to gradually develop deeper comprehension and skills, reducing frustration and encouraging engagement.

The third design move will be adding a quick “test” round before the main Teachable Machines activity, aligning with the constructionist principle of tinkering, exploration, and productive failure. This provides students with an opportunity to experiment and make mistakes in a low-stakes setting before they attempt the main task. The test round acts as a space for trial and error, allowing students to explore the AI tool, understand its functionality, and learn from any initial mistakes.

4.5. Red Circle Cluster Analysis

Students in the red circle cluster (see Figure 6) experienced struggles while participating in the evil plan activity. This activity requires students to collaborate with friends to create an evil plan for using AI in bad ways. The purpose of the activity was to increase their ethical awareness of the potential negative effects of AI. Some of the students’ quotes are: “The difficult part when planning a crime is because I don’t like crime”; and another student reported “The difficult part is when [looking for] ways to use AI for crime”. On the other hand, some other students in this cluster experienced successes in the evil plan activity, as well as the activity involving manipulating data with AI. In the data manipulation activity, students used Instagram to edit images using AI camera filters. The aspects of the experience that students in this cluster reported as working particularly well were in line with principles from learning theories in the cluster. First, in social constructivist theory, the zone of proximal development (ZPD) refers to the distance between what a learner can do independently and what they can achieve with assistance. This concept was evident in the students’ use of Instagram filters. A student reported on what they found easy: “Using Instagram filters because it is not difficult to use them”. Another student said “Taking photos using IG filters because it’s fun/cool. It’s easy because assisted by AI and mentors”. They quickly learned to use these filters independently because it was not difficult, and they reported that it was fun or cool. Their learning was further supported by AI and mentors, which could be seen as scaffolding in the ZPD. Second, the transformative theory principle of developing new perspectives was related to other nodes in this cluster. One of the students realized that AI can be misused for criminal activities. The student said, “I just found out that AI can be used for criminal activities”. This shows a shift in perspective about AI technology. This student was previously unaware of the potential misuse of AI, but now had a broader understanding, demonstrating transformative learning. Also, in this cluster was the transformative theory principle of changing beliefs and assumptions. One student said, “[The activity that worked well was] Deepfakes/filters can change voice, face, etc., can be used for good or evil”. In this case, the statement about deepfakes and filters being able to change voices, faces, and possibly being used for good or evil represented a change in understanding. Before developing this knowledge, students might have assumed these tools are purely for entertainment. After learning about their potential misuse, however, their beliefs about these tools’ scope and potential risks evolved.

4.6. Red Circle Cluster Design Moves

In order to address struggles that the students in the red cluster encountered, a set of design moves were constructed regarding session 3 activities. The evil plan activity will have an introduction in a form of discussion about an analogy: “How do police detectives find out who did something bad? Do they have to imagine how bad people think? So, good people can do good work by imagining what evil genius people might be thinking so that we can stop them or protect ourselves and our loved ones.” Using this analogy aligns with the social constructivist theory principles of work within the zone of proximal development (ZPD) as it offers the necessary assistance to bridge the gap between what students can understand independently and the new concepts they are trying to grasp. The analogy also provides a context for the exercise, thus helping to shift the focus away from the problematic framing of “planning a crime” to thinking strategically about how AI could be misused and how such misuse could be prevented. This can alleviate students’ discomfort with the idea of “planning a crime,” making the activity more palatable. Also, the analogy encourages students to step into the shoes of others (in this case, police detectives), prompting them to think from a different perspective. This is aligned with transformative theory, which emphasizes how learning often involves a change in perspective. It can help students see AI from a different angle—as a tool that, like any other, can be used for both good and evil purposes, deepening their understanding of the ethical dimensions of AI.

4.7. Grey Diamond Cluster Analysis

Students in the grey diamond cluster (see Figure 6) experienced two kinds of struggles, which were related to handling technical errors in using computers and having difficulties in working with classmates or collaborating with friends. Students said, “There’s something difficult, which is drawing using auto draw, because I don’t like to draw”, and another one said, “The difficult part is when playing IG [Instagram]”. Yet, another student reported that “The difficult part is when working in a group”. On the other hand, these students reported positive experiences of interacting with AI when they use, play, talk, and ask questions. A student said, “I like it when AI listens to our voice when we want to ask something because AI provides the answer to what we ask of it”. These students reported experiences that aligned with the social constructivist theory principle of mediating artifacts, tools, and technologies. In this case, AI was acting as a mediating artifact. Another student said, “I just found out that AI can turn on lights or fans using a Google [home speaker], so we can turn on lights or fans just by speaking”. In social constructivist theory, artifacts are objects that are used to support learning, and AI was being used to facilitate the students’ collaborative knowledge construction. The students were learning by interacting with AI, posing questions to it, and then getting responses that aided their understanding.

4.8. Grey Diamond Cluster Design Moves

The design moves from the grey diamond cluster will address the struggles students encountered in both session 2 and session 3. In session 2, the drawing activity will be changed so that students can sit together with their friends that enjoy drawing. Those that do not want to draw may sit together with their friends and help by contributing ideas for drawing. This will avoid any technical difficulties for students who do not enjoy or have confidence in their technical skills of drawing. Allowing students to group together based on their comfort level with drawing can alleviate some of the stress for those who are not confident in their drawing skills. This approach aligns with the social constructivist theory principle that learning is inherently a product of social interaction. For students who enjoy drawing, they will serve as mediators, helping their peers understand the process. This supports the social constructivist principle of learning with mediating artifacts, tools, and technologies: in this case, the more confident drawers become “human tools” assisting their peers.

In session 3, the struggles with working with classmates and collaboration showed up primarily in the evil plan activity. This activity will be modified to start with a short discussion about why collaboration is more powerful than individual work, framing collaboration through the concept of the result being greater than the sum of the parts. Initiating a discussion on the advantages of teamwork prior to the evil plan activity is expected to reinforce the idea that collective brainstorming can lead to richer, more innovative, and more diverse ideas. This is a key component of social constructivist theory, which emphasizes that knowledge is co-constructed through social interaction and shared experiences. This can help students understand that everyone’s input is valuable and that a variety of perspectives can enhance problem-solving. Another design move is that the evil plan activity will be modified to let students form their own groups so they can have a more playful atmosphere with their friends. Allowing students to form their own groups for the evil plan activity aligns with the social constructivist view of learning as a socially mediated activity. By working with friends, students may feel more comfortable and engaged, fostering a more positive and productive collaborative experience. This creates a supportive environment where students can express their ideas freely, fostering more effective collaborative learning and mutual support.

5. Discussion

This section discusses the design principles for developing AI literacy in elementary students, grounded in the study findings and learning theories underpinning the intervention. The results are interpreted in relation to the literature, and strengths, limitations, contributions, and future directions are described.

5.1. Design Principles for Developing AI Literacy in Elementary School Students

Based on the design moves in iteration four, in order to develop AI literacy for elementary schools’ students, we need to first ground the intervention based on learning theories, create safe and comfortable learning environments, and solve the students’ struggles through scaffolding. Figure 7 shows the design principles we derived through categorizing the design moves constructed for iteration four.

We believe that to enhance students’ AI literacy, it is crucial to ground all interventions across every iteration in effective learning theories. These theories include cognitive constructivism, social constructivism, constructionism, and transformative learning theory, in which student–student collaboration and student–AI collaboration are central to the learning process. This type of interaction will increase students’ creativity and empower students to make stuff instead of merely remembering things. The unique and supportive interactions that were created in the study served as the foundation for a safe and comfortable learning environment, enabling students to select their own groups or pairs, thus promoting optimal learning, and allowing them to freely express their ideas. This learning environment must allow tinkering, exploration, and productive failure in learning. Students will also feel safe to voice their ideas, learn from their mistakes, and ask questions without any hesitation. Should students encounter any difficulties in their learning journey, the instructor needs to employ scaffolding techniques. These could involve asking questions that guide towards solutions. For instance, when students struggled with failures in training the AI model during a Teachable Machine activity, the instructor provided tailored scaffolding by asking guiding questions such as, “What do you think would happen if you removed any blurry pictures and replaced them with better quality photos?” and “Could you check if there is too much similarity between categories?” This enabled students to troubleshoot issues and incrementally build the skills needed to train an effective model. In order to reduce cognitive load, the instructor will need to employ framing to help students interact with difficult concepts, for example by constructing analogies. As a result, students are more likely to engage deeply with the material and overcome any problems they encounter.

5.2. Interpretation of the Results in the Context of the Literature Review

The findings of this study are in line with Long and Magerko’s [17] claim that programming ability is not a prerequisite for AI literacy. Our results demonstrate that students could effectively understand AI concepts and interact with AI technologies without necessarily having any programming skills. This highlights the feasibility of integrating AI literacy into broader STEAM education, aligning with the arguments of Lee et al. [13] and How and Hung [14], who have emphasized the importance of AI literacy for preparing students for future AI-empowered roles. Additionally, the data aligns with the principle that generative AI, as defined by Gong et al. [10], can foster creativity in STEAM education. The students in our study demonstrated increased creativity when using generative AI tools, affirming the notions of Epstein et al. [12] who found generative AI’s significant application in the creative arts. Our findings also reinforced the importance of understanding the ethical and societal implications of AI, as suggested by Zhang et al. [15]. Participants showed a growing awareness of AI’s broader societal impacts, illustrating the success of an AI literacy approach that incorporates ethical considerations. Moreover, the use of constructivist, constructionist, social constructivism, and transformative learning theories [20,21,22,23] in shaping the learning experience and design moves generated positive outcomes. Consistent with these theories, students constructed meaning from their individual and collaborative experiences, were active in the learning process, and demonstrated changes in their assumptions and beliefs about AI. The results are also in line with Touretzky et al.’s [18] claim that hands-on experiences with AI technologies are vital for fostering AI literacy. Our data suggest that hands-on engagement enhanced the students’ ability to critically evaluate AI technologies and articulate the ethical or societal impacts of AI applications.

5.3. Strengths and Limitations of the Study

The study design was grounded in established learning theories, such as cognitive constructivism, social constructivism, and transformative learning theory, providing a robust framework for understanding and analyzing students’ experiences and outcomes. The use of learning experience network analysis to help address students’ problems in learning activities turned out to be an effective evidence-based approach to crafting practical solutions to real challenges. However, there are some limitations in this study. First, the results of this study may not be generalized to all student populations, as the participants were a specific group of students in Indonesia and the source data were qualitative in nature. More research will be needed to validate these findings across diverse student populations. The study was also limited to the immediate context and did not include any long-term follow-up to assess the sustainability of the improvement in students’ AI literacy. It will be interesting to track these students over time to see how their AI literacy develops in the long term.

5.4. Contribution to the Field of AI Literacy and STEAM Education

This research study offers several valuable contributions to the field of AI literacy and STEAM education. Firstly, it emphasizes the importance of grounding AI education in multiple established learning theories. It showcases how cognitive constructivism, social constructivism, constructionism, and transformative learning theory can be used to guide the design and iteration of educational interventions. These theoretical foundations provide students with a more holistic and profound understanding of AI, fostering both theoretical knowledge and practical skills.

Secondly, this research identifies the necessity of a safe and comfortable learning environment in promoting optimal learning outcomes. The study demonstrates that students are more likely to engage deeply with the material when they feel at ease to voice their ideas, embrace learning from their mistakes, and ask questions without fear of retribution. This highlights the role of the learning environment as not just a backdrop but a crucial element in students’ AI literacy development.

Thirdly, our findings emphasize the role of scaffolding in supporting students’ learning. This is especially important in a complex and rapidly evolving field like AI, where students may struggle with the technical intricacies and theoretical concepts. By providing appropriate support at the right moments, educators can help students overcome these challenges, thereby fostering deeper learning and comprehension.

In addition, the study highlights the potential of integrating AI into more traditional STEAM subjects, demonstrating how AI can be a tool for enhancing learning in these areas. This not only boosts students’ AI literacy but also helps them see the real-world applications of their STEAM knowledge, which can enhance motivation and engagement.

Finally, this research contributes a model for designing AI literacy interventions in elementary school settings. By articulating specific “design moves”, it offers a framework that other educators can adapt and apply in their own contexts. This represents a significant step towards creating more effective and inclusive AI literacy development for younger learners.

5.5. Future Directions for Research and Practice

Future research directions can expand the demographic scope of this study to include diverse student populations from different geographic regions, educational levels, and socio-economic backgrounds. Implementing long-term follow-up studies could also help understand the impact of AI literacy improvements over time. Educators and policymakers should also consider incorporating the learning experience network analysis approach into their teaching methodologies, especially in areas with a significant emphasis on AI literacy. Furthermore, considering the rapidly changing nature of AI technologies, continuous revision and updating of AI literacy instructional design would be crucial to keep pace with industry trends and future careers.

6. Conclusions

The study found that meaningful collaboration and the use of analogies and scaffolding support grade 5 students in understanding AI concepts without needing programming skills, thereby supporting the integration of AI literacy into broader STEAM education and highlighting students’ creativity and awareness of AI’s ethical implications. Grounding AI literacy education in theories like cognitive constructivism, social constructivism, constructionism, and transformative learning can create an optimal learning environment. Techniques like scaffolding and analogies enhance engagement, while integrating AI with traditional STEAM subjects boosts literacy and real-world application [16,20,21,22,23]. Fostering AI literacy from an early age prepares children for an AI-empowered future, with early exposure contributing to responsible and ethical use, potentially mitigating ethical issues associated with AI [18]. This study offers a tentative set of design principles for incorporating AI literacy in elementary schools, paving the way for future research and practice. Expanding the demographic scope and including long-term studies could deepen understanding of AI literacy development strategies’ impact over time.