Leveraging Gamification in ICT Education: Examining Gender Differences and Learning Outcomes in Programming Courses

Abstract

This study investigates the differential effects of gamification on learning outcomes, motivation, and usability perceptions in an introductory programming course, focusing on gender differences. While gamification has shown promise for increasing student engagement in educational settings, its impact may vary across genders. An experimental study was conducted with 88 university students randomly assigned to gamified and non-gamified groups. Learning gains were assessed through pre- and post-tests, motivational factors were measured via questionnaires, and usability perceptions were evaluated using the Technology Acceptance Model (TAM) questionnaire. Results revealed that women learned significantly more than men in the non-gamified condition, while men outperformed women in the gamified condition. Furthermore, men reported higher enjoyment, usefulness, and comfort with the gamified tool than women. Interestingly, both genders indicated greater satisfaction with the non-gamified version. These findings contribute nuanced insights into how gamification impacts genders differently in programming education, suggesting that gamification may hinder women’s learning while modestly benefiting men. The study highlights the importance for practitioners to carefully consider gender dynamics when implementing gamified approaches, potentially offering customization options or blended techniques to optimize learning outcomes for all students in programming education.

1. Introduction

Gamification, defined as integrating game-like elements into non-game contexts, has emerged as a promising strategy to enhance motivation and facilitate learning in various educational settings [1,2]. However, while gamification has proven effective in increasing student engagement, its impact may not be uniform across all groups, particularly when considering gender as a variable of interest. Recent studies have highlighted significant differences in how men and women respond to gamification, suggesting that these dynamics could differentially influence educational outcomes [3].

Gender differences in learning and motivation are particularly pronounced in computer programming. Women tend to adopt more versatile approaches to information processing and skill acquisition, while men often show a greater affinity for competitive structures common in gamification [4]. However, research on how gamification could influence the gender gap in programming skill acquisition is limited and often contradictory. Some studies suggest that gamification benefits both genders equally [5,6], while others indicate that it may exacerbate inequalities, favoring men more significantly [7,8].

Numerous studies have focused on determining how gamification affects student motivation and engagement [9,10,11,12]; however, there remains a gap in longitudinal studies that assess the impact of gamification on learning [13,14,15], particularly in programming [16]. The current evidence is insufficient to generalize the results due to the diversity of protocols and assessment tools [17].

This experimental study aims to analyze how gamification affects gender differences in programming skill acquisition. An experiment was conducted with university students in Chile, randomly assigned to gamified and non-gamified groups. Learning outcomes were assessed through pre- and post-tests, while motivational factors and usability perceptions were evaluated using an adaptation of the Technology Acceptance Model (TAM) questionnaire. The findings reveal that women learn significantly more in the non-gamified condition, while men moderately benefit from gamification. These results underscore the importance of considering gender dynamics when implementing gamified approaches in programming education.

1.1. Research Questions

• RQ1: Are there learning differences by gender when applying gamification techniques?
• RQ2: Are there gender differences in motivation levels when applying gamification in terms of enjoyment (a), usefulness (b), and satisfaction (c)?
• RQ3: Are there gender differences in ease-of-use levels when applying gamification regarding correct usage (a) and comfort (b)?

1.2. Document Structure

The document is structured into several key sections that address the study of the impact of gamification in programming education, with a particular focus on gender differences. It begins with an abstract that presents the objectives and main findings. This is followed by an introduction that contextualizes the topic and highlights the importance of gamification in education. Next, the literature review provides a detailed analysis of previous studies on gamification, its applications in education, and gender differences in learning. The experimental design section then describes the methodology, study subjects, and measurement instruments. The results are presented in detail, accompanied by statistical analyses highlighting differences in learning outcomes and motivation between males and females under gamified and non-gamified conditions. Finally, the document concludes with discussions and conclusions that interpret the findings in the context of the study, suggesting implications for educational practice and possible directions for future research.

2. Literature Review

2.1. Gamification

Gamification is a technique that incorporates elements of games into non-game contexts to improve engagement, motivation, and performance [2,18,19,20]. This strategy focuses on psychological theories of motivation and co-creating individual value [21]. By introducing game elements, such as rewards, competitions, and achievements, into non-game settings, the goal is to stimulate people’s participation and concentration on tasks or activities that might otherwise be monotonous or uninteresting [18].

Existing studies show that gamification has been used in education, the workplace, and marketing [12,22,23], with benefits such as increased motivation and engagement, but there remains limited understanding of its long-term effects and best practices [22,24,25].

The gamification techniques employed in this study are derived from PBL (Points, Badges, and Leaderboards), which has been utilized in numerous studies related to gamification [26,27]. In our studies, we will use two techniques from PBL:

• Scores: This is one of the most common elements in gamification. They involve rewarding user points for performing specific desired actions in a gamified system [28]. Studies utilizing scores demonstrate positive effects, as seen in [29], where implementation of a scoring system in a programming course increased student motivation and engagement with learning. Furthermore, the obtained score correlated with academic performance. In the study by [30], it is noted that scores generate a positive feedback loop by providing users with instant information about their performance. This reinforces desired behavior in the learning process. However, placing too much emphasis on scores may foster unhealthy student competition. Focusing on skill mastery rather than score accumulation is recommended [31].
• Leaderboards: These correspond to ranking boards that publicly display a sorted list of participants based on their performance in a specific activity. In learning, leaderboards allow students to see how they compare to their peers in academic tasks. Studies have shown that leaderboards can positively affect student motivation and engagement. For instance, in [7], a leaderboard was implemented in a university programming course, and it was found that students performed 28% more voluntary tasks when a leaderboard was visible. Students reported that it motivated them to track and compare their progress. In the case of [32], a study with computer science students demonstrated that leaderboards stimulated healthy competition among peers and helped improve participation in programming tasks. Additionally, [31] implemented leaderboards and observed increased student motivation, satisfaction, and commitment. However, some studies warn about the potential adverse effects of excessive competition promoted by leaderboards, such as anxiety and reduced collaboration among students. A systematic review by [33] identified a performance loss as the most common adverse effect, with leaderboards being the most cited game design element associated with adverse effects. Therefore, it is essential to implement leaderboards carefully, balancing them and aligning them with pedagogical objectives. In our case, the leaderboard is automatically constructed based on the cumulative scores of students.

2.2. Use of Gamification in Education

The main uses of gamification in education are to increase student motivation, engagement, and interactions while improving learning outcomes and assessment [10,34,35]. Over recent years, practitioners have explored game design’s motivational ‘power’ in domains as diverse as work, fitness tracking, health and well-being, education, commerce, learning, crowdsourcing, information retrieval, and organization engagement [36]. A systematic review of gamification in science education highlighted its positive impact on student motivation and learning [37]. It fosters active learning and engagement by immersing students in interactive activities, promoting motivation, and enhancing learning outcomes [38]. Studies focus on exploring how to use gamification to motivate students, enhance their skills, and improve the learning process [22]. Some studies highlight the effectiveness of gamified learning in improving students’ academic performance, with no significant differences between genders, offering recommendations for educators to integrate these strategies into the curriculum [39]. According to the study by [12], gamification is not only increasingly used as an effective learning tool to create more engaging educational environments, but it also contributes to the long-term retention and application of newly acquired skills, which is crucial for lasting learning [40]. For example, a workshop that utilizes game-based elements to teach protein biosynthesis demonstrates how gamification can effectively enhance the understanding and retention of complex concepts in scientific education [41]. Gamification has been predominantly applied in educational contexts to enhance student engagement, motivation, and performance, effectively addressing challenges such as low student participation and lack of motivation [42,43].

Research on gamification is growing, especially in university education, as analyzed by [10]. Their study, which systematically reviewed 14 studies on educational gamification from 2016 to 2020, largely demonstrated that well-designed gamification, with various game elements, improves student motivation, engagement, and academic performance. In this context, the study conducted by [44] highlights the effectiveness of gamification as a strategy to enhance student engagement in programming courses. It demonstrated a significant improvement in pass rates and a reduction in dropout rates, thereby underscoring the potential of this methodology to transform education in technical disciplines. Moreover, gamified applications have shown a significant increase in both intrinsic and extrinsic motivation and the academic performance of students with learning disabilities, with high participation and engagement being critical factors for academic success in these cases [45]. Emotions are known to impact behavior, including learning behavior significantly. As various authors have discussed, positive emotions like happiness enhance learning, while negative emotions such as fear and anxiety can inhibit it [46].

Thus, the findings support the idea that educational gamification can have a significant effect on academic performance, engagement, and motivation of students, something consistent with the study [47], which determines that gamification has a positive effect on the learning process of students, regarding their involvement and performance, particularly enjoyment, engagement, motivation, achievement, satisfaction, and attitudes developed by the student.

There are also challenges, as in the study case [34], which, despite evidence of increased motivation, also identifies the potential risk of decreased motivation, classroom management challenges, and technological problems associated with using gamification in educational tools. Therefore, while it was noted that it is possible to use gamification applications to assess teaching and provide advantages such as stimulating motivation and preventing academic fraud [24], they also pointed out certain limitations, such as difficulties in classroom management and technology-related problems [34]. Gamified training programs, in particular, have been shown to improve skills when integrated with real-world experiences. Contextual factors such as culture, socioeconomic status, and technological progress significantly influence gamification’s effectiveness [48]. There is agreement on the benefits of increased motivation, engagement, enjoyment, and learning outcomes when appropriately used, but it is important to note that studies focus on the cognitive development of higher education students, and more research is needed on socio-emotional development [49]. In addition, it is challenging for technologies to be used with a student-centered approach, following appropriate educational methods, and considering students’ knowledge, interests, unique characteristics, and personality traits.

Therefore, as [35] pointed out, gamification significantly increases student motivation and participation in computer science, but its effectiveness depends on how gamification elements are applied. Particularly in software engineering education, gamification improves the quality of learning by making the process more interactive and understandable, which, in turn, positively influences the perception of ease of use, student satisfaction, and perceived usefulness, factors that impact students’ intention to participate and continue learning.

2.3. Difficulty in Learning Programming

Research indicates that learning to program computers can provide several cognitive benefits, including improved problem-solving skills, enhanced thinking skills, and positive impacts on mental development. However, some studies suggest that students learning to code may experience cognitive overload if presented with overly complex concepts too quickly [50,51,52].

Despite these advances, there is a lack of consensus on how gender differences may influence these processes when gamification is introduced, highlighting a significant gap in the literature. This lack of specific research motivates the research questions formulated in this study, which seek to clarify whether gamification affects men and women differently regarding learning, motivation, and ease of use.

Therefore, effective computer science education should maximize the cognitive advantages of learning to program while remaining aware of the potential for overload, especially among beginners. A balanced approach can allow students to acquire computational thinking skills and experience cognitive growth without undue frustration or confusion from introducing too much material too fast.

Studies show that learning to program is difficult for students due to deficiencies in deep learning and problem-solving skills, inadequate learning environments, and a focus on language and tools rather than problem-solving. However, it can be improved with systematic instruction and engaging methods [53,54,55,56,57,58], even in insufficient support environments [59]. The difficulty in learning to program cuts across different countries and cultures [60]. Students face similar challenges in the East and West, such as deficiencies in deep learning and problem-solving skills, and the difficulty relates to teaching focused on language and tools rather than solving problems by designing programs [61]. In summary, learning programming presents cross-cutting difficulties that can be improved by changing the approach and teaching methods.

As Ref. [62] points out, programming courses have high failure rates, suggesting that mastering coding skills is complex. Analysis of common errors highlights deficiencies in high-level programming fundamentals, like program structure, rather than just syntax. This points to the need to improve instruction focused on developing solid conceptual understanding and problem-solving skills in core areas.

2.4. Gamification and Computer Programming

The studies associated with gamification in teaching programming are diverse, such as that of [63], where the user experience of a gamified learning website for programming languages using points, badges, and levels is analyzed. The instrument used was a questionnaire that evaluated 48 elements in nine constructs, such as perceived enjoyment, usefulness, and immersion; all constructs scored above 4 out of 5, indicating good usability. Regarding motivation, this was assessed through a pre-test and post-test based on the ARCS motivation model. The results showed a significant increase in motivation after using the gamified website; therefore, there was a positive correlation between usability and motivation, indicating that better usability leads to greater learning motivation. Gamification has proven to be particularly effective in enhancing learning in programming courses, as demonstrated in a recent study where students who employed gamification techniques achieved significantly higher performance than those who did not, even in activities of greater taxonomic complexity [64].

There are also cases of teaching older programming languages like C, such as [65], which, using an open-source gamified platform called UDPiler to teach C programming, conducted a validation experiment with 817 first-year engineering students from Diego Portales University in Chile. Students who used UDPiler obtained statistically significantly higher exam scores compared to those who used the non-gamified platform, leading to the conclusion that gamification is a promising approach to improve engagement and learning in programming courses, especially for first-year students new to programming. This coincides with the results of [66], showing that the use of gamification concepts can significantly contribute to the teaching–learning process of programming concepts for students in the early years and adolescent students without any prior knowledge of programming concepts.

Therefore, gamification can positively impact learning programming by enhancing students’ intrinsic motivation. However, that effect changes over time depending on prior familiarity with programming, as shown by [67], where the aim was to conduct an experimental study on the use of gamification in teaching programming to undergraduate students, as applied to 19 students in a university algorithms course in Brazil through questionnaires on the Moodle platform. The assignment was random for the gamified and non-gamified questionnaires for six weeks. The learning outcomes measured were cognitive (pre/post-test), motivational (intrinsic motivation scales), and behavioral (number of completed questionnaires). The results showed that gamification positively affected programming learning by improving intrinsic motivation, which was strongly evidenced in the learning gains. However, the effect on motivation depended on the duration of the intervention and familiarity with programming, as the effect was positive initially but decreased over time for students less familiar with programming. In conclusion, gamification can improve programming learning through motivation, but the effects change over time and depend on students’ prior knowledge. It is important to note that this study highlights the limitation of a small sample size, lack of motivation data, and very restricted duration of the intervention.

Some variations include social aspects in the implementation of gamification, as in the case of [68], where a study that analyzed the use of a gamified social learning platform in a graduate course on mobile application programming, incorporating game elements such as points, achievements, leaderboards, and a virtual reward store, along with social features like forums, Q&A, blogs, and microblogging. As a result, the researchers observed that students who used the gamified platform showed better learning performance, especially in basic programming skills, compared to a control group using a traditional e-learning platform. In addition, there was a positive correlation between the number of interactions on the platform and learning outcomes; they also showed that students felt more engaged, motivated, and involved when using the gamified platform compared to the traditional e-learning platform, since the social features of the platform seemed to enhance communication and relationship ties among students. Importantly, this study also has the limitation of a small group of students and an uneven gender distribution.

Regarding the quality of programmed code, the study by [69] aims to motivate students to write higher-quality code using leaderboards and code complexity metrics in an online judge system for programming assignments with 35 students. The result was that students’ submitted analysis of the code showed greater usage of techniques like reducing duplicate code and lower cyclomatic complexity compared to control groups. Additionally, it has been demonstrated that implementing gamification elements in an introductory programming course significantly enhances student collaboration, increasing course communication efficiency by 88% and reducing response times to inquiries, especially outside working hours [70]. In addition, in terms of overall impact, gamification techniques have been used successfully in a programming course, demonstrating greater student participation and a high percentage of students passing [71], coinciding with a reduction in the high failure rate observed among computer programming students [72].

Recent studies have also confirmed these findings, such as the case of [73], which investigated learning outcomes in a programming course using the FGPE platform, where students showed significant improvement in motivation and academic performance compared to non-gamified MOOCs. Additionally, a study by [74] found that the perception of gamified elements and their impact on improving programming skills varied by gender, highlighting the importance of personalizing gamification to maximize its effectiveness.

Finally, gamification can effectively increase student activity when developing educational software [75].

2.5. Learning Differences by Gender

It is known that there are differences in how male and female students approach learning; men prefer deep/strategic learning behaviors like thoroughly understanding the material and using effective study techniques; instead, women are more versatile in their learning styles, integrating different approaches in an organized manner [76]. The differences are related to learning processes rather than achievement motivation. The results generally suggest that gender differences arise regarding learning styles [77]. Additionally, a meta-analysis revealed a small but significant female advantage in overall scholastic achievement, particularly in language courses [78]. These differences in learning styles are further supported by findings that suggest that girls outperform boys in overall GPA across various subjects, a trend partially explained by higher levels of self-discipline observed in girls compared to boys [79]. Male brains are optimized for intrahemispheric connectivity, which facilitates the coordination between perception and action, while female brains are better at enhancing interhemispheric communication, supporting the integration of analytical and intuitive processing [80]. Therefore, given that there are observable differences between male and female students in terms of their preferred learning styles, it is recommended that educators consider the strengths of the learning styles of both sexes to optimize outcomes for all students [81].

Further research has indicated that although stereotype threat did not affect the performance of girls in the single-sex school, it negatively impacted the performance of females in the coed school, suggesting that school context may moderate the effects of stereotype threat on academic performance [82]. In addition, gender (identity) differences in learning styles do not vary between teachers and, with one exception, do not vary between subjects [4]. In conclusion, the significant gender differences in different learning strategies imply their importance in teaching [83]. Another study determined that a profile of female university students with prior experience in programming languages could achieve high levels of computational thinking, particularly in the dimensions of simple algorithms and loops. This suggests that early exposure to programming may be beneficial for developing these skills in women [84].

2.6. Differences in Learning Programming Using Gamification by Gender

Research on integrating students’ learning styles and preferences with gamification in education still needs to be completed. For example, in [85], 24 studies on gamification applied to different areas of knowledge were reviewed, and none examined learning styles similar to what was shown in the systematic mapping [86]. Learning differences in programming follow the same analysis discussed above, and it could be an unsettled issue. A meta-analysis indicated that men have more favorable attitudes towards using digital games than women, highlighting gender differences in attitudes towards games, including gamification [87].

The study by Campo [88] found that implementing gamification techniques, such as virtual points and leaderboards, helped improve the academic performance of programming and software engineering students of all genders. Specifically, male and female students showed better results when these game-like elements were incorporated into the learning experience. Not all studies are consistent, as [89] revealed gender differences in gamification, where women tend to favor learning activities while men focus on obtaining rewards, underscoring the need for differentiated approaches in gamification to maximize participation based on gender. Some studies explore how gender-stereotyped gamified design can influence women’s educational experience in STEM fields; the findings indicate that gamification based on gender stereotypes can have detrimental effects on women, particularly regarding their participation and performance in educational environments. However, the unexpected results related to negative thinking call for a deeper understanding of these dynamics and their implications for future research and design practices [90]. This suggests that gamification can be an effective gender-neutral strategy to boost engagement and performance for all students, regardless of gender. Some studies suggest that men are slightly more inclined towards personalized gamification than women, with men showing approximately 5% higher participation in gamified activities [91]. Despite boys’ greater involvement and experience with computer gaming, the learning gains achieved by both boys and girls through digital game-based learning (DGBL) did not differ significantly, suggesting that DGBL can be equally effective for both genders in high school computer science education [92]; since virtual points and leaderboards generated gains for both male and female participants, the results indicate that gamification does not favor one gender over the other. Instead, it is an inclusive approach that can motivate and improve learning for both women and men. The study concluded that gamification has the potential to close gender gaps in academic performance by improving outcomes for female students to match those of men.

A study on DGBL found that third-grade girls initially exhibited lower self-efficacy than boys, but this gap diminished post-intervention, suggesting that gamification can enhance girls’ confidence in learning contexts [93]. This finding indicates that gamification may address gender-based confidence and self-efficacy disparities over time.

These findings are consistent with recent studies suggesting that gamification, when adapted to students’ individual characteristics, can reduce gender differences in education. Another study found that although there were no significant gender differences in the pre-test and post-test results, women’s participation and perceived content acquisition increased by 2.8 points compared to a 0.8-point increase for men in the post-test [94]. Furthermore, a recent empirical study demonstrated that gamification improves programming skills in engineering students, and although there is a gender perception gap in the acceptance of gamification, this strategy remains effective for both genders [74]. Another recent study confirmed that personalizing gamification according to students’ preferences and characteristics can significantly improve its effectiveness, regardless of gender [95]. Other studies provide evidence that gender differences exist in the effect of playfulness on students’ attitudes toward technology and their intention to use it. In females, playfulness directly influences attitude toward using the system. In males, this influence is mediated by perceived usefulness [96]. Finally, research published in 2023 highlights that gamified platforms can be particularly effective in improving programming learning outcomes when combined with non-gamified educational resources, demonstrating the versatility and effectiveness of gamification as an educational tool [73].

The above is consistent with that stated by [6], examining the relationship between students’ player types, gender, and preferences for gamification elements and finding no significant differences between them. When the researchers analyzed students’ player-type profiles and observed their choices for gamification techniques such as points, badges, and leaderboards, there were no statistically significant differences between male and female students for most of these elements. The data suggest that gender does not play an essential role in determining preferences for how game elements are incorporated into education. These findings prove that gender differences could dissipate over time within gamified software-based collaborative learning experiences [97]. There is also the case of the study by [98], which investigated the effects of age, gender, and frequency of use on the perception of game elements and student use, finding that these factors do not affect the perception of the game but do affect the perception of the use of game elements, which can be used to provide personalized and gamified learning experiences. Similarly, investigating the determinants of age and gender differences in the acceptance of mobile learning, [99] found that age and gender significantly influenced the intention to use mobile learning, suggesting that these demographic factors should be considered when designing educational technologies. Cultural context influences how genders engage with gaming, leading to varying gaming practices and preferences [100].

However, there are opposing results, such as that of [101], which analyzed whether gamification, using intelligent bands with gameplay functions, is an effective strategy to help participants perceive greater self-efficacy in the chosen sport or exercise routine. Perceived self-efficacy refers to one’s belief in their ability to perform tasks and is an essential factor in adopting and maintaining exercise habits. It was proposed that the experience of participating in a gamified program is intrinsically motivating (providing feelings of competence, autonomy, and social relationship) and fun. It is assumed that these motivational aspects positively influence perceived self-efficacy. The moderating effects of gender and age on the relationship between the gamification experience and perceived self-efficacy were also analyzed, where it was proposed that women and older adults have greater intrinsic motivation and, therefore, obtain more self-efficacy benefits from gamification. An online questionnaire was conducted with 233 smartband users in Spain for validation. The results supported the hypothetical model in which the gamification experience predicted greater perceived self-efficacy. This relationship was more significant for women and older adults than men and younger adults. In addition, the findings highlight the potential of gamification through intelligent wearable devices to enhance perceived self-efficacy and motivation for physical activity, especially for groups that tend to be less active (women and older adults). Something similar occurred in the study [3], which showed that gamification could improve learning in introductory programming courses, amplifying the impact of practice. Still, it must be carefully designed to prevent students from playing instead of learning. Gender was the only significant moderator of its effect, being positive for women but not for men.

Despite the numerous studies that have explored the effects of gamification in the educational context, there is a notable lack of research specifically addressing gender differences in learning programming through gamification techniques. Moreover, existing studies offer mixed results, highlighting the need for more focused research exploring how gamification might benefit or disadvantage students of different genders unequally. This leads us to formulate the following research questions, which aim to fill this critical gap in the current literature.

3. Experimental Design

The study presented in this research is designed at an experimental level using a quantitative research methodology similar to that used in [64]; for this purpose, a control group and an experimental group are considered, with both groups undergoing a pre-test and a post-test.

3.1. Test Subjects

The participants in this study are students in a digital literacy course taught in the third semester (second year) of the Accounting Auditor program at a university in Chile. This is the first time the students face a course that teaches them programming, Microsoft Excel, and algorithms.

The study was conducted in the first semester of 2022; the total sample initially consisted of 81 students, but 19 students did not complete the activity or did not want to participate in it, leaving a final total of 62 students in the sample, 31 men and 31 women, divided into experimental and control groups. The assignment of students to each group was random. The control group consisted of 30 students, and the experimental group 32. Both groups had an initial class where the content to be covered was explained.

3.2. Contents

The Information Systems 1 course has a workload of 4 pedagogical hours per week, divided into 2 h of theory and 2 h of practice. The experiment was conducted over two weeks, totaling 8 h of class. A total of 3 h was used for initial instruction on the content to be covered, and the remaining time was used to practice using the tools. The learning outcomes addressed with this experiment were the following:

• LO1: Students recognize the technical terminology of the Java language to program algorithmic solutions to fundamental mathematical problems.
• LO2: Students understand using control and iterative structures in Java to program algorithmic solutions to fundamental business problems.
• LO3: Students integrate one-dimensional arrays into algorithmic solutions in Java to provide solutions to fundamental business problems.

The main business problems to be addressed had to do with content related to the student’s professional work, such as calculating sales amounts, taxes, and inventories, among others.

The previous learning outcomes are contained in the second learning unit of the course, which proposes that students learn to build basic interactive applications in Java that they can then apply to business problems.

Using Bloom’s and Anderson’s taxonomies [102,103], we indicate that LO1 is of a lower order since it requires the use of memory (level 1: remember); LO2 is taxonomically classified as medium-low since it requires the development of the skill of constructing meaning from educational material, such as reading or teacher explanations (level 2: understand); for LO3 we assign a higher-order taxonomic complexity classification since it requires the ability to use past ideas to create new ones, generalize from supplied data, relate knowledge, and predict derived conclusions (level 5: synthesize).

3.3. Process

The process on which the experiment was based is shown in Figure 1, and the details are:

• At moment 1, the material associated with the content is available in Moodle five days before moment 2.
• At moment 2, a face-to-face session is held to provide an introductory overview of the content available and resolve students’ conceptual doubts. At this moment, the content is not explained, nor are the students taught the content related to the study. All students (experimental and control groups) participated in this activity, including those who later decided not to continue studying.
• At moment 3, a pre-test is administered to both groups, with 21 questions divided into seven for each learning objective. The test applied was the same for the entire sample of students; the time available to answer the test was 30 min. In addition, an acceptance test based on TAM (Technology Acceptance Model) is taken.
• At moment 4, students must generate a list of questions and answers to be applied in a “Rosco” (wheel) format. The students develop these questions collaboratively in an autonomous flipped class modality. The teacher later consolidates and reviews this list. The teacher corrects the questions, which become part of the question bank for the gamified activity.
• At moment 5, the students are divided into experimental and control groups, and use the tool according to their assigned group; the time allotted for this stage is fourteen days, during which students can freely access the activity according to their frequency and schedule.
• At moment 6, a test is conducted again with the same number of questions, assigned time, and difficulty as the initial test. In addition, an acceptance test based on TAM is taken.

It is important to note that students do not know about the division between a control group and an experimental group, since the division of the groups is generated using Moodle’s “groups” tool. The subjects (students from both groups) were unaware of the difference between the tool with gamification and the one without. Moments 1, 2, 3, and 4 lasted 2 h. Moment 5 covered fourteen days. Moment 6 lasted 30 min; students completed it when they finished time 5. For all students participating in the study and who completed all the stages, a bonus of 0.5 points was given for the formal evaluation of the course.

3.4. Instruments

To evaluate the achievement of learning outcomes, the gamified tool had an extensive database of isomorphic questions, with around 100 problems for each expected learning outcome. It is important to note that the experimental and control groups responded to the same pre- and post-test questions designed to assess the same learning outcomes. Although specific questions for each test were randomly selected from a larger pool, all questions covered the same learning objectives and had an equivalent difficulty level.

The quiz questions were randomly selected from a pre-developed question bank. This bank contained equivalent questions in terms of content and difficulty, all aligned with the defined learning outcomes. However, within each group (control and experimental), the questions were identical, ensuring that differences in results were not due to variations in the tests.

Additionally, the problems each student had to solve during the activities were randomly drawn from a predefined set, ensuring that all activities maintained the same consistency regarding challenge and content. This allowed for measuring the intervention’s impact without introducing biases about the difficulty or type of problems presented.

Furthermore, to explain the determinants of users’ technological acceptance, five questions based on the Technology Acceptance Model (TAM) [104] were applied to provide a vision of the degree to which students believed that using the learning tool would improve their performance and productivity in addition to determining whether it would require little or no effort. This model has been used and psychometrically validated in existing studies. These factors directly influence the intention of use, which determines the effective use of the technological system. This model has been widely used in information systems and interactive technologies research. For example, in [105], TAM is used to determine the relationship between gamification features and the satisfaction of intrinsic needs, reaching results concerning gamification’s ability to satisfy needs of competence, autonomy, and relationship. In the case of [106], researchers used TAM to study how social influence affects the intention to use a gamified exercise application; the results showed that social influence positively affects perceived usefulness and intention of use. The authors of [107] studied how different game design elements (dynamics) impact user engagement in gamified systems; in addition, they used an adaptation of the TAM model, determining as a result that elements such as points, levels, and leaderboards were positively correlated with the predictors and therefore increased the intention of use. The questions associated with TAM that were used in our study had a binary response (yes/no) and were:

• Did you like playing a game to practice programming?
• Did you find the tool useful to learn programming?
• Could you use the tool correctly?
• Did you feel satisfied with your results with the tool?
• Did you feel comfortable with the tool?

3.5. Software Tool

Students in both the control and experimental groups used the EducaPlay learning support tool (https://es.educaplay.com, accessed on 17 August 2024) for learning. Both the pre-test and post-test were physically taken in person. Each user created a username and password using their university access through the Google Suite platform. The activities used were restricted to the following:

• Word wheel: The objective is for the student to provide answers related to specific vocabulary associated with a letter of the alphabet, for example: “Begins with the letter C and corresponds to a procedure to transform a primitive variable from one type to another”. The wheel is completed as the student answers the questions.
• Matching columns: In this case, definitions or questions are matched with their respective solution or answers. For example: “Java reserved word that allows declaring an integer number value” is compared with “int”.
• Tests: This is the most conventional activity used, but it allowed us greater versatility in questions, such as detecting code errors and definition of calculations, among other benefits. The questions and four correct alternative answers were posed, from which the student had to select one. An example question was: “What is the correct way to determine the VAT (Value Added Tax) calculation?”; the alternatives presented were “(a) int vat = sales * 0.19; (b) int vat = sales * 0.19; (c) double vat = sales * 0.19; (d) double vat = (int) sales * 0.19”.

During moment 4, students accessed the EducaPlay platform online from anywhere in their free time. The number of attempts was unlimited. Figure 2, Figure 3 and Figure 4 show examples of the activities performed. Figure 2 depicts the activity of “Guess the Word in Java”, where the student is provided with a description of a Java-specific term along with the word’s initial letter. The student is then required to write the word in the provided space.

Figure 3 illustrates the “Matching Game (Matching columns)” activity. Java terms are presented on the left column, while their corresponding definitions are displayed on the right column. The student is required to match the concepts with their respective definitions.

Figure 4 presents an example of a multiple-choice question. In this question, the student is tasked with determining the correct calculation for a tax value using Java.

The integration functionalities already present in the EducaPlay platform were utilized for the gamification techniques. The ranking used for the experimental group included the student’s name and the accumulated score in the game, which fostered competitiveness and student engagement with the activity. It is important to note that, as it was a cumulative score per game, this ranking added up all attempts made by the student in that type of activity, ensuring that each interaction counted towards their ranking position. For this to be possible, the student belonging to the experimental group had to be logged into the platform during their participation. The visualization of scores and rankings was updated in real time, allowing students to see the impact of their performance on their position immediately.

In contrast, the control group used the game anonymously, meaning no score was accumulated, and no ranking was displayed. This ensured that the control group’s experience differed, avoiding any motivational effect derived from competition.

Although the EducaPlay tool is public, the game designed for this experiment was kept hidden and only accessible through an exclusive link provided to the students. This measure was key to preserving the experiment’s integrity, ensuring that only designated participants could access the activities and that the learning environment was controlled and consistent for all students involved.

4. Results

Data were processed with SPSS software version 29 for the statistical analyses and graphs generation. For the analysis of learning effects, ANOVA and ANCOVA tests were considered. Also, partial eta squared (η²) was used as a measure of effect size, with values of 0.01, 0.06, and 0.14 for small, medium, and large effect sizes, respectively, as suggested by Cohen. For the motivational and ease-of-use perception questionnaire, Chi-square tests were considered with Cramer-V (φ_c) for measuring effect sizes for significant differences. Also, to assess the internal consistency of the perception questionnaire, a Cronbach alpha was calculated, with 0.69 for the questions on motivation and ease-of-use perceptions.

4.1. Learnings

Table 1. Descriptive statistics of pre-test, post-test, and gain per sex and condition.

Dependent Variable: Difference
Sex	Condition	Mean	Std. Deviation	N
Female	Not Gamified	0.27	0.10	16
	Gamified	0.17	0.17	15
	Total	0.22	0.15	31
Male	Not Gamified	0.16	0.17	14
	Gamified	0.23	0.17	17
	Total	0.20	0.17	31
Total	Not Gamified	0.22	0.14	30
	Gamified	0.20	0.17	32
	Total	0.21	0.16	62

shows descriptive statistics for the pre-test, post-test, and learning gains across sex and experimental conditions. To assess the learning gains per sex and condition (gamification use or non-use), a two-way ANCOVA was performed using the pre-test as a covariate. Levene’s test was F(3, 58) = 1.57, p = 0.233, indicating that the error variance of learning gains (dependent) was equal across groups. Also, an F test was taken to check for heteroskedasticity, with F(1, 60) = 1.45, p = 0.539, indicating that error variances did not depend on independent variable values.

The ANCOVA results showed non-significant differences of sex (F(1, 57) = 0.29, p = 0.588) and experimental condition (F(1, 57) = 0.45, p = 0.503) for learning gains. However, a significant two-way interaction resulted in F(1, 57) = 5.45, p = 0.023, η² = 0.09, and observed power of 0.63, indicating possible differences in learning gains when considering sex and condition together.

The results of the ANCOVA analysis for the pre-tests indicate that no significant differences were found in the scores between the groups (F(1, 57) = 0.0001, p = 0.991). In addition, when performing an ANOVA analysis, it was confirmed that there are no significant differences between the pre-test scores of both groups (F(1, 57) = 0.0001, p = 0.991).

Analysis of the main effect for gender across conditions showed females learning significantly more than males in the non-gamified condition, with F(1, 57) = 4.02, p = 0.050, η² = 0.07, and males learning more than females in the gamified condition, although this was not statistically significant (p = 0.203). Next, analysis of the main effect for condition among gender showed that for males, there were non-significant differences in learning gains across experimental conditions (p = 0.246), while for females, learning gains were significantly higher in the non-gamified version than in the gamified version, with F(1, 57) = 4.55, p = 0.037, η² = 0.07. Figure 5 illustrates an interaction between gender and the gamification condition on learning gains, where women achieve higher results in the non-gamified condition, while men demonstrate greater learning gains in the gamified condition.

4.2. Motivational Perceptions

First, a contingency table with the frequencies of motivational questions per sex and condition is provided (see

Table 2. Contingency table of question: Did you enjoy using a game to practice programming?

Condition			No		Yes		Total
			N	%	N	%	N	%
Not Gamified	Sex	Female	5	50.00%	11	55.00%	16	53.30%
		Male	5	50.00%	9	45.00%	14	46.70%
	Total		10	100.00%	20	100.0%	30	100.00%
Gamified	Sex	Female	7	77.80%	8	34.80%	15	46.90%
		Male	2	22.20%	15	65.20%	17	53.10%
	Total		9	100.00%	23	100.00%	32	100.00%
Total	Sex	Female	12	63.20%	19	44.20%	31	50.00%
		Male	7	36.80%	24	55.80%	31	50.00%
	Total		19	100,00%	43	100.00%	62	100.00%

,

Table 3. Contingency table of question: Did you find the tool useful to learn programming?

Condition			No		Yes		Total
			N	%	N	%	N	%
Not Gamified	Sex	Female	5	55.60%	11	52.40%	16	53.30%
		Male	4	44.40%	10	47.60%	14	46.70%
	Total		9	100.00%	21	100.00%	30	100.00%
Gamified	Sex	Female	8	80.00%	7	31.80%	15	46.90%
		Male	2	20.00%	15	68.20%	17	53.10%
	Total		10	100.00%	22	100.00%	32	100.00%
Total	Sex	Female	13	68.40%	18	41.90%	31	50.00%
		Male	6	31.60%	25	58.10%	31	50.00%
	Total		19	100.00%	43	100.00%	62	100.00%

and

Table 4. Contingency table of question: Were you satisfied with your results using the tool?

Condition			No		Yes		Total
			N	%	N	%	N	%
Not Gamified	Sex	Female	0	0.00%	16	55.20%	16	53.30%
		Male	1	100.00%	13	44.80%	14	46.70%
	Total		1	100.00%	29	100.00%	30	100.00%
Gamified	Sex	Female	10	55.60%	5	35.70%	15	46.90%
		Male	8	44.40%	9	64.30%	17	53.10%
	Total		18	100.00%	14	100.00%	32	100.00%
Total	Sex	Female	10	52.60%	21	48.80%	31	50.00%
		Male	9	47.40%	22	51.20%	31	50.00%
	Total		19	100.00%	43	100.00%	62	100.00%

). Three-way Chi-square tests were carried out to evaluate the possible differences of sex and condition on the answers regarding the motivational factors of enjoyment, usefulness, and satisfaction.

In the gamified version, a significantly higher proportion of males reported enjoyment when contrasted against females (χ² (1, N = 32) = 4.80, p = 0.028, φ_c = 0.39). But there was no significant association between sex and enjoyment in the non-gamified condition (χ² (1, N = 30) = 0.06, p = 0.796), nor overall in both conditions (χ² (1, N = 62) = 1.89, p = 0.168). When focusing on the effects of the condition by gender, we observed that there were no significant differences across gamified/non-gamified conditions for males (χ² (1, N = 31) = 2.51, p = 0.112), females (χ² (1, N = 31) = 0.78, p = 0.38), or overall (χ² (1, N = 62) = 0.19, p = 0.66).

When considering subjects’ perceptions of usefulness, in the gamified version, males were more likely than females to find the software tool useful (χ² (1, N = 32) = 6.4, p = 0.011, φ_c = 0.45). On the other hand, there was no significant association between sex and enjoyment in the non-gamified condition (χ² (1, N = 30) = 0.03, p = 0.873), or when allowing for both conditions together (χ² (1, N = 62) = 3.72, p = 0.054). In addition, in the analysis of the condition (gamified/non-gamified) by gender, we observed no significant differences in males (χ² (1, N = 31) = 1.389, p = 0.239), females (χ² (1, N = 31) = 1.55, p = 0.213), and overall (χ² (1, N = 62) = 0.01, p = 0.915).

There were no significant differences by sex in the perception of results satisfaction, either gamified (χ² (1, N = 32) = 1.25, p = 0.265), non-gamified (χ² (1, N = 30) = 1.18, p = 0.277), or overall (χ² (1, N = 62) = 0.08, p = 0.783). However, when focusing on the effects of the condition by gender, we observed that the non-gamified condition presented a significantly higher proportion of subjects reporting as satisfied when compared to the gamified version, for males (χ² (1, N = 31) = 5.98, p = 0.015, φ_c = 0.44), females (χ² (1, N = 31) = 15.75, p < 0.001, φ_c = 0.71), and overall (χ² (1, N = 62) = 24.40, p < 0.001, φ_c = 0.57).

4.3. Ease-of-Use Perceptions

First, a contingency table with the frequencies of ease-of-use questions per sex and condition is provided (see

Table 5. Contingency table of question: Did you use the tool correctly?

Condition			No		Yes		Total
			N	%	N	%	N	%
Not Gamified	Sex	Female	0	0.00%	16	55.20%	16	53.30%
		Male	1	100.00%	13	44.80%	14	46.70%
	Total		1	100.00%	29	100.00%	30	100.00%
Gamified	Sex	Female	0	0.00%	15	46.90%	15	46.90%
		Male	0	0.00%	17	53.10%	17	53.10%
	Total			100.00%	32	100.00%	32	100.00%
Total	Sex	Female	0	0.00%	31	50.80%	31	50.00%
		Male	1	100.00%	30	49.20%	31	50.00%
	Total		1	100.00%	61	100.00%	62	100.00%

and

Table 6. Contingency table of question: Did you feel comfortable with the tool?

Condition			No		Yes		Total
			N	%	N	%	N	%
Not Gamified	Sex	Female	3	75.00%	13	50.00%	16	53.30%
		Male	1	25.00%	13	50.00%	14	46.70%
	Total		4	100.00%	26	100.00%	30	100.00%
Gamified	Sex	Female	8	100.00%	7	29.20%	15	46.90%
		Male	0	0.00%	17	70.80%	17	53.10%
	Total		8	100.00%	24	100.00%	32	100.00%
Total	Sex	Female	11	91.70%	20	40.00%	31	50.00%
		Male	1	8.30%	30	60.00%	31	50.00%
	Total		12	100.00%	50	100.00%	62	100.00%

). Chi-square tests were carried out to evaluate the possible differences of sex and condition in the answers regarding the factors of adequate use and comfort.

Regarding the adequate use of the software tool, only one subject (a male in the non-gamified condition) answered ‘No’ to that question. Therefore, no significant differences were found across gender and condition in general or in particular for any combination of factors.

As for subjects’ perceptions of comfort, in the gamified version, the relation between gender and comfort was significant (χ² (1, N = 32) = 12.09, p < 0.001, φ_c = 0.62), meaning that males felt more comfortable than females in the gamified version. However, there was no significant association between sex and comfort in the non-gamified condition (χ² (1, N = 30) = 0.87, p = 0.351). When considering both conditions together, males also felt comfortable in a higher proportion than females (χ² (1, N = 62) = 10.33, p = 0.001, φ_c = 0.41). In addition, the analysis of the condition (gamified/non-gamified) by gender showed that there were no significant differences in males (χ² (1, N = 31) = 1.26, p = 0.263) and overall (χ² (1, N = 62) = 1.35, p = 0.245). However, females in the non-gamified condition presented a significantly higher proportion of comfort than the females in the gamified version (χ² (1, N = 31) = 4.04, p = 0.044, φ_c = 0.36).

5. Discussion

5.1. Learning

Firstly, it is essential to note that the study’s results demonstrated that women learned significantly more than men in the non-gamified condition, and women in the non-gamified version learned more than those in the gamified version. In keeping with this, various studies have examined gender differences in learning without gamification, suggesting that women tend to outperform men in certain aspects. For example, in the meta-analysis by [78] on academic grades, a small but significant advantage in women’s performance was found in most subjects, including mathematics and sciences. Other researchers point to neurobiological differences between men and women that could influence learning; for instance, the cerebral hemispheres are more interconnected in the female brain, facilitating learning by integrating information across regions [80]. Some studies attribute this phenomenon to women’s tendency to have a greater inclination toward consistent study habits [79]. Additionally, it should be considered in these results that gender stereotypes may have generated anxiety in men about not meeting societal expectations [82].

On the other hand, studies comparing the impact of gamification on learning between men and women are limited and show mixed results. Our study revealed that the incorporation of gamification elements hindered the learning of female participants. Specifically, we found that the women exposed to gamification obtained significantly poorer results than the female control group that received the same instruction without these elements. While men in the gamification group scored higher than their female counterparts, this difference was insignificant. In summary, our findings suggest that gamification may have a counterproductive effect on the academic performance of female students. This aligns with research suggesting greater effectiveness for men in gamified environments, such as the study by [32], which implemented a gamification system in an introductory university programming course, finding a more significant increase in engagement and motivation among male students than their female counterparts. Other research on gender differences in gamification [91] indicates a 5% higher interest in game customization for men than women, where scoreboards are the most attractive gamified component for both genders. They agree that more studies are needed to understand specific gender preferences and strategically adapt gamified elements, maximizing motivation and equal learning.

However, Ref. [7] found no significant gender differences in the motivational effects of gamification in a programming course. The evidence suggests that implementing game elements in introductory programming courses can increase student motivation and engagement, as gamification amplifies the benefits of practice in these courses; gender appears to moderate the effects, being positive for women but not necessarily for men [3]. In [101], it was found that participating in gamified programs improves self-confidence in women and older adults more than in men and younger individuals; the researchers concluded that gamification design should consider these demographic differences. In [94], it is revealed that implementing badge-based gamification in higher education improved content acquisition and increased the perception of gender equity among students, with a more notable increase in participation and perceived learning among women than men. Hence, more studies were needed to understand under what conditions and why gamification use benefits certain groups more and how to optimize its equitable impact through motivational and psychological adaptations based on gender and age. The evidence on the differential gender impact of gamification for learning is still limited and contradictory. It likely depends on specific instructional design, content, sociocultural context, and individual student differences. More rigorous studies are needed in this regard, carefully adapting gamification strategies to optimize motivation and engagement for both men and women.

Finally, in response to RQ1, the study revealed significant gender differences in learning outcomes when applying gamification techniques. Females in the non-gamified condition demonstrated notably higher learning gains compared to their counterparts in the gamified condition. On the other hand, males showed a slight, albeit not statistically significant, improvement in learning gains in the gamified condition. These findings suggest that gamification may enhance learning outcomes for males yet could potentially impede the learning process for females. This highlights the critical need to consider gender-specific effects when implementing gamified educational tools.

5.2. Motivational Perceptions

Studies indicate that rewards, absorption, and autonomy in gamification positively enhance users’ enjoyment and help meet their psychological needs. In this regard, our results showed that in the gamified version, women reported lower enjoyment than men. However, there was no significant association between gender and enjoyment in the non-gamified condition. Additionally, there were no significant differences across gamified and non-gamified conditions in men. The results in the literature regarding enjoyment, however, are mixed. For instance, in the study [7], results on computer programming learning showed no significant differences in enjoyment between men and women.

In contrast, the study by [36], implementing a gamification system in a statistics course, found that enjoyment was 7% higher in male students than females. Men have a more pronounced orientation towards rewards, and competition could increase their enjoyment when these dynamics are present in gamified design [20]. Gender stereotypes associated with games and technology also play a role [100].

This can be explained by men preferring action, strategy, and sports games, while women favor puzzles, word games, and social simulations. Software design may be biased towards the former. Additionally, men generally have more experience in digital games, facilitating engagement with gamified mechanics. Gender gaps in digital skills applied to software use may reduce perceived ease of use and fun in women. Different motivational patterns, such as women being more oriented towards intrinsic goals compared to extrinsic goals in men, could be a key factor in enjoyment.

The current evidence on gender differences in enjoyment of gamified educational systems is limited and contradictory. It depends heavily on specific instructional design, content, and context. Therefore, more studies controlling for confounding variables and utilizing robust methodologies are needed before concluding that there is an inherent difference between men and women in the enjoyment of learning through gamification.

Regarding usefulness, our study resulted in more women in the gamified version finding the software tool useless than men. This aligns with other studies that have found gender differences in the perception of the utility of gamified educational software. For instance, Ref. [1] measured the perceived utility of a gamified platform for learning programming in university students, finding utility scores were 6% higher on average for men than for women. However, our results contradict those indicated by [7], which found no significant gender differences in perceived utility scores of gamified software for learning programming. Also, Ref. [42] did not identify conclusive differences between men and women in the perception of utility. This may be due to the software design ensuring that it is equally appealing and usable for both genders, incorporating effective gamification and pedagogy elements to engage both men and women equally. The mixed results could also be due to the small or non-representative sample of participants, making it challenging to find statistically significant differences between subgroups. Therefore, it is essential to ensure larger sample sizes. Individual factors such as motivation, learning style, and previous experience with video games or technology may have a more substantial effect than gender. This is compounded by whether the participants’ environment promotes gender equality in technology education and the assessment of educational software, ensuring that there are no ingrained biases. It is crucial to study whether the perceived utility measurement instrument is sensitive enough to detect nuances by gender. More research is needed to investigate and control for variables and interactions that may alter results.

It is also important to note that our study showed no significant differences by gender in the perception of satisfaction in both groups. Studies including this variable in gamification evaluations are scarce and inconclusive, suggesting no gender differences in satisfaction with gamification use, consistent with our results. Two studies supporting this are [7], which found no significant differences between men and women in satisfaction, and [1], both in gamification for learning computer programming. A possible justification for these results is that the design and content of educational software, with or without gamification, are equally oriented to meet the needs and expectations of both men and women. Additionally, satisfaction strongly depends on individual factors such as motivation, prior digital competence, and self-efficacy in technology use. Therefore, focusing on the overall learning experience of the individual student rather than generalities about men and women allows the development of educational software that is equally satisfying from a gender perspective.

However, in our study, by focusing on the effects of gender conditions, we found that both men and women who used the non-gamified software tool reported greater satisfaction than those who used the gamified version. This is because the task’s difficulty (in this case, programming) generated greater compliance from the subjects regarding the results obtained in the non-gamified version. Without elements of competition or external rewards, satisfaction is derived mainly from the ability to complete the task. This contradicts the limited existing evidence. For example, in [70], there was slightly higher, although not statistically significant, satisfaction among men than women in a gamified algorithm learning environment. This could be attributed to social and narrative drivers in the gamified design, which could increase women’s satisfaction, while competition could satisfy men more [42]. In summary, there is no conclusive evidence on whether men or women inherently experience greater satisfaction with gamification. As with the previous variables, it depends mainly on the specific instructional design and how it caters to the interests and motivations of each gender. More in-depth studies analyzing the behavior of these variables are needed.

In response to RQ2, the study found distinct gender differences in motivation levels when applying gamification. Specifically, males reported significantly higher enjoyment and perceived usefulness levels in the gamified condition than females. Conversely, females reported greater satisfaction in the non-gamified condition, indicating that the gamified environment may not align as effectively with their motivational needs. These findings suggest that while gamification can enhance motivation for males, it may not have the same effect for females, emphasizing the need for a more tailored approach when implementing gamified educational tools.

5.3. Perceptions of Ease of Use

Our study found no significant differences between genders and groups (with and without gamification) in the perception of adequate use of the EducaPlay tool. On the one hand, this supports the idea that the software used and its activities were suitable, and there were no significant issues of interaction, usability, or others that could have affected the results. This aligns with some studies supporting the absence of significant gender differences in the perception of appropriate use of gamified educational software and that gender does not determine ease of use [96,99], as seen in the case of [92], where research on educational video games for primary school science learning found that a simple interface, clear objectives, appropriate progression of challenges, and familiar contexts were crucial for a high perception of use equally among boys and girls. Similarly, Ref. [46], a study on digital games in higher education, determined that differences in the perception of use were more related to the individual experience and motivation of the students. Therefore, it can be argued that, probably through a user-centric design experience, the perception of appropriate use of gamified educational software can be promoted without gender distinction, leveling the playing field for both.

Another aspect we observed was that the results showed that men felt more comfortable than women in using the gamified tool. This aligns with studies suggesting that men might feel slightly more comfortable with gamification, as reported by [43], which indicated a higher initial cheerful disposition of male students toward a gamified system, although this difference disappeared over time. Additionally, some authors argue that gender stereotypes associated with interactive software or video games could generate more initial apprehension in women, creating the previously studied effect [100].

Another finding in the same line was that women who did not use the gamified tool had a significantly higher proportion of comfort compared to female students who did use gamification. The higher level of comfort experienced by women in the non-gamified version compared to the gamified one explains, at least partially, the disparities in learning between women with and without gamification. In other words, women under gamification learned less and reported feeling less comfortable than those without it. There are studies indicating specific gender differences in attitudes and perceptions toward gamification use, with women reporting less comfort or initial reluctance. However, there are nuances in the results obtained, such as in the case of [85], which showed that women initially had more negative attitudes than men toward a gamified educational system. Still, subsequent research by the same authors failed to replicate this gender difference [43]. Under this condition, Ref. [20] argued that stereotypes associated with software tools could generate more initial skepticism in women, affecting the comfort of use. However, this effect is mitigated with careful implementation of gamification. Therefore, it becomes essential that the preferences and motivations of each gender should be considered in instructional design [42].

In response to RQ3, the study identified significant gender differences in ease-of-use levels when applying gamification. Males reported a higher level of comfort with the gamified tool than females, who indicated greater comfort in the non-gamified condition. Interestingly, no significant gender differences were observed regarding the correct tool usage, suggesting that both genders were equally capable of using the software correctly. However, the disparity in comfort levels highlights the potential need to adjust gamified tools to better cater to female users, ensuring a more inclusive and comfortable learning environment for all students.

5.4. Limitations

This study presents several limitations. First, the sample size, consisting of approximately 60 subjects, may not provide sufficient statistical power to detect small effects, and the duration of the experiment may not be adequate to generalize the results to other populations or educational contexts. Additionally, the study was conducted at a university in Chile, which limits the generalizability of the findings to other cultural and educational settings. Variables such as motivation and perceived usefulness were measured through self-reported questionnaires, which could introduce response biases.

Another limitation is the lack of data regarding students’ familiarity with video games, which could have influenced their perception of the tools used and, consequently, the results obtained. For future studies, it is recommended to include an initial assessment of prior experience with video games to control for this potential confounding factor.

Moreover, the random assignment of students to groups, although a common practice, did not account for individual characteristics such as programming experience or initial motivation, which prevents confirming the statistical equivalence of the groups. This could influence the results; thus, future studies should consider a prior analysis of these variables.

6. Conclusions

In this study, we addressed the challenge that learning programming is complicated for students due to deficiencies in problem-solving skills and an excessive focus on language rather than problem-solving through program design. Previous experiences have shown positive but inconclusive impacts of gamification in programming courses, generating effects such as increased motivation, engagement, and student performance. However, these effects vary based on the intervention duration and prior knowledge of the subjects.

Furthermore, we aimed to answer questions associated with whether gamification led to differences in learning by gender, motivation levels in terms of enjoyment, utility, and satisfaction, and levels of ease of use concerning adequate use and comfort. To address these questions, we designed an experiment using the EducaPlay tool to reinforce programming concepts. The gamified version included ranking and scoring elements, while the non-gamified version did not.

In this experimental study, 88 students in a programming course participated, randomly assigned to control and experimental groups. The experimental group used a gamified platform. Initially, a pre-test was administered to all students (both groups), and a post-test was conducted at the end, along with a questionnaire measuring the usage of perception variables, among others. The evaluation included learning, motivational perceptions, and ease of use among men and women.

The results revealed that women learned significantly more than men in the non-gamified group and more than in the gamified group. Although men in the gamified group performed better than those in the non-gamified group, this difference was not statistically significant. These results suggest that gamification’s competitive elements and extrinsic rewards may favor men’s learning while women benefit more from traditional teaching methods without gamification.

To design effective gamified educational tools, it is crucial to consider gender differences in preferences and motivations. Women may need more support and resources to develop digital skills and benefit from teaching methods incorporating intrinsic motivators and less competitive elements. These differences may be attributed to psychological and sociocultural factors. Future studies should explore how to tailor gamification to optimize educational benefits for both genders, taking into account diverse motivational and design approaches.

Regarding gender differences in motivation levels when applying gamification, women reported lower perceptions of enjoyment and usefulness than men. This can be attributed to the orientation of gamification techniques towards reward and competition, as men may experience a greater enjoyment of gamified educational systems incorporating reward and competitive elements than women. This aligns with gender stereotypes associated with games and technology, as men adapt more readily to these tools. To design effective gamified educational tools, it is crucial to consider gender differences in preferences and motivations. Women may need more support and resources to develop digital skills and benefit from teaching methods incorporating intrinsic motivators and less competitive elements. Furthermore, different motivational patterns should be considered, such as intrinsic goals in women and extrinsic goals in men. Thus, gamification design should incorporate reward and competitive elements attractive to both genders, provide support and resources to help women develop their digital skills, and utilize intrinsic motivation elements that appeal to both genders.

Finally, regarding gender differences in ease of use when applying gamification, no significant differences were found in the perception of adequate use of the EducaPlay tool between genders and groups (with and without gamification). This suggests that the software and its activities were appropriate, and no usability or interaction issues could affect the results. This can be attributed to EducaPlay being designed for educational gamification and free of performance, access, and usage issues which have been reported in other studies that did not experience significant gamification benefits. Additionally, the results show that women felt less comfortable than men with the gamified tool. This aligns with previous studies indicating that men may initially feel more comfortable with gamification. Therefore, further studies with gamification techniques focused on motivational aspects aligned with social work and low competitiveness are needed. These would be more suitable for female students. As a recommendation, when designing an educational tool with gamification, consideration should be given to the preferences and motivations of each gender in the instructional design of gamified educational tools to mitigate potential differentiated effects. This can be achieved through questionnaires or interviews with students to understand their interests and learning styles, and which types of rewards or game mechanics are most motivating for them; by offering customization options and flexibility in the tool for each student to adapt aspects such as their profile, progression, or types of challenges according to their preferences; or using inclusive themes, narratives, and images that do not reinforce harmful gender stereotypes in learning content and rewards, among other measures.

In summary, women found gamification less enjoyable, comfortable, and useful than men. Additionally, women felt more comfortable without gamification than with it, making them learn less under gamification. These results attempt to explain motivational and usage factors for the gender-based differences in learning. Particularly, they shed light on the reasons behind the negative outcomes for women under gamification. Therefore, more rigorous studies are needed on the differential impact of gamification on the learning and motivation of men and women, controlling for confounding variables and employing solid experimental designs.