1. Introduction
Gamification, defined as integrating game-like elements into non-game contexts, has emerged as a promising strategy to enhance motivation and facilitate learning in various educational settings [
1,
2]. However, while gamification has proven effective in increasing student engagement, its impact may not be uniform across all groups, particularly when considering gender as a variable of interest. Recent studies have highlighted significant differences in how men and women respond to gamification, suggesting that these dynamics could differentially influence educational outcomes [
3].
Gender differences in learning and motivation are particularly pronounced in computer programming. Women tend to adopt more versatile approaches to information processing and skill acquisition, while men often show a greater affinity for competitive structures common in gamification [
4]. However, research on how gamification could influence the gender gap in programming skill acquisition is limited and often contradictory. Some studies suggest that gamification benefits both genders equally [
5,
6], while others indicate that it may exacerbate inequalities, favoring men more significantly [
7,
8].
Numerous studies have focused on determining how gamification affects student motivation and engagement [
9,
10,
11,
12]; however, there remains a gap in longitudinal studies that assess the impact of gamification on learning [
13,
14,
15], particularly in programming [
16]. The current evidence is insufficient to generalize the results due to the diversity of protocols and assessment tools [
17].
This experimental study aims to analyze how gamification affects gender differences in programming skill acquisition. An experiment was conducted with university students in Chile, randomly assigned to gamified and non-gamified groups. Learning outcomes were assessed through pre- and post-tests, while motivational factors and usability perceptions were evaluated using an adaptation of the Technology Acceptance Model (TAM) questionnaire. The findings reveal that women learn significantly more in the non-gamified condition, while men moderately benefit from gamification. These results underscore the importance of considering gender dynamics when implementing gamified approaches in programming education.
1.1. Research Questions
RQ1: Are there learning differences by gender when applying gamification techniques?
RQ2: Are there gender differences in motivation levels when applying gamification in terms of enjoyment (a), usefulness (b), and satisfaction (c)?
RQ3: Are there gender differences in ease-of-use levels when applying gamification regarding correct usage (a) and comfort (b)?
1.2. Document Structure
The document is structured into several key sections that address the study of the impact of gamification in programming education, with a particular focus on gender differences. It begins with an abstract that presents the objectives and main findings. This is followed by an introduction that contextualizes the topic and highlights the importance of gamification in education. Next, the literature review provides a detailed analysis of previous studies on gamification, its applications in education, and gender differences in learning. The experimental design section then describes the methodology, study subjects, and measurement instruments. The results are presented in detail, accompanied by statistical analyses highlighting differences in learning outcomes and motivation between males and females under gamified and non-gamified conditions. Finally, the document concludes with discussions and conclusions that interpret the findings in the context of the study, suggesting implications for educational practice and possible directions for future research.
3. Experimental Design
The study presented in this research is designed at an experimental level using a quantitative research methodology similar to that used in [
64]; for this purpose, a control group and an experimental group are considered, with both groups undergoing a pre-test and a post-test.
3.1. Test Subjects
The participants in this study are students in a digital literacy course taught in the third semester (second year) of the Accounting Auditor program at a university in Chile. This is the first time the students face a course that teaches them programming, Microsoft Excel, and algorithms.
The study was conducted in the first semester of 2022; the total sample initially consisted of 81 students, but 19 students did not complete the activity or did not want to participate in it, leaving a final total of 62 students in the sample, 31 men and 31 women, divided into experimental and control groups. The assignment of students to each group was random. The control group consisted of 30 students, and the experimental group 32. Both groups had an initial class where the content to be covered was explained.
3.2. Contents
The Information Systems 1 course has a workload of 4 pedagogical hours per week, divided into 2 h of theory and 2 h of practice. The experiment was conducted over two weeks, totaling 8 h of class. A total of 3 h was used for initial instruction on the content to be covered, and the remaining time was used to practice using the tools. The learning outcomes addressed with this experiment were the following:
LO1: Students recognize the technical terminology of the Java language to program algorithmic solutions to fundamental mathematical problems.
LO2: Students understand using control and iterative structures in Java to program algorithmic solutions to fundamental business problems.
LO3: Students integrate one-dimensional arrays into algorithmic solutions in Java to provide solutions to fundamental business problems.
The main business problems to be addressed had to do with content related to the student’s professional work, such as calculating sales amounts, taxes, and inventories, among others.
The previous learning outcomes are contained in the second learning unit of the course, which proposes that students learn to build basic interactive applications in Java that they can then apply to business problems.
Using Bloom’s and Anderson’s taxonomies [
102,
103], we indicate that LO1 is of a lower order since it requires the use of memory (level 1: remember); LO2 is taxonomically classified as medium-low since it requires the development of the skill of constructing meaning from educational material, such as reading or teacher explanations (level 2: understand); for LO3 we assign a higher-order taxonomic complexity classification since it requires the ability to use past ideas to create new ones, generalize from supplied data, relate knowledge, and predict derived conclusions (level 5: synthesize).
3.3. Process
The process on which the experiment was based is shown in
Figure 1, and the details are:
At moment 1, the material associated with the content is available in Moodle five days before moment 2.
At moment 2, a face-to-face session is held to provide an introductory overview of the content available and resolve students’ conceptual doubts. At this moment, the content is not explained, nor are the students taught the content related to the study. All students (experimental and control groups) participated in this activity, including those who later decided not to continue studying.
At moment 3, a pre-test is administered to both groups, with 21 questions divided into seven for each learning objective. The test applied was the same for the entire sample of students; the time available to answer the test was 30 min. In addition, an acceptance test based on TAM (Technology Acceptance Model) is taken.
At moment 4, students must generate a list of questions and answers to be applied in a “Rosco” (wheel) format. The students develop these questions collaboratively in an autonomous flipped class modality. The teacher later consolidates and reviews this list. The teacher corrects the questions, which become part of the question bank for the gamified activity.
At moment 5, the students are divided into experimental and control groups, and use the tool according to their assigned group; the time allotted for this stage is fourteen days, during which students can freely access the activity according to their frequency and schedule.
At moment 6, a test is conducted again with the same number of questions, assigned time, and difficulty as the initial test. In addition, an acceptance test based on TAM is taken.
It is important to note that students do not know about the division between a control group and an experimental group, since the division of the groups is generated using Moodle’s “groups” tool. The subjects (students from both groups) were unaware of the difference between the tool with gamification and the one without. Moments 1, 2, 3, and 4 lasted 2 h. Moment 5 covered fourteen days. Moment 6 lasted 30 min; students completed it when they finished time 5. For all students participating in the study and who completed all the stages, a bonus of 0.5 points was given for the formal evaluation of the course.
3.4. Instruments
To evaluate the achievement of learning outcomes, the gamified tool had an extensive database of isomorphic questions, with around 100 problems for each expected learning outcome. It is important to note that the experimental and control groups responded to the same pre- and post-test questions designed to assess the same learning outcomes. Although specific questions for each test were randomly selected from a larger pool, all questions covered the same learning objectives and had an equivalent difficulty level.
The quiz questions were randomly selected from a pre-developed question bank. This bank contained equivalent questions in terms of content and difficulty, all aligned with the defined learning outcomes. However, within each group (control and experimental), the questions were identical, ensuring that differences in results were not due to variations in the tests.
Additionally, the problems each student had to solve during the activities were randomly drawn from a predefined set, ensuring that all activities maintained the same consistency regarding challenge and content. This allowed for measuring the intervention’s impact without introducing biases about the difficulty or type of problems presented.
Furthermore, to explain the determinants of users’ technological acceptance, five questions based on the Technology Acceptance Model (TAM) [
104] were applied to provide a vision of the degree to which students believed that using the learning tool would improve their performance and productivity in addition to determining whether it would require little or no effort. This model has been used and psychometrically validated in existing studies. These factors directly influence the intention of use, which determines the effective use of the technological system. This model has been widely used in information systems and interactive technologies research. For example, in [
105], TAM is used to determine the relationship between gamification features and the satisfaction of intrinsic needs, reaching results concerning gamification’s ability to satisfy needs of competence, autonomy, and relationship. In the case of [
106], researchers used TAM to study how social influence affects the intention to use a gamified exercise application; the results showed that social influence positively affects perceived usefulness and intention of use. The authors of [
107] studied how different game design elements (dynamics) impact user engagement in gamified systems; in addition, they used an adaptation of the TAM model, determining as a result that elements such as points, levels, and leaderboards were positively correlated with the predictors and therefore increased the intention of use. The questions associated with TAM that were used in our study had a binary response (yes/no) and were:
Did you like playing a game to practice programming?
Did you find the tool useful to learn programming?
Could you use the tool correctly?
Did you feel satisfied with your results with the tool?
Did you feel comfortable with the tool?
3.5. Software Tool
Students in both the control and experimental groups used the EducaPlay learning support tool (
https://es.educaplay.com, accessed on 17 August 2024) for learning. Both the pre-test and post-test were physically taken in person. Each user created a username and password using their university access through the Google Suite platform. The activities used were restricted to the following:
Word wheel: The objective is for the student to provide answers related to specific vocabulary associated with a letter of the alphabet, for example: “Begins with the letter C and corresponds to a procedure to transform a primitive variable from one type to another”. The wheel is completed as the student answers the questions.
Matching columns: In this case, definitions or questions are matched with their respective solution or answers. For example: “Java reserved word that allows declaring an integer number value” is compared with “int”.
Tests: This is the most conventional activity used, but it allowed us greater versatility in questions, such as detecting code errors and definition of calculations, among other benefits. The questions and four correct alternative answers were posed, from which the student had to select one. An example question was: “What is the correct way to determine the VAT (Value Added Tax) calculation?”; the alternatives presented were “(a) int vat = sales * 0.19; (b) int vat = sales * 0.19; (c) double vat = sales * 0.19; (d) double vat = (int) sales * 0.19”.
During moment 4, students accessed the EducaPlay platform online from anywhere in their free time. The number of attempts was unlimited.
Figure 2,
Figure 3 and
Figure 4 show examples of the activities performed.
Figure 2 depicts the activity of “Guess the Word in Java”, where the student is provided with a description of a Java-specific term along with the word’s initial letter. The student is then required to write the word in the provided space.
Figure 3 illustrates the “Matching Game (Matching columns)” activity. Java terms are presented on the left column, while their corresponding definitions are displayed on the right column. The student is required to match the concepts with their respective definitions.
Figure 4 presents an example of a multiple-choice question. In this question, the student is tasked with determining the correct calculation for a tax value using Java.
The integration functionalities already present in the EducaPlay platform were utilized for the gamification techniques. The ranking used for the experimental group included the student’s name and the accumulated score in the game, which fostered competitiveness and student engagement with the activity. It is important to note that, as it was a cumulative score per game, this ranking added up all attempts made by the student in that type of activity, ensuring that each interaction counted towards their ranking position. For this to be possible, the student belonging to the experimental group had to be logged into the platform during their participation. The visualization of scores and rankings was updated in real time, allowing students to see the impact of their performance on their position immediately.
In contrast, the control group used the game anonymously, meaning no score was accumulated, and no ranking was displayed. This ensured that the control group’s experience differed, avoiding any motivational effect derived from competition.
Although the EducaPlay tool is public, the game designed for this experiment was kept hidden and only accessible through an exclusive link provided to the students. This measure was key to preserving the experiment’s integrity, ensuring that only designated participants could access the activities and that the learning environment was controlled and consistent for all students involved.
4. Results
Data were processed with SPSS software version 29 for the statistical analyses and graphs generation. For the analysis of learning effects, ANOVA and ANCOVA tests were considered. Also, partial eta squared (η2) was used as a measure of effect size, with values of 0.01, 0.06, and 0.14 for small, medium, and large effect sizes, respectively, as suggested by Cohen. For the motivational and ease-of-use perception questionnaire, Chi-square tests were considered with Cramer-V (φc) for measuring effect sizes for significant differences. Also, to assess the internal consistency of the perception questionnaire, a Cronbach alpha was calculated, with 0.69 for the questions on motivation and ease-of-use perceptions.
4.1. Learnings
Table 1 shows descriptive statistics for the pre-test, post-test, and learning gains across sex and experimental conditions. To assess the learning gains per sex and condition (gamification use or non-use), a two-way ANCOVA was performed using the pre-test as a covariate. Levene’s test was F(3, 58) = 1.57,
p = 0.233, indicating that the error variance of learning gains (dependent) was equal across groups. Also, an F test was taken to check for heteroskedasticity, with F(1, 60) = 1.45,
p = 0.539, indicating that error variances did not depend on independent variable values.
The ANCOVA results showed non-significant differences of sex (F(1, 57) = 0.29, p = 0.588) and experimental condition (F(1, 57) = 0.45, p = 0.503) for learning gains. However, a significant two-way interaction resulted in F(1, 57) = 5.45, p = 0.023, η2 = 0.09, and observed power of 0.63, indicating possible differences in learning gains when considering sex and condition together.
The results of the ANCOVA analysis for the pre-tests indicate that no significant differences were found in the scores between the groups (F(1, 57) = 0.0001, p = 0.991). In addition, when performing an ANOVA analysis, it was confirmed that there are no significant differences between the pre-test scores of both groups (F(1, 57) = 0.0001, p = 0.991).
Analysis of the main effect for gender across conditions showed females learning significantly more than males in the non-gamified condition, with F(1, 57) = 4.02,
p = 0.050, η
2 = 0.07, and males learning more than females in the gamified condition, although this was not statistically significant (
p = 0.203). Next, analysis of the main effect for condition among gender showed that for males, there were non-significant differences in learning gains across experimental conditions (
p = 0.246), while for females, learning gains were significantly higher in the non-gamified version than in the gamified version, with F(1, 57) = 4.55,
p = 0.037, η
2 = 0.07.
Figure 5 illustrates an interaction between gender and the gamification condition on learning gains, where women achieve higher results in the non-gamified condition, while men demonstrate greater learning gains in the gamified condition.
4.2. Motivational Perceptions
First, a contingency table with the frequencies of motivational questions per sex and condition is provided (see
Table 2,
Table 3 and
Table 4). Three-way Chi-square tests were carried out to evaluate the possible differences of sex and condition on the answers regarding the motivational factors of enjoyment, usefulness, and satisfaction.
In the gamified version, a significantly higher proportion of males reported enjoyment when contrasted against females (χ2 (1, N = 32) = 4.80, p = 0.028, φc = 0.39). But there was no significant association between sex and enjoyment in the non-gamified condition (χ2 (1, N = 30) = 0.06, p = 0.796), nor overall in both conditions (χ2 (1, N = 62) = 1.89, p = 0.168). When focusing on the effects of the condition by gender, we observed that there were no significant differences across gamified/non-gamified conditions for males (χ2 (1, N = 31) = 2.51, p = 0.112), females (χ2 (1, N = 31) = 0.78, p = 0.38), or overall (χ2 (1, N = 62) = 0.19, p = 0.66).
When considering subjects’ perceptions of usefulness, in the gamified version, males were more likely than females to find the software tool useful (χ2 (1, N = 32) = 6.4, p = 0.011, φc = 0.45). On the other hand, there was no significant association between sex and enjoyment in the non-gamified condition (χ2 (1, N = 30) = 0.03, p = 0.873), or when allowing for both conditions together (χ2 (1, N = 62) = 3.72, p = 0.054). In addition, in the analysis of the condition (gamified/non-gamified) by gender, we observed no significant differences in males (χ2 (1, N = 31) = 1.389, p = 0.239), females (χ2 (1, N = 31) = 1.55, p = 0.213), and overall (χ2 (1, N = 62) = 0.01, p = 0.915).
There were no significant differences by sex in the perception of results satisfaction, either gamified (χ2 (1, N = 32) = 1.25, p = 0.265), non-gamified (χ2 (1, N = 30) = 1.18, p = 0.277), or overall (χ2 (1, N = 62) = 0.08, p = 0.783). However, when focusing on the effects of the condition by gender, we observed that the non-gamified condition presented a significantly higher proportion of subjects reporting as satisfied when compared to the gamified version, for males (χ2 (1, N = 31) = 5.98, p = 0.015, φc = 0.44), females (χ2 (1, N = 31) = 15.75, p < 0.001, φc = 0.71), and overall (χ2 (1, N = 62) = 24.40, p < 0.001, φc = 0.57).
4.3. Ease-of-Use Perceptions
First, a contingency table with the frequencies of ease-of-use questions per sex and condition is provided (see
Table 5 and
Table 6). Chi-square tests were carried out to evaluate the possible differences of sex and condition in the answers regarding the factors of adequate use and comfort.
Regarding the adequate use of the software tool, only one subject (a male in the non-gamified condition) answered ‘No’ to that question. Therefore, no significant differences were found across gender and condition in general or in particular for any combination of factors.
As for subjects’ perceptions of comfort, in the gamified version, the relation between gender and comfort was significant (χ2 (1, N = 32) = 12.09, p < 0.001, φc = 0.62), meaning that males felt more comfortable than females in the gamified version. However, there was no significant association between sex and comfort in the non-gamified condition (χ2 (1, N = 30) = 0.87, p = 0.351). When considering both conditions together, males also felt comfortable in a higher proportion than females (χ2 (1, N = 62) = 10.33, p = 0.001, φc = 0.41). In addition, the analysis of the condition (gamified/non-gamified) by gender showed that there were no significant differences in males (χ2 (1, N = 31) = 1.26, p = 0.263) and overall (χ2 (1, N = 62) = 1.35, p = 0.245). However, females in the non-gamified condition presented a significantly higher proportion of comfort than the females in the gamified version (χ2 (1, N = 31) = 4.04, p = 0.044, φc = 0.36).
5. Discussion
5.1. Learning
Firstly, it is essential to note that the study’s results demonstrated that women learned significantly more than men in the non-gamified condition, and women in the non-gamified version learned more than those in the gamified version. In keeping with this, various studies have examined gender differences in learning without gamification, suggesting that women tend to outperform men in certain aspects. For example, in the meta-analysis by [
78] on academic grades, a small but significant advantage in women’s performance was found in most subjects, including mathematics and sciences. Other researchers point to neurobiological differences between men and women that could influence learning; for instance, the cerebral hemispheres are more interconnected in the female brain, facilitating learning by integrating information across regions [
80]. Some studies attribute this phenomenon to women’s tendency to have a greater inclination toward consistent study habits [
79]. Additionally, it should be considered in these results that gender stereotypes may have generated anxiety in men about not meeting societal expectations [
82].
On the other hand, studies comparing the impact of gamification on learning between men and women are limited and show mixed results. Our study revealed that the incorporation of gamification elements hindered the learning of female participants. Specifically, we found that the women exposed to gamification obtained significantly poorer results than the female control group that received the same instruction without these elements. While men in the gamification group scored higher than their female counterparts, this difference was insignificant. In summary, our findings suggest that gamification may have a counterproductive effect on the academic performance of female students. This aligns with research suggesting greater effectiveness for men in gamified environments, such as the study by [
32], which implemented a gamification system in an introductory university programming course, finding a more significant increase in engagement and motivation among male students than their female counterparts. Other research on gender differences in gamification [
91] indicates a 5% higher interest in game customization for men than women, where scoreboards are the most attractive gamified component for both genders. They agree that more studies are needed to understand specific gender preferences and strategically adapt gamified elements, maximizing motivation and equal learning.
However, Ref. [
7] found no significant gender differences in the motivational effects of gamification in a programming course. The evidence suggests that implementing game elements in introductory programming courses can increase student motivation and engagement, as gamification amplifies the benefits of practice in these courses; gender appears to moderate the effects, being positive for women but not necessarily for men [
3]. In [
101], it was found that participating in gamified programs improves self-confidence in women and older adults more than in men and younger individuals; the researchers concluded that gamification design should consider these demographic differences. In [
94], it is revealed that implementing badge-based gamification in higher education improved content acquisition and increased the perception of gender equity among students, with a more notable increase in participation and perceived learning among women than men. Hence, more studies were needed to understand under what conditions and why gamification use benefits certain groups more and how to optimize its equitable impact through motivational and psychological adaptations based on gender and age. The evidence on the differential gender impact of gamification for learning is still limited and contradictory. It likely depends on specific instructional design, content, sociocultural context, and individual student differences. More rigorous studies are needed in this regard, carefully adapting gamification strategies to optimize motivation and engagement for both men and women.
Finally, in response to RQ1, the study revealed significant gender differences in learning outcomes when applying gamification techniques. Females in the non-gamified condition demonstrated notably higher learning gains compared to their counterparts in the gamified condition. On the other hand, males showed a slight, albeit not statistically significant, improvement in learning gains in the gamified condition. These findings suggest that gamification may enhance learning outcomes for males yet could potentially impede the learning process for females. This highlights the critical need to consider gender-specific effects when implementing gamified educational tools.
5.2. Motivational Perceptions
Studies indicate that rewards, absorption, and autonomy in gamification positively enhance users’ enjoyment and help meet their psychological needs. In this regard, our results showed that in the gamified version, women reported lower enjoyment than men. However, there was no significant association between gender and enjoyment in the non-gamified condition. Additionally, there were no significant differences across gamified and non-gamified conditions in men. The results in the literature regarding enjoyment, however, are mixed. For instance, in the study [
7], results on computer programming learning showed no significant differences in enjoyment between men and women.
In contrast, the study by [
36], implementing a gamification system in a statistics course, found that enjoyment was 7% higher in male students than females. Men have a more pronounced orientation towards rewards, and competition could increase their enjoyment when these dynamics are present in gamified design [
20]. Gender stereotypes associated with games and technology also play a role [
100].
This can be explained by men preferring action, strategy, and sports games, while women favor puzzles, word games, and social simulations. Software design may be biased towards the former. Additionally, men generally have more experience in digital games, facilitating engagement with gamified mechanics. Gender gaps in digital skills applied to software use may reduce perceived ease of use and fun in women. Different motivational patterns, such as women being more oriented towards intrinsic goals compared to extrinsic goals in men, could be a key factor in enjoyment.
The current evidence on gender differences in enjoyment of gamified educational systems is limited and contradictory. It depends heavily on specific instructional design, content, and context. Therefore, more studies controlling for confounding variables and utilizing robust methodologies are needed before concluding that there is an inherent difference between men and women in the enjoyment of learning through gamification.
Regarding usefulness, our study resulted in more women in the gamified version finding the software tool useless than men. This aligns with other studies that have found gender differences in the perception of the utility of gamified educational software. For instance, Ref. [
1] measured the perceived utility of a gamified platform for learning programming in university students, finding utility scores were 6% higher on average for men than for women. However, our results contradict those indicated by [
7], which found no significant gender differences in perceived utility scores of gamified software for learning programming. Also, Ref. [
42] did not identify conclusive differences between men and women in the perception of utility. This may be due to the software design ensuring that it is equally appealing and usable for both genders, incorporating effective gamification and pedagogy elements to engage both men and women equally. The mixed results could also be due to the small or non-representative sample of participants, making it challenging to find statistically significant differences between subgroups. Therefore, it is essential to ensure larger sample sizes. Individual factors such as motivation, learning style, and previous experience with video games or technology may have a more substantial effect than gender. This is compounded by whether the participants’ environment promotes gender equality in technology education and the assessment of educational software, ensuring that there are no ingrained biases. It is crucial to study whether the perceived utility measurement instrument is sensitive enough to detect nuances by gender. More research is needed to investigate and control for variables and interactions that may alter results.
It is also important to note that our study showed no significant differences by gender in the perception of satisfaction in both groups. Studies including this variable in gamification evaluations are scarce and inconclusive, suggesting no gender differences in satisfaction with gamification use, consistent with our results. Two studies supporting this are [
7], which found no significant differences between men and women in satisfaction, and [
1], both in gamification for learning computer programming. A possible justification for these results is that the design and content of educational software, with or without gamification, are equally oriented to meet the needs and expectations of both men and women. Additionally, satisfaction strongly depends on individual factors such as motivation, prior digital competence, and self-efficacy in technology use. Therefore, focusing on the overall learning experience of the individual student rather than generalities about men and women allows the development of educational software that is equally satisfying from a gender perspective.
However, in our study, by focusing on the effects of gender conditions, we found that both men and women who used the non-gamified software tool reported greater satisfaction than those who used the gamified version. This is because the task’s difficulty (in this case, programming) generated greater compliance from the subjects regarding the results obtained in the non-gamified version. Without elements of competition or external rewards, satisfaction is derived mainly from the ability to complete the task. This contradicts the limited existing evidence. For example, in [
70], there was slightly higher, although not statistically significant, satisfaction among men than women in a gamified algorithm learning environment. This could be attributed to social and narrative drivers in the gamified design, which could increase women’s satisfaction, while competition could satisfy men more [
42]. In summary, there is no conclusive evidence on whether men or women inherently experience greater satisfaction with gamification. As with the previous variables, it depends mainly on the specific instructional design and how it caters to the interests and motivations of each gender. More in-depth studies analyzing the behavior of these variables are needed.
In response to RQ2, the study found distinct gender differences in motivation levels when applying gamification. Specifically, males reported significantly higher enjoyment and perceived usefulness levels in the gamified condition than females. Conversely, females reported greater satisfaction in the non-gamified condition, indicating that the gamified environment may not align as effectively with their motivational needs. These findings suggest that while gamification can enhance motivation for males, it may not have the same effect for females, emphasizing the need for a more tailored approach when implementing gamified educational tools.
5.3. Perceptions of Ease of Use
Our study found no significant differences between genders and groups (with and without gamification) in the perception of adequate use of the EducaPlay tool. On the one hand, this supports the idea that the software used and its activities were suitable, and there were no significant issues of interaction, usability, or others that could have affected the results. This aligns with some studies supporting the absence of significant gender differences in the perception of appropriate use of gamified educational software and that gender does not determine ease of use [
96,
99], as seen in the case of [
92], where research on educational video games for primary school science learning found that a simple interface, clear objectives, appropriate progression of challenges, and familiar contexts were crucial for a high perception of use equally among boys and girls. Similarly, Ref. [
46], a study on digital games in higher education, determined that differences in the perception of use were more related to the individual experience and motivation of the students. Therefore, it can be argued that, probably through a user-centric design experience, the perception of appropriate use of gamified educational software can be promoted without gender distinction, leveling the playing field for both.
Another aspect we observed was that the results showed that men felt more comfortable than women in using the gamified tool. This aligns with studies suggesting that men might feel slightly more comfortable with gamification, as reported by [
43], which indicated a higher initial cheerful disposition of male students toward a gamified system, although this difference disappeared over time. Additionally, some authors argue that gender stereotypes associated with interactive software or video games could generate more initial apprehension in women, creating the previously studied effect [
100].
Another finding in the same line was that women who did not use the gamified tool had a significantly higher proportion of comfort compared to female students who did use gamification. The higher level of comfort experienced by women in the non-gamified version compared to the gamified one explains, at least partially, the disparities in learning between women with and without gamification. In other words, women under gamification learned less and reported feeling less comfortable than those without it. There are studies indicating specific gender differences in attitudes and perceptions toward gamification use, with women reporting less comfort or initial reluctance. However, there are nuances in the results obtained, such as in the case of [
85], which showed that women initially had more negative attitudes than men toward a gamified educational system. Still, subsequent research by the same authors failed to replicate this gender difference [
43]. Under this condition, Ref. [
20] argued that stereotypes associated with software tools could generate more initial skepticism in women, affecting the comfort of use. However, this effect is mitigated with careful implementation of gamification. Therefore, it becomes essential that the preferences and motivations of each gender should be considered in instructional design [
42].
In response to RQ3, the study identified significant gender differences in ease-of-use levels when applying gamification. Males reported a higher level of comfort with the gamified tool than females, who indicated greater comfort in the non-gamified condition. Interestingly, no significant gender differences were observed regarding the correct tool usage, suggesting that both genders were equally capable of using the software correctly. However, the disparity in comfort levels highlights the potential need to adjust gamified tools to better cater to female users, ensuring a more inclusive and comfortable learning environment for all students.
5.4. Limitations
This study presents several limitations. First, the sample size, consisting of approximately 60 subjects, may not provide sufficient statistical power to detect small effects, and the duration of the experiment may not be adequate to generalize the results to other populations or educational contexts. Additionally, the study was conducted at a university in Chile, which limits the generalizability of the findings to other cultural and educational settings. Variables such as motivation and perceived usefulness were measured through self-reported questionnaires, which could introduce response biases.
Another limitation is the lack of data regarding students’ familiarity with video games, which could have influenced their perception of the tools used and, consequently, the results obtained. For future studies, it is recommended to include an initial assessment of prior experience with video games to control for this potential confounding factor.
Moreover, the random assignment of students to groups, although a common practice, did not account for individual characteristics such as programming experience or initial motivation, which prevents confirming the statistical equivalence of the groups. This could influence the results; thus, future studies should consider a prior analysis of these variables.
6. Conclusions
In this study, we addressed the challenge that learning programming is complicated for students due to deficiencies in problem-solving skills and an excessive focus on language rather than problem-solving through program design. Previous experiences have shown positive but inconclusive impacts of gamification in programming courses, generating effects such as increased motivation, engagement, and student performance. However, these effects vary based on the intervention duration and prior knowledge of the subjects.
Furthermore, we aimed to answer questions associated with whether gamification led to differences in learning by gender, motivation levels in terms of enjoyment, utility, and satisfaction, and levels of ease of use concerning adequate use and comfort. To address these questions, we designed an experiment using the EducaPlay tool to reinforce programming concepts. The gamified version included ranking and scoring elements, while the non-gamified version did not.
In this experimental study, 88 students in a programming course participated, randomly assigned to control and experimental groups. The experimental group used a gamified platform. Initially, a pre-test was administered to all students (both groups), and a post-test was conducted at the end, along with a questionnaire measuring the usage of perception variables, among others. The evaluation included learning, motivational perceptions, and ease of use among men and women.
The results revealed that women learned significantly more than men in the non-gamified group and more than in the gamified group. Although men in the gamified group performed better than those in the non-gamified group, this difference was not statistically significant. These results suggest that gamification’s competitive elements and extrinsic rewards may favor men’s learning while women benefit more from traditional teaching methods without gamification.
To design effective gamified educational tools, it is crucial to consider gender differences in preferences and motivations. Women may need more support and resources to develop digital skills and benefit from teaching methods incorporating intrinsic motivators and less competitive elements. These differences may be attributed to psychological and sociocultural factors. Future studies should explore how to tailor gamification to optimize educational benefits for both genders, taking into account diverse motivational and design approaches.
Regarding gender differences in motivation levels when applying gamification, women reported lower perceptions of enjoyment and usefulness than men. This can be attributed to the orientation of gamification techniques towards reward and competition, as men may experience a greater enjoyment of gamified educational systems incorporating reward and competitive elements than women. This aligns with gender stereotypes associated with games and technology, as men adapt more readily to these tools. To design effective gamified educational tools, it is crucial to consider gender differences in preferences and motivations. Women may need more support and resources to develop digital skills and benefit from teaching methods incorporating intrinsic motivators and less competitive elements. Furthermore, different motivational patterns should be considered, such as intrinsic goals in women and extrinsic goals in men. Thus, gamification design should incorporate reward and competitive elements attractive to both genders, provide support and resources to help women develop their digital skills, and utilize intrinsic motivation elements that appeal to both genders.
Finally, regarding gender differences in ease of use when applying gamification, no significant differences were found in the perception of adequate use of the EducaPlay tool between genders and groups (with and without gamification). This suggests that the software and its activities were appropriate, and no usability or interaction issues could affect the results. This can be attributed to EducaPlay being designed for educational gamification and free of performance, access, and usage issues which have been reported in other studies that did not experience significant gamification benefits. Additionally, the results show that women felt less comfortable than men with the gamified tool. This aligns with previous studies indicating that men may initially feel more comfortable with gamification. Therefore, further studies with gamification techniques focused on motivational aspects aligned with social work and low competitiveness are needed. These would be more suitable for female students. As a recommendation, when designing an educational tool with gamification, consideration should be given to the preferences and motivations of each gender in the instructional design of gamified educational tools to mitigate potential differentiated effects. This can be achieved through questionnaires or interviews with students to understand their interests and learning styles, and which types of rewards or game mechanics are most motivating for them; by offering customization options and flexibility in the tool for each student to adapt aspects such as their profile, progression, or types of challenges according to their preferences; or using inclusive themes, narratives, and images that do not reinforce harmful gender stereotypes in learning content and rewards, among other measures.
In summary, women found gamification less enjoyable, comfortable, and useful than men. Additionally, women felt more comfortable without gamification than with it, making them learn less under gamification. These results attempt to explain motivational and usage factors for the gender-based differences in learning. Particularly, they shed light on the reasons behind the negative outcomes for women under gamification. Therefore, more rigorous studies are needed on the differential impact of gamification on the learning and motivation of men and women, controlling for confounding variables and employing solid experimental designs.