1. Introduction
The number of foreigners visiting, working, and studying in Indonesia has increased recently for a variety of reasons. One of the reasons for this increase is the resilience of the Indonesian economy during the coronavirus pandemic and its attractiveness for foreign investment and expansion [
1]. This can be proved by the fact that the total number of international visitors to Indonesia in 2023 rose 98.30 percent compared to the previous year, surpassing 11 million people [
2]. Among them, 168,048 people are registered as foreign workers, an increase from the 2022 record of 111,537 people [
3,
4].
However, the language barrier brings stress to many non-native foreign students and workers in Indonesia [
5,
6]. This situation is made worse by the lack of comprehensive language support services intended to help these people not only surmount communication challenges but also integrate fully into Indonesian society. As of September 2024, only 1.2 percent of all websites in the world use Bahasa Indonesia or Indonesian language [
7]. Furthermore, the 2023 Duolingo Language Report did not include Bahasa Indonesia as one of the top 10 most popular languages studied on the application, despite its larger population compared to Japan and South Korea [
8].
A milestone in the global recognition of Bahasa Indonesia was reached when the United Nations Educational, Scientific, and Cultural Organization (UNESCO) General Assembly held on 20 November 2023 declared Bahasa Indonesia as the 10th official language [
9]. Mohamad Oemar, the Indonesian Ambassador to the Republic of France and the Permanent Delegation of the Republic of Indonesia to UNESCO, stated that the recognition of Bahasa Indonesia as an official language enhances cross-cultural communication and fosters stronger diplomatic ties between nations, both nationally and internationally [
10].
This is in harmony with one of the research studies which resulted in a strategy to globalize Bahasa Indonesia, a program called Bahasa Indonesia for Foreign Speakers, or popularly known as BIPA [
11]. Based on the data from the Agency for Language Development and Cultivation Ministry of Education and Culture, there are 428 institutions holding BIPA programs around the world. About 52 countries have adopted the Bahasa Indonesia curriculum, with the estimation of Bahasa Indonesia second-language speakers reaching 150,000 people [
10,
12].
Table 1 summarizes a pre-survey conducted by a researcher among the “Bahas Bahasa” group members on the messaging platform Telegram. It showed a lack of satisfaction with the existing BIPA resources offered by the Ministry of Education and Culture. These criticisms include poorly arranged content, broken web links, passive social media promotion, and limited access to several pages within the website. The suggestions collected from them were the need for a better website service and wider promotion through the Indonesian embassy or local universities or organizations.
Despite the emergence of Indonesian courses on platforms like Babbel and Duolingo, the resources for learning Indonesian are limited. When compared to the popular languages such as Spanish, French, and Arabic, the interactive features and comprehensive content are lacking. A search through the Google Play Store led to only one application that fits the purpose. This application contains an electronic book with few interactive features. While this application has a perfect rating of 5.0 from a few users, this merely shows that more dynamic and creative learning tools are desired because of its lack of competitors.
To bridge this gap, the implementation of gamification emerges as a reasonable solution to enhance satisfaction in Bahasa Indonesia learning platforms. Gamification can be defined as a process of enhancing a service with affordance of game-like experiences in order to support users’ overall value creation [
13]. Gamification involves applying elements of gaming, such as achievements, competition, and rewards, to non-game situations, rather than full-fledged contests [
14]. This approach aims to increase engagement and motivation among participants, with findings evidencing its superiority over traditional game-based learning across a range of educational settings [
15,
16,
17].
In undergraduate education and employee training, gamification has been proven to increase workforce recruitment and retention, improve program adaption, and enhance job performance [
17]. Giving rewards within a social issues-related context not paid for by the teachers will motivate students to participate in extra learning activities, which otherwise they would probably neglect. In addition, people will see those jobs as enjoyable and satisfying [
16].
Therefore, this research will answer the following research questions:
RQ1 How can the gamification method be utilized to develop a BIPA learning application?
RQ2 How significant is gamification as a method to acquire a foreign language?
RQ3 What are the usability challenges of a gamified learning system?
In addition, this research will measure the effect of satisfaction with gamification on increasing the Bahasa Indonesia ability of non-native speakers.
2. Materials and Methods
Krath [
18] examines how gamification works in ten principles. Gamification is a technology that effectively communicates goals, sets personal goals, gives feedback, rewards performance, connects users, adapts tasks to individual abilities and knowledge, and nudges users toward necessary actions, offering multiple options to reach a goal. Thus, it is a valuable tool for achieving goals of user-friendliness and simplifies the content for the user.
Figure 1 shows three elements that shape gamification: mechanics, components, and dynamics. The mechanics of a game are abstract concepts that define potential actions and states of users, influencing their behavior and guiding the progression of the game, despite the ongoing evolution of specific components. Achievements, challenges, cooperation, feedback, ownership, progression, transaction, and stochastic elements are examples of the mechanics [
19].
The selection of components in a system depends on its purpose, target user group, and software tools. Intelligent use of these components is challenging and requires careful consideration. Examples of the components include points, badges, leaderboards, quests, competitions/challenges, virtual goods, gifting/sharing, and levels [
19].
The dynamics of a gamified system involve the interactions between users, influenced by the components and mechanics described. These dynamics depend on the user’s nature and experience, ensuring that the system adapts to their unique behaviors. Elements that we can find include emotions, relationships, and also narratives and storylines [
19].
The most common framework adopted by game developers to optimize gamification is Mechanics, Dynamics, and Aesthetics (MDA). Mechanics defines the game’s particular elements, encompassing data representation and algorithms. Dynamics deals with the real-time behavior of mechanical systems that engage with player inputs and interact among themselves. Aesthetics refers to the emotional responses that the player has when engaging with the game system [
20].
Table 2 describes the comparison of several studies on learning Bahasa Indonesia, such as a prototype web design of BIPA for distance learning [
21], an offline application for learning BIPA at the beginner level [
22], and a web-based design of a BIPA placement test [
23].
Among the studies conducted, only the prototype web design developed for BIPA distance learning had a statistical evaluation. The evaluation carried out by the learning media expert was satisfactory with a percentage score of 60.29%, while the evaluation by the BIPA teaching expert was also satisfactory with a score of 81.57%. The overall level of applicability for the application was found to be 71.52%. Nonetheless, this study did not have a targeted user base of non-native Indonesian speakers.
There are also alternatives, such as using BIPA online materials available on the official BIPA website. Arabic language education program students, as prospective teachers and content creators, create video content that can improve students’ fluency in Arabic. Targeting BIPA teachers for students or foreign speakers from Arabic-speaking countries requires more effort [
24].
Table 2.
Comparison with prior studies on Bahasa Indonesia.
Researcher | Research Methods | Results | Evaluation |
---|
F. I. Maulana et al., 2020 [21] | Waterfall model, sequential approach | Web-based application prototype named BIPALOKA with features such as tutor appointment and tutor scheduling. | There are no quantitative results, and only a group of actors, such as admins, users, and tutors, can edit and view certain menus on the account page. |
Boeriswati et al., 2023 [23] | Design-based research and bibliometric analysis | Web-based application prototype with BIPA placement level test as a main feature. | Overall, 89% of respondents agreed on the importance of developing BIPA program placement testing instruments. |
A. Maulana et al., 2022 [25] | Borg and Gall’s “Ten steps of research and development” approach | Web-based application prototype named BIPA KITA. | The feasibility level of the prototype product reached 71.52%. The validation result of BIPA experts was 60.59% and that of learning media experts was 81.57%. |
Yulianeta et al., 2022 [26] | 4-D Model (define, design, develop, and disseminate) | Web-based application prototype. | The average score of BIPA participant satisfaction reached 91.285. The BIPA teacher satisfaction predicate was very good with an average score of 89. |
Daniar et al., 2022 [27] | Iterative design | Mobile application based on Android. | Overall, 81% of students rated the product positively with the category of “very interesting”. The level of effectiveness of using game media in learning Indonesian is 82%, or very effective. |
Progressive Web Apps (PWAs) are an idea from Google Chrome software engineer Alex Russell and designer Frances Berriman. In a blog post, Russell [
28] wondered about a system that can store an application mediator that facilitates content searching rather than clicking on one link to access the content that user want to watch. Chris Wilson, a Google Chrome engineer, added that PWA creates a new level of user experience quality [
29].
Figure 2 illustrates the architecture of Progressive Web Apps (PWAs), which can operate on both desktop browsers and mobile operating systems like Android and Apple’s iOS. PWAs leverage modern web technologies to deliver experiences similar to native applications while remaining accessible through web browsers. On Android, PWAs utilize WebAPK, allowing them to be installed as native apps, along with Shortcuts for quick access and QuickApp for efficient resource management. PWAs saved as Web.app on iOS offer native app features but have limited access to system-level APIs.
Service workers facilitate both offline and online capabilities for PWAs across different platforms. When online, service workers connect with Backend Servers to provide real-time data. In offline mode, they rely on Browser Cache Storage to keep the application running. PWAs present a viable alternative to native apps for cross-platform development, as their architecture enhances cross-platform functionality, lowers development costs, and boosts user accessibility.
Developers must fulfill specific requirements for a web application to be installable from a supported browser. Among them is a web application manifest in the form of an additional link tag that connects the manifest file. The manifest file contains a single JSON object consisting of several members that contain aspects of the appearance or rules of the PWA [
30].
This research applied a quantitative approach with an experimental design to assess the efficacy of gamification in improving user satisfaction in language acquisition. The data were gathered from multiple sources, including questionnaires, internet observations, and user usability measurements, as shown in
Table 1. User feedback and engagement metrics were also analyzed to identify the overall success rate of the gamified elements.
This research began with identifying requirements, developing prototypes, conducting usability testing, and evaluating data. To identify requirements, we conducted online observation. The website we observed was an official website of the BIPA program. The identified requirements are reflected in the use case diagram that can be seen in
Figure 3. After identifying problems and constructing a research question, supporting data were collected from a pre-survey and conversation with experts.
Belajar Indo was developed in a web-based application using Progressive Web App and Next.js as the frameworks. The language material inside Belajar Indo was based on the BIPA website with several changes as suggested by a Bahasa Indonesia expert. This will create a system with an interface adaptable to many resolutions, particularly for online and mobile devices. Research findings can be highlighted with the hope that they will contribute to the advancement of BIPA applications.
This usability evaluation used a third-party application called Lyssna, with three prototype task questions and several questionnaires. We made sure that the participants involved in this assessment were between the ages of 25 and 54 years old, did not speak Indonesian well, and had at least a college degree. These requirements come from the fact that in European Union countries people know at least one foreign language [
31].
There were three tasks to be completed based on the use case diagram requirements, such as sign in to create an account, select to enroll in a course to begin learning, and view own report to track the learning progress. This research used usability testing and evaluated the results using the Net Promoter Score. The participants were asked to complete the tasks while thinking aloud so as to gain insight into their thought process.
The success rate is determined by the total number of successful outcomes. It is calculated by dividing the total number of successful tasks completed by all respondents by the total number of tasks performed by all respondents. This calculation offers a clear insight into how effectively the respondents accomplished their tasks. A higher success rate reflects a greater level of efficiency and skill within the group.
The Net Promoter Score will help determine the overall satisfaction level of the users with the language learning platform, providing valuable feedback for future improvements. It works by gathering data from customers who have used the product or service and assessing their likelihood of recommending it to others. It classifies customers as promoters, passively satisfied, and critics based on ratings from 0 to 10 in their feedback. Promoters include those whose ratings fall between 9 and 10 and indicate customers who are highly possible to recommend the platform. Passively satisfied individuals are those with ratings from 7 to 8, while critics, who are extremely unlikely to recommend, have ratings below 6 [
32].
3. Results
The design results include gamification design and application design. Gamification design is utilized to meet the needs of users for BIPA learning. A wireframe application framework design is developed to make it easier for designers in developing an application.
Figure 3 illustrates the use case of the application.
The registered users can log in. Users can update their profiles and view their profiles after registering. In order to learn the Indonesian language material, users register for courses so they can take quizzes and earn prizes in the form of points, badges, and others that can be exchanged. In addition, users can view the available courses, including descriptions and resources. It allows the user to track their progress by showing a leaderboard with their rating and points earned. Users can also check the report to see their progress as another means of transparency.
There are gamification features in the Belajar Indo application, such as badges, leaderboards, and points.
Figure 4 illustrates the application’s implementation of gamification through a high-fidelity wireframe. Users can earn points from attending courses and completing quizzes. At some point levels, users can earn a badge as an achievement.
The participants were predominantly from the United Kingdom, comprising a substantial majority of seventeen individuals, while the United States contributed six participants. Australia and the Netherlands gave five individuals, Malaysia provided two participants, and Singapore supplied one. The age of the participants ranged from 25 to 54 years old. Users had diverse educational levels, from some college education to postgraduate degrees.
Table 3 displays the task completion status of each participant.
Table 4 presents the assessment of user job accomplishment, including total successful clicks and misclicks. Among the 36 users who participated, the total number that reached the target screen for each task was 32, 35, and 34, respectively. The average percentage of participants who finished the whole task was 93.51%. The average number of clicks in the first task was eight clicks with 42% misclicks. The second task recorded two clicks with 3% misclicks, while in the last task, it was five clicks with 27% misclicks.
Table 5 represents the success rate across gender and education level. The data obtained for the male users suggest that the most consistent rate of success came from those in a higher educational bracket. While all males with postgraduate education and some with a college education had a complete success rate of 100%, most of those with a college showed a slightly lower rate, at 91.67%; for male users with some postgraduate education, that number dropped to 83.33%. Female users, on the other hand, showed the same variation in recordings where the success rate always tended to be high, though slightly lower as compared to male user records. Records for the groups with some college education were unavailable for representation. All female users who had a postgraduate education recorded a success rate of 100%, while those with some college education and college degree holders had an 88.89% success rate.
Table 6 summarizes user success rates by both device type and operating system in use. Among the male users, there was a slightly lower success rate of 83.33% using desktop devices with Windows operating system, while for Mac, it stood at 100%. Female users who used Windows on desktop devices achieved full success, while on Mac, this was only 77.78%. The success rates for males and females on mobile devices were similarly very high, with Android users at 100% and iOS users varying, with 88.89% for males, while females were a little higher, with 94.44%. This could point to user preference or comfort levels for the operating system.
As shown in
Figure 5, twenty-seven participants provided a favorable rating, ranging from 8 to 10. The remaining responses had unfavorable evaluations, with two people scoring 1, one individual scoring 5, and another scoring 6. Those who gave a score of 1 said that they found the prototype confusing, especially the bottom menu. In addition, the design can certainly be enhanced, as the current version looks like a low-fidelity mockup with a splash of color. Another response from the participants scoring the app 5 and 6 also mentioned that, while visually stunning and easy to navigate, the design felt incomplete because of all the empty space and white space, which might hinder the usability of the user interface.
4. Discussion
A high success rate of reaching the goal screen shows that participants can go through the application with ease. However, the high number of misclicks in the first and third tasks does show some room for improvement in terms of making this user interface smoother and clutter-free of unwanted interactions. They also rated 8.11 out of 10 on the likelihood to recommend this application to others. The overall usability test for the Belajar Indo application was positive.
On average, 21% of misclicks were made; that is, the first and third tasks contributed 42% and 27%, respectively, to errors, which means that the user interface should be improved in order to avoid less important or wrong user interactions. Although the prototype was tested with the iOS interface, it is worth noting that Windows users and Android users understood the application pretty quickly, hence displaying its platform versatility. Mac users and iOS users scored poorly, resulting in most users quitting before proceeding with the tasks.
Despite the majority of testers saying that the tasks were clear, and the interface was straightforward and easy to navigate, there were several post-testing comments suggesting that some of the design still looked unfinished, which influenced the high rate of misclicks. One participant said, “In the beginning the task said to log in as a new user which I assumed would mean I have to create a new account, but I was actually expected to log in to an existing account, which had me a bit confused.” Another participant said, in a simple way, that they were slightly confused with the task and took a long time to understand it.
Most participants experienced difficulties during the login process because the instructions were not clear. In the heatmap analysis, it was shown that most users clicked on wrong areas or revisited previous pages due to confusion. The developers can make the user experience better by fixing excessive clicks and misclicks. This, in theory, should enhance the completion times of tasks and user satisfaction. These usability test results helped put some key recommendations toward further development of the Belajar Indo application in an effort to make it more user-friendly. The results from the analysis of heat maps hint that the first task may have been difficult due to the complexity of the sign-in process.
The heat maps shown in
Figure 6 proved one of the concerns raised by several participants, noting that the design made them confused because there were no clear instructions. These heat maps are significant for explaining the root causes of the high percentage of misclicks in the first task. The heat maps further show a lack of clarity in the instructions on this page, leading to user frustration and inefficiency. It supports our recommendations for simplifying the interface layout and providing clearer guidance to enhance the user experience.
These misclicks, along with the relatively high click count in the first task, could suggest that either the interface is too complicated or the users had some difficulties regarding how to proceed further with parts of the application. Lastly, it was not clear to the respondents what the description of the task was, so they revisited it several times and navigated backwards to the home section and the sign-in page. To improve the layout, simplifying the menu structure and using concise instructions should be considered, which can help reduce errors and user frustration. Additionally, incorporating user feedback and conducting usability testing can further enhance the user experience.
The average rating given by participants regarding their likelihood of recommending this application to others was 8.11 out of 10. Overall, usability testing of the Belajar Indo application yielded positive results. This encouragement is well received, yet it also points toward specific areas, such as optimizing task flows and reducing click errors, where usability enhancements can be made.
Despite a high average rating for recommending the application, the occurrence of misclicks and the potential for enhancing the user interface indicate that usability concerns remain to be resolved. Simply depending on user ratings may not yield a thorough evaluation of the application’s usability. Therefore, usability testing is crucial in addressing specific issues in the Belajar Indo application, ensuring a seamless and efficient user experience. This comprehensive approach, which includes observing user behavior, conducting surveys, and analyzing heat maps, leads to increased satisfaction and loyalty.
To check if gamification is effective for learning Bahasa Indonesia, another analysis was conducted using a nonparametric analysis. This aimed to show symmetric measures on some variables compared with the response time, as shown in
Table 7. Based on the
p-values that are much greater than 0.05, it is indicated that these variables are insignificant and the results are in line with the inclusive design of the system, developed to accommodate a wide range of user profiles.
One surprising fact was found, where the user rating had a negative correlation with p-values under 0.05. This result might indicate that users who rated the system highly experienced some usability issues but liked either the potential of the platform or the gamification elements. Alternatively, this might reflect differences in individual expectations or learning styles.
Such issues will have to be taken care of in the future with concrete measures, including simplifying the layout, offering clearer instructions, and improving user guidance. Revising the layout, adapting the information architecture and navigation guides, and clarifying instructions could improve efficiency and reduce errors, possibly lowering user frustration. Combining usability testing and feedback loops with analyses such as heat maps and task completion rates ensures that changes are brought effectively to the users and improvements are warranted.
Although no research has revealed a comprehensive usability testing result or user satisfaction rating, the researchers hope that the findings will have an impact on improving satisfaction and usability. This will empower users to conveniently learn Bahasa Indonesia with the application, fostering language acquisition and cultural understanding. Additionally, the researchers aim to improve the target for non-Bahasa Indonesia speakers to communicate effectively in Bahasa Indonesia.
The diverse cultural and linguistic backgrounds of participants might influence their perception of, and also the effectiveness of, gamified elements, so this could endanger the generalizability of this research’s findings. Although our study included a rather heterogeneous sample size of participants, further research will be needed to determine whether such cultural differences significantly impact gamified language learning. Further research in this respect could categorize people according to their cultural background in order to examine unequal effects.
While traditional language learning relies on structured lessons, textbooks, and repetitive exercises, gamification brings play into the process of learning. Traditional methods are passive in that students receive information provided and practice it through writing exercises or classroom drills. In contrast, gamification incorporates challenges, rewards, and progress tracking that call for active engagement and immediate feedback, which raise engagement and motivation in learning.
In general, the results demonstrate how gamification can enhance user satisfaction, as in previous studies like Daniar et al. (2022) [
27] and Yulianeta et al. (2022) [
26]. However, Maulana et al. (2022) [
21] suggested a lower (71.52%) feasibility score for their web-based BIPA application based on closed observation from the perspective of Bahasa Indonesia teachers and education technology experts; we should acknowledge their result and consider it in future research.
Furthermore, the comprehensive usability testing in our study gives a better understanding of system effectiveness through metrics such as task success rate and misclick analysis. The heatmap analysis visualization also helped us to decipher what caused users to misclick, which affected the task success rate. This underscores the distinctive contributions of our research, which integrates gamification with a user-centered methodology to enhance the overall learning experience for non-native speakers of Bahasa Indonesia.