1. Introduction
Gamification encompasses multiple areas, such as technology, culture, society, healthcare, and economics, while helping make the reality more gameful to make humans more creative, skilled, productive, engaged, and motivated [
1]. A gamified system is an information system which uses game principles in a non-gaming context for increasing user motivation and engagement. The purpose of a gamified information systems is to provide an experience and motivation similar to games, affecting the user behaviour [
2]. Therefore, when developing a gamified system, one should strive towards not only a good functionality but also player engagement, as that is necessary to achieve the main goal of gamification—user motivation and active involvement in the activity that is being gamified [
3]. This creates new challenges compared to the development of traditional information systems.
Gamification is increasingly associated with the concepts of user interface (UI) and user experience (UX) [
4], a high quality of which is crucial in order to meet the rapidly changing user expectations and demands [
5]. The user interface defines components that allow the users to interact and communicate with the interactive system in order to complete specified tasks [
6]. The user experience, on the other hand, is the quality of the user’s interaction with the system, which includes user reactions and impressions [
6].
The crucial component of user experience is usability, which is a software quality attribute that determines the extent to which the specific user in a specific context can use the system to achieve specific goals effectively and efficiently [
7]. According to the ISO 25010 standard, there are six characteristics of usability, namely: appropriateness/readability; learnability; operability; user error protection; accessibility; and user interface aesthetics [
8]. However, these criteria can be broken down into even finer characteristics, for example, learnability can be treated as dependent on predictability, synthesisability, familiarity, generalisability, and the consistency of the system. The generalised usability guidelines, also known as heuristics, typically build on such a set of usability characteristics. For example, the widely cited Jacob Nielsen heuristics for user interface design cover ten principles: visibility of system status; match between system and the real world; user control and freedom; consistency and standards; error prevention; recognition rather than recall; flexibility and efficiency of use; aesthetic and minimalist design; helping users recognize, diagnose, and recover from errors; and help and documentation [
9]. At a more detailed level, there are hundreds of concrete usability recommendations and guidelines, which, in most cases, are of a universal nature.
Usability is important in the development of gamified systems, as it has a direct impact on the user engagement and the achievement of individual or organizational goals [
5]. If the system does not ensure sufficient usability, it can cause problems and affect the user, for example, to annoy, distract, or mislead him [
10]. This fact is among the basic principles behind the human-centred design approach, which incorporates the knowledge and techniques of human ergonomics and usability into the development of interactive systems, with the aim of making them usable [
6].
Therefore, it is important to ensure that the gamified system meets the usability requirements and complies with user experience and human-centred design guidelines, as poor usability can lead to a loss of user engagement. In addition, if the system is insufficiently easy to use, the user may choose to avoid the system or use it improperly, even if other aspects of the system (functionality, security, speed of operation, and others) are very well thought out.
As far as usability in gamified systems is concerned, this field is under-explored and the systematic reviews on usability recommendations for gamified systems are lacking. However, there are several studies focusing on the user experience and user interface recommendations for learning-oriented gamified systems. Handayani et al. [
5] presented a systematic review of the gamification UX and UI concepts, aimed at identifying specific design recommendations for the gamified e-learning systems. They specified three research questions about criteria, usability evaluation tools, and gamification elements for the e-learning application. The review shows that the gamification elements must be tailored to the target audience (gamified system users), while also stressing the importance of the user experience and user interface evaluation and testing with actual users. Mustafa and Karimi [
11] reviewed the elements of gamification in gamified online learning and their impacts on user experience and engagement. They outlined four research questions related to targeted education level, gamified subjects, combination of gamification elements, and the influence of gamification on the user experience. The authors concluded that the elements of gamification could affect the online learning user experience and engagement.
As a result, even though concepts such as usability, user interface, user experience, human and user-centred design are used in systems development in general, it is not clear what specific usability, UI, and UX guidelines and methodologies would be relevant when developing new gamified systems and evaluating existing ones. Having such a tailor-made toolkit would allow those working with gamified systems to concentrate their usability-related efforts more efficiently. This review aims to collect and analyse usability and user experience guidelines, intended specifically for gamified systems, and identify their relations to the gamification target audience and application domain. The findings are relevant to usability or gamification researchers seeking to better understand the current trends and possible work directions. On the other hand, they are also valuable to the developers and owners of gamified systems. Here, there are several potential benefits, including:
- (a)
improved ability to concentrate on the usability essentials when designing or developing a gamified system, instead of trying to meet all of the universal recommendations out there, thus saving time and resources;
- (b)
increased likelihood that the usability improvement measures taken will indeed be beneficial to the efficiency of the gamification, rather than some other aspect of the gamified system;
- (c)
improved ability to pick the usability recommendations based on the characteristics of the gamification in question, i.e., gamification domain and features of the target audience.
2. Methods
A systematic review is performed following the recommendations outlined in the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) [
12] methodology.
This review aims to examine the state of research in usability and related issues in the context of gamified systems. Three research questions were raised to further specify the review goals:
RQ1. What is the nature of specific usability recommendations for gamified systems?
RQ2. Do gamified systems usability recommendations depend on the gamification target audience?
RQ3. Do gamified systems usability recommendations depend on the system application domain?
The above questions influenced the key literature search criteria. Additionally, publication date, type, and other similar attributes were also defined. The full list of constraints applied to basic properties, as well as the titles and abstracts of papers, is given below.
Paper inclusion criteria:
Publishing date 2008 to the first quarter of 2022;
Document type is article and/or conference paper;
Usability, user experience, human-centred design, user-centred design guidelines for gamified systems are given;
Discussion is related to software gamified systems (i.e., not just gamification per se).
Paper exclusion criteria:
The literature search was carried out between February 2022 and March 2022 in five databases: Web of Science, Scopus, IEEE Xplore, ScienceDirect, and ACM Digital Library. The relevant papers were identified by searching in papers’ metadata (title, abstract, keywords), using keyword search queries. The keywords identified from the research questions were as follows: gamification; usability; user experience; human-centred design; user-centred design; guidelines; and their synonyms. The search queries and number of search results for each database can be found in
Table 1.
The following data about the identified papers were imported into the reference management tool: publication type; year of publication; title, abstract; author(s); publication language. Based on the eligibility criteria, the titles and abstracts were manually screened to determine which of the papers were relevant, irrelevant, or indeterminate and thus requiring full-text screening. The manual screening of both the abstracts and full-text was performed by two of the authors in a pair and, in case of disagreement, the paper under consideration was discussed to reach a consensus.
3. Results
The databases’ search resulted in the identification of 814 candidate records in total. Before screening the abstracts, filters were used to exclude the papers that were not articles and/or conference papers, were published before 2008, were not in the field of computer science, and/or were not in English. The selected references were imported into Zotero [
13], a reference management tool, which was used to remove the duplicates and manage the list. The number of papers included and excluded in each stage is shown in
Figure 1. A total of 396 records were removed before screening: 242 were removed using filters and 154 were duplicates. After screening the abstracts, another 177 records were removed. After full-text screening, 104 articles were excluded for the following reasons:
did not provide usability, UX, or human/user-centred design guidelines;
were not written in English;
papers were review articles;
full-text versions were unavailable.
Finally, an additional paper search was carried out following the process of snowball search for primary paper references and citations, and one additional paper that met the inclusion criteria was detected. In the end, 20 papers were selected as eligible for inclusion in the analysis.
After selecting the papers, the years and types of publications were analysed. The findings show that the selected papers were published in the last ten years, and even though the years 2017 and 2018 were the most productive, the totals were low. Most of the papers were conference proceedings (
n = 12). The numbers of papers per year and per publication type are given in
Figure 2 and
Figure 3, respectively.
After analysing the relevant guidelines and recommendations for gamified systems described in the selected papers, it was discovered that the recommendation lists provided by the different authors overlapped heavily. The different authors would often mention the same principle or recommendation with the only difference being the wording. Therefore, to facilitate the analysis of the recommendations, an initial grouping was carried out. In the end, the thorough review of 259 guidelines and recommendations resulted in the list of 30 items or generalised recommendations R1–30, most of which were mentioned in one way or the other in more than one publication (see
Table 2). It must be noted that the level of detail that the different authors provided in their recommendations varied significantly. Below are some of the recommendations and guidelines that were generalised into R1 and R2:
R1. Ensure user control and freedom:
“Allow users to control of the application.” [
14];
“The system provides users with choices on what to do or how to do something, which are interesting but also limited in scope according to each user’s capacity.” [
15];
“Allow the children/user to be in control of the application through self-pace progression, navigation and support elements.” [
16].
R2. Provide feedback:
“The feedback is affirmative, positive in tone, short, precise, polite and non-offensive.” [
14];
“The systems always inform users immediately of any changes or accomplishments in an easy and graspable way…. Feedback always tells users where they stand and what is the path ahead for progression.” [
15];
“When a system provides feedback, users are aware of the impact of their actions. In a game this enables them to know when they succeeded or failed and why.” [
17].
The list of generalised recommendations served as an input for the second phase of classification, as described further in this section.
The selected publications were grouped based on the target gamification domain and the audience outlined by the authors of the particular paper. The authors of seven papers did not define the characteristics of their recommendations’ target audience, while others concentrated on either children, the elderly, or adults (without defining whether such positioning includes the elderly) only (see
Table 3 for distribution data). Similarly, there was a good variety of gamification application domains that the presented recommendations were aimed at. Those included the development of gamified solutions for medicine, education, sport, and business. Only a minority of the analysed recommendations’ lists were positioned by the authors as applicable independent of the nature of gamified activity (see
Table 4). It was presumed that not defining the target audience and/or target gamification domain implied that these factors were deemed as having no influence on the importance of the defined usability recommendations.
It is important to emphasise that not all of the authors positioned their recommendations as aimed at improving the usability or user experience of gamified systems, even if most of the given recommendations could, in fact, be interpreted as usability recommendations. Instead, these authors claimed that their recommendations’ lists were intended for serious games developers and/or those striving to improve gamification mechanisms, user motivation, user engagement, adherence to the human/user centred design principles, or a combination of the above.
The relationships between all of the three mentioned aspects of how the analysed lists were positioned are given in
Figure 4.
To gain a better understanding of the nature of the recommendations proposed in the selected papers, the generalised recommendations R1–R30 were further analysed one by one and assigned to one or more general classes, based on their nature. Two classifications were used to group recommendations, namely:
Nielsen’s heuristics for user interface design [
9]. Heuristics serve as the most general usability principles available. Each recommendation had to be assigned to one or more of the following heuristics:
N1. Visibility of system status;
N2. Match between system and the real world;
N3. User control and freedom;
N4. Consistency and standards;
N5. Error prevention;
N6. Recognition rather than recall;
N7. Flexibility and efficiency of use;
N8. Aesthetic and minimalist design;
N9. Help users recognize, diagnose, and recover from errors;
N10. Help and documentation.
Usability characteristics defined in ISO 25010 [
8]. Usability is a complex concept and its characteristics provide a more granular view. Again, each recommendation had to address one or more of the following characteristics:
I1. Appropriateness recognizability;
I2. Learnability;
I3. Operability;
I4. User error protection;
I5. Accessibility;
I6. User interface aesthetics.
The mapping to Nielsen’s heuristics was completed, based on the descriptions of the said heuristics given in [
9] and its accompanying articles. Similarly, the mapping to the ISO 25010 usability characteristics was completed, based on the descriptions of these characteristics given in the standard and in the textbook literature on usability. Due to the lack of purely formal definitions, such mapping is deemed to be at least somewhat subjective, but this risk was mitigated by the fact that two of the authors of this paper have extensive experience in both the teaching of Nielsen’s heuristics and usability characteristics and in using them as classifiers for software usability problems.
Table 5 shows the mappings between the generalised recommendations collected from the papers and Nielsen’s heuristics, as well as the ISO 25010 usability characteristics. Note that the recommendations R9, R19, R21, R22, R23, R26, R27 and R29 were dismissed from further analysis as they concerned the methodological or social aspects of gamification and were not related to the usability of gamified software systems.
To better understand the importance of each of the usability classes, we traced back from the generalised recommendations to the pre-merger ones to have a look at how many individual recommendations of the given nature were mentioned in the selected papers (see
Figure 5).
Although the totals given at the bottom of the
Table 5 already provide certain hints, the summary given in
Figure 5 clearly shows that the majority of the identified recommendations were aimed at improving the I2-learnability of gamified systems (
n = 78), with I3-operability a distant second (
n = 40). Speaking of the classes based on Nielsen’s heuristics, it must be noted that there were only 12 recommendations overall related to consistency and standards (N2) and 14 related to help and documentation (N10). The matters concerning user interface appearance or aesthetics seemed to be the least important to the researchers.
Table 6 reveals a relationship between Nielsen heuristics, ISO 25010 usability characteristics and all 20 of the selected publications that helps to better understand the importance of these aspects to the different authors. One usability characteristic and four heuristics have dependencies with all of the publications, namely: N1-visibility of system status; N2-match between system and the real world; N5-error prevention; N9-helping users recognize, diagnose, and recover from errors; I2-learnability. On the other hand, three heuristics and one usability characteristic could only be mapped to less than half of all of the selected publications. Those were heuristics N4-consistency and standards (
n = 6), N8-aesthetic and minimalist design (
n = 9), N10-help and documentation (
n = 7) and the usability characteristic I6-user interface aesthetics (
n = 9). Additionally, it is noteworthy that only three publications have outlined recommendations related to all of the usability heuristics and characteristics.
After establishing the dependencies between publications and usability heuristics/characteristics, we wanted to understand the relationship between the way usability recommendations were positioned by the authors and the nature of these recommendations.
Figure 6 shows how many of the selected papers mentioned at least one recommendation of the given nature and for a given gamification target audience or application domain. We can see, for example, that all six papers aimed at gamified systems for children, mentioned at least one usability recommendation concerning learnability, but only two of such papers mentioned the recommendation(-s) related to user interface aesthetics. This suggests that user aesthetics were considered less critical than learnability when developing gamified systems for young audiences. Note that there are only seven intersection points on the diagram, where there were no recommendations of the given nature for the specific target audience or gamification application domain.
To gain an even better understanding of these relationships, we again traced back from the generalised recommendations to the initial ones presented in the selected papers and repeated the calculations given in
Figure 6. The result is presented in
Figure 7. Here, we can see that the learnability recommendations were suggested considerably more often when talking about gamified systems for children (
n = 31) than when giving recommendations aimed at gamified systems for the elderly (
n = 13). Of course, one must not forget that there were six papers that talked about children-oriented gamified systems and four that discussed those for the elderly (
Table 4). In addition, recall that the same recommendation could be assigned to multiple classes, depending on its nature.
4. Discussion
The results of the review presented above served as the input answering the research questions RQ1–RQ3. The key observations are presented below.
RQ1. What is the nature of specific usability recommendations for the gamified systems?
The authors of the selected papers positioned their findings not only as usability recommendations, but also as recommendations for building better, more serious games, ensuring a better user experience, higher motivation or engagement and better compliance with human/user-centred design principles. Despite that, many of the 259 identified guidelines and recommendations addressed the same issues and therefore a summarised list from all of the 20 papers included just 30 items, 7 of which were dismissed as not relevant to this analysis.
In terms of the nature of the recommendations, the variety was good, with all ten Nielsen heuristics and all six ISO 25010 usability characteristics, that were used to group recommendations, covered. The usability characteristic that was addressed the most was learnability, while the user interface aesthetics aspect turned out to be the least important for the gamified systems researchers. The examples of generalised recommendations aimed at improving the learnability of gamified systems were R2-provide feedback, R3-use familiar vocabulary, R4-ensure that available actions are relevant to goals, R5-use modality principle, R6-provide help to users, R7-ensure onboarding, R8-use information segmentation, R11-ensure consistency of elements, R13-provide clear error and warning messages, R14-provide challenges that grow with user skills and R28-ensure easy navigation.
As can be seen from the list above, many of the recommendations were what one could classify as universally accepted, e.g., using familiar vocabulary, providing good feedback, error prevention, information segmentation, ensuring short response times, etc. On the other hand, the authors often recommended using modality and signalling principles as well as ensuring onboarding (relevant information before the task) and providing fatigue management features (adaptable time limitations and speed of work). Such guidelines are not applicable to all software systems, but clearly should be taken into consideration when developing gamified ones. Overall, all ten of Nielsen’s heuristics were covered by the analysed authors in their recommendations, led by N5-error prevention (38 individual recommendations) with N4-consistency and standards coming last (12 individual recommendations).
Most of the recommendations provided in the selected papers were of a general nature, and concrete instructions that could be used by gamification software designers and developers were relatively rare.
RQ2. Do gamified system usability recommendations depend on the gamification target audience?
The majority of the selected papers addressed the usability of gamified systems aimed specifically at either children or adults (30% and 30%, respectively) indicating that age is an important factor when making the decision on how to build or improve gamified systems. Out of six papers presenting adults-oriented recommendations, half were aiming specifically at the elderly gamification audience. The authors of the remaining 40% of the selected publications did not specify any characteristics of the gamification target audience for whom they were providing recommendations. Other user characteristics that typically influence usability-related system development decisions, such as user goals, level of motivation, education or IT skills, were not mentioned.
However, upon further analysis, no drastic differences were found between the nondescript recommendation sets and those aimed at gamified systems for specific age groups. The recommendations concerning learnability were most often mentioned: for children n = 31 (5.16 per paper on average); for adults, including elderly n = 21 (avg. = 3.5); undefined n = 26 (avg. = 3.25). The user interface aesthetics turned out to be the least-addressed usability characteristic. This came as a surprise, as aesthetics have a big influence on the mood the system creates and thus one could presume that for gamified systems, which are of an inherently ambiguous nature and must balance playfulness and seriousness, this factor should be quite important. The principle of consistency and standards was also not stressed by the analysed authors, which may indicate that the gamified systems, as a relatively new class of software, does not rely so heavily on following recognized conventions. On the other hand, it cannot be forgotten that certain usability recommendations are considered to be universally applicable to all systems without exceptions, and hence they may have been deliberately omitted by the analysed authors.
RQ3. Do gamified system usability recommendations depend on the system application domain?
As can be seen from the data given in
Table 3, gamification in the domains of education and medicine was discussed most often (30% and 25% of the analysed papers, respectively). The authors of 35% of the selected publications did not indicate any gamification domain(s) that their recommendations were for. Not surprisingly, the largest total number of recommendations were positioned as aimed at the gamification of education. However, similar to the question RQ2, the review results show that the gamified system application domain did not have a substantial impact on the nature of outlined usability recommendations. In all of the application domains (medicine, education, business, sports and undefined), the most mentioned recommendations were related to the learnability aspect of the systems’ usability: medicine
n = 18 (avg. = 3.6); education = 30 (avg. = 5); sport
n = 5 (avg. = 5). business
n = 4 (avg. = 4); undefined
n = 21 (avg. = 2.63).
Speaking of the limitations of the conducted review, the generalizability of its results is somewhat compromised by the fact that the analysis only focussed on research journals and conferences. The usability of gamified systems is a highly practical topic and therefore it can be discussed in books as well as online blogs, forums and similar non-academic sources, which were not the subject of this analysis. Furthermore, the number of papers that were selected was not high–20 in total. One could observe a spike in relevant research activity in years 2017 and 2019, but, overall, the numbers of publications on the analysed topics were consistently low. This may have compromised the analysis with regards to the research questions RQ2 and RQ3, as the differences or, more precisely, similarities in the nature of the various recommendation sets could have been due to chance.