A Heuristic Method for Evaluating Accessibility in Web-Based Serious Games for Users with Low Vision

Abstract

Nowadays, serious games have become a beneficial resource in the learning process; they are part of our culture and promote social inclusion. Designing accessible serious games is a complete challenge, even more for non-experts. Most existing serious games do not meet accessibility standards because of a lack of methods that include standards and help create more accessible serious games. For this reason, our research presents a heuristic method with three modifications to Giorgio Brajnik’s barrier walkthrough method and based on the Web Content Accessibility Guidelines 2.1 (WCAG 2.1). We defined 28 barriers for the users with low vision and the related impact and persistence variables by defining severity ranges to evaluate accessibility. This method allows measuring the accessibility of web-based serious games; the method proposed in this article can be a good help for non-experts. As a case study, this heuristic method was applied to 40 web-based serious games. The evaluators concluded that serious games should apply WCAG 2.1 to achieve an adequate and inclusive accessibility level. However, this study has limitations; the heuristic method depends on the evaluators’ experience. This work can contribute to studies related to accessibility heuristics in serious games; it can also help construct a software tool that applies WCAG 2.1 and helps experts and non-experts evaluate accessibility in serious games.

1. Introduction

At present, web-based serious games significantly contribute to our culture and can promote social inclusion. Additionally, students’ participation and motivation in the classroom are of great importance in the learning process, so the concept of gamification [1] has taken on great importance. Gamification allows (1) stimulation and support of creativity; (2) player motivation through modifications and new levels of play; (3) use as a teaching aid; (4) inclusion as a trendy means of recreation; and (5) mind training [2].

The International Game Developers Association (IGDA), according to its survey of 1116 developers in 2019, has revealed that 85% of developers considered diversity in the games industry to be essential [3]. On the other hand, the Game Accessibility Special Interest Group (SIG) aims to support the design of universally accessible games [4]. This group has worked since 2003 to aid the game industry to develop more accessible video games.

The accessibility requirements themselves are described in the comprehensive legislation, specifically section 14.21 of the Century Communications and Video Accessibility Act (CAVAA) [5]. Similarly, Microsoft [6,7,8], and Sony [9] are committed to ensuring that products and services are designed for everyone, including the one billion people with disabilities, and therefore apply sets of guidelines to design and evaluate the accessibility of software and hardware developed by companies.

According to the World Health Organization (WHO), more than one billion people have some disability; this corresponds to the fact that there is 15% disability worldwide [10]. Therefore, accessibility in serious games is essential to improving the interaction between users and serious games. The primary motivation for building serious accessible games is to offer easy access to many users, including people with disabilities.

Therefore, accessibility in the context of serious games [11] aims to make applications usable by the maximum number of people, regardless of their knowledge or personal abilities and technical characteristics of the equipment used to access the serious games. Accessibility implies the way users perceive, navigate, understand, and interact with serious games.

This study considered the four principles of accessibility of the Web Content Accessibility Guidelines (WCAG) 2.1 [12]; we used a heuristic method to assess accessibility in serious games. The proposed method is based on the barrier walkthrough (BW) suggested by Brajnik [13,14,15] and WCAG 2.1 [12]. Therefore, a barrier is an impediment that hinders the interaction between the serious game and the user.

The barriers found through the assessment can produce obstacles in interacting with serious games, especially for people with disabilities. In this study, the authors considered the impact and persistence of each barrier faced by users with disabilities. After establishing scales for impact and persistence, the authors determined the severity of each barrier.

This study was developed from previous works [15,16,17]; in the current work, the authors apply three modifications to the BW method. The first modification comprises widening the scale to examine the persistence and impact between zero (0) and three (3). The second modification is applied to persistence, which depends on the number of barriers presented in the assessment. The third applies the scales defined to evaluate effectiveness, productivity, satisfaction, security, and impact.

The heuristic method proposed has eight phases: (1) choose the web-based serious games; (2) describe the category of user; (3) determine user activities; (4) investigate the serious games; (5) make a list of the barriers; (6) evaluate with the heuristic method; (7) record the data; and (8) interpret and analyze the data.

As a case study, this method was applied to a total of 40 web-based serious games selected with simple random probability sampling from various websites for educational and learning purposes. During the evaluation of the serious games with the heuristic method, two evaluators participated, with accessibility experience in serious games from 2015. In this case study, the authors applied the 28 guidelines for users with low vision. This heuristic method can be replicated for other categories of disabilities by considering the appropriate barriers with the type of disability. However, this heuristic method is manual and requires experience and training in web accessibility by the evaluators. This investigation can provide a beginning step for future studies associated with accessibility heuristics in web-based serious games.

The rest of the document is structured as follows: Section 2 defines the background and work related to accessibility in serious games; Section 3 defines the heuristic method; Section 4 presents the case study; Section 5 explains the results; Section 6 presents the discussion of the results. Finally, Section 7 describes the perspectives and future work.

2. Background

In this section, we explain serious games, considered as a resource to support learning [18]. Some authors maintain that serious games improve the comprehension capacity of students when they learn [19]. Accessibility in serious games implies that users can perceive, understand, navigate, and interact [12] without the application’s problems. The evaluation methods of the accessibility in serious games, several previous studies about the subject, are presented and introduced to our study. The heuristic method addresses several studies and concepts applied in this research.

2.1. Serious Games

Serious games [20] or training games have a different primary purpose than entertainment; they must also link the real and virtual world. They promote [21] the construction of knowledge and skills through exposure to different situations, cases, or problems. They allow simulations of a real situation to interact in the environment in the same way they would in a real environment. They are aimed at different audiences and can be of any kind, uses different technologies, and are designed for different platforms, computers, consoles, virtual worlds, and social networks. According to its final objective, [22], three categories are identified:

(1)

Edutainment, or educational games so that the game takes part in the education, the game provides an ideal environment for training by transmitting information and testing the knowledge.

(2)

Advergaming, a game to promote a product, one of the most popular applications of serious games is advertising communication.

(3)

Simulation facilitates learning represents a simplified model of reality, either with group games of physical or intellectual skills that reproduce the study’s process.

The video gaming industry is vast; in 2015, there were almost two billion video players worldwide; it is estimated that this number will increase to more than three billion gamers by 2023 [23]. According to statistics [24], the educational serious games market’s income will grow from 3.5 billion USD in 2018 to 24 billion USD in 2024.

According to Google Trends [25] trend on “serious games” and “educational games” in the last five years worldwide, taken from 15 November 2015 to 15 November 2020, in Figure 1a, we observe that “educational games” tend to grow from 2019. Figure 1b shows that the peak corresponds to 22 March 2020, coinciding with the coronavirus pandemic’s confinement in many countries. Because teachers, parents, and students were looking for “educational games” to reinforce students’ learning processes and compensate for the fact that they could not use face-to-face classes.

2.2. Accessibility in Serious Games

As we have mentioned, accessibility [12] is the inclusive practice of ensuring that serious games and technologies are designed and developed so that people with disabilities can use them. In the context of serious games, accessibility includes aids such as high-contrast or large font size, screen magnifiers, screen readers and reviewers, voice recognition software, keyboards, and adapted devices according to user needs.

The study [26] argues that application designers face a significant challenge in applying accessibility guidelines because they are complex and ambiguous. For this reason, the authors propose following the WCAG with a user-centered approach. They offer designers an innovative and engaging solution for broader adoption of accessibility guidelines that can create accessible games.

The study [27] indicates that games offer a traditional channel where all people can be involved; however, this only happens when creating inclusive design environments to achieve this challenge. Therefore, they suggest developing an accessibility vocabulary and language for games. The authors recommend including the terminology used, showing the controls that enable the play’s tasks and skills.

The investigation [28] indicates that play accessibility is about removing barriers for users with disabilities. Besides which, the authors argue that the challenge is that web-based games can better access the WCAG 2.1 and achieve an adequate accessibility level in the construct of more accessible games.

Previous studies by the study [29] indicate that it is essential to evaluate the users’ learning styles, for which they applied the Honey-Alonso test [30] for the development of serious games. The Honey-Alonso test includes active, reflective, theoretical, and practical techniques. The authors proposed constructing an educational video game with a Honey-Alonso test script and applied the WCAG.

The research [31] argues that making serious games universal and accessible are a challenge for interface creators and game designers. The authors indicate that more effort is needed and propose research with recommendations for accessibility in serious games. Consequently, the authors recommend ten strategies for designing games for visually impaired users. The results indicated that only three out of ten games were accessible.

Concerning accessibility, in June 2018, the World Wide Web Consortium (W3C) [32] established the WCAG 2.1 recommendations for making web content more accessible; furthermore, this proposal can be applied to achieve accessibility in serious games. WCAG 2.1 covers a wide range of recommendations to make web content more accessible. Applying [12] these guidelines will make serious game content more accessible to people with disabilities, including adaptations for blindness and low vision, deafness and hearing loss, limited movement, speech disabilities, photosensitivity, and combinations, and some adaptations for learning disabilities and cognitive limitations. WCAG 2.1 comprises four principles, 13 guidelines, 78 compliance criteria, combined with an undefined number of sufficient techniques and advisory techniques. The four principles are described below:

(1)

Perceivable, users must distinguish content in audio, video, and text.

(2)

Operable; users must be able to use and interact with the interface modules.

(3)

Understandable, users must be able to recognize self-controls of the interface.

(4)

Robust, users must be able to access comfortably and understand consistently in any case of the present and potential technologies.

Therefore, the compliance criteria include three levels. Level A offers a minimum of accessibility; not achieving implies that users cannot access the content. Level AA allows an intermediate level of accessibility; it satisfies the levels A and AA; it indicates that the users cannot access all the content. Furthermore, Level AAA is the highest level of accessibility. Achieving this level means that users can easily access the content.

2.3. Accessibility Evaluation Methods

Serious games and gamification approaches in the educational context [33] allows for greater student participation in skill development. Due to the growing multidisciplinary interest in learning, serious games are considered a useful pedagogical tool. Therefore, the need arises to build serious games accessible and inclusive to access these learning resources. This section describes several of the accessibility evaluation works, which contribute with tools that guide the design of web-based serious games.

The research [34] evaluates accessibility related to visual impairments in educational video games to suggest improvements in this type of application’s design. A second study by the authors indicates that the evaluation of mobile serious games’ accessibility for people with hearing disabilities argue that accessibility has not been considered in the development of serious games. As a case study, they evaluated a mobile game’s accessibility, where they proposed an analysis tool. In a third study, the investigation [35] argues that accessibility is a feature that is not considered in the design of serious games, so they are not easily accessible. The study shares a compilation of accessibility guidelines for developing video games; they propose categorizing the existing guidelines to evaluate a video game’s accessibility for people with motor disabilities.

A previous study by the authors [36] proposes a method to motivate software developers to improve accessibility. The authors present a technique for evaluating the accessibility of serious games applying WCAG 2.1, which incorporates barriers for users with cognitive disabilities. The proposed approach can be replicated for people with other disabilities and can be used throughout the game to create a cycle to guarantee more inclusive and accessible applications. A second study [37] proposes to apply a combined method to evaluate accessibility in serious games, taking into account the WCAG 2.1. As a case study, the accessibility of 82 serious games developed by Physical Education Technology Interactive Simulations at the University of Colorado was evaluated. The authors suggest replicating the combined approach for users with various disabilities, considering the accessibility barriers. As future work, they suggest generating a heuristic accessibility evaluation focused on serious games to develop applications more inclusive.

2.4. Heuristic Method

In the literature review for this research, we did not find any automatic review methods or heuristics applied to serious games, so we started from the existing information on evaluating the heuristics of accessibility for websites; we have the following studies. The study [38] argued that accessibility heuristics are created as a complement to accessibility guidelines. Additionally, heuristics allow evaluators to identify which sections of an application have accessibility issues effectively. The results revealed that using accessibility heuristics can locate a broader range of barriers to solving accessibility problems. However, heuristics do not provide evaluators with compliance information, so heuristics studies are not a substitute for user experience.

The research [39] argued that accessibility evaluation methods help user-centered accessibility. The author notes that accessibility evaluation methods can find accessibility issues, for instance, guideline infringements, modes of failure, defects, or user implementation rates. An evaluation method can prescribe how to rank and rate topics in conditions of severity and priority. It presents a discussion of several evaluation methods, one of which is the BW method, which has the following advantages: low complication, validates knowledge, more excellent correction than conformance review, and produces severity indices.

A study by the authors [17] argues that determining whether interfaces are accessible is essential, and it requires accessibility evaluation. One form of assessment uses automated tools, which are not sufficient; human testing with experts and users with disabilities is already necessary. The authors present a study to evaluate accessibility and barriers present on the application for Banco do Brazil employing the BW method, which records and defines a sequence of barriers that are susceptible to each classification of the user. This survey’s objective was to examine whether this approach can be treated as a supplement to automatic assessment when observations involving users cannot be made—the method allowed for rapid and timely evaluation, decreasing the assessment’s complication. With the suggested study, they detected usability problems associated with accessibility, for which they presented some suggestions.

The study [40] argued that accessibility assessment methodologies assume that all guidelines must be met to achieve universal accessibility. The authors noted that the BW approach addresses issues of applying the guidelines to diverse groups of users. The process allows evaluators to assess the severity of the barrier being inspected. They determined that barriers could be combined with the BW method to evaluate the impact of an application on users and effectively relate to the subject.

A previous study by the authors [16] indicated that there are qualitative and quantitative methods for checking whether an application is accessible. The authors presented a modification to the BW method offered by Giorgio Brajnik based on WCAG 2.1. The change involved incorporating persistence to work out the severity of an accessibility barrier. With the process used in the experiment, the evaluators found that the method allows for (1) measuring the accessibility of applications; (2) testing a heuristic process; (3) assisting in manual evaluations, and (4) contributing to studies associated with accessibility heuristics.

Therefore, the heuristic method based on the BW suggested in the studies [13,14,15] can assist in supplementing accessibility assessment in serious games. The heuristic method is a systematic method built on testing and error inspections in which an evaluator thinks of some potential accessibility barriers corresponding to the WCAG 2.1 accessibility principles. Barriers incorporate characteristics according to user type, purpose, the perspective of use, and serious gaming site to draw appropriate conclusions related to effectiveness, productivity, satisfaction, and security [13,14], with severity scores designated to each accessibility barrier.

In this study, the barriers are defined in the conditions of the next parameters: (1) Effectiveness indicates the degree of compliance to accomplish the user’s aim accurately; (2) Productivity is associated with the time, effort, and cognitive load prerequisite to achieving a specific level of effectiveness; (3) Satisfaction characterizes the contentment and acceptance by the user; it implies the capacity to acclimate, and (4) Security represents the established weakness in the serious game assessed. The BW method [13,15] is an accessibility inspection technique created from accessibility designed principles, taking into account the user group, the type of assistive technology, the influence of users, the types of serious games, and the effects caused.

The study [41] considers that inspection-based evaluation techniques are standard, as they involve less formal instruction. It applies a method that combines task-oriented heuristic assessments that are easy to understand and use. Sears recommends methods for checking the authenticity and consistency of the assessment techniques. The results of heuristic assessments allow for more significant problems to be identified than in cognitive reviews.

The severity of a barrier, according to the BW method [13,15], is related to the physical and cognitive characteristics of the user and the context in which the activities are performed; in this way, it is possible to reach conclusions with effectiveness, productivity, satisfaction, and security, to achieve the applicable parameters of severity. This method recommends utilizing two factors to assess the severity of a barrier: the impact of the barrier on satisfaction and productivity user and the persistence with which the barrier is present [15].

The BW method states that the evaluator can measure the severity of the barrier on a scale between one (1) and three (3), where the value of one (1) corresponds to a “minor problem,” implying that the user can overcome and avoid the barrier [13,42]. An identified barrier may affect productivity or user satisfaction but not effectiveness and security. A barrier that seriously affects a task’s execution is assigned the value of two (2) and corresponds to a “significant problem.” Finally, when it is difficult to avoid the barrier, security and effectiveness are reduced. The value of three (3) is assigned, implying a “critical problem,” in which context users often fail to achieve the objectives. Therefore, the barrier causes a negative effect that could affect (1) productivity and effectiveness; and (2) security and user satisfaction.

For this research, we start with the authors’ previous studies of authors [16] on websites’ heuristic methods. The authors apply the heuristic method based on the barrier path using ranges and severity scales to evaluate websites. In this research for the heuristic evaluation of accessibility in serious games, we apply the concept of heuristics suggested by the authors [43], which involves a method of inspection that supports an interactive evaluation of the system; three modifications to the BW heuristic method [13] are proposed by Giorgio Brajnik.

3. Method

The authors tested an initial part of this method in previous studies, the authors [16] related to evaluating accessibility heuristics in websites. Therefore, in this study, we applied a method that considers three variations used for the first time in the heuristic evaluation of accessibility in serious games. This method consists of prioritizing the impacts of the barriers corresponding to the applied perspective. With this heuristic method, we can identify a more significant number of accessibility barriers that prevent interaction between serious games and users.

Once these barriers are identified, they can be corrected by the developers of serious games. Using the detailed scale in

Table 1. Enlarged Scale.

Scale	Meaning
0	Null
1	Minor
2	Significant
3	Critical

, we applied three modifications to the BW method in this investigation; this contains the scale and the meaning of the severity with the first modification of the BW [14,15,16]; it consisted of expanding the scale to evaluate the efficiency, productivity, satisfaction, security, and impact.

The second modification affects persistence, which depends on the number of barriers presented in the evaluation; we apply the persistence scale detailed in

Table 2. Barrier–Persistence.

Barriers Number	Persistence
0 and 1	0
2 and 3	1
4 and 5	2
Greater than 5	3

. The third modification was to apply the severity according to the value of impact and persistence, shown in

Table 3. Impact, Persistence, and Severity value.

Impact	Persistence	Severity
0	0	Null
1	0	Null
2	0	Null
3	0	Null
1	1	Minor
1	2	Minor
1	3	Significant
2	1	Significant
2	2	Significant
2	3	Critical
3	1	Critical
3	2	Critical
3	3	Critical

.

The heuristic method to evaluate web-based serious games is condensed into eight phases, corresponding to Figure 2; each phase of the method is detailed in the case study.

4. Case Study

This method was applied to 40 web-based serious games, selected with a simple random probability sampling, from several websites for educational and learning purposes. The websites that contain the web-based serious games are shown in

Table 4. Websites where serious games have been obtained.

URL	Description
https://phet.colorado.edu/sims/html/	The site created by Physical Education Technology (PhET) Interactive Simulations at the University of Colorado [18], which contains several free serious games for educational use in different subjects
https://www.vascak.cz/	The simulations and animations of physics, electromagnetism, mechanics, gravitational field, molecular physics and thermodynamics, and particular relativity theory.
https://www.freeriderhd.com/	The random selection of various websites that contain serious learning games, including some of the most popular game lists.
http://www.raptjs.com/play/
http://www.sinuousgame.com
https://worldsbiggestpacman.com/
https://ludominga-aff09.firebaseapp.com/
https://funhtml5games.com/

.

In this research, we applied accessibility barriers for low vision users; the evaluators specified heuristics associated with impact and severity involving serious games. The evaluators previously raised the accessibility barriers based on the WCAG 2.1, applied according to the disability and user impact, characteristics, and effects of the serious games evaluated.

Phase 1: Select the serious games; in this phase, we used the formula for calculating the sample size when the population size is unknown. Before calculating the sample size, we need to determine: (1) the size of the population defined as a collection of objects or individuals that have similar characteristics; (2) the margin of error or confidence interval, which is a statistic that expresses the amount of random sampling error in the results; (3) the confidence level that refers to the random intervals that are used to constrain a value with a given high probability. For our case, we applied Equation (1).

$$n=\frac{Z_a^2 \ast p \ast q}{d^2}$$

(1)

where: n = size of the desired sample, Z = confidence level = 0.631, p = probability of success = 50%, q = (1 − p) probability of failure = 50%, d = accuracy, accepted estimation error = 5%.

When applying the calculations, we obtained a value of 39.8; therefore, a total of 40 serious games listed in

Table 5. List of selected serious games.

ID	URL	Subject
SG01	https://phet.colorado.edu/sims/html/acid-base-solutions/latest/acid-base-solutions_en.html	Chemistry
SG02	https://phet.colorado.edu/sims/html/area-builder/latest/area-builder_en.html	Math
SG03	https://phet.colorado.edu/sims/html/area-model-algebra/1.2.1/area-model-algebra_en.html	Math
SG04	https://phet.colorado.edu/sims/html/coulombs-law/1.0.9/coulombs-law_en.html	Physics
SG05	https://phet.colorado.edu/sims/html/energy-skate-park-basics/1.1.19/energy-skate-park-basics_en.html	Physics
SG06	https://phet.colorado.edu/sims/html/expression-exchange/1.1.14/expression-exchange_en.html	Math
SG07	https://phet.colorado.edu/sims/html/fractions-intro/1.0.12/fractions-intro_en.html	Math
SG08	https://phet.colorado.edu/sims/html/function-builder-basics/1.0.14/function-builder-basics_en.html	Math
SG09	https://phet.colorado.edu/sims/html/gravity-and-orbits/1.1.15/gravity-and-orbits_en.html	Physics
SG10	https://phet.colorado.edu/sims/html/masses-and-springs-basics/1.0.9/masses-and-springs-basics_en.html	Physics
SG11	https://phet.colorado.edu/sims/html/ph-scale/1.3.4/ph-scale_en.html	Earth science
SG12	https://phet.colorado.edu/sims/html/rutherford-scattering/1.1.9/rutherford-scattering_en.html	Chemistry
SG13	https://phet.colorado.edu/sims/html/vector-addition/1.0.0/vector-addition_en.html	Math
SG14	https://phet.colorado.edu/sims/html/waves-intro/1.0.2/waves-intro_en.html	Earth science
SG15	https://phet.colorado.edu/sims/html/build-a-molecule/latest/build-a-molecule_en.html	Chemistry
SG16	https://phet.colorado.edu/sims/html/density/latest/density_en.html	Chemistry
SG17	https://phet.colorado.edu/sims/html/energy-skate-park/latest/energy-skate-park_en.html	Physics
SG18	https://phet.colorado.edu/sims/html/number-line-integers/1.0.1/number-line-integers_en.html	Math
SG19	https://phet.colorado.edu/sims/html/molecule-polarity/1.0.15/molecule-polarity_en.html	Chemistry
SG20	https://phet.colorado.edu/sims/html/graphing-lines/1.3.10/graphing-lines_en.html	Math
SG21	http://www.raptjs.com/play/#/rapt/Bomberland/	Skills
SG22	https://funhtml5games.com/?play=angrybirds	Strategy
SG23	https://www.vascak.cz/data/android/physicsatschool/template.php?s=mech_sily&l=en	Physics
SG24	https://www.vascak.cz/data/android/physicsatschool/template.php?s=mech_pohyb&l=en	Physics
SG25	https://sumonhtml5.ludei.com/	Math
SG26	http://arcade.lostdecadegames.com/onslaught-arena/	Skills
SG27	https://worldsbiggestpacman.com/play/#-1,1	Skills
SG28	https://www.freeriderhd.com/t/727781-the-great-oak	Racing
SG29	http://www.sinuousgame.com/	Skills
SG30	https://ludominga-aff09.firebaseapp.com/boingBoingGame	Skills
SG31	https://ludominga-aff09.firebaseapp.com/facesAndGesturesGame	Skills
SG32	https://ludominga-aff09.firebaseapp.com/huntingFiguresGame	Skills
SG33	https://ludominga-aff09.firebaseapp.com/puzzleGame	Skills
SG34	https://www.vascak.cz/data/android/physicsatschool/template.php?s=mech_rovnobeznik&l=en	Physics
SG35	https://www.vascak.cz/data/android/physicsatschool/template.php?s=mech_pruzna&l=en	Physics
SG36	https://www.vascak.cz/data/android/physicsatschool/template.php?s=mech_maxwell&l=en	Physics
SG37	https://www.vascak.cz/data/android/physicsatschool/template.php?s=gp_newton_zakon&l=en	Physics
SG38	https://www.vascak.cz/data/android/physicsatschool/template.php?s=gp_sourad_geol&l=en	Physics
SG39	https://www.vascak.cz/data/android/physicsatschool/template.php?s=mf_parnistroj&l=en	Physics
SG40	https://www.vascak.cz/data/android/physicsatschool/template.php?s=opt_newtonkotouc&l=en	Physics

should be evaluated, containing the ID that is the identifier assigned to the web-based serious game, the URL, and the subject of the serious game.

Phase 2: Characterize the type of users; in this experiment, the guide for low vision users [44] was applied, who are characterized as those who have a situation in which the user’s vision cannot be rectified with glasses, affecting everyday events, reading, and driving. Low vision is most frequent among the elderly but can occur in people of any age because of degenerative diseases.

The WHO defines low vision [45] as a visual abnormality that restricts the ability to perform everyday visual tasks. The leading causes of visual impairment are uncorrected refractive errors, cataracts, age-related macular degeneration, glaucoma, diabetic retinopathy, corneal opacity, and trachoma. The fundamental principle of accessibility for users with low vision is “perceivable” [12].

Phase 3: Define user activities; the evaluators define scenarios to analyze the serious game and how users interact with the game to achieve the objective. The task was to enter each serious game, review each game’s functionality, check the menus, images, messages, and help options to see any accessibility barriers for low vision users. Additionally, in this phase, we describe the level of accessibility (A, AA, AAA) that would be assessed; in this situation, we operate up to the level AA. It was also necessary to detect accessibility support with a listing of web browsers that must be friendly. In this investigation, the evaluators used Google Chrome version 84.0.4147.125, Mozilla Firefox version 77.0.1, and Opera version 70.0.3728.106.

In this case study, a barrier for a user with low vision means that he or she cannot interact effectively with the serious game, implying that it is difficult for him or her to move precisely over the content of the serious game due to the complexity of his or her visual perception. The two evaluators identified the scenarios made up of users with low vision, the aims, and the activities that users must complete in the testing to apply the approach suggested in this investigation. A list of accessibility barriers for low vision users based on WCAG 2.1 [12] is listed in

Table 6. Barrier vs. Web Content Accessibility Guidelines 2.1 (WCAG 2.1.)

ID	Barrier	WCAG 2.1	Success Criteria	Level
G01	Accessible keyboard	Operable	2.1.1	A
G02	No luminance flash failures	Operable	2.3.1	A
G03	Animation from Interactions	Operable	2.3.3	AAA
G04	Content hovering over focus	Perceivable	1.4.13	AA
G05	Easy to read font	Perceivable	1.4.4	AA
G06	Text alternatives	Perceivable	1.1.1	A
G07	Subtitled	Perceivable	1.2.4	AA
G08	Automatic transcripts	Perceivable	1.2.5	AA
G09	Sign language	Perceivable	1.2.6	AAA
G10	Information and relationships	Perceivable	1.3.1	A
G11	Adjust display settings	Perceivable	1.3.4	AA
G12	Interface rearrangement	Perceivable	1.3.5	AA
G13	Use of color	Perceivable	1.4.1	A
G14	Contrast without text	Perceivable	1.4.11	AA
G15	Well-spaced elements	Perceivable	1.4.12	A
G16	Good audio techniques	Perceivable	1.4.2	A
G17	Contrast	Perceivable	1.4.3	AA
G18	Images as sharp as possible	Perceivable	1.4.5	AA
G19	Visual presentation	Perceivable	1.4.8	AAA
G20	Pause, stop, hide	Perceivable	2.2.2	A
G21	Sensory Characteristics	Perceivable	1.3.3	A
G22	Screen reader support	Robust	4.1.2	A
G23	Status messages	Robust	4.1.3	AA
G24	Language	Understandable	3.1.1	A
G25	Consistent navigation	Understandable	3.2.3	AA
G26	Labels or instructions	Understandable	3.3.2	A
G27	Help	Understandable	3.3.5	AAA
G28	On Focus	Understandable	3.2.1	A

. The evaluators recommended this method to accomplish the aim, considering effectiveness, productivity, security, and user satisfaction. It was essential to recognize the scale of severity and range of persistence of each barrier that characterizes the low vision user’s impediment to achieve the aim.

Phase 4: Explore the serious games; the expert explores each serious game with the instructions defined in phase 3. The tasks that the user had to perform were: (1) interact with the first interface of the serious game; (2) review the functionality of each round; (3) identify whether the games have a screen magnifier; (4) review whether the games have the option to adjust the game parameters; (5) determine whether the game has the opportunity to help the user; (6) identify whether the user can read each message of the serious game; (7) review whether the images have adequate clarity and are apparent to the user; and (8) check whether the serious game has the option to change the language.

At this stage, the evaluators navigate all the serious game features to detect potential accessibility barriers. Finally, in this phase, the evaluators review the behaviors agreeing to the device used, the browser, and the structure applied.

Phase 5: List the barriers; we selected 28 barriers based on WCAG 2.1 for low vision users to apply this BW-based heuristic method. Table 6 contains the guideline identifier, the barrier, the WCAG 2.1 principle, the success criteria, and the level of accessibility.

Phase 6: Evaluate with the heuristic method, the evaluators carried out the following process: (1) run the serious game; (2) carefully examine whether each barrier meets the parameters of effectiveness, productivity, security, and satisfaction; we applied Table 1. (3) Find the impact related to the parameters defined in Table 1; (4) record the persistence of each barrier according to the parameters defined in Table 2; (5) apply the persistence scale defined in Table 2; (6) finally, apply the severity scale according to the impact and resulting persistence defined in Table 3.

To analyze the severity of a barrier, we need two parameters: (1) the impact of the barrier as a result of the effectiveness, productivity, security, and satisfaction of the user performing an activity; (2) the value of persistence, denoted by the number of times the barrier is repeated, is essential when evaluating a serious game.

For example, when checking the barrier “Accessible keyboard” in the game SG01 “Acid-base solutions”, we apply the following process: First, for the efficiency parameters, we register the value zero (0), in productivity the value of one (1), in satisfaction zero (0) and security the value of one (1). Second, we analyze whether the barrier affects productivity, satisfaction, effectiveness, or safety. In that case, the impact was minimal; when analyzing these parameters, the impact’s value is two (2). Thirdly, we evaluate the number of times the barrier corresponding to the persistence parameter presented by the barrier “Accessible keyboard” is presented. Fourth, we apply the persistence scale defined in Table 2. Fifth, we determine the severity defined in Table 3 for impact and persistence; for the barrier “Accessible keyboard” we have in the impact the value of two (2) and for the persistence the value of two (2), therefore the severity for the barrier “Accessible keyboard” is significant.

Phase 7: Record the data; the data from each serious game evaluated was documented in a spreadsheet.

Table 7. Evaluation with the BW heuristic method for the serious game SG01 “acid-base-solutions”.

ID	Barrier	E	P	St	Sc	I	P1	P2	P3	P4	Pt	P	Severity
G01	Accessible keyboard	0	1	0	1	2			4		4	2	Significant
G02	No luminance flash failures	1	0	1	2	1		2			2	1	Minor
G03	Animation from Interactions	1	0	1	2	1		3			3	1	Minor
G04	Content hovering over focus	0	0	0	0	3				6	6	3	Critical
G05	Easy to read font	1	0	1	2	1		2			2	1	Minor
G06	Text alternatives	0	1	0	1	2			5		5	2	Significant
G07	Subtitled	0	1	0	1	2			4		4	2	Significant
G08	Automatic transcripts	0	1	0	1	2			4		4	2	Significant
G09	Sign language	0	1	0	1	2			5		5	2	Significant
G10	Information and relationships	1	0	1	2	1		3			3	1	Minor
G11	Adjust display settings	1	1	1	3	0	0				0	0	Null
G12	Interface rearrangement	1	1	1	3	0	1				1	0	Null
G13	Use of color	1	0	1	2	1		2			2	1	Minor
G14	Contrast without text	1	0	1	2	1		2			2	1	Minor
G15	Well-spaced elements	1	1	1	3	0	0				0	0	Null
G16	Good audio techniques	0	1	0	1	2			5		5	2	Significant
G17	Contrast	0	1	1	2	1		2			2	1	Minor
G18	Images as sharp as possible	1	1	1	3	0	0				0	0	Null
G19	Visual presentation	1	1	1	3	0	0				0	0	Null
G20	Pause, stop, hide	0	1	1	2	1				6	6	3	Significant
G21	Sensory characteristics	1	0	1	2	1				6	6	3	Significant
G22	Screen reader support	1	0	1	2	1				6	6	3	Significant
G23	Status messages	0	0	1	1	2				6	6	3	Critical
G24	Language	0	0	1	1	2				6	6	3	Critical
G25	Consistent navigation	1	1	1	3	0	0				0	0	Null
G26	Labels or instructions	0	0	0	0	3				6	6	3	Critical
G27	Help	0	0	0	0	3				6	6	3	Critical
G28	On focus	0	0	0	0	3				6	6	3	Critical

Where: E = Effectiveness, P = Productivity, St = Satisfaction, Sc = Security, I = Impact, P1 = 0 and 1, P2 = 2 and 3, P3 = 4 and 5, P4 > 5, Pt = Persistence total, P = Persistence.

is an example since it collects the data of only one set; it contains the identifier assigned to the pattern, the barrier, the effectiveness, the productivity, the satisfaction, the safety, the impact, the range of persistence, the total persistence, the result of persistence and the severity of the serious set with the code SG01 that corresponds to “acid-based solutions” of the PhET project of the University of Colorado. For the G01 “Accessible keyboard” barrier, we applied Table 1 and obtained the following results: in effectiveness we recorded zero (0), in productivity one (1), in satisfaction zero (0), and in safety one (1). Then we analyzed the impact; if the barrier affects productivity or satisfaction marginally, but not effectiveness or safety, the impact is minimal; when analyzing these parameters, we found impact equal to two (2) according to the scale of Table 1 this corresponds to significant. To evaluate persistence, we apply Table 2; we find that the barrier is present for more than four times; therefore, persistence is equal to two (2). Finally, we applied Table 3 to find severity, with the impact value equal to two (2) and persistence equal to (2); therefore, this barrier presents significant severity.

The recorded data are available in the repository [46] so that the assessment can be reproduced. Data recording is essential; the data allow evaluators to manage and condense information by diverse categories.

Phase 8: Interpret and analyze the data; the evaluators offer the analysis of the severities taken when applying the heuristic method; First, we organize and group the information by categories. To manage each serious game’s severity information, we use the dendrogram shown in Figure 3; it contains the distances between each serious web-based game evaluated consecutively. In the results section and the discussion section, a more in-depth analysis of the results related to the severity of the barriers, the accessibility principles, the levels, and the success criteria of the WCAG 2.1 is carried out.

5. Results

Table 8. Summary of the severities of the serious games evaluated.

ID	Null	Minor	Significant	Critical
SG01	6	7	9	6
SG02	14	0	1	13
SG03	14	1	0	13
SG04	14	3	0	11
SG05	11	3	0	14
SG06	14	1	0	13
SG07	13	1	0	14
SG08	13	2	1	12
SG09	17	2	0	9
SG10	14	1	3	10
SG11	14	2	1	11
SG12	14	3	0	11
SG13	12	2	1	13
SG14	16	0	0	12
SG15	12	4	1	11
SG16	12	1	1	14
SG17	15	1	0	12
SG18	14	1	0	13
SG19	13	2	0	13
SG20	14	0	1	13
SG21	5	3	5	15
SG22	15	0	2	11
SG23	11	0	3	14
SG24	12	0	2	14
SG25	10	2	3	13
SG26	11	3	1	13
SG27	12	4	0	12
SG28	3	1	6	18
SG29	7	4	2	15
SG30	12	3	1	12
SG31	12	3	1	12
SG32	12	3	1	12
SG33	12	3	1	12
SG34	11	0	2	15
SG35	11	0	1	16
SG36	11	0	1	16
SG37	10	1	1	16
SG38	10	1	1	16
SG39	10	1	1	16
SG40	10	1	1	16
Total	473	70	55	522

summarizes the 40 serious games evaluated; it contains the serious games’ code, the severity of null, minor, significant, and critical.

The results obtained in this case study show that the total null severity has a value of 473, representing 42.2% of the total. The minor severity corresponds to the amount of 70, representing 6.3%. The significant severity is 55, which corresponds to 4.9%; finally, the critical severity has a value of 522 and represents 46.6%. A summary of WCAG 2.1 principles and the severity of the serious games’ barriers are presented in

Table 9. Summary of WCAG 2.1 principles and severity of barriers.

Principle	Null	% Null	Minor	% Minor	Significant	% Significant	Critical	% Critical
Perceivable	302	63.8	53	75.7	23	41.8	342	65.5
Operable	57	12.1	14	20.0	12	21.8	37	7.1
Understandable	88	18.6	3	4.3	6	10.9	103	19.7
Robust	26	5.5	0	0.0	14	25.5	40	7.7

.

6. Discussion

Of the 40 serious games evaluated, we found that:

(1)

The SG01 “acid-base solutions” set has a value of six in the critical severity evaluation.

(2)

The SG09 “severity-and-orbits” set has nine and corresponds to the group’s most accessible serious games with less critical barriers.

(3)

The two web-based serious games correspond to the University of Colorado’s PhET project.

The serious games with the highest number of critical barriers correspond to SG28 “freeriderhd” with a value of 18, followed by SG35 “physicsatschool” with 16. The results indicate: (1) The most significant number of serious games is in the critical and significant severity rating; (2) For serious games to be accessible at an acceptable level, application programmers must correct the most significant number of accessibility barriers through serious game programming software; and (3) the alternative is to use adaptive hardware by applying assistive technology or digital ramps that allow users to interact with serious games easily.

Furthermore, to find similarities between the serious games evaluated with the heuristic method, we apply the hierarchical grouping that allows the finding of similarities between the serious games accommodated in groups so that the groups are well separated and that the serious games within them are as close as possible. The similarity between the serious games is graphed using a tree called a dendrogram. In this process, we apply the following steps: (1) the number of clusters or groups is formed according to the number of serious games; (2) the serious games are grouped based on a measure of similarity between them; (3) the groups are grouped based on their distance, forming a tree called a dendrogram; (4) they continue to group until only one grouping remains.

Figure 3 shows the dendrogram of the 40 serious games’ evaluations with the heuristic method considering the null, minor, significant, and critical variables to know the groups that obtained similar characteristics among the serious games. To measure the similarity between the serious games, we applied the Euclidean distance (2).

$$d\left(x,y\right)=\Vert x-y\Vert \sqrt{{{\displaystyle \sum }}_{i=1}^n{\left(xi-yi\right)}^2}$$

(2)

The Euclidean distance method allows us to see how similar the evaluation data are for each serious game. The dendrogram in Figure 3 shows the height of the branches that indicates the similarity between the clusters, and contains five horizontal lines in green indicating the number of groups into which they are divided; in this case, it was divided into six groups containing similar values, which are detailed below:

(1)

The serious game SG01 is in a single group because it records low values in null, minor, significant, and critical; this implies that SG01 is the most accessible serious game of the 40 evaluated.

(2)

The SG28 serious game is in the second group. It is still not as accessible because the critical severity is 18, indicating that it contains some accessibility barriers that must be corrected even though null, minor, and significant are low.

(3)

The two serious games, SG21 and SG29, are in the third group are not very accessible because they have a critical severity of 15.

(4)

The five serious games, SG10, SG22, SG09, SG14, and SG17, are in the fourth group and are not very accessible because they have a critical severity.

(5)

The 12 serious games SG05, SG23, SG24, SG25, SG26, SG34, SG35, SG36, SG37, SG38, SG39, SG40 have a high critical severity and are therefore not accessible.

(6)

The 19 serious games SG02, SG03, SG04, SG06, SG07, SG08, SG11, SG12, SG13, SG15, SG16, SG18, SG19, SG20, SG27, SG30, SG31, SG32, SG33 are among the least accessible.

Figure 4a shows that the examined critical games violate some principles of WCAG 2.1.

In the 40 evaluated serious games, the evaluators found: (1) 342 barriers with critical severity, where 65.5% characterize the perceptible principle. (2) 37 barriers with critical severity represent 7.1% of the operable. (3) 19.7% corresponding to the understandable, and (4) 7.7% corresponding to the robust. These statistics indicate that accessibility barriers affect web-based serious games and should be corrected.

Figure 4b shows a summary of the accessibility levels corresponding to critical severity; we observe that for level A, 210 barriers representing 40.2%, for level AA, 227 barriers corresponding to 43.5%, and level AAA has 85 barriers corresponding to 16.3% of the total.

Figure 5 presents the success criteria for critical severity; these criteria can be found in Table 7; we observe that:

(1)

The on-focus barriers correspond to criterion 3.2.1. The help barrier of the success criterion 3.3.5 is the most critical with a value of 40.

(2)

It is followed by the content hovering over the focus barrier of criterion 1.4.13, subtitled criterion 1.2.4, and the automatic transcripts barrier of criterion 1.

(3)

With the value of 38, the barriers text alternative is sign language.

(4)

Through 37, the criterion status messages.

(5)

By 36, interface rearrangement.

(6)

With 32, we have adjusted the display settings and pause, stop, hide.

(7)

With 30, the criterion accessible keyboard.

(8)

Together with 23, the criterion useful audio techniques.

(9)

The rest of the criteria have values lower than 12.

We suggest incorporating solutions using digital ramps or software and hardware assistance technologies to have serious games accessible. We propose to include support tools that help users with sensory, motor, and visual disabilities with configuration options to enable an environment controlled by the user of the serious game. We propose that the serious game contains (1) A tool to control the serious game’s video, the subtitles, and playback. (2) A tool to support motor, sensory, and cognitive disabilities.

The tool in Figure 6 suggested should contain (1) an option to control the serious game’s execution that allows the configuration of subtitles, descriptions, keyboard, and automatic transcription; (2) the subtitle settings should include options to manage the position, font, font size, text color, background, and opacity; (3) an option to configure the serious game’s speed, subtitles, and audio replay in the application. The user must have the control to manage and customize the speed of video, subtitles, and audio according to the user’s disability.

Figure 7 suggests including a tool to support users with disabilities that allows voice commands, and the option for them to use the VoiceOver application that narrates what happens on the screen of any device; voice assistants are a great help for users with visual or motor disabilities. Another option is to include a chatbot based on artificial intelligence, which feeds the database as the user learns.

A system can be included to transcribe the video into text or sign language and transform the text into sign language for deaf users.

People with motor disabilities usually read the information presented by computer output devices through the monitor or printer without difficulty, but often have difficulty handling input devices such as the keyboard and mouse. For people with motor disabilities, filters can be incorporated to facilitate the use of the keyboard and mouse by a trackball, joystick, buttons, and custom devices.

As additional add-ons, serious games can include changing the language to customize and choose the audio description and subtitles’ language. Although sign language is not similar globally, this option should be included to help deaf and dumb users and feed a database with each region’s sign language. Therefore, assistive technologies have become an essential resource to overcome the barriers of access to digital technologies that positively impact people’s quality of life with disabilities.

7. Conclusions

This heuristic method can be used to evaluate the accessibility of any serious web-based game. One of these heuristic methods’ contributions is to evaluate each barrier by the parameters of effectiveness, productivity, satisfaction, and security related to impact and persistence. We recommend reproducing this heuristic accessibility evaluation method for any serious game considering the accessibility barriers that correspond to the type of motor, cognitive, or sensory disability. The evaluators identified some critical severity barriers, including the following two. The first barrier with the highest critical severity is the focus, and the second barrier is the help, both of which are related to the understandable principle.

These barriers occur when no contextual help is available, and any user interface component receives the focus but does not initiate a context change. Consequently, the evaluators recommend: (1) Applying techniques and tools to reduce the barrier help’s critical severity can use clear labels that can act as contextual help. (2) To reduce the on-focus barrier’s severity, the user interface components can be implemented as a programmable element to perceive the parts as separate controls. To assess the accessibility of serious games, evaluators recommend fusing automatic assessment tools with heuristic methods. One must remember that no instrument can substitute manual assessment by a serious game accessibility expert. The proposed method can be tested with serious game developers throughout the software development cycle to identify new barriers and correct them as they are built to achieve more inclusive applications.

Future research may propose new methods of evaluating multimedia resources concerning videos and sound recordings; in both cases, a transcript of the dialogues, a description of the sounds, and control of the playback speed should be provided. However, inappropriate use of multimedia elements may create a barrier to user access. To achieve more accessible serious games, we suggest testing with users with sensory, motor, and cognitive disabilities to identify the barriers users face with disabilities.

Future work should continue to refine heuristic methods related to serious games and socialize best accessibility practices, designing software that helps experts evaluate serious games’ accessibility by considering this heuristic method and the WCAG 2.1. Furthermore, we propose to survey serious game developers to find out if they know the WCAG 2.1 and if they have tried to apply some of its guidelines to design inclusive products that make an essential contribution in the field of accessible serious games.