To build the case studies, a group of seven people participated in the simultaneous development of applications following MEEXU2 methodology. Each participant was assigned multiple roles according to their skills and knowledge. In this way, an interdisciplinary group was formed, consisting of graphic designers, programmers, experts in the learning area, 3D designers and testing and evaluation. Below are the case studies carried out in this project.
4.1. Technical Aspects and Common Characteristics
Unity3D, an engine for videogame creation that has grown significantly in recent years, was used to build both applications [
48]. A tangible interface to “Learning with Pesos” was built with a 30 cm ∗ 40 cm piece of acrylic, a Logitech 1080 p webcam, two 2-inch iron presses, and LED lighting (see
Figure 4a). In addition, reacTIVision 1.5.1, an open source, cross-platform computer vision framework, was used for fast and robust tracking of fiducial markers attached to physical objects, as well as for multi-touch finger tracking. The tangible objects used are composed of a fiducial marker used to identify the object within the application and a coin attached front and back (
Figure 4b).
Moreover, virtual reality glasses Oculus Quest 2, developed by META, were used to create a “Street Simulator” application, and Oculus integration assets were implemented to create the virtual interactions.
Although in both case studies, different technologies were used (TUI and XR), they share common characteristics, as the development methodology and the activities to implement the gamification mechanics are the same (see
Table 1). This shows that the same gamification elements can be used for different technologies and disabilities to motivate users.
4.2. Case Study 1: Application for Teaching the Mexican Currency to Blind People
Blind people face a number of visual challenges every day, from reading the label on a frozen dinner, to figuring out if they are at the right bus stop, to even identifying if they are paying with the right coin [
49]. For this reason, when making different forms of coins and banknotes, it is important to take into account the fact that there are people who are visually impaired, but still need to understand the different values of the banknotes that exist [
50]. Considering the importance of knowledge of currencies, the first case study proposes an application to teach currencies called “Learning with Pesos”, whose objective is to teach the Mexican currency using tangible interfaces, gamification mechanics and the UDL. We decided to use tangible interfaces with the objective of taking advantage of the main means of interaction of blind people, their hands. In addition, through TUI, innovative educational and cognitive interventions can be maintained to listen and stimulate the narratives of visually impaired people [
23]. Additionally, there are multiple projects which demonstrate favorable results in improving the learning of blind people with concepts of shadows, programming, music and Braille, among others [
51,
52,
53,
54].
The application is composed of two main sections (see
Figure 5a); the first section has the objective of showing descriptive information about the different coins/bills of the Mexican currency. for this, the user places an object with a coin attached to it on the tangible interface, then the application recognizes it and provides information through audio about its physical characteristics and the amount it represents, so that the blind person can identify it more easily (see
Figure 5b).
The second section consists of a series of mathematical problems (addition and subtraction) that allude to possible events that the blind person could face in the real world, with the aim of improving their fluency in identifying Mexican currencies. In these sections it is possible to unlock different achievements by answering correctly and using different tangible objects.
Figure 6a shows how the 2-peso badge is obtained after correctly answering an exercise, and
Figure 6b shows the rewards interface where the achievements and their description are displayed; in the case of blind people these achievements are described through audio.
To select mechanics and dynamics, we followed the gamification techniques selection process shown in
Section 2, as described below.
What are user characteristics? Users for whom this application is intended have visual problems, such as total blindness or partial loss of sight, their sense of touch and hearing being their main means of interaction.
Which gamification dynamics and mechanics are the most suitable for the user according to their characteristics? Based on research that has previously used gamification techniques with blind children, the most appropriate dynamics are levels, challenges and missions, while the use of rewards as mechanics is considered appropriate [
4,
55,
56,
57].
How can we adapt these techniques according to user characteristics? Considering that their main means of interaction are the senses of touch and hearing, we intend to take advantage of the latter by adding a representative sound to each gamification element for its quick identification by blind people.
Challenges are oriented to the overcoming of different activities in a timely manner, encouraging the user to achieve as many challenges as possible;
Table 2 shows some of the established challenges.
Additionally, missions were created that are unlocked by completing different challenges;
Table 3 shows some of the established missions.
With this application we demonstrated the use of gamification elements for blind people through different mechanics such as levels, challenges, missions and rewards. For each participant we provided a representative sound facilitating its identification by the blind person. In addition, with the use of the tangible interface, we facilitated the manipulation of the different coins; in this way, the users were able to interact with them and identify the characteristics mentioned in the application for easy recognition through their hands. Finally, with the UDL we present multiple means of representation, such as audio, text and images, to reach more users.
Evaluation
A usability evaluation was conducted to measure the satisfaction of blind people when using the application. The instrument used was system usability scale (SUS) [
58], which has ten statements related to the use of the system where users must indicate on a scale of 0 to 5 whether they totally disagree or totally agree with each of the statements. Three blind people of 31, 44 and 45 years old belonging to the Sistema Nacional para el Desarrollo Integral de la Familia (National System for the Integral Development of the Family), Aguascalientes (DIF) were recruited; this number is within Nielsen’s recommended range for getting the most out of usability issues, which recommends three to five participants, since a higher number does not provide much additional information [
59].
At the beginning of the activity, they used the application with the assistance of a person who showed them where to place the tangible objects; later, they used it independently, and after some time the instrument was applied (see
Figure 7a,b).
An average result of 91.7 was obtained. A result higher than 68 on the SUS indicates that the level of satisfaction is acceptable.
Table 4 shows the age and the results obtained by each participant.
4.3. Case Study 2: Application for Behavioral Rehabilitation and Transit Development for People with Autism Spectrum Disorder (ASD)
Children with autism spectrum disorders (ASD) experience difficulties in social skills and may find understanding other people’s nonverbal cues and social behaviors a challenge [
60]. Commonly, the condition is classified as high-functioning autism in those with autism spectrum disorder without intellectual disability [
61], and as low-functioning autism in those who are unable to follow instructions, or who do not let anyone touch them; they may be aggressive or show unreasonable behavioral reactions that are difficult to explain [
62,
63]. However, if given enough attention and therapy, they can overcome their symptoms. Several notable people, such as Albert Einstein (scientist), Bobby Fischer (Chess Grandmaster) have had ASD and success in their lives [
6]. Moreover, in children with ASD, we find an absence in the acquisition of notions of space and time; consequently, they may be subjected to experiences where their physical safety is put at risk, such as walking along edges at heights, or crossing streets without caution [
64].
With the aim of providing a rehabilitation proposal to help children with ASD to better cope in the environment around them, “Street Simulator” was created, a VR video game with gamification mechanics and UDL in which they follow instructions provided by avatars to reach a particular destination (simulating the real world). It was decided to use VR because it can bring many advantages to the treatment of ASD symptomatology [
8]. In addition, several studies have shown significant improvements in various cognitive indices, such as task learning, attention, executive functioning and daily living skills with the use of VR; therefore, it can be successfully used as an educational tool for children with ASD [
65,
66,
67].
This game is composed of a main screen with the options of starting the game, modifying options, and exiting (see
Figure 8a). Once the game is started, a story is displayed so that the child with ASD knows the destination to which he/she must go (see
Figure 8b).
Once inside the environment, instructions are provided through audio and text to offer different means of representation to access the information. In the virtual environment, it is possible to encounter different avatars representing police officers who give instructions to reach the established objective when the user approaches them (see
Figure 9a,b).
To complete the activity, children with ASD have a time limit of three minutes; if they do not reach the indicated destination in that time, the activity is restarted. This time is not shown at the time of using the game; it is shown at the end of the activity so the user may know how long it took to complete it. Other restrictions within the environment are that the child with ASD cannot cross the street unless he/she is in a corner; in the case that this is ignored, a warning is issued, indicating that this is forbidden. Once the child with ASD reaches his goal, visual and auditory feedback is given according to their performance (
Figure 10a), and if the child does not reach the established place, encouragement is given to keep trying (
Figure 10b).
To select the mechanics and dynamics, the gamification techniques selection process shown in
Section 2 was followed, as described in the following section.
Challenges and missions were established for this video game to encourage the child with ASD to complete them and continue interacting with the virtual environment. Some of the challenges established are shown in
Table 5.
Additionally, missions were created that are unlocked by completing different challenges;
Table 6 shows some of the established missions.
Through the “Street Simulator” application, we demonstrate the use of gamification for people with autism and its advantages to encourage learning and use of virtual environments; the use of challenges, missions, and rewards (stars according to their performance) makes the environment more interactive, facilitating its main function, which is that the child with ASD manages to function better in the environment that surrounds them. UDL use is observed in the representation media used, such as audio and text. These media were chosen because some children with ASD can read; however, for those who cannot, instructions can be received through audio and text.
Evaluation
For this preliminary evaluation, we conducted an evaluation of “Street Simulator” with a person diagnosed with high-functioning autism since the age of four with a higher-than-average IQ (see
Figure 11a). The objective of this evaluation was to observe their behavior with the virtual reality technology, their interaction with the environment and how well they were able to complete the mission. The parameters for this evaluation were qualitative in nature, as they were based on the observation, opinions and experiences of the child with autism. Their actions within the virtual environment were monitored by our development team through a computer (see
Figure 11b).
It is important to mention that prior to this evaluation, the child with ASD had not experienced the use of VR, which is why they were puzzled and disoriented in their first experience with the application. The result of their first attempt was unsuccessful, since they exhausted the starting time, ignoring the indications indicated by the avatars; however, this first attempt allowed us to observe that the individual developed a quite acceptable control ability within the virtual environment, considering that it was their first experience with VR. In the second test, the person successfully completed the mission, and a developed orientation skill was observed, since they did not need to interact with the avatars to reach the established point. As a conclusion of this evaluation, it is observed that the use of VR attracted the attention of the child with ASD; therefore, it can be a great tool to subject them to different virtual scenarios and improve their interactions with the real world in decision making. Finally, it is important to mention that as this is a preliminary evaluation, tests with more individuals are necessary to confirm what is stated here.
4.4. Implementation of Universal Design for Learning
The UDL aims to improve and optimize teaching and learning for all people [
9]. Its application in both case studies is observed in the proportion of different means of representation to provide the information to our target user; however, these means of representation indirectly help the case studies to be accessible to different types of people (see
Figure 12).
On one side, in “Learning with Pesos”, blind people receive the information through audio; indirectly, this means of representation can help people who cannot read or write or have motor difficulties to access the information of the application. Likewise, information is shown through text that, although is not accessible to blind people, is accessible to people with hearing problems or motor disabilities who have the ability to read. In addition, the use of physical objects within the tangible interface can help people who are blind, deaf or unable to read or write to use the application.
On the other hand, in “Street Simulator”, audio and text are used to provide information, so in case the person with ASD does not know how to read, he/she can receive the information through audio and follow the established dynamics. In both cases, it was decided to use gamification as a means of engagement due to the multiple missions, rewards and challenges established to encourage the use of the applications in a fun way to generate learning that can serve users in their daily lives.