*4.1. Overview*

In this paper, we propose an auditory card game system that presents the card's contents by sound so as to make a game accessible for people with visual impairments. We developed a game called "Auditory *Uta-Karuta*" by using the proposed system, and conducted an evaluation experiment in which participants played against each other. The purpose of this experiment is to test the applicability of auditory card game system to board games. For this purpose, the game performance of three different types of players was revealed thorough the experiment. Since the proposed system assumes that people with visual impairments can participate in a board game together with sighted people, we prepared sighted people and people with visual impairments as experimental participants. In addition to those, we also prepared sighted players with blindfolds and conducted a game between sighted players and sighted players with blindfolds to clarify how much their game performance is affected due to the presence of vision. Furthermore, by examining a game played between a visually impaired person and a sighted person wearing a blindfold, it was possible to examine the difference in auditory abilities between people with visual impairments and sighted people. In this experiment, since the participants were required to touch/grab audible cards, the sighted participants who were able to recognize the positions of the audible cards through their vision were considered to have an advantage in the card-grabbing situation. On the other hand, visually impaired participants spend their daily lives using their hearing ability and, thus, have a higher capacity for recognizing auditory information than sighted participants. "Auditory *Uta-Karuta*" does not present visual contents to all of the participants, and it is necessary to recognize the audible cards mainly with one's hearing ability. As mentioned above, while sighted participants are expected to be better at touching/grabbing the audible cards, visually impaired participants are expected to be better at detecting the correct audible cards due to their high auditory abilities. Therefore, an evaluation experiment was conducted to find

the extent to which the respective advantages and disadvantages of people with visual impairments and sighted people affect the game's results. As a comparison, we prepared Braille *Karuta*, which is representative of existing tactile presentation methods.

### *4.2. Stimuli and Equipment*

For each audible card, an iPod touch (6th generation) was used. Six audible cards were placed horizontally in a row between two participants, with a distance of 70 mm between each card. No information was shown on the screen of the audible cards, and the players had to obtain information only by hearing. The two participants sat face to face with a distance of about 1 m between them. The sounds used as stimuli were consisted of six different animal crying sounds obtained from [30]. All sounds were 5 s in length at Phase 1 to ensure clarity of the card's contents and positions. Sounds were sampled at 44.1 kHz with 16-bit resolution, normalized for amplitude across conditions, and presented in mono. The experiment was programmed in Swift and presented using the speaker of the iPod touch (6th generation).

We used bridge cards (56 mm × 88.9 mm), one of the common standards for playing cards, as the Braille cards for comparison. These cards had animal names written in Braille and Japanese characters on both ends of the card so that participants could understand them, whether they have a visual impairment or not. They were placed horizontally in a row between two participants, in the same position as the audible cards.

There were 18 participants (14 males and four females) aged from 19 to 42 years old. Among the participants, there were six people with clear vision (four males and two females), six people with visual impairment (five males and one female), and six people with a blindfold (five males and one female). All visually impaired participants could read Braille. In contrast, all sighted participants could not read Braille. All participants had normal hearing and no physical or language disabilities. They had practiced the game before the experiment and understood the rules well.

### *4.3. Procedure and Evaluation*

At the beginning, the participants listened to the six sounds used in the experiment and grasped the meaning of the cards they indicated. For example, a cat crying sound corresponded to the card's content "Cat". The participants sat face-to-face and checked the positions of the six audible/Braille cards in front of them, either visually or by touching them with their hands. In addition, a test phase was set up—the sound used in the experiment was randomly generated by audible cards laid out in front of the participants, and the participants were asked to answer with the meanings of the sounds. The purpose of this phase was to make sure that the participants heard each audible card correctly. After confirming the position of the audible cards and the sounds and practicing touching and grabbing the audible cards several times, they moved on to the main experiment. In the main experiment, the participants played "Auditory *Uta-Karuta*" using the procedure described in Section 4.2. In addition, they played both Type *α* and Type *β*. The six sounds used in each experiment were not changed, but the order of the six types of cards and the cards to be read were randomly changed each time. In the case of using Braille cards, which is the comparison object, the sighted participants grasped the contents of the cards visually, and the visually impaired and blindfolded participants grasped the contents of the cards by touching the Braille. In this case, the time used in Phase 1 was 30 s, as in the case of audible cards. The game was played in the following three states.

#### **Case 1:** Person with a visual impairment vs. sighted person

**Case 2:** Person with a visual impairment vs. sighted person with a blindfold

**Case 3:** Sighted person vs. sighted person with a blindfold

Case 1 was a competition between a person with a visual impairment and a sighted person, the aim of which was to verify whether the "Auditory *Uta-Karuta*" allowed people with visual impairments to be equally competitive. If there was a difference in the results in Case 1, we could conclude that the accessibility of "Auditory *Uta-Karuta*" for people with visual impairments was inadequate. Case 2 was a competition between a person with a visual impairment and a sighted person with a blindfold, the aim of which was to verify whether an equal match was possible when participants were completely deprived of visual information. Furthermore, if there was a difference in the results in Case 2, the difference in hearing ability between people with visual impairments and sighted people would be confirmed. Case 3 was a competition between a sighted person and a sighted person with a blindfold, the aim of which was to clarify the extent to which the sighted person was affected by their vision. If there was a difference in the results in Case 3, we could conclude that game performance in "Auditory *Uta-Karuta*" is affected due to the presence of vision for sighted players. The participants were informed about the degree of visual impairment of their opponents before the experiment.

The evaluation method is described below. One of the theories that examines the situations in which people find games interesting is flow theory [31]. According to this theory, people feel enjoyment when the difficulty of the challenge is in competition with their own abilities, and when it is close to the limit of their abilities. These "flow states" have been adopted in theories of game enjoyment [32–34]. Abuhamdeh [35] found that games with better opponents and closer games gave more enjoyment to people. In the game of *Karuta*, the winning rate and the strength of the opponent should be close to one's own, which is considered to be a condition that enables the game to be played for an interesting and close game. In addition, since *Karuta* is a game that includes an element of luck—it is possible to guess a card correctly before the read-out phase—it is hard to consider that the winning rate and the feeling of competitiveness are in perfect correspondence. Therefore, for the system evaluation experiment, we used the winning rate as a quantitative evaluation and the subjective evaluation of the opponent as a qualitative evaluation.

### **(a) Winning Rate:**

Winning rate of the people with visual impairments in Cases 1 and 2 and of those with a blindfold in Case 3.

### **(b) Subjective Evaluation of the Opponent:**

The opponent's strength [0, 100] when the player strength is 50.

The winning rate is determined based on players who do not have visual information (people with visual impairments and people with a blindfold) in *Karuta*. The more equal the competition is, the closer the winning rate will be to 50%. Subjective evaluations of the opponents were used for the purpose of subjectively evaluating the strength of an opponent's game performance based on the competition, regardless of the winning rate. The participants' own strength was defined as 50, and they chose an integer value from 0 to 100 to rate the strength of their opponent. The value of 0 is the weakest and 100 is the strongest. Thus, if the value is close to 50, which is the strength of the players themselves, the competition can be considered psychologically equal.

### *4.4. Results and Discussion*

In this experiment, we evaluated the results by having participants play "Auditory *Uta-Karuta*" with three types of experiments: sighted person, visually impaired person, sighted person with a blindfold. Figure 4 shows the result of the evaluation experiment. For the winning rate, a Welch's *t*-test was conducted to examine the significant differences (*p* < 0.05) from the winning percentage of 50%. For the subjective evaluation of the opponent, Fisher's Least Significant Difference (LSD) test was conducted to examine the significant differences (*p* < 0.05) from the existing method using Braille cards.

**Figure 4.** (**a**) Winning rate of the people with visual impairments in Cases 1 and 2 and those with a blindfold in Case 3, (**b**) subjective evaluation of the opponent: The opponent's strength [0, 100] when the player strength is 50.

Figure 4a shows the winning rate of the evaluation experiment. Although there were significant differences in the method using Braille cards, there was no significant difference in Cases 2 and 3. In other words, the null hypothesis that the winning rate was 50%, was rejected when the tactile presentation was used, whereas the null hypothesis was not rejected when the proposed method was used. Therefore, it was possible to enable fair competition, in which the players can be given the same degree of victory opportunity, even without visual information.

Figure 4b shows the subjective evaluation of the opponent of the evaluation experiment. There were significant differences between using Braille cards and using audible cards in Cases 2 and 3. In other words, the null hypothesis that there is no difference in the evaluation value of the opponents between the previous and proposed methods, was rejected. In addition, the results in the proposed method were close to 50. Therefore, the proposed method makes it possible to boost the sense of rivalry, even without visual information compared to using Braille *Karuta*.

Furthermore, Figure 5 shows the questionnaire results of the enjoyment of each game. The participants rated their enjoyment of playing Braille *Karuta* and "Auditory *Uta-Karuta*" on a scale of 1 to 5. Welch's *t*-test was conducted to examine the significant differences (*p* < 0.05) from the existing method using Braille cards. For all the participants in the experiment, "Auditory *Uta-Karuta*" was more enjoyable. While the sighted people can recognize the Braille cards by reading the characters on them, the people with visual impairments and the blindfolded people can only recognize Braille cards by touching them. Therefore, the sighted people could easily win the game when they played Braille *Karuta*, and could not feel enough enjoyment. Although the people with visual impairments can understand Braille cards, they were unable to win in a match against a sighted player. Similarly, when playing against a blindfolded player, it was impossible to realize close competition because blindfolded players could not understand Braille cards. For these reasons, it is assumed that the people with visual impairments were not able to feel enjoyment playing Braille *Karuta*. The blindfolded participants were not familiar with Braille and could not fully understand the Braille cards, so they did not enjoy Braille *Karuta*. On the other hand, in the game of "Auditory *Uta-Karuta*" all the participants recognized the cards by sound, and the game became closer to an equal match, which resulted in a sense of enjoyment. Furthermore, compared to Braille *Karuta*, the difference in the evaluation of enjoyment between the attributes of the experimental participants tended to be smaller for "Auditory *Uta-Karuta*". It is assumed that this is because the audible cards can make the performance of recognizing them closer to the same level regardless of the attributes of the experimental participants compared to the Braille cards.

**Figure 5.** Results of the questionnaire. The participants of the experiment rated their enjoyment of playing Braille *Karuta* and "Auditory *Uta-Karuta*" on a scale of 1 to 5.

Thus, the results showed that "Auditory *Uta-Karuta*" has the potential to be played equally regardless of whether they have a visual impairment or not compared to using Braille cards. Furthermore, it proved to be possible to realize not only equal competition, but also more enjoyable games for the players. This result suggests that the proposed system may be used as one of the means to convey public information in board games that are accessible to the visually impaired. Although they showed that the game was significantly effective in a match between a visually impaired person and a sighted person with a blindfold, there was no significant difference in a match between a visually impaired person and a sighted person. In other words, the experimental game could not completely enable fair competition, regardless of visual impairment. However, it is interesting that the visually impaired persons had a higher winning rate compared with sighted persons with a blindfold, even though neither of them could obtain visual information. Similarly, the visually impaired persons were highly evaluated by sighted persons with a blindfold. These results sugges<sup>t</sup> that the people with visual impairments may have better auditory memorization and directional resolution abilities than the sighted people. This suggestion is assumed would be proper in light of the fact that the visually impaired live mainly using their sense of hearing and touch instead of their sense of sight. This is verified by previous studies [36,37]. In spite of the above facts, the reason for the lack of significant difference between the visually impaired and the sighted is due to the way they touch/grab the audible cards. In fact, in the experiment, it was observed that many visually impaired participants failed to touch/grab the cards. In other words, in this game of "Auditory *Uta-Karuta*", the advantage of sighted people's ability to accurately recognize the positions of the audible cards had a greater impact on the outcome of the game than the high auditory abilities of the visually impaired people. However, it is easy to assume that it would be difficult for visually impaired people to perform as well as or better than sighted people in the card-grabbing situation, unless the sighted people were given a handicap or the visually impaired people were given an advantage. As mentioned above, the degree of usefulness of an auditory card game system can change depending on the characteristics of the players and the rules of the game. Therefore, in the future, it is possible to consider a flexible game design that presents images in order to control the game performance of sighted players. In order to design such a flexible game in the future, it is first necessary to verify the effects of an auditory card game system that uses only auditory cues for the players by changing each valuable parameter. This process reveals the values of the parameters that allow each player to maximize the recognition of audible cards, as well as the parameters that affect game performance depending on the players' characteristics. In the following sections, we investigate each valuable parameter of the system and explain the limitations of the proposed system and appropriate game design based on the results of various existing studies.

### **5. Suggestion for Suitable Auditory Cues in Auditory Card Game System** *5.1.Overview*

In the previous sections, we developed "Auditory *Uta-Karuta*" and verified the usefulness of this system by actually having the game played by visually impaired people, sighted people, and sighted people with a blindfold. As a result, we could not conclude that "Auditory *Uta-Karuta*" had a completely equal game design, as it favored the sighted participants. However, compared to Braille *Karuta*, "Auditory *Uta-Karuta*" approached a more favorable result. This result suggests that, compared to games that use tactile means of presentation, "Auditory *Uta-Karuta*" has a greater potential to make games accessible, regardless of visual impairment. Although we used animal calls and arranged the audible cards horizontally in "Auditory *Uta-Karuta*", we believe that we can find specific valuable parameters that will enable even more equal games by considering a comprehensive range of possible game designs. In other words, it is necessary to search for the elements required for an equal game design that takes the characteristics of sighted people—who have vision—and visually impaired people—who have high hearing abilities—into account.

In this section, we investigate each valuable parameter of the system and explain the limitations of the proposed system and appropriate game design based on the results of various existing studies.

### *5.2. Valuable Parameters*

The main components of our system are the degree of visual impairment, the selection of the audio type, and the style of sound playback. Details are described below.


Hence, as shown in Table 3, we prepared parameters for three types of degree of visual impairment, three audio types, and two types of sound playback.


**Table 2.** The list of audio types using experiment.



### *5.3. Stimuli and Equipment*

For the audible cards, iPod touch devices (6th generation) were used. A total of 12 audible cards were placed in front of the participants, and each card was arranged as shown in Figure 6. No information was shown on the screen of the audible cards, and the participants had to obtain information only by hearing. Sounds were sampled at 44.1 kHz with 16-bit resolution, normalized for amplitude across conditions, and presented in mono. The experiment was programmed in Swift and presented using the speaker of the iPod touch (6th generation). The representational and abstract sounds were obtained from [30]. The verbal sounds were an adult female voice (Kyoko) in Japanese, created using Apple's speech reading software. Representational sounds were environmental sounds which were selected based on a strong relationship between the sound and the event with which it was associated. For example, a cat crying sound was used for the card's content "Cat". The detail of correspondence between the card's contents and the verbal/representational sound is shown in Table 4. Abstract sounds were designed to enable discrimination between multiple abstract sounds and to ensure localization performance, referring to guidelines and experimental results in previous studies [40–42]. Brewster [40] suggested using musical timbres with multiple overtones and including a range from 0.125 to 5 kHz in his guidelines for abstract sound design. In addition, Brewster recommended to add delay of about 0.1 s between each abstract sound when playing them one after another. Patterson [41] found that similar rhythms can confuse abstract sounds even when there are large differences in spectra. Morikawa [42] concluded that auditory stimuli consisting of only frequencies below 2 kHz or above 12 kHz cannot provide sufficient horizontal sound localization.

**Figure 6.** The state of experiment: The subject sat on the chair and described the content and position of the audible cards laid out in front of them.

**Table 4.** Verbal Sound and representational sound sets used in the experiment. Verbal Sound sets are pronounced in Japanese, and written in international phonetic symbols.


Based on these guidelines and results, the abstract sounds were designed as follows. They consisted mainly of musical timbres with a wide bandwidth from 0.5 to 16 kHz and overtones as much as possible to ensure both the guideline and the localization performance. Furthermore, each abstract sound was designed with a different rhythm to reduce confusion. In the Type *α* experiment, a 0.1 s gap was inserted between each sound to enable the participant to determine when one sound ends and the next begins. Abstract sounds were randomly assigned to the contents of the card.

Participants were tested on six stimulus combinations of three audio types and two methods of presenting multiple sounds. Blocks of demonstrations and experiments were presented to participants with a new random order of stimuli in each block. In the blocks of demonstrations, the participants listened to the 12 sounds used in the experiment and grasped the meaning of the cards they indicated. The maximum memorization time was 10 min for each audio type. The participants sat and checked the positions of the 12 audible cards in front of them, either visually or by touching them with their hands. In addition, the test sounds were played from each of the 12 cards in turn, and the participants had to confirm correspondence between the sound source and the card's position. In the experiment block, each of the 12 audible cards emitted a different sound. In the Type *α* experiment, all sounds were presented randomly for 5 s each. The participants described orally the meaning and the position of the sound presented by each audible card. There was no time limit for this, and the participants were asked to describe as much as possible. In this experiment, the identification rate and recognition rate were evaluated. The identification rate indicates how many of the 12 audible cards the participants were able to distinguish. The recognition rate indicates how many of the 12 audible cards of which the participants were able to correctly grasp the location and contents.

There were 18 participants (10 males and eight females) aged from 19 to 42 years old. Among the participants, there were five people with clear vision (four males and one females), seven people with visual impairments (three males and four females) and six people with a blindfold (three males and three females). All participants had normal hearing and no physical or language disabilities.
