*4.2. Interaction*

One of our design requirements was to make interaction with the IMG as simple to learn and use as possible. Requiring to remember the location and use of buttons as well as gestures or commands can be burdensome for most people since it will be the first time that they use a device. Moreover, many users often skip instructions, even if they are short. Because of this, the IMG only has three user interactions, pressing buttons with a single-use, and double and triple tapping on the relief model to access localized information and audio. A simple interaction interface has its benefits. It makes the system easy to learn to use (Table 4-S7) and avoiding the feeling of the burden that can come when facing a new device (Table 4-S10) as evidenced by one participant's response, *"EP8: With the talking exhibit you don't need to know anything, you just stand there, touch something, and it starts talking to you about the picture."* By keeping consistency throughout the IMGs, once a user knows how to use one IMG, it knows how to use the rest. Unfortunately, tactile graphics have drawbacks. Experience goes a long way to read tactile graphics proficiently, and every time the user faces a new tactile graphic, it will need to learn the meaning of the texture and line styles to recognize their meaning. As expressed by one of the participants, *"EP15: You need to know Braille to read the tactile drawings with Braille and that takes time and effort."* Moreover, the lack of Braille proficiency affects the experience across all the exhibits, since the burden is on the user.

The participants reported a higher degree of inconsistency (Table 4-S6) for the IMG. Upon further investigation, we found out that it was due to a failure in some of the IMG prototypes to register some touch gestures correctly, causing the wrong audio feedback to trigger or not at all. Similarly, at the exhibition, not all the interactive zones in some of the artworks had ambient sound audio feedback, causing some users to believe that the system was malfunctioning or that their gesture was not recognized when they tapped the area and audio was not reproduced. Audio feedback should be added to the interaction zones that lack audio tracks, like empty or background space, to manage user expectations. Non-obtrusive audio or vibrotactile feedback could be added to help the user become aware that their input is sensing.

### **5. Conclusions and Future Work**

In this work, we have presented the development of an interactive multimodal guide for improving the independent access and understanding of visual artworks. The IMG design was developed following the needs uncovered through a formative study in collaboration with people with vision impairments, art museums and gallery staff, and artists. Through an evaluation with eighteen participants, results demonstrate that the multimodal approach coupled with a simple to learn interaction interface is more effective in comparison to tactile graphics guides in providing independent access across a diverse style of artworks. Feedback collected during the multiple exhibition points in new directions for our work. As seen in Figure 5b, the IMG is sometimes used as a collaboration tool to socially interact with art. We would like to explore this possibility, as this could alleviate the perceived burden that some participants expressed when going to the art gallery with an acquaintance. Moreover, our current prototype was designed for use in an exhibition environment. Art educators at schools have expressed their interest in using the guide as an educational tool in class. To this end, more research is needed to explore the difference in audio description content and delivery methods to provide tailored information, while

making it manageable for users with different content needs. The current prototypes only make use of tactile and audio modalities. We look forward to develop new experiences with other modalities such as smell, and explore how they might improve visual artworks exploration.

**Figure 5.** Exhibition visitors using the interactive multimodal guide. (**a**) Stand alone use (**b**) Social interaction.

> **Author Contributions:** Conceptualization, methodology, formal analysis, data curation, writing— original draft preparation, visualization, and project administration, L.C.Q.; Software, J.I.B. and L.C.Q.; Investigation, L.C.Q., J.I.B. and J.C.; Resources, writing—review and editing, supervision, funding acquisition, J.C. All authors have read and agreed to the published version of the manuscript.

> **Funding:** This research was funded by the 2018 Science Technology and Humanity Converging Research Program of the National Research Foundation of Korea gran<sup>t</sup> number 2018M3C1B6061353.

> **Institutional Review Board Statement:** The study was conducted according to the guidelines of the Declaration of Helsinki, and approved by the Institutional Review Board (or Ethics Committee) of Sungkyunkwan University (SKKU202011005-UE003 2020-11-05).

> **Informed Consent Statement:** Informed consent was obtained from all subjects involved in the study.

**Data Availability Statement:** The data presented in this study are available within the article.

**Acknowledgments:** We would like to thank all volunteers for their participation and the reviewers for their insights and suggestions.

**Conflicts of Interest:** The authors declare no conflict of interest.
