**1. Introduction**

Multimodal (or multisensory) integration refers to the neural integration or combination of information from different sensory modalities (the classic five senses of vision, hearing, touch, taste, and smell, and, perhaps less obviously, proprioception, kinesthesis, pain, and the vestibular senses), which gives rise to changes in behavior associated with the perception of and reaction to those stimuli. Information is typically integrated across sensory modalities when the sensory inputs share certain common features [1].

Cross-modality refers to the interaction between two different sensory channels. Crossmodal correspondence is defined as the surprising associations that people experience between seemingly unrelated features, attributes, or dimensions of experience in different sensory modalities.

The intuitive strategies based on cross-modal analogy, association, and symbolism are suitable for creating a design that provides connections between the senses, which directly appear appropriate and easy to interpret [2].

**Citation:** Cho, J.-D.; Lee, Y. ColorPoetry: Multi-Sensory Experience of Color with Poetry in Visual Arts Appreciation of Personswith Visual Impairment. *Electronics* **2021**, *10*, 1064. https://doi.org/ 10.3390/electronics10091064

Academic Editors: Juan M. Corchado and Rui Pedro Lopes

Received: 2 March 2021 Accepted: 25 April 2021 Published: 30 April 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

Although many studies have been conducted on the cross-sensation between the sight and other senses, there are not many studies on the cross-modality between the non-visual senses. Through the investigation of weak synesthesia that is perceived at the same time by intersecting various senses such as audio, touch, smell, etc., other sensory information can be connected to a specific form of color information. In this task, so far, efforts have been made to create different single-mode sensory perceptions for color, for example, color–sound, color–tactile pattern, color–odor, etc.

In our previous studies, colors were expressed as embossed tactile patterns for recognition by finger touches, temperature, sound, and smell to provide a rich art experience to person with visual impairments. The Black Book of Colors by Cottin [3] described the experience of a fictional blind child named Thomas, who describes color through association with certain elements in his environment. This book highlights the fact that blind people can gain experience through multisensory interactions: "Thomas loves all colors because he can hear, touch, and taste them."

An accompanying audio explanation provides a complementary way to explore the overall color composition of an artwork. When tactile patterns are used for color transmission, the image can be comprehensively grasped by delivering graphic patterns, painted image patterns, and color patterns simultaneously [4].

This suggests to us the possibility and the justification for developing a new way of appreciating works in which the colors used in works are subjectively explored through the non-visual senses. In other words, it can be inferred that certain senses will be perceived as being correlated with certain colors and concepts through unconscious associations constructed with the concepts. However, while this helps visually impaired users perceive and understand colors in completely different ways, it becomes necessary to integrate all perceptual sensations into a single multi-sensory system, where all the senses are perfectly connected and interchanged. This concept is in line with the theory of emotional intelligence, which defends the organic functioning of people, as a whole, in interaction between the emotional, the sensory, and the cognitive [5,6]. According to Damásio [5], feelings are basically the mind's interpretation of the state of the body, and in order for reason to be truly rational, it must be based on emotional cues from the body. Emotional memories are not easily erased from memory and remain the longest. Tacit memories that did not easily emerge above consciousness can suddenly come to mind through the opportunity of contact with any part of one's body [5].

Audio description does provide a useful service for those with low or no visual acuity [7,8]. Schifferstein [9] observed that vivid images occur in all sensory modalities. The quality of some types of sensory images tends to be better (e.g., vision, audition) than of others (e.g., smell and taste) for sighted people. The quality of visual and auditory images did not differ significantly. Therefore, training these multi-sensory experiences introduced in this paper may lead to more vivid visual imageries or seeing with the mind's eye. However, such descriptions are often monolithic accounts, missing the opportunity for alternative messages to be delivered and received. Making art accessible to the visually impaired requires the ability to convey explicit and implicit visual images through nonvisual forms. It argues that a multi-sensory system is needed to successfully convey artistic images. It also designed a poetic text audio guideline that blends sounds and effects to translate the work into an ambiguous artistic sound text [9].

Audio or verbal description on an artwork generally attempts to explain the painting without expressing the individual subjectivity. When making visual art accessible to the visually impaired, it is not enough to describe the colors and situations portrayed because that objective information does not attach to anything in their experience. Using expressions that contain the sensibility of poems, color, and situation can be matched with the parts of each painting.

Poem is a piece of writing that has features of both speech and song, whereas poetry is the art of creating these poems. Throughout the poem, seeing and hearing are used to understand color. Imagery is a literary device that refers to the use of figurative language

to evoke a sensory experience or create a picture with words for a reader. By utilizing effective descriptive language and figures of speech, writers appeal to a reader's senses of sight, taste, smell, touch, and sound, as well as internal emotion and feelings. Poetic imagery provides sensory details to create clear and vivid descriptions. This appeals to a reader's imagination and emotions as well as their senses [10].

The connection between color and poem has not been studied through existing research. In this paper, we explore how color can be explained by poetry so that visually impaired people feel or perceive color through poetry. However, the connection between color and poetry should be clearly matched. When appreciating color in artworks using poetry, we need to explore whether color concept and poem interfere with each other, causing confusion in color perception, or synergizing with each other to positively affect color perception. A system usability test is performed to evaluate whether it helps to easily recognize the color.
