**1. Introduction**

People with visual impairments (PVI) can form concepts regarding real-world object properties, including abstract conceptualization and relative differences between colors [1–3]. Symbolism, preferences, terminology, and other daily life concepts associate with colors by engaging psychological and aesthetic stimuli. These color associations serve a significant role in culture, faith, art, commercial branding, and everyday lifestyle. Therefore, comprehensive color information is necessary for PVI to think, consider, make actions, and cause reactions more prudently [4].

Color recognition through non-visual stimuli has been an active research area for PVI [5]. The color-sound cross modular associations convey visual color information through auditory senses. These associations code color characteristics of hue, chroma, and value through a combination of music instrument, music tone, pitch variations, etc. The auditory codings might be used for conveying color information independently, or in conjunction with tactile sense. Cho et al. [6] have studied these relationships and proposed two sound coding color melodies. They considered the tone, intensity, and pitch of melody sounds extracted from classic music to express the brightness and saturation of colors. The sound code system represented 18 chromatic and five achromatic colors with using classical music sounds played on different instruments. While using sound to depict color, tapping a relief-shaped embossed outline area transforms the color of that area into the sound of an orchestra instrument. Furthermore, the overall color composition of Van Gogh's "The Starry Night" was expressed as a single piece of music that accounted for color using the tone, key, tempo, and pitch of the instruments. The shape can be distinguished by touching it with a hand, but the overall color composition can be conveyed

**Citation:** Jabbar, M.S.; Lee, C.-H.; Cho, J.D. ColorWatch: Color Perceptual Spatial Tactile Interface for People with Visual Impairments. *Electronics* **2021**, *10*, 596. https:// doi.org/10.3390/electronics10050596

Academic Editor: Calogero Maria Oddo

Received: 3 February 2021 Accepted: 28 February 2021 Published: 4 March 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

as a single piece of music, thereby reducing the effort required to recognize color from that needed to touch each pattern one by one. Gilbert et al. confirmed that humans have an association mechanism that unconsciously associates a specific scent with a specific color through olfactory sense [7]. Temperature differences have also been studied to convey color information to PVI. However, as the temperature is a single modality, merely one color characteristic can be coded through temperature levels. Bartolomé et al. [8] proposed thermal stimuli to code depth of colors in visual artworks. Cavadini et al. proposed an automatic color perception system, which provides haptic feedback at three points on the palm corresponding to Red, Green, and Blue channel values of detected colors through color sensors [9]. This method uses a wearable glove, and wearing the glove can hinder everyday tasks for PVI. In addition, the RGB color system uses a combination of red, green, and blue channel numerical values to represent any arbitrary color, which does not match the human perception of colors. Researchers reflect that spatial perception, tactile perception, and linguistics engage the brain's visual cortex. In addition, not having to process the visual stimuli makes PVIs' performance superior for tactile tasks [10], which suggests the preference of tactile stimuli for PVI perception. The cognitive aspects of multi-sensory activation and recognition by the sense of touch are covered by Lawrence et al. [11].

### *1.1. Tactile Representation of Colors*

Tactile color patterns (TCP) are embossed surface patterns for conveying color information through touch to PVI. TCPs might be helpful as they can be used in conjunction with other tactile modalities, like contour information or object boundaries in the artwork. They offer immediate color perception by tapping onto the artwork relief pattern, unlike audio description which needs to be triggered. TCPs as an assistive tool for visual aspects of artwork can be supplemented to the audio description; this helps shorten the audio description and improve localization of artworks' objects information. Ramsamy-Iranah et al. [12] designed color symbols for children. The design process for the symbols was influenced by the children's prior knowledge of shapes and links to their surroundings. For example, a small square box was associated with dark blue reflecting the blue square soap, a circle represented red as it was associated with the 'bindi' [13]. Yellow was represented by small dots reflecting the pollen of flowers. Since orange is a mixture of yellow and red, circles of smaller dimension were used to represent orange. Horizontal lines represented purple and curved lines were associated with green representative of bendable grass stems. Shin et al. [4] coded nine colors (pink, red, orange, yellow, green, blue, navy, purple, brown, and achromatic) using a grating orientation (a regularly spaced collection of identical, parallel, elongated elements). The texture stimuli for color were structured by matching variations of orientation to hue, width of the line to chroma, and the interval between the lines to value. The eight chromatic colors were divided into 20º angles and achromatic at 90º. Each color has nine levels of value and of chroma. Levels 1–4 used a different grating interval to represent value, levels 6–9 used a different grating width to represent chroma, and level 5 represented the pure hue. In the survey on whether or not 3D printed colors were distinguished by texture, color identification tests were performed on five visually impaired people to distinguish the direction, width, and spacing of the proposed color patterns. Adjacent color hues in this scheme are oriented at an angular distance of 20°, but research suggests that tactile accuracy for grating orientation is significantly distinguishable for 30° or 45° angle [14,15]. The Munsell color system based TCP schemes by using ideographic characters were proposed by Cho et al. [16]. They employed an experimental investigation and adaption based approach for representing wide color gamu<sup>t</sup> of 29 and 53 colors shades in the basic and extended versions for three TCP schemes, respectively. Taras et al. [17] presented a color code created for viewing on braille devices. The primary colors, red, blue, and yellow, were each coded by two dots. Mixed colors, for example violet, green, orange, and brown, were coded as combinations of dots representing the primary colors. In addition, the light and dark shades were added by using the 2nd and 3rd dots in the left column of the Braille cell. In addition, color can be represented by

using the spatial color wheel that expresses the angular orientation. This is another way of color information transmission. Our proposed system codes a similar number of colors compared to other color patterns. Moreover, the color wheel depicts essentially the visible spectrum of colors enclosed by a circle, and is a useful tool for describing what happens when you mix paints together, complementary color relationships, and adjacent colors.

It can be difficult for individuals with visual impairments to fully participate in the visual arts due to the lack of inclusion and assistive technologies. Their participation in the visual culture of the world and visual art is important as the inclusion opportunities improve their life quality and help them gain skills crucial to their education and employment opportunities [18]. The visual centricity of exhibitions and museums has typically been a barrier regarding visit and appreciation of PVIs [19]. However, as "The event happens as a question mark before happening as a question (Lyotard, 1989: 197)," many of these institutions are now focusing on accessibility to enhance PVIs' experiences by incorporating assistive technologies, contextual information delivery, and multisensory experiences [20].

The TCP schemes in recent literature are mainly focused on tactile color translation for artwork, whereas the significance of colors in daily life imparts the need to convey comprehensive color information to PVI for them to participate in society more prudently. Most of these schemes present static interpretation of colors, which causes the need to arise to develop assistive technologies that can dynamically translate detected colors from reallife objects or artwork for PVI. Moreover, the usability of these tactile patterns is limited, as learnability of these tactile patterns may be required for individual TCP associations. In this paper, we propose a tool for people with visual impairment (i.e., congenital blind people who have not experienced color and the acquired blind people whose color has disappeared from memory) that can intuitively recognize and understand the three elements of color, that is, color hue, lightness, and saturation, taking advantage of the timepiece watch design.

### *1.2. Timepiece Watches for PVI*

There are many wearable wristwatches for PVI in the context of timepiece operation. Nevertheless, the common way of applying it has been using the braille interface for conveying numeric braille patterns similar to the digital watches for sighted people, since interactable hands of the traditional analog watch are fragile and prone to damage by heavy touch. The electromagnetic solenoid based cost-effective electronic braille display was proposed by Adnan et al. [21]. Tyler [22] used the 4-dot condensed braille code to introduce a design scheme of wearable timepiece wristwatch for PVI. The mechanical design was adapted to represent numeric braille cells for time and date, with time/date adjustment and alarm options. Dot watch [23] is a commercially designed smartwatch based on a similar concept which uses braille as its interface using braille coding. In addition to presenting time information by tactile stimuli, the Dot watch can be connected to a smartphone via Bluetooth to perform few basic tasks such as caller identification, and either pick or decline the call. Another design scheme consisting of a disk, a plate, an actuator, and a plurality of four pins mounted to a slide within the four respective holes was proposed for the braille timepiece wristwatch by Anderson et al. [24]. A haptic feedback scheme for a digital smartwatch display was proposed by Twyman et al. [25], which engaged side-mounted piezoelectric actuators to cause glass screen vibrations via ultrasonic frequencies. The braille literacy of PVI is on the decline, and it is projected to decrease in upcoming years, with the widespread use of smart devices. Velázquez [26] provided a comprehensive study on workload, learnability, design concept understanding, and latest advancements in wearable assistive devices for blind people and recognized low user acceptance for these devices. This led our research to investigate the durable design of the analog watch and its possible mapping with color perception for the PVI.

### *1.3. Review of the Color Systems*

The Munsell color system arranges colors by accounting for the human visual response into systematic color space [27]. It considers hue, chroma, and lightness as three properties of color for color space organization as shown in Figure 1a. The hues or basic colors are arranged in a circular manner placed apart at each horizontal circle Figure 1b. Chroma or saturation of color is measured by the distance from the center of the circle to the edge. High chroma implies clearness of color or pure colors, while low chroma implies less saturated color with the lowest chroma colors being achromatic. The lightness or value of colors varies vertically where white is represented by the highest lightness and black holds the lowest lightness value.

The Goethe's color triangle is an excellent model for color relationships and the relative differences between them, such as additive color mixing and complementary colors [28]. These differences can be interactively simulated. This model arranges colors as primary, secondary, and tertiary colors. The three vertices of triangle are associated by primary colors of red, blue, and yellow. The secondary colors orange, green, and purple are obtained by mixing primary colors on either side of them (Figure 1c). The tertiary colors are obtained by mixing primary colors adjacent to them. Goethe arranged the primary, secondary, and tertiary colors based on physical grounds, as well as for their emotional content linked with them. An American educator Josef Albers extended Goethe's work for studying and teaching colors through an experimental way [29]. The particular arrangemen<sup>t</sup> of Goethe's color triangle retains the particular emotional or psychological states as per his description. This color arrangemen<sup>t</sup> is also similar to the RYB color model [30]. Its warm colors are red, orange, and yellow, and its cool colors are green, blue, and purple. The three primary colors in the RYB color model are red, yellow, and blue. Mixing the primary colors causes the mixtures to absorb light wavelengths to create other colors. The three secondary colors are orange, green, and purple. Mixing the three primary colors together creates an almost black color. We have considered the color gamu<sup>t</sup> for our proposed system based on these color models as shown in Figure 1d,e for simplified and extended versions of watch patterns, respectively. This primary and secondary color information is extended with the integration of color tones of light, saturated, and dark from Munsell's color system.

**Figure 1.** (**a**) Lighted (L), saturated (S), and Dark (D) color tones for red and achromatic colors. (**b**) 3D representation of Munsell color system renotations with a slice cut away for visualization [31]; (**c**) Goethe's Colour Triangle with primary and secondary colors; (**d**) color wheel for simplified watch pattern; (**e**) color wheel for extended watch pattern.
