*2.4. Sound-Color Cross-Modality*

Color is a continuous spectrum for which there have been several representation models [15]. Munsell's color model is one of the earliest ones. In it, the color is organized into three dimensions: hue, chroma (or saturation), and value (or brightness). Hue refers to the color itself. Brightness is an indication of the amount of white or black of the color. The brighter the color, the closer it is to white, and vice versa. Saturation is an indicator of the vividness (clearness) of a color. Another common dimension is the warm-cold spectrum of colors. The closer a color is to the red end of the visible spectrum, the warmer it is. On the contrary, the closer a color is to the blue end of the spectrum, the colder it is [16].

Wang et al. [17] explored the putative existence of cross-modal correspondences between sound attributes and beverage temperature. The results, after an online pre-study and the main study itself, confirmed that the experience of drinking cold water is associated with significantly higher pitches and faster tempo. One possible explanation for this kind of effect is the formation of emotional associations [18].

Hamilton-Fletcher et al. [19] presented a color-sound sensory substitution device which consisted on a color image explored by the user on a tablet device. The color was then turned into sound, which the user was able to listen to while moving the stylus over the image. In addition, the device, together with other two sensory-substitution devices, was given to ten blind users which, among other things, addressed the importance of the sounds to be not only understandable but also aesthetically engaging.

Regarding aesthetics in sounds for color substitution, Cho et al. [6] investigated possibilities for creating beautiful sounds for representing colors by replicating the three main characteristics of color: hue, chroma, and value, by matching them to three features of sound: timbre, intensity, and pitch. Then, two sets of musical sounds for expressing colors were designed and tested: VIVALDI and CLASSIC. User tests were conducted with eight sighted adults and 12 visually impaired users. The results showed that both sound-color mappings were useful and engaging for the participants and that users were able to identify with high accuracy the different colors by hearing the musical sounds of both sets of audios. The present work is a continuation of that work. Here, those two sound-color mappings are implemented into a complete sound-temperature-color coding. In addition, the relative usefulness of each set (VIVALDI or CLASSIC) for the multi-sensory system is also investigated, in order to choose the best one among the two for the multi-sensory system.
