**3. Method**

### *3.1. Previous Method*

As we stated above, this paper addresses and aims to improve previous works in which unisensory color-coding was performed and tested. While in [5] a sound-color mapping algorithm was designed, in [6] the developed algorithm and system was a temperature–color one. It is our goal to mix both these methods for increasing the number of colors the user can experience and for making the sensory experience more stimulating and engaging. However, for developing a multi-sensory system mixing both of them, it is necessary to find the best way of implementing both methods in a multi-sensory way in order to increase the easiness of color recognition and the comfortability of the sensory experience.

The sound–color code designed in [6] for each of the colors and their saturated, bright, and dark variants can be seen in Table 1. There, each of the 18 colors is represented by a musical excerpt either from a sound set consisting of variations on a Vivaldi piece or from a more ample classical repertoire sound set. Each one of these sound sets received a name: VIVALDI, and CLASSIC. The melody, rhythm, harmony, instrument, and tessitura of the sound files of both sound sets were selected in order to make a comprehensible representation of the hue and quality of each color. While more information about the musical selection and processing method can be seen in [6], it is important to notice that, regardless of the method, each sound file was created so its musical excerpt represents both the hue of the color and the quality of the color. Therefore, all sounds belonging to the dark colors have differences, which allows the user to distinguish the hue, but also have similarities that represent the dark dimension of the color. It is important to remember that this set of sounds come in two different versions, CLASSIC and VIVALDI, since one of the research's goals will be finding out which one of those set of sounds is more convenient for being implemented in the multi-sensory system.

**Table 1.** Sound coding colors with saturated/bright/dark colors (Adapted from ref. [6]). The sound wav files are provided separately as a supplementary materials.


Similarly, a temperature–color mapping was designed based on the previous work from [5], where it was proved that users were able to discern with high accuracy temperatures with intervals larger than 3 ◦C. As a result, any division of the comfortable temperature range with more than 3 ◦C between temperatures can be used for discerning several colors or the other dimensions of the colors. Consequently, the following temperature-color codes were designed, one for expressing hues (Table 2) and the other for expressing other color dimensions (Table 3). The original main method, based on [5], would be to have the user feel two temperatures, one expressing hue and the second one expressing the color dimensions of that same hue.




**Table 3.** Temperature-Color coding for color warm/bright/dark and cold dimensions.

### *3.2. Multi-Sensory Improved Method*

It was possible to mix both those methods to create a multi-sensory experience of colors if both sound–color and temperature–color mappings were simplified and mixed, dividing the roles each one had. For example, hue could be expressed only through sound while the color quality could be expressed only by means of the temperature cues, or vice versa. This created a multi-sensory experience of the colors more engaging and convenient than the original unisensory experiences. In addition, the separation of hue and the other color dimensions into different sensory modalities may make memorization of the mapping easier and faster for the user. Both novel multi-sensory sound-temperature-color systems can be seen in Tables 4 and 5. In Table 4, a coding where color hue is expressed as sound, and the other color dimensions are expressed as temperature, is presented. On the contrary, in Table 5 the color hue is expressed through temperature and the other dimensions are expressed by means of sound cues. We created a naming convention for these two possible mappings by following the order of the sensory experience: sound-temperature-color coding and temperature-sound-color coding, respectively.

### **Table 4.** Designed sound–temperature–color coding.




For the satisfactory development of a multi-sensory color-coding system based on these methods, three are the elements that needed to be figured out:


3. Finding out which sound set from the two options (VIVALDI and CLASSIC) is in general terms more suitable to express the different dimensions of color.

Tests were performed for answering these questions. These tests and their analysis and results will be explored in the following sections.
