Towards a More Sustainable and Less Invasive Approach for the Investigation of Modern and Contemporary Paintings

Abstract

In Heritage Science, sampling is frequently performed for the subsequent diagnostics of modern and contemporary paintings using invasive analytical techniques. However, it endangers the integrity of artworks, and thus, it should be carefully planned and carried out only as a last resort by specialists. Pigment mixtures have commonly been employed by modern and contemporary artists due to the ease of combining paints on the color palette. Hence, a painting might include both primary/secondary paints and mixtures of those. Therefore, obtaining a sample from a mixture might be sufficient for the identification of the individual primary-colored paints. This study focused on the creation of a user-friendly computational workflow for the analysis of images of paintings for the identification of mixtures using cluster analysis (K-means and Fuzzy C-means clustering). Sixteen modern and contemporary paintings that belong to the International Gallery of Modern Art Ca’ Pesaro in Venice have been selected: seven of them by Guido Cadorin (1892–1976), six by Andreina Rosa (1924–2019), and three by Boris Brollo (b. 1944), and the artworks of the latter being examined for the first time in this study (using Raman and ER–FTIR spectroscopies). It was found that mixtures can be identified in unvarnished paintings that consist of both non-overlapping and vibrant-colored paint layers, like those of Boris Brollo, and overlapping paint layers, like those of Andreina Rosa. Moreover, K-means clustering performs better in the case of non-overlapping colors, whereas Fuzzy C-means in the case of overlapping colors. In contrast, paintings that have been rendered with dark colors and that present a varnish layer, like those of Guido Cadorin, cannot be preliminary investigated in the proposed manner.

1. Introduction

1.1. Sustainable Approaches—The Answer to the Need for a More Effective and Less Invasive Analysis of Modern and Contemporary Paintings

Modern and contemporary paintings are arguably the most displayed cultural heritage objects since they marvel through their unconventionality, and they emphasize the artists’ inner worlds in a completely new manner. Therefore, museums face the challenge of their conservation, which is complicated by the fact that these paintings include a wide range of synthetic organic and inorganic materials (pigments, binders, and additives) that might not have been specifically designed for artistic purposes [1,2]. Green conservation treatments are gradually implemented to ensure both the health of the conservator and the low impact on the environment [3,4]. However, prior to proposing a conservation treatment, diagnostics should be performed by various analytical techniques that are often (micro) invasive and, thus, require samples being obtained. In this context, the well-established principle in art conservation that originated in medicine, namely, primum non nocere (first, do not do any harm) [5], should always be taken into consideration as, first and foremost, cultural heritage has to be preserved for future generations. Therefore, a sampling strategy should be designed to limit the number of samples and ensure their representativeness. It should be mentioned that the number of visitors of museums and sites around the world is increasing each year and UNWTO forecasted that there will be 1.8 billion international tourists by 2030 [6]. Thus, an adequate sampling strategy would not only lead to better conservation but also would benefit tourism as the artworks would be kept in the best possible condition, which in turn would enhance sustainable economic development. It should also be noted that the analysis of samples using the techniques that are generally employed in Heritage Science tends to require substantial electricity consumption. This could also be diminished by obtaining less and more representative samples.

Since the digitization of paintings gained momentum in these last years, researchers began to study the potential of a digital camera, or even a mobile phone’s camera, to preliminarily identify the pigments used by artists [7,8,9,10,11]. A positive outcome of this inquiry would not only simplify the process of the diagnostics of paintings but also would require fewer financial resources since the necessary equipment has a lower cost in comparison with other analytical techniques. However, the procedure of pre- and post-processing is highly dependent on the illuminants and surfaces’ reflectance at the moment of taking of the images, and these two parameters are difficult and sometimes almost impossible to control [12]. Fortunately, it has been proven by Trombini et al. [13] that polynomial regressions give quite good results in the case of the color correction of images for the color analysis of pigments when studying clusters of data that have been collected from the chromatic and chemical points of view. Despite this, to the authors’ knowledge, the acquisition of the images of paintings coupled with computational approaches for data extraction and processing has not been previously considered as a way of guiding the sampling procedure.

Therefore, this study sought to test whether such an approach would have a satisfactory outcome in this regard.

1.2. The Paintings Included in this Study

The sixteen paintings that were selected for this study belong to the International Gallery of Modern Art Ca’ Pesaro in Venice (Italy), and they have been depicted by three Venetian artists, seven of them by Guido Cadorin (1892–1976), six by Andreina Rosa (1924–2019), and three by Boris Brollo (b. 1944).

Guido Cadorin was an important Italian modern art painter, who was notorious for the use of tempera [14] in spite of the developments brought about the paint industry. The seven paintings that have been previously studied [14,15] and that were selected for this research: Ritratto di mio padre (1921–1923), Fanciulla (1930), Nudo femminile (c.1930), La Navicella di San Pietro (1942), Ritratto di Giovanni Giurati Junior (1942), Ritratto di Benno Geiger (1943), and La tavola (1951), which are depicted in Figure 1a. The color palettes that were used in these artworks were formerly examined, and thus, the existence or lack of mixtures has already been ensured [14,15]. Moreover, Guido Cadorin varnished his artworks, which means that the visual appearance of the pigments is the result of the presence of the varnish on top. This aspect can be quite problematic in the context of color analysis since protective coating layers strongly influence the color appearance of the pigments underneath. This is not only caused by the progressive darkening of the resin that composes the varnish but also by the gloss of the surface that reflects the incoming light [16,17]. However, these artworks were considered appropriate for testing the performance of the proposed workflow also on the paintings that consisted of a protective coating.

Andreina Rosa was a prolific Italian artist that took part in multiple editions of the Venice Biennale (from 1950 to 1970) [18]. She preferred to use paints with a high tinting strength without applying varnish layers on her finished artworks. The six paintings that have been included in this study have been previously analyzed and, thus, similar to Guido Cadorin, and the presence or lack of mixtures was already known [19,20]. Their titles are Nudo Coricato (1948–1949), Natura morta con lanterna (1950–1953), Giulia con Michele (1954), Natura Morta (1954–1955), Case sul lago (1954–1955), and Variazione sul Tema Maiastra: Il Fiore per la Principessa (1989), and are also shown in Figure 1b.

Boris Brollo was born in 1944, and he is currently active as a curator, being known for organizing exhibitions such as Propilei, Concordia Sagittaria (VE) in 1986; Sensi, Verona in 1991; and Al Limite, Linz (Austria) in 1993 [21]. He started his art career as a student at the Accademia di Belle Arti di Venezia from 1970 to 1974, where he enhanced his painting skills. In 1972, he created the three artworks that are included in this study, namely, America America, L’Amore, and La Verità (Figure 1c). All three paintings show the affinity of the painter for the pop art movement. The artworks seem to be comic book excerpts and depict the situation of the world at that time, highlighting aspects such as American consumerism and poverty. Their selection was motivated by the predilection of the artist for the use of almost pure paints from paint tubes, whose colors had not been altered by the creation of tints (lightening the colored paint with white) or shades (darkening the colored paint with black). Moreover, it was highly possible that he generally employed synthetic organic pigments (SOPs) that have a high tinting strength even if they are actually present in low concentrations in paints [22]. According to the data provided by the International Gallery of Modern Art Ca’ Pesaro in Venice, all three artworks were depicted with acrylic paints. However, they have not been formerly investigated in an analytical manner, so this information was also verified in this study.

1.3. The Aim of this Research

Mixing paints that were applied straight from paint tubes onto the color palette was most probably a routine activity for modern and contemporary artists, not only due to financial reasons, but also due to the uniqueness of the hues that might have been obtained [23]. As a result, paintings might have included regions where only primary pigments have been used, which might have also been mixed and then employed in the same artwork. Therefore, obtaining samples from the mixtures might be sufficient to detect the individual pigments. This might guide the subsequent sampling procedure, diminish the endangering of the artworks’ integrity, and increase the representativeness of the samples. However, there might have been the possibility that other primary-colored pigments than those used in some regions of the paintings were mixed, but in this study, it was considered rather low since these modern and contemporary artists were characterized by their fast manner of painting. Thus, it is highly probable that they generally employed a reduced color palette and mixed the same primary colors.

This study continues the research published in [20], which mainly focused on the identification of mixtures from the same six paintings of Andreina Rosa using the paletteR package in R. Here, the main aim is to present a better-optimized and user-friendly workflow for the extraction of colors from the corrected images of modern and contemporary paintings using two clustering methods, namely, K-means and Fuzzy C-means, the subsequent simulation of mixtures of the extracted primary colors, and the measurement of the color difference. The latter could emphasize the presence or the lack of pigment mixtures. The computational methodology was applied on the sixteen paintings by Guido Cadorin, Andreina Rosa, and Boris Brollo and the outcomes were confronted with the analytical results in order to validate it. This paper mainly presents the results obtained in the case of Boris Brollo as the materials that have been identified in the artworks of the other two Venetian artists had been already described in the previous publications [14,15,19,20]. In the case of Andreina Rosa, a comparison was required between the use of K-means clustering that was also employed in [20] and Fuzzy C-means clustering to check whether the latter performed better.

2. Materials and Methods

2.1. The Non-Invasive Analytical Techniques Used for the Identification of the Painting Materials of Boris Brollo

2.1.1. Raman Spectroscopy

Raman spectroscopy was carried out using a BRAVO Handheld Raman Spectrometer (Bruker Optik GmbH, Leipzig, Germany). The instrument includes a dual-laser excitation that can provide a simultaneous double excitation at 785 and 852 nm. Its spectral resolution is 10–12 cm⁻¹. Raman spectra were registered in the 300–3200 cm⁻¹ spectral range with a scanning time from 1 s to 300 s. The diameter of the tip of the instrument was 5 mm, which ensured an adequate registration of the data as each observed color could be individually examined. The elaboration and the interpretation of data were carried out using the following softwares: OPUS (version 8.2 provided by Bruker Optik GmbH), Spectragryph (version 1.2.16.1), and Origin Pro 2021 (version 9.8.0.200). The synthetic organic pigments’ spectra that were included in the SOPRANO database [24] created by the Royal Institute for Cultural Heritage (KIK-IRPA) served as references.

2.1.2. External Reflection FTIR Spectroscopy (ER–FTIR)

ER–FTIR analysis was applied to provide a further validation of the Raman results in order to confirm/deny the presence of pigment mixtures. This spectroscopic technique was applied on only one of the paintings of Boris Brollo, namely, B1. The spectra were recorded in the 7500–350 cm⁻¹ range using an ALPHA II Fourier Transform IR Spectrometer (Bruker Optik GmbH). The analysis points were investigated with 128 scans. The resolution was 4 cm⁻¹. The aperture of the instrument was 6 mm, and the acquisition time was set to 3 min.

2.2. The Proposed Computational Approach for the Preliminary Identification of Pigment Mixtures

2.2.1. Camera, Color Target, and Spectrocolorimetry

The images could not be taken in a dark room, and, therefore, they had to be transformed to respect the conditions of the D65 standard illuminant since they were acquired around midday on sunny days. A Samsung NX3300 digital camera that had been modified by MADAtec Srl (Pessano Con Bornago, Italy) was employed. The lens was mounted on a camera manufactured by Olympus (Tokyo, Japan). A UV/IR cut-off Hoya filter was attached to the lens to allow only visible light to pass.

The camera was placed on a tripod at a distance of approximately 50 cm from the paintings. The images were acquired in the sRGB color space, and their resolution was high, namely, 3648 × 5472. They had a 16-bit quantization level. Therefore, the FADGI performance level was between two and three stars [25], as the digital camera allowed a maximum of 5472 pixels on the long dimension, which matched the former level. The main camera settings are shown in

Table 1. Camera Settings.

Variable	Value
Flash	Off
ISO	400
Operation Mode	Manual
Exposure Time	1/125
Quality	RAW
F-stop	1.0
Color Space	sRGB

.

The ColorChecker^® Classic Mini Target (CC) created by X-Rite (Grand Rapids Michigan, MI, USA) [26] was placed near the artworks during the acquisition of the images to act as a colorimetric reference. This color target includes 24 colors that are generally encountered in photography and industry [27]. The CC used in this research was acquired specifically for this color analysis on images, and it has been stored in its pouch to limit color fading that might have been caused by the exposure to light.

Spectrocolorimetry was carried out for recording the L*a*b* coordinates of each of the 24 color patches of the color target. This data was useful for performing the color correction of the images. A Konica Minolta CM-26d Spectrophotometer (Tokyo, Japan) coupled with SpectraMagic NX software (version 3.31) was used. The instrument was set on a D65 illuminant, a 10° standard observer, a SCI (Specular Component Included) acquisition mode, and an acquisition time of 10 s.

2.2.2. Color Correction of the Images

The images were loaded into Imatest Master software (version 2021.2.6) [28] without being pre-processed since all the corrections including white balance could be performed within the same software and directly on the RAW format. The color target was considered the region of interest (ROI) and was manually selected in each of the photos since the use of a gold standard image was not feasible in this case due to the variance in lighting and the position of the color target (CC). A 3 × 3 color correction matrix (CCM) was obtained using the spectrocolorimetric data from the color target (CC) and the measurement of the error was carried out using ΔE₀₀ (CIEDE2000) [29]. All the set parameters are summarized in

Table 2. The settings for the calculation of the color correction matrix (CCM) in Imatest Master.

Variable	Value
Optimize	ΔE00
Type of Matrix	3 × 3
Linearization	Gamma polynomial data fit
Weighting	Using the L* color coordinate (which represents lightness)
Optimization Constraint	No constraints
Type of Calculation	Estimation of input RGB values, calculation of the XYZ matrix and xy primaries, and conversion to output RGB values.
Chroma Multiplier	The default value of 1 was kept.

.

2.2.3. Image Resizing and Data Frame Arrangement

The images were composed of 19,961,856 pixels each. Therefore, their number was too great for the computational resources of the employed laptop (i.e., hardware) to handle in a rapid manner, while avoiding any crashes. Therefore, the corrected and calibrated images were compressed to a common resolution to ensure that the comparison between them was made in a standardized manner. Each image was compressed to a resolution of 500 × 500, thus containing 250,000 pixels. This particular procedure of resizing images at the same width and height was previously used on images of artworks by Andrea Ialenti [30], who extracted the color palette of Jackson Pollock. It is true that the shape of the artworks and, thus, of the initial images varied from a square to a rectangle, but for the purpose of this study, their compression to a square shape was considered optimal since colors were uniformly dispersed in the analyzed paintings, and thus, none of them would have been underrepresented. This procedure was carried out using the resize image online tool provided by Adobe Photoshop free of charge [31].

The sRGB images were then loaded into R. However, since they were arrays, which are tridimensional, the last dimension representing the color channel (1 stands for red, 2 for green, and 3 for blue), they had to be rearranged into a data frame [32]. This allowed the analysis of the RGB color coordinates of each pixel. The data frames that were employed for each image included five columns and each row represented a pixel. Thus, the first two columns described the position of the pixel, using the x and y axes, and the last three highlighted the values of the three-color channels.

2.2.4. Image Sampling and the Selection of the Number of Clusters

Due to computational constraints, the analysis was conducted on a random sample of 10,000 pixels from each resized image. Our sampling process can be reproduced using the set.seed() function in R. To ensure the representativeness of the sample, we performed a Kolmogorov–Smirnov test [33] between our sample and the distributions of the entire image across the five numerical variables. We only included observations whose distributions were not statistically significantly different (at a 5% level) in each dimension.

Regarding the selection of the number of color clusters in each image, there are three methods that are commonly applied for approaching this problem, namely, the elbow, average silhouette, and gap statistic [34], and all three have been employed in this study. The elbow method is generally suitable for a dataset that has a limited number of K values and is based on the measurement of WCSS (Within Cluster Sum of Squares) [35]. The silhouette method assigns a silhouette value to all datapoints, and the mean of all these values is then calculated in order to determine the number of clusters, where the silhouette value (between −1 and 1) indicates the similarity between a particular datapoint and its cluster upon comparison to the rest of the clusters or centroids [34]. Gap statistic generates a gap between the curve of the reference and that of the original data, and when the gap reaches its maximum value, it means that the optimal number of clusters has been obtained [36]. Its advantage is that it can identify the presence of a single cluster, a situation that cannot be properly managed by its competing algorithms [36].

The aforementioned methods were implemented into R using: (a) the NbClust package [37], which was employed for the application of both the elbow and silhouette methods that led to the generation of the plots that emphasized the optimal number of clusters; and (b) the clusGap() function from the cluster package [38] or performing the gap statistic.

However, the obtained K-values sometimes resulted in suboptimal clustering, as they were either too high or too low [39]. In these situations, the chosen approach involved iteratively running the algorithm and adjusting the value of K until the best result was achieved based on the visually observed colors in the images of the paintings (see Section 2.2.6).

2.2.5. 3D and 2D Color Histograms (sRGB Color Space)

To analyze the distribution of colors, the 3D histograms of the images were created in the sRGB color space using the colordistance [40] and the rgl [41] packages in R. This data visualization provided not only the distribution of color from the resized images but also of the samples that have been extracted. As a result, it could be understood whether the 10,000 pixels that had been selected from each image were significant also from the visual point of view, namely, if they seemed to be grouped in color clusters and if they covered the entire sRGB color space evenly. A concentrated point cloud underlined the lack of an adequate color recognition from the image, whereas a well-dispersed color distribution offered an easily comprehensible visualization of data [42]. It could also be added that the denser the points the deeper they are embedded in the cloud, which highlights that there is a great difficulty in distinguishing them from the more marginal ones [43].

2.2.6. K-Means and Fuzzy C-Means Clustering for the Extraction of the Color Palettes

Clustering refers to the process of dividing a set of data, in this case pixels, into a specific number of groups. Clustering can be “hard” or “fuzzy”. ‘’Hard” clustering allows a pixel to be assigned to a single cluster, whereas “fuzzy” clustering allows it to be part of more clusters, thus the latter being more adequate when clusters tend to overlap [44,45]. One of the most ubiquitous hard clustering methods is K-means, which separates data into K groups [46], and was considered appropriate for grouping the pixels into clusters in the present study. It attempts to find K non-overlapping clusters, where each of these groups is represented by a centroid (which is typically the mean of the points in that cluster) [47]. However, due to the overlapping of paint layers and different colors in most of the analyzed artworks, fuzzy clustering was also used. Specifically, the Fuzzy C-means algorithm was employed to determine if better results could be achieved compared to the K-means algorithm. “C” in C-means represents the number of clusters, which are groups of points, and each point having a degree of belonging to the clusters [48]. However, both K-means and Fuzzy C-means require the number of clusters to be specified manually before running the analysis.

The color palettes were extracted from the samples using the k-means (from the stats package [49]) and cmeans (from the e1071 package [50]) functions implemented in R. The number of centroids corresponded to the number of clusters that had been previously revealed. The set.seed() function was used for making the results reproducible. The functions were applied multiple times, until the optimal color palettes were obtained, depending on whether the same color hues were extracted more than once. In all cases, the maximum number of iterations was set to 3000 to ensure optimal results.

2.2.7. Simulation of Mixtures of Pigments

The CompuPhase algorithm was used to measure the similarity between the simulated and extracted mixtures. This algorithm measures the distance between two colors (represented by RGB values), and the result of the equation that quantifies the difference is the similarity score [51], where $\overline{\mathrm{r}}$ is the weighing factor and it represents the mean of the intensity of the red channel level of the two colors, and ΔR, ΔG, and ΔB show the differences between the RGB color coordinates of the two colors:

$$\mathrm{S}\mathrm{i}\mathrm{m}\mathrm{i}\mathrm{l}\mathrm{a}\mathrm{r}\mathrm{i}\mathrm{t}\mathrm{y} \mathrm{s}\mathrm{c}\mathrm{o}\mathrm{r}\mathrm{e}=\sqrt{(2+\frac{\overline{\mathrm{r}}}{256})\mathsf{\Delta }{\mathrm{R}}^2+4\mathsf{\Delta }{\mathrm{G}}^2+(2+\frac{255-\overline{\mathrm{r}}}{256})\mathsf{\Delta }{\mathrm{B}}^2}$$

A low value of the similarity score indicates a higher similarity of the colors. Generally a score of 100 or less could indicate a match [51]. Therefore, the RGB values are weighted in order to imitate human perception better and to get the best color approximation possible [52]. The limitations of this algorithm are its publication in an informal paper and the unavailability of the scientific data that led to its development [53,54]. Despite this, to the authors’ knowledge, it is the only optimized manner to quantify the color difference of two colors represented by RGB coordinates. The colorscience [55] package in R was used for the implementation of the algorithm in this study.

2.2.8. Summarized Workflow

The proposed workflow is summarized in Figure 2, which presents the steps that start from the acquisition of the images, followed by their color correction using the color target (CC), and end with the extraction of the colors using K-means and/or Fuzzy C-means clustering, the simulation of the mixtures, and the measurement of the color differences between the secondary/tertiary colors extracted from the analyzed images and the simulated ones.

3. Results

3.1. The Pigments and Binding Media Used in the Paints of Boris Brollo

3.1.1. White Paint

According to the results provided by Raman spectroscopy, white paints are mainly composed of titanium dioxide (TiO₂) in the rutile stable phase that could be identified by the presence of two out of its four active modes, namely, E_g mode that corresponds to the symmetric stretching at 448 cm⁻¹ and A_1g mode that is linked to the antisymmetric bending vibrations at 611 cm⁻¹ [56] (see Appendix A, Figure A1a). Additionally, the wavenumber at 1008 cm⁻¹, which was assigned to gypsum, and at 1087 cm⁻¹, which was attributed to calcite, might highlight the use of a preparatory layer that contained gesso (CaSO₄·2H₂O) and/or chalk/calcite (CaCO₃). However, they might have been added to the paint by the manufacturers to act as extenders, as usually found in commercial paints [57,58].

3.1.2. Blue Paint

According to the sharp peak observed at 547 cm⁻¹, the blue paint was identified as ultramarine blue [59] both in B1 and B2 (see Appendix A, Figure A3). The presence of titanium white in the rutile form together with calcite and gypsum could also be observed.

3.1.3. Red Paint

Red paint (see Appendix A, Figure A2a) consisted mainly of Pigment Red 4 (PR4) that was firstly manufactured in 1907 [60] and is a synthetic monoazo β-naphthol colorant, being mainly used as an industrial paint, but also for producing painting and drawing materials such as paint tubes and pencils [61]. The wavenumbers that underlined the presence of this particular pigment are included in

Table 3. The main constituents of Boris Brollo’s paints identified using Raman spectroscopy in both B1 and B2.

Paint Color	Main Constituents	Band Wavenumber (cm−1)
Blue	Ultramarine blue	547 s
	Titanium white (Rutile)	447 m, 611 m
	Gypsum	1007 w
	Calcite	1087 w
White	Titanium white (Rutile)	447 s, 611 s
	Gypsum	1009 m
	Calcite	1087 m
Yellow	Monoazo yellow pigment	358 w, 389 w, 509 w, 785 m, 823 w, 949 w, 998 m, 1137 m, 1216 m, 1254 m, 1310 s, 1337 m, 1386 m, 1448 w, 1484 vs, 1534 w, 1562 w, 1619 s, 1670 w
	Titanium white (Rutile)	447 s, 611 s
Green	Phthalocyanine green (PG7)	683 s, 738 m, 773 m, 1214 m, 1280 m, 1333 m, 1440 w, 1531 s
	Calcite	1085 w
Red	Pigment Red 4 (PR4)	421 w, 593 w, 626 w, 708 w, 767 w, 892 w, 985 w, 1095 w, 1123 m, 1181 w, 1224 w, 1266 w, 1337 vs, 1397 m, 1452 w, 1485 w, 1553 w, 1586 s
	Gypsum	1010 w
Brown (present only in B1)	Pigment Red 4 (PR4) and phthalocyanine green (PG7)	683 w, 1214 w, 1280 m, 1531 w (these peaks are characteristic to PG7), 1337 w, 1394 w, 1450 w (might be characteristic to both PG7 and PR4)
	Titanium white (Rutile)	447 s, 611 s

Intensity is labelled as w = weak, m = medium, s = strong, vs = very strong.

.

3.1.4. Yellow Paint

The characteristic wavenumbers of the yellow paint revealed the presence of a monoazo pigment, most probably PY1, a type of Hansa yellow (see Appendix A, Figure A2b). However, the spectra of PY3 and PY83, which belong to the bisacetoacetarylide and diarylide yellow class of pigments, are very similar to PY1. The differentiation was accomplished by the presence of the 845 cm⁻¹ peak that can be identified only in the case of PY1. PY3 was also discarded because of the absence of the bands at 746 and 650 cm⁻¹, the former being attributed to the C-H bonding, while the latter to the C-Cl stretching [62].

3.1.5. Green Paint

Based on the recorded spectra for the green paint (see Appendix A, Figure A3a), the pigment that was used was phthalocyanine green (most probably the PG7 type) in the case of both analyzed paintings, according to the bands shown in Table 3. PG7 was first commercialized in 1936, and it is still used to this day as it consists of a copper polychloro phthalocyanine that includes 14 to 15 chlorine atoms [63], and thus, it has a less toxic nature with respect to other traditional pigments. Titanium white in rutile form could not be detected, signaling an almost pure green.

3.1.6. Brown Paint

Only brown paint could be identified as a mixture in the case of B1. The spectrum highlighted in Appendix A, Figure A3b, suggests the presence of a blend of pigments, namely, green and red. However, green paint seems to be more abundant as the bands corresponding to it are more pronounced, namely, the 1537, 1286, 1212, 1085, 818, 740, and 684 cm⁻¹ wavenumbers that were previously assigned to phthalo green (PG7). This might be due to the presence of titanium white in rutile form that was confirmed due to the 611 and 446 cm⁻¹ bands and that might diminish the bands attributed to Pigment Red 4. However, there are three bands within the brown paint spectrum that might correspond to the red synthetic organic pigment, namely, at 1450 cm⁻¹, 1396 cm⁻¹ (very weak), and 1338 cm⁻¹. It should be mentioned that they might also be attributed to Pigment Green 7.

3.1.7. Binding Media

Despite the previous assumptions of the use of an acrylic binder, it was observed that its characteristic bands were absent in the Raman spectra. Instead, the wavenumbers that might be characteristic to poly(vinyl) acetate were identified. Figure A4a,b from Appendix A indicate a maximum band at 2938 cm⁻¹ and a second band at around 2878 cm⁻¹ and are quite consistent with the results obtained by De Sá et al. [64] regarding the identification of the spectral markers of poly(vinyl) acetate. Moreover, ER–FTIR spectroscopy seemed to further strengthen the use of these synthetic polymer-based paints. This binder might be confirmed by the bands at 1438 and 1380 cm⁻¹ that were assigned to δ(CH₂/CH₃) (see Appendix A, Figure A4c). The existence of this doublet does not only diminish the probability of the use of an alkyd binder since it would have lacked these two bands but also that of acrylic. In the latter case, the wavenumbers would have been found at 1460 and 1390 cm⁻¹, and their intensity would have been reversed, the former being medium and the latter weak. Other characteristic bands are shown in Appendix A, Figure A4d, together with the two wavenumbers that highlight the presence of gypsum at 5144 cm⁻¹ and 5064 cm⁻¹ that belong to the hydration water within gypsum, namely, ν₁/ν₃ OH and ν₂ OH [65]. The vinyl-based binder can be mainly distinguished through the presence of the 4439 and 4344 cm⁻¹ bands that were assigned to the ν + δ(CH), and the second overtone of δ(CH₃), visible at around 4030 and 3980 cm⁻¹ [66]. However, the degradation products of poly(vinyl) acetate could not be individualized with the use of ER–FTIR. It should be noted that the degradation of poly(vinyl) acetate can be enhanced by the presence of ultramarine blue and calcite as they can speed the scission rate of the polymer [67]; thus, more research is recommended in order to fully understand the degradation process of Boris Brollo’s artworks. Moreover, to fully confirm the presence of PVAc, Py-GC/MS could be used.

3.1.8. The Technique of Boris Brollo

The obtained results in the case of the color palette of B1 and B2, which are indicated in Table 3 point out the use of the same pigments, even when a brown paint was employed in the case of the former, since it was observed that it was most probably a mixture of red (Pigment Red 4) and green (Pigment Green 7). Blue, yellow, and brown paints seem to have been depicted over the white paint as titanium white (TiO₂) in rutile form was found in all spectra corresponding to these three colors. The binding media that was employed according to Raman and ER–FTIR techniques was poly(vinyl). The preparatory layer is most probably composed of gesso (CaSO₄·2H₂O) and calcite (CaCO₃); however, the binder that was used was unidentifiable using Raman spectroscopy.

3.2. The Application of the Proposed Computational Workflow

3.2.1. The Color Correction of the Images

ΔE₂₀₀₀ (input–reference) was automatically measured using the software, which represented the color difference between the CIELAB coordinates of the CC in the original image (measured using Imatest Master (version 2021.2.6)) and the corresponding color coordinates of the implemented data of the CC (measured with a spectrophotometer). Its values ranged from 5.79 to 21.34, and significantly decreased after performing the color correction. The average color difference (corrected-reference) for all ten analyzed paintings was 4.14, which is considered to be within the acceptable limit for commercial reproduction as it is between 3 and 6 [68].

As shown in

Table 4. The average ΔE₂₀₀₀ (corrected-reference) of each artist’s paintings and all the sixteen paintings.

ΔE2000 (corrected-reference) (Guido Cadorin’s paintings)	4.77
ΔE2000 (corrected-reference) (Andreina Rosa’s paintings)	3.68
ΔE2000 (corrected-reference) (Boris Brollo’s paintings)	3.59

, this value was strongly influenced by the images of the paintings of Guido Cadorin, most probably due to the presence of varnish layers in C1, C3, C4, C5, and C6 that covered the real hues of the colors and made them seem darker. Moreover, the registered L* coordinate of the black patch (F4) of the reference CC might have been too high since this is one of the main limitations of spectrocolorimetry, black color appearing lighter than it actually is [69]. In addition, the software reported the highly saturated level of several color patches resulting from the color correction using the reference CC in the case of the four aforementioned paintings. These cannot be optimally corrected, and, as a consequence, they are not altered and are marked as “Sat” [70]. Therefore, the increase in the saturation of colors could have been triggered by the presence of a varnish layer. In spite of this, the images corresponding to the paintings of Andreina Rosa and Boris Brollo presented a color difference (corrected-reference) of less than 4. These results might be explained by the absence of varnish layers and the presence of vibrant colors. It might be hypothesized that the more limited the color palette of the artist is, the lower the color difference is and, thus, the better the result is.

3.2.2. The Selected pixel Samples and Their Color Histograms

In the case of Guido Cadorin, the color histograms provided an additional confirmation of the inadequate distribution of colors. The 3D color histograms indicated that the pixels were quite concentrated in the middle of the sRGB color space, highlighting the poor representation of colors, especially in the case of C4 (Figure 3a). Similarly, the two main components extracted through PCA for creating a 2D histogram seemed to show the predominance of the red color, which was clearly untrue. Conversely, the R3 painting of Andreina Rosa showed a more uniform distribution consisting of three principal colors, namely, red, green, and purple/blue (Figure 3b). However, the color histograms of Boris Brollo were considered to be the most representative as they allowed a clear first glimpse of the color palettes, especially when they corresponded to the 10,000 pixels samples that had been selected. Moreover, at least five non-overlapping clusters could be visually distinguished in the sample of B2, which have been marked accordingly in Figure 3c. Since the 3D color space of the images was well defined, the quality of the images could be confirmed once more. Nonetheless, observing all the obtained color histograms, it could be hypothesized whether the cluster analysis would provide satisfactory results, as it was assumed that the better dispersed and separated the colors were, the better the extraction of the color palette was.

3.2.3. The Selection of the Number of Clusters

The selection of the K-number of color clusters showed unclear elbow inflection points in the case of Guido Cadorin and Andreina Rosa since the points that followed the automatically selected optimal elbow were not minimally changing as indicated in the literature [35]. As a result, it was concluded that this method was not suitable for the detection of the number of clusters in 13 images out of 16 in the present study.

Similarly, in the case of Guido Cadorin and Andreina Rosa, the average silhouette coefficient $\overline{\mathrm{s}}$ $\in$ (0.3, 0.4) accounted for 61.66% of the identified K values, which highlights a poorly reliable partition [71]. Despite this, for the three images corresponding to Boris Brollo $\overline{\mathrm{s}}$ $\in$ (0.6, 0.8) represented 70% of the K values, which suggests a reliable/very reliable partition.

The third method that was employed, namely, gap statistic, tended to both underestimate and overestimate the number of clusters [72]. The former was probably caused by the high vicinity of some clusters, which were at the same time too far apart from the rest. Instead, the latter might have been generated by the high vicinity of all clusters, and therefore, neither one of them could be well-differentiated from the others. For Guido Cadorin, this method offered satisfactory results only in the case of C2 and C6. Interestingly, for C1, which was clearly darker than the rest of the artworks, nine clusters were identified that might seem to be an acceptable outcome, but still the result was discarded due to the presence of a highly oxidized varnish layer that could have influenced the K-value. Furthermore, the images of Andreina Rosa’s paintings did not prove to be appropriate for the gap statistic method probably because the artist had a predilection for the use of overlapped brushstrokes, and, thus, the division of data into color clusters was not feasible. In contrast, in the case of Boris Brollo, the gap statistic seem to have performed particularly well in the analysis B2 and B3 as the number of clusters corresponded to that of the visually observed colors. However, the identified K-values of B1 were slightly underestimated.

Consequently, it was noticed that in the paintings that were characterized by a multitude of overlapped brushstrokes, neither one of the three methods performed well enough. However, if paints were applied uniformly, the clusters could be more easily distinguished, as in the case of Boris Brollo. As a result, the number of identified clusters corresponded not only to the visually observed colors employed by the artist but also to whether they could be well distinguished from one another (

Table 5. The selected number of clusters for each of the 10,000-pixel samples of the images. The last column shows the number of centroids that were used as input in the clustering algorithms.

Painting Index	Number of Clusters (Elbow Method)	Number of Clusters (Silhouette Method)	Number of Clusters (Gap Statistic Method)	The Final Selected Number of Clusters
C1	5	2	9	6
C2	4	2	6	7
C3	6	2	2	8
C4	5	2	5	3
C5	7	2	3	7
C6	5	3	11	10
C7	7	3	1	4
R1	5	2	1	15
R2	8	2	4	6
R3	6	2	1	12
R4	6	2	3	8
R5	9	2	14	12
R6	6	3	4	8
B1	7	9	6	9
B2	6	9	8	7
B3	8	7	7	7

).

3.2.4. The Extracted Color Palettes Using K-Means and Fuzzy C-Means Clustering

The extracted palettes of Guido Cadorin seem to have been largely influenced by the presence of varnish and dark colors. However, trials were conducted using the selected number of clusters, which generally reported uncharacteristic colors. Even in the case of C2 that had not been covered with a protective coating, the colors did not seem to represent the artwork. Moreover, the use of black paint was observed in the case of the hair of the character, and the carbon-black pigment was also identified in the red, blue, and purple colors according to the results of Raman spectroscopy [15].

Thus, the correction of the colors might have been substantially influenced by the difference in the black patch (F4) between the CC that was present in the image and its reference CIELAB coordinates that proved to be 5.55; thus, darker colors appeared brighter than they actually were [69]. Accordingly, the more the black pigments were used the higher the color difference and the poorer the extraction of the colors. Additionally, a positive correlation was observed between the length of the paintings and the color difference corrected-reference of the black patch (F4), thus, the greater the length the higher the difference and vice versa. This might have occurred due to the fact that the CC did not cover more than 10% of the image [73] in the case of larger paintings, which made the color correction difficult to carry out properly.

As a result, neither one of the color palettes that had been extracted from the analyzed paintings of Guido Cadorin through clustering could be used for the simulation of mixtures since the primary colors could not be identified, and the secondary ones were not considered representative, and colors were generally characterized by the presence of black paint that consisted of rather small patches of pure pigment or presented a varnish layer.

Nevertheless, since the paintings were not protected by a varnish layer, the extraction of the color palettes could be better performed in the case of Andreina Rosa. The two clustering methods were efficient only in R1, R3, R5, and R6, probably because they were characterized by bright colors that did not include carbon-black pigments. However, Fuzzy C-means clustering gave a slightly better performance than K-means clustering in all the four images, most probably due to the abundance of overlapped colors as shown in Figure 4. Consequently, mixtures could be simulated in the further step, using the individual primary colors, namely, red and blue (for obtaining purple) and yellow and blue (for obtaining green).

In the case of Boris Brollo’s paintings, colors were well separated in all the three images. K-means and Fuzzy C-means clustering provided quite similar outcomes. Moreover, the selection of the color clusters was straightforward, probably due to both the vibrancy of colors and their presence on a large surface of the paintings. The extracted colors generally corresponded to the visually observed ones except for gray that was identified in B1 and beige in B3, respectively. Through the plotting of the pixels according to the colors that they have been assigned in the clustering process, the gray color was identified where black had been used for rendering the contour lines of the represented silhouettes, most probably due to their non-uniformity and closeness to white paint. The two beige colors extracted from B3 were specific to the newspaper that was attached to the canvas and conveyed a part of the artwork’s message. Figure 5 shows the color palettes that have been extracted from the image of B1 and highlights the possibility of simulating the green color using blue and yellow and the brown color using red and green.

3.2.5. The Simulated Mixtures and the Measurement of the Color Difference

In the case of Andreina Rosa, Fuzzy C-means clustering led to a better simulation of the mixtures and to a measured colored difference that was less than the one resulting from the K-means clustering extraction. Therefore, the presence of color mixtures could be more clearly observed in the case of R3, R5, and R6 after blending the individual primary/secondary colors that were the outcome of the clustering procedure. However, the similarity score was still less than 100 as it was previously observed after applying the K-means clustering algorithm. However, Fuzzy C-means seemed to segment colors better when overlapping is present (

Table 6. The similarity scores of the simulated mixtures created for the images of paintings that allowed a proper segmentation of the colors, namely, R1, R3, R5, R6, B1, B2, and B3.

Painting Index	Mixed Colors	Color to be Achieved	Percentage of Colors in the Mixture	CompuPhase Color Difference (Colors Extracted Using K-Means Clustering)	CompuPhase Color Difference (Colors Extracted Using Fuzzy C-Means Clustering)
R1	blue and red	purple	50% blue + 50% red	24.01	11.59
			60% blue + 40% red	4.26	1.76
			70% blue + 30% red	17.42	13.64
			80% blue + 20% red	37.38	26.29
R3	blue and yellow	green	40% blue + 60% yellow	53.21	34.4
			50% blue + 50% yellow	37.33	31.91
			60% blue + 40% yellow	31.02	43.54
			70% blue + 30% yellow	38.97	61.61
	red and blue	purple	40% red + 60% blue	54.71	45.28
			50% red + 50% blue	52.37	44.60
			60% red + 40% blue	52.63	46.50
			70% red + 30% blue	55.45	50.70
R5	red and green	brown	50% red + 50% green	51.92	37.34
			60% red + 40% green	56.48	43.10
			70% red + 30% green	61.96	49.97
			80% red + 20% green	68.15	57.58
R6	red and blue	purple	50% red + 50% blue	52.15	30.97
			60% red + 40% blue	49.98	29.54
			70% red + 30% blue	50.58	32.18
			80% red + 20% blue	53.83	38.03
B1	red and green	brown	50% red + 50% green	51.68	64.6
			60% red + 40% green	28.19	38.7
			70% red + 30% green	52.12	54.93
			80% red + 20% green	93.42	95.37
	yellow and blue	green	30% yellow and 70% blue	124.4	130.03
			40% yellow and 60% blue	120.4	147.6
			50% yellow and 50% blue	144.53	186.02
			60% yellow and 40% blue	186.7	235.74
B2	yellow and blue	green	30% yellow and 70% blue	127.13	127.48
			40% yellow and 60% blue	125.61	126.2
			50% yellow and 50% blue	148.14	148.75
			60% yellow and 40% blue	186.96	187.44
B3	yellow and blue	green	30% yellow and 70% blue	126.48	147.25
			40% yellow and 60% blue	125.57	165.52
			50% yellow and 50% blue	149.07	200.29
			60% yellow and 40% blue	188.86	245.24

).

As it has been previously remarked, Boris Brollo used a limited color palette, preferring vibrant colors that have not been modified through tinting and shading. The sole color mixture that seems to have been present in the artist’s paintings according to the Raman spectra consisted of red and green; thus, a brown color was observed in B1. This could be simulated using the RGB values of the almost pure colors that were previously extracted with the use of R that corresponded to red and green, namely, to Pigment Red 4 and Pigment Green 7. Four mixtures were generated, which are shown in Figure 6, and the measured color differences between the original extracted brown color and the simulated mixtures are shown in Table 5. The similarity scores after the application of K-means and Fuzzy C-means were quite similar, but their values were more reduced in the case of the former, which might highlight that this clustering method is sufficient when colors are not overlapped. In spite of this, it was concluded that the optimal color mixture was obtained with 60% red and 40% green, whose similarity score after K-means was 28.19, which is considerably less than 100, the value that marks the beginning of the mismatch region between colors [51]. However, it is worth mentioning that the other three mixtures also had a CompuPhase color difference of less than 100, confirming once again the presence of a mixture, but with a slight change in the color percentages.

To further validate the computational approach presented in this study, green paint was also simulated in the case of B1, B2, and B3 by combining yellow and blue as it was already known that the pigment that was used for depicting it was green and not a mixture. Therefore, a high similarity score was expected between the color obtained through simulation and the one extracted from the corrected and resized image. The results shown in Table 6 emphasize values that surpass the 100-threshold, both after the application of K-means and Fuzzy C-means clustering, which indicates a strong mismatch between the analyzed colors. Thus, the lack of a mixture of yellow and blue could be confirmed. Figure 7 shows that the visual color difference between the simulated green and the extracted green from B1, B2, and B3 is significant.

Moreover, even though B3 had not been previously investigated, it was observed that the similarity scores between the extracted and the simulated green were similar to those of the same color in B1 and B2 as shown in Figure 8 since the curves corresponding to the color difference of each of the simulated green paints follow a very similar trend. Therefore, it is highly probable that the artist not only used Pigment Green 7 in B3 but also Pigment Yellow 1 and ultramarine blue.

4. Conclusions

The presented computational workflow proved to be efficient in the preliminary identification of mixtures of primary/secondary−colored pigments in the case of modern and contemporary paintings that have been depicted using vibrant colors and without the subsequent addition of varnish layers. The procedure is quite straightforward, and it can be easily carried out through the use of a digital camera, a color target, a spectrocolorimeter, and two computational resources, namely, Imatest Master and R. Regarding the extraction of colors, K-means clustering seems to perform better when paint layers are not overlapped, whereas Fuzzy C-means clustering gives a more optimized color palette when paint layers are overlapped.

Consequently, this computational approach might be particularly successful in the preliminary analysis of paintings such as those of Andreina Rosa and Boris Brollo, as it could indeed guide the sampling procedure. Therefore, a sample from the mixture might also lead to the identification of the individual constituent pigments. However, further research is recommended for the understanding of the effects of the binding media on the color of the pigments. Thus, a following step would be the creation of mock-ups using various paints and binders in order to observe both the behavior of the materials and their change in the color difference over time using the proposed methodology.

In spite of this, the paintings of Guido Cadorin served as proof that this methodology is not adequate for varnished and dark-colored paintings since the extracted colors do not correspond with the true colors of the paint layers even after performing color correction. Therefore, this workflow should be applied strictly when the paint layers have vibrant colors, have not been covered by varnish layers, and have not suffered from fading.

However, the computational approach presented in this study has the potential to contribute to the development of a less invasive method for investigating and diagnosing painted and unvarnished artworks that contain synthetic organic colorants. Reducing the number of samples obtained from the paintings could help in preserving them for future generations. This approach would also contribute to supporting tourism, as visitors would be attracted to artworks that have been maintained in the best possible condition. Moreover, an additional benefit would be a decrease in electricity consumption, as the analytical instruments would need to be used for shorter durations.