Cognitive Computing for Understanding and Restoring Color in Renaissance Art

Abstract

In this article, for the first time on this topic, we analyze the historical color palettes of Renaissance oil paintings by using machine-learning methods and digital images. Our work has two main parts: we collect data on their historical color palettes and then use machine learning to predict the original colors of paintings. This model studies color ratios, enhancement levels, symbolic meanings, and historical records. It looks at key colors, measures their relationships, and learns how they have changed. The main contributions of this work are as follows: (i) we develop a model that predicts a painting’s original color palette based on multiple factors, such as the color ratios and symbolic meanings, and (ii) we propose a framework for using cognitive computing tools to recover the original colors of historical artworks. This helps us to rediscover lost emotional and cultural details.

1. Introduction

Art is a human way of sharing emotions, ideas, and social messages. It touches us through our senses, often without our full awareness. We can see art as both a physical object and a form of communication. Understanding how these aspects shape our response helps us to value art more deeply. Cognitive computing (CC) allows us to analyze and mimic human artworks. CC systems use many kinds of data to understand context, such as text, images, and historical records. They consider details like time, place, and user preferences. Within art, one key factor is color symbolism. Colors can set moods, highlight important parts, or stand for ideas like hope or betrayal. For example, Leonardo da Vinci’s “The Last Supper” shows Judas in shadowy tones, suggesting darkness and deceit [1,2]. Vincent van Gogh’s “Starry Night” uses swirling blues and yellows to show turbulence and emotional intensity. The blues hint at melancholy, while the yellows represent hope and light [3]. In Paul Gauguin’s “The Yellow Christ”, the bright yellow shows isolation and religious devotion, while the warmer autumn tones suggest a desire to escape from urban life [4]. Over time, colors in old paintings may fade or change, making their original meanings hard to grasp. Learning about these lost colors can help us to understand the painting’s original emotional and cultural message. Art can be seen as a unique form of cognitive engineering [5]. Art often expresses personal and collective experiences and can serve as social commentary. It is both a physical object and a communicative medium. At times, viewers may not be aware of the cultural and emotional elements behind the art they see. Distinguishing between art as a physical stimulus and art as a deeper expression of human experience [6] helps us to understand how people appreciate artworks.

Cognitive computing models can analyze and even mimic human-made art today. CC systems use various information sources, such as text, images, historical data, and user preferences. They consider context, including time, place, domain, and the goals of their users. CC simulates human thinking in a computer model. Within cognitive science, many studies focus on how our minds process visual media and aesthetics. Research suggests that the human brain evolved basic image-processing skills for survival [7]. Later, as culture developed, humans created art. In response, the mind recognizes artistic style as distinct from the subject matter [7,8]. This difference in how we process art versus real-world images is essential. It shows that the style itself is a cognitive marker.

As CC systems evolve, they can handle both structured and unstructured data. This includes visual information, historical records, symbolic cues, and artistic details. Over time, oil paintings may lose their original colors due to aging or damage. New computational tools can help us to restore or predict the original colors. Many recent studies from computer science, computer vision, color science, digital forensics, and pattern recognition have been devoted to the analysis, conservation, classification, and virtual restoration of works of art [8,9,10,11,12,13,14,15]. Techniques include digital image acquisition, indexing, web-based sharing, diagnostics, condition evaluation, and virtual restoration. Still, these methods are not yet standard among conservators, information scientists, and art historians.

Each artwork is unique. Factors like the style, dimensions, materials, and color usage vary. Renaissance artists, as well as others like Rembrandt, often used color to convey meaning. Conservation and restoration have long been studied at the intersection of art and science. However, the traditional, physical method could be harmful and alter color during the restoration process. Computer-based methods for painting analysis have several advantages. First, they are more objective, since they rely on data from many artworks. Traditional methods often depend on conservators’ personal preferences, which may alter the painting’s original character. Second, digital restoration does not physically change the artwork, avoiding irreversible damage. Third, digital restoration allows for multiple experiments, helping computers to learn from past outcomes and improve future predictions.

Oil painting gained popularity in the 15th and 16th centuries. Renaissance artists had a wide range of colors, making color composition crucial. Colors helped to make scenes more realistic, evoked emotional responses, and served as visual guides to the viewer [16]. For example, the deep blue pigment often represented the biblical Mary [17]. Leonardo da Vinci blurred edges between objects in the “Mona Lisa” to achieve soft transitions. Such techniques influence how we use machine learning to analyze and predict the original color palettes of Renaissance oil paintings.

In our approach, CC techniques were used to guide the analysis and prediction of original color palettes. We combine data from art history sources, image analyses, and known pigment properties. Our framework uses machine learning models to study color relationships, symbolic meanings, and historical records. The system can learn patterns and predict how colors originally appeared by applying CC. This involves extracting color ratios, segmentation, enhancement, and measuring distances between colors. The CC model refines its predictions through iterative learning, moving closer to the probable original hues.

In this paper, we analyze the historical color palettes of Renaissance oil paintings with machine learning methods. Our work has two main parts:

• We develop a model to predict a painting’s original color palette. This model uses color ratios, enhancement measurements, symbolic meanings, historical data, and new color distance calculations. The images are segmented to identify primary colors and these colors are then analyzed to understand their original appearance;
• We propose a CC-based framework to explore how to predict the original colors of Renaissance oil paintings. Our framework was tested on over 105 images from three famous artists—Raphael, Leonardo da Vinci, and Rembrandt—sourced from Olga’s Gallery (http://www.abcgallery.com/ accessed on 9 September 2017).

This framework will help to forecast the original color palettes of Renaissance oil paintings and other artworks. By understanding the colors, symbolism, and original ratios, we can gain deeper insights into these masterpieces’ artistic intent and historical significance.

Paper Organization: Section 2 briefly reviews related works and presents the framework of our method. Section 3 explains the symbolic interpretation of primary colors in Byzantine, Gothic, and Renaissance paintings. Section 4 analyzes painting colors and palettes, and introduces a method of color correction using color ratios. Section 5 and Section 6 present and analyze experimental results.

2. Related Works

Color degradation in paintings can result from aging binders, oxidation of pigments, exposure to light, or atmospheric pollutants. Consequently, art conservators and researchers have long sought ways to recover or preserve the original appearance of historic works. Traditional restoration involves physical methods such as surface cleaning, varnish removal, and pigment consolidation. More recently, digital and machine learning (ML) techniques have been introduced to perform non-invasive “virtual restoration” or to guide physical interventions with greater precision.

Traditional (physical) color restoration methods: Historically, restorers used organic solvents and precise manual techniques to remove degraded varnish layers or accumulated dirt. This practice can be found in Cennino Cennini’s book from the 15th century [18]. In modern conservation manuals, conservators use pigment with specific chemical properties, such as reversible, light-fast paints, to integrate abraded or missing areas. Strict ethical guidelines (the “minimum intervention” principle) ensure that any additions remain distinguishable under ultraviolet light [19]. This process can alter color if not done carefully, highlighting the need for minimally invasive techniques. Since the early twentieth century, art historians and conservation scientists have worked together, using methods like X-ray analysis [20], microscopic sampling [21], and microchemical analysis [22] for color restoration.

Digital (non-invasive) color restoration approaches: As computer science advanced, researchers began applying digital methods to overcome the disadvantages of traditional restoration methods. Genetic algorithms have been used to reconstruct murals [23], and 3D printing has been employed to recreate paintings’ appearances [24]. Computer graphics algorithms are also used to reconstruct missing or damaged regions in paintings [25]. These methods allow for the virtual reconstruction of large missing areas while preserving texture without modifying the physical materials on the painting.

Machine learning-based restoration techniques: Machine learning and deep learning now support tasks like classifying ancient paintings by their color features [26,27]. In cases where reference “before-damage” data do not exist, self-supervised learning is applied. The work [28] leveraged style transfer and colorization techniques from large sets of related artworks to infer restorations. GAN-based frameworks can auto-generate missing or faded details by learning the distribution of intact paintings. Models can be fine-tuned for a specific artist’s palette or style, potentially offering highly realistic reconstructions, such as CAN [29] and ConvSRGAN [30].

While AI and machine learning have shown promise in art restoration and color reconstruction, several challenges remain. The accuracy of AI predictions depends heavily on the quality and quantity of training data, which can be limited for certain artists or periods. Additionally, AI-generated restorations must be carefully evaluated to ensure that they do not introduce unintended visual discrepancies or compromise the artwork’s authenticity, especially generative models, such as GAN-based models [31].

In summary, digital techniques and AI have introduced innovative tools for art restoration, particularly in color reconstruction and damage detection. However, ongoing research and collaboration are necessary to harness AI’s potential while preserving the integrity and authenticity of historical artworks. The restored artwork should reflect the culture, art styles, and historical facts in the period when the work was painted. To address the mentioned problems, we proposed our framework based on cognitive computing models that can analyze different information sources, such as context, time, place, images, historical data, and user preferences for color restoration.

3. Accessing Images of Paintings

This section considers an image processing model with the color enhancement and prediction model presented on the block diagram displayed in Figure 1. The original image is processed in the frequency domain, and then, the color ratios are analyzed in both actual and enhanced images. Then, the color components of these two images are processed to keep the ratios of colors equal to the average color ratios in the images. The enhanced images and the images after color ratio correction (modification) represent, in most cases, high-quality images that can be considered examples of restoration being close to the original images. Different measurements of the images are analyzed, including the new measurements that relate to relationships between the ratios of primary colors in the paintings. The analysis of different images calculated from the original image using the enhancement methods and method of color ratio correction allows us to select a high-quality image and consider it as a predicted image for the painting.

Here, we describe in more detail the proposed method of processing color images. The images are considered in the color RGB model. The discrete color image $f_{n,m}$ of size $N\times M$ pixels is considered in the quaternion space. For this, different types of quaternion arithmetic can be used, noncommutative [32] and commutative [33]. We consider traditional noncommutative arithmetic. The quaternion number $q$ is presented as the number with one real part and a three-component imaginary part, $q=a+(ib+jc+kd)$. Here, $a,b,c,$ and $d$ are real, and $i,j$, and $k$ are the imaginary units $i,j$, and $k$, which satisfy the following multiplication rules: (1) $ij=-ji=k$; (2) $jk=-kj=i$; (3) $ki=-ik=j$; (4) $i^2=j^2=k^2=ijk=-1$; and (5) $1i=i$. The quaternion number $q$ can be considered a point in the 4D space. In the RGB color model with three primary colors, red, green, and blue, plus the gray color, the image at each pixel can be presented by a 4D vector. Thus, the image is presented as a quaternion image, or the 4D image of size $N\times M$ pixels, which is defined as (for more detail, see [32,34]):

$$f_{n,m}=a_{n,m}+(i{r}_{n,m}+j{g}_{n,m}+k{b}_{n,m})$$

(1)

The real part $a_{n,m}$ is the grayscale component, being the average of three primary colors, $a_{n,m}=(r_{n,m}+g_{n,m}+b_{n,m})/3$. The primary colors are red $r_{n,m}$, green $g_{n,m}$, and blue $b_{n,m}$, which constitute the three-component imaginary part of the quaternion image. The real part of such an image can also be calculated as the brightness of the image, $a_{n,m}=\left({0.30r}_{n,m}+{0.59g}_{n,m}+0.11b_{n,m}\right)$ at pixel ($n,m$). Such a quaternion image can also be used without a real part, i.e., when $a_{n,m}=0$ [35]. We will treat each color image as a unit using quaternion data, while the traditional approach treats each color component separately.

When enhancing color images in the frequency domain, the different concepts of the 2-D quaternion discrete Fourier transform (QDFT) can be used, since the quaternion multiplication is not commutative. Our experimental results show that the left, right, and both-side 2-D QDFTs almost do not differ in image enhancement [35,36,37,38]. Therefore, we consider the concept of the right-side 2D QDFT. This transform of the quaternion image $f_{n,m}$ and its inverse are calculated by

$$F_{p,s}=\sum _{n=0}^{N-1}\left(\sum _{m=0}^{M-1}{f}_{n,m}{W}_k^{ms}\right)W_j^{np}, f_{n,m}={\displaystyle \frac{1}{NM}}\sum _{m=0}^{M-1}\left(\sum _{n=0}^{N-1}{F}_{p,s}{W}_j^{-np}\right)W_k^{-ms},$$

(2)

where we consider the basic quaternion exponential functions with the quaternion pure units $k$ and $j$, that is, $W_k^{ms}=coscos 2\pi ms/M-ksinsin 2\pi ms/M$ and $W_j^{np}=coscos 2\pi np/N-jsinsin 2\pi np/N$. Fast algorithms of the 2D QDFT have been developed (see for detail [33,39]). Therefore, the 2D QDFT can be effectively used for processing the quaternion and color images in the frequency domain.

One of the most effective methods for improving the quality of color images is the alpha-rooting method with the 2D QDFT. It is a parameterized enhancement when the only magnitude of the 2D QDFT is changed by the real exponential function [40]. The quaternion image ${f=\{f}_{n,m}\}$ is processed into an enhanced one $g{=\{g}_{n,m}\}$ calculated by

$$G_{p,s}={\left|F_{p,s}\right|}^{\alpha -1}{F}_{p,s}, p,s=0:\left(N-1\right),\left(M-1\right).$$

(3)

The selection of best values of $\alpha$ for enhancing the color image $f_{n,m}$ is based on the known color image enhancement measures on the image $g_{n,m}$, such as the EMEC functions [38,40]. We consider this measure of quality color images in the RGB model, wherein the color image ${f=\{f}_{n,m}\}$ is the triplet of red, green, and blue color components, $f=\{f_R, f_G, f_B\}$. The enhancement measure of the color image $g=\left\{g_R, g_G, g_B\right\},$ or the imaginary part of the quaternion image, is calculated as follows:

$$EMEC\left(g\right)={\displaystyle \frac{1}{k_1{k}_2}}\sum _{k=1}^{k_1}\sum _{l=1}^{k_2}20{\displaystyle \frac{(g_R, g_G, g_B) }{\left(g_R, g_G, g_B\right)+0.01}},$$

(4)

where the discrete image is divided by $k_1{k}_2$ blocks of size $L\times L$ pixels each, and integers $k_1\le N/L$ and $k_2\le M/L$. In all examples given below, the block size $7\times 7$ is used. The maximum $(g_R, g_G, g_B)$ and minimum $(g_R, g_G, g_B)$ values of the image in the $(k,l)$-th block are calculated component-wise at pixels in the block. The blocks with zero $(g_R,g_G, \mathrm{and} g_B)$ are not considered in this measure (or one can be added to colors for such blocks).

For a full quaternion image $g=\left\{g_a,g_R, g_G, g_B\right\}$ with the grayscale component $g_a$ in the real part, a similar enhancement measure of the image is calculated [32,34]

$$EMEQ\left(g\right)={\displaystyle \frac{1}{k_1{k}_2}}\sum _{k=1}^{k_1}\sum _{l=1}^{k_2}20{\displaystyle \frac{({g_a,g}_R,g_G, g_B) }{\left({g_a,g}_R,g_G, g_B\right)+\epsilon }},$$

(5)

where $\epsilon$ is a small number, such as 0.001. The maximum and minimum values of colors are calculated together with the gray values in each block.

These measures calculate the average range of colors in the logarithmic scale according to the definitions. Both measures are calculated on the original color image $f_{n,m}$ and enhanced image $g_{n,m}$, and they then are compared. Typically, visually enhanced image scores are high for these measures. In the alpha-rooting method, the enhanced image is parameterized by $\alpha$; the same is the enhancement measure. Therefore, the best parameter for color enhancement is considered the value of $\alpha$ with a maximum of $EMEQ\left(g\right)$ or, in some cases, minimum of $EMEQ\left(g\right)$ Our experimental results show that the measures EMEQ and EMEC are practical tools for selecting the best or optimal parameters to receive high-quality images [40,41]. Other measures for estimating the quality of enhanced images can also be used, including color image contrast and quality measures [42,43].

We introduce the following three measures (also used as features) to estimate the relationship between the primary colors in the RGB model. They are the measurements of the red-to-blue, green-to-blue, and blue-to-red colors, which we denote by $M_1\left(g\right)=M_{R/B}\left(g\right)$, $M_2\left(g\right)=M_{G/B}\left(g\right)$, and $M_3\left(g\right)=M_{B/R}\left(g\right)$, respectively. Here, the measurement of two colors, for instance, red-to-blue, is defined as

$$M_1\left(g\right)=M_{R/G}\left(g\right)={\displaystyle \frac{1}{NM}}\sum _{n=0}^{N-1}\sum _{m=0}^{M-1}{\displaystyle \frac{{{(g}_R)}_{n,m}-\left(Mean{(g}_R\right)) }{{{(g}_B)}_{n,m}-\left(Mean{(g}_B\right))+\epsilon }},$$

(6)

where $\epsilon$ is a small number, such as 0.001. $Mean{(g}_R)$ and $Mean{(g}_B)$ are the mean values of the red and blue color components of the image, respectively. The other two measures are defined similarly. As the mean average of these measurements, we consider the measure

$$M\left(g\right)=\sqrt[3]{M_{R/B}\left(g\right)M_{G/B}\left(g\right)M_{B/R}\left(g\right)}.$$

(7)

Different measurements of the images are analyzed for the selection (Step-2 in Figure 2), and the main steps of the selection method are as follows:

• Generate a class of images from a received image using the alpha-rooting method, including the high-quality image with the best parameter using EMEQ and EMEC. For example, the best oil painting image quality is the image (c), which was generated using $\alpha =0.87$ rooting method, where EMEQ (short $E_q\left(\alpha \right)$) attains the maximum value (see Figure 2a);
• Use 11 features, such as colors, symbols, and others, and pick the best-predicted original color palette of Renaissance oil artworks from the generated class of images. These features included five colors and six image quality-related features;
• Calculate four-color distance values (CCD) between the color of the analyzed image and the color of the material used in that period (CCD1), the artist’s painting preference of colors (CCD2), art movement color preference (CCD3), and color symbolism of painting contents (CCD4);
• Apply PCA to 11 values and list the sum of the top 1–8 values for each alpha;
• Utilize a ranked voting system (in Section 6) using PCA on 11 measurements (M1, M2, M3, M, EMEC, EMEC2, CCD1, CCD2, CCD3, CCD4, and CRD) and select the alpha-rooting image corresponding to historical color palettes.

4. Colors and Symbols

This section briefly considers the symbolic interpretation of the primary colors used in Byzantine, Gothic, and Renaissance paintings. The following colors were used in Renaissance art [44,45,46,47,48,49,50,51].

• Gold—used as a background color or in a halo; symbolizes purity, royalty, and glory of life after death. Gold is associated with wealth, royalty, and heavenly rewards and riches;
• Blue—symbolizes purity; The Virgin Mary; Virgin and Child; and The Immaculate Conception;
• Purple—symbolizes Christ in Majesty in Byzantine-Style paintings. Important Holy figures wear purple robes outlined in red;
• Red—a symbol of greediness and lust. Denotes sin; sins of mankind, original sin; temptation, Judas, Harrowing of Hell, The Fiery Furnace, Slaughter of the Innocents, and Apocalypse. Red also denotes power, wealth, and authority. Many Renaissance artists were praised for the virtuoso use of red pigments in their paintings. For example, Titian was recognized for his brilliant reds, particularly vermillion. Additionally, Jan Van Eyck, draped figures and interiors in his paintings in rich crimson to signify their high social status or sanctity. In the past, red pigments were rare and expensive during various periods of color history, which helped to pave their way into the wardrobes and palettes of rich and powerful people. Only the rich and powerful could afford to wear clothes of red color;
• Green—symbolizes the Resurrection, the Ascension, and Baptism. Green also symbolizes peace, spring, spiritual renewal, rebirth, and new life;
• White—is a sign of innocence; Birth, Youth, Betrothal, and Marriage; The Virgin Mary; Virgin and Child; The Immaculate Conception, The Holy Family, and angels;
• Grey/Black/Dark Brown—symbolic of the Entombment, Crosses, Crucifixes, darkness, misdeeds, death, and witchery;
• Yellow—symbolizes a remembrance of the spiritual world, miracles, harmony, and soul sustenance. Yellow also symbolizes spring, spiritual renewal, rebirth, and new life;
• Pink—symbolizes eternal innocence; The Virgin Mary; Virgin and Child; and The Immaculate Conception;
• Orange—a symbol of materialism and desire for worldly goods in favor of spiritual health. Orange also denotes indulgence, carnal desires, and original sin.

In the Renaissance, colors were used and chosen based on several factors, including the following:

• Color costs: for example, deep blue color ingredients were so expensive that they were saved for only special parts of a picture (for the clothing of the biblical Mary or a woman wearing clothes), because they were made from minerals, such as the gemstone lapis lazuli, in a fine powder and mixed with other ingredients;
• Aesthetic or Technical Purposes:
  • To create a three-dimensional space, which means making a scene look as if you could almost walk into it.
  • The color was often chosen, and also served as a symbol in selected cases.
  • Context determines the meaning of color.
• Local Culture and Geographic Area.

Table 1. Color and color symbolism.

Color and Its Properties	Renaissance	Medieval
Red Wavelength interval of 700–635 nm Frequency interval of 430–480 THz RGB (255, 0, 0)	It is associated with high social status, royalty, gentlemen, and men of justice (Scotland, the Holy Roman Empire, England’s Court of Common Pleas, occasionally by peers in English Parliament or a man with access to international trading centers) [11,13] It is associated with high government posts (Venice and Florence), royal magistrates, and the king’s chancellor (France) Titian was renowned for his brilliant reds, particularly vermillion. Purple/red is the color of royalty, wealth, and power. Is associated with power and prestige [11] Is associated with authority, Pentecostal fire, the blood of Christ, martyrdom, crucifixion, and Christian charity Symbolizes the satanic and color of hellfire [11] Is associated with medicine (at the universities of Padua and Bologna) [10]	A lover wears vermilion, like blood’ (later Middle Ages) [14] A sign of otherworldly power in European legends and folktales Protection: red thread to ward off witches, red coral necklaces to guard against illness [12] The Virgin Mary’s robes [12] Identified with kingly virtues of valor and success in war [8] Fire [12]
Green Wavelength interval of 560–520 nm Frequency interval of 540–580 THz RGB (0, 255, 0)	Symbolizes love and joy [11] Associated with youth [10] Associated within the secular sphere, chastity [13] Symbolizes nature and health [44]
Blue Wavelength interval of 490–450 nm Frequency interval of 610–670 THz RGB (0, 255, 0)	Light blue represented a young marriageable woman [11] It was the traditional color of servitude (England). Servants or members of a city company were to wear bright blue or gray Renaissance clothing [46] Deep blue is associated with chastity in the sacred sphere [13] “turquoise was a sure sign of jealousy“. [12] Symbolized purity. It was used for robes of Christ and the Virgin [17] Associated with wealth [17]	Blue replaced royal purple in the mantle of the Virgin Mary and robes and heraldry (especially in France) [51] A lover wears blue for fidelity (late) [14] Blue was associated with darkness, evil. Later, blue became associated with light [11]
Orange Wavelength interval of 635–590 nm Frequency interval of 480–510 THz RGB (0, 255, 0)	Is associated with peasants and middle-ranked persons who imitate upper-class reds, by dyeing their Renaissance clothes with cheaper orange-red and russet dyes [12].	Illness
Yellow Wavelength interval of 590–560 nm Frequency interval of 510–540 THz RGB (255, 255, 0)	Is associated with prostitution (in almost all Italian cities, it was required to wear yellow) [11] Is associated with Jews (in Venice, they were demanded to sew a yellow circle on their clothes) [11] Is very useful in attracting attention Sunshine, happiness, and optimism	Yellow color expresses the balance between the red of justice and the white of compassion [15]
Purple Wavelength interval of 450–400 nm Frequency interval of 670–750 THz RGB (255, 255, 0)	Is associated with the Medici family in Florence [11] (Imperial purple that disappeared in 1453) [12]
Gray	Is associated with modest and religious dress [48] Symbolizes poverty [12] Symbolizes lower classes (for example, in England, servants or members of a city company were required to wear bright blue or gray [49]).
Brown RGB (152, 72, 7)	Symbolizes modest and religious dress [12] Is associated with the color of poverty [12] Dull browns were worn by lower classes (In England) [12] Symbolizes simplicity, health, and nature [52] Represents gradations from light to dark [53]
Black	Symbolizes seriousness [51] Symbolizes mourning [11] Symbolizes nobility and wealth, representing refinement and distinction [12]
White	Symbolizes purity [8] Symbolizes light

summarizes colors and color symbolism use and meanings. Pariqurerly indicates that the colors are highly related to the painting contents (for example, it shows specific associations between colors and a person’s emotions, behavior, or cultural differences) and can partially represent the original color palette.

Table 2. Colors, pigment color, and historical period.

Color	Pigment Color	Used Period
Blue RGB (0, 255, 0)	Azurite, RGB (49, 80, 101)	Mid 1400s–1500s [17]
	Ultramarine, RGB (18, 10, 143)	1300s–1600s [17]
	Smalt, RGB (0, 51, 153)	1400s–Late-1500s [17]
Yellow RGB (255, 255, 0)	Lead Tin Yellow, RGB (255, 230, 74)	Early 1400s–Mid-1700s [54]
	Naples Yellow, RGB (250, 218, 94)	Mid-1700s–Mid-1800s [55]
Green RGB (0, 255, 0)	Verdigris, RGB (67, 179, 174)	1400s–1600s [56]
	Malachite, RGB (78, 129, 96)	1400s–1500s [57]
Reds RGB (255, 0, 0)	Vermilion, RGB (227, 66, 52)	1600s–1700s [58]
	Red Lead, RGB (204, 51, 0)	1400s–1800s [59]
White	Lead White, RGB (240, 235, 229)	Ancient–1800s [60]
Brown RGB (152, 72, 7)	Vandyke Brown, RGB (68, 54, 47)	1500s–Present [61]
	Umber, RGB (88, 62, 35)	1400s–1800s [62]
Black RGB (0, 0, 0)	Ivory Black, RGB (35, 31, 32)	Ancient–Present [63]

shows the color with pigment colors and the historical period of their use.

5. Color Palettes and Color Distance

We have seen in Section 3 that colors are highly related to a painting’s contents (for example, it shows specific associations between colors and a person’s emotions, behavior, or cultural differences) and can partially represent the original color palette [64]. The color palette is the range of color characteristics of a particular artist, painting, or school of art [65,66]. This section uses information on color palettes (Table 1) to calculate the color distance (CD). In this section, we aim to list all the color-related and quality-related values for each painting. We will analyze 11 features, including five color-related and six quality-related, to make the final decision for alpha. We only demonstrate one color palette. The other all-color palette can be created similarly.

Color palettes have been used to analyze painting colors. They refer to the major colors one image contains or the colors of pigments the artist used most in their painting. However, there are no standard original color palettes for Renaissance Arts. We propose a way to generate these original color palettes. Based on the observation and some art history research [17,52,53], we believe that the actual color will be impacted by (1) the material used in that period, (2) the artist’s painting preference for colors, (3) art movement color preference, and (4) the color symbolism of painting contents. These original color palettes will be treated as target color palettes.

To generate principal colors for predicted color palettes, we tested three color clustering algorithms: K-means [26], fuzzy C-means [67], and K-medoids [68]. To compare the performance of these algorithms, we selected the color palettes generated by Google Art Palette [69] as the standard color palettes. Google Art Palette produces five principal colors for each image. We calculated the color distance (CD) value of standard color palettes and then predicted color palettes generated by the three clustering algorithms. The color distance is calculated by the following equation:

$$CD=CD\left(P,S\right)={\displaystyle \frac{1}{k}}\sum _{i=1}^{k}argarg \{Colordis(P_i,S)\},$$

(8)

where the predicted color palettes are denoted by $P_i$, and $S$ is the standard color palette. In this equation, $i$ represents the color index in palette $P$, and $k$ is the number of colors in $P$. The function ‘$Colordis$’ outputs a list of the Euclidean distance values between the color $P_i$ and all the colors in $S$. The length of the output list is the same as the length of $S$. For example, if $S$ consists of three colors, Colordis($P_i,S$) will output a list with three values representing the Euclidean distances between $P_i$ and each color in $S$. The function argmin{.} will provide the smallest distance in this list. This distance value will be regarded as the color distance between color $P_i$ and the closest color in $S$. The distance between two colors $C_1=(r_1,g_1,b_1)$ and $C_2=(r_2,g_2,b_2)$ in the RGB color model is calculated as $d\left(C_1,C_2\right)=\sqrt{{\left(r_1-r_2\right)}^2+{\left(g_1-g_2\right)}^2+{\left(b_1-b_2\right)}^2}$. Thus, the average value of the smallest color distances describes how close the two-color palettes are. A smaller CD value means that the algorithm obtained similar color palettes to Google Art Palette. The CSR is a value for estimating the similarity rate between the two-color sets. The higher the value is, the closer the two-color sets are. The color similarity rate is calculated by $CSR\left(P,S\right)=1-CD/\left(255\sqrt{3}\right)$, where $255\sqrt{3}$ is the largest color distance between the black (0, 0, 0) and white (255, 255, 255) colors.

Table 3. Comparison of existing essential color palettes (5 colors). G.T.: ground-truth color palette generated by google art, PR: predicted color palette generated by our method, CSR: color similarity rate, CD: average color distance (0~442).

Original Image	GT	PR	Methods	CSR	CD	Original Image	GT	PR	Methods	CSR	CD
Madonna of The Yarnwinder			K-means	0.963	16.507	Canigiani Family			K-means	0.968	14.270
			Fuzzy C-means	0.928	31.734				Fuzzy C-means	0.919	35.778
			K-medoids	0.932	30.638				K-medoids	0.920	35.513
The Night Watch			K-means	0.958	18.625	Mona Lisa			K-means	0.951	21.734
			Fuzzy C-means	0.944	24.532				Fuzzy C-means	0.953	20.634
			K-medoids	0.955	20.032				K-medoids	0.947	23.261
Femme assise dans un fauteuil			K-means	0.956	19.395	Tobit and Anna			K-means	0.959	18.339
			Fuzzy C-means	0.920	35.439				Fuzzy C-means	0.973	11.865
			K-medoids	0.915	37.453				K-medoids	0.960	17.616

shows the results of the tested algorithms for six renowned paintings. Based on these results, we conclude that K-means generally achieves the smallest color distance and highest CSR. Fuzzy C-means achieves the best CSR for “Mona Lisa” and “Tobit and Anna with the Kid”. However, the difference between K-means and Fuzzy C-means was less than 0.02 in CSR. Therefore, we consider using K-means as the clustering algorithm for our analysis.

Now, we use K-means algorithm to cluster/generate $k$ representative colors for each target color palette. The K-Means algorithm will cluster the color samples into $k$ classes, and the center of each class will be the representative color. After running the K-means algorithm, one list of k representative colors will be generated. The K-means algorithm calculates each representative color palette on each generated image. We chose $k=16$, based on the experiment results in [47]. The target color palette for the High Renaissance was generated from 25 most famous paintings. The artist’s color preference palette was generated from a group of famous paintings.

For example, Da Vinci’s color preference palette was generated from nine of his most famous paintings, including “Mona Lisa”, “Madonna and the Child”, and “Lady with an Ermine”. Color symbolism and color pigment palettes were generated based on Table 1 and Table 2. Figure 3 and Figure 4 show the color palettes and color percentages calculated from each transformed image by K-means clustering. After generating the target color palettes, comparisons between the predicted color palette and target can be made to make decisions regarding the best parameter $\alpha$ for the 2D QDFT $\alpha$-rooting.

By applying α-rooting 2D QDFT with different α values from those of the original image, the generated images correlated with α values will be the potential predictions of the original color image. Several representative color palettes will be generated for each generated image by the color segmentation method. To choose the best α, these representative color palettes will be compared with the target color palettes. Comparisons between representative and target color palettes will be compared to choose the best α for the original image. The comparison method we used was for calculating the modified color distance value (MCD) [15] and CIEDE 2000 [70] between two palettes. The MCD is defined as

$$MCD={\displaystyle \frac{1}{k}}\sum _{c=1}^k{\displaystyle \frac{1}{argarg \{Colordis(I_c,colorlist)\}}},$$

(9)

where $I_c$ is the representative color palette, $colorlist$ is the target color palettes, and $k$ is the number of colors in the color palette. $Colordis(.)$ is the same function explained in Equation (8). This function outputs a vector of Euclidean distance values between color $I_c$ and all the colors in $colorlist$. The vector has the same length as $colorlist$.

The color difference of CIEDE 2000 is another color distance calculation algorithm for the CIELAB color space, which covers the entire range of human color perception, or gamut [71]. The value of CIEDE color distance describes how different two color sets are. To estimate the similarity of two color palettes, we use the concept of combined color distance (CCD), which was calculated by the following equation: $CCD=MC{D}_n+{(1/CIEDE)}_n$, where ${MCD}_n$ is the normalized MCD value and $(1/CIEDE{)}_n$ is the normalized inversion value of CIEDE. Figure 3 shows the four target color palettes and illustrates how to calculate the CCD between the transformed images and target color palettes.

The analysis of images of paintings by well-known artists shows that many artists had unique ratios in colors. Such ratios include, for instance, the well-known golden ratio (GR) and aesthetic ratio [72,73]. The GR is a rule of the proportionality of the whole and the parts that constitute the whole, which can be written as $(a+b)/b=b/a=\Psi =1.6180339\dots$. We recall that three numbers $x,y,z$ are in gold proportion if one of the permutations of colors $\left(x,y,z\right)$, $(x,z,y)$, $(y,x,z)$, $(y,z,x)$, $(z,y,x)$, or $(z,x,y)$ equals $a\times (\Psi ,\sqrt{\Psi }, 1)$, where $a$ is a positive number, and $\Phi =\sqrt{\Psi }=1.2720$. If the numbers $x,y,$ and $z$ in the gold proportion represent the primary color components R, G, and B, respectively, of the image at pixel $(n,m)$, then the color (R,G,B) is called the golden color at this pixel [73]. It is assumed that each recognized artist has their own color ratio, which we denote by $\Phi$, not necessarily being the number 1.2720 of the GR. This ratio can be estimated by the average of colors when analyzing several dozens of paintings by the artist. For instance, our research shows that, for Leonardo Da Vinci, $\Phi =1.46$; for Pablo Picasso, $\Phi =1.49;$ for Vincient van Gogh, $\Phi =1.38$; for Rafaello Sanzio, $\Phi =1.61$; and for Rembrandt van Rijn, $\Phi =1.65$. Then, the question arises of how the images of paintings look after the correction of color components at all pixels. All image colors will be in the Golden proportion or $\Psi$-proportion. For the given $\Psi$-proportion, the method of color correction, which is called color modeling via color ratio (CMCR), is described in [74].

6. Results of Image Processing

To illustrate the proposed method of processing color images, Figure 4a shows the color image of 526 × 744 pixels of Leonardo da Vinci’s painting “Portrait of Cecilia Gallerani (Lady with an Ermine)”. The image in part (b) is the same image after color ratio correction by the CMCR. The image in part (c) is the 2D QDFT 0.94-rooted image, and the image in part (d) is the CMCR-based processed image. We can see a 16-color palette with a percentage of each color generated by the K-means algorithm.

Figure 5 shows an image of Leonardo da Vinci’s painting “Madonna of the Yarnwinder”. We can see the different color palettes of the processed images. In part (e), we present a symbolism color palette, which is based on the content of the drawing and color pixel distribution in the RGB coordinate system. From the clustered and original distribution of pixels, we can see that the color ratio of this painting (the plots “R-G” and “G-B” in part (e)) are concentrated in a certain range, which also supports the results that we received from our color ratio research.

Figure 6, Figure 7, Figure 8, Figure 9, Figure 10, Figure 11, Figure 12, Figure 13, Figure 14 and Figure 15 show the original image and processed images with the alpha-rooting by the 2-D QDFT when using different values of alpha $\alpha$. Our color clustering algorithm generated the color palette under each image. In Figure 6, we consider the color image of 586 × 802 pixels of one of Rembrandt’s paintings and nine images of α-rooting, for $\alpha =0.80:0.96$ with step 0.02. The mean of the ratios $\psi$ of the image $f_{n,m}$ is calculated by

$$CR=CR\left(f\right)=Mean\left(\psi \right)={\displaystyle \frac{1}{NM}}\sum _{n=0}^{N-1}\sum _{m=0}^{M-1}{\psi }_{n,m},$$

(10)

where ${\psi }_{n,m}$ is the color ratio at pixel $(n,m)$.

We also demonstrate the symbolism of the color palette for each image in the associated figures. The table after each figure lists the data of the color ratio and the measurements for the processed image. For the image of Rembrandt’s painting “Tobit and Anna with the Kid”, Figure 7 shows the graph of the calculation of the color ratio after the 2D QDFT alpha-rooting. The color ratio as the function of alpha, $CR\left(\alpha \right)$, is shown in the interval [0,7,1] with step 0.01. The maximum color ratio 2.0205 is a point ${\alpha }_0=0.89$. Figure 8 shows the original image in part (a), and the 0.89-rooting in part (b). The average color ratio of this image is CR = 2.0205, and the image, after reducing the color ratio to a value of 2.0205, is shown in part (c). For comparison, the original image has the color ratio CR = 2.4387. After reducing the color ratio to this value, the image is shown in part (d). Both images have a good quality and color palette. From

Table 4. Data for the images in Figure 6.

α	0.80	0.82	0.84	0.86	0.88	0.90	0.92	0.94	0.96	0.98
CR	1.83	1.89	1.94	1.98	2.02	2.01	1.96	1.86	1.75	1.65
M1	1.14	1.14	1.36	1.15	1.30	1.31	1.87	1.18	1.04	1.04
M2	1.07	1.10	1.30	1.14	1.24	1.27	1.64	1.16	1.06	1.05
M3	1.62	2.61	2.97	5.59	3.17	3.52	3.39	3.37	3.54	5.05
M	1.25	1.49	1.74	1.94	1.73	1.80	2.19	1.66	1.57	1.77
EMEC	2.66	3.14	3.77	4.69	6.36	8.18	11.60	15.72	19.57	23.67
EMEC2	5.47	6.29	7.33	8.83	11.12	14.76	19.36	24.87	29.57	33.43

, one can note the quality of the 0.92-rooting image, for which the measurements CR and M1 had high values.

EMEC2 is the EMEC value of the image after modifying the color ratio to the mean ratio, given in Table 4.

Figure 9 shows the original image of the “Madonna Litta” in Leonardo Da Vinci’s Painting and the alpha-rooting 2D QDFT, for $\alpha =$ 0.80, 0.82, …, 0.92, 0.94, and 0.96. From

Table 5. Data for the images in Figure 9.

α	0.80	0.82	0.84	0.86	0.88	0.90	0.92	0.94	0.96	0.98
CR	1.96	2.02	2.05	2.04	2.02	1.97	1.91	1.84	1.76	1.64
M1	1.48	1.84	1.07	1.07	1.16	1.20	1.19	1.10	1.17	1.06
M2	1.39	1.65	1.07	1.08	1.15	1.18	1.18	1.10	1.14	1.04
M3	2.09	5.09	4.31	3.57	4.40	5.25	4.74	5.57	7.66	6.54
M	1.63	2.49	1.70	1.60	1.80	1.96	1.88	1.89	2.17	1.93
EMEC	7.28	7.68	8.28	9.32	10.41	12.17	14.62	17.45	21.08	23.87
EMEC2	7.27	8.35	9.26	10.17	11.45	13.99	17.08	20.62	23.74	27.17

, one can notice an increase in image quality when the alpha value is changed after 0.90. Figure 10 shows the color image of the “Virgin and Child with St. Anne” in Leonardo Da Vinci’s Painting and the alpha-rooting for nine values $\alpha =$ 0.80, 0.82, …, 0.96. The 0.90- and 0.92-rooting showed high-quality images; all colors and details in these images can be seen in a beautiful color palette. In

Table 6. Data for the images in Figure 10.

α	0.80	0.82	0.84	0.86	0.88	0.90	0.92	0.94	0.96	0.98
CR	1.76	1.77	1.77	1.76	1.77	1.78	1.75	1.65	1.55	1.49
M1	1.19	1.37	1.07	1.55	1.04	2.97	2.29	3.64	1.92	1.08
M2	1.04	1.15	0.99	1.24	0.97	1.99	1.62	2.33	1.41	0.97
M3	1.90	2.46	2.76	2.98	3.40	19.33	5.26	4.26	4.02	4.51
M	1.33	1.57	1.43	1.79	1.51	4.85	2.69	3.31	2.22	1.67
EMEC	5.26	5.89	6.61	7.75	8.95	10.47	12.62	15.52	18.48	22.23
EMEC2	8.07	9.30	11.32	13.70	16.29	19.35	22.93	26.16	28.06	26.74

, the measures M and CR had high values for 0.90-rooting. Figure 11 shows the color image of the “The Vision of a Knight” in Raphael’s painting and the alpha-rooting by the 2D QDFT, for the same values of alpha $\alpha =$ 0.80, …, 0.96. One can notice that the color palette changed slowly with an increase in alpha values.

Table 6 displays the generated alpha rooting method-based class of images from the paintings in Figure 10 using the alpha (from 0.80 to 0.98) in seven features and average color ratio (CR) distance-related measurements.

Table 7. Data of seven feature measurements of α-rooting method-based class images for the painting “The Vision of a Knight”.

α	0.80	0.82	0.84	0.86	0.88	0.90	0.92	0.94	0.96
CR	2.26	2.34	2.39	2.49	2.53	2.53	2.43	2.23	1.92
M1	1.48	1.58	4.73	1.69	1.79	1.93	1.85	1.89	1.91
M2	1.47	1.57	4.77	1.69	1.81	1.98	1.91	1.99	2.02
M3	2.98	2.11	5.33	2.49	3.04	7.17	3.20	5.60	6.53
M	1.87	1.73	4.94	1.92	2.14	3.02	2.24	2.76	2.93
EMEC	7.25	7.84	8.13	8.56	9.10	9.78	11.19	12.69	15.52
EMEC2	8.09	9.03	10.30	11.13	11.97	13.69	14.69	17.15	22.16

displays the generated (from Figure 11 paintings) alpha rooting method-based class images, their seven feature measurements, and color ratio distance-related average results.

Figure 11, Figure 12, Figure 13, Figure 14 and Figure 15 show images of four of Raphael’s paintings processed by the 2D QDFT alpha-rooting, for α = 0.90, 0.92, 0.94, 0.96, and 0.98.

Table 8. Data of seven feature measurements of α-rooting method-based class images for the painting “Canigiani Holy Family”.

α	0.80	0.82	0.84	0.86	0.88	0.90	0.92	0.94	0.96	0.98
CR	1.91	1.86	1.82	1.83	1.90	2.03	2.23	2.27	1.99	1.64
M1	1.39	6.55	3.29	1.53	1.54	1.70	1.51	1.62	1.41	1.78
M2	1.28	5.18	2.56	1.26	1.27	1.40	1.29	1.40	1.27	1.54
M3	1.72	2.12	2.25	2.82	4.72	3.09	3.28	3.66	4.38	4.16
M	1.41	4.16	2.67	1.75	2.10	1.95	1.86	2.02	1.98	2.25
EMEC	8.12	8.20	8.52	8.75	9.31	10.07	11.49	13.79	17.24	21.09
EMEC2	8.45	9.60	10.56	11.55	12.41	13.32	14.71	17.21	23.39	27.74

,

Table 9. Data of feature measurements of α-rooting method-based class images for the painting “Madonna and Child with Book”.

α	0.80	0.82	0.84	0.86	0.88	0.90	0.92	0.94	0.96	0.98
CR	1.96	2.09	2.20	2.24	2.28	2.30	2.20	2.03	1.81	1.69
M1	1.73	13.19	2.13	1.46	1.82	1.98	1.77	1.65	1.51	1.45
M2	1.44	8.85	1.69	1.30	1.52	1.58	1.45	1.38	1.27	1.21
M3	1.49	2.39	2.30	2.90	3,29	3.86	7.14	4.08	4.84	4.90
M	1.55	6.53	2.02	1.77	2.09	2.30	2.64	2.10	2.10	2.05
EMEC	3.29	3.49	3.58	3.87	4.55	5.49	7.41	10.79	14.57	17.48
EMEC2	6.35	6.85	7.49	8.00	9.16	10.73	13.61	19.46	24.06	27.31

,

Table 10. Data of feature measurements of α-rooting images for the painting “Colonna Madonna”.

α	0.80	0.82	0.84	0.86	0.88	0.90	0.92	0.94	0.96	0.98
CR	1.90	1.96	2.07	2.18	2.30	2.40	2.44	2.32	2.01	1.68
M1	1.58	2.33	2.13	1.98	2.26	2.46	2.23	2.45	2.53	2.73
M2	1.48	1.96	1.66	1.50	1.61	1.64	1.48	1.53	1.50	1.55
M3	1.60	4.61	1.77	5.79	5.23	2.75	5.54	4.27	5.10	8.12
M	1.55	2.76	1.84	2.58	2.67	2.23	2.64	2.52	2.69	3.25
EMEC	11.23	11.49	11.91	12.33	12.79	13.26	13.89	14.65	16.39	18.80
EMEC2	10.91	12.26	12.52	12.76	12.82	13.23	13.54	14.83	16.86	19.21

and

Table 11. Data of feature measurements of α-rooting method-based class images for the painting “Catherine of Alexandria”.

α	0.80	0.82	0.84	0.86	0.88	0.90	0.92	0.94	0.96	0.98
CR	2.15	2.21	2.26	2.29	2.28	2.21	2.05	1.96	1.68	1.57
M1	1.36	1.46	1.20	1.47	1.28	3.27	1.15	1.65	1.29	1.14
M2	1.34	1.42	1.27	1.39	1.21	2.85	1.12	1.56	1.27	1.16
M3	7.65	2.40	6.32	3.88	4.73	3.74	4.52	4.96	5.32	6.41
M	2.41	1.71	2.19	1.99	1.95	3.27	1.80	2.34	2.06	2.03
EMEC	8.74	8.38	8.34	8.53	8.81	9.24	10.96	12.98	15.79	20.15
EMEC2	10.29	11.59	13.15	14.53	15.50	16.87	19.40	22.92	26.00	26.13

display the measurement data of the four Raphael paintings’ color ratios and their features, including CR, M1, M2, M3, M, EMEC, and EMEC2. Based on these measurements, we may observe that 0.90- and 0.92-rooted images (Figure 12) had high values of CR and M primary measures. Visually, we can also note that these images are of high quality and are probably the best candidates to be selected for color prediction for this painting.

7. Results of Image Color Prediction

This section describes the results of historical color prediction of the artists’ paintings. The difference in color ratio between the generated image and the average value of the artist’s paintings was combined with the previous results from the CCD values to make the final predicted color image decision. The measurements for each image were the color distance value to the High Renaissance color palette (CCD1), Leonardo Da Vinci’s color palette (CCD2), High Renaissance material color palette (CCD3), painting’s symbolism color palette (CCD4), and color ratio difference (CRD). The color ratio difference is the L1 loss between color ratio values and the artist’s color ratio.

Now, we consider the results of the developed method tested on two paintings, “Madonna Litta” by Da Vinci. Figure 16 shows the result of the image processing using the proposed method. The color image measurements for different alpha-rooting of this image are given in

Table 12. Measurements of processed images.

α	CCD1	CCD2	CCD3	CCD4	CRD	M1	M2	M3	M	EMEC	EMEC2
0.80	0.2147	0.225	0.2226	0.2165	0.5	1.48	1.39	2.09	1.63	7.28	7.27
0.82	0.2472	0.2182	0.2253	0.2249	0.56	1.84	1.65	5.09	2.49	7.68	8.35
0.84	0.2221	0.2392	0.2199	0.2211	0.59	1.07	1.07	4.31	1.7	8.28	9.26
0.86	0.2392	0.2437	0.2197	0.2207	0.58	1.07	1.08	3.57	1.6	9.32	10.17
0.88	0.2168	0.2152	0.2176	0.2226	0.56	1.16	1.15	4.4	1.8	10.41	11.45
0.90	0.2081	0.2135	0.2229	0.2247	0.51	1.2	1.18	5.25	1.96	12.17	13.99
0.92	0.2192	0.2232	0.2191	0.2181	0.45	1.19	1.18	4.74	1.88	14.62	17.08
0.94	0.2391	0.2138	0.2269	0.2255	0.38	1.1	1.1	5.57	1.89	17.45	20.62
0.96	0.1936	0.2082	0.2258	0.226	0.3	1.17	1.14	7.66	2.17	21.08	23.74

.

Table 13. Top component values after principal component analysis.

α	Top1	Top2	Top3	Top4	Top5	Top6	Top7	Top8
0.80	−2.3859	−1.8312	−3.7225	−2.6850	−2.1274	−2.0834	−2.0460	−2.0335
0.82	−0.3212	4.1938	4.7600	4.5406	4.2164	4.3240	4.3326	4.3392
0.84	−1.9399	−3.1562	−2.4624	−2.9127	−2.7433	−2.0728	−2.2016	−2.1016
0.86	−2.3892	−3.6643	−2.4188	−2.0191	−2.1756	−1.9049	−1.7412	−1.8276
0.88	−0.7759	−1.4759	−1.6936	−2.9685	−2.9446	−3.8138	−3.6845	−3.6347
0.90	0.8552	0.8143	0.6425	−0.2265	0.4831	0.4284	0.2712	0.1624
0.92	−0.0110	−1.0206	−1.7941	−1.5270	−2.7993	−2.9996	−3.1293	−3.1523
0.94	2.4122	2.1147	3.2998	4.6102	4.9642	4.3527	4.2984	4.3335
0.96	4.5598	4.0253	3.389	3.1880	3.1265	3.7694	3.9003	3.9147

shows that the best $\alpha$ for Da Vinci’s “Madonna Litta” was 0.94, which achieved the highest voting score and was ranked the highest in CCD3. It also exceeded 9% of the second score achieved by 0.82. In Table 13, “Top-4”, “Top-5”, and “Top-6” features give the same best $\alpha =0.94$. Although the “Top-7” and “Top-8” features suggest $\alpha =0.82$, it only exceeds the second-best value 0.94 by 0.1%. Based on these observations, 0.94 was considered to be the best $\alpha$ for candidate images of “Madonna Litta”.

The final score of each generated image was calculated by a ranked voting system and principal component analysis (PCA). The ranked voting system and PCA worked independently to predict the α value (see Table 13). The highest-ranking α was assigned the highest-ranking score (RS) in a ranked voting system. The highest score was the total number of different α values. When the ranking decreased by 1, the score decreased by 1. The final score will be the sum of each α value’s ranking scores. The highest voting score achieved by the specific α will be the value we choose for generating the color image predicted to have the closest color palettes to the original color palettes. In Table 13, a list of the essential features is given. The number in the names of the columns (‘Top1’, …, ‘Top8’) means the number of most important features being combined. We sum up the essential feature values after PCA because the values have already been weighted.

Table 14. Mean voting score and CCD value estimation of the “Madonna Litta” painting image.

α	CCD1		CCD 2		CCD 3		CCD 4		CRD		Voting Score
	Value	RS	Value	RS	Value	RS	Value	RS	Value	RS
0.80	0.2147	3	0.2250	7	0.2226	5	0.2165	1	0.5	6	22
0.82	0.2472	9	0.2182	5	0.2253	7	0.2249	7	0.56	4	32
0.84	0.2221	6	0.2392	8	0.2199	4	0.2211	4	0.59	2	24
0.86	0.2392	8	0.2437	9	0.2197	3	0.2207	3	0.58	3	26
0.88	0.2168	4	0.2152	4	0.2176	1	0.2226	5	0.56	4	18
0.90	0.2081	2	0.2135	2	0.2229	6	0.2247	6	0.51	5	21
0.92	0.2192	5	0.2232	6	0.2191	2	0.2181	2	0.45	7	22
0.94	0.2391	7	0.2138	3	0.2269	9	0.2255	8	0.38	8	35
0.96	0.1936	1	0.2082	1	0.2258	8	0.2260	9	0.3	9	28

also shows each column’s ranking scores (RC) and the voting scores in the final column. We applied PCA to help us to make decisions based on 11 measurements (M1, M2, M3, M, EMEC, EMEC2, CCD1, CCD2, CCD3, CCD4, and CRD).

To generate the artist’s color ratio value, we collected paintings of different artists. We chose three well-known artists for the final experiment. One hundred and five images of paintings by Rafael, Leonardo Da Vinci, and Rembrandt were selected and processed. The images were taken from Olga’s Gallery at the address http://www.abcgallery.com/ (accessed on 9 September 2017). The data of color ratio and measurements discussed above were calculated for each image. These measurements were also averaged (see

Table 15. Averages of Raphael’s oil paintings in seven feature measurements and color ratio (CR) distance-related average results.

Artist: Raphaello Sanzio
Painting Name	CR	M1	M2	M3	M	EMEC	EMEC2
‘The Vision of a Knight’ Figure 11	1.50	2.35	2.58	4.62	3.03	23.08	35.42
‘Madonna and Child with Book’ Figure 13	1.64	1.51	1.22	4.86	2.08	19.89	28.91
‘Madonna Estergazi’	1.54	2.11	2.43	5.71	3.08	24.36	32.07
‘Canigiani Holy Family’ Figure 12	1.50	1.70	1.46	3.48	2.05	23.97	27.74
‘Colonna Madonna’ Figure 14	1.49	2.97	1.63	7.89	3.37	20.46	20.98
‘Madonna and Child Enthroned with Saints’	1.38	1.12	1.11	4.18	1.73	16.39	38.68
‘St. Catherine of Alexandria’ Figure 15	1.51	1.06	1.09	7.27	2.03	25.14	24.09
‘Crucifixion’	1.41	1.61	1.65	2.29	1.83	16.91	28.21
‘The Niccolini-Cowper Madonna’	1.45	2.20	1.59	4.04	2.42	25.52	21.03
……	…	…	…	…	…	…	…
Average (over 30 paintings)	1.61	1.59	1.65	6.39	2.44	22.31	31.66

,

Table 16. Averages of Da Vinci’s oil paintings in seven feature measurements and color ratio (CR) distance-related average results.

Artist: Leonardo da Vinci
Painting Name	CR	M1	M2	M3	M	EMEC	EMEC2
‘Madonna Litta’ Figure 9 and Figure 13	1.56	1.53	1.36	5.24	2.22	25.72	29.98
‘The Virgin and Child with St. Anne’ Figure 10	1.46	1.60	1.24	4.39	2.06	26.87	24.68
‘Madonna Benois’	1.55	1.71	2.18	5.86	2.80	29.62	32.44
‘Madonna with the Carnation’	1.57	2.45	2.46	4.47	3.00	23.02	20.42
‘St. Hieronymus’	1.61	3.64	3.86	4.90	4.10	26.49	37.01
‘Madonna of the Rocks’	1.48	0.79	0.96	6.78	1.73	24.46	30.95
‘Madonna of the Yarnwinder’ Figure 5	1.48	1.18	1.08	6.43	2.02	22.43	29.86
‘Portrait of Cecilia Gallerani (Lady with an Ermine)’ Figure 4	1.60	4.95	6.17	2.15	4.03	24.20	30.53
……	…	…	…	…	…	…	…
Average (over 25 paintings)	1.46	2.86	2.84	5.84	2.67	24.35	26.35

and

Table 17. Averages of Rembrandt’s oil paintings in 7 feature measurements and color ratio (CR) distance-related average results.

Artist: Rembrandt
Painting Name	CR	M1	M2	M3	M	EMEC	EMEC2
‘Tobit and Anna with a Kid’ Figure 6 and Figure 8	1.58	1.08	1.05	3.68	1.61	27.56	35.74
‘Self-Portrait’	1.67	2.31	2.20	2.31	2.28	21.16	25.08
‘Christ in the Storm on the Lake of Galilee’	1.46	1.58	1.52	1.30	1.46	17.61	21.04
‘Philosopher Reading’	1.44	2.86	3.24	3.57	3.21	29.85	23.25
‘Portrait of a Young Woman with the Fan’	1.42	2.27	2.44	1.80	2.15	20.19	23.07
‘Feast of Belshazzar’	1.49	1.65	1.80	4.19	2.32	21.40	33.48
……	…	…	…	…	…	…	…
Average (over 50 paintings)	1.65	2.71	2.96	3.13	2.70	22.01	26.65

).

Table 18. Predicted original painting colors of six High Renaissance paintings.

Rembrandt’s “Tobit and Anna with a Kid”		Raphael’s “Canigiani Holy Family”
Existing image	Predicted color (α = 0.96)	Existing image	Predicted color (α = 0.98)
Raphael’s “Madonna and Child with Book”		Raphael’s “Colonna Madonna”
Existing image	Predicted color (α = 0.98)	Existing image	Predicted color (α = 0.98)
Raphael’s “St. Catherina of Alexandria”		Da Vinci’s “Virgin and Child with Saint Anne”
Existing image	Predicted color (α = 0.94)	Existing color image	Predicted color (α = 0.90)

shows the results of our work. The proposed method was applied to six different paintings: Rembrandt’s “Tobit and Anna with the Kid”, Raphael’s “Canigiani Holy Family”, “Madonna and Child with Book”, “Colonna Madonna”, “St. Catherina of Alexandria”, and Da Vinci’s “Virgin and Child with Saint Anne”. The best α value for Rembrandt’s “Tobit and Anna with Kid” was 0.96. The best α values for Raphael’s paintings were all 0.98, except for “St. Catherina of Alexandria”, which presented a value of 0.94. The best α value for Da Vinci’s painting was 0.90. The results show that this method could simultaneously predict the original image color. Most of the best α values for predicted color were high, at 0.94~0.98, for which high EMEC and EMEC2 values were achieved.

8. Conclusions

This study presents a novel approach to estimating and reconstructing the original color palettes of Renaissance oil paintings using cognitive computing (CC) and machine-learning techniques. This research successfully applies quaternion-based transformations and alpha-rooting with 2D quaternion discrete Fourier transform (QDFT) to predict historical color schemes. The key results include:

• Color Distance Caculation: The proposed method effectively reconstructs color palettes by analyzing color distances between the modified state of the painting and its expected historical palettes using the K-means machine-learning model, which outperformed fuzzy C-means and K-medoids in most cases;
• Symbolism and Historical Accuracy: This study confirms the strong correlation between color usage in the time and region in which the painting was created and symbolic meanings in Renaissance art, demonstrating that enhanced images aligned with historical and cultural knowledge.
• Color Ratios: Analysis of the relative proportions of colors in the painting to infer likely original hues;
• Enhancement Measurements: Enhancement techniques reveal underlying features and colors that may have faded or degraded over time. The technique is efficient in generating images with highly probable original color palettes;
• Feature Analysis: Through principal component analysis (PCA), 11 key features—including color ratios, enhancement metrics, and color distance measures—were identified as crucial for optimal color restoration.

To conclude, this research contributes to the field of digital art restoration by introducing a systematic, machine-learning-driven framework for estimating the original colors of Renaissance oil paintings. The approach integrates historical knowledge with modern computational techniques, enabling more accurate color predictions that align with the known artistic conventions and material compositions of the period. By examining symbolic color choices, historical data, and the relationships between colors, we have shown the potential of CC and machine-learning tools for understanding and restoring the visual essence of old masterpieces.

By leveraging quaternion arithmetic, this study successfully modeled color transformations in a multi-dimensional space, leading to better preservation of the color balance and symbolic significance inherent in these artworks while also improving image quality. Furthermore, the ranked voting system and PCA-based selection ensure that the reconstructed colors are not only visually pleasing, but also historically plausible.

Our findings suggest that further in-depth exploration in this field is warranted. Although we did not address contrast–brightness balance, we acknowledge that its role in shaping the perceived quality and authenticity of color warrants deeper study in the future. We plan to use more extensive datasets, such as the images available at http://imag.pub.ro/pandora/pandora_download.html (accessed on 2 April 2025) and the jenaesthetics dataset (Amirshahi et al., 2012) [75]. These richer data sources will help us to refine our model’s predictions, enhance its robustness, and provide a broader foundation for understanding and preserving historical artworks’ original color palettes and aesthetic qualities. In addition, we will conduct several comparison experiments with the traditional color restoration methods and the recently developed AI-based approach to highlight the performance of our method and provide a valuable reference for our findings and results.