1. Introduction
All over the world, objects of historic and cultural values are exhibited and preserved in museum collections. To protect objects in collections from degradation they are held and exhibited in controlled environments, where ideally all ambient conditions are adjusted to minimize future damage to the heritage materials. This strategy, focused in preventing future deterioration acting on the potential causes of deterioration, is known as preventive conservation.
However, the perfect environment is impossible to achieve, and sometimes humidity, pollutants or lighting problems can reach a point where the material an exhibited cultural heritage item is made of starts suffering a quick degradation [
1,
2,
3]. Sometimes harmful emissions come from inside. Building materials from furniture, showcases or panels, such as paints, woods or adhesives or cleaners, and even some objects from collections can emit gaseous pollutants. This problem is even more serious if the exhibited artifacts are stored in non-ventilated display [
4,
5].
Thus, even in the best conservation-suited environment, additional needs for surveillance of cultural heritage are likely to be required. In order to lighten human efforts on the task of surveillance, automatic monitoring resources are encouraged and proposed. These may take any mean for quantifying the possible decay, like sensing humidity or light or checking the change in color of some material present when it is undergoing a degradation process [
6].
One of the most evident signs of deterioration of heritage materials are color alterations. Color is probably the most significant property of cultural heritage objects. Not only in painting, but in sculpture and decorative arts color is carefully chosen, selecting materials, finishes and surface treatments. Aging of materials frequently produces color changes that can be related to degradation in different aspects and degrees of intensity. In some cases as in irreversible alterations of pigments and dyes it represents a major concern for conservators. Therefore, the availability of tools to monitor color change is of interest in cultural heritage conservation. At the same time, color change of certain reference materials can been used for indoor environment evaluation as a preventive conservation tool.
Colorimetric measurements are in fact frequently used for detecting degradation in different materials, using colorimeters or spectrophotometers [
7,
8]. The use of color change as an indicative of environmental changes has also been suggested [
9,
10]. Color monitoring is clearly relevant in the conservation field; it does not require complex equipment or specialized personnel, as a colorimeter is easy to operate. However, it is a time-consuming method, and has some application limitations on very irregular or delicate surfaces, as it requires contact.
Based on these premises, our research group has proposed a system for monitoring environmental conditions in museums based on the measurement of color changes by image processing. The objective of this system, besides being a low-cost preventive conservation tool, is to increase social awareness in relation to heritage conservation. For this purpose, the system uses crowdsourced pictures taken by visitors with their cell phones [
11] (
Figure 1). The usage of crowdsourcing techniques is acknowledged as a useful resource, which has been used for long in monitoring of biodiversity, astronomy, and other fields [
12]. More recently has been proposed as a monitorization tool for cultural heritage preservation, using smartphone technologies [
13,
14,
15].
Photographic cameras represent the acquired information analogously to the human visual system [
16,
17]. But, considering fabrication and physical component differences between models, images representing the same scene by disjoint cameras are likely to appear visually different to the same observer [
18]. This is the main concern regarding the crowdsourced information, therefore a homogenisation via color calibration is needed to perform an effective monitoring of the pieces. This requires the presence of a color chart when acquiring the content to be calibrated.
The mentioned approaches use mobile images individually adjusted using X-Rite charts as color reference. However, the X-Rite color chart has been designed provide bright colors for outdoor photography, and some of the tiles have too saturated colors [
13], and its materials have not been tested to be compatible with heritage objects. Our monitoring system pretends a further step on these first approaches, designing and testing an optimized color chart, an accurate calibration method and developing an efficient color change detection system.
The previous stages of this project, the conception of the color chart, the verification of the suitability of the materials for the monitoring system, and an adaptive calibration algorithm for image processing have already been presented [
11,
19]. The color chart is mounted on an inert substrate together with several metal coupons used as sensitive materials.
This reference color chart has been specifically designed to cover all the range of the sRGB color space, which is shared by most phone cameras in usage. It is placed in the display case or exhibition space to be monitored in this project. Alongside the chart, metal coupons are placed. Possible inadequate environmental conditions that may affect the heritage artifacts will induce corrosion in the coupons. This corrosion process might take the form of a tarnishing, more or less uniform colored corrosion layers, or colored spots on the surface of the metal. These colorimetric cues are the relevant information to be extracted from the crowdsourced pictures, as warnings of unsuitable environmental conditions for the conservation of heritage assets.
The calibration process is one of the most critical parts of the monitoring system. Assuming most crowdsourced pictures are taken with different cameras, a quality calibration for every image is needed for homogenizing color information. According to this, the color chart needs to be consistent and stable enough to be used as a reliable reference for the color calibration algorithm.
So, the materials employed to build the display and chart and the coupons need to be carefully selected so they do not undergo possible strong chromatic deviations from the initial state; and they should not produce alterations in the metal coupons and, more important, in the heritage artifacts [
11]. It is utmost important that the reference color chart remains unaffected so the quality of the picture calibration does not lower when deployed for a prolonged time. In addition, if by any reason the color references change, it is imperative to check how this change can affect the outcomes of the calibration process.
Thus, the aim of this paper is to evaluate robustness of the crowdsourced color evaluation system against environmental degradation of the reference display. The criterion adopted is mathematical robustness to discoloration. Since the main agent causing color modification in organic materials (including boards, papers and inks) is light, variations in the color chart on different materials due to exposure to UV radiation have been evaluated. For this purpose, periodic measurements of the exposed references have been carried out.
These measures are processed in different mathematical ways to evaluate the robustness of environmentally-induced color degradation in them. From this information, it is evaluated how possible light induced color variations may affect the accuracy of the calibration method.
Finally, conclusions are drawn and the selection of the most suitable material to build the crowdsourcing-based monitoring display are proposed.
2. Materials and Methods
2.1. Color Chart Design
The color chart has been conceived so the calibration to perform does not incur in unacceptable errors. It consists in 64 color patches of dimensions 1 cm × 1 cm arranged on 11 columns and 9 rows. There is a free space of 7 columns × 5 rows in the centre, where the metal coupons (or other references) can be placed (
Figure 2).
The calibration process is performed by calculating a transfer function that projects the R, G and B information of each channel of the taken pictures into a reference space, where their corresponding coordinates take values that must be present in the calibrated image. Thus, all pictures after this operation should depict consistent color information [
19], regardless of their camera of origin.
With this conception in mind, the depicted colors present regularly sparse grey tones range in sRGB acting as valid as luminance values too, including maximum black and white, and the primaries red, green, blue, cyan, magenta and yellow. The remaining colors are combinations of equal shares of each RGB axis, and are furthermore designed to exist in the perceptual spaces as sRGB and inside CIELAB’s boundary [
19]. This avoids transformation errors when colors exist regardless of the color space to consider in further application. It is also assured that the possible range of values is covered equally, and inferring the projection of colors not present within the set of anchor colors not much accuracy is lost.
It is imperative that the manufactured color charts to be used in museum showcases are stable for a prolonged usage whilst maintaining the color characteristics mentioned above. This way they are expected to withstand the adverse ambient conditions that this project, in its global scope, is intended to monitor.
2.2. Color Chart Materials
Color charts have been printed by a graphic arts printer using selected materials for the support and UV resistant inks. Four candidate materials for the support have been evaluated: acid-free laminated paper, acid-free non-laminated paper, Forex Smart and Glasspack. The choice of these papers, within the options available on the market, has been based on the British Museum’s Oddy Test database, to ensure their safety [
20]. In addition, their light weight, firmness, low cost and ease of cutting and handling making it easy to build an exhibition structure with any of them, are suitable for the intended purpose.
The ink set tested for the color chart was LED-UV Xtreme Pro, by Deutsche Druckfarben. These inks are used for printing advertising posters intended for outdoor display, and they are known to resist solar radiation. Their lightfastness is rated 5 (yellow and magenta) and 8 (cyan and black) to blue wool standard according to the manufacturer [
11].
The thickness of the ink has been measured using a Helios Digitrix II micrometer, with 1 µm resolution, by difference of the thickness of the blank paper and the printed patches. The thickness strongly depends on the color, ranging from 2–3 µm in the ligthest patches, up to 20–25 µm in the more saturated colors and 40 µm in the black patch.
2.3. Color Chart Evaluation
Therefore, a display structure is made from each one of the four support materials and the set of inks. Each test display has been exposed to a constant irradiation of UV light at 50
C, using fluorescent UVA-340 lamps, following the ISO4892-3 standard in a Q-Lab QUV chamber [
21].
Since the color charts are intended to be exposed in museum cases for a long time, the UV exposure time has been calculated to be equivalent to several museum years. Q-Lab QUV chamber lamps reproduce daylight conditions, which are equivalent to a 30,000 lux illuminance. As museum illuminance for non-sensitive materials is set at 300 lux, the illuminance in the chamber is (maximum) 100 times that in the museum. Thus, we can estimate that in the worst scenario (a non UV-filtered illumination), 144 h chamber exposure (six 24 h periods) can be approximated to 4 years of museum (144 h × 100 = 14,400 h of exposure /10 h per day = 1440 days = 3.9 years.
Measures of the color coordinates have been taken for hours 0 (without degradation before exposure), 2,4, 6, 24, 48, 72, 96, 120 and 144 using a Konica Minolta spectrophotometer CM-700-d, with 6 mm diameter mask, D65 as illuminant and at 10 degrees to the observer, in color space CIELAB. Every measure taken consists in an automatic average of three measurements on areas of 1 cm. Its accuracy specifications reveal a repeatability tolerance of a standard deviation between 0 and of the spectral reflectance, and within of the detected colorimetric value. As a representative set of colors of the printed charts to be evaluated, the patches corresponding to white, black, red, green, blue, cyan, magenta and yellow are taken. It is to be noted that the color patch corresponding to the white tone does not have any ink printed upon it, so it consists in the naked material. Therefore, the effect of the test on the substrate material can be tested when analysing the white patch. Consequently, the other color tones are considered indicators of the effect of the degrading ambiance on a combination of substrate and inks. Thus, the effect of the degrading ambiance on the naked substrate and on different mixtures of different amounts of inks can be observed, and its possible effects on the calibration process inferred.
The results are a list of progressive CIELAB coordinates along time for given color patches in every material combination. The CIE1976 L*a*b* color space, or CIELAB, is usually employed in heritage science matters as accurate color specification [
22], while being uniform, device-independent and suitable for calculations such as calibrations [
19].
Several means to describe the degradation have been considered. A graphical evolution along time of the CIELAB coordinates for the selected color patches has been observed (
Figure 3). In addition, a good descriptor of degradation is a difference metric. The
CIEDE2000 difference formula has been selected for this purpose, since it is designed to reflect CIELAB color differences according to human perception [
23]. Equation (
1) reflects its nature [
24]. It defines a metric similar to a mathematical distance in the CIELAB color space, employing differences related to the physical lightness, chroma and hue (
and
H). The
S functions and parametric factors
k define weights to the former quantities, and in order to improve its effectiveness for the blue range it employs an interactive term between
C and
H (
).
The difference formula has been employed to show a progressive degradation. Each CIELAB measure from each one of the time checkpoints is compared to the original zero-hour measurement, its difference calculated and its progress shown.
The analysis for the white, ink-free patch is particularly interesting for what it may reveal about the nature of degradation on the bare support material, without the ink acting as a cover. Special consideration has been given to measurements of CIELAB values and differences concerning these patches. Their effect is measured by performing calibration processes using these degraded white values as reference, and the calibration error as CIEDE2000 difference is then evaluated for all materials.
The next section will address the results, explaining the material to be chosen and which ones could be discarded or used as a temporary substitute. The choice ensures that the display will guarantee a stable calibration with consistent results, so the monitoring via crowdsourced images can be enabled for an extended period in a museum.
Aside, in order to demonstrate the capabilities of the conceived charts, an experiment that involves the calibration scheme outlined in [
11,
13] over pictures taken with commercial brands of phones has been performed. With the material chosen with the best results (explained in the following sections), a chart has been printed and photographed. Pictures have been taken with the following models: Xiaomi Mi10 lite 5G, Poco X3 NFC, Poco F3, Huawei P30, iPhone SE and Microsoft Surface Pro (
Figure 4). This way, the crowdsourcing disparity is achieved. After having collected the pictures, the calibration process has been ran over them and the robustness evaluated. The calibrated color patches of each of the pictures have been compared with the reference color values in sRGB using the CIEDE2000 difference formula.
In addition to this, the mean digital CIELAB coordinates for every patch in each calibrated picture have been compared to the original spectrophotometer measurements so the accuracy of the calibrated values with respect to the original data can be assessed.
3. Results and Discussion
The evolution of CIELAB coordinates of each of the considered color patches are listed in the
Table A1,
Table A2,
Table A3 and
Table A4. It should be noted that the white coordinate, corresponding to the ink-free spot of the color chart, undergoes a stronger variation in comparison to the other colors under study in all cases. This degradation, seen as a progressive shift of values, is especially remarkable in the b* coordinate. This implies a process of yellowing, extremely pronounced for the Glasspack (
Table A4). The inks act as a “shield” for the substrate material, given their greater stability and less intense degradation. The most stable of the materials considering its low variation is the acid-free laminated paper (
Table A1).
Based on those tables, the CIEDE2000 differences for each one of the four materials under study are shown graphically (
Figure 3). In general, the color patches with inks printed upon them show a relatively slow degrading, with a barely steep slate, staying in most cases under the
CIEDE2000 < 1.5 threshold, implying a minimal perceivable color difference, only noticeable by trained eyes [
25,
26,
27]. This indicates good stability and robustness in the considered exposition period.
Considering that materials such as papers usually are not as resistant towards chromatic deviation as other, more expensive materials like ceramics, it is promising to observe how small the induced error can be for the considered application.
It is interesting to note that on half of the tested materials, the color patches with only one ink printed (black, yellow, cyan or magenta) show a subtly stronger degradation than the other colors, which need more than one ink to be printed, following the additive YCMK system. This reinforces the idea of the nature of the inks as protective covers of the substrate material.
Therefore, the white ink-free patch shows a stronger degradation compared to the rest in the described graphs. Furthermore, its degradation can exceed a CIEDE2000 difference of value 3 for the acid-free paper and the Glasspack. It can be stated that the degradation of the ink-free spots is especially necessary to consider for the whole experiment (
Figure 5), since an excessive difference with the zero-hour original value may lead to an erroneous color calibration process. This fact could compromise the whole monitoring mechanism.
Therefore, a deeper observation of the white spot for all the considered materials has been done. Several “dummy” color charts have been generated. The colored spots have been conserved, but the value corresponding to white patch has been substituted in each one of the materials with the RGB color coordinates corresponding to the CIELAB of every taken measure for the given time checkpoints. This is done for all four substrates. Then, the calibration process as described in [
19] has been run on each of them with respect to the corresponding zero-hour color chart as the reference. Thus, the drifting effect of the decoloring towards the yellow of the white spot on the whole calibration process can be measured.
The subjects of calibration have been seven color stripes, in order to assess its effect on a variety of surfaces. Four of them represent silver and copper extracted textures from other photographs, since the indicator material coupons for the monitoring system are will be made of both metals. The three others represent a neutral grey hue (RGB = 128, 128, 128), a uniform copper color (RGB = 199, 140, 98) and a bright yellow (RGB = 255, 230, 128). They are used to see the effects of calibration with an erroneous white reference on different zones of the color space: the center, the region of copper tones, considering silver will mainly engulf luminance-predominant hues, and the yellow stripe represents the yellowing shown by aged whites (
Figure 6). After each process, the stripes are extracted and their mean CIELAB coordinates and CIEDE2000 difference calculated. Results are presented in
Table 1,
Table 2,
Table 3 and
Table 4.
For the time interval considered the calibrations of the modified color charts present a moderate difference, all values remaining in a close range even between the least and most deteriorated state, in every case smaller than the acceptability threshold of CIEDE2000. The most stable material is the acid-free paper (
Table 1), that undergoes practically no deviation, whilst the others still fluctuate, even if slightly. Nevertheless, the acid-free non-laminated paper presents the best balance on lower mean error (lower than the acid-free paper) with acceptable variation (
Table 2).
However, it is interesting to note that the differences shown in the
Table 1,
Table 2,
Table 3 and
Table 4 are not continuous, but they fluctuate at each time step. A reason for this can be understood when the “white” values for every paper are represented in their 3D color space (
Figure A1,
Figure A2,
Figure A3,
Figure A4 and
Figure A5) and their evolution along time observed. The white tones, represented by dots in space, do not follow a linear evolution over time, but evidently fluctuate in space when under the chromatic deviation process, following an irregular evolution from the initial white tone in different directions during each time step. This is represented using arrows, as seen in the figures. For every paper, the arrows in the a*L*, b*L* and a*b* planes (
Figure A1,
Figure A2,
Figure A3 and
Figure A4), and in the whole L*a*b* space (
Figure A5) are represented. Depending on the position in space of any affected color during the calibration process, its distance from its projected reference value will be of different magnitude and orientation depending on its initial position, closer or farther away from the state of the white value in the chart in the moment the calibration is performed. This is extensible to any other reference color in the chart. This irregular variability of the reference color depending on degradation and time explains the fluctuating results when calibrating.
Aside of that, it is also remarkable to see how, even when performing a calibration with deteriorated color charts, the results presented here are of a higher quality than other similarly oriented cultural heritage conservation works with ideal charts [
14]. Whilst state-of art accepted CIEDE2000 differences exceed the minimal perceivable difference threshold with an ideal chart, our physically printed charts on every material among the tested ones under a chromatic deviation equivalent to 4 years still depict values inferior to the threshold in all the cases, and most of them with a CIEDE2000 value between 0 and 1 after application of the calibration process.
Overall, concluding from the color difference values, it can be observed that the four materials under consideration (laminated acid-free paper, acid-free paper, Forex Smart and Glasspack) are robust enough to resist the effects of chromatic degradation over a long period without incurring in errors noticeable to the human eye in the calibration process, in spite of the material degradation that still occurs, even if slight, when observing the metrics. If the selection of the material would only depend on the mathematical chromatic stability criterion, the most suitable and balanced one would be the acid-free laminated paper. Even if the differences are subtle from a numerical perspective, Glasspack would be discarded, and acid-free paper and Forex Smart would be seen as the second and third choices, respectively.
Out of the performed tests, when it comes to deploying the structures. the selected materials should fulfil some basic requirements in order to be used in a museum environment. The most important one is that the material should be innocuous to the heritage assets in its environment. Many materials are known to release different pollutants that can cause degradation of sensitive materials [
5,
28]. Additionally, since the chart is to be used in combination with metallic coupons (acting as sensitive materials for the environment), the materials of the chart should also be interred to the metal coupons. In order to check these aspects, the four candidate materials have been subject to the Oddy Test. Based on these results, which can be found in detail in a related work by the authors [
11], acid-free laminated paper has been chosen, considering the balance between color stability and emission of harmful pollutants.
Thus, considering the facts above, the most suitable material for building the calibration display has been the acid-free laminated paper. Its chromatic stability is the best ones among all the tests, whilst fulfilling the real-life safety requirement. The calibration process for the crowdsourced images can be therefore considered as safe to perform for a prolonged time.
Having decided the definitive materials for building the charts, one has been printed to evaluate its suitability to endure the disparity effects on crowdsourced imaging. Six pictures with different phones, as outlined in the previous section, have been taken in the lab and the CIEDE2000 differences for each color patch extracted. Also, the standard deviation for all the metrics corresponding to the same patch has been calculated (
Table A5).
As seen there, not only the quality metrics are excellent, but also the standard deviation is close to 0, meaning the color calibration is adequately performed on every picture and there is and extremely low variability between the processed images, regardless of their camera of source.
In
Table A6 and
Table A7 the accuracy of the calibration scheme thanks to the robustness of the chart is demonstrated. The average digital L*, a* and b* coordinates for the color patches of each calibrated picture have been extracted and compared with measurements from the printed chart made with the spectrophotometer. Their extreme similarity in all cases reveal the good performance of the calibration scheme, which is allowed with the robust printing on the chart.
5. Future Steps
Considering that the work presented in this paper is engulfed in a the framework of a bigger project, further steps to consider from this point on are the natural continuations of the premises exposed in the previous sections.
The calibration of crowdsourced images, to obtain reliable color coordinates from RGB images obtained with different cameras is a challenging question. Although the same accuracy of a dedicated instrument (spectrophotometer) can not be expected, results shown in this and previous papers by the authors [
11,
12,
13,
14,
15,
19] and other colleagues [
13,
14,
15] show promising results of this approach for our intended application (detection of degradation of heritage artifacts).
After these initial tests, deployment of the color charts in real locations is currently under way. Charts have been industrially printed, and with the corresponding metal coupons, they have been installed in museums, where they are being tested during an one-year period. Their performance and robustness will be evaluated, and possible improvements of the calibration and the crowdsourcing system will be made during that time.
Calibration of crowdsourced images, from different cameras and acquired under non-ideal illumination conditions and angles, is a challenging question that will be addressed in this step. While the accuracy of a dedicated instrument (spectrophotometer or calibrated professional camera) can no be expected, preliminary results suggest that can be enough to detect color changes in the sensitive materials, thus serving as alert system of environmental conditions dangerous for the conservation of heritage artifacts.
When a good performance can be assessed, the whole system can be officially launched and installed.