4.1. Using the RGBF Camera
Table 1 shows the assessment metric statistics for the test samples using the LUT method, where the cameras are the D5100 and RGBF. The LUT method for the D5100 camera is the same as the quadcolor camera shown in
Section 3.2, except that the signal space is reduced from 4D to 3D. As can be seen from
Table 1, using the RGBF camera reduced the mean
ERef, Δ
E00 and SCI of the inside samples and increased the mean GFC of the inside samples compared to the D5100 camera. The mean
ERef and GFC values of the outside samples using the RGBF camera were even smaller and larger than the inside samples using the D5100 camera, respectively. While non-zero as expected, the color difference Δ
E00 was small for most of the test samples. Compared to the D5100 camera, the mean
ERef of the test samples, inside samples and outside samples using the RGBF camera was reduced by 31.98%, 35.81% and 35.82%, respectively. Compared to the D5100 camera, the RGF99 of the test samples, inside samples and outside samples using the RGBF camera was increased from 0.9343, 0.9375 and 0.9208 to 0.9887, 0.9972 and 0.9706, respectively.
Table 2 is the same as
Table 1 except that the wPCA method was used. The wPCA method for the D5100 camera is the same as the quadcolor camera shown in
Section 3.1, except that three basis spectra were used. The inside and outside samples using the wPCA method were the same as those using the LUT method for comparison. The optimized
γ = 1.7 and 1.2 for the cases of using the D5100 and RGBF cameras, respectively. From
Table 2, the mean assessment metrics of the outside samples were worse than those of the inside samples. Compared to the D5100 camera, the mean
ERef values of the test samples, inside samples and outside samples using the RGBF camera were reduced by 21.6%, 27.3% and 24.9%, respectively. Compared to the D5100 camera, the RGF99 values of the test samples, inside samples and outside samples using the RGBF camera increased from 0.9493, 0.9676 and 0.8713 to 0.9765, 0.9945 and 0.9382, respectively. The improvement of RGF99 on outside samples using the RGBF camera is significant.
From
Table 1 and
Table 2, it can be seen that the LUT method outperformed the wPCA method using the RGBF camera. Note that the wPCA method outperformed the LUT method using the D5100 camera except for about two orders of magnitude longer computation time [
27,
29]. In [
29], the LUT method outperformed the wPCA method using the D5100 camera because the value of
γ was not optimized for the wPCA method, where
γ = 1.0.
Figure 8a–d show the
ERef, GFC, Δ
E00, and SCI histograms for the test samples, respectively, where the three shown cases are (i) using the D5100 camera and the wPCA method, (ii) using the RGBF camera and the LUT method, and (iii) using the RGBF camera and the wPCA method.
From
Figure 8a–d, the numbers of test samples in the “
ERef > 0.05”, “GFC < 0.99”, “Δ
E00 > 2.0”, and “SCI > 20” bins using the LUT method are less than those using the wPCA method. From
Table 1 and
Table 2, using the RGBF camera, the maximum
ERef = 0.0567 and 0.0742 for the cases of using the LUT and wPCA method, respectively. These results show that using the LUT method is more reliable for spectral reflectance recovery. However, when using the LUT method or the wPCA method, the assessment metrics were improved using the RGBF camera compared to the D5100 camera.
Since the mean assessment metrics of outside samples are worse than those of inside samples, the spectrum reconstruction of outside samples was investigated in more detail.
Figure 9a–f show the recovered spectral reflectance
SRefRec using the LUT method from the light reflected from 2.5G 7/6, 10P 7/8, 2.5R 4/12, 2.5Y 9/4, 10BG 4/8 and 5PB 4/12 color chips, respectively, where their target reflectance
SRef values are also shown. In addition to the D5100 and RGBF cameras,
Figure 9a–f also show the results using other cameras, which will be considered in the following subsections. The same color chips were used as examples in [
29] to show the spectral reflectance recovery using the D5100 camera and the LUT method. All the cases in
Figure 9a–f are outside examples. The case in
Figure 9a is an inside sample using the D5100 camera, but it becomes an outside sample using the RGBF camera.
Figure 10a–f are the same as
Figure 9a–f, respectively, except that spectra were recovered using the wPCA method.
Table 3 shows the
ERef values for the cases shown in
Figure 9a–f and
Figure 10a–f.
Table 4 is the same as
Table 3 except that the values of Δ
E00 are shown. Values for
ERef and Δ
E00 larger than 0.03 and 1.0, respectively, are shown in bold. The cases where the error
ERef of using the RGBF camera is larger than that of using the D5100 camera are the cases of
Figure 9e,f and the case of
Figure 10a. The cases where the difference Δ
E00 of using the RGBF camera is larger than that of using the D5100 camera are the cases of
Figure 9b,f and the cases of
Figure 10a,b,f. Compared to the D5100 camera, using the RGBF camera effectively improved the statistics of the assessment metrics, but it does not guarantee better spectral reflectance recovery for every color chip tested.
4.2. Using the RGBC Camera
Since there are red, green and blue channels, the fourth channel is reasonably designed to be either a cyan channel or a yellow channel. From the spectral sensitivity of the silicon sensor shown in
Figure 2b, if the fourth channel is a cyan channel, its sensitivity in long wavelength is suppressed. Using a compensation color filter shown in
Figure 3a modifies the greenish yellow channel in
Figure 2b to a yellowish green channel in
Figure 3b instead of a cyan channel. If the applied color filter has a much smaller mid- and long-wavelength transmittance than the filters shown in
Figure 3a, the fourth channel becomes the blue or greenish blue channel. The S channel of the RGBS camera is an example of such a case, which will be considered in
Section 4.3. The case with spectral sensitivity shown in
Figure 2b has been considered in
Section 4.1, i.e., the RGBF camera. This subsection will consider the cases with spectral sensitivities shown in
Figure 3b.
It was found that among the five color filters in
Figure 3a, the spectrum reconstruction error was the smallest when the IEC 131K filter was applied to the fourth channel. Using the LUT method, the mean
ERef = 0.0091, 0.0128, 0.0093, 0.0099 and 0.0125 for the cases with the IEC 131K, 501, 508, 518 and 578 filters, respectively; RGF99 = 0.9897, 0.9060, 0.9878, 0.9803 and 0.9240 for the cases with the IEC 131K, 501, 508, 518 and 578 filters, respectively. The mean spectrum reconstruction error increased as the spectral transmittance of filter decreased in the long wavelength region.
Table 5 shows the assessment metric statistics for the test samples of the RGBC camera with the IEC 131K filter using the LUT and wPCA methods. For the case of using the wPCA method, the optimized
γ = 1.2.
Figure 11a–d show the
ERef, GFC, Δ
E00, and SCI histograms, respectively, for the test samples of the optimized RGBC camera with the IEC 131K filter, using the LUT method.
Figure 12a–d are the same as
Figure 11a–d, except for using the wPCA method.
Figure 9a–f and
Figure 10a–f also show the spectral reflectance recovery examples using the optimized RGBC camera, where the values of
ERef and Δ
E00 are shown in
Table 3 and
Table 4, respectively.
For ease of comparison, the assessment metric statistics for using the RGBF camera are also shown in
Table 5. As can be seen from
Table 5, the assessment metric statistics using the optimized RGBC camera were about the same as the RGBF camera but with an additional compensation color filter. The use of the compensation color filter to suppress the spectral sensitivity of the fourth channel in the long wavelength region did not improve the performance of spectrum reconstruction.
4.3. Using the RGBS and RGBL Cameras
The above results show that using the fourth channel without a compensation color filter produced a slightly smaller mean
ERef. Further reduction in the mean
ERef is possible if the spectral sensitivity of the fourth channel can be appropriately modified using a color filter. This subsection considers the cases of using the short-pass and long-pass filters defined in
Section 2.1 as the color filter applied to the fourth channel.
Figure 13a shows the mean
ERef and 1– GFC of the test samples using the LUT and wPCA methods versus the edge wavelength
λS of the short-pass optical filter.
Figure 13b is the same as
Figure 13a except that the mean Δ
E00 and SCI are shown.
Figure 14a,b are the same as
Figure 13a,b, respectively, except that the long-pass optical filter was used. As can be seen from
Figure 13a and
Figure 14a, the mean 1– GFC roughly followed the mean
ERef. From
Figure 13b and
Figure 14b, the mean SCI roughly followed the mean Δ
E00.
Figure 15 shows the optimized value of
γ for the case of using the wPCA method in
Figure 13 and
Figure 14. For the case of using the RGBS camera, the optimized value of
γ for the wPCA method is larger around 530 nm, as shown in
Figure 15. We will show that the minimum CMFMisF is at this wavelength. From
Figure 13b and
Figure 14a, it can be seen that the RGBS and RGBL cameras are suitable designed for low color difference and low spectral error, respectively.
From
Figure 13a,b, using the LUT method, the quadcolor camera optimized for the minimum mean Δ
E00 is the RGBS camera using the short-pass optical filter of
λS = 528 nm. For this optimized RGBS camera, the mean
ERef = 0.0115 and Δ
E00 = 0.1666, where the mean
ERef is larger than the RGBF camera but smaller than the D5100 camera; the mean Δ
E00 is much smaller than the RGBF and D5100 cameras; the maximum Δ
E00 is only 1.0169. Using the wPCA method, the edge wavelength of the optimized RGBS is
λS = 529 nm and the optimized
γ = 1.9. For this optimized RGBS camera, the mean
ERef = 0.0141 and Δ
E00 = 0.158, where the mean
ERef and Δ
E00 are larger and smaller, respectively, than the D5100 and RGBF cameras.
From
Figure 14a,b, using the LUT method, the quadcolor camera optimized for the minimum mean
ERef is the RGBL camera using the long-pass optical filter of
λL = 585 nm. The mean
ERef = 0.0083 and Δ
E00 = 0.3506 using the optimized RGBL camera with
λL = 585 nm are smaller than the mean
ERef = 0.0089 and Δ
E00 = 0.3992 using the RGBF camera, respectively. Using the wPCA method, the edge wavelength of the optimized RGBL is
λL = 587 nm and the optimized
γ = 1.3. The mean
ERef = 0.0087 and Δ
E00 = 0.3862 using the optimized RGBL camera are smaller than the mean
ERef = 0.0095 and Δ
E00 = 0.4257 using the RGBF camera, respectively.
For the cases of using the LUT method, the spectral transmittance of the optimized long-pass and short-pass optical filters and their corresponding fourth channel spectral sensitivities are shown in
Figure 3b. The histograms of assessment metrics for using the optimized RGBS and RGBL cameras are shown in
Figure 11a–d for the case of using the LUT method.
Figure 12a–d are the same as
Figure 11a–d, respectively, except for the case of using the wPCA method. From
Figure 11a–d and
Figure 12a–d, it can be seen that the histogram characteristics for the RGBC and RGBL cameras are similar when using the LUT method or the wPCA method. The histogram characteristics for the RGBS camera are quite different from those for the RGBC and RGBL cameras when using the LUT method or the wPCA method. For the case of using the RGBS camera and the LUT method, there are 28, 52 and 2 test samples in the “
ERef > 0.05”, “GFC < 0.99” and “SCI > 20” bins, respectively, while there is only 1 test sample in the “Δ
E00 = 1.1” bin and no test sample in the larger Δ
E00 bins. For this case, the color difference of all test samples is low despite the large spectral error. For the case of using the RGBS camera and the wPCA method, there are 52, 82, 4 and 14 test samples in the “
ERef > 0.05”, “GFC < 0.99”, “Δ
E00 > 2.0” and “SCI > 20” bins, respectively. For this case, most of the test samples have a low color difference, but there are few test samples with a large color difference. Therefore, if the RGBS camera is used as an imaging colorimeter, it is more reliable to reconstruct spectra using the LUT method.
Table 5 shows the assessment metric statistics for the test samples of the optimized RGBS and RGBL cameras using the LUT and wPCA methods. For the case of using the LUT or wPCA method, the mean
ERef and mean Δ
E00 using the optimized RGBS camera are the largest and smallest, respectively, compared to the other quadcolor cameras.
Figure 9a–f and
Figure 10a–f also show the spectral reflectance recovery examples using the optimized RGBS and RGBL cameras, where the values of
ERef and Δ
E00 are shown in
Table 3 and
Table 4, respectively. Notably,
Figure 10f shows poor spectral reflectance recovered using the RGBS camera and the wPCA method, where the reflectance is 1.486 at 700 nm and zero around 580 nm. The zero is due to the negative value calculated from Equation (5).
The small color difference using the optimized RGBS camera can be explained using the CMFMisF defined in
Section 3.4.
Figure 16 shows the CMFMisF value versus the edge wavelength of the optical filter. Comparing
Figure 16 with
Figure 13b and
Figure 14b, it can be seen that the mean Δ
E00 closely relates to CMFMisF for the RGBS camera. The optimized edge wavelength of the RGBS camera using the LUT method or the wPCA method is about the wavelength of the minimum value in
Figure 16, where the minimum CMFMisF = 0.09537 at
λS = 530 nm. CMFMisF = 0.1495, 0.1495, 0.1486, 0.09543 and 0.1493 for the RGB, RGBF, optimized RGBC, optimized RGBS (
λS = 528 nm) and optimized RGBL (
λL = 585 nm) cameras, respectively. Note that the CMFMisF values are the same for the RGB and RGBF cameras, since the fourth channel of the RGBF camera contributes negligibly to the fit. The CMFs can be better fitted using the spectral sensitivities of the optimized RGBS camera.
Figure 17a,b show the least squares fits of the CMF vectors using the spectral sensitivity vectors of the RGBF and optimized RGBS camera (
λS = 528 nm), respectively. Although the CMF vectors were not well fitted in
Figure 17a, if the spectrum of the test sample is well reconstructed using the RGBF camera, the color difference Δ
E00 of the test sample will not be small.
As a comparison, the authors of [
29] showed that for the same test samples using the CMF camera and optimized CMY ARS filters, the mean
ERef, GFC, Δ
E00 and SCI are 0.0132, 0.9972, 0.0 and 4.1869, respectively. The CMF camera is the artificial tricolor camera with spectral sensitivity functions that are the same as the CMFs. While the mean Δ
E00 is zero, the mean
ERef of the CMF camera is about the same as the D5100 camera.
4.4. Cross Comparison
From the results shown above, the LUT method outperformed the wPCA method in spectrum reconstruction. We first discuss the case of using the LUT method. Compared to the D5100 camera, the mean ERef using the RGBF and optimized RGBL cameras were reduced by 32.0% and 36.8%, respectively. Compared to the D5100 camera, the mean ΔE00 using the RGBF and optimized RGBS cameras were reduced by 5.3% and 60.5%, respectively. Compared to the RGBF camera, the advantage of using the optimized RGBL camera to obtain a smaller mean ERef was not significant; the advantage of using the optimized RGBS camera to obtain a smaller mean ΔE00 was significant. Compared to the RGBF camera, the mean ΔE00 using the optimized RGBS camera was reduced by 58.3%, but the mean ERef was increased by 28.6%. However, since the mean ΔE00 was as small as 0.3992, using the RGBF camera may be better than the optimized RGBS camera in spectral reflectance recovery due to the smaller mean ERef and no need to use the color filter applied to the fourth channel.
In this paragraph, the use of the wPCA method is discussed. Compared to the D5100 camera, the mean ERef values using the RGBF and optimized RGBL cameras were reduced by 21.6% and 28.2%, respectively. Compared to the D5100 camera, the mean ΔE00 values using the RGBF and optimized RGBS cameras were increased by 9.6% and reduced by 59.3%, respectively. Compared to the RGBF camera, the advantage of using the optimized RGBL camera to obtain smaller spectral error was also not significant. Compared to the RGBF camera, using the optimized RGBS camera had a 62.9% smaller mean ΔE00 but a 49.5% larger mean ERef.
The RGBF camera is a compromised design for the case of using the LUT or wPCA method. If a mean ERef smaller than that of the RGBF camera is required, a suitable long-pass optical filter can be applied to the fourth channel. If an ultra-small color difference is required, a suitable short-pass optical filter can be applied to the fourth channel, but care must be taken not to add too much spectral error.
The computation time required for the LUT method is about two orders of magnitude faster than that required for reconstruction methods using basis spectra that emphasize the relationship between the test and training samples for the 3D case [
27,
29]. However, for the 4D case, the computation time required to use the LUT method is longer than the wPCA method, because it takes much longer time to locate the simplex for interpolation than the 3D case. For the case of using the quadcolor camera, the ratio of the computation time required to use the LUT method and wPCA method was 1:0.49, where samples were reconstructed from their signal vector
C to the spectral reflectance vector
SRefRec using MATLAB on the Windows 10 platform. The improvement of the algorithm to locate the simplex faster is a computational geometry problem and is beyond the scope of this paper.