Next Article in Journal
Recognition and Grasping of Disorderly Stacked Wood Planks Using a Local Image Patch and Point Pair Feature Method
Previous Article in Journal
Development of Washable Silver Printed Textile Electrodes for Long-Term ECG Monitoring
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Accuracy of an Affordable Smartphone-Based Teledermoscopy System for Color Measurements in Canine Skin

1
Biophotonics Laboratory, Institute of Atomic Physics and Spectroscopy, University of Latvia, 19 Rainis Blvd., LV-1586 Rīga, Latvia
2
Vetamplify SIA, Veterinary Services, 57/59–32 Krišjāņa Valdemāra Str., LV-1010 Rīga, Latvia
*
Author to whom correspondence should be addressed.
Sensors 2020, 20(21), 6234; https://doi.org/10.3390/s20216234
Submission received: 21 October 2020 / Revised: 30 October 2020 / Accepted: 30 October 2020 / Published: 31 October 2020
(This article belongs to the Section Biomedical Sensors)

Abstract

:
Quality smartphone cameras and affordable dermatoscopes have enabled teledermoscopy to become a popular medical and veterinary tool for analyzing skin lesions such as melanoma and erythema. However, smartphones acquire images in an unknown RGB color space, which prevents a standardized colorimetric skin analysis. In this work, we supplemented a typical veterinary teledermoscopy system with a conventional color calibration procedure, and we studied two mid-priced smartphones in evaluating native and erythematous canine skin color. In a laboratory setting with the ColorChecker, the teledermoscopy system reached CIELAB-based color differences ΔE of 1.8–6.6 (CIE76) and 1.1–4.5 (CIE94). Intra- and inter-smartphone variability resulted in the color differences (CIE76) of 0.1, and 2.0–3.9, depending on the selected color range. Preliminary clinical measurements showed that canine skin is less red and yellow (lower a* and b* for ΔE of 10.7) than standard Caucasian human skin. Estimating the severity of skin erythema with an erythema index led to errors between 0.5–3%. After constructing a color calibration model for each smartphone, we expedited clinical measurements without losing colorimetric accuracy by introducing a simple image normalization on a white standard. To conclude, the calibrated teledermoscopy system is fast and accurate enough for various colorimetric applications in veterinary dermatology.

1. Introduction

Smartphones have become a popular tool for monitoring human health and well-being [1]. Recent advances in device display, sensors, battery, and connectivity have enabled stand-alone detection of cardiovascular parameters (heart or respiratory rate), sleep patterns, activity levels, cognitive functions, and changed health conditions in skin and eyes [1]. In veterinary medicine, smartphones were similarly applied for activity monitoring, electrocardiography, and endoscopy in various pet and farm animals [2,3,4]. By attaching adapters, holders, and additional mechanical, electrical, or optical modules, smartphones can also be suitable for various laboratory applications [5]. Colorimetric analyses of urine- or blood-related strips [6] and immunochromatographic assays [7] are among the most popular. Furthermore, smartphones have been exploited as analyzers of the enzyme-linked immunosorbent assay (ELISA) microplates, microfluidic system modules, and microscopes [8].
Smartphones have also contributed to developing a new subfield of dermatology – smartphone-based teledermoscopy [9,10]. Optical systems joining capable smartphone cameras and mobile dermatoscopes allow users to take and share quality images of their skin lesions, which could serve as preventive screening for cancer [9], especially where professional assistance is not available face to face. Compared to in-person dermatological services, smartphone-based teledermoscopy achieved the same sensitivity but lower specificity for detecting melanoma [11]. Furthermore, teledermoscopy can be complemented with image processing algorithms for automated skin lesion classification [12]. Rizvi and others even speculated that many regular consultations on skin lesions with the specialist could be replaced by the use of smartphone applications [13].
In human dermatology, diagnosing skin lesions usually relies on a dermatoscope, enabling detailed lesion observation according to the ABCD rules or 7-point checklist [14]. However, because melanomas in animals are less frequent and malignant, veterinarians typically employ dermatoscopes for studying other skin conditions such as alopecia (e.g., hair loss) [15]. Our research group recently used an affordable ($300) dermatoscope, the DermLite DL1 basic (3Gen, Inc., San Juan Capistrano, CA, USA) for the objective estimation of skin erythema severity in atopic dogs [16]. There was a high correlation (r = 0.82–0.85) and prediction ability (r2 = 0.72–0.73) between a dermatologist’s visual erythema estimates and an erythema index (EI), calculated directly from the image RGB values.
However, each smartphone acquires images in the camera-dependent RGB color space due to the sensor’s unique spectral response, preventing standardized colorimetric skin assessments. One option to overcome this limitation is to perform a transformation into a device-independent color space (e.g., CIELAB [17]) with the help of color calibration targets (CCT). The optical systems and transformation between color spaces naturally cannot guarantee perfect color accuracy. Therefore, an error between the measured and reference color appears called color difference (ΔE). The original, CIE76, Euclidian distance-based formula for ΔE estimation was updated twice (i.e., CIE94, CIEDE2000), but expanded versions are less often listed in the literature.
The CIELAB color space-based color difference ΔE can be transposed to the human eye capability to spot a difference between two colors. The just-noticeable difference (JND) usually has a ΔE estimated to be 2.3, but the detection threshold can be up to 5.0 [17]. Based on the popular CCT, such as the ColorChecker, various optical systems achieved similar colorimetric accuracies. The stand-alone use of popular cameras (e.g., SX200 IS, Canon, Tokyo, Japan; D3100, Nikon, Tokyo, Japan; or ST60, Samsung Electronics Co. Ltd., Suwon, Korea) produced color differences (CIE76) in the range of 3.7–5.9 [18,19]. An attached dermatoscope (Dermlite Pro) lowered the CIE76 ΔE to between 2.5 and 4.0 [20]. Furthermore, a camera (IDS UI-2240-C, IDS Imaging Development Systems GmbH, Obersulm, Germany) attached to the microscope resulted in a CIE 94 color distance of 2.2 [21].
The color constancy of smartphones (Samsung Galaxy S4 and S7, iPhone 5S) was slightly worse, with ΔEs respectively ranging between 3.5–8.4 [22], 7.4–10.2 (CIE76) [23], and 3.0–4.4 (CIEDE2000) [24]. Finally, the optical system with a smartphone and a miniature microscope ensured the CIE 94 color difference between 2.0 and 2.4 [8].
This work’s overarching goal is to set up a color calibration on a typical veterinary smartphone-based teledermoscopy system and study the colorimetric accuracy of two mid-priced smartphone cameras on canine skin. First, we transformed the acquired data from device-dependent (RGB) to device-independent (CIELAB, sRGB) color space. We continued with estimating colorimetric accuracy in a laboratory setting on the standard color calibration target (CCT), ColorChecker. We also made preliminary measurements of canine skin in terms of CIELAB color and erythema indices (sRGB). Additionally, we evaluated color variability for each (intra-) and between both smartphones (inter-smartphone accuracy). Finally, we confirmed that a simple image normalization on the white standard could expedite clinical measurements without decreasing colorimetric accuracy.

2. Materials and Methods

2.1. Optical System

We employed two mid-priced smartphones. The first (SP1) was a Nokia 6 smartphone (v.2017, HMD Global, Espoo, Finland) equipped with a 16 MP camera (f/2.0). JPEG images (90% quality, 4608 × 3456 pixels) were acquired in the program Open Camera (v1.47.3, Mark Harman, Cambridge, UK) with the following settings: photo mode (STD), white balance (manual, fluorescent), scene mode (steady photo), color effect (none), ISO (50), and focus (auto). The second smartphone (SP2) was a Samsung Galaxy A51 (Samsung Electronics Co. Ltd., Suwon, Korea) with a 48 MP camera (f/2.0). The images (90% quality, 4000 × 3000 pixels) were taken in the Open Camera program with the following settings: photo mode (STD), white balance (manual, cloudy), color effect (none), ISO (100), and focus (auto). To both smartphones, we attached DermLite DL1 basic a dermatoscope, an epiluminescence microscopy device with a 15 mm lens (Figure 1). The illumination consisted of four white LEDs with irradiance fluctuations being expected to be up to 0.9% [23]. Additionally, a polarizer reduced reflections from the skin surface. The camera was held 3.1 cm away from the skin, resulting in a circular 3.0 cm2 sampling area.

2.2. Laboratory Measurements

Two ColorCheckers (Figure 2, X-rite, Grand Rapids, MI, USA) served as the reference color calibration targets (CCTs). For testing color calibration models on the whole color range, we exploited all patches on the Classic ColorChecker (full-range CCT). Only the white and skin patches (E5, D-J 7&8) on the Digital SG ColorChecker (skin-range CCT) were employed for skin color analysis.
Figure 3 displays flowcharts of different color calibration procedures, described below. First, we acquired six repeated images of each CCT (Ifull/skin1–6), thus having 12 individual image sets. For each set separately, the brightest white patch (19 for the full-range and E5 for the skin-range CCT) served for data normalization and setting a fixed camera exposure time, which did not change within image set. A median value of central 750 pixels in both dimensions served for the calculation of the image’s RGB values: R (red), G (green), and B (blue).
For each image set (Figure 3a), we then built a linear-regression model between an unknown device-dependent RGB and a device-independent CIELAB color space: LAB = ffull/skin1–6(RGB). The mathematical model was based on the best performing polynomial from our previous study [23]:
L A B = a 0 + a 1 R + a 2 G + a 3 B ,           { L * , a * , b * }   L A B ,
where L* (lightness), a* (green-red), and b* (blue-yellow) are coordinates of the CIELAB color space, and a0–3 are regression coefficients, estimated with the function fitlm (Matlab, R2016a, MathWorks, Natick, MA, USA). The color difference (ΔE, CIE76) was calculated from fitting residuals as a Euclidean distance between estimated and true color [17]:
Δ E = ( Δ L * ) 2 + ( Δ a * ) 2 + ( Δ b * ) 2 .
CIE94′s ΔE was estimated with the openly available Matlab code [25].
Based on the full-range CCT, we additionally tested a few modified color calibration procedures, which could improve, deteriorate, simplify, or expedite clinical measurements:
(1)
Longer exposure time. If the exposure time is not locked on the brightest CCT’s patch (e.g., patch 19), the camera sensor’s capacity limit can be reached, leading to saturation. We acquired additional six full-range CCT image sets (Sfull1–6) with the exposure time locked on the slightly gray patch (full-range CCT: patch 20, neutral 8 (.23 *), L* = 81.3) and a transformation (regression) model was built in the same manner as described before (Figure 3a). Finally, the retrieved color differences were calculated and compared to the calibration on the image sets Ifull/skin1–6.
(2)
Full normalization (Figure 3b) was done across the whole lightness range based on the grayscale patches (No. 19–24). As guaranteeing a color constancy by acquiring all 24 patches for each clinical measurement could be burdensome and time-consuming, we expedited the color calibration procedure by introducing an initial transformation from the subsequent (Ifull 2–6) to the first image set (Ifull 1) RGB color space: RGB1 = g(RGB2–6), where g was a quadratic polynomial. Six grayscale patch images served for the determination of function’s coefficients with the regression model fit. Finally, RGB values were converted to the CIELAB color space based on the known f1 (Figure 3a).
(3)
Single-point normalization (Figure 3c) expedited a color calibration procedure since the normalization was implemented with only the brightest (No. 19) instead of the six grayscale patches. As before, the transformation to CIELAB color space was based on the known model f1 from the image set Ifull 1.

2.3. Clinical Measurements

Clinical measurements were ethically approved by the Administration of the Republic of Slovenia for Food Safety, Veterinary Sector and Plant Protection under the number U3440-187/2020/7.
In addition to the image sets Ifull 1–6, we acquired skin images of six dogs in the inguinal region. For each measurement site, we repeated the image acquisition on the same spot. The typical skin RGB values were calculated as the pixels mean from two manually selected small squares [26]. We transformed the RGB values into the CIELAB color space based on the known models f1–6 (Figure 3a). We additionally compared skin color estimates between repetitions (intra-smartphone variability) and both smartphones (inter-smartphone variability), according to Equation (2) above. Finally, LAB skin values were determined with a single-point normalization (Figure 3c) and compared to the original fits.
We calculated the erythema index (EI) for canine skin measurements and for the selected full-range CCT color patches (1–3, 5, 8, 9, and 19–24), which have physiological EI values. First, device-dependent RGB color data were transformed to the standard RGB color space (sRGB, Figure 3d) based on the reference values of grayscale patches (No. 19–24). Then, we calculated the erythema index EIBRG:
E I B R G = B · R G .
We opted for an EIBRG since it was correlated the most with the canine erythema severity [26]. We additionally calculated EI with two additional (and faster) color calibration procedures (i.e., the full and single-point normalizations, Figure 3b,c). Finally, we studied agreements on EI estimation between both smartphones and against the full-range CCT reference values.

3. Results

Laboratory measurements on the full-range CCT showed that the Nokia smartphone (SP1) achieved a slightly better color accuracy than the Samsung smartphone (SP2) since their average CIE76 color differences were 6.45 and 6.60, respectively (Figure 4, Table 1). The average variability in color differences for each smartphone (i.e., intra-smartphone variability) was up to CIE76 ΔE of 0.12 (CIE94 ΔE = 0.08). When we studied the differences in color estimates between both smartphones (i.e., inter-smartphone variability), CIE76 ΔE increased to 3.91 (CIE94 ΔE = 2.60). When only the skin color patches were involved in the color calibration, the color differences were significantly smaller (Figure 4, Table 1).
Furthermore, the impact of all three modified color calibration procedures (i.e., a longer exposure time, or full-, or single-point normalizations) on the colorimetric accuracy was limited (Figure 5). On average, CIE76 ΔE fell for 0.48, when SP1 operated closer to its sensor’s capacity limit. On the other hand, P2′s CIE76 color difference increased by 0.07. Normalization procedure on the grayscale patches (i.e., full normalization) or the single white patch (single-point normalization) did not improve or deteriorate colorimetric accuracy significantly; the change in CIE76 color difference (ΔE) was between −0.16 and 0.11.
We performed preliminary clinical measurements on the skin of six dogs to determine and compare the LAB values to those of human skin based on the specifications from the full- and skin-range CCT (Figure 6a). Both, human and canine skin exhibited a similar lightness range, but the latter appeared to be less red and yellow (i.e., lower a* and b* values). Furthermore, the skin color estimations between both smartphones resulted in the mean CIE76 color difference of 2.97 (i.e., inter-smartphone variability, Figure 6b). The mean ΔE between the original and repeated color estimates of the single SP1 smartphone (i.e., intra-smartphone variability) was 2.18. Finally, when we studied the simplest color calibration of a single-point normalization on the brightest color patch (Figure 3c), the additional change in color difference was around 0.47.
We additionally discovered significant absolute differences between the acquired and referential RGB values studying the grayscale patches on the full-range CCT (Figure 7).
The first smartphone (SP1) generally produced smaller RGB differences on the bright patch (i.e., normalized referential RGB values of around 0.8), but the SP2-based system performed better with darker images (Figure 7a). However, both smartphones’ sensors responses were completely depressed while acquiring the black patch with a normalized referential RGB value (sRGB) of around 0.2 a.u. Furthermore, prolonging the exposure time by the image normalization on the gray patch 20 (instead of the white patch 19) significantly affected the sensor’s response with the increased RGB values up to 0.05 in the referential RGB range between 0.50 and 0.66 a.u. (Figure 7b). Although the differences to the standard grayscale CCT patches were smaller, the absolute differences remained significant, that is up to 0.20. Finally, we found that even individual RGB sensor elements have different spectral responses with channel-specific RGB differences of up to 0.04 a.u. (Figure 7c).
Finally, we studied differences in erythema estimates (EIBRG) with various color calibration procedures (Table 2). The transformation to the standard RGB color space (sRGB) significantly improved the EIBRG agreement since absolute differences in erythema estimates dropped to around 0.04 compared to the calculations in the device-dependent RGB color space (0.29–0.60). We also found that the full normalization on six CCT patches did not outperform the shorter and simpler normalization with a single white patch. Repeated measurements on the CCT resulted in the EIBRG differences (i.e., intra-smartphone variability) of 0.005 and 0.003 for SP1 and SP2, respectively. Clinical skin measurements exhibited an increased intra-smartphone variability of 0.008 ± 0.007 and 0.011 ± 0.011 for SP1 and SP2, respectively. Single differences in EIBRG of the selected CCT patches and canine skin between both smartphones are shown in Figure 8.

4. Discussion

In this work, we characterized the color calibration using a typical veterinary smartphone-based teledermoscopy system. We studied its colorimetric accuracy in both laboratory (CIELAB color space with the color calibration target ColorChecker) and clinical settings (RGB-based erythema index of canine skin). We discovered that the calibrated teledermoscopy system is accurate enough since the CIELAB-based color differences (ΔE) were up to 6.6. Furthermore, estimating the severity of skin erythema with an erythema index led to errors lower than 3%. Finally, we showed that we could achieve comparably accurate colorimetrical measurements with a simple image normalization on the white surface without the need for time-consuming, full-color calibration.
In the laboratory settings on the full-range CCT, the proposed teledermoscopic setup with a smartphone and a dermatoscope achieved a decent color constancy with the average CIE76 color difference between 6 in 7 (Figure 4). Based on a polynomial of degree one with only four estimated parameters, a robust transformation model resulted in the color differences in the range of existing studies (ΔE between 2.5 and 10.2 [8,18,19,20,21,22,23]). However, some of the studies (e.g., Zhang et al. [8]) employed the quadratic polynomial with ten parameters included. When the same polynomial was applied to our measurements, the CIE76 ΔE decreased to 3.8 and 0.9 for the full- and skin-range CCT, respectively. Nevertheless, we do not recommend using a complex polynomial with a small number of measurements (in our case: 14–24) since it can easily lead to overfitting. Furthermore, we mostly listed the CIE76 color difference, which seemed to have an increased value in the used color range compared to the CIE94 and CIEDE2000 standards. For example, ΔE decreased to 3.9 (CIE94) or 3.8 (CIEDE2000) when the updated equations for color difference were applied. Furthermore, adding a complex polynomial to the transformation model resulted in the color difference (CIE94) of 2.3, which corresponds to the lower limit of the reported ΔE range [8].
Wang and Zhang [18] showed that utilizing CCT with a narrower color range halved the color differences (e.g., from 10.4 to 5.4). We observed a similar phenomenon since the transformation models on the skin-range CCT had a significantly lower ΔE (Figure 4, ΔE of 2–3). We also discovered comparably low color differences between the color estimates from both smartphones. Since the retrieved ΔE fell within the range of the human eye’s capability to spot a color difference (i.e., just noticeable difference, JND), our results point towards the adequacy of smartphones for dermatological use.
Selecting a longer exposure time (Figure 5) based on the gray patch (No. 20) increased the image’s RGB values. On the one hand, this improved the contrast of dark images, which would otherwise have similar, depressed RGB values. For example, longer exposure times decreased the percentage of low-intensity color patches (normalized RGB < 0.01) from 8.9% to 8.3% and from 15.0% to 13.1% for SP1 and SP2, respectively. On the other hand, prolonged exposures can lead the camera sensor to be close to its saturation charge level. The image pixels of the brightest patch (No. 19) held almost maximal RGB values, i.e., 0.92 and 0.96 for SP1 and SP2, respectively. Additionally, as shown in our previous study [23], brighter images with higher contrast can sometimes contribute to a better color constancy (e.g., with the first smartphone SP1). However, the effect of a longer exposure time can also have a negligible or even adverse impact, probably due to image saturation. Simultaneously, full range or single-point normalizations (Figure 3b,c) were more straightforward color calibration procedures. Both approaches sped up the clinical measurements without affecting the color constancy of both smartphones (Figure 5). Therefore, a single-point normalization is the most clinically appropriate since it is the fastest and the same accurate as full normalization.
We showed similarities between the lightness of human and canine skin in Figure 6a. However, it seems that canine skin is less red and yellow. Its average CIELAB’s a* and b* were 10.0 and 6.9, compared to 15.8 and 15.9 (full-range CCT), 20.2 and 25.3 (skin-range CCT) of human skin, resulting in color differences (CIE76 ΔE) of 10.7 (canine vs. human full-range CCT skin) and 10.3 (human full- vs. skin-range CCT skin). One could object that the color difference between our estimates and the full-range CCT skin specifications appeared due to the average smartphone error (ΔE = 6.5, Figure 4). However, our models (ffull 1–6) overestimated both skin patches (No. 1, 2) of the full-range CCT for ΔE of 6.0 (mean estimated and referential a* and b* values were 18.8 & 21.1, and 15.8 & 15.9, respectively). Therefore, misestimates due to the smartphone and model errors would compensate in the worst-case scenario, keeping the color difference between canine and human skin of around 10. On the other hand, the color differences between full- and skin-range CCT are expected since the semi-gloss finish of the skin-range CCT causes overestimates in a* and b* for ΔE of around 7 [27]. We should note that these conclusions are based on the standard human skin colors of the ColorChecker. If the normal canine skin color range was estimated, in vivo human skin measurements would contribute to the more real color comparison. However, these data will not improve the proposed optical system’s accuracy study since the actual colors (i.e., gold standard) would remain unknown.
Comparing skin color estimates between smartphones (Figure 6b) resulted in the ΔE of 3.0, similar to the results on the full-range CCT (Figure 4). However, we can attribute a significant portion of this color disagreement to the variability in clinical measurements (ΔE = 2.2). Since sampling the representative canine skin was done manually, it was a challenge to guarantee entirely repeatable skin area selection. Furthermore, a single-point normalization proved, again, to be sufficient for color calibration after the basic color transformation model for each smartphone was made.
This work and many other studies relied on the colorimetric parameters retrieved after the transformation between RGB and CIELAB color spaces, which guaranteed a certain level of colorimetric standardization. However, when a less experienced reader encounters parameters characterized directly in the device-dependent RGB color space, he or she could assume that these results are standardized and comparable between smartphones as well. As we showed in Figure 7 and Table 2, performing only an image normalization with a single reference point resulted in a significant disagreement (up to 60%) between both smartphones and the reference. We can attribute these misestimates in EI to two basic reasons. First, smartphone cameras (and their separate RGB channels) have individual spectral responses with different sensitivities to detect a certain lightness level (Figure 7a,c). For example, the first smartphone images had significantly lower absolute RGB values compared to the second smartphone. Secondly, the smartphone camera response is not linear; therefore, setting a specific exposure time affected the acquired data in the skin lightness range (Figure 7b). Therefore, we needed to construct a transformation to the device-independent standard RGB (sRGB) color space before any RGB-based parameter calculations (Figure 3d).
In our previous study [16], we estimated the EIRGB range of canine skin between 1.0 (healthy skin) and 2.0 (severely erythematous skin). After the transformation to sRGB, our teledermatoscopic setup achieved intra-smartphone EI variability up to 0.01, which corresponds to a 0.5–1.0% margin of error. Additionally, disagreements in the erythema index between both smartphones were, on average, around 0.03, matching an error between 1.5–3%. These results are superior to some of the other smartphone-based optical systems that assess skin chromophores such as hemoglobin [28], which reported disagreements between smartphones exceeding 25%.
Finally, we shall point out that only six dogs were enrolled in this study. However, it seems that 17 clinical (Figure 8b), in addition to the numerous laboratory measurements are sufficient for confirming the colorimetric reliability of the proposed optical system. Moreover, the results indicated that the measurement site (re-)selection and image sampling are significant factors for the color disagreement (Figure 6b). Therefore, the measuring and sampling procedures should be better specified before any comprehensive clinical study.

5. Conclusions

Although smartphone cameras acquire images in an unknown RGB color space, a simple color space transformation ensured accurate colorimetric measurements with color differences ΔE (CIE76) of 1.8–6.6 (CIE94 ΔE = 1.1–4.5). Since the results are within the range of JND (i.e., the human eye’s capability to spot a color difference), the proposed teledermoscopic optical setup with a smartphone and dermatoscope has the potential to replace the visual skin color assessment. We assume that we could achieve even better color constancy with the help of canine skin-tailored color patches. However, native canine skin color should be determined first. Besides the erythema index, we could study additional colorimetric parameters for erythema estimation (e.g., a* of the CIELAB color space). To conclude, the proposed affordable teledermatoscopy system has the capability to be employed in colorimetric studies in veterinary dermatology.

Author Contributions

Conceptualization, and data processing, B.C.; methodology, B.C. and E.Š.; measurements, E.Š.; writing—original draft preparation, B.C. and E.Š.; writing—review and editing, B.C. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by the Marie Skłodowska-Curie Actions of the European Union (IF, DogSPEC, 745396) and by the Latvian State Education Development Agency under the grant agreement 1.1.1.2/VIAA/3/19/455 (GoBVM).

Acknowledgments

We would like to express our gratitude to Inga Saknīte (Vanderbilt University Medical Center, Nashville, TN, USA), Thierry Olivry (NC State University, Raleigh, NC, USA), Jānis Spīgulis and Uldis Rubīns (University of Latvia, Riga, Latvia) for all the help with preparing this paper. We also thank Katja Lužar and Urška Kovče from the veterinary clinic Zamba (Celje, Slovenia) for their help in measuring canine skin color.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Majumder, S.; Deen, M.J. Smartphone Sensors for Health Monitoring and Diagnosis. Sensors 2019, 19, 2164. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  2. Huynh, M. Smartphone-Based Device in Exotic Pet Medicine. Vet. Clin. Exot Anim. 2019, 22, 349–366. [Google Scholar] [CrossRef] [PubMed]
  3. Debauche, O.; Mahmoudi, S.; Andriamandroso, A.L.H.; Manneback, P.; Bindelle, J.; Lebeau, F. Cloud services integration for farm animals’ behavior studies based on smartphones as activity sensors. J. Ambient Intell. Hum. Comp. 2019, 10, 4651–4662. [Google Scholar] [CrossRef] [Green Version]
  4. Freitag, F.A.V.; Muehlbauer, E.; Martini, R.; Froes, T.R.; Duque, J.C.M. Smartphone otoscope: An alternative technique for intubation in rabbits. Vet. Anaesth. Analg. 2020, 47, 281–284. [Google Scholar] [CrossRef] [PubMed]
  5. Quesada-González, D.; Merkoçi, A. Mobile phone-based biosensing: An emerging “diagnostic and communication” technology. Biosens. Bioelectron. 2017, 92, 549–562. [Google Scholar] [CrossRef] [Green Version]
  6. Kim, S.D.; Koo, Y.; Yun, Y. A Smartphone-Based Automatic Measurement Method for Colorimetric pH Detection Using a Color Adaptation Algorithm. Sensors 2017, 17, 1604. [Google Scholar] [CrossRef] [Green Version]
  7. Hou, Y.; Wang, K.; Xiao, K.; Qin, W.; Lu, W.; Tao, W.; Cui, D. Smartphone-Based Dual-Modality Imaging System for Quantitative Detection of Color or Fluorescent Lateral Flow Immunochromatographic Strips. Nanoscale Res. Lett. 2017, 12, 291. [Google Scholar] [CrossRef] [PubMed]
  8. Zhang, Y.; Wu, Y.; Zhang, Y.; Ozcan, A. Color calibration and fusion of lens-free and mobile-phone microscopy images for high-resolution and accurate color reproduction. Sci. Rep.-UK 2016, 6, 27811. [Google Scholar] [CrossRef]
  9. Bruce, A.F.; Mallow, J.A.; Theeke, L.A. The use of teledermoscopy in the accurate identification of cancerous skin lesions in the adult population: A systematic review. J. Telemed. Telecare 2018, 24, 75–83. [Google Scholar] [CrossRef]
  10. Lee, K.J.; Finnane, A.; Soyer, H.P. Recent trends in teledermatology and teledermoscopy. Dermatol. Pract. Concept. 2018, 8, 214–223. [Google Scholar] [CrossRef]
  11. Vestergaard, T.; Prasad, S.C.; Schuster, A.; Laurinaviciene, R.; Andersen, M.K.; Bygum, A. Diagnostic accuracy and interobserver concordance: Teledermoscopy of 600 suspicious skin lesions in Southern Denmark. J. Eur. Acad. Dermatol. 2020, 34, 1601–1608. [Google Scholar] [CrossRef]
  12. Jaworek-Korjakowska, J.; Kleczek, P. eSkin: Study on the Smartphone Application for Early Detection of Malignant Melanoma. Wirel. Commun. Mob. Comput. 2018, 2018, 5767360. [Google Scholar] [CrossRef] [Green Version]
  13. Rizvi, S.M.H.; Schopf, T.; Sangha, A.; Ulvin, K.; Gjersvik, P. Teledermatology in Norway using a mobile phone app. PLoS ONE 2020, 15, e0232131. [Google Scholar] [CrossRef] [PubMed]
  14. Abbasi, N.R.; Shaw, H.M.; Rigel, D.S.; Friedman, R.J.; McCarthy, W.H.; Osman, I.; Kopf, A.W.; Polsky, D. Early Diagnosis of Cutaneous Melanoma: Revisiting the ABCD Criteria. JAMA 2004, 292, 2771–2776. [Google Scholar] [CrossRef] [PubMed]
  15. Zanna, G.; Roccabianca, P.; Zini, E.; Legnani, S.; Scarampella, F.; Arrighi, S.; Tosti, A. The usefulness of dermoscopy in canine pattern alopecia: A descriptive study. Vet. Dermatol. 2017, 28, 161-e34. [Google Scholar] [CrossRef] [Green Version]
  16. Cugmas, B.; Olivry, T. Evaluation of skin erythema severity by dermatoscopy in dogs with atopic dermatitis. Vet. Dermatol. 2020. accepted. [Google Scholar]
  17. Mokrzycki, W.; Tatol, M. Color difference Delta E - A survey. Mach. Graph. Vis. 2011, 20, 383–411. [Google Scholar]
  18. Wang, X.; Zhang, D. A New Tongue Colorchecker Design by Space Representation for Precise Correction. IEEE J. Biomed. Health 2013, 17, 381–391. [Google Scholar] [CrossRef]
  19. Goñi, S.M.; Salvadori, V.O. Color measurement: Comparison of colorimeter vs. computer vision system. J. Food Meas. Charact. 2017, 11, 538–547. [Google Scholar] [CrossRef]
  20. Quintana, J.; Garcia, R.; Neumann, L. A novel method for color correction in epiluminescence microscopy. Comp. Med. Imaging Graph. 2011, 35, 646–652. [Google Scholar] [CrossRef]
  21. Charrière, R.; Hébert, M.; Trémeau, A.; Destouches, N. Color calibration of an RGB camera mounted in front of a microscope with strong color distortion. Appl. Opt. 2013, 52, 5262–5271. [Google Scholar] [CrossRef] [PubMed]
  22. Amani, M.; Falk, H.; Jensen, O.D.; Vartdal, G.; Aune, A.; Lindseth, F. Color Calibration on Human Skin Images. In Proceedings of the Computer Vision Systems; Tzovaras, D., Giakoumis, D., Vincze, M., Argyros, A., Eds.; Springer International Publishing: Cham, Switzerland, 2019; pp. 211–223. [Google Scholar]
  23. Cugmas, B.; Pernuš, F.; Likar, B. Color constancy in dermatoscopy with smartphone. Proc. SPIE 2017, 10592, 105920G. [Google Scholar]
  24. Mulcare, D.C.; Coward, T.J. Suitability of a Mobile Phone Colorimeter Application for Use as an Objective Aid when Matching Skin Color during the Fabrication of a Maxillofacial Prosthesis. J. Prosthodont. 2019, 28, 934–943. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  25. Westland, S. Computational Colour Science Using MATLAB 2e. Available online: https://se.mathworks.com/matlabcentral/fileexchange/40640-computational-colour-science-using-matlab-2e (accessed on 3 July 2020).
  26. Cugmas, B.; Olivry, T.; Olivrī, A.; Spīgulis, J. Skimager for the objective erythema estimation in atopic dogs. Proc. SPIE 2020, 11211, 1121110. [Google Scholar]
  27. BabelColor The Problem with the ColorChecker Digital SG. Available online: http://www.babelcolor.com/colorchecker-3.htm#CCP3_SGproblem (accessed on 10 June 2020).
  28. Kuzmina, I.; Lacis, M.; Spigulis, J.; Berzina, A.; Valeine, L. Study of smartphone suitability for mapping of skin chromophores. J. Biomed. Opt. 2015, 20, 090503. [Google Scholar] [CrossRef] [Green Version]
Figure 1. In this study, we applied a Smartphone-based teledermatoscopy system with a smartphone (one of two smartphones applied, SP1, Nokia 6 v.2017) and the DL1 basic dermatoscope for colorimetric measurements in canine skin.
Figure 1. In this study, we applied a Smartphone-based teledermatoscopy system with a smartphone (one of two smartphones applied, SP1, Nokia 6 v.2017) and the DL1 basic dermatoscope for colorimetric measurements in canine skin.
Sensors 20 06234 g001
Figure 2. ColorChecker Classic (left, full-range CCT) and Digital SG (right, skin-range CCT). Skin patches used in our study are placed inside the red square.
Figure 2. ColorChecker Classic (left, full-range CCT) and Digital SG (right, skin-range CCT). Skin patches used in our study are placed inside the red square.
Sensors 20 06234 g002
Figure 3. Flowcharts of different color calibration procedures. (a) Image sets (Ifull/skin1–6) of the selected CCT color patches were acquired and normalized against patches 19 (full-range CCT) or E5 (skin-range CCT). For each set independently, we built a transformation model between an unknown RGB and a standard CIELAB color space (LAB = f(RGB)). (b) Full normalization. RGB values of I2–6 were first transformed to the RGB color space of I1 and later to the CIELAB based on the known f1. (c) Single-point normalization. Normalized images (I2–6) with the brightest patch (No. 19) were directly transformed to the CIELAB color space based on the known f1. (d) Erythema Index (EI) was calculated after the colorimetric data was transformed to the standard RGB color space (sRGB).
Figure 3. Flowcharts of different color calibration procedures. (a) Image sets (Ifull/skin1–6) of the selected CCT color patches were acquired and normalized against patches 19 (full-range CCT) or E5 (skin-range CCT). For each set independently, we built a transformation model between an unknown RGB and a standard CIELAB color space (LAB = f(RGB)). (b) Full normalization. RGB values of I2–6 were first transformed to the RGB color space of I1 and later to the CIELAB based on the known f1. (c) Single-point normalization. Normalized images (I2–6) with the brightest patch (No. 19) were directly transformed to the CIELAB color space based on the known f1. (d) Erythema Index (EI) was calculated after the colorimetric data was transformed to the standard RGB color space (sRGB).
Sensors 20 06234 g003
Figure 4. Mean CIE76 color differences (ΔE) with standard deviations as error bars for the full-range or skin-range CCT, obtained from the six image sets which were acquired by the first (SP1, Nokia) or second (SP2, Samsung) smartphone, respectively. SP1–SP2 marks the color differences between both smartphones’ estimates.
Figure 4. Mean CIE76 color differences (ΔE) with standard deviations as error bars for the full-range or skin-range CCT, obtained from the six image sets which were acquired by the first (SP1, Nokia) or second (SP2, Samsung) smartphone, respectively. SP1–SP2 marks the color differences between both smartphones’ estimates.
Sensors 20 06234 g004
Figure 5. Changes in CIE76 color difference (ΔE) due to the modified color calibration procedures: increased exposure time reaching the sensor’ capacity limit (Exp. Time), full (Full Norm.), and single-point normalization (Single-P Norm.). A negative change marks the improvement in colorimetric accuracy. Boxplots: the central mark represents the median; the bottom and top box edges indicate the 25th and 75th percentiles, respectively. Whiskers extend to the most extreme data point.
Figure 5. Changes in CIE76 color difference (ΔE) due to the modified color calibration procedures: increased exposure time reaching the sensor’ capacity limit (Exp. Time), full (Full Norm.), and single-point normalization (Single-P Norm.). A negative change marks the improvement in colorimetric accuracy. Boxplots: the central mark represents the median; the bottom and top box edges indicate the 25th and 75th percentiles, respectively. Whiskers extend to the most extreme data point.
Sensors 20 06234 g005
Figure 6. Clinical measurement of canine skin color. (a) Canine and human (full-range CCT: bold x-es, skin-range CCT: boxplots) skin color in the CIELAB color space. (b) Color differences (ΔE) in skin color between both smartphones’ estimates (SP1–SP2), between the original and repeated measurement (Repeated), and when a single-point normalization was applied (Single-P Norm.). Boxplots: the central mark represents the median; the bottom and top box edges indicate the 25th and 75th percentiles, respectively. Whiskers extend to the most extreme data point. The outlier is marked as a plus (+). († retrieved from the CCT specifications).
Figure 6. Clinical measurement of canine skin color. (a) Canine and human (full-range CCT: bold x-es, skin-range CCT: boxplots) skin color in the CIELAB color space. (b) Color differences (ΔE) in skin color between both smartphones’ estimates (SP1–SP2), between the original and repeated measurement (Repeated), and when a single-point normalization was applied (Single-P Norm.). Boxplots: the central mark represents the median; the bottom and top box edges indicate the 25th and 75th percentiles, respectively. Whiskers extend to the most extreme data point. The outlier is marked as a plus (+). († retrieved from the CCT specifications).
Sensors 20 06234 g006
Figure 7. Absolute differences (ΔRGB) between the acquired and normalized referential RGB values (sRGB) comparing (a) both smartphones (SP1, SP2), (b) shorter (full-range CCT’s patch 19) and longer (patch 20) exposure times, and (c) separate SP1′s RGB channels. Units a.u. are retrieved from the normalization against the white standard (i.e., patches 19 or 20).
Figure 7. Absolute differences (ΔRGB) between the acquired and normalized referential RGB values (sRGB) comparing (a) both smartphones (SP1, SP2), (b) shorter (full-range CCT’s patch 19) and longer (patch 20) exposure times, and (c) separate SP1′s RGB channels. Units a.u. are retrieved from the normalization against the white standard (i.e., patches 19 or 20).
Sensors 20 06234 g007
Figure 8. Bland-Altman plots on erythema index agreement between both smartphones. EIBRG is based on (a) the full-range CCT patches (No: 1–3, 5, 8, 9, and 19–24) or (b) canine skin. Mean difference and 95% limits of agreement (±1.96 STD) are marked as full and dotted lines, respectively.
Figure 8. Bland-Altman plots on erythema index agreement between both smartphones. EIBRG is based on (a) the full-range CCT patches (No: 1–3, 5, 8, 9, and 19–24) or (b) canine skin. Mean difference and 95% limits of agreement (±1.96 STD) are marked as full and dotted lines, respectively.
Sensors 20 06234 g008
Table 1. The color difference (ΔE) means and the whole ranges for individual patches (in brackets) on the full- and skin-range CCTs for the first (SP1) and second (SP2) smartphone and between both smartphones (SP1-SP2, i.e., inter-smartphone variability).
Table 1. The color difference (ΔE) means and the whole ranges for individual patches (in brackets) on the full- and skin-range CCTs for the first (SP1) and second (SP2) smartphone and between both smartphones (SP1-SP2, i.e., inter-smartphone variability).
Color CalibrationSP1SP2SP1–SP2
Full-range CCT (CIE76)6.45 (2.52–13.19)6.60 (1.66–12.80)3.91 (0.36–9.30)
Full-range CCT (CIE94)3.96 (1.51–9.25)4.46 (1.02–9.75)2.60 (0.36–7.28)
Skin-range CCT (CIE76)3.19 (0.93-9.04)1.76 (0.47–3.65)2.01 (0.34–7.18)
Skin-range CCT (CIE94)1.74 (0.40–3.64)1.16 (0.25–3.10)1.05 (0.13–3.08)
Table 2. Absolute differences in EIBRG between CCT’s referential data and both smartphones’ estimates (SP1, SP2) with different EI color calibration procedures. 1: EI was calculated directly from the device-dependent SP1′s and SP2′s RGB color spaces, 2: Colorimetric data was transformed to the standard RGB color space (sRGB) by the full normalization on six grayscale patches, 3 and 4: Colorimetric data was transformed to the standard RGB (sRGB) color space by the single-point normalization on the brightest white CCT patch (No. 19).
Table 2. Absolute differences in EIBRG between CCT’s referential data and both smartphones’ estimates (SP1, SP2) with different EI color calibration procedures. 1: EI was calculated directly from the device-dependent SP1′s and SP2′s RGB color spaces, 2: Colorimetric data was transformed to the standard RGB color space (sRGB) by the full normalization on six grayscale patches, 3 and 4: Colorimetric data was transformed to the standard RGB (sRGB) color space by the single-point normalization on the brightest white CCT patch (No. 19).
Color CalibrationCCT–SP1CCT–SP2SP1–SP2
1. Without (RGB) L0.40 ± 0.930.60 ± 1.160.29 ± 0.40
2. Full Norm. (sRGB) L0.04 ± 0.060.04 ± 0.090.03 ± 0.04
3. Single-P Norm. (sRGB) L0.04 ± 0.060.04 ± 0.100.03 ± 0.04
4. Single-P Norm. (sRGB) C//0.02 ± 0.01
L Laboratory settings with the full-range CCT. C Clinical skin measurements.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Cugmas, B.; Štruc, E. Accuracy of an Affordable Smartphone-Based Teledermoscopy System for Color Measurements in Canine Skin. Sensors 2020, 20, 6234. https://doi.org/10.3390/s20216234

AMA Style

Cugmas B, Štruc E. Accuracy of an Affordable Smartphone-Based Teledermoscopy System for Color Measurements in Canine Skin. Sensors. 2020; 20(21):6234. https://doi.org/10.3390/s20216234

Chicago/Turabian Style

Cugmas, Blaž, and Eva Štruc. 2020. "Accuracy of an Affordable Smartphone-Based Teledermoscopy System for Color Measurements in Canine Skin" Sensors 20, no. 21: 6234. https://doi.org/10.3390/s20216234

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop