Performance of Automated Oral Cancer Screening Algorithm in Tobacco Users vs. Non-Tobacco Users

Featured Application

Development of “smart” adjunct oral cancer screening approaches.

Abstract

Oral non-neoplastic and neoplastic lesions have similar clinical manifestations, increasing the risk of inaccurate screening decisions that adversely affect oral cancer (OC) outcomes. Tobacco-use-related changes in the oral soft tissues may affect the accuracy of “smart” oral screening modalities. Because smoking is such a strong predictor of OC risk, it may overwhelm the impact of other variables on algorithm performance. The objective was to evaluate the screening accuracy in tobacco users vs. non-users of a previously developed prototype smartphone and machine-learning algorithm-based oral health screening modality. 318 subjects with healthy mucosa or oral lesions were allocated into either a “tobacco smoker” group or a “tobacco non-smoker” group. Next, intraoral autofluorescence (AFI) and polarized white light images (pWLI), risk factors as well as clinical signs and symptoms were recorded using the prototype screening platform. OC risk status as determined by the algorithm was compared with OC risk evaluation by an oral medicine specialist (gold standard). The screening platform achieved 80.0% sensitivity, 87.5% specificity, 83.67% agreement with specialist screening outcome in tobacco smokers, and 62.1% sensitivity, 82.9% specificity, 73.1% agreement with specialist screening outcome in non-smokers. Tobacco use should be carefully weighted as a variable in the architecture of any imaging-based screening algorithm for OC risk.

1. Introduction

Five-year survival rate in the U.S.A. for oral cancer (OC) approximates 62% and has not improved over decades despite considerable progress in the outcomes of all other major cancers [1]. Additionally, 3–84% of the U.S. population manifests oral potentially premalignant lesions (OPMLs) [2,3], which carry a risk of malignant transformation of up to 10% per year [4]. The poor survival rate for OC is mainly due to late diagnosis, leading to late treatment onset, reduced treatment options, and greater treatment morbidity and mortality [5]. The primary determinant affecting survival is the cancer’s stage at diagnosis [5,6,7,8,9,10]. As more than 2/3 of OC lesions are detected late, after the onset of metastasis, treatment outcomes and prognoses are extremely poor [1,11].

A study in 200,000 individuals found that carefully executed screening for OC in tobacco users resulted in considerably earlier OC detection and a significantly lower rate of OC deaths [12]. However, a preponderance of other studies has reported low screening accuracy, and consequently little or no impact by screening on OC outcomes within the target populations [13,14]. OPMLs and OC are typically asymptomatic until they reach an advanced stage, so that patients do not experience symptoms that might motivate them to seek professional diagnosis and treatment [1,11]. Thus, effective, and regular screening in at-risk individuals is essential. Nevertheless, because OPMLs and early stage OC can have a similar appearance as many other intraoral lesions, they are easily overlooked or misdiagnosed during non-specialist screening. This is especially in tobacco-users, where 60–80% manifest confounding tobacco use-related soft tissue changes [15]. Because tobacco use commonly changes the appearance of the oral soft tissues, and because this habit itself is considered the most important predictor for OC risk [2,16,17,18,19,20,21,22,23,24,25], it is important that any imaging-based, and/or algorithmic approaches to evaluating OC risk are able to correctly and appropriately evaluate the relevance of each variable to OC risk. This process is essential to ensure that each variable is assigned the appropriate significance or “weighting” in the architecture of the analytical process. To the best of our knowledge, this important matter remains to be addressed.

In low-resource environments, 80–90% of the population has no ready access to oral medicine or otolaryngology specialists skilled at diagnosing OC [26,27,28]. Specialist access and compliance with specialist referral are dependent on factors such as physical proximity of a specialist center, the availability and cost of transportation, child- and eldercare duties, language, cultural, financial, and social barriers, fear, and unfamiliarity with specialist and academic centers. Thus, there exists a strong impetus to develop and upgrade community-based OC screening programs to improve screening and referral accuracy, and to establish new pathways for overcoming barriers to referral compliance. In tobacco users, the oral mucosa commonly manifests a panoply of visible mucosal changes which may impact screening accuracy [15,29]. Because the accuracy of non-specialist screenings is low (30–60%) [26,30,31], providing additional non-specialist screening manpower will have little impact on OC outcomes unless tools are developed that improve screening accuracy. The primary barrier to better OC outcomes is the lack of an effective screening tool at the local point-of-care level.

A range of adjunctive screening tools to support effective OC screening in low resource and community settings is under investigation [32,33,34,35,36,37]. The accuracy and usefulness of these approaches is predicated on many variables, depending on the specific imaging modality and the applications under consideration [37,38,39,40,41]. Barriers to the adoption and the effectiveness of such approaches, especially in community-based and low-resource settings, are considerable. Perhaps at a very basic level there are the usual limitations defined by a community and low-resource environment: tools need to be low-cost, portable, simple to use, and easy to understand even for the occasional and inexperienced user. Outcomes measures (such as “no risk”, “low risk”, “high risk”) should translate directly into improved clinical decision-making, and categorical cutoffs should be clearly defined, and outcome based. Moreover, measurements and outcomes should be consistently reproducible, and longitudinal evaluation of changes (for example, in lesion risk levels) should be possible. To improve the accuracy of adjunct screening tools, researchers are pairing emerging optical imaging modalities with novel artificial intelligence-based approaches to provide “smart”, inexpensive, robust, non-specialist chairside screening platforms [42,43,44]. Artificial intelligence approaches combined with imaging can have considerable impact on OC outcomes, with modalities ranging from low-cost screening with smartphone-based probes to algorithm-guided detection of oral lesion heterogeneity and margins using optical coherence tomography [42,45,46,47,48].

Our long-term goal is to improve OC outcomes by developing a smartphone-based oral screening platform for non-specialist use that informs on OC risk levels based on multimodal intraoral imaging, clinical signs and symptoms, and risk factors. The specific goal of this study was to evaluate the effect of tobacco use on the accuracy of an imaging-and risk factor-based screening algorithm for OC that is being developed in our laboratories.

2. Materials and Methods

This clinical study was designed as a prospective study. It was approved by the University of California Irvine’s IRB protocol #2002-2805 and all clinical procedures were conducted in accordance with the Helsinki Declaration of 1975, as updated in 2013.27. No significant changes were made in the study design and execution after commencement of the study. Written informed consent was obtained from all subjects involved in the study, all of whom completed the study in full compliance with the approved protocol.

2.1. Overview

A total of 2198 images of healthy oral mucosa and oral lesions suspicious for OPML or OC were collected from 318 subjects, together with a full documentation of any clinical signs and symptoms as well as an evaluation of all individual risk factors and behaviors. Images and data were recorded using a prototype intraoral camera system and algorithm specifically designed for non-specialists working in a low resource community setting. The accuracy of OC risk status as determined by the screening platform was compared in tobacco smokers vs non-smokers. Specialist screening outcome served as gold standard for evaluating the platform’s screening accuracy.

2.2. Subjects

This study was performed in 318 subjects from three clinical sites. Individuals attending Concorde College of Dental Hygiene Dental Clinics in Garden Grove, California, or West Coast University Dental Hygiene Clinics in Anaheim, California, as well as those referred to the University of California, Irvine Clinics with oral lesions suspicious of OPML or OC were recruited, as well as individuals with no visible oral lesions. Informed consent was obtained from all subjects prior to study begin. Study participants were first divided into “smoker” and “non-smoker” groups. They were classified as tobacco smokers if they self-reported consuming 10 or more cigarettes per day and 70 cigarettes per week over the past year or longer. Study participants were classified as tobacco non-smokers if they had not smoked any tobacco within the past year or more. Light smokers (less than 10 cigarettes per day within the past year), and individuals who had indulged in any form of vaping, cigar smoking, chewing tobacco or similar, snuff, or water pipe/hookah use within the past year were excluded from the study, as were individuals who self-reported consuming more than two glasses of beer or wine weekly, or who regularly imbibed any stronger forms of alcoholic beverages. These criteria were established to avoid some of the most common multifactorial causes of increased OC risk and of visible changes to the oral mucosa. For example, individuals who regularly consumed any more than minimal amounts of alcohol were excluded from the study, as this behavior potentiates OC risk in tobacco smokers, and in this study our goal was to single out tobacco use as a variable in the algorithm architecture. Inclusion/exclusion criteria are shown in

Table 1. Recruitment inclusion/exclusion criteria.

Eligibility Criteria
Inclusion Criteria	Exclusion Criteria
- Minimum 18 years of age - Able to understand and carry out instructions - Normal oral tissue or oral lesions based on visual inspection by dentist - Tobacco non-smoker or tobacco smoker consuming >10 cigarettes weekly over >1 year	- Vaping, chewing tobacco or similar, snuff, water pipe/hookah use within the past 6 months. - Smoking 1–9 cigarettes weekly - Consuming >2 servings of alcohol weekly - Previous treatment for an oral lesion - History of malignancy, diabetes, autoimmune disorders, hepatitis, or HIV infection - Severe, advanced periodontal disease

.

2.3. Protocol

Dental Hygiene students at Concorde College of Dental Hygiene Dental Clinics in Garden Grove, California, and West Coast University Dental Hygiene Clinics in Anaheim, California and Biomedical Science students at the University of California, Irvine, recorded multimodality image sets, clinical signs and symptoms, and risk factor data from each subject using the prototype screening platform. These students were selected to represent community health workers, who are the intended future users of the screening platform. At each study site, the students all received one day of classroom teaching by the same instructor, where they learned about OPML and OC causes, risk factors, and pathology, as well as its clinical manifestations, prognosis, treatment, and outcomes. Next, directly prior to study begin, the students all attended a ½ day of clinical training at their respective school, which was taught by the same instructor at all centers. During this clinic, the students learned to operate the screening platform’s prototype scanner pen and App. Throughout the study duration, the students recorded study participants multimodality images, risk factors, clinical signs and symptoms with the scanner pen and App and they then documented the ensuing algorithm-based screening outcome as either “no increased OC risk” or “increased OC risk”. Imaging sites were selected in each subject as follows: in all subjects with lesions, all soft tissue areas manifesting any kind of visual changes were imaged. Then, where available, contralateral lesion-free tissues were also imaged in the same subject. In healthy subjects, 12 intraoral sites were imaged, including buccal and vestibular mucosa, dorsal, ventral, and lateral surfaces of tongue, floor of mouth, palate, soft palate, and tonsillar regions. Finally, one oral medicine specialist performed a full standard of care OC screening and she subsequently recorded a screening outcome as either “no increased risk” or “increased risk” for each study participant (Figure 1). During this project the same oral medicine specialist performed all screening in all participants at all three clinical sites according to the standard of care, combining clinical examination with risk factors and patient history. The specialist screening outcome served as the gold standard for evaluating screening platform accuracy. Finally, all study participants with increased OC risk according to the specialist screening were informed of the screening outcome. They were then referred to a specialist for diagnosis and entry into the pathway of care.

2.4. Intraoral Camera Platform

In this study, a previously developed prototype OC risk screening platform was used. Extensive details of the screening device are provided in our previous papers [49,50,51]. This platform was specifically developed for use in low-resource settings. It is very low cost, very easy to operate, robust, can be charged by solar power, and all data collection and storage is located in a HIPAA-compliant cloud-based repository. A simple disposable sheath allows for immediate, on-site infection control. In this study, a simple, low-cost Android smartphone was used, but the entire platform is freely transferrable to almost any basic smartphone, to ensure usability in a wide range of low-resource settings. Briefly, the prototype OC risk screening platform consists of a commercial Moto G5 Android smartphone, an intraoral imaging probe, a light-emitting diode (LED) driver, a rechargeable lithium battery, and our prototype mobile application. Intraoral imaging is performed by means of a multimodality autofluorescence imaging (AFI) and polarized white light imaging (pWLI) probe or scanner pen that is connected to the smartphone (Figure 2 and Figure 3). The system utilizes four 405-nm Luxeon UV U1 LEDs (Lumileds, Amsterdam, Netherlands) to enable intraoral autofluorescence imaging and four 4000-K Luxeon Z ES LEDs for intraoral white-light imaging. The illumination LEDs are driven by a switching boost voltage regulator (LT1815, Linear Technology, Milpitas, CA, USA). A custom Android application drives the simple user interface, which has drop-down menus to make all functions as simple and intuitive as possible for the user. The screen-front interface controls the phone and probe, captures dual-modal intraoral images, and implements our oral image classification approach. The intraoral probe simultaneously captures dual-modal intraoral images and is connected to the smartphone to record and classify the images. Figure 2 shows the intraoral camera and its flexible tip to improve intraoral access, which uniquely allows the screener to record clear, in-focus images from all areas of the oral cavity, including the tonsillar pillars, the soft palate, and the base of the mouth. Figure 3 shows the intraoral screening platform during use in a community setting.

2.5. Statistical Analysis

Sensitivity, specificity, agreement, false positive rate, false negative rate, and positive and negative predictive values were estimated from the observed results. Standard errors (SE) and 95% confidence intervals were calculated for all the rates. A level of p < 0.05 was used to achieve statistical significance.

3. Results

All 318 subjects enrolled in the study participated for its entire duration and in full compliance with the IRB protocol.

Table 2. Subject demographics.

Identifying Gender	Race/Ethnicity	Age
137 Female	124 White/Hispanic	Range: 37–89 years of age
181 Male	91 White/Non-Hispanic	Mean: 54 years of age
	66 Asian	Median: 41 years of age
	19 more than one race
	14 African American
	4 Pacific Islander

shows a breakdown of subject demographics, of whom 247 self-identified as non-smokers, and 71 as smokers. About 1099 pairs (AFI and pWLI) of intraoral images, or a total of 2198 individual images were collected. These image pairs were categorized as follows: Category (A): “no increased risk” contained 786 image pairs and category (B): “increased risk” contained 313 image pairs. The smoking group contained 119 “no increased risk” image pairs and 126 “increased risk” image pairs. The non-smoking group contained 667 “no increased risk” image pairs and 187 “increased risk” image pairs. All of these categorizations were based on the standard-of-care specialist screening outcome.

All of the algorithm training was completed using a VGG19 model pre-trained with ImageNet. The experiments were implemented in the python platform using Tensorflow and Keras tools. For each training pathway, the dataset was randomly split into training and validation to perform fourfold cross-validation, and the results from the folds were averaged. The batch size was 32, the learning rate was 0.001, the epoch number was 300, and Adam optimizer was used for each experiment. Since the non-smoking group is imbalanced, we randomly over sampled (ROS) the minority class identified as carrying “increased risk” using data augmentation, and randomly under sampled (RUS) the majority class that was identified as having “no increased risk”. After ROS and RUS, the dataset for training and cross-validation contained 374 “no increased risk” image pairs and 374 “increased risk” image pairs. Our previous study has shown the class bias induced by an imbalanced OC image dataset can be reduced using these data-level approaches [52]. For dual-modal image (white light and autofluorescence images) data input, we used an image fusion method reported in a previous study [34], which created a new three-channel image dataset from autofluorescence and white light images and this was then fed into the network for training.

A total of 1099 pairs (pWLI and AFI) of intraoral images were collected from the 318 study subjects (Figure 4 and Figure 5). Figure 4 shows image pairs of typical erythematous lesions in the anterior tonsillar pillar region obtained from a smoker and a non-smoker. In the pWLI, the smoker’s tissues appear hyperkeratinised, thickened, and fibrous (Figure 4i(a)), which is typical of smoking-related changes in the oral soft tissues. These changes are also clearly visible in the AFI of the smoker (Figure 4i(b)). Because thickening and whitening of the soft tissues can also be an indicator of OPML or OC, there is the potential for a confluence of smoking-related and neoplasia-related mucosal changes in these individuals, as shown in these images. In contrast, the same type of lesion in the non-smoker appears more moist and considerably less fibrosed, less white, and less thick in the pWLI and the AF images (Figure 4ii(a) and Figure 4ii(b)). Figure 5 shows image pairs of papillary hyperplasia from a smoker ((Figure 5i), and a non-smoker ((Figure 5ii)). The thick, red, dry looking fibrosed surface layer in the smoker’s gingival tissues extends across the entire visible gingival surface, with a quite uniform white color, coupled with increased bulk across the entire spread of the tissues, from the gingival margin extending up to the mucosa (Figure 5i(a)). In the non-smoker, there is an evident localization of change in color and texture, as well as swelling, which appears inflammatory rather than fibrosed in nature (Figure 5ii(a)). The smoker’s tissues evidence considerably more backscattering in the AFI (Figure 5i(b)) vs. the non-smoker (Figure 5ii(b)).

Table 3. Screening performance algorithm vs. specialist in non-tobacco smoker vs tobacco Smoker.

		VALUE	SE	LOWER CI	UPPER CI
Non-Smoker	Sensitivity	0.622	0.022	0.600	0.644
	Specificity	0.835	0.023	0.812	0.858
	False Positive Rate	0.165	0.024	0.142	0.188
	False Negative Rate	0.378	0.022	0.356	0.400
	Agreement with Specialist	0.733	0.010	0.726	0.742
	Positive predictive values	0.774	0.019	0.755	0.793
	Negative predictive values	0.710	0.012	0.700	0.720
Tobacco-Smoker	Sensitivity	0.800	0.033	0.768	0.832
	Specificity	0.864	0.041	0.824	0.904
	False Positive Rate	0.135	0.040	0.0959	0.174
	False Negative Rate	0.200	0.033	0.168	0.232
	Agreement with Specialist	0.833	0.026	0.807	0.858
	Positive predictive values	0.861	0.037	0.825	0.897
	Negative predictive values	0.805	0.026	0.780	0.830

summarizes the comparison of the screening algorithm performance vs. the oral medicine specialist. The algorithm achieved 80.0% sensitivity, 86.4% specificity, and 83.3% agreement with the specialist screening outcome in tobacco smokers, and 62.2% sensitivity, 83.5% specificity, and 73.3% agreement with the specialist screening outcome in non-smokers. For comparison purposes, we also evaluated the algorithm screening outcomes for all subjects combined, achieving an overall sensitivity of 71.0%, a specificity of 75.4%, and an agreement of 73.2% with the oral medicine specialist screening outcome for the two groups combined.

4. Discussion

The primary goal of this study was to gain a better understanding of the significance of tobacco smoking in the performance of an imaging- and risk-factor-based screening algorithm for OC risk. To date, the algorithms that are under development all use the same algorithm pathway and determinants in all subjects undergoing OC risk screening, regardless of the extrinsic factors which may mask or modify cancer risk-related changes in the visual appearance of the oral soft tissues. Our concerns about the impact of the obvious visual changes to the appearance and texture of the oral mucosa of tobacco smokers provided impetus for this study. Because our prototype screening algorithm combines image-based evaluations with clinical signs and symptoms and risk factors, we were concerned that smoking-related mucosal changes might reduce the accuracy of the screening algorithm overall. However, the study findings did not support this premise, actually demonstrating greater algorithm-based screening accuracy in smokers than in non-smokers.

A second impetus for this study was the entire topic of how to address “weighting” each individual variable in the screening algorithm. For example, how much importance (“weight”) does the algorithm attribute to tobacco use vs. alcohol use as an indicator of oral risk, and how should the algorithm-based classification of risk levels be modified when a subject indulges in both behaviors, which potentiate each other to a degree that is not just additive? It is only if each variable is assigned the appropriate level of significance within the algorithm architecture that the maximum amount of information can be mined from the data and the images, to achieve the most accurate screening outcome possible. Because tobacco smoking is such a powerful predictor of OC risk, an optimized screening algorithm will need to reflect an appropriate “weighting” of this risk factor without overwhelming the additional significance of other risk factors. From the results of this first study, it is apparent that our OC risk screening algorithm achieved considerably higher accuracy and agreement with the specialist screening outcome in the smoking group vs the non-smoking group. These findings suggest that the weighting of tobacco smoking as a variable within our first-generation algorithm needs to be fine-tuned to prevent this variable from canceling out useful information obtained from other variables. The current level of weighting for tobacco use is clearly too high, whereas the weighting of the imaging-based criteria needs to be increased. Similarly, future work is needed to allocate the appropriate level of significance to other non-tobacco-smoking-related risk factors such as ulceration, texture, and color change within the oral cavity, high-risk behaviors, or a concerning health history. The finding from this study that the screening accuracy of our algorithm is currently lower in non-tobacco users than in smokers implies that the importance attributed to each non-tobacco-use-related finding vs. tobacco use per se within the algorithm architecture needs to be re-visited and fine-tuned. Other planned additions and improvements to the algorithm will also need to address interaction and potentiation mechanisms between different risk factors and behaviors that relate to OC risk, such as tobacco smoking with alcohol consumption. In order to avoid the known potentiating effect on OC risk of regular alcohol use, subjects who consumed more than a minimal amount of alcohol were excluded from this study, and we will investigate this important topic in an upcoming study.

Overall, the findings of this study indicate that the next steps in refining the screening algorithm could be based on a multi-layered approach that first addresses tobacco use and potential potentiators of its effects such as alcohol, then incorporates other variables such as diet, lifestyle, health history, as well as an imaging-based analysis of the oral tissues. These next steps will bring us closer to a tailored non-specialist tool for oral cancer risk screening that should help to improve the accuracy of screening outcomes and resultant specialist referrals, while reducing unnecessary referrals.

5. Conclusions

Our long-term goal is to improve OC outcomes in remote and low-resource individuals through earlier and more accurate evaluation of OC risk to allow earlier and more accurate specialist referral, diagnosis, and treatment. In this study, a prototype low cost, simple “smart” screening platform designed specifically for use by the non-specialists who typically screen and surveil the underserved individuals who carry the greatest OC risk performed with greater accuracy in smokers than in non-smokers. Thus, next steps must include a re-calibration and fine-tuning of the algorithm architecture to better reflect the risk levels posed by each evaluation criterion.