1. Introduction
Barrett’s esophagus (BE) is characterized by the replacement of the normal squamous epithelial cells lining the esophagus with intestinal metaplastic tissue, resulting in a visibly reddened and thickened appearance. Gastrointestinal reflux disease (GERD), marked by the retrograde flow of stomach acid into the esophagus, leading to chronic inflammation and irritation, is a major risk factor for BE [
1,
2,
3]. BE is most commonly diagnosed in patients over 50, with an average diagnosis at 55 years, and exhibits a higher occurrence among males compared to females [
1,
4,
5,
6]. Other risk factors include smoking, alcohol use, obesity, and family history [
7,
8,
9,
10,
11].
Patients diagnosed with BE are at a 10–55× risk of developing esophageal adenocarcinoma (EAC), which is among the most lethal cancers, with a 21% 5-year survival rate [
1,
6,
12,
13]. Due to the increased risk and the lethality of EAC, the American College of Gastroenterology guidelines suggest that patients with BE be enrolled into endoscopic surveillance programs with the goal of detecting dysplasia or early cancer, both of which can be treated effectively with endoscopic eradication therapy (EET) [
14]. However, the detection of dysplasia is fraught with challenges, including interobserver variability and subjectivity, establishing the need for better risk stratification tools [
15,
16,
17].
Methylation biomarkers have been validated as methods for earlier detection and prognostication of cancers and their precursor conditions, including BE [
18,
19,
20,
21,
22]. In background studies performed at Johns Hopkins University and validation studies at Previse, four specific methylation markers (
p16,
RUNX3,
HPP1, and
FBN1) have demonstrated consistent statistically significant performance as early harbingers of neoplastic progression in BE, presenting an opportunity for improved prognostication in BE patients [
23,
24,
25,
26].
Esopredict
TM is a high-complexity, laboratory-developed test (LDT) developed at Previse (Baltimore, MD; CLIA #21D2256153). This tissue test (FFPE) is indicated for BE patients with non-dysplastic BE (NDBE), indefinite for dysplasia (IND), or low-grade dysplasia (LGD) to predict future progression to high-grade dysplasia (HGD) or esophageal adenocarcinoma (EAC). This epigenetic test, Esopredict
TM, has been clinically validated in two independent studies: a 110-patient study conducted across six clinical sites, and 224 samples across 80 patients in a spatial/temporal cohort [
25]. These studies have demonstrated the robustness of Esopredict
TM’s performance to identify a subset of patients who are at either higher or lower risk than average of advancing to HGD or EAC to help guide treatment decisions by physicians through either increased surveillance or decreased surveillance, respectively.
While clinical validation is vital to establish the accuracy of the test with respect to a clinical outcome, analytical validation is equally essential to establish the accuracy and reliability in measuring the signal across analytical conditions. The CLIA ’88 guidelines and Clinical Laboratory Standards Institute (CLSI) provide well-accepted guidance for analytical validation for nucleic acid assays, including reproducibility and repeatability, sensitivity, and specificity. In this study, we aimed to demonstrate the robustness of EsopredictTM within acceptable ranges across multiple conditions (e.g., reagents, instruments, runs, operators), through a comprehensive analysis of samples from 26 patients with BE, cell lines, and methylated and unmethylated controls. We aim to show through rigorous analysis that EsopredictTM is an assay with high precision, accuracy, analytical sensitivity (as measured by the limit of detection (LOD)) and analytical specificity (as measured by the limit of blank (LOB)) for the determination of the risk of progression to HGD or EAC.
2. Materials and Methods
2.1. Test Overview
Esopredict
TM is a CLIA-developed LDT for high-complexity analysis of methylation markers in tissue from Barrett’s esophagus patients for predicting the risk of future progression to EAC or HGD within five years. The test was developed from a series of studies performed at the Johns Hopkins School of Medicine, Baltimore, MD, USA and validated in the CLIA Laboratory at Previse in Baltimore, MD, USA [
23,
24,
25]. DNA is extracted from FFPE tissue of esophageal origin in Barrett’s patients. Following bisulfite conversion, methylation-specific PCR (MSP) is performed to assess methylation levels of four genes (
HPP1,
p16,
RUNX3, and
FBN1) as well as a control gene
beta-actin. Normalized methylation values and age are used to generate a risk score and risk category (low vs. high) as well as the probability of progression within five years. The test is intended for patients > 18 years.
2.2. Validation Samples
A total of 26 samples were evaluated including FFPE patient specimens, cell lines, DNA and contrived controls (see
Appendix A). Tissue Samples: Sixteen Barrett’s esophagus patient samples were de-identified and submitted from collaborating sites. Serial sections were cut from formalin-fixed paraffin-embedded (FFPE) tissue blocks. Two additional esophageal tissue samples were purchased from Biochain Institute Inc., Newark, CA, USA (C509126, human normal esophagus) and Tissue Array, Derwood, MD, USA (HuCat041, human EAC esophagus).
Cell lines: Three EAC-derived cell lines from unique patients were obtained, including two human esophageal cancer cell lines from Sigma Aldrich Burlington, MA, USA (FLO-1, SK-GT-4) and a proprietary esophageal cell line, JH-EsoAd1, from Dr. Stephen J. Meltzer’s laboratory at the Johns Hopkins School of Medicine, Baltimore, MD, USA [
27].
DNAs: Fully methylated-100% methylated DNA, (HCT116 DKO Methylated DNA, D5014-2), and unmethylated DNA-0% methylated DNA (Human HCT116 DKO Non-Methylated DNA, D5014-1) were obtained from Zymo Research, Irvine, CA, USA. The 100% methylated sample was used as a positive assay control and the 0% methylated sample was used as a negative assay control for EsopredictTM.
Contrived samples were engineered for limit-of-detection (LOD) analysis using the fully methylated DNA (100% methylated) in a background unmethylated DNA (0% methylated) by mixing DNA from both methylated and unmethylated samples at compositions of 5%, 3%, and 1.5% methylated DNA. The fully unmethylated DNA was used as the limit of blank (LOB).
We note that the cell lines, DNA and contrived samples are not derived from FFPE, which is the source material for EsopredictTM. Given the difficulty in finding an unlimited FFPE source of sufficient quality and quantity for this study, we feel the cell lines and DNA provide an appropriate surrogate for assessing assay performance. Additionally, amplicons were designed to accommodate the fragmented nature of FFPE samples with amplicons < 200 base pairs. FFPE specimens were also selected across a range of Esoscores and DNA input levels.
Ethical statement for human subjects: Histological tissues from patients diagnosed with Barrett’s were prepared at the time of endoscopy and not collected for this study. Archival tissues were anonymized and de-identified. The study was exempted by the institutional ethics committee of Johns Hopkins University, Baltimore, MD, USA.
2.3. Test Method
DNA was extracted from anonymized FFPE tissue samples from BE patients using the Qiagen, Germantown, MD, USA QIAamp DNA FFPE Advanced Kit (56604). DNA was extracted from cell lines using the Qiagen DNeasy Blood and Tissue Kit (69504). DNA was quantified using the Qubit 4.0 and Qubit ds high-sensitivity kit (Q32854). For bisulfite treatment, 100–500 ng of DNA was used in the Zymo, Irvine, CA, USA Research EZ DNA Methylation-Lightning Kit (D5030T), which is within the range of the kit and where 100 ng is the minimal amount for EsopredictTM.
Methylation-specific PCR was performed on the Quantstudio 3 software version 1.5.2 (ThermoFisher, Carlsbad, CA, USA) using a set of 5 sets of primers and probes (IDT) for four methylation genes (
HPP1,
p16,
RUNX3, and
FBN1) as well as a control gene
beta-actin (see
Appendix B). All probes were labeled with a universal quencher and FAM. A five-point standard curve was made by dilution of a 100% methylated control from Zymo Research, Irvine, CA, USA (D5015). Fivefold dilutions were constructed in the range of 50 ng DNA to 0.08 ng DNA. Briefly, 20 µL reactions were set up consisting of 3–17 ng DNA, 0.5 µM forward and reverse primers, 0.25 µM probe, and TaqMan Fast Advanced Master Mix (no UNG) (ThermoFisher, Carlsbad, CA, USA, A44360). Cycling conditions were 95 °C 5 min, followed by 40 cycles of 95 °C 0:15, 60 °C 0:25, and 72 °C 0:30. Each sample was run in triplicate, and quantities were averaged. Samples where amplification was detected but below the assay limit of quantification or lower than the lowest standard of 0.08 ng DNA were imputed to LLOQ/2 (0.04 ng). Each gene was analyzed, and normalized methylation values (NMV) were obtained for all methylation genes by dividing by control gene beta-actin:
2.4. Data Analysis
Using the normalized methylation values combined with age at biopsy, a classification algorithm was developed to calculate an Esoscore on a scale of 0–100, which translates into four risk levels for progression to HGD or EAC within 5 years: low risk, low-moderate risk, high-moderate risk and high risk, as shown in
Table 1. In summary, the algorithm transforms NMV values for each gene and uses a linear regression model with age to determine the Esoscore. Given the high correlation between
HPP1 and
FBN1, an equally weighted average combines these two markers for analysis. The score for a BE patient with non-dysplastic BE (NDBE), indeterminate dysplasia (IND), or low-grade dysplasia (LGD) is translated into a probability of progressing to HGD or EAC within 5 years (see
Appendix C). Note: For Esopredict
TM calculations for contrived DNA samples and cell lines, we used a patient age of 55 years, which is the average age of diagnosis for Barrett’s esophagus [
4,
5,
6].
2.5. Assay Performance Characteristics
2.5.1. Limit of Blank (LOB)
LOB was determined by calculating the Esoscore for the unmethylated DNA sample (0% methylation). Seventy-five independent preps of the unmethylated sample were analyzed across 66 runs. The LOB was determined as the highest value likely to be observed within confidence limits for the unmethylated material. Non-template controls or water blanks were also analyzed in each assay but not used in LOB analysis since there was no amplification detected in any genes, including the control gene; therefore, no Esoscore can be determined. LOB was calculated using the formula:
2.5.2. Limit of Detection (LOD)
To determine the assay, LOD 100% methylated DNA was mixed in a background of unmethylated DNA at various low-level concentrations to determine the lowest analyte concentration to be reliably distinguished from the LOB. A 5%, 3%, and 1.5% methylated sample was analyzed using the assay minimum input of 100 ng DNA for the bisulfite reaction and 3.5 ng per qPCR. Contrived samples were analyzed across four runs with an average of 5 replicates per run. LOD was determined using the formula:
2.5.3. Accuracy, Precision (Reproducibility and Repeatability)
The average agreement between sample measurements was determined by repeating 18 BE patient samples five times across three runs (two runs in singlicate, one run in triplicate) on different days using different equipment, operators, and reagents. Due to the limitation of DNA extracted from FFPE samples and the quantification limitations of the assay, triplicates could only be performed in one run for patient samples. Cell lines were also run in triplicate across three independent runs to compensate for the lack of tissue material. Two of the three cell lines were analyzed in triplicate at the highest and lowest input levels (100 ng and 500 ng). As there is no ground truth or gold standard when comparing replicates, ANA (average negative agreement) and APA (average positive agreement) were calculated. APA is the number of higher-risk level matches as a proportion of the total number of higher-risk results observed, and ANA is the number of lower-risk level matches as a proportion of the total number of lower-risk results observed as calculated in
Table 2 [
28]:
We also include the distinction between the two main risk categories (low vs. high risk) as those distinguish populations with a probability of progressing to HGD or EAC less than the average BE population versus those who have a probability of progressing greater than the average BE population [
25] and represent potential reduced vs. increased esophagogastroduodenoscopy surveillance populations. During initial model development, risk levels were further refined to include four clinically relevant actionable risk levels, thereby creating a more substantial impact on decisions [
25].
Additionally, intra-assay and inter-assay precision were determined by calculating the coefficients of variation (CV) for the Esoscore for all sample types and patient samples and across all risk levels.
4. Discussion
Esophageal cancer is the second most lethal cancer in the United States with a 5-year survival of <21% and is amongst the most rapidly rising cancers today, with a more than 7-fold incidence increase over the past few decades [
2,
29,
30,
31,
32]. Thus, patients diagnosed with EAC’s precursor, Barrett’s esophagus, are enrolled in endoscopic surveillance programs. Historically, such risk stratification has been a challenge, with surveillance decisions primarily being determined by the presence of dysplasia (or lack thereof). However, there is high interobserver variability amongst pathologists in diagnosing dysplasia, and diagnosis is not a great predictor of future neoplastic progression [
15,
16,
17]. Further, in patients with non-dysplastic Barrett’s esophagus, from which 50% of esophageal adenocarcinoma cases arise, gastroenterologists lack the tools to identify which patients may or may not progress. For surveillance programs to be successful, methods of improving risk-stratification are needed to determine how often to administer endoscopy, and potentially, whom to administer endoscopic eradication therapy.
EsopredictTM is a proprietary epigenetic laboratory test to predict 5-year risk of progression to high-grade dysplasia or esophageal adenocarcinoma in patients with Barrett’s esophagus. This solution has been clinically validated, and its performance could lead to better treatment options such as more or less frequent surveillance, or increased surveillance or EET, based on risk categorization. It uses available biopsy material from endoscopic exams, so no additional biopsy material is needed from the patient and can be run using standard molecular laboratory equipment and techniques.
In this analytical validation study, we have demonstrated that Esopredict
TM has high analytical sensitivity, analytical specificity, and reproducibility for the intended population. Analytical specificity and sensitivity were measured by LOB and LOD analyses where as low as 1.5% methylation composition may be detected from a background of unmethylated DNA. Furthermore, 1–2% methylation differences may be detectable. Given that Esopredict
TM scores are also age-dependent, it is worth noting that very old patients (>95-year-old) will rarely have a risk level lower than high-moderate due to the LOD for that age. However, the average patient age is 55 years [
1,
4,
5,
6] and an older age group >95 years is unlikely to receive testing or surveillance. The test was also highly reproducible across a range of Esoscores that span all risk categories with an average CV < 10%. High reproducibility was also demonstrated by ANA (>90%) and APA (>88%) analysis. As Esopredict
TM is reported as a continuous result with an associated probability of HGD or EAC and is not reported solely as dichotomous (e.g., positive or negative), it is not surprising that samples near the cutoff in the intermediate range (low-intermediate and high-intermediate) will shift upon repeat testing. It is also worth noting that all repeats in the lowest risk category scored low risk (i.e., risk levels low or low-moderate) and all repeats in the highest risk category scored high risk (i.e., risk levels high-moderate or high). This analytical validation study demonstrates that Esopredict
TM is a robust and reproducible assay.
5. Conclusions
We demonstrate that the prognostic test EsopredictTM performs to the expectations of such accrediting bodies as CMS CLIA with high repeatability and reproducibility and with high analytical sensitivity and specificity, as shown by the data presented here. This assay shows a limit of detection of 1.5% methylation in samples with a minimal DNA input of 100 ng. Overall inter and intra-assay precision for Esoscores demonstrated high reproducibility over a range of risk levels CV < 9% and <5%, respectively, and are within recommendations for clinical assays according to CLIA/CLSI guidelines (CV < 20%). Inter and intra-assay average positive agreement (APA) demonstrates high concordance of 88% and 94%, respectively, in addition to average negative agreement (ANA) values of 90% and 94%, respectively, for distinguishing low-risk from high-risk populations.