Performance Characteristics of Oncomine Focus Assay for Theranostic Analysis of Solid Tumors, A (21-Months) Real-Life Study

Abstract

Next generation sequencing analysis is crucial for therapeutic decision in various solid tumor contexts. The sequencing method must remain accurate and robust throughout the instrument lifespan allowing the biological validation of patients’ results. This study aims to evaluate the long-term sequencing performances of the Oncomine Focus assay kit allowing theranostic DNA and RNA variants detection on the Ion S5XL instrument. We evaluated the sequencing performances of 73 consecutive chips over a 21-month period and detailed the sequencing data obtained from both quality controls and clinical samples. The metrics describing sequencing quality remained stable throughout the study. We showed that an average of 11 × 10⁶ (±0.3 × 10⁶) reads were obtained using a 520 chip leading to an average of 6.0 × 10⁵ (±2.6 × 10⁵) mapped reads per sample. Of 400 consecutive samples, 95.8 ± 16% of amplicons reached the depth threshold of 500X. Slight modifications of the bioinformatics workflow improved DNA analytical sensitivity and allowed the systematic detection of expected SNV, indel, CNV, and RNA alterations in quality controls samples. The low inter-run variability of DNA and RNA—even at low variant allelic fraction, amplification factor, or reads counts—indicated that our method was adapted to clinical practice. The analysis of 429 clinical DNA samples indicated that the modified bioinformatics workflow allowed detection of 353 DNA variants and 88 gene amplifications. RNA analysis of 55 clinical samples revealed 7 alterations. This is the first study showing the long-term robustness of the Oncomine Focus assay in clinical routine practice.

1. Introduction

Theranostic analysis of solid tumors gains continuous complexity requiring the search for point mutations, short insertions or deletions, copy number variations, or gene fusions in always larger panels of genes [1,2]. Next generation sequencing (NGS) allows to keep up with the search for a growing number of genetic alterations in limited amounts of biologic material with a temporality compatible with clinical needs. NGS may allow reporting results to clinicians in a minimum of five days from the acquiring of formalin-fixed, paraffin-embedded (FFPE) samples. Implementation of a new NGS method follows well-defined procedures that are usually defined by academic societies or by state medical agencies [3,4]. Following the new NGS method set up, identification of factors that contribute to alter data quality over time is of critical importance. Indeed, several environmental, material, or human factors may impact NGS long-term performances (i.e., analytical sensitivity), possibly affecting the choice of clinical treatments [2,5]. Indeed, the initial validation does not ensure that the method will be robust enough to produce accurate results throughout the instrument lifespan.

Two major NGS methods are commonly used in diagnostic laboratories. Although both methods are based on sequencing by synthesis, Thermo Fisher and Illumina platforms use different principles for sequencing. Thermo Fisher Ion Torrent^TM technology directly converts nucleotide sequences into digital information using a semiconductor chip measuring variation of pH [6]. Illumina “bridge amplification” technology is based on incorporation and detection of fluorescent nucleotides in DNA fragments immobilized on a glass slide [7]. A recent study reported that both technologies achieved comparable NGS performances (i.e., mean read coverage, mean coverage uniformity, and variant allele frequency) and are suitable for diagnosis purpose [8].

Our laboratory implemented the Oncomine^TM Focus Assay (OFA) on an Ion S5XL (ThermoFisher, Les Ulis, France). This assay is dedicated to the identification of all types of theranostic variations in FFPE tissues and enables the concurrent analysis of DNA and RNA to detect variations in 52 genes in solid tumors [9,10]. To our knowledge, no study has been dedicated to investigate the evolution of the performances of OFA over a long period of time.

The aim of this study was to evaluate retrospectively the evolution of the performances of the OFA during a 21-month period in order to verify whether it is suitable for hospital routine practice.

2. Materials and Methods

2.1. Sample Section

This study was performed at Hospital Avicenne, Molecular and Biochemistry Laboratory (93000-Bobigny-France), from January 2020 to September 2021 and approved by the Local Ethic Comity (Avicenne Hospital). Formalin-fixed, paraffin-embedded (FFPE) tissues originating from surgical biopsies were systematically reviewed by a skilled pathologist who determined the tumor cellularity. Commercial quality samples were used for DNA assay validation (Quantitative Multiplex Reference Standard (cat. no. HD200, Horizon Diagnostics, Waterbeach, Cambridge, UK), EGFR Gene-Specific Multiplex Reference Standard (FFPE) 5% Variant Allelic Fraction (VAF) (cat. no. HD300, Horizon Diagnostics), Structural Multiplex Reference Standard (FFPE) (cat. no. HD789, Horizon Diagnostics). RNA assay characteristics performances were assessed using ALK-RET-ROS1 Fusion FFPE RNA (cat. no. HD784, Horizon Diagnostics) and Seraseq FFPE Tumor Fusion RNA Reference Material v4 (cat. no. SER0710-0496, Seracare, Ozyme, Saint Cyr l’Ecole, France).

Anonymized FFPE patient samples or quality control (QC) samples originating from 73 consecutive sequencing runs were used for quality metrics assessment. DNA alterations were analyzed from 420 patient samples originating from various tumors including 277 lung, 64 colon, 34 skin, and 45 other tissues (18 pancreas, 16 bile ducts, 5 breast, 3 thyroids, and 3 spleens) over a 21-month period. RNA fusions and exon skipping were analyzed from 55 tissue samples (45 lung, 8 bile duct, 1 pancreas, and 1 thyroid) over a 12-month period.

2.2. DNA and RNA Extraction

DNA and RNA were isolated using the Maxwell 16 Instrument using Maxwell^® RSC DNA FFPE Kit and Maxwell^® RSC RNA FFPE Kit, respectively, according to the manufacturer’s protocols (Promega, Madison, WI, USA). DNA and RNA quantification were performed on a Qubit 2.0 Fluorometer using Qubit dsDNA and Qubit HS RNA assay, respectively (Thermo Fisher, Les Ulis, France).

2.3. Library Preparation and Sequencing

For RNA assay, prior to library preparation, cDNA were synthetized using SuperScript VILO cDNA synthesis kit (catalog no. 11756050, Thermo Fisher). Library preparations were carried out using Oncomine Focus Assay, Chef-Ready Library (Thermo Fisher) using an Ion Chef Instrument (Thermo Fisher) following the manufacturer’s instructions using a total of 10ng of DNA or 2ng of RNA.

The DNA panel is designed to identify hotspot mutations of 35 genes (AKT1, ALK, AR, BRAF, CDK4, CTNNB1, DDR2, EGFR, ERBB2, ERBB3, ERBB4, ESR1, FGFR2 FGFR3, GNA11, GNAQ, HRAS, IDH1, IDH2, JAK1, JAK2, JAK3, KIT, KRAS, MAP2K1, MET, MTOR, NRAS, PDGFRA, PIK3CA, RAF1, RET, ROS, and SMO), copy number variations of 19 genes (ALK, AR, BRAF, CCND1, CDK4, CDK6, EGFR, ERBB2, FGFR1, FGFR2, FGFR3, FGFR4, KIT, KRAS, MET, MYC, MYCN, PDGFRA, and PIK3CA), fusions drivers of 21 genes (ABL1, ALK, AKT1, AXL, BRAF, ERBB2, ERG, ETV1, ETV4, ETV5, FGFR1, FGFR2, FGFR3, NTRK1, NTRK2, NTRK3, PDGFRA, PPARG, RAF1, RET, and ROS1), and exon skipping of 2 genes (EGFR and MET).

RNA and DNA libraries were equalized at 100pM using the Ion Chef instrument and pooled before templating (Ion 510™ & Ion 520™ & Ion 530™ Kit—Chef, cat. no. A34461, Thermo Fisher). Eight DNA samples or both eight DNA samples and eight RNA samples were loaded on a 520 chip (Ion 520™ Chip Kit, catalog no. A27762 Thermo Fisher). Sequencing was performed on an Ion S5XL instrument (Thermo Fisher).

The Tumor Hot Spot assay (THS, Multiplicom, Les Ulis, France), on a MiSeq sequencer (Illumina, Paris, France), was used for OFA inter-run variability comparison.

2.4. Bioinformatic Pipeline

For each run, quality metrics were assessed including chip loading density, number of total reads, percentage of clonality, percentage of adapter dimer, percentage of low quality, read length, and alignment of the reads to the hg19 human reference genome (Torrent server, version 5.12, Thermo Fisher). The “coverage Analysis” plugging was applied to assess the quality of sample sequencing. Each sample must reach strict validation criteria, i.e., yield a minimum of 400,000 reads, 98% of the amplicons with a minimal sequencing depth of 500X, 90% of the reads located within the target region boundaries, 80% of the amplicons being read from end-to-end, and 90% of the amplicons being read without strand bias.

For DNA variant annotation, the Ion Reporter software was carried out (version 5.10). The following default parameters were modified from the “Oncomine Focus w2.4—DNA—Single Sample” workflow: variants were reported with allele view, complex variants were allowed, down-sampled to coverage was set to 5000 reads, and the variants were generated at a minimum VAF of 0.03. The hotspot file was also modified: some hotspot mutations were added to report minor variants within BRAF, KRAS, and NRAS genes (Appendix A). The minimal value for VAF detection was decreased to 0.02 for positions of theranostic interest within BRAF, EGFR, KRAS, and NRAS genes. CNV were reported by the pipeline if the MAPD (median absolute pairwise difference) was below 0.4 and if the amplification factor was above 4.

For RNA variant annotation, the “Oncomine Focus—520—w2.4—Fusions—Single Sample” workflow was used with default parameters. The validation criteria for RNA samples were as follows: each sample must generate at least 20,000 reads and have a minimum mean read length of 50 pb. Additionally, at least three of the five RNA internal controls (TBP, LRP1, ITGB7, MYC, and HMBS) must be called. Finally, RNA alterations were reported only if a minimum number of reads was reached: 20 for targeted fusions, 250 for non-targeted fusions, and 120 for exon skipping.

For the inter-run variability comparison study, BAM files generated by the MiSeq instrument were analyzed using a tailored Sophia Genetics pipeline (Sophia Genetics, Lausanne, Switzerland). Variant calls were confirmed using the Integrative Genomics Viewer tool (Broad Institute, Cambridge, MA, USA) when necessary.

2.5. Immunohistochemistry

Immunohistochemistry for ROS1, ALK, and ERBB2 was performed as previously described [11,12]. Tissues were incubated with rabbit anti-ALK monoclonal antibody prediluted at 1:2 (clone K52076; Dako, Agilent, les Ulis, France) and anti-ROS1 monoclonal antibody (clone 1A4; Origene, Clinisciences, Nanterre, France). ERBB2 status was assessed using the HercepTest™ (kit K5207, Dako-Agilent, Les Ulis, France).

3. Results

3.1. Sequencing Performances

The quality metrics of the OFA DNA panel were assessed using 73 consecutive runs on 520 chips including QC and clinical samples. An average of 11 × 10⁶ (±0.3 × 10⁶; CV = 3.4%) reads were initially generated per run. The polyclonal ion sphere particles (ISP), primer dimers, low quality reads, and test quality sequences were then filtered out (5.8 × 10⁶ ± 0.8 × 10⁶ reads). Hence, 49 ± 7% (CV = 13%) of the amplicons presented the required quality for further bioinformatics analysis (Figure 1a). Among an average number of sequenced bases of 77 × 10⁶ (±2.5 × 10⁶), 72 × 10⁶ (±2.4 × 10⁶) bases were sequenced with a quality score of 20 (Q20). The mean sequencing depth for a given on-target base was 2246X ± 939X. All the metrics, independent of DNA quality, were measured at each run during the 21-month period and did not reveal any loss of performance over time, as assessed by the low coefficients of variation.

For each sample, an average of 6.0 × 10⁵ (±2.6 × 10⁵; CV = 44%) reads were mapped to hg19 genome reference and 98.2 ± 7.1% of the reads were aligned over a target region. The mean read length was 114 bp (± 4 bp). The uniformity of base coverage, defined as the percentage of on-target bases covered by at least 20% of mean coverage depth, was 98.5 ± 8.4%.

Analysis of 400 consecutive samples revealed an average of 95.8 ± 16% (CV = 16.7%) of amplicons with at least 500X, 98.4 ± 2.0% (CV = 2.0%) of amplicons with no strand bias, and 97.3 ± 5.4% (CV = 5.6%) of amplicons with end-to-end sequencing (Figure 1b). A total of 64 of these 400 samples (16%) failed to reach Thermo Fisher quality requirements due to insufficient depth coverage (n = 58), strand bias (n = 2), or default of end-to-end sequencing (n = 4) (Figure 1b).

A more detailed analysis of 136 consecutive samples indicated that 97.8% (133/136) of all samples reached an average depth of 500X with a mean sequencing depth of 2695X ± 432X across all amplicons. Some amplicons reached 5066X ± 2535X (OCP1_MET_8). The amplicons more likely to be covered with less than 500X were OCP1_DCUN1D1_9: 845X ± 386X, OCP1_KRAS_10: 892X ± 403X, and OCP1_AR_8: 938X ± 506X (Figure 1c).

3.2. Long Term Inter-Run Variability for SNVs and Indels

Inter-run variability for SNVs and indels analysis was performed using both QC materials and patient samples. To assess the inter-run performance of the OFA using S5XL instrument, different QC materials were analyzed (HD300: n = 42 runs; HD200: n = 35 runs). All expected mutations in the reference material were called at expected VAFs. Coefficient of variation of measured VAF ranged from 5.6% (p.His1047Arg, expected VAF = 17.5%) to 37.7% (p.Thr790Met, expected VAF = 1%) (

Table 1. Inter-run variability of VAF obtained with HD200 and HD300 QC materials on S5XL and MiSeq instruments.

		Oncomine Focus Assay				Tumor Hot Spot
	Expected VAF (%)	n	Mean VAF (%) (Mean ± SD)	CV (%)	N	Mean VAF (%) (Mean ± SD)	CV (%)
BRAF p.Val600Glu	11	42	11.8 (11.1–12.5)	6.0	43	11.6 (9.8–13.4)	7.7
EGFR p.Glu746_Ala750del	2	42	1.5 (1.2–1.9)	22.4	43	1.9 (1.6–2.3)	19.2
EGFR p.Glu746_Ala750del	5	35	4.0 (3.4–4.5)	14.5	42	5.1 (4.5–5.7)	11.5
EGFR p.Gly719Ser	5	35	5.2 (4.5–6.0)	14.6	42	5.7 (5.2–6.2)	9.4
EGFR p.Thr790Met	1	42	1.1 (0.7–1.5)	37.7	43	1.2 (0.6–1.7)	46.5
EGFR p.Leu861Gln	5	35	5.5 (4.9–6.0)	10.3	42	4.7 (4.3–5.2)	10.0
EGFR p.Leu858Arg	3	42	3.4 (2.7–3.9)	13.3	43	3.4 (3.0–4.0)	14.1
KRAS p.Gly12Asp	6	42	6.2 (5.5–7.0)	12.3	43	6.5 (4.8–8.7)	12.9
KRAS p.Gly13Asp	15	42	14.2 (13.1–15.4)	8.1	43	15.2 (11.7–18.7)	11.5
NRAS p.Gln61Lys	13	42	12.9 (11.6–14.2)	9.8	43	11.8 (9.4–14.1)	10.0
PIK3CA p.Glu545Lys	9	42	7.5 (5.4–9.5)	11.9	43	8.3 (6.7–10.0)	10.1
PIK3CA p.His1047Arg	18	42	17.5 (16.4–18.4)	5.6	43	17.7 (15.3–20.1)	6.8

; Figure 2 and Figure S1). The follow-up of the VAF QC material by Levey–Jennings charts assessed the low inter-run variability for VAF ranging from 1% to 11%. Furthermore, no drift of VAF values was observed. Indeed, during the 21-month period, for the lowest expected VAF (EGFR p.Thr790Met), only one value (2.3%) overpassed the limit of 3SD (Figure 2).

These data were also compared to a similar inter-run variability study performed using the THS on the Miseq instrument (HD300: n = 42 runs; HD200: n = 43 runs). The comparison of the VAF obtained for expected variants indicated a strong correlation between both methods (Pearson’s r = 0.990) (Table 1).

To further evaluate inter-run variability, clinical samples from colorectal and lung tissues were analyzed within six different runs. Indeed, PIK3CA p.His1047Arg, EGFR p.Leu747_Ala750del, AKT1 p.Gly17Leu, BRAF p.Val600Glu, and KRAS p.Gly12Val mutations previously detected using the THS assay were called at each run using the OFA (Figure 3). The VAF coefficients of variation ranged from 6.7% (BRAF p.Val600Glu) to 33.1% (KRAS p.Glu12Val).

3.3. Detection of SNVs and Indels

During the 21-month study of the OFA, the pipeline allowed to detect a large variety of mutations at a broad range of VAF. Among 420 consecutive DNA clinical samples, the pipeline reported SNV or indel in 294 samples (70%). The distribution of the most frequently mutated genes was KRAS 42.2% (n = 158), EGFR 13.9% (n = 52), PIK3CA 10.2% (n = 38), BRAF 9.1% (n = 34), NRAS 4.0% (n = 15), and CTNNB1 2.9% (n = 11) (Figure 4).

During our study, 353 DNA variants were detected. Sixty-four mutations had a VAF comprised between 2 and 10%. The original pipeline was designed to detect hotspot mutations with a VAF above 3%. The modification of the pipeline allowed the detection of 10 (3.4%) mutations with VAF below 3% with potential theranostic impact: EGFR p.Leu747_Thr751del (2.9%), p.Leu858Arg (2.6%); KRAS p.Gly12Ala (2.4% and 2.4%) p.Gly12Asp (2.1% and 2.9%), p.Gly12Cys (2.7%), p.Gly13Cys (2.9%), p.Lys117Asn (2.3%); and NRAS (2.7%).

3.4. Performances of CNV Detection

To assess inter-run variability, 20 QC samples were analyzed in different runs. The MAPD scores ranged from 0.272 to 0.356, assessing a low read coverage noise (

Table 2. Inter-run variation of gene amplification analysis using HD789 QC material in 20 consecutive runs.

Gene	Expected CNV	Measured CNV (Mean ± SD)	CV (%)	5% Limit Variability of OFA Confidence Interval (Mean ± SD)	95% Limit Variability of OFA Confidence Interval (Mean ± SD)
MET	4.5	5.8 ± 0.3	5.2%	4.5 ± 0.4	7.4 ± 0.4
MYCN	8.5	14.1 ± 1.0	3.7%	11.3 ± 1.0	17.6 ± 0.8

). Furthermore, the inter-run variability of MAPD was low (mean: 0.305 ± 0.032), indicating the long-term performance stability of CNV determination. MET and MYCN gene amplifications were systematically reported close to expected levels. In addition, the variation of inter-run CNV quantification remained low as assessed by the low coefficient of variation for each CNV level. No drift of the measured CNV value was observed, indicating that the performance of the method remained stable over time.

This study was extended to three clinical samples, which were sequenced twice. KRAS, EGFR, FGFR2, CCND1, CDK4, and ERBB2 genes amplifications were reported in the two runs with equivalent amplification rates.

Furthermore, we confirmed CNV detection of ERBB2 gene by studying protein expression level using immunochemistry assay in four lung tumors samples (

Table 3. ERBB2 gene amplification evaluation by NGS and immunochemistry analysis.

Sample	MAPD	ERBB2 CNV	CNV Confidence Interval (5–95%)	Immunochemistry Staining Intensity
Patient 1	0.323	28.0	22.2–35.2	Strong (+++)
Patient 2	0.360	9.5	7.5–12.0	Strong (+++)
Patient 3	0.282	5.4	4.2–6.8	Intermediate (++/+++)
Patient 4	0.334	12.0	8.3–16.5	Low (+)

; Figure 5). Nuclei were stained in blue (hematoxylin), cytoplasm of positives cells for ERBB2 were stained in brown, while negative cells remained unstained. A large view of bone tissue section showing negative staining for ERBB2 is shown in Figure S2.

3.5. Detection of Gene Amplification in Routine Practice

The pipeline revealed 88 (20.7%) gene amplifications among 420 clinical samples. These amplifications were detected in 56 lung, 6 colon, 3 bile duct, 3 pancreas, 2 skin, and 2 breast samples. The genes exhibiting most frequently CNV were CDK4 (n = 20), EGFR and MYC (n = 13 each), CCND1 (n = 7), ERBB2 (n = 6), and KRAS and MET (n = 5 each) (Figure 6a). The amplification factors reported by the OFA pipeline ranged from 5 to 75. Amplification factors remained below 6x in eight cases and were not reported to clinicians (7%) (Figure 6b).

3.6. Gene-Fusion and Exon-Skipping Detection

RNA samples from HD784 and Seracare QC materials were sequenced in 29 and 7 runs, respectively. Quality criteria were satisfactory for each sample, allowing further data analysis. Expected RNA fusions and exon skipping variations were systematically detected. Fusion reads represented 1.6 to 17.8% of total reads counts (

Table 4. Inter-run variation for gene-fusion and exon skipping detection in QC samples.

HD784 QC Material (n =29)
Total Mapped Fusion Allele Reads: 95,753 (57,060–134,445); CV: 40.4%
	Fusion Reads Counts		Fusion Reads (%)
	Mean (Mean ± SD)	CV (%)	Mean (Mean ± SD)	CV (%)
CCDC6(1)-RET(12)	4034 (1867–6212)	54.0	4.6 (3.5–5.7)	23.2
SLC34A2(4)-ROS1(32)	14,637 (7927–21,348)	45.8	17.8 (13.0–22.7)	27.1
EML4(13)-ALK(20)	4289 (1457–7121)	35.2	4.5 (2.9–6.1)	27.2
Seracare QC Material (n =7)
Total Mapped Fusion Allele Reads: 98,353 (73,474–123,233); CV: 23.3%
	Fusion Reads Counts		Fusion Reads (%)
	Mean (Mean ± SD)	CV (%)	Mean (Mean ± SD)	CV (%)
FGFR3(17)-BAIAP2L1(2)	4436 (3849–5023)	13.2	4.6 (3.9–5.3)	15.1
CCDC6(1)-RET(12)	1589 (1194–1985)	24.9	1.6 (1.4–1.9)	17.5
KIF5B(24)-RET(11)	3639 (24,511–4828)	32.7	3.7 (2.9–4.5)	21.8
NCOA4(8)-RET(12)	1706 (1458–1955)	14.6	1.8 (1.6–1.9)	9.5
TMPRSS2(1)-ERG(2)	3282 (1437–5126)	56.2	3.2 (2.2–4.2)	31.1
PAX8(9)-PPARG(3)	3909 (2476–5343)	36.7	3.9 (3.2–4.7)	18.1
EML4(13)-ALK(20)	2837 (2464–3211)	13.2	3 (2.3–3.7)	22.2
SLC45A3(1)-BRAF(8)	5436 (1681–9192)	69.1	5.2 (3.3–7.0)	36.0
SLC34A2(4) -ROS1(34)	3437 (2111–4761)	38.6	3.4 (2.9–4.0)	15.9
CD74(6)-ROS1(34)	7090 (5700–8479)	19.6	7.4 (5.8–8.9)	20.9
LMNA(2)-NTRK1(10)	1701 (531–2874)	68.8	1.6 (1.0–2.6)	38.4
TFG(5)-NTRK1(10)	1323 (923–1723)	30.2	1.3 (1.4–1.5)	14.9
TPM3(7)-NTRK1(9)	2822 (2127–3517)	24.6	2.9 (2.6–3.1)	7.9
ETV6(5)-NTRK3(15)	2352 (1908–2796)	18.9	2.4 (2.0–2.9)	17.4
EGFR(1)-EGFR(8) (vIII)	2605 (1390–3821)	46.6	2.6 (2.0–3.0)	18.3
MET(13)-MET(15)	2451 (2012–2889)	17.9	2.6 (1.9–3.3)	27.0

). The inter-run CV of the percentage of total reads counts ranged from 7.9 to 38.4%. No drift in the detection of both fusions and exon skipping was observed, indicating that the performance of the method remained stable over time.

To complete the validation of MET exon 14 skipping detection, we analyzed RNA material from two patients with MET alterations previously detected at DNA level with the THS assay (MET c.3082 +1G>C and MET c.2888-2A>C). These mutations were detected at the RNA level with the OFA in two independent experimentsIn order to validate ALK and ROS detection, two gene fusions determined by OFA [(EML4(6)—ALK(20) (reads counts: 2024, total mapped fusion reads: 49,617) and CD74(6)—ROS1(34) (reads counts: 1027, total mapped fusion reads: 31,980)] were confirmed by immunochemistry (Figure 7). Nuclei were stained in blue (hematoxylin), cytoplasm of positives cells for ALK and ROS1 were stained in brown, while negative cells remained unstained. Negative ROS1 staining of non-tumoral lung tissue is shown in Figure S3.

3.7. Detection of Fusion and Exon Skipping in Routine Practice

Among 55 RNA clinical samples, 11 samples were not suitable for analysis because of poor sequencing quality and no RNA modification was detected for 37 samples. The OFA pipeline detected three MET exon skipping alterations, two ROS fusion transcripts in lung tissues (CD74(6)—ROS1(34) and SLC34A2(13)—ROS1(34)), one ALK fusion transcript in lung tissue (EML4(6)—ALK(20)), and one PAX8 fusion transcript in thyroid tissue (PAX8(9)—PPARG(2)).

4. Discussion

Theranostic analysis of tumors becomes continuously more complex requiring the study of ever more larger panels of genes in search of point mutations, short insertions and deletions, CNV, or gene fusions. In addition, the development of less invasive sampling techniques such as needle biopsy lead to the decrease of available material. Despite these limitations, genetic analysis must remain compatible with a rapid determination of therapeutic strategy. NGS is the method of choice to fulfill these challenges, assuming that optimal analytical performances are maintained over time.

The aim of this study was to determine whether the analytical performances of the Oncomine Focus Assay remain suitable for clinical practice over a 21-month period. For this purpose, we evaluated the sequencing results obtained from both QC and clinical samples along 73 consecutive runs on an Ion S5 XL sequencer. During this period, we observed no technical failure and the inter-runs variability of critical parameters was adapted to clinical practice.

We first observed that the template generation was reproducible over time. Indeed, the coefficient of variation of the chips loading was 3.4% (mean: 11 × 10⁶ amplicons). Subsequent bioinformatic processing indicated that 49% (CV = 13%) of these amplicons had a sufficient quality for further processing. Additionally, we showed that 6.0 × 10⁵ (CV = 44%) reads per sample were mapped to hg19 reference genome. These CV reflected also the variation in DNA quality of samples. However, our results were consistent with other studies using the same technique. Indeed, Bartlett et al. showed that intra-laboratory variability of mapped reads generation may range from 0.18 × 10⁶ to 1.6 × 10⁶ (10). In our study, the quality of mapped reads within both QC and clinical samples was maintained over time, assessed by the low variability of percentage of end-to-end reads (97.3%; CV = 5.6%) and reads without strand bias (98.4%; CV 2.0%).

The main point of variation, influenced by DNA quantity, was the percentage of amplicons reaching 500X per sample [2]. In our routine practice, although a depth below 500X may remain informative when mutations are at a high allelic frequency, we consider that the depth should reach 500X to validate a negative result. We showed that, for a given sample, 95.8% (CV = 16.7%) of the amplicons reached a depth of 500X. Moreover, all amplicons were individually covered with at least 845X (Figure 1). These results are comparable with those of Williams et al.’s study reporting 89.7% of samples with an average amplicon coverage above 500X [9]. In our analysis, all hotspot regions were sequenced with at least 1574X. Of note, the amplicons sequenced with a lower depth targeted only CNV positions (Figure 1c; Appendix B). We successfully sequenced 98% of clinical samples, a proportion comparable to the 95% described by Williams et al. [9]. These data suggest (i) the tissular heterogeneity of samples did not affect the quality of the sequencing and (ii) the performance of sequencing did not decline over time. Considering that numerous samples were tissue biopsies, we show that the OFA is adapted to this limited amount of material.

The call of the mutations with the Ion Reporter software could fail because of inadequacy of the bioinformatic pipeline. Therefore, we modify some parameters of the workflow. Clinical purpose led us to use samples with a very low acid nucleic concentration, potentially causing limited sequencing depth. In order to improve robustness of our analysis, we modified the hotspot file (Appendix A). We set the detection threshold down to 3% of AF for SNV and indel, and down to 2% for driver variations in BRAF, ERBB2, EGFR, KRAS, and NRAS genes. In addition, we increased the down sampled coverage up to 5000 reads (the minimum reads randomly considered), allowing to improve the inter-run variability. Finally, we and others observed incorrect nomenclature calls [9]. To attribute the mutation position to the correct variant annotation, the “allele view” and the “allow complex” parameters were selected.

In order to validate our modifications of the bioinformatic pipeline, we analyzed the inter-run variability of two QC materials. We observed similar inter-run variabilities between the OFA and THS kits (Table 1). As expected, the variability was higher for low VAFs. However, the low coefficients of variation allowed the detection of mutations down to 1% VAF at each run. Furthermore, mutations from 3 to 5% expected VAF were systematically called (Table 1; Figure 2 and Figure S1). This long-term follow-up of VAF values highlighted that there were no loss of performance and no drift over time. This low variability was appropriate for clinical sample analysis. An inter-laboratory study involving six teams and analyzing the same QC materials obtained similar VAF for most mutations. However, we observed differences only for mutations with VAF < 5%. Five of the six laboratories did not report the EGFR p.Leu858Arg SNP with a VAF of 4% and none reported the EGFR p.Thr790Met and p.Glu746_Ala750del variations at VAF 1% and 2%, respectively [10]. This increased sensibility may be due to our optimization of the bioinformatic pipeline.

Using FFPE clinical samples, the observed inter-run variability was higher (Figure 3). These results, which were expected, were possibly due to lower DNA quality. Nevertheless, this larger variability remained sufficient to systematically report each expected mutation.

In our routine practice, we include a QC sample at each library preparation for quality purpose. Considering clinical results interpretation, to qualify a variant as both authentic and relevant, we take into consideration the number of reads at the corresponding genomic position, the tumor cellularity, and the clinical context.

In the case of patient follow-up or for investigation of therapeutic resistance mechanism, the communication to clinicians of variants with VAF below the threshold of 5% may have a significant impact on prognosis [13,14]. During the 21-month period of DNA analysis, we detected 353 mutations among tissue samples. The increased sensitivity of the bioinformatic analysis allowed the detection of 10 mutations with VAF below 3%. The mutation with the lowest VAF (KRAS p.Gly12Asp; 2%) was detected in a colon sample. When such low VAF mutation are detected, the biologist has to consider also detailed metrics and genomic alignment visualization to appreciate its theranostic relevance [13,14,15].

Considering gene amplification, the CNV value is interpreted according to the MAPD score. The analysis of inter-run variability using QC material revealed that both expected gene amplifications were detected and associated MAPD scores were below the threshold of 0.4, giving confidence to these results. The coefficients of variation were notably low for MET and MYCN amplification (5.2% and 3.7%, respectively), attesting the inter-run stability over time. In four clinical samples, ERBB2 gene amplifications were confirmed by immunocytochemistry (Table 3; Figure 5). This limited comparison suggests that, using OFA, the CNV represented more a confidence level of positivity than an absolute value [16]. In order to avoid false positive, we choose to report only CNV with a “5% confidence lower limit” above 6 [17]. Among the 88 amplifications called by the pipeline, we reported 79 (91%) results to the clinicians.

The scarcity of patients presenting RNA alteration imposed a validation protocol using two different QC materials in order to increase the diversity of fusion transcripts. The expected fusion transcripts were systematically detected throughout the 12-month period. Despite a high heterogeneity of determined inter-run variability CV, fusions reads counts were always above the manufacturer recommended thresholds (i.e., 20 for targeted fusions, 250 for non-targeted fusions, and 120 for exon skipping). Like others, we chose to determine the ratio of fusion reads counts to total mapped reads counts (Table 4) [10]. This ratio, similar to VAF, was useful to interpret clinical samples with low total RNA quantity. In our QC study, a fusion transcript could be reported with a percentage as low as 1% of total mapped reads. In contrast, in samples with a high amount of total RNA mapped reads, a ratio below 10% may indicate a false positive result.

Regarding HD784 QC material, we obtained similar fusion reads and fusion reads ratios to those reported by Bartlett et al.’s study [10]. In our clinical samples study, we reported seven fusions or exon skipping variants; among them, two were validated by immunochemistry (Figure 7).

Our results demonstrated that the sensitivity and the accuracy of OFA were suitable for molecular analysis of low quality and/or limited amounts of nucleic acids obtained from FFPE tumor samples in clinical routine practice. Slight modifications of the bioinformatic pipeline allowed to improve detection of low VAF variants, which may be useful to early detect genetic alterations involved in resistance mechanisms. This long-term study conducted in real-life conditions demonstrated that the performances of the OFA remained stable over time and ensured the reliability of results with theranostic impact.