Managing Deviating EQA Results: A Survey to Assess the Corrective and Preventive Actions of Medical Laboratories Testing for Oncological Biomarkers
Abstract
:1. Introduction
2. Materials and Methods
3. Results
3.1. Management of Deviating EQA Scheme Results
3.2. Management of Deviating Results during Routine Processing
3.3. CAPA of Deviating Biomarker Results Related to Laboratory Characteristics
3.4. Improvement of Correct Testing in Next EQA Schemes
4. Discussion
4.1. Management of Deviating EQA Scheme Results
4.2. Management of Deviating Results during Routine Processing
4.3. CAPA of Deviating Biomarker Results Related to Laboratory Characteristics
4.4. Improvement of Correct Testing in Next EQA Schemes
4.5. Recommendations for Future Error Management
5. Conclusions
Supplementary Materials
Author Contributions
Funding
Acknowledgments
Conflicts of Interest
References
- Cree, I.; Deans, Z.; Ligtenberg, M.J.L.; Normanno, N.; Edsjö, A.; Rouleau, E.; Solé, F.; Thunnissen, E.; Timens, W.; Schuuring, E.; et al. Guidance for laboratories performing molecular pathology for cancer patients. J. Clin. Pathol. 2014, 67, 923–931. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- The Food and Drug Administration (FDA), Center for Medicaid Services (CMS), and the Centers for Disease Control and Prevention (CDC). The Clinical Laboratory Improvement Amendments of 1988. Health. Care Financ. Rev. 1989, 10, 141–146. [Google Scholar]
- International Organization for Standardization. Medical Laboratories—Particular Requirements for Quality and Competence; ISO: Geneva, Switzerland, 2012. [Google Scholar]
- International Organization for Standardization. Conformity Assessment—General Requirements for Proficiency Testing; ISO: Geneva, Switzerland, 2010. [Google Scholar]
- European Society of Pathology (ESP). Available online: https://www.esp-pathology.org/esp-foundation/eqa-schemes.html (accessed on 18 March 2020).
- Patton, S.; Normanno, N.; Blackhall, F.H.; Murray, S.; Kerr, K.M.; Dietel, M.; Filipits, M.; Benlloch, S.; Popat, S.; Stahel, R.; et al. Assessing standardization of molecular testing for non-small-cell lung cancer: Results of a worldwide external quality assessment (EQA) scheme for EGFR mutation testing. Br. J. Cancer 2014, 111, 413–420. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- College of American Pathologists (CAP). Available online: https://documents.cap.org/documents/2019-surveys-catalog.pdf (accessed on 18 March 2020).
- Eurostat. Available online: https://ec.europa.eu/eurostat/statistics-explained/index.php/Construction_production_(volume)_index_overview (accessed on 15 July 2020).
- Haffajee, Z.A.M.; Kumar, B.; Francis, G.; Peck, M.; Badrick, T. The Royal College of Pathologists of Australasia Quality Assurance Program. Appl. Immunohistochem. Mol. Morphol. 2019, 27, 339–344. [Google Scholar] [CrossRef] [PubMed]
- Immunohistochemistry Quality Control (CIQC). Available online: http://cpqa.ca/main/ (accessed on 18 March 2020).
- Qualitätssicherungs-Initiative Pathologie (Quip). Available online: https://quip.eu/wp-content/uploads/2019/07/004_QuIP-Programm_25072019.pdf (accessed on 18 March 2020).
- Dequeker, E.M.; Keppens, C.; Egele, C.; Delen, S.; Lamy, A.; Lemoine, A.; Sabourin, J.-C.; Andrieu, C.; Ligtenberg, M.; Fetique, D.; et al. Three Rounds of External Quality Assessment in France to Evaluate the Performance of 28 Platforms for Multiparametric Molecular Testing in Metastatic Colorectal and Non-Small Cell Lung Cancer. J. Mol. Diagn. 2016, 18, 205–214. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Normanno, N.; Fenizia, F.; Castiglione, F.; Barberis, M.; Taddei, G.L.; Truini, M.; De Rosa, G.; Pinto, C.; Marchetti, A. External quality assessment for EGFR mutations in Italy: Improvements in performances over the time. ESMO Open 2017, 2, e000160. [Google Scholar] [CrossRef] [Green Version]
- Sociedad Espanola de Aanatomia Patologica (SEAP). Available online: https://www.seap.es/calidad (accessed on 18 March 2020).
- Richman, S.D.; Fairley, J.; Butler, R.; Deans, Z.C. RAS screening in colorectal cancer: A comprehensive analysis of the results from the UK NEQAS colorectal cancer external quality assurance schemes (2009–2016). Virchows Arch. 2017, 471, 721–729. [Google Scholar] [CrossRef]
- Stichting Kwaliteitsbewaking Medische Laboratoriumdiagnostiek (SKML). Available online: https://www.skml.nl/ (accessed on 18 March 2020).
- Wetenschappelijk Instituut Volksgezondheid/Institut scientifique de la Santé publique. Available online: https://www.wiv-isp.be/QML/Informatiebrochure_EKE.pdf (accessed on 18 March 2020).
- Kristensen, G.B.; Meijer, P. Interpretation of EQA results and EQA-based trouble shooting. Biochem. Med. 2017, 27, 49–62. [Google Scholar] [CrossRef] [PubMed]
- Sciacovelli, L.; Secchiero, S.; Padoan, A.; Plebani, M. External quality assessment programs in the context of ISO 15189 accreditation. Clin. Chem. Lab. Med. 2018, 56, 1644–1654. [Google Scholar] [CrossRef]
- Tack, V.; Schuuring, E.; Keppens, C.; Hart, N.T.; Pauwels, P.; Van Krieken, H.; Dequeker, E.M. Accreditation, setting and experience as indicators to assure quality in oncology biomarker testing laboratories. Br. J. Cancer 2018, 119, 605–614. [Google Scholar] [CrossRef]
- Lindeman, N.I.; Cagle, P.T.; Beasley, M.B.; Chitale, D.A.; Dacic, S.; Giaccone, G.; Jenkins, R.B.; Kwiatkowski, D.J.; Saldivar, J.-S.; Squire, J.; et al. Molecular Testing Guideline for Selection of Lung Cancer Patients for EGFR and ALK Tyrosine Kinase Inhibitors: Guideline from the College of American Pathologists, International Association for the Study of Lung Cancer, and Association for Molecular Pathology. Arch. Pathol. Lab. Med. 2013, 137, 828–860. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Lindeman, N.I.; Cagle, P.T.; Aisner, D.L.; Arcila, M.E.; Beasley, M.B.; Bernicker, E.H.; Colasacco, C.; Dacic, S.; Hirsch, F.R.; Kerr, K.; et al. Updated Molecular Testing Guideline for the Selection of Lung Cancer Patients for Treatment With Targeted Tyrosine Kinase Inhibitors. J. Mol. Diagn. 2018, 20, 129–159. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Van Krieken, J.H.; Normanno, N.; Blackhall, F.H.; Boone, E.; Botti, G.; Carneiro, F.; Celik, I.; Ciardiello, F.; Cree, I.A.; Deans, Z.C.; et al. Guideline on the requirements of external quality assessment programs in molecular pathology. Virchows Arch. 2012, 462, 27–37. [Google Scholar] [CrossRef] [PubMed]
- Dufraing, K.; Fenizia, F.; Torlakovic, E.; Wolstenholme, N.; Deans, Z.C.; Rouleau, E.; Vyberg, M.; Parry, S.; Schuuring, E.; Dequeker, E.M.C. Biomarker testing in oncology—Requirements for organizing external quality assessment programs to improve the performance of laboratory testing: Revision of an expert opinion paper on behalf of IQNPath ABSL. Virchows Arch. 2020, in press. [Google Scholar] [CrossRef] [PubMed]
- National Quality Assessment Advisory Panel Terms of Reference. Available online: https://www.rcpath.org/profession/committees/jwgqa/genetics-nqaap.html (accessed on 18 March 2020).
- Tack, V.; Ligtenberg, M.J.; Tembuyser, L.; Normanno, N.; Borght, S.V.; Van Krieken, J.H.; Dequeker, E. External Quality Assessment Unravels Interlaboratory Differences in Quality of RAS Testing for Anti-EGFR Therapy in Colorectal Cancer. Oncologist 2015, 20, 257–262. [Google Scholar] [CrossRef] [Green Version]
- Keppens, C.; Tack, V.; Hart, N.T.; Tembuyser, L.; Ryska, A.; Pauwels, P.; Zwaenepoel, K.; Schuuring, E.; Cabillic, F.; Tornillo, L.; et al. A stitch in time saves nine: External quality assessment rounds demonstrate improved quality of biomarker analysis in lung cancer. Oncotarget 2018, 9, 20524–20538. [Google Scholar] [CrossRef] [Green Version]
- Keppens, C.; Schuuring, E.; Dequeker, E.M.C. Causes behind error rates for predictive biomarker testing: The utility of sending post-EQA surveys. Virchows Arch. 2020, in press. [Google Scholar]
- Dufraing, K.; De Hertogh, G.; Tack, V.; Keppens, C.; Dequeker, E.; Van Krieken, J.H.; Van Krieken, H.J. External Quality Assessment Identifies Training Needs to Determine the Neoplastic Cell Content for Biomarker Testing. J. Mol. Diagn. 2018, 20, 455–464. [Google Scholar] [CrossRef] [Green Version]
- Broek, D.V.D.; Hiltermann, T.J.N.; Biesma, B.; Dinjens, W.N.M.; Hart, N.A.T.; Hinrichs, J.W.J.; Leers, M.P.G.; Monkhorst, K.; Van Oosterhout, M.; Scharnhorst, V.; et al. Implementation of Novel Molecular Biomarkers for Non-small Cell Lung Cancer in the Netherlands: How to Deal With Increasing Complexity. Front. Oncol. 2020, 9, 15–21. [Google Scholar] [CrossRef]
- Keppens, C.; Dufraing, K.; Van Krieken, J.H.; Siebers, A.G.; Kafatos, G.; Lowe, K.A.; Demonty, G.; Dequeker, E. European follow-up of incorrect biomarker results for colorectal cancer demonstrates the importance of quality improvement projects. Virchows Arch. 2019, 475, 25–37. [Google Scholar] [CrossRef] [Green Version]
- Dufraing, K.; Keppens, C.; Siebers, A.G.; Kafatos, G.; Lowe, K.; Demonty, G.; Dequeker, E.M.C.; Van Krieken, J.H. Pre-Analytical Challenges during RAS Testing: Tissue Quality and the Estimation of Neoplastic Cell Percentage. Biomark. J. 2019, 5, 2. [Google Scholar] [CrossRef]
- Keppens, C.; Boone, E.; Gameiro, P.; Tack, V.; Moreau, E.; Hodges, E.; Evans, P.; Brüggemann, M.; Carter, I.; Lenze, D.; et al. Evaluation of a worldwide EQA scheme for complex clonality analysis of clinical lymphoproliferative cases demonstrates a learning effect. Virchows Arch. 2020, in press. [Google Scholar]
- Pillay, B.; Wootten, A.C.; Crowe, H.; Corcoran, N.; Tran, B.; Bowden, P.; Crowe, J.; Costello, A.J. The impact of multidisciplinary team meetings on patient assessment, management and outcomes in oncology settings: A systematic review of the literature. Cancer Treat. Rev. 2016, 42, 56–72. [Google Scholar] [CrossRef] [PubMed]
- Salto-Tellez, M.; De Castro, D.G. Next-generation sequencing: A change of paradigm in molecular diagnostic validation. J. Pathol. 2014, 234, 5–10. [Google Scholar] [CrossRef] [PubMed]
- O’Brien, C.; Taylor, S.E.; O’Leary, J.J.; Finn, S.P. Molecular testing in oncology: Problems, pitfalls and progress. Lung Cancer 2014, 83, 309–315. [Google Scholar] [CrossRef]
- Koopman, B.; Van Der Wekken, A.J.; Ter Elst, A.; Hiltermann, T.J.N.; Vilacha, J.F.; Groves, M.R.; Berg, A.V.D.; Hiddinga, B.I.; Hijmering-Kappelle, L.B.M.; Stigt, J.A.; et al. Relevance and Effectiveness of Molecular Tumor Board Recommendations for Patients With Non–Small-Cell Lung Cancer With Rare or Complex Mutational Profiles. JCO Precis. Oncol. 2020, 4, 393–410. [Google Scholar] [CrossRef]
- Jones, B.A.; Darcy, T.; Souers, R.J.; Meier, F.A. Staffing Benchmarks for Clinical Laboratories: A College of American Pathologists Q-Probes Study of Laboratory Staffing at 98 Institutions. Arch. Pathol. Lab. Med. 2012, 136, 140–147. [Google Scholar] [CrossRef]
- Tack, V.; Spans, L.; Schuuring, E.; Keppens, C.; Zwaenepoel, K.; Pauwels, P.; Van Houdt, J.; Dequeker, E.M. Describing the Reportable Range Is Important for Reliable Treatment Decisions. J. Mol. Diagn. 2018, 20, 743–753. [Google Scholar] [CrossRef] [Green Version]
- Deans, Z.C.; Costa, J.-L.; Cree, I.; Dequeker, E.; Edsjö, A.; Henderson, S.; Hummel, M.; Ligtenberg, M.J.; Loddo, M.; Machado, J.C.; et al. Integration of next-generation sequencing in clinical diagnostic molecular pathology laboratories for analysis of solid tumours; an expert opinion on behalf of IQN Path ASBL. Virchows Arch. 2016, 470, 5–20. [Google Scholar] [CrossRef] [Green Version]
Year | 2015 | 2016 | 2017 | 2018 | Study Total | ||
---|---|---|---|---|---|---|---|
Analyzed laboratories | EQA participations to different subschemes | 329 | 445 | 733 | 712 | 2219 | |
Unique laboratories participating | 197 | 234 | 259 | 241 | 410 * | ||
Unique laboratories who received the survey (laboratories with at least one error in any of the subschemes) | 88 | 120 | 153 | 175 | 315 * | ||
Unique laboratories who replied to the survey | 39 | 44 | 90 | 99 | 185 * | ||
Analyzed surveys | Surveys sent | 105 | 154 | 234 | 298 | 791 | |
Surveys with response | 40 | 53 | 108 | 124 | 325 | ||
Analyzed cases | Cases tested in the scheme | 4224 | 5134 | 6276 | 5902 | 21536 | |
Deviating EQA results included in survey | 162 | 225 | 362 | 418 | 1167 | ||
Deviating EQA results with response | 51 | 74 | 181 | 208 | 514 | ||
NSCLC | FISH | ALK | 2 | 4 | 18 | 1 | 25 |
ROS1 | 6 | 3 | 7 | 33 | 49 | ||
FISH digital | ALK | 1 | 0 | 4 | 2 | 7 | |
ROS1 | 4 | 3 | 5 | 21 | 33 | ||
IHC | ALK | 0 | 4 | 20 | 7 | 31 | |
ROS1 | 9 | 0 | 2 | 6 | 17 | ||
PD-L1 | N/A | N/A | 35 | 28 | 63 | ||
IHC technical * | ALK | 8 | 6 | 7 | 15 | 36 | |
ROS1 | N/A | 1 | 0 | 2 | 3 | ||
PD-L1 | N/A | N/A | N/A | 6 | 6 | ||
IHC digital | PD-L1 | N/A | N/A | 11 | 19 | 30 | |
Variant analysis | EGFR (mandatory) | 21 | 19 | 41 | 27 | 108 | |
KRAS (optional) | N/A | N/A | 2 | 11 | 13 | ||
BRAF (optional) | N/A | N/A | 0 | 3 | 3 | ||
mCRC | Variant analysis | KRAS (mandatory) | N/A | 23 | 23 | 22 | 68 |
NRAS (mandatory) | N/A | 7 | 2 | 3 | 12 | ||
BRAF (optional) | N/A | 4 | 4 | 2 | 10 |
Question | n (%) | ISO 15189:2012 Clause [3] |
---|---|---|
When are EQA results evaluated (time after release by the provider)? | 407 | 5.6.3.1 |
Within a week | 238 (58.5) | |
Within two weeks‒within a month | 163 (40.0) | |
More than 1 month | 6 (1.5) | |
When was the error detected? | 435 | 5.6.2.3 5.7.1 |
Before release of the EQA report | 70 (16.1) | |
After release of the EQA report | 365 (83.9) | |
Are the results discussed in the laboratory? | 330 | 4.1.2.6 5.6.3.4 |
Yes, always | 270 (81.8) | |
Only in the case of deviating results | 54 (16.4) | |
No | 6 (1.8) | |
Which action was performed for this specific error? | 514 | 5.6.3.4 |
Additional EQA participation | 7 (1.4) | |
Change method/control tissue | 48 (9.3) | |
Contact manufacturer | 38 (7.4) | |
Contact EQA provider | 17 (3.3) | |
Implement/optimise procedure | 64 (12.5) | |
Protocol revision | 75 (14.6) | |
Retest sample | 26 (5.1) | |
Staff training | 78 (15.2) | |
Unknown | 19 (3.7) | |
None | 142 (27.6) | |
How many people were involved in the follow-up of the deviating result for this case? | 430 | 4.1.2.5 |
1 | 241 (56.0) | |
2 | 114 (26.5) | |
3 or more | 75 (17.4) | |
Who was responsible for this action? a | 430 | 4.9 |
Quality manager | 50 (11.6) | |
Lead laboratory technician | 72 (16.7) | |
Laboratory technician | 96 (22.3) | |
Molecular biologist | 172 (40.0) | |
Pathologist | 200 (46.5) | |
Laboratory director | 87 (20.2) | |
Other | 32 (7.4) | |
Were the EQA samples treated differently in any way? | 315 | 5.6.3.3 |
No | 265 (84.1) | |
Yes | 50 (15.9) | |
Was the personnel aware they were handling EQA samples? | 318 | 5.6.3.3 |
No | 28 (8.8) | |
Yes | 290 (91.2) | |
Importance given to EQA participation | 318 | 4.2.2 5.6.4 |
7 or less | 31 (9.7) | |
8 | 49 (15.4) | |
9 | 51 (16.0) | |
10 | 187 (58.8) |
Question | n (%) | ISO 15189:2012 Clause [3] |
---|---|---|
Who interprets the results? a | 474 | 4.1.2.1 |
Lead laboratory technician | 21 (4.4) | |
Laboratory technician | 83 (17.5) | |
Molecular biologist | 215 (45.4) | |
Pathologist | 308 (65.0) | |
Laboratory director | 40 (8.4) | |
KMBP/molecular pathologist | 19 (4.0) | |
Geneticist | 13 (2.7) | |
Other | 23 (4.9) | |
Is additional training received to perform the interpretation? a | 435 | 5.1.6 5.1.9 |
No additional training, degree only | 113 (26.0) | |
Performing validations (internal) | 170 (39.1) | |
Laboratory meetings (internal) | 98 (22.5) | |
Learning from colleagues with gradually more independence (internal) | 153 (35.2) | |
Training by the manufacturer (external) | 33 (7.6) | |
Training course/workshop (external) | 202 (46.4) | |
Literature study (internal) | 9 (2.1) | |
EQA/laboratory exchange (internal) | 10 (2.3) | |
Other training | 6 (1.4) | |
Who reports the results? a | 460 | 5.8.1 5.9.1 |
Lead laboratory technician | 13 (2.8) | |
Laboratory technician | 62 (13.5) | |
Molecular biologist | 159 (34.6) | |
Pathologist | 314 (68.3) | |
Laboratory director | 41 (8.9) | |
KMBP/molecular pathologist | 8 (1.7) | |
Other | 31 (6.7) | |
Is an additional sample always requested? | 388 | 4.7 |
No | 45 (11.6) | |
Depends on sample availability/type | 9 (2.3) | |
Yes in routine practice but not for EQA | 122 (31.4) | |
Yes, always | 212 (54.6) | |
Do you submit your results to a database? | 289 | 5.7.1 |
No, never | 15 (5.2) | |
No, a report for the oncologist is made only | 95 (32.9) | |
No, our results are research use only | 7 (2.4) | |
Yes, unspecified | 9 (3.1) | |
Yes, a local oncology database with patient follow-up | 7 (2.4) | |
Yes, a national pathology database | 25 (8.7) | |
Yes, our local pathology database | 131 (45.3) | |
Do you correlate molecular results with relevant literature? | 300 | 4.1.2.2 |
No | 73 (24.3) | |
Yes | 227 (75.7) | |
Do you ask for follow-up of the patient’s results? | 302 | 4.14.7 |
No | 77 (25.5) | |
No, although I would be interested | 105 (34.8) | |
Yes, unspecified | 14 (4.6) | |
Yes, during a multi-disciplinary team meeting | 59 (19.5) | |
Yes, occasionally for patients with specific variants | 47 (15.6) | |
Do you take part in continuous education? | 324 | 5.1.8 |
No | 98 (30.2) | |
Yes | 226 (69.8) |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Keppens, C.; Schuuring, E.; Dequeker, E.M. Managing Deviating EQA Results: A Survey to Assess the Corrective and Preventive Actions of Medical Laboratories Testing for Oncological Biomarkers. Diagnostics 2020, 10, 837. https://doi.org/10.3390/diagnostics10100837
Keppens C, Schuuring E, Dequeker EM. Managing Deviating EQA Results: A Survey to Assess the Corrective and Preventive Actions of Medical Laboratories Testing for Oncological Biomarkers. Diagnostics. 2020; 10(10):837. https://doi.org/10.3390/diagnostics10100837
Chicago/Turabian StyleKeppens, Cleo, Ed Schuuring, and Elisabeth MC Dequeker. 2020. "Managing Deviating EQA Results: A Survey to Assess the Corrective and Preventive Actions of Medical Laboratories Testing for Oncological Biomarkers" Diagnostics 10, no. 10: 837. https://doi.org/10.3390/diagnostics10100837