Deep Learning Algorithm Trained with COVID-19 Pneumonia Also Identifies Immune Checkpoint Inhibitor Therapy-Related Pneumonitis
Abstract
:Simple Summary
Abstract
1. Introduction
2. Materials and Methods
2.1. Participants
2.2. Chest CT Imaging Protocol
2.3. Artificial Intelligence Analysis
2.4. Statistical Analysis
3. Results
3.1. Study Population Characteristics
3.2. AI Model Performance
4. Discussion
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
Abbreviations
SARS-CoV-2 | severe acute respiratory syndrome coronavirus 2 |
COVID-19 | coronavirus disease 2019 |
WHO | World Health Organization |
RT-PCR | real-time reverse transcription polymerase chain reaction |
CT | computed tomography |
ICI | immune checkpoint inhibitors |
AI | artificial intelligence |
ROC | receiver operating characteristic |
AUC | area under the curve |
SPSS | Statistical Package for the Social Sciences |
References
- Fauci, A.S.; Lane, H.C.; Redfield, R.R. Covid-19 Navigating the Uncharted. N. Engl. J. Med. 2020, 382, 1268–1269. [Google Scholar] [CrossRef]
- Gorbalenya, A.E.; Baker, S.C.; Baric, R.S.; de Groot, R.J.; Drosten, C.; Gulyaeva, A.A.; Haagmans, B.L.; Lauber, C.; Leontovich, A.M.; Neuman, B.W.; et al. Severe acute respiratory syndrome-related coronavirus: The species and its viruses—A statement of the Coronavirus Study Group. BioRxiv 2020. [Google Scholar] [CrossRef] [Green Version]
- WHO Director-General’s Opening Remarks at the Media Briefing on COVID-192020 09/04/2020. Available online: https://www.who.int/dg/speeches/detail/who-director-general-s-opening-remarks-at-the-media-briefing-on-covid-19---11-march-2020 (accessed on 1 November 2020).
- Fehr, A.R.; Perlman, S. Coronaviruses: An overview of their replication and pathogenesis. Methods Mol. Biol. 2015, 1282, 1–23. [Google Scholar] [PubMed] [Green Version]
- CDC. Coronavirus Disease 2019 (COVID-19)2020 10/04/2020. Available online: https://www.cdc.gov/coronavirus/2019-ncov/hcp/clinical-criteria.html (accessed on 1 November 2020).
- Chung, M.; Bernheim, A.; Mei, X.; Zhang, N.; Huang, M.; Zeng, X.; Cui, J.; Xu, W.; Yang, Y.; Fayad, Z.A.; et al. CT Imaging Features of 2019 Novel Coronavirus (2019-nCoV). Radiology 2020, 295, 202–207. [Google Scholar] [CrossRef] [Green Version]
- Caruso, D.; Zerunian, M.; Polici, M.; Pucciarelli, F.; Polidori, T.; Rucci, C.; Guido, G.; Bracci, B.; De Dominicis, C.; Laghi, A. Chest CT Features of COVID-19 in Rome, Italy. Radiology 2020, 296, E79–E85. [Google Scholar] [CrossRef] [PubMed]
- Rubin, G.D.; Ryerson, C.J.; Haramati, L.B.; Sverzellati, N.; Kanne, J.P.; Raoof, S.; Schluger, N.W.; Volpi, A.; Yim, J.-J.; Martin, I.B.K.; et al. The Role of Chest Imaging in Patient Management during the COVID-19 Pandemic: A Multinational Consensus Statement from the Fleischner Society. Radiology 2020, 296, 172–180. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Li, L.; Qin, L.; Xu, Z.; Yin, Y.; Wang, X.; Kong, B.; Bai, J.; Lu, Y.; Fang, Z.; Song, Q.; et al. Artificial Intelligence Distinguishes COVID-19 from Community Acquired Pneumonia on Chest CT. Radiology 2020, 200905. [Google Scholar] [CrossRef]
- Kalisz, K.R.; Ramaiya, N.H.; Laukamp, K.R.; Gupta, A. Immune Checkpoint Inhibitor Therapy-related Pneumonitis: Patterns and Management. Radiographics 2019, 39, 1923–1937. [Google Scholar] [CrossRef] [PubMed]
- Granier, C.; De Guillebon, E.; Blanc, C.; Roussel, H.; Badoual, C.; Colin, E.; Saldmann, A.; Gey, A.; Oudard, S.; Tartour, E. Mechanisms of action and rationale for the use of checkpoint inhibitors in cancer. ESMO Open 2017, 2, e000213. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Martinez-Lostao, L.; Anel, A.; Pardo, J. How Do Cytotoxic Lymphocytes Kill Cancer Cells? Clin. Cancer Res. 2015, 21, 5047–5056. [Google Scholar] [CrossRef] [Green Version]
- Naidoo, J.; Wang, X.; Woo, K.M.; Iyriboz, T.; Halpenny, D.; Cunningham, J.; Chaft, J.E.; Segal, N.H.; Callahan, M.K.; Lesokhin, A.M.; et al. Pneumonitis in Patients Treated With Anti-Programmed Death-1/Programmed Death Ligand 1 Therapy. J. Clin. Oncol. 2017, 35, 709–717. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Nishino, M.; Giobbie-Hurder, A.; Hatabu, H.; Ramaiya, N.H.; Hodi, F.S. Incidence of Programmed Cell Death 1 Inhibitor-Related Pneumonitis in Patients With Advanced Cancer: A Systematic Review and Meta-analysis. JAMA Oncol. 2016, 2, 1607–1616. [Google Scholar] [CrossRef] [Green Version]
- Kaehler, K.C.; Piel, S.; Livingstone, E.; Schilling, B.; Hauschild, A.; Schadendorf, D. Update on immunologic therapy with anti-CTLA-4 antibodies in melanoma: Identification of clinical and biological response patterns, immune-related adverse events, and their management. Semin Oncol. 2010, 37, 485–498. [Google Scholar] [CrossRef]
- Stucci, S.; Palmirotta, R.; Passarelli, A.; Silvestris, E.; Argentiero, A.; Lanotte, L.; Acquafredda, S.; Todisco, A.; Silvestris, F. Immune-related adverse events during anticancer immunotherapy: Pathogenesis and management. Oncol. Lett. 2017, 14, 5671–5680. [Google Scholar] [CrossRef] [Green Version]
- Bersanelli, M. Controversies about COVID-19 and anticancer treatment with immune checkpoint inhibitors. Immunotherapy 2020, 12, 269–273. [Google Scholar] [CrossRef] [Green Version]
- Di Giambenedetto, S.; Ciccullo, A.; Borghetti, A.; Gambassi, G.; Landi, F.; Visconti, E.; Dal Verme, L.Z.; Bernabei, R.; Tamburrini, E.; Cauda, R.; et al. Off-label Use of Tocilizumab in Patients with SARS-CoV-2 Infection. J. Med. Virol. 2020, 92, 1787–1788. [Google Scholar] [CrossRef]
- Stroud, C.R.; Hegde, A.; Cherry, C.; Naqash, A.R.; Sharma, N.; Addepalli, S.; Cherukuri, S.; Parent, T.; Hardin, J.; Walker, P. Tocilizumab for the management of immune mediated adverse events secondary to PD-1 blockade. J. Oncol. Pharm. Pract. 2019, 25, 551–557. [Google Scholar] [CrossRef]
- Gabay, C.; Emery, P.; van Vollenhoven, R.; Dikranian, A.; Alten, R.; Pavelka, K.; Klearman, M.; Musselman, D.; Agarwal, S.; Green, J.; et al. Tocilizumab monotherapy versus adalimumab monotherapy for treatment of rheumatoid arthritis (ADACTA): A randomised, double-blind, controlled phase 4 trial. Lancet 2013, 381, 1541–1550. [Google Scholar] [CrossRef]
- Ronneberger, O.; Fischer, P.; Brox, T. U-Net: Convolutional Networks for Biomedical Image Segmentation. In Medical Image Computing and Computer-Assisted Intervention—MICCAI; Lecture Notes in Computer Science; Navab, N., Hornegger, J., Wells, W., Frangi, A., Eds.; Springer: Cham, Switzerland, 2015; p. 9351. [Google Scholar]
- Kermany, D.S.; Goldbaum, M.; Cai, W.; Valentim, C.C.S.; Liang, H.; Baxter, S.L.; McKeown, A.; Yang, G.; Wu, X.; Yan, F.; et al. Identifying Medical Diagnoses and Treatable Diseases by Image-Based Deep Learning. Cell 2018, 172, 1122–1131.e9. [Google Scholar] [CrossRef]
- Depeursinge, A.; Chin, A.S.; Leung, A.N.; Terrone, D.; Bristow, M.; Rosen, G.; Rubin, D.L. Automated classification of usual interstitial pneumonia using regional volumetric texture analysis in high-resolution computed tomography. Invest. Radiol. 2015, 50, 261–267. [Google Scholar] [CrossRef]
- Anthimopoulos, M.; Christodoulidis, S.; Ebner, L.; Christe, A.; Mougiakakou, S. Lung Pattern Classification for Interstitial Lung Diseases Using a Deep Convolutional Neural Network. IEEE Trans. Med. Imaging 2016, 35, 1207–1216. [Google Scholar] [CrossRef] [PubMed]
- Huang, L.; Han, R.; Ai, T.; Yu, P.; Kang, H.; Tao, Q.; Xia, L. Serial Quantitative Chest CT Assessment of COVID-19: Deep-Learning Approach. Radiol. Cardiothorac. Imaging 2020, 2, e200075. [Google Scholar] [CrossRef] [Green Version]
- Quattrocchi, C.C.; Mallio, C.A.; Presti, G.; Beomonte Zobel, B.; Cardinale, J.; Iozzino, M.; Della Sala, S.W. The challenge of COVID-19 low disease prevalence for artificial intelligence models: Report of 1,610 patients. Quant. Imaging Med. Surg. 2020, 10, 1891–1893. [Google Scholar] [CrossRef]
- Wang, M.; Xia, C.; Huang, L.; Xu, S.; Qin, C.; Liu, J.; Cao, Y.; Yu, P.; Zhu, T.; Zhu, H.; et al. Deep learning-based triage and analysis of lesion burden for COVID-19: A retrospective study with external validation. Lancet Digit. Health 2020, 10, e506–e515. [Google Scholar] [CrossRef]
- Simpson, S.; Kay, F.U.; Abbara, S.; Bhalla, S.; Chung, J.H.; Chung, M.; Henry, T.S.; Kanne, J.P.; Kligerman, S.; Ko, J.P.; et al. Radiological Society of North America Expert Consensus Statement on Reporting Chest CT Findings Related to COVID-19. Endorsed by the Society of Thoracic Radiology, the American College of Radiology, and RSNA. J. Thorac. Imaging 2020. [Google Scholar] [CrossRef]
- Yang, R.; Li, X.; Liu, H.; Zhen, Y.; Zhang, X.; Xiong, Q.; Luo, Y.; Gao, C.; Zeng, W. Chest CT Severity Score: An Imaging Tool for Assessing Severe COVID-19. Radiol. Cardiothorac. Imaging 2020, 2, e200047. [Google Scholar] [CrossRef] [Green Version]
- Choi, H.; Qi, X.; Yoon, S.H.; Park, S.J.; Lee, K.H.; Kim, J.Y.; Lee, Y.K.; Ko, H.; Kim, K.H.; Park, C.M.; et al. Extension of Coronavirus Disease 2019 (COVID-19) on Chest CT and Implications for Chest Radiograph Interpretation. Radiol. Cardiothorac. Imaging 2020, 2, e204001. [Google Scholar] [CrossRef] [Green Version]
- Wang, Y.; Dong, C.; Hu, Y.; Li, C.; Ren, Q.; Zhang, X.; Shi, H.; Zhou, M. Temporal Changes of CT Findings in 90 Patients with COVID-19 Pneumonia: A Longitudinal Study. Radiology 2020, 296, E55–E64. [Google Scholar] [CrossRef] [Green Version]
- Hanley, B.; Lucas, S.B.; Youd, E.; Swift, B.; Osborn, M. Autopsy in suspected COVID-19 cases. J. Clin. Pathol. 2020, 73, 239–242. [Google Scholar] [CrossRef] [Green Version]
- Porcu, M.; De Silva, P.; Solinas, C.; Battaglia, A.; Schena, M.; Scartozzi, M.; Bron, D.; Suri, J.S.; Willard-Gallo, K.; Sangiolo, D.; et al. Immunotherapy Associated Pulmonary Toxicity: Biology Behind Clinical and Radiological Features. Cancers 2019, 11, 305. [Google Scholar] [CrossRef] [Green Version]
- Russano, M.; Citarella, F.; Napolitano, A.; Dell’Aquila, E.; Cortellini, A.; Pantano, F.; Vencenzi, B.; Tonini, G.; Santini, D. COVID-19 pneumonia and immune-related pneumonitis: Critical issues on differential diagnosis, potential interactions, and management. Expert Opin. Biol. Ther. 2020, 20, 959–964. [Google Scholar] [CrossRef] [PubMed]
- Mallio, C.A.; Quattrocchi, C.C.; Zobel, B.B.; Parizel, P.M. Artificial intelligence, chest radiographs, and radiology trainees: A powerful combination to enhance the future of radiologists? Quant. Imaging Med. Surg. 2021. [Google Scholar] [CrossRef]
Property | COVID-19 | ICI Therapy-Related | Pneumonia-Free |
---|---|---|---|
Patients (n) | 34 | 21 | 30 |
Female/male (n) | 15/19 | 7/14 | 13/17 |
Age (years) | 67 (38–87) | 72 (46–82) | 59 (32–88) |
SARS-CoV-2 RT-PCR (positive/negative/n.a.) | 34/0/0 | 0/0/21 | 0/14/16 |
Primary cancer | n/a | NSCLC (n = 17) | n/a |
ccRCC (n = 1) | |||
Breast cancer (n = 1) | |||
HCC (n = 1) | |||
SCLC (n = 1) | |||
Immunotherapy drug | n/a | Nivolumab (n = 7) | n/a |
Pembrolizumab (n = 12) | |||
Atezolizumab (n = 2) |
Group | Sensitivity % | Specificity % | PPV % | NPV % | Acc % | AUC |
---|---|---|---|---|---|---|
COVID-19 vs. pneumonia-free | 97.1 (88.6, 97.1) | 100 (90.4, 100) | 100 (91.2, 100) | 96.8 (87.5, 96.8) | 98.4 | 0.99 (0.98, 1.00) |
COVID-19 vs. ICI therapy-related | 97.1 (90.9, 99.8) | 14.3 (4.3, 18.8) | 64.7 (60.6, 66.6) | 75.0 (22.6, 98.7) | 60.0 | 0.62 (0.55, 0.70) |
ICI therapy-related vs. pneumonia-free | 85.7 (71.3, 85.7) | 100 (89.9, 100) | 100 (83.2, 100) | 90.9 (81.7, 90.9) | 94.1 | 0.97 (0.91, 1.00) |
ICI Therapy | COVID-19 | ICI Therapy-Related | p-Value | |
---|---|---|---|---|
TOTAL | (%) | 2.95 (1.22–8.89) | 1.68 (0.28–9.62) | 0.27 |
(cm3) | 105.54 (44.68–257.07) | 52.03 (6.95–225.0) | 0.14 | |
RUL | (%) | 0.78 (0.15–5.12) | 0.17 (0–10.84) | 0.43 |
(cm3) | 7.3 (1.21–31.42) | 1.92 (0–14.22) | 0.17 | |
ML | (%) | 0.24 (0–3.89) | 0.28 (0–3.47) | 0.66 |
(cm3) | 1.01 (0–7.92) | 1.00 (0–13.58) | 0.61 | |
RLL | (%) | 3.54 (1.19–11.06) | 1.48 (0.02–11.80) | 0.09 |
(cm3) | 27.14 (8.20–83.30) | 4.83 (0–41.02) | 0.05 | |
LUL | (%) | 0.73 (0.05–5.70) | 0 (0–2.81) | 0.10 |
(cm3) | 7.22 (0.84–54.28) | 0 (0–15.42) | 0.04 | |
LLL | (%) | 3.99 (0.46–17.56) | 0 (0–2.10) | <0.01 |
(cm3) | 16.35 (3.66–85.61) | 0 (0–10.97) | <0.01 | |
COVID-19 RISK (%) | 41.85 (34.52–51.12) | 34.4 (27.2–46.4) | 0.16 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Mallio, C.A.; Napolitano, A.; Castiello, G.; Giordano, F.M.; D'Alessio, P.; Iozzino, M.; Sun, Y.; Angeletti, S.; Russano, M.; Santini, D.; et al. Deep Learning Algorithm Trained with COVID-19 Pneumonia Also Identifies Immune Checkpoint Inhibitor Therapy-Related Pneumonitis. Cancers 2021, 13, 652. https://doi.org/10.3390/cancers13040652
Mallio CA, Napolitano A, Castiello G, Giordano FM, D'Alessio P, Iozzino M, Sun Y, Angeletti S, Russano M, Santini D, et al. Deep Learning Algorithm Trained with COVID-19 Pneumonia Also Identifies Immune Checkpoint Inhibitor Therapy-Related Pneumonitis. Cancers. 2021; 13(4):652. https://doi.org/10.3390/cancers13040652
Chicago/Turabian StyleMallio, Carlo Augusto, Andrea Napolitano, Gennaro Castiello, Francesco Maria Giordano, Pasquale D'Alessio, Mario Iozzino, Yipeng Sun, Silvia Angeletti, Marco Russano, Daniele Santini, and et al. 2021. "Deep Learning Algorithm Trained with COVID-19 Pneumonia Also Identifies Immune Checkpoint Inhibitor Therapy-Related Pneumonitis" Cancers 13, no. 4: 652. https://doi.org/10.3390/cancers13040652