Contrast-Enhancing Lesion Segmentation in Multiple Sclerosis: A Deep Learning Approach Validated in a Multicentric Cohort
Abstract
:1. Introduction
2. Materials and Methods
2.1. Dataset
2.2. Preprocessing and Sampling Strategy
- -
- For scans with CELs: 32 patches of 64 × 64 × 64 mm3 were randomly cropped, with one out of every three patches centered on CELs (positive patches) and the remaining two centered on WMLs (negative patches).
- -
- For scans without CELs: 32 patches of 64 × 64 × 64 mm3 were randomly cropped, all centered on WMLs.
2.3. Network Architecture
2.4. Metrics
2.5. Training Pipeline
3. Results
3.1. Comparative Analysis of Loss Functions
3.2. Model Performance
4. Discussion
5. Conclusions
Supplementary Materials
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
Appendix A
Center | MRI Scanner Model | Number of Scans | Sequence | Resolution (mm3) | Flip Angle (degree) | Repetition Time (ms) | Echo Time (ms) | Inversion Time (ms) |
---|---|---|---|---|---|---|---|---|
FLAIR | 1 × 1 × 1 | 120 | 5000 | 398 | 1800 | |||
Lausanne | SIEMENS Skyra 3T | 51 | T1n | 1 × 1 × 1.2 | 9 | 2300 | 2.9 | 900 |
T1ce | 1 × 1 × 1 | 9 | 2000 | 2.03 | 1100 | |||
FLAIR | 1 × 1 × 1 | 120 | 5000 | 395 | 1600 | |||
Geneva | SIEMENS Skyra 3T | 16 | T1n | 1 × 1 × 1 | 9 | 2000 | 2.03 | 1100 |
T1ce | 1 × 1 × 1 | 9 | 2000 | 2.03 | 1100 | |||
FLAIR | 1 × 1 × 1 | 120 | 5000 | 335 | 1800 | |||
Bern | SIEMENS Skyra 3T | 26 | T1n | 1 × 1 × 1 | 15 | 1790 | 2.58 | 1100 |
T1ce | 1 × 1 × 1 | 15 | 2060 | 53 | 1100 | |||
FLAIR | 1 × 1 × 1 | 120 | 5000 | 280 | 1600 | |||
Basel | SIEMENS Skyra 3T | 245 | T1n | 1 × 1 × 1 | 9 | 2300 | 1.96 | 900 |
T1ce | 1 × 1 × 3 | 30 | 30 | 11 | 0 | |||
FLAIR | 1 × 1 × 1.3 | 120 | 7500 | 317 | 3000 | |||
Aarau | SIEMENS Skyra 3T | 14 | T1n | 1 × 1 × 1 | 15 | 1970 | 3.14 | 1100 |
T1ce | 1 × 1 × 1 | 9 | 2100 | 4.78 | 1800 | |||
FLAIR | 1 × 1 × 1 | 120 | 5000 | 373 | 900 | |||
Lugano | SIEMENS Skyra 3T | 7 | T1n | 1 × 1 × 1 | 9 | 2300 | 2.98 | 950 |
T1ce | 1 × 1 × 1 | 120 | 600 | 11 | 0 | |||
FLAIR | 1 × 1 × 1 | 120 | 6000 | 355 | 1850 | |||
St. Gallen | SIEMENS Avanto 1.5T | 13 | T1n | 1 × 1 × 1 | 8 | 2700 | 2.96 | 950 |
T1ce | 1 × 1 × 1 | 8 | 2700 | 2.96 | 950 |
References
- Lassmann, H.; van Horssen, J.; Mahad, D. Progressive multiple sclerosis: Pathology and pathogenesis. Nat. Rev. Neurol. 2012, 8, 647–656. [Google Scholar] [CrossRef] [PubMed]
- Brück, W. The pathology of multiple sclerosis is the result of focal inflammatory demyelination with axonal damage. J. Neurol. 2005, 252, v3–v9. [Google Scholar] [CrossRef]
- Miller, D.H.; Barkhof, F.; Nauta, J.J.P. Gadolinium enhancement increases the sensitivity of MRI in detecting disease activity in multiple sclerosis. Brain 1993, 116, 1077–1094. [Google Scholar] [CrossRef]
- Granziera, C.; Reich, D.S. Gadolinium should always be used to assess disease activity in MS—Yes. Mult. Scler. J. 2020, 26, 765–766. [Google Scholar] [CrossRef] [PubMed]
- Montalban, X.; Gold, R.; Thompson, A.J.; Otero-Romero, S.; Amato, M.P.; Chandraratna, D.; Clanet, M.; Comi, G.; Derfuss, T.; Fazekas, F.; et al. ECTRIMS/EAN Guideline on the pharmacological treatment of people with multiple sclerosis. Mult. Scler. J. 2018, 24, 96–120. [Google Scholar] [CrossRef]
- Kira, J.-I. Redefining use of MRI for patients with multiple sclerosis. Lancet Neurol. 2021, 20, 591–592. [Google Scholar] [CrossRef] [PubMed]
- Guo, B.J.; Yang, Z.L.; Zhang, L.J. Gadolinium Deposition in Brain: Current Scientific Evidence and Future Perspectives. Front. Mol. Neurosci. 2018, 11, 335. [Google Scholar] [CrossRef] [PubMed]
- Tsantes, E.; Curti, E.; Ganazzoli, C.; Puci, F.; Bazzurri, V.; Fiore, A.; Crisi, G.; Granella, F. The contribution of enhancing lesions in monitoring multiple sclerosis treatment: Is gadolinium always necessary? J. Neurol. 2020, 267, 2642–2647. [Google Scholar] [CrossRef]
- Wattjes, M.P.; Ciccarelli, O.; Reich, D.S.; Banwell, B.; de Stefano, N.; Enzinger, C.; Fazekas, F.; Filippi, M.; Frederiksen, J.; Gasperini, C.; et al. 2021 MAGNIMS–CMSC–NAIMS consensus recommendations on the use of MRI in patients with multiple sclerosis. Lancet Neurol. 2021, 20, 653–670. [Google Scholar] [CrossRef] [PubMed]
- Lesjak, Ž.; Galimzianova, A.; Koren, A.; Lukin, M.; Pernuš, F.; Likar, B.; Špiclin, Ž. A Novel Public MR Image Dataset of Multiple Sclerosis Patients With Lesion Segmentations Based on Multi-rater Consensus. Neuroinformatics 2018, 16, 51–63. [Google Scholar] [CrossRef]
- Mortazavi, D.; Kouzani, A.Z.; Soltanian-Zadeh, H. Segmentation of multiple sclerosis lesions in MR images: A review. Neuroradiology 2012, 54, 299–320. [Google Scholar] [CrossRef]
- Filippi, M.; Preziosa, P.; Banwell, B.L.; Barkhof, F.; Ciccarelli, O.; De Stefano, N.; Geurts, J.J.G.; Paul, F.; Reich, D.S.; Toosy, A.T.; et al. Assessment of lesions on magnetic resonance imaging in multiple sclerosis: Practical guidelines. Brain 2019, 142, 1858–1875. [Google Scholar] [CrossRef]
- Danieli, L.; Roccatagliata, L.; Distefano, D.; Prodi, E.; Riccitelli, G.; Diociasi, A.; Carmisciano, L.; Cianfoni, A.; Bartalena, T.; Kaelin-Lang, A.; et al. Nonlesional Sources of Contrast Enhancement on Postgadolinium “Black-Blood” 3D T1-SPACE Images in Patients with Multiple Sclerosis. Am. J. Neuroradiol. 2022, 43, 872–880. [Google Scholar] [CrossRef]
- Lundervold, A.S.; Lundervold, A. An overview of deep learning in medical imaging focusing on MRI. Z. Fur Med. Phys. 2019, 29, 102–127. [Google Scholar] [CrossRef]
- Kayalibay, B.; Jensen, G.; van der Smagt, P. CNN-based Segmentation of Medical Imaging Data. arXiv 2017, arXiv:1701.03056. [Google Scholar]
- Gaj, S.; Ontaneda, D.; Nakamura, K. Automatic segmentation of gadolinium-enhancing lesions in multiple sclerosis using deep learning from clinical MRI. PLoS ONE 2021, 16, e0255939. [Google Scholar] [CrossRef] [PubMed]
- Coronado, I.; Gabr, R.E.; Narayana, P.A. Deep learning segmentation of gadolinium-enhancing lesions in multiple sclerosis. Mult. Scler. J. 2020, 27, 519–527. [Google Scholar] [CrossRef] [PubMed]
- Krishnan, A.P.; Song, Z.; Clayton, D.; Gaetano, L.; Jia, X.; de Crespigny, A.; Bengtsson, T.; Carano, R.A.D. Joint MRI T1 Unenhancing and Contrast-enhancing Multiple Sclerosis Lesion Segmentation with Deep Learning in OPERA Trials. Radiology 2022, 302, 662–673. [Google Scholar] [CrossRef]
- Schlaeger, S.; Shit, S.; Eichinger, P.; Hamann, M.; Opfer, R.; Krüger, J.; Dieckmeyer, M.; Schön, S.; Mühlau, M.; Zimmer, C.; et al. AI-based detection of contrast-enhancing MRI lesions in patients with multiple sclerosis. Insights Into Imaging 2023, 14, 123. [Google Scholar] [CrossRef] [PubMed]
- Weiss, K.; Khoshgoftaar, T.M.; Wang, D.D. A survey of transfer learning. J. Big Data 2016, 3, 1345–1459. [Google Scholar] [CrossRef]
- Wahlig, S.G.; Nedelec, P.; Weiss, D.A.; Rudie, J.D.; Sugrue, L.P.; Rauschecker, A.M. 3D U-Net for automated detection of multiple sclerosis lesions: Utility of transfer learning from other pathologies. Front. Neurosci. 2023, 17, 1188336. [Google Scholar] [CrossRef] [PubMed]
- Huang, L.; Zhao, Z.; An, L.; Gong, Y.; Wang, Y.; Yang, Q.; Wang, Z.; Hu, G.; Wang, Y.; Guo, C. 2.5D transfer deep learning model for segmentation of contrast-enhancing lesions on brain magnetic resonance imaging of multiple sclerosis and neuromyelitis optica spectrum disorder. Quant. Imaging Med. Surg. 2024, 14, 273–290. [Google Scholar] [CrossRef] [PubMed]
- Disanto, G.; Benkert, P.; Lorscheider, J.; Mueller, S.; Vehoff, J.; Zecca, C.; Ramseier, S.; Achtnichts, L.; Findling, O.; Nedeltchev, K.; et al. The Swiss Multiple Sclerosis Cohort-Study (SMSC): A Prospective Swiss Wide Investigation of Key Phases in Disease Evolution and New Treatment Options. PLoS ONE 2016, 11, e0152347. [Google Scholar] [CrossRef]
- La Rosa, F.; Abdulkadir, A.; Fartaria, M.J.; Rahmanzadeh, R.; Lu, P.-J.; Galbusera, R.; Barakovic, M.; Thiran, J.-P.; Granziera, C.; Cuadra, M.B. Multiple sclerosis cortical and WM lesion segmentation at 3T MRI: A deep learning method based on FLAIR and MP2RAGE. NeuroImage Clin. 2020, 27, 102335. [Google Scholar] [CrossRef]
- Khaleeli, Z.; Ciccarelli, O.; Mizskiel, K.; Altmann, D.; Miller, D.; Thompson, A. Lesion enhancement diminishes with time in primary progressive multiple sclerosis. Mult. Scler. J. 2010, 16, 317–324. [Google Scholar] [CrossRef] [PubMed]
- Klein, S.; Staring, M.; Murphy, K.; Viergever, M.A.; Pluim, J.P.W. elastix: A Toolbox for Intensity-Based Medical Image Registration. IEEE Trans. Med. Imaging 2009, 29, 196–205. [Google Scholar] [CrossRef]
- Isensee, F.; Schell, M.; Pflueger, I.; Brugnara, G.; Bonekamp, D.; Neuberger, U.; Wick, A.; Schlemmer, H.-P.; Heiland, S.; Wick, W.; et al. Automated brain extraction of multisequence MRI using artificial neural networks. Hum. Brain Mapp. 2019, 40, 4952–4964. [Google Scholar] [CrossRef]
- Chlap, P.; Min, H.; Vandenberg, N.; Dowling, J.; Holloway, L.; Haworth, A. A review of medical image data augmentation techniques for deep learning applications. J. Med. Imaging Radiat. Oncol. 2021, 65, 545–563. [Google Scholar] [CrossRef] [PubMed]
- Rice, L.; Wong, E.; Kolter, J.Z. Overfitting in Adversarially Robust Deep Learning. 2020. Available online: https://github.com/ (accessed on 10 August 2023).
- Cardoso, M.J.; Li, W.; Brown, R.; Ma, N.; Kerfoot, E.; Wang, Y.; Murrey, B.; Myronenko, A.; Zhao, C.; Yang, D. MONAI: An open-source framework for deep learning in healthcare. arXiv 2022, arXiv:2211.02701. [Google Scholar]
- La Rosa, F.; Beck, E.S.; Maranzano, J.; Todea, R.; van Gelderen, P.; de Zwart, J.A.; Luciano, N.J.; Duyn, J.H.; Thiran, J.; Granziera, C.; et al. Multiple sclerosis cortical lesion detection with deep learning at ultra-high-field MRI. NMR Biomed. 2022, 35, e4730. [Google Scholar] [CrossRef]
- ECTRIMS 2019—Poster Session 1. Mult. Scler. J. 2019, 25, 131–356. [CrossRef]
- Lin, T.Y.; Goyal, P.; Girshick, R.; He, K.; Dollar, P. Focal loss for dense object detection. IEEE Trans. Pattern Anal. Mach. Intell. 2020, 42, 318–327. [Google Scholar] [CrossRef] [PubMed]
- Ma, Y.-d.; Liu, Q.; Qian, Z.-B. Automated Image Segmentation Using Improved PCNN Model Based on Cross-entropy. In Proceedings of the 2004 International Symposium on Intelligent Multimedia, Video and Speech Processing, Hong Kong, China, 20–22 October 2004. [Google Scholar]
- Sudre, C.H.; Li, W.; Vercauteren, T.; Ourselin, S.; Cardoso, M.J. Generalised dice overlap as a deep learning loss function for highly unbalanced segmentations. In Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Springer International Pubishing: Berlin/Heidelberg, Germany, 2017. [Google Scholar]
- Kingma, D.P.; Ba, J. Adam: A Method for Stochastic Optimization. arXiv 2014, arXiv:1412.6980. [Google Scholar]
- Krishnan, R.; Rajpurkar, P.; Topol, E.J. Self-supervised learning in medicine and healthcare. Nat. Biomed. Eng. 2022, 6, 1346–1352. [Google Scholar] [CrossRef] [PubMed]
- Imran, S.M.A.; Saleem, M.W.; Hameed, M.T.; Hussain, A.; Naqvi, R.A.; Lee, S.W. Feature preserving mesh network for semantic segmentation of retinal vasculature to support ophthalmic disease analysis. Front. Med. 2023, 9, 1040562. [Google Scholar] [CrossRef] [PubMed]
Number of Patients with MS | Number of MRI Scans | Number of CELs | Mean Number of CELs per Scan | Mean Volume of CELs (mm3) | |
---|---|---|---|---|---|
Data with CELs | 162 | 208 | 654 | 3 | 136 |
Data without CELs(control cases) | 118 | 164 | 0 | 0 | - |
Total | 280 | 372 | 654 | 1.7 |
Demographic and Clinical Data | n = 372 MRI Scans |
---|---|
Female, No. (%) | 264 (71) |
Male, No. (%) | 108 (29) |
Age at closest visit, mean (SD), y | 41.6 (11.5) |
Disease duration at closest visit, mean (SD), y | 11.8 (9.4) |
EDSS at closest visit, median (IQR) | 2.0 (1.5, 3.5) |
CIS, No. (%) | 7 (1.9) |
PPMS, No. (%) | 7 (1.9) |
RRMS, No. (%) | 336 (91.1) |
SPMS, No. (%) | 19 (5.1) |
Dataset | Loss Function | DSC in Patches | DSC in Whole Images | Number of TP Lesions | Number of FN Lesions | Number of FP Lesions |
---|---|---|---|---|---|---|
Validation | Starting | 0.78 | 0.80 | 39 | 8 | 5 |
Validation | Weighted | 0.78 | 0.82 | 40 | 7 | 6 |
Test | Starting | 0.67 | 0.72 | 50 | 10 | 3 |
Test | Weighted | 0.72 | 0.76 | 56 | 4 | 1 |
Lesion Volume (mm3) | Number of TP Lesions | Number of FN Lesions | Number of FP Lesions | Dice Score Coefficient |
---|---|---|---|---|
3–10 | 6 | 3 | 1 | 0.88 |
10–20 | 5 | 1 | 0 | 0.56 |
20–30 | 6 | 0 | 0 | 0.73 |
30–40 | 4 | 0 | 0 | 0.82 |
40–50 | 3 | 0 | 0 | 0.61 |
50–100 | 16 | 0 | 0 | 0.79 |
100–200 | 11 | 0 | 0 | 0.82 |
200–300 | 1 | 0 | 0 | 0.85 |
>300 | 4 | 0 | 0 | 0.83 |
All | 56 | 4 | 1 | 0.76 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Greselin, M.; Lu, P.-J.; Melie-Garcia, L.; Ocampo-Pineda, M.; Galbusera, R.; Cagol, A.; Weigel, M.; de Oliveira Siebenborn, N.; Ruberte, E.; Benkert, P.; et al. Contrast-Enhancing Lesion Segmentation in Multiple Sclerosis: A Deep Learning Approach Validated in a Multicentric Cohort. Bioengineering 2024, 11, 858. https://doi.org/10.3390/bioengineering11080858
Greselin M, Lu P-J, Melie-Garcia L, Ocampo-Pineda M, Galbusera R, Cagol A, Weigel M, de Oliveira Siebenborn N, Ruberte E, Benkert P, et al. Contrast-Enhancing Lesion Segmentation in Multiple Sclerosis: A Deep Learning Approach Validated in a Multicentric Cohort. Bioengineering. 2024; 11(8):858. https://doi.org/10.3390/bioengineering11080858
Chicago/Turabian StyleGreselin, Martina, Po-Jui Lu, Lester Melie-Garcia, Mario Ocampo-Pineda, Riccardo Galbusera, Alessandro Cagol, Matthias Weigel, Nina de Oliveira Siebenborn, Esther Ruberte, Pascal Benkert, and et al. 2024. "Contrast-Enhancing Lesion Segmentation in Multiple Sclerosis: A Deep Learning Approach Validated in a Multicentric Cohort" Bioengineering 11, no. 8: 858. https://doi.org/10.3390/bioengineering11080858