RFS+: A Clinically Adaptable and Computationally Efficient Strategy for Enhanced Brain Tumor Segmentation
Abstract
:Simple Summary
Abstract
1. Introduction
- This paper, for the first time in the literature, provides a thorough investigation of different normalization techniques of MRI scans on segmentation tasks for DL models;
- A novel strategy called RFS+ is introduced as a versatile solution for any DL model, optimizing brain tumor segmentation through a fusion of various segmentation approaches, and normalization techniques with ensemble learning, thereby enhancing accuracy and generalizability across different datasets;
- For each region, RFS+ method provides the best DSC score by investigating the effect of normalization techniques on U-net models. It helps in the selection of the best method for each region when the aim is to use only one model. For example, transferring the trained models with one approach (such as multi-class) on ET, TC, and WT to segment GTVs cannot always give the best results. In contrast, RFS+ gives the best model for specific contours when transferring information from one contour style (TC) to another (GTV) [12];
- RFS+ offers ensemble learning by using models with the top three DSC scores on the training dataset from the proposed models. The segmentation outcomes, when utilizing a 2D U-net model through ensemble learning, outperform those of the state-of-the-art model, indicating a substantial enhancement in segmentation accuracy provided by RFS+. Additionally, the introduction of RFS+ led to a DSC enhancement up to 1% when compared to its predecessor;
- A state-of-the-art model, with its original Docker image that triumphed in the BraTS 2021 challenge, was evaluated using a local dataset, addressing a notable gap as most models, trained on the BraTS training dataset, have predominantly been tested on the BraTS validation and test datasets, leaving the exploration on local datasets largely untouched. This study seeks to illuminate the generalizability of DL models by showcasing the segmentation results of a state-of-the-art DL model on local datasets, serving as a pivotal guide for future research and applications in this domain.
2. Related Work
3. Method
3.1. The Proposed Strategy: (RFS+)
3.2. Normalization of MRI Scans
3.3. Network Architectures
3.3.1. Segmentation Approaches
3.3.2. Loss Function
3.4. Dataset
3.5. Data Pre-Processing
3.6. The Details of the Implementation
4. Results and Discussion
4.1. Model Selection via the BraTS 2021 Dataset
4.2. Benchmarking RFS+ Method: Comparative Study to the BraTS 2021 Winner Model
4.3. Ablation Study
4.4. Validating RFS+ on Local Dataset
5. Conclusions
Supplementary Materials
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Sung, H.; Ferlay, J.; Siegel, R.L.; Laversanne, M.; Soerjomataram, I.; Jemal, A.; Bray, F. Global Cancer Statistics 2020: GLOBOCAN Estimates of Incidence and Mortality Worldwide for 36 Cancers in 185 Countries. CA Cancer J. Clin. 2021, 71, 209–249. [Google Scholar] [CrossRef] [PubMed]
- Siegel, R.L.; Miller, K.D.; Jemal, A. Cancer statistics, 2019. CA Cancer J. Clin. 2019, 69, 7–34. [Google Scholar] [CrossRef] [PubMed]
- Ostrom, Q.T.; Gittleman, H.; Truitt, G.; Boscia, A.; Kruchko, C.; Barnholtz-Sloan, J.S. CBTRUS Statistical Report: Primary Brain and Other Central Nervous System Tumors Diagnosed in the United States in 2011–2015. Neuro Oncol. 2018, 20, iv1–iv86. [Google Scholar] [CrossRef] [PubMed]
- Louis, D.N.; Perry, A.; Wesseling, P.; Brat, D.J.; Cree, I.A.; Figarella-Branger, D.; Hawkins, C.; Ng, H.K.; Pfister, S.M.; Reifenberger, G.; et al. The 2021 WHO Classification of Tumors of the Central Nervous System: A summary. Neuro Oncol. 2021, 23, 1231–1251. [Google Scholar] [CrossRef] [PubMed]
- Levine, H., III. Medical Imaging; Bloomsbury Publishing USA: New York, NY, USA, 2010; Available online: http://ebookcentral.proquest.com/lib/cardiff/detail.action?docID=617068 (accessed on 15 October 2023).
- Kamnitsas, K.; Ledig, C.; Newcombe, V.F.; Simpson, J.P.; Kane, A.D.; Menon, D.K.; Rueckert, D.; Glocker, B. Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation. Med. Image Anal. 2017, 36, 61–78. [Google Scholar] [CrossRef]
- van Timmeren, J.E.; Cester, D.; Tanadini-Lang, S.; Alkadhi, H.; Baessler, B. Radiomics in medical imaging—‘how-to’ guide and critical reflection. Insights Imaging 2020, 11, 91. [Google Scholar] [CrossRef]
- Ghaffari, M.; Sowmya, A.; Oliver, R. Automated Brain Tumor Segmentation Using Multimodal Brain Scans: A Survey Based on Models Submitted to the BraTS 2012–2018 Challenges. IEEE Rev. Biomed. Eng. 2019, 13, 156–168. [Google Scholar] [CrossRef]
- Bengio, Y.; Courville, A.; Vincent, P. Representation learning: A review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intell. 2013, 35, 1798–1828. [Google Scholar] [CrossRef]
- Pereira, S.; Pinto, A.; Alves, V.; Silva, C.A. Brain tumor segmentation using convolutional neural networks in MRI images. IEEE Trans. Med. Imaging 2016, 35, 1240–1251. [Google Scholar] [CrossRef]
- Jones, D. ICRU Report 50—Prescribing, Recording and Reporting Photon Beam Therapy. Med. Phys. 1994, 21, 833–834. [Google Scholar] [CrossRef]
- Duman, A.; Whybra, P.; Powell, J.; Thomas, S.; Sun, X.; Spezi, E. PO-1620 Transferability of deep learning models to the segmentation of gross tumour volume in brain cancer. Radiother. Oncol. 2023, 182, S1315–S1316. [Google Scholar] [CrossRef]
- Wen, P.Y.; Chang, S.M.; Bent, M.J.V.D.; Vogelbaum, M.A.; Macdonald, D.R.; Lee, E.Q. Response Assessment in Neuro-Oncology Clinical Trials. J. Clin. Oncol. 2017, 35, 2439–2449. [Google Scholar] [CrossRef]
- Wang, R.; Lei, T.; Cui, R.; Zhang, B.; Meng, H.; Nandi, A.K. Medical image segmentation using deep learning: A survey. IET Image Process. 2022, 16, 1243–1267. [Google Scholar] [CrossRef]
- Maqsood, S.; Damasevicius, R.; Shah, F.M. An Efficient Approach for the Detection of Brain Tumor Using Fuzzy Logic and U-NET CNN Classification. In Computational Science and Its Applications—ICCSA 2021; Gervasi, O., Murgante, B., Misra, S., Garau, C., Blečić, I., Taniar, D., Apduhan, B.O., Rocha, A.M.A.C., Tarantino, E., Torre, C.M., Eds.; Springer International Publishing: Cham, Switzerland, 2021; pp. 105–118. [Google Scholar]
- Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A.N.; Kaiser, Ł.; Polosukhin, I. Attention is all you need. In Proceedings of the Advances in Neural Information Processing Systems 30 (NIPS 2017), Long Beach, CA, USA, 4–9 December 2017; Volume 30. [Google Scholar]
- Liu, Z.; Lv, Q.; Yang, Z.; Li, Y.; Lee, C.H.; Shen, L. Recent progress in transformer-based medical image analysis. Comput. Biol. Med. 2023, 164, 107268. [Google Scholar] [CrossRef]
- Qin, L.; Zhao, H.; Zhang, S.; Tang, Z. Automated brain tumor segmentation using cascaded bootstrapping model. In Eleventh International Conference on Graphics and Image Processing (ICGIP 2019); SPIE: Bellingham, WA, USA, 2020; pp. 356–360. [Google Scholar]
- Beers, A.; Chang, K.; Brown, J.; Sartor, E.; Mammen, C.P.; Gerstner, E.; Rosen, B.; Kalpathy-Cramer, J. Sequential 3d u-nets for biologically-informed brain tumor segmentation. arXiv 2017, arXiv:1709.02967. [Google Scholar]
- Hashemi, N.; Masoudnia, S.; Nejad, A.; Nazem-Zadeh, M.-R. A Memory-efficient Deep Framework for Multi-Modal MRI-based Brain Tumor Segmentation. In Proceedings of the 2022 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), Glasgow, UK, 11–15 July 2022; pp. 3749–3752. [Google Scholar]
- Baid, U.; Ghodasara, S.; Mohan, S.; Bilello, M.; Calabrese, E.; Colak, E.; Farahani, K.; Kalpathy-Cramer, J.; Kitamura, F.C.; Pati, S.; et al. The rsna-asnr-miccai brats 2021 benchmark on brain tumor segmentation and radiogenomic classification. arXiv 2021, arXiv:2107.02314. [Google Scholar]
- Wang, G.; Li, W.; Ourselin, S.; Vercauteren, T. Automatic Brain Tumor Segmentation Using Cascaded Anisotropic Convolutional Neural Networks. In Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries; Crimi, A., Bakas, S., Kuijf, H., Menze, B., Reyes, M., Eds.; Springer International Publishing: Cham, Switzerland, 2018; pp. 178–190. [Google Scholar]
- Myronenko, A. 3D MRI Brain Tumor Segmentation Using Autoencoder Regularization. In Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries; Crimi, A., Bakas, S., Kuijf, H., Keyvan, F., Reyes, M., van Walsum, T., Eds.; Springer International Publishing: Cham, Switzerland, 2019; pp. 311–320. [Google Scholar]
- Jiang, Z.; Ding, C.; Liu, M.; Tao, D. Two-Stage Cascaded U-Net: 1st Place Solution to BraTS Challenge 2019 Segmentation Task. In Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries; Crimi, A., Bakas, S., Eds.; Springer International Publishing: Cham, Switzerland, 2020; pp. 231–241. [Google Scholar]
- Isensee, F.; Jäger, P.F.; Full, P.M.; Vollmuth, P.; Maier-Hein, K.H. nnU-net for brain tumor segmentation. In International MICCAI Brainlesion Workshop; Springer: Berlin/Heidelberg, Germany, 2020; pp. 118–132. [Google Scholar]
- Berkley, A.; Saueressig, C.; Shukla, U.; Chowdhury, I.; Munoz-Gauna, A.; Shehu, O.; Singh, R.; Munbodh, R. Clinical capability of modern brain tumor segmentation models. Med. Phys. 2023, 50, 4943–4959. [Google Scholar] [CrossRef] [PubMed]
- Leithner, D.; Nevin, R.B.; Gibbs, P.; Weber, M. ComBat Harmonization for MRI Radiomics: Impact on Nonbinary Tissue Classification by Machine Learning. Invest. Radiol. 2023, 58, 697–701. [Google Scholar] [CrossRef] [PubMed]
- Ronneberger, O.; Fischer, P.; Brox, T. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention; Springer: Berlin/Heidelberg, Germany, 2015; pp. 234–241. [Google Scholar]
- Luu, H.M.; Park, S.-H. Extending nn-UNet for Brain Tumor Segmentation. In Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries; Crimi, A., Bakas, S., Eds.; Springer International Publishing: Cham, Switzerland, 2022; pp. 173–186. [Google Scholar]
- Reinhold, J.C.; Dewey, B.E.; Carass, A.; Prince, J.L. Prince, Evaluating the impact of intensity normalization on MR image synthesis. In Medical Imaging 2019: Image Processing; SPIE: Bellingham, WA, USA, 2019; pp. 890–898. [Google Scholar]
- Magadza, T.; Viriri, S. Deep learning for brain tumor segmentation: A survey of state-of-the-art. J. Imaging 2021, 7, 19. [Google Scholar] [CrossRef] [PubMed]
- Casamitjana, A.; Puch, S.; Aduriz, A.; Vilaplana, V. 3D convolutional neural networks for brain tumor segmentation: A comparison of multi-resolution architectures. In International Workshop on Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries; Springer: Berlin/Heidelberg, Germany, 2016; pp. 150–161. [Google Scholar]
- Hussain, S.; Anwar, S.M.; Majid, M. Brain tumor segmentation using cascaded deep convolutional neural network. In Proceedings of the 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Jeju, Republic of Korea, 11–15 July 2017; pp. 1998–2001. [Google Scholar] [CrossRef]
- Wang, G.; Li, W.; Ourselin, S.; Vercauteren, T. Automatic brain tumor segmentation based on cascaded convolutional neural networks with uncertainty estimation. Front. Comput. Neurosci. 2019, 13, 56. [Google Scholar] [CrossRef]
- Pemberton, H.G.; Wu, J.; Kommers, I.; Müller, D.M.J.; Hu, Y.; Goodkin, O.; Vos, S.B.; Bisdas, S.; Robe, P.A.; Ardon, H.; et al. Multi-class glioma segmentation on real-world data with missing MRI sequences: Comparison of three deep learning algorithms. Sci. Rep. 2023, 13, 18911. [Google Scholar] [CrossRef]
- Zeineldin, R.A.; Karar, M.E.; Burgert, O.; Mathis-Ullrich, F. Multimodal CNN Networks for Brain Tumor Segmentation in MRI: A BraTS 2022 Challenge Solution. In Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries; Bakas, S., Crimi, A., Baid, U., Malec, S., Pytlarz, M., Baheti, B., Zenk, M., Dorent, R., Eds.; Springer Nature: Cham, Switzerland, 2023; pp. 127–137. [Google Scholar]
- Henry, T.; Carre, A.; Lerousseau, M.; Estienne, T.; Robert, C.; Paragios, N.; Deutsch, E. Brain Tumor Segmentation with Self-ensembled, Deeply-Supervised 3D U-Net Neural Networks: A BraTS 2020 Challenge Solution. In Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries; Crimi, A., Bakas, S., Eds.; Springer International Publishing: Cham, Switzerland, 2021; pp. 327–339. [Google Scholar]
- Duman, A.; Powell, J.; Thomas, S.; Sun, X.; Spezi, E. Generalizability of Deep Learning Models on Brain Tumour Segmentation. In Proceedings of the Cardiff University School of Engineering—Research Conference 2023, Cardiff, UK, 12 July 2023. [Google Scholar]
- Çiçek, Ö.; Abdulkadir, A.; Lienkamp, S.S.; Brox, T.; Ronneberger, O. 3D U-Net: Learning dense volumetric segmentation from sparse annotation. In Medical Image Computing and Computer-Assisted Intervention–MICCAI 2016: 19th International Conference, Athens, Greece, October 17–21 2016, Proceedings, Part II 19; Springer: Berlin/Heidelberg, Germany, 2016; pp. 424–432. [Google Scholar]
- Cardoso, M.J.; Li, W.; Brown, R.; Ma, N.; Kerfoot, E.; Wang, Y.; Murrey, B.; Myronenko, A.; Zhao, C.; Yang, D.; et al. Monai: An open-source framework for deep learning in healthcare. arXiv 2022, arXiv:2211.02701. [Google Scholar]
- Menze, B.H.; Jakab, A.; Bauer, S.; Kalpathy-Cramer, J.; Farahani, K.; Kirby, J.; Burren, Y.; Porz, N.; Slotboom, J.; Wiest, R.; et al. The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS). IEEE Trans. Med. Imaging 2015, 34, 1993–2024. [Google Scholar] [CrossRef] [PubMed]
- Bakas, S.; Akbari, H.; Sotiras, A.; Bilello, M.; Rozycki, M.; Kirby, J.S.; Freymann, J.B.; Farahani, K.; Davatzikos, C. Advancing the cancer genome atlas glioma MRI collections with expert segmentation labels and radiomic features. Sci. Data 2017, 4, 170117. [Google Scholar] [CrossRef] [PubMed]
- Rohlfing, T.; Zahr, N.M.; Sullivan, E.V.; Pfefferbaum, A. The SRI24 multichannel atlas of normal adult human brain structure. Hum. Brain Mapp. 2010, 31, 798–819. [Google Scholar] [CrossRef] [PubMed]
- Davatzikos, C.; Rathore, S.; Bakas, S.; Pati, S.; Bergman, M.; Kalarot, R.; Sridharan, P.; Gastounioti, A.; Jahani, N.; Cohen, E.; et al. Cancer imaging phenomics toolkit: Quantitative imaging analytics for precision diagnostics and predictive modeling of clinical outcome. J. Med. Imaging 2018, 5, 011018. [Google Scholar] [CrossRef]
- Pati, S.; Singh, A.; Rathore, S.; Gastounioti, A.; Bergman, M.; Ngo, P.; Ha, S.M.; Bounias, D.; Minock, J.; Murphy, G.; et al. The cancer imaging phenomics toolkit (CaPTk): Technical overview. In Brainlesion; Crimi, A., Bakas, S., Eds.; In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Springer: Berlin/Heidelberg, Germany, 2020; pp. 380–394. [Google Scholar] [CrossRef]
- Isensee, F.; Schell, M.; Pflueger, I.; Brugnara, G.; Bonekamp, D.; Neuberger, U.; Wick, A.; Schlemmer, H.-P.; Heiland, S.; Wick, W.; et al. Automated brain extraction of multisequence MRI using artificial neural networks. Hum. Brain Mapp. 2019, 40, 4952–4964. [Google Scholar] [CrossRef]
- Kingma, D.P.; Ba, J. Adam: A method for stochastic optimization. arXiv 2014, arXiv:1412.6980. [Google Scholar]
- Gwynne, S.; Spezi, E.; Wills, L.; Nixon, L.; Hurt, C.; Joseph, G.; Evans, M.; Griffiths, G.; Crosby, T.; Staffurth, J. Toward Semi-automated Assessment of Target Volume Delineation in Radiotherapy Trials: The SCOPE 1 Pretrial Test Case. Int. J. Radiat. Oncol. Biol. Phys. 2012, 84, 1037–1042. [Google Scholar] [CrossRef]
Model | ET | TC | WT |
---|---|---|---|
nnU-net | 83.96 | 88.34 | 92.53 |
3D U-net | 83.21 | 87.55 | 91.67 |
2.5D U-net | 84.34 | 88.55 | 91.64 |
2D U-net | 84.99 | 89.71 | 91.65 |
Models | GTV |
---|---|
nnU-net | 77.45 |
3D U-net | 75.74 |
2.5D U-net | 70.35 |
2D U-net | 78.43 |
Intensity Norm. Tech | Segmentation Approach | ET | TC | WT |
---|---|---|---|---|
Nyul | multi-class | 79.44 | 79.53 | 88.98 |
multi-label | 83.52 | 88.78 | 92.05 | |
binary class | 84.21 | 89.42 | 90.30 | |
Z-score | multi-class | 84.99 | 89.71 | 91.65 |
multi-label | 82.29 | 87.27 | 92.24 | |
binary class | 85.19 | 89.48 | 92.18 |
Models | DSC(ET) (%) | DSC(TC) (%) | DSC(WT) (%) |
---|---|---|---|
Extended nnU-net [27] | 84.51 | 87.81 | 92.75 |
nnU-net | 78.65 | 85.96 | 91.67 |
3D U-net | 78.89 | 81.05 | 91.16 |
2.5D U-net | 78.80 | 84.23 | 90.90 |
2D U-net | 77.45 | 82.14 | 90.82 |
Z-Score Normalization | Nyul Normalization | Combined Method | GTV Dice Score (%) | ||||||
---|---|---|---|---|---|---|---|---|---|
Multi-Class | Multi-Label | Binary | Multi-Class | Multi-Label | Binary | Union | Ensemble | ||
Base U-net (Multi-class) | Yes | 78.43 | |||||||
Multi-label | Yes | 77.91 | |||||||
Binary | Yes | 78.22 | |||||||
Base U-net (Multi-class) | Yes | 77.61 | |||||||
Multi-label | Yes | 78.20 | |||||||
Binary | Yes | 78.91 | |||||||
RFS | Yes | Yes | Yes | Yes | 78.51 | ||||
RFS+ (only Z-score normaliz-ation) | Yes | Yes | Yes | Yes | 78.69 | ||||
Proposed RFS+ | Yes | Yes | Yes | Yes | 79.22 |
Models | DSC ↑ | HD95 ↓ | Sensitivity ↑ | Specificity ↑ |
---|---|---|---|---|
extended nnU-net [27] | 79.09 | 7.8 | 74.07 | 99.97 |
nnU-net (base nnU-net) | 77.83 | 10.72 | 74.65 | 99.95 |
nnU-net with RFS+ | 78.30 | 8.2 | 73.59 | 99.97 |
2D U-net (base U-net) | 78.43 | 8.8 | 77.24 | 99.94 |
2D U-net with RFS [12] | 78.51 | 11.33 | 78.48 | 99.93 |
2D U-net with RFS+ | 79.22 | 8.1 | 76.93 | 99.95 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Duman, A.; Karakuş, O.; Sun, X.; Thomas, S.; Powell, J.; Spezi, E. RFS+: A Clinically Adaptable and Computationally Efficient Strategy for Enhanced Brain Tumor Segmentation. Cancers 2023, 15, 5620. https://doi.org/10.3390/cancers15235620
Duman A, Karakuş O, Sun X, Thomas S, Powell J, Spezi E. RFS+: A Clinically Adaptable and Computationally Efficient Strategy for Enhanced Brain Tumor Segmentation. Cancers. 2023; 15(23):5620. https://doi.org/10.3390/cancers15235620
Chicago/Turabian StyleDuman, Abdulkerim, Oktay Karakuş, Xianfang Sun, Solly Thomas, James Powell, and Emiliano Spezi. 2023. "RFS+: A Clinically Adaptable and Computationally Efficient Strategy for Enhanced Brain Tumor Segmentation" Cancers 15, no. 23: 5620. https://doi.org/10.3390/cancers15235620