Improving Breast Tumor Multi-Classification from High-Resolution Histological Images with the Integration of Feature Space Data Augmentation
Abstract
:1. Introduction
- Performing the classification and augmentation in a feature latent space allows the model to deal with the local and global information of gigapixel WSIs;
- Working in a feature latent space reduces the computational cost while preserving the class label of the input;
- Combining a two-stage augmentation process into an all-in-one model enables the network to generate and classify the WSIs during the training without a separate classifier, increasing the classification accuracy on the testing dataset.
2. Method
2.1. Grid-Based Features Map Extraction
2.2. GFM Augmentation
2.2.1. Standard Data Augmentation
2.2.2. Conditional Generative Adversarial Network (cGAN)
- Discriminator Loss Real: this quantifies how well the discriminator correctly identifies real data as real;
- Discriminator Loss Fake: quantifies how well the discriminator correctly identifies generated data as fake.
2.3. Convolutional Neural Network (CNN) Classifier
3. Experiments and Results
3.1. Dataset
- Task 1: a 3-class WSI classification is required by grouping the original six tumor subtypes into the following three lesion types: non-cancerous (PB+UDH), pre-cancerous (ADH+FEA), and cancerous (DCIS+IC);
- Task 2: a 6-class WSI classification is required to perform a fine-grained subtyping of the tumors within the WSIs. The following six tumor subtypes have been considered: PB, UDH, ADH, FEA, DCIS, and IC.
3.2. Training Protocol
3.2.1. cGAN Training
3.2.2. Classifier Training
3.3. Result and Discussion
4. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Huang, Y.; Zhao, W.; Wang, S.; Fu, Y.; Jiang, Y.; Yu, L. ConSlide: Asynchronous Hierarchical Interaction Transformer with Breakup-Reorganize Rehearsal for Continual Whole Slide Image Analysis. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Paris, France, 2–6 October 2023; pp. 21349–21360. [Google Scholar]
- Wetstein, S.C.; de Jong, V.M.; Stathonikos, N.; Opdam, M.; Dackus, G.M.; Pluim, J.P.; van Diest, P.J.; Veta, M. Deep learning-based breast cancer grading and survival analysis on whole-slide histopathology images. Sci. Rep. 2022, 12, 15102. [Google Scholar] [CrossRef] [PubMed]
- Khalifa, N.E.; Loey, M.; Mirjalili, S. A comprehensive survey of recent trends in deep learning for digital images augmentation. Artif. Intell. Rev. 2022, 55, 2351–2377. [Google Scholar] [CrossRef] [PubMed]
- Naveed, H.; Anwar, S.; Hayat, M.; Javed, K.; Mian, A. Survey: Image mixing and deleting for data augmentation. arXiv 2021, arXiv:2106.07085. [Google Scholar] [CrossRef]
- Ghiasi, G.; Cui, Y.; Srinivas, A.; Qian, R.; Lin, T.Y.; Cubuk, E.D.; Le, Q.V.; Zoph, B. Simple copy-paste is a strong data augmentation method for instance segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA, 19–25 June 2021; pp. 2918–2928. [Google Scholar]
- Wang, F.; Wang, H.; Wei, C.; Yuille, A.; Shen, W. CP 2: Copy-Paste Contrastive Pretraining for Semantic Segmentation. In Proceedings of the 17th European Conference, Tel Aviv, Israel, 23–27 October 2022; pp. 499–515. [Google Scholar]
- Chen, Y.; Yang, X.H.; Wei, Z.; Heidari, A.A.; Zheng, N.; Li, Z.; Chen, H.; Hu, H.; Zhou, Q.; Guan, Q. Generative adversarial networks in medical image augmentation: A review. Comput. Biol. Med. 2022, 144, 105382. [Google Scholar] [CrossRef] [PubMed]
- AlAmir, M.; AlGhamdi, M. The Role of generative adversarial network in medical image analysis: An in-depth survey. ACM Comput. Surv. 2022, 55, 1–36. [Google Scholar] [CrossRef]
- Gecer, B.; Aksoy, S.; Mercan, E.; Shapiro, L.G.; Weaver, D.L.; Elmore, J.G. Detection and classification of cancer in whole slide breast histopathology images using deep convolutional networks. Pattern Recognit. 2018, 84, 345–356. [Google Scholar] [CrossRef] [PubMed]
- Zhang, X.; Liu, C.; Li, T.; Zhou, Y. The whole slide breast histopathology image detection based on a fused model and heatmaps. Biomed. Signal Process. Control 2023, 82, 104532. [Google Scholar] [CrossRef]
- Brancati, N.; De Pietro, G.; Riccio, D.; Frucci, M. Gigapixel histopathological image analysis using attention-based neural networks. IEEE Access 2021, 9, 87552–87562. [Google Scholar] [CrossRef]
- Tomita, N.; Abdollahi, B.; Wei, J.; Ren, B.; Suriawinata, A.; Hassanpour, S. Attention-based deep neural networks for detection of cancerous and precancerous esophagus tissue on histopathological slides. JAMA Netw. Open 2019, 2, e1914645. [Google Scholar] [CrossRef]
- Tellez, D.; Litjens, G.; van der Laak, J.; Ciompi, F. Neural image compression for gigapixel histopathology image analysis. IEEE Trans. Pattern Anal. Mach. Intell. 2019, 43, 567–578. [Google Scholar] [CrossRef] [PubMed]
- Ma, P.; Lu, H.; Yang, B.; Ran, W. GAN-MVAE: A discriminative latent feature generation framework for generalized zero-shot learning. Pattern Recognit. Lett. 2022, 155, 77–83. [Google Scholar] [CrossRef]
- Wu, H.; Gao, R.; Sheng, Y.P.; Chen, B.; Li, S. SDAE-GAN: Enable high-dimensional pathological images in liver cancer survival prediction with a policy gradient based data augmentation method. Med. Image Anal. 2020, 62, 101640. [Google Scholar] [CrossRef] [PubMed]
- Mirza, M.; Osindero, S. Conditional generative adversarial nets. arXiv 2014, arXiv:1411.1784. [Google Scholar]
- Wang, X.; Xiang, J.; Zhang, J.; Yang, S.; Yang, Z.; Wang, M.H.; Zhang, J.; Yang, W.; Huang, J.; Han, X. SCL-WC: Cross-slide contrastive learning for weakly-supervised whole-slide image classification. Adv. Neural Inf. Process. Syst. 2022, 35, 18009–18021. [Google Scholar]
- Marini, N.; Wodzinski, M.; Atzori, M.; Müller, H. A Multi-Task Multiple Instance Learning Algorithm to Analyze Large Whole Slide Images from Bright Challenge 2022. In Proceedings of the 2022 IEEE International Symposium on Biomedical Imaging Challenges (ISBIC), Kolkata, India, 28–31 March 2022; pp. 1–4. [Google Scholar]
- Zhang, J.; Zhang, X.; Ma, K.; Gupta, R.; Saltz, J.; Vakalopoulou, M.; Samaras, D. Gigapixel whole-slide images classification using locally supervised learning. In Proceedings of the 25th International Conference, Singapore, 18–22 September 2022; pp. 192–201. [Google Scholar]
- Brancati, N.; Anniciello, A.M.; Pati, P.; Riccio, D.; Scognamiglio, G.; Jaume, G.; De Pietro, G.; Di Bonito, M.; Foncubierta, A.; Botti, G.; et al. BRACS: A dataset for breast carcinoma subtyping in h&e histology images. Database 2022, 2022, baac093. [Google Scholar] [PubMed]
- Macenko, M.; Niethammer, M.; Marron, J.S.; Borland, D.; Woosley, J.T.; Guan, X.; Schmitt, C.; Thomas, N.E. A method for normalizing histology slides for quantitative analysis. In Proceedings of the 2009 IEEE International Symposium on Biomedical Imaging: From Nano to Macro, Boston, MA, USA, 28 June–1 July 2009; pp. 1107–1110. [Google Scholar]
- Neary-Zajiczek, L.; Beresna, L.; Razavi, B.; Pawar, V.; Shaw, M.; Stoyanov, D. Minimum resolution requirements of digital pathology images for accurate classification. Med. Image Anal. 2023, 89, 102891. [Google Scholar] [CrossRef] [PubMed]
- Mayouf, M.S.; Dupin de Saint-Cyr, F. Curriculum Incremental Deep Learning on BreakHis DataSet. In Proceedings of the 2022 8th International Conference on Computer Technology Applications, Vienna, Austria, 12–14 May 2022; pp. 35–41. [Google Scholar]
- He, K.; Zhang, X.; Ren, S.; Sun, J. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016; pp. 770–778. [Google Scholar]
- Kingma, D.P.; Ba, J. Adam: A method for stochastic optimization. arXiv 2014, arXiv:1412.6980. [Google Scholar]
- Loshchilov, I.; Hutter, F. Decoupled weight decay regularization. arXiv 2017, arXiv:1711.05101. [Google Scholar]
Non-Cancerous | Pre-Cancerous | Cancerous | Total | ||||
---|---|---|---|---|---|---|---|
PB | UDH | FEA | ADH | DCIS | IC | ||
Training | 131 | 65 | 30 | 36 | 49 | 112 | 423 |
Validation | 16 | 9 | 11 | 12 | 12 | 20 | 80 |
Testing | 34 | 33 | 33 | 33 | 33 | 34 | 200 |
Non-Cancerous | Pre-Cancerous | Cancerous | Total | |
---|---|---|---|---|
Original Dataset | 196 | 66 | 161 | 423 |
Augmented Dataset | 1960 (196 × 10) | 1320 (66 × 20) | 1932 (161 × 12) | 5212 |
Experiment | F1-Average | F1-Non-Cancerous | F1-Pre-Cancerous | F1-Cancerous |
---|---|---|---|---|
[19] | 65.3 | 68.0 | 54.0 | 74.2 |
WINM 1° rank * | 71.6 | 72.5 | 62.3 | 80.0 |
WINM 2° rank * | 69.6 | 70.8 | 52.7 | 85.3 |
[18] * | 65.0 | 71.8 | 51.6 | 71.7 |
Proposed Method | 69.5 | 74.0 | 59.5 | 75.2 |
Experiment | F1-Average | F1-Non-Cancerous | F1-Pre-Cancerous | F1-Cancerous |
---|---|---|---|---|
AT_NcGAN | 61.6 | 71.0 | 44.2 | 69.4 |
NAT_cGAN | 62.4 | 70.5 | 47.8 | 68.9 |
AT_3GAN | 62.1 | 66.7 | 48.2 | 71.3 |
Proposed Method | 69.5 | 74.0 | 59.5 | 75.2 |
Experiment | F1-Average | F1-PB | F1-UDH | F1-FEA | F1-ADH | F1-DCIS | F1-IC |
---|---|---|---|---|---|---|---|
[19] | 40.3 | 46.1 | 41.5 | 31.8 | 13.0 | 45.3 | 64.1 |
Proposed Method | 41.5 | 49.6 | 28.0 | 40.0 | 20.0 | 40.8 | 71.2 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Brancati, N.; Frucci, M. Improving Breast Tumor Multi-Classification from High-Resolution Histological Images with the Integration of Feature Space Data Augmentation. Information 2024, 15, 98. https://doi.org/10.3390/info15020098
Brancati N, Frucci M. Improving Breast Tumor Multi-Classification from High-Resolution Histological Images with the Integration of Feature Space Data Augmentation. Information. 2024; 15(2):98. https://doi.org/10.3390/info15020098
Chicago/Turabian StyleBrancati, Nadia, and Maria Frucci. 2024. "Improving Breast Tumor Multi-Classification from High-Resolution Histological Images with the Integration of Feature Space Data Augmentation" Information 15, no. 2: 98. https://doi.org/10.3390/info15020098