Insights and Considerations in Development and Performance Evaluation of Generative Adversarial Networks (GANs): What Radiologists Need to Know
Abstract
:1. Introduction
2. GAN Architecture, Hierarchy, and Variants
2.1. Where Do the Generated Images Originate? Noise vs. Image
2.2. Improvement of Quality for Generated Images
3. Selecting the Appropriate GAN for the Research Objectives
3.1. Three Considerations
3.1.1. Image-to-Image Translation GANs and Interindividual Anatomic Variance
3.1.2. High-Quality Image Resolution and Contrast
3.1.3. Unsupervised Detection Models
3.2. Examples of GAN Application to Brain MR Imaging
3.2.1. CycleGAN—Brain Infarction Images for Augmentation
3.2.2. pSp Encoder-Combined StyleGAN—Brain Vessel Images for Unsupervised Anomaly Detection
4. Input Data Training
4.1. Image Data Preprocessing Protocols
4.2. Training Saturation
4.3. Performance Improvement and Ablation Study
5. Performance Evaluation
5.1. Quantitative Evaluation
- (1)
- Metrics
- (2)
- Adversarial evaluation
5.2. Qualitative Evaluation
- (1)
- Qualitative evaluation of CycleGAN: synthetic normal and acute infarction images on DWI sequences
- (2)
- Qualitative evaluation of pSp-encoder-combined Style–GAN generator: MIP images of TOF-MRA of normal intracranial arteries
5.3. Diagnostic Performance Evaluation
6. Conclusions
7. Future Directions
Supplementary Materials
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Gao, L.; Zhang, L.; Liu, C.; Wu, S. Handling imbalanced medical image data: A deep-learning-based one-class classification approach. Artif. Intell. Med. 2020, 108, 101935. [Google Scholar] [CrossRef] [PubMed]
- Gao, M.; Huang, J.; Huang, X.; Zhang, S.; Metaxas, D.N. Simplified labeling process for medical image segmentation. Med. Image Comput. Comput. Assist. Interv. 2012, 15 Pt 2, 387–394. [Google Scholar]
- Xun, S.; Li, D.; Zhu, H.; Chen, M.; Wang, J.; Li, J.; Chen, M.; Wu, B.; Zhang, H.; Chai, X.; et al. Generative adversarial networks in medical image segmentation: A review. Comput. Biol. Med. 2022, 140, 105063. [Google Scholar] [CrossRef]
- Osuala, R.; Kushibar, K.; Garrucho, L.; Linardos, A.; Szafranowska, Z.; Klein, S.; Glocker, B.; Díaz, O.; Lekadir, K. A Review of Generative Adversarial Networks in Cancer Imaging: New Applications, New Solutions. arXiv 2021, arXiv:2107.09543. [Google Scholar]
- Huang, G.; Jafari, A.H. Enhanced balancing GAN: Minority-class image generation. Neural Comput. Appl. 2023, 35, 5145–5154. [Google Scholar] [CrossRef] [PubMed]
- Kannan, R.J.; Krithika alias Anbu Devi, M.; Suganthi, K.; Kommers, P.S.A.; Quadir Md, A. Review of Medical Image Synthesis using GAN Techniques. ITM Web Conf. 2021, 37, 01005. [Google Scholar]
- Karras, T.; Laine, S.; Aila, T. A Style-Based Generator Architecture for Generative Adversarial Networks. In Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, 15–20 June 2019; pp. 4396–4405. [Google Scholar]
- Bermano, A.H.; Gal, R.; Alaluf, Y.; Mokady, R.; Nitzan, Y.; Tov, O.; Patashnik, O.; Cohen-Or, D. State-of-the-Art in the Architecture, Methods and Applications of StyleGAN. Comput. Graph. Forum 2022, 41, 591–611. [Google Scholar] [CrossRef]
- Karras, T.; Aila, T.; Laine, S.; Lehtinen, J. Progressive Growing of GANs for Improved Quality, Stability, and Variation. arXiv 2017, arXiv:1710.10196. [Google Scholar]
- Radford, A.; Metz, L.; Chintala, S. Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks. arXiv 2015, arXiv:1511.06434. [Google Scholar]
- Zhu, J.Y.; Park, T.; Isola, P.; Efros, A.A. Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks. In Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy, 22–29 October 2017; pp. 2242–2251. [Google Scholar]
- Isola, P.; Zhu, J.Y.; Zhou, T.; Efros, A.A. Image-to-Image Translation with Conditional Adversarial Networks. In Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 21–26 July 2017; pp. 5967–5976. [Google Scholar]
- Mirza, M.; Osindero, S. Conditional generative adversarial nets. arXiv 2014, arXiv:1411.1784. [Google Scholar]
- Xia, X.; Pan, X.; Li, N.; He, X.; Ma, L.; Zhang, X.; Ding, N. GAN-based anomaly detection: A review. Neurocomputing 2022, 493, 497–535. [Google Scholar] [CrossRef]
- Park, S.; Lee, K.; Ko, B.; Kim, N. Unsupervised anomaly detection with generative adversarial networks in mammography. Sci. Rep. 2023, 13, 2925. [Google Scholar] [CrossRef] [PubMed]
- Lee, S.; Jeong, B.; Kim, M.; Jang, R.; Paik, W.; Kang, J.; Chung, W.J.; Hong, G.S.; Kim, N. Emergency triage of brain computed tomography via anomaly detection with a deep generative model. Nat. Commun. 2022, 13, 4251. [Google Scholar] [CrossRef]
- Goodfellow, I.; Pouget-Abadie, J.; Mirza, M.; Xu, B.; Warde-Farley, D.; Ozair, S.; Courville, A.; Bengio, Y. Generative Adversarial Networks. In Proceedings of the Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014, Montreal, QC, Canada, 8–13 December 2014; Volume 3. [Google Scholar]
- Alotaibi, A. Deep Generative Adversarial Networks for Image-to-Image Translation: A Review. Symmetry 2020, 12, 1705. [Google Scholar] [CrossRef]
- Quintana-Quintana, O.J.; De León-Cuevas, A.; González-Gutiérrez, A.; Gorrostieta-Hurtado, E.; Tovar-Arriaga, S. Dual U-Net-Based Conditional Generative Adversarial Network for Blood Vessel Segmentation with Reduced Cerebral MR Training Volumes. Micromachines 2022, 13, 823. [Google Scholar] [CrossRef]
- Anirudh, R.; Thiagarajan, J.J.; Bremer, P.-T.; Kim, H. Lung nodule detection using 3D convolutional neural networks trained on weakly labeled data. In Proceedings of the Medical Imaging 2016: Computer-Aided Diagnosis, San Diego, CA, USA, 27 February–3 March 2016. [Google Scholar]
- Hwang, S.; Kim, H.-E. Self-Transfer Learning for Weakly Supervised Lesion Localization. In Proceedings of the Medical Image Computing and Computer-Assisted Intervention—MICCAI 2016: 19th International Conference, Athens, Greece, 17–21 October 2016. [Google Scholar]
- Feng, X.; Yang, J.; Laine, A.; Angelini, E. Discriminative Localization in CNNs for Weakly-Supervised Segmentation of Pulmonary Nodules. In Proceedings of the Medical Image Computing and Computer Assisted Intervention—MICCAI 2017: 20th International Conference, Quebec City, QC, Canada, 11–13 September 2017; Volume 10435. [Google Scholar]
- Richardson, E.; Alaluf, Y.; Patashnik, O.; Nitzan, Y.; Azar, Y.; Shapiro, S.; Cohen-Or, D. Encoding in style: A stylegan encoder for image-to-image translation. In Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA, 20–25 June 2021; pp. 2287–2296. [Google Scholar]
- Tov, O.; Alaluf, Y.; Nitzan, Y.; Patashnik, O.; Cohen-Or, D. Designing an encoder for StyleGAN image manipulation. ACM Trans. Graph. 2021, 40, 133. [Google Scholar] [CrossRef]
- Zhou, T.; Li, Q.; Lu, H.; Cheng, Q.; Zhang, X. GAN review: Models and medical image fusion applications. Inf. Fusion 2023, 91, 134–148. [Google Scholar] [CrossRef]
- Abdusalomov, A.B.; Nasimov, R.; Nasimova, N.; Muminov, B.; Whangbo, T.K. Evaluating Synthetic Medical Images Using Artificial Intelligence with the GAN Algorithm. Sensors 2023, 23, 3440. [Google Scholar] [CrossRef]
- Wicaksono, K.P.; Fujimoto, K.; Fushimi, Y.; Sakata, A.; Okuchi, S.; Hinoda, T.; Nakajima, S.; Yamao, Y.; Yoshida, K.; Miyake, K.K.; et al. Super-resolution application of generative adversarial network on brain time-of-flight MR angiography: Image quality and diagnostic utility evaluation. Eur. Radiol. 2022, 33, 936–946. [Google Scholar] [CrossRef]
- Belue, M.J.; Harmon, S.A.; Masoudi, S.; Barrett, T.; Law, Y.M.; Purysko, A.S.; Panebianco, V.; Yilmaz, E.C.; Lin, Y.; Jadda, P.K.; et al. Quality of T2-weighted MRI re-acquisition versus deep learning GAN image reconstruction: A multi-reader study. Eur. J. Radiol. 2024, 170, 111259. [Google Scholar] [CrossRef]
- Chen, Y.; Lin, H.; Zhang, W.; Chen, W.; Zhou, Z.; Heidari, A.A.; Chen, H.; Xu G:, I.C.y.c.l.e.-G.A.N. Improved cycle generative adversarial networks for liver medical image generation. Biomed. Signal Process. Control 2024, 92, 106100. [Google Scholar] [CrossRef]
- Song, Y.; Chong, N.Y. S-CycleGAN: Semantic Segmentation Enhanced CT-Ultrasound Image-to-Image Translation for Robotic Ultrasonography. arXiv 2024, arXiv:2406.01191. [Google Scholar]
- Esmaeili, M.; Toosi, A.; Roshanpoor, A.; Changizi, V.; Ghazisaeedi, M.; Rahmim, A.; Sabokrou, M. Generative Adversarial Networks for Anomaly Detection in Biomedical Imaging: A Study on Seven Medical Image Datasets. IEEE Access 2023, 11, 17906–17921. [Google Scholar] [CrossRef]
- Lyu, J.; Fu, Y.; Yang, M.; Xiong, Y.; Duan, Q.; Duan, C.; Wang, X.; Xing, X.; Zhang, D.; Lin, J.; et al. Generative Adversarial Network-based Noncontrast CT Angiography for Aorta and Carotid Arteries. Radiology 2023, 309, e230681. [Google Scholar] [CrossRef] [PubMed]
- Wang, B.; Pan, Y.; Xu, S.; Zhang, Y.; Ming, Y.; Chen, L.; Liu, X.; Wang, C.; Liu, Y.; Xia, Y. Quantitative Cerebral Blood Volume Image Synthesis from Standard MRI Using Image-to-Image Translation for Brain Tumors. Radiology 2023, 308, e222471. [Google Scholar] [CrossRef] [PubMed]
- Armanious, K.; Jiang, C.; Fischer, M.; Küstner, T.; Hepp, T.; Nikolaou, K.; Gatidis, S.; Yang, B. MedGAN: Medical image translation using GANs. Comput. Med. Imaging Graph. 2020, 79, 101684. [Google Scholar] [CrossRef] [PubMed]
- Kong, L.; Lian, C.; Huang, D.; Li, Z.; Hu, Y.; Zhou, Q. Breaking the Dilemma of Medical Image-to-image Translation. In Proceedings of the 2021 Conference on Neural Information Processing Systems, Online, 6–14 December 2021. [Google Scholar]
- Makhlouf, A.; Maayah, M.; Abughanam, N.; Catal, C. The use of generative adversarial networks in medical image augmentation. Neural Comput. Appl. 2023, 35, 24055–24068. [Google Scholar] [CrossRef]
- Saeed, A.Q.; Sheikh Abdullah, S.N.H.; Che-Hamzah, J.; Abdul Ghani, A.T.; Abu-ain, W.A.k. Synthesizing Retinal Images using End-To-End VAEs-GAN Pipeline-Based Sharpening and Varying Layer. Multimed. Tools Appl. 2024, 83, 1283–1307. [Google Scholar] [CrossRef]
- Khan, S.; Huh, J.; Ye, J.C. Contrast and Resolution Improvement of POCUS Using Self-consistent CycleGAN. In Domain Adaptation and Representation Transfer, and Affordable Healthcare and AI for Resource Diverse Global Health; Springer International Publishing: Cham, Switzerland, 2021; pp. 158–167. [Google Scholar]
- Suryawati, E.; Pardede, H.F.; Zilvan, V.; Ramdan, A.; Krisnandi, D.; Heryana, A.; Yuwana, R.S.; Kusumo, R.B.S.; Arisal, A.; Supianto, A.A. Unsupervised feature learning-based encoder and adversarial networks. J. Big Data 2021, 8, 118. [Google Scholar] [CrossRef]
- Schlegl, T.; Seeböck, P.; Waldstein, S.; Langs, G.; Schmidt-Erfurth, U. f-AnoGAN: Fast Unsupervised Anomaly Detection with Generative Adversarial Networks. Med. Image Anal. 2019, 54, 30–44. [Google Scholar] [CrossRef]
- Sabuhi, M.; Zhou, M.; Bezemer, C.-P.; Musilek, P. Applications of Generative Adversarial Networks in Anomaly Detection: A Systematic Literature Review. IEEE Access 2021, 9, 161003–161029. [Google Scholar] [CrossRef]
- Schlegl, T.; Seeböck, P.; Waldstein, S.M.; Schmidt-Erfurth, U.; Langs, G. Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker Discovery. In Information Processing in Medical Imaging; Springer International Publishing: Cham, Switzerland, 2017; pp. 146–157. [Google Scholar]
- Akcay, S.; Atapour-Abarghouei, A.; Breckon, T.P. GANomaly: Semi-supervised Anomaly Detection via Adversarial Training. In Computer Vision—ACCV 2018; Springer International Publishing: Cham, Switzerland, 2019; pp. 622–637. [Google Scholar]
- Chen, L.; Li, Y.; Deng, X.; Liu, Z.; Lv, M.; Zhang, H. Dual Auto-Encoder GAN-Based Anomaly Detection for Industrial Control System. Appl. Sci. 2022, 12, 4986. [Google Scholar] [CrossRef]
- Nakao, T.; Hanaoka, S.; Nomura, Y.; Murata, M.; Takenaga, T.; Miki, S.; Watadani, T.; Yoshikawa, T.; Hayashi, N.; Abe, O. Unsupervised Deep Anomaly Detection in Chest Radiographs. J. Digit. Imaging 2021, 34, 418–427. [Google Scholar] [CrossRef]
- Goceri, E. Medical image data augmentation: Techniques, comparisons and interpretations. Artif. Intell. Rev. 2023, 56, 12561–12605. [Google Scholar] [CrossRef]
- Shorten, C.; Khoshgoftaar, T.M. A survey on Image Data Augmentation for Deep Learning. J. Big Data 2019, 6, 60. [Google Scholar] [CrossRef]
- Meyes, R.; Lu, M.; Puiseau CWd Meisen, T. Ablation Studies in Artificial Neural Networks. arXiv 2019, arXiv:1901.08644. [Google Scholar]
- Sheikholeslami, S.; Meister, M.; Wang, T.; Payberah, A.H.; Vlassov, V.; Dowling, J. AutoAblation: Automated Parallel Ablation Studies for Deep Learning. In Proceedings of the 1st Workshop on Machine Learning and Systems 2021, Edinburgh, UK, 26 April 2021. [Google Scholar]
- Singh, R.; Sethi, A.; Saini, K.; Saurav, S.; Tiwari, A.; Singh, S. Attention-guided generator with dual discriminator GAN for real-time video anomaly detection. Eng. Appl. Artif. Intell. 2024, 131, 107830. [Google Scholar] [CrossRef]
- Mindlin, D.; Schilling, M.; Cimiano, P. ABC-GAN: Spatially Constrained Counterfactual Generation for Image Classification Explanations. In Explainable Artificial Intelligence; Springer Nature: Cham, Switzerland, 2023; pp. 260–282. [Google Scholar]
- Woodland, M.; Wood, J.; Anderson, B.M.; Kundu, S.; Lin, E.; Koay, E.; Odisio, B.; Chung, C.; Kang, H.C.; Venkatesan, A.M.; et al. Evaluating the Performance of StyleGAN2-ADA on Medical Images. In Simulation and Synthesis in Medical Imaging; Springer International Publishing: Cham, Switzerland, 2022; pp. 142–153. [Google Scholar]
- Long, X.; Tian, C. Spatial and channel attention-based conditional Wasserstein GAN for direct and rapid image reconstruction in ultrasound computed tomography. Biomed. Eng. Lett. 2024, 14, 57–68. [Google Scholar] [CrossRef]
- Liang, D.; Zhang, S.; Zhao, Z.; Wang, G.; Sun, J.; Zhao, J.; Li, W.; Xu, L.X. Two-stage generative adversarial networks for metal artifact reduction and visualization in ablation therapy of liver tumors. Int. J. Comput. Assist. Radiol. Surg. 2023, 18, 1991–2000. [Google Scholar] [CrossRef]
- Alshawi, R.; Hoque, M.T.; Ferdaus, M.M.; Abdelguerfi, M.; Niles, K.; Prathak, K.; Tom, J.; Klein, J.; Mousa, M.; Lopez, J.J. Dual Attention U-Net with Feature Infusion: Pushing the Boundaries of Multiclass Defect Segmentation. arXiv 2023, arXiv:231214053. [Google Scholar]
- Jeong, J.J.; Tariq, A.; Adejumo, T.; Trivedi, H.; Gichoya, J.W.; Banerjee, I. Systematic Review of Generative Adversarial Networks (GANs) for Medical Image Classification and Segmentation. J. Digit. Imaging 2022, 35, 137–152. [Google Scholar] [CrossRef] [PubMed]
- Laino, M.E.; Cancian, P.; Politi, L.S.; Della Porta, M.G.; Saba, L.; Savevski, V. Generative Adversarial Networks in Brain Imaging: A Narrative Review. J. Imaging 2022, 8, 83. [Google Scholar] [CrossRef] [PubMed]
- Mudeng, V.; Kim, M.; Choe, S.-w. Prospects of Structural Similarity Index for Medical Image Analysis. Appl. Sci. 2022, 12, 3754. [Google Scholar] [CrossRef]
- Nečasová, T.; Burgos, N.; Svoboda, D. Chapter 25—Validation and evaluation metrics for medical and biomedical image synthesis. In Biomedical Image Synthesis and Simulation; Burgos, N., Svoboda, D., Eds.; Academic Press: Cambridge, MA, USA, 2022; pp. 573–600. [Google Scholar]
- Zhang, H.; Li, H.; Dillman, J.R.; Parikh, N.A.; He, L. Multi-Contrast MRI Image Synthesis Using Switchable Cycle-Consistent Generative Adversarial Networks. Diagnostics 2022, 12, 816. [Google Scholar] [CrossRef] [PubMed]
- Yoshida, M.; Teramoto, A.; Kudo, K.; Matsumoto, S.; Saito, K.; Fujita, H. Automated Extraction of Cerebral Infarction Region in Head MR Image Using Pseudo Cerebral Infarction Image by CycleGAN. Appl. Sci. 2022, 12, 489. [Google Scholar] [CrossRef]
- Ji, J.; Zhao, Y.; Zhang, Y.; Wang, C.; Ma, X.; Huang, F.; Yao, J. Infrared and visible image fusion of generative adversarial network based on multi-channel encoding and decoding. Infrared Phys. Technol. 2023, 134, 104853. [Google Scholar] [CrossRef]
- Borji, A. Pros and cons of GAN evaluation measures: New developments. Comput. Vis. Image Underst. 2022, 215, 103329. [Google Scholar] [CrossRef]
- Kalantar, R.; Messiou, C.; Winfield, J.M.; Renn, A.; Latifoltojar, A.; Downey, K.; Sohaib, A.; Lalondrelle, S.; Koh, D.M.; Blackledge, M.D. CT-Based Pelvic T(1)-Weighted MR Image Synthesis Using UNet, UNet++ and Cycle-Consistent Generative Adversarial Network (Cycle-GAN). Front. Oncol. 2021, 11, 665807. [Google Scholar] [CrossRef] [PubMed]
- Van Nguyen, H.; Trong, T.H. Brain MRI Images Generating Method Based on Cyclegan. J. Sci. Technol. Issue Inf. Commun. Technol. 2022, 20, 13–18. [Google Scholar] [CrossRef]
- Avbersek, L.K.; Repovs, G. Deep learning in neuroimaging data analysis: Applications, challenges, and solutions. Front. Neuroimaging 2022, 1, 981642. [Google Scholar] [CrossRef]
- Baur, C.; Wiestler, B.; Muehlau, M.; Zimmer, C.; Navab, N.; Albarqouni, S. Modeling Healthy Anatomy with Artificial Intelligence for Unsupervised Anomaly Detection in Brain MRI. Radiol. Artif. Intell. 2021, 3, e190169. [Google Scholar] [CrossRef] [PubMed]
- Ahmad, B.; Sun, J.; You, Q.; Palade, V.; Mao, Z. Brain Tumor Classification Using a Combination of Variational Autoencoders and Generative Adversarial Networks. Biomedicines 2022, 10, 223. [Google Scholar] [CrossRef] [PubMed]
- Tariq, U.; Qureshi, R.; Zafar, A.; Aftab, D.; Wu, J.; Alam, T.; Shah, Z.; Ali, H. Brain Tumor Synthetic Data Generation with Adaptive StyleGANs. In Proceedings of the 2023 Irish Conference on Artificial Intelligence and Cognitive Science, Letterkenny, Ireland, 7–8 December 2023; Springer Nature: Cham, Switzerland, 2023; pp. 147–159. [Google Scholar]
- Information eXtraction from Images Dataset. Available online: http://brain-development.org/ixi-dataset/ (accessed on 31 July 2024).
Paper | Target Organ | Modality | Purpose | GAN Variants | Performance Evaluation |
---|---|---|---|---|---|
Wicaksono et al. (2023) [27] | Intracranial artery | TOF-MRA * | Enhances resolution | Modified pix2pix | MS-SSIM *, 0.87 vs. 0.73 ISSM *, 0.60 vs. 0.35; Improved sensitivity and specificity in detecting aneurysms, stenoses, and occlusions |
Mason et al. (2023) [28] | Prostate | mpMRI | Reconstruction | CycleGAN * | Improved quantitative deep learning score No qualitative improvement |
Ying et al. (2024) [29] | Liver | CT and MRI | Augmentation | ICycleGAN * | Superior visual quality (SSIM *, PSNR *, NMAE *, FID *) |
Yuhan S. and Nak Young C. (2024) [30] | Abdomen | CT → US | Segmentation and reconstruction | S-CycleGAN | Absent suitable metrics and evaluation |
Marzieh et al. (2023) [31] | Brain, breast, and blood cancer | MRI Mammography | Unsupervised anomaly detection | f-anoGAN, GANomaly, and multi-KD | Unreliable performance for detecting abnormalities in medical images |
Seungjun et al. (2022) [16] | Brain | CT | Unsupervised anomaly detection | CN *-StyleGAN | Shorter post-ADA * triage than pre-ADA * triage by 294 s in an emergency cohort median wait time |
Jinhao et al. (2023) [32] | Neck and abdomen | NCCT * | CTA * reconstruction | CTA-GAN | Diagnostic accuracy for vascular diseases (accuracy = 94%) |
Wang et al. (2023) [33] | Brain tumor | DSC MRI * | CBV * map reconstruction | Feature-consistent GAN + three-dimensional encoder–decoder network | The highest synthetic performance (SSIM *, 86.29% ± 4.30) Accuracy of grading gliomas (AUC *, 0.857 vs. 0.707) |
Seungju et al. (2023) [15] | Breast cancer | Mammography | Unsupervised anomaly detection | StyleGAN2 | AUC *, sensitivity, and specificity of the classification performance (70.0%, 78.0%, and 52.0%) |
Architecture | GAN Variants | Detailed Characteristics | |
---|---|---|---|
Image-to-image translation | Suitable for medical image: preserve anatomical structure | Pix2pix [12] | Applies conditions to generate detailed features Can generate high-quality images Requires paired dataset for training |
CycleGAN * [11] | Uses unpaired data to generate images from different domains Suitable for image augmentation Unsuitable for organs with significant variance in shape and location | ||
Encoder-combined GANs [23,24] | Generates high-resolution images Enables unsupervised anomaly detection | ||
Noise-to-image translation | Not suitable for the medical images by itself: needs combination with an encoder or other GANs | cGAN * [13] | Assigns conditions to generate required features Needs class labeling Can complicate the training process |
DCGAN * [10] | Generates large-scale and high-quality images Unstable training process | ||
PGGAN * [9] | Generates high resolution with fine image features Requires high computational costs and a large amount of training data | ||
StyleGAN [7,8] | Generates high resolution with fine image features Requires high computational costs and a large amount of training data Allows for the selective modification of desired image features |
Main Categories | Subcategories | Examples | Characteristics |
---|---|---|---|
Quantitative | Pixel-level metrics | PSNR *, SSIM *, and RMSE * | An objective evaluation method does not always correlate with radiologists’ evaluations |
Distribution metrics | FID * and IS * | ||
Adversarial evaluation | Another discriminator | Immediate feedback to the generator objective evaluation method does not always correlate with radiologists’ evaluations | |
Qualitative | Radiologist evaluation | Rating, ranking, preference, or pair wise comparison | Applicable to clinical diagnosis, requires time and effort, and there are different standards and biases within the evaluators |
Combined | Diagnostic performance | Accuracy, sensitivity, specificity, ROC * curve, and AUC * | Highest reliability in clinical diagnosis requires significant time and effort |
Normal → Synthetic Infarction | Infarction → Synthetic Normal | Total | |
---|---|---|---|
No. of generated images | 12 | 12 | 24 |
No. of consistent images | 9 | 10 | 19 |
Image production rate | 9/15 (60%) | 10/15 (67%) | 19/30 (63%) |
Black Background | Color Augmentation | ||||||
---|---|---|---|---|---|---|---|
Large Vessel | Medium Vessel | Small Vessel | Large Vessel | Medium Vessel | Small Vessel | ||
Rater 1 | Outer margin | 1.1 | 1.1 | 1.3 | 1.4 | 1.8 | 2.1 |
Diameter consistency | 1.0 | 1.5 | 1.5 | 1.3 | 1.3 | 1.9 | |
Separability | 0.1 | 0.4 | 1.0 | 0.3 | 0.5 | 0.6 | |
Rater 2 | Outer margin | 1.4 | 1.6 | 1.9 | 1.3 | 1.3 | 2.8 |
Diameter consistency | 1.5 | 1.8 | 2.3 | 1.9 | 2.4 | 2.4 | |
Separability | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | |
Rater 3 | Outer margin | 1.0 | 1.1 | 1.3 | 1.0 | 1.1 | 1.6 |
Diameter consistency | 1.0 | 1.1 | 1.0 | 1.0 | 1.1 | 1.3 | |
Separability | 0.7 | 0.8 | 1.0 | 0.4 | 0.8 | 0.8 | |
Rater 4 | Outer margin | 1.5 | 1.6 | 2.1 | 1.1 | 1.3 | 2.0 |
Diameter consistency | 1.5 | 1.6 | 2.6 | 1.5 | 1.8 | 1.9 | |
Separability | 1.0 | 1.0 | 1.0 | 0.4 | 0.5 | 0.9 | |
Total Average | Outer margin | 1.3 | 1.4 | 1.6 | 1.2 | 1.3 | 2.1 |
Diameter consistency | 1.3 | 1.5 | 1.8 | 1.4 | 1.6 | 1.9 | |
Separability | 0.7 | 0.8 | 1.0 | 0.5 | 0.7 | 0.8 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Yoon, J.T.; Lee, K.M.; Oh, J.-H.; Kim, H.-G.; Jeong, J.W. Insights and Considerations in Development and Performance Evaluation of Generative Adversarial Networks (GANs): What Radiologists Need to Know. Diagnostics 2024, 14, 1756. https://doi.org/10.3390/diagnostics14161756
Yoon JT, Lee KM, Oh J-H, Kim H-G, Jeong JW. Insights and Considerations in Development and Performance Evaluation of Generative Adversarial Networks (GANs): What Radiologists Need to Know. Diagnostics. 2024; 14(16):1756. https://doi.org/10.3390/diagnostics14161756
Chicago/Turabian StyleYoon, Jeong Taek, Kyung Mi Lee, Jang-Hoon Oh, Hyug-Gi Kim, and Ji Won Jeong. 2024. "Insights and Considerations in Development and Performance Evaluation of Generative Adversarial Networks (GANs): What Radiologists Need to Know" Diagnostics 14, no. 16: 1756. https://doi.org/10.3390/diagnostics14161756
APA StyleYoon, J. T., Lee, K. M., Oh, J. -H., Kim, H. -G., & Jeong, J. W. (2024). Insights and Considerations in Development and Performance Evaluation of Generative Adversarial Networks (GANs): What Radiologists Need to Know. Diagnostics, 14(16), 1756. https://doi.org/10.3390/diagnostics14161756