Next-Generation Diagnostics: The Impact of Synthetic Data Generation on the Detection of Breast Cancer from Ultrasound Imaging

Abstract

Breast cancer is one of the most lethal and widespread diseases affecting women worldwide. As a result, it is necessary to diagnose breast cancer accurately and efficiently utilizing the most cost-effective and widely used methods. In this research, we demonstrated that synthetically created high-quality ultrasound data outperformed conventional augmentation strategies for efficiently diagnosing breast cancer using deep learning. We trained a deep-learning model using the EfficientNet-B7 architecture and a large dataset of 3186 ultrasound images acquired from multiple publicly available sources, as well as 10,000 synthetically generated images using generative adversarial networks (StyleGAN3). The model was trained using five-fold cross-validation techniques and validated using four metrics: accuracy, recall, precision, and the F1 score measure. The results showed that integrating synthetically produced data into the training set increased the classification accuracy from 88.72% to 92.01% based on the F1 score, demonstrating the power of generative models to expand and improve the quality of training datasets in medical-imaging applications. This demonstrated that training the model using a larger set of data comprising synthetic images significantly improved its performance by more than 3% over the genuine dataset with common augmentation. Various data augmentation procedures were also investigated to improve the training set’s diversity and representativeness. This research emphasizes the relevance of using modern artificial intelligence and machine-learning technologies in medical imaging by providing an effective strategy for categorizing ultrasound images, which may lead to increased diagnostic accuracy and optimal treatment options. The proposed techniques are highly promising and have strong potential for future clinical application in the diagnosis of breast cancer.

1. Introduction

Ultrasound diagnostics is vital for the early identification and diagnosis of many disorders, including cancers. Ultrasound, in particular, is utilized to analyze formations in soft tissues such as the breast, allowing for the diagnosis of whether they are malignant or benign. The accuracy of such a diagnosis has a direct influence on the patient’s subsequent treatment options and prognosis. However, despite advancements in ultrasound technology, interpreting ultrasound pictures remains a difficult endeavor due to their high variability and the subjectivity of professional assessments. For example, investigations have demonstrated that radiologists can differ greatly in their diagnosis accuracy; in fact, as many as 30% of instances have been found to have discrepancies [1].

Breast cancer is a serious health issue worldwide, which has a significant impact on people’s personal health and public health [2]. According to the World Health Organization (WHO), an estimated 2.3 million women were diagnosed with breast cancer in 2020, with more than 684,000 dying from the illness [3,4]. Accounting for 11.7% of newly diagnosed cases and 6.9% of cancer-related fatalities worldwide, breast cancer has a high incidence rate, which increases its global burden [5]. Early and precise identification is critical for effective treatment, with a five-year survival rate of over 90% for localized breast cancer compared to only 15% for metastatic stages [6].

Breast cancer screening can greatly increase survival rates and is essential for early detection. A variety of approaches are used in screening, such as non-invasive imaging modalities and physical examinations by medical professionals. The imaging modalities that are most frequently used include histology, ultrasound, CT scan, MRI, X-ray, and mammography [7]. Every method has unique benefits and drawbacks that are suited to certain therapeutic situations. When changes are seen on a screening mammography, for example, mammography is usually utilized to provide comprehensive images of the breast to help with diagnosis. For example, mammography has been proven to lower breast cancer mortality in women aged 50–69 by around 20% [8]. Breast ultrasonography is useful in differentiating among solid and fluid-filled lumps; solid lumps may signify benign fibrous growths or malignancy, while fluid-filled lumps are often benign. Compared to mammography, breast MRI provides more information, which makes it useful for identifying microscopic tumors or minor changes in the breast that would not be apparent on a mammogram. For instance, the breast MRI has proved to be 30% more sensitive than mammography in identifying breast cancer in its early stages [8]. If more assessment is required, a breast biopsy might be carried out. To help with a final diagnosis and treatment planning, an insignificant amount of tissue or a few cells are taken out of the suspicious place and examined. All things considered, a mix of screening techniques and imaging modalities enables thorough assessment and early identification of breast cancer, eventually leading to better patient outcomes [9,10].

A significant challenge in medical imaging, especially when associated with breast cancer, is the lack of annotated data. Training reliable diagnostic models requires high-quality, labeled datasets, yet obtaining such data may be time- and resource-consuming. As an illustration, annotated medical-imaging datasets are frequently restricted to small, specialized cohorts, which can lead to models that are not very generalizable to a wide range of populations. The lack of data makes it difficult to create advanced diagnostic devices and restricts the effectiveness of machine-learning models, especially in uncommon or under-represented instances.

This problem may be solved by synthetic data generation, especially when using advanced techniques like generative adversarial networks (GANs) [11,12,13]. GANs may enhance existing datasets by producing high-quality synthetic images that closely resemble real-world data. This adds more training examples to the dataset, which enhances the versatility and effectiveness of the model. For instance, it has been demonstrated in a number of applications that using synthetic data may increase the model training efficiency and accuracy by up to 15% [5,8]. The use of synthetic data can assist in overcoming the constraints imposed by small datasets, improve machine-learning models’ ability to generalize, and eventually, increase the efficiency and accuracy of diagnosis. A potent strategy for overcoming the problems associated with data scarcity is the use of synthetic data in medical imaging, which helps to advance the creation of more sophisticated diagnostic instruments.

The ability to automate the process of interpreting ultrasound images has evolved in recent years due to advancements in artificial intelligence and machine-learning techniques. This can greatly improve the diagnosis accuracy and efficiency. The use of deep learning, particularly with generative adversarial networks and EfficientNet designs, brings up new possibilities for classifying benign and malignant lesions in ultrasound images. Global health is seriously threatened by the rising incidence and mortality rates of breast cancer, which highlights the critical need for innovative research and creative solutions. Breast cancer significantly impairs women’s emotional health and quality of life, in addition to their physical health, on a global scale.

Because breast cancer can take many different forms and progress at different rates, it is complicated and requires an integrated strategy to enhance early detection, precise diagnosis, and successful treatment plans. The urgent need to address the rising incidence of breast cancer is what drives this study. Through delving more deeply into the specifics of breast anatomy, the ways that cancer progresses, and the powers of diagnostic methods, the key contributions are as follows:

• Synthetic data integration: We successfully integrate StyleGAN3-generated synthetic ultrasound images with actual data to improve the outcome of the proposed model for breast cancer detection.
• Improved metrics: Employing synthetic data considerably improves the classification accuracy, improving the metrics values by around 3% to 4%.
• Statistical validation: Our results are validated by thorough statistical analyses, including t-tests and Wilcoxon signed-rank tests, which reveal significant enhancements (p-values < 0.0001).
• Detailed assessment: We investigate our model’s performance using a variety of metrics (accuracy, precision, recall, and F1 score) for experiments on both real and synthetic datasets.
• Advancement in medical imaging: Our technique demonstrates that GANs could address data scarcity and increase diagnosis accuracy in medical-imaging applications.
• Scalability and efficiency: By utilizing EfficientNet-B7, our model is scalable and has low computing needs, making it appropriate for a range of medical-imaging applications.

2. Literature Review

Breast cancer detection is a significant area of medical research, and several machine-learning and deep-learning approaches have been used to increase the diagnosis accuracy. We conducted a literature evaluation to identify research gaps in breast cancer detection. This review highlights current breakthroughs in this sector, concentrating on diverse approaches and their performance indicators.

Sadad et al. (2018) [14] employed mammography scans to apply decision trees to the MIAS and DDSM datasets, achieving a remarkable 98.20% accuracy. Decision trees are well known for their simplicity and interpretability, making them an efficient diagnostic tool. The study’s drawbacks include a limited ability to capture complex connections in large datasets. Mughal et al. (2018) [15] and Kavitha et al. (2022) [16] employed backpropagation neural networks on the MIAS and DDSM datasets, as well as mammography modalities, and obtained accuracy of 99.00% and 98.50%, respectively, confirming neural networks’ usefulness in breast cancer detection. The problem is that the model is prone to overfitting due to its high complexity.

Vijayarajeswari et al. (2019) [17] and Kaur et al. (2019) [18] explored the utilization of SVM on the MIAS dataset and reached 94.00% accuracy. SVM is a convincing classifier that outclasses in high-dimensional spaces and is frequently applied in medical image analysis, especially breast cancer imaging techniques. The limitation of this study is that it imposes sensible collection of kernel functions and settings for best performance. Haris et al. (2024) [19] also employed SVM, acquiring the CBIS-DDSM dataset for mammography modality, and attained 98.8% accuracy. The authors exhibited excellent performance in medical image processing in this study by using structured data for breast cancer diagnosis tasks. A constraint of this study is that it imposes thorough optimization for the best results in terms of sensitivity to the kernel function and parameter choices.

Valvano et al. (2019) [20] and Mahesh et al. (2024) [21] used CNNs to achieve accuracies of 98.20% and 95.2%, respectively, using datasets such as CBIS-DDSM and Mini-DDSM, as well as breast histopathology images. CNNs are widely renowned for their capacity to automatically extract characteristics from images, making them ideal for medical-imaging applications for breast cancer detection. The limitation is that training requires vast volumes of annotated data, which is computationally costly. Cai et al. (2020) [22] achieved 93.7% accuracy with DCNN on the Local and INbreast datasets, proving the deep-learning model’s capacity to gather comprehensive visual information for accurate diagnosis. However, the restriction might be vanishing or inflating gradients in very deep networks. Vaka et al. (2020) [23] employed dense neural networks to assess a private hospital’s histopathology dataset, with an accuracy of 97.21%. This approach exhibits dense networks’ versatility while processing non-image input, such as histopathology slides. The model may struggle with high-dimensional data and may necessitate significant hyperparameter modification, indicating a drawback of our work. Ur Rehman et al. (2021) [24] employed FC-DSCNN on the DDSM and PINUM datasets and achieved 90.00% accuracy. This approach improves the diagnostic performance by combining deep and supervised learning. This study may be vulnerable to overfitting owing to its completely linked layers and vast parameter space.

Ragab et al. (2022) [25] applied CSO-MLP to the BUSI dataset and achieved an accuracy of 97.09%. CSO-MLP is a tailored method that improves the neural network performance of cancer detection applications. The potential sensitivity to hyperparameter selections and dataset features, which requires strong validation across multiple datasets to assure generalizability, appears to be the drawback of this work. Sheeba et al. (2023) [26] used the histopathology datasets Bisque and BreakHis to diagnose breast cancer using TCL-RAM, with a 97.00% accuracy rate. TCL-RAM improves the performance in histopathology image analysis by using transfer-learning techniques in pretrained models and adapting them to fresh datasets. The limitation of this study is that the variations in tissue preprocessing and imaging processes may restrict the transferability of previously learned properties. Yan et al. (2023) [27] used an ensemble classifier on the DDSM and MIAS datasets and achieved 93.26% accuracy. Ensemble approaches use many learning algorithms to increase the overall performance and resilience. The model appears to have limitations due to its increased complexity and significant processing cost. Asadi and Memon (2023) [28] achieved 96.80% accuracy using ResNet50 on the INbreast, BCDR, and WDBC datasets. Advanced designs, like as ResNet50, are recognized for their depth and capacity to tackle challenging image categorization problems. The model may necessitate substantial computational resources and skill for installation and training.

Huynh et al. (2023) [29] used EfficientNet and ConvNeXt designs on six mammography datasets, reaching an accuracy of 92.00%. These advanced CNN designs aim to strike a compromise between model complexity and computational economy, displaying strong performance over a wide range of datasets. The variability in dataset attributes and imaging techniques between institutions may impact the model generalizability, which is the model’s limitation. Bouzar-Benlabiod et al. (2023) [30] achieved 86.71% accuracy on the CBIS-DDSM dataset by combining SE-ResNet-101 with CBR. This method highlights the significance of customized deep-learning models designed for specific datasets. The applied model may not generalize well to different datasets or modalities.

Oyelade et al. (2024) [31] achieved 97.7% accuracy with TwinCNN on the MIAS and BreakHis datasets. TwinCNN models are helpful tools for diagnosing breast cancer since they are designed to handle a variety of imaging modalities. This investigation’s constraints include the need for perfect synchronization across many networks due to the increased complexity of model creation and training. Kadadevarmath and Reddy (2024) [32] proposed the DualNet-DL mammography model, which achieved 94.29% accuracy using the CBIS-DDSM and MIAS datasets. This model architecture combines two networks to improve the feature extraction and classification in breast cancer detection tasks. The drawback of this study is the potential complexity of model training and the interpretability caused by the dual network integration. AlSalman et al. (2024) [33] proposed the federated +DCNN approach for mammography-based breast cancer diagnosis across a number of datasets, including VINDR-MAMMO, CMMD, and INBREAST, with an impressive accuracy of 98.90%. This method use federated learning to train deep convolutional neural networks (DCNNs) jointly across several decentralized data sources while maintaining data privacy. The limitation of this study is the potential for diversity in the data quality and distribution between participating institutions, which may impair the model generalizability.

3. Materials and Methods

Our proposed methodology demonstrates the impact of synthetic data generation using the StyleGAN3 model over the conventional augmentation technique. Figure 1 presets a block diagram depicting the compete methodology utilized for the detection of breast cancer from the ultrasound image datasets. In the first stage, we have acquired the dataset from 4 different sources of breast cancer images, which mainly include 2 pathology types, benign and malignant. Since the images are collected from various sources, preprocessing techniques such as cropping and resizing to a uniform size have been applied. Then, the data have been distributed into the training and test set in the ratio of 80:20, after which conventional data augmentation and StyleGAN3 have been applied to the training dataset, whereas the test dataset has been reserved for validating the model. The EfficientNet-B7 DL model has been utilized for taring the datasets using 5-fold cross-validations techniques, and at last, model has been verified on the test dataset. The complete details about each step are presented in the following sections.

3.1. Data Collection

The foundation of this research was a dataset comprised of 3186 ultrasound images of breast lesions, collected from four different publicly available sources, including breast lesions ultrasound (BrEaST), BUSI, Thammasat, and HMSS (Figure 2). The images were categorized into two groups, malignant and benign, according to the clinical annotations provided. Our study was underpinned by an extensive analysis of ultrasound images of breast lesions, classified as malignant or benign. To achieve the most accurate and generalizable results, a comprehensive dataset was compiled from various sources. This process ensured a broad spectrum of characteristics and features in the ultrasound images, which is necessary for training and testing deep-learning models.

• BrEaST-Lesions USG—offers a collection of ultrasound images of breast lesions, equipped with detailed annotations, including clinical data and histological analysis results. We have utilized 252 images from this dataset, including 154 benign types and 98 malignant types [34].
• Dataset BUSI with GT—includes breast ultrasound images with precise demarcation of tumor boundaries, providing valuable data for training recognition and classification models. From this dataset, we have utilized 454 benign cancers and 211 malignant cancers, giving a total of 665 images [35].
• Thammasat University Dataset—contains images gathered by researchers at Thammasat University, including images with various artifacts, contributing to model training in conditions close to real clinical practice. From this open-source dataset, we have utilized 2006 breast ultrasound images, including 846 benign and 1160 malignant lesions [9].
• HMSS Ultrasound Cases—provides an extensive collection of ultrasound diagnostic cases, including images of breast formations, enriching the dataset with a variety of cases. From this dataset, we have used 263 images, in which 120 are benign and 263 are malignant [36].

Figure 2. The distribution of breast ultrasound images by pathology types across the datasets.

Figure 2. The distribution of breast ultrasound images by pathology types across the datasets.

3.2. Data Preprocessing

To create a standardized training set, data from all the sources underwent thorough processing and standardization. A key step in the preparation was the standardization of all the image sizes and the cropping operation to remove edges containing the ultrasound device interface from the images. This process helped eliminate irrelevant diagnostic elements and focus the model’s attention on the medical content of the images. We have developed a preprocessing approach that incorporates cropping and scaling images to increase the detection accuracy of breast cancer. Standardizing pictures from different sources requires this method to ensure uniformity in resolution and to focus on pertinent sections of breast tissue [37,38].

We utilized image-cropping techniques in order to differentiate between the breast tissue, which is vital to the diagnosis of cancer. Usually, areas of clinical relevance, such as tumors or certain breast regions, are highlighted by annotations or masks that define the region of interest (ROI). The bounding box coordinates ($x_{min},x_{max},y_{min},x_{max}$) of the ROI are found during the cropping process using either automatic segmentation methods or predetermined criteria [39]. By doing this, it is ensured that only relevant and potentially abnormal regions of the breast image are retained for further investigation.

We have then used the technique of resizing the cropped image to the specified target dimensions (512 × 512 pixels) after the ROI has been created and retrieved. In order to maintain image quality and details during transformation, bilinear interpolation has been used for resizing. We have employed an approach that is intended to standardize and resize clinical images to a standard resolution of 512 × 512 pixels, uniformly across all the utilized breast cancer datasets ($D_{\mathrm{BrEaST}},D_{\mathrm{BUSI}},D_{\mathrm{Thammasat}},D_{\mathrm{HMSS}}$). In order to ensure consistency in the image dimensions across the various datasets for accurate analysis and dependable training of models in breast cancer detection, standardization is critical in the medical-imaging field [40].

We started by defining our standard target dimensions for resizing as $\left(W_{\mathrm{target}}=512\right)$ and $\left(H_{\mathrm{target}}=512\right)$ pixels. In accordance with the best standards in medical imaging, these dimensions were chosen to strike the right balance between computing efficiency and the preservation of image information. Each dataset (D), including images from $D_{\mathrm{BrEaST}},D_{\mathrm{BUSI}},D_{\mathrm{Thammasat}},D_{\mathrm{HMSS}},$ is iterated through by the algorithm, which processes each image separately. In the dataset (D), we used bilinear interpolation for every image $I_i$. Because it preserves the image quality when scaling by smoothly interpolating the pixel values depending on nearby pixels, this approach is ideal for medical imaging [41]. To start resizing the cropped images, scaling factors are first calculated, as given by Equation (1).

$$\begin{gathered} {\mathrm{scale}}_w={\displaystyle \frac{W_{\mathrm{in}}}{W_{\mathrm{target}}}} \\ {\mathrm{scale}}_h={\displaystyle \frac{H_{\mathrm{in}}}{H_{\mathrm{target}}}} \end{gathered}$$

(1)

where $W_{\mathrm{in}}$ and $H_{\mathrm{in}}$ are the width and height of the original image $I_i.$ We used bilinear interpolation to standardize pictures of uniform width. The coordinates $\left(x_{\mathrm{in}},y_{\mathrm{in}}\right)$ in the original image $I_i$ corresponding to each pixel in the scaled image $I_i^{\prime }$ have been calculated using Equation (2).

$$x_{\mathrm{in}}={\mathrm{scale}}_w\times x_{\mathrm{out}}, y_{\mathrm{in}}={\mathrm{scale}}_h\times y_{\mathrm{out}}$$

(2)

Here, $\left(x_{\mathrm{out}},y_{\mathrm{out}}\right)$ denotes the coordinates in the resized image $I_i^{\prime }$, which ranges from $0 \mathrm{to} (W_{\mathrm{target}}-1)$ for $x_{\mathrm{out}}$ and $0 \mathrm{to} H_{\mathrm{target}}-1$ for $y_{\mathrm{out}}$. The variables $\left(x_{\mathrm{in}},y_{\mathrm{in}}\right)$ represent the corresponding coordinates in the original image $I_i$. In order to carry out bilinear interpolation with accuracy, the method uses Equation (3) to find the closest integer coordinates $\left(x_1,y_1\right)$ and their subsequent coordinates $\left(x_2,y_2\right)$ from the original image.

$$\begin{gathered} x_1=\left[x_{\mathrm{in}}\right], { y}_1=\left[y_{\mathrm{in}}\right] \\ {x}_2=\min \left(x_1+1,W_{\mathrm{in}}-1\right), y_2=\min \left(y_1+1,H_{\mathrm{in}}-1\right) \end{gathered}$$

(3)

In bilinear interpolation, the final pixel value in the scaled image is computed using the fractional weights alpha and beta. They are determined by taking into account the target pixel’s relative position across the original image frame. The horizontal interpolation weight between the two nearest pixels in a row is represented by α. In a given column, the weight given to the vertical interpolation between the two adjacent pixels is represented by β. The alpha and beta values are obtained using Equation (4) [42].

$$\alpha =x_{\mathrm{in}}-x_1, \beta =y_{\mathrm{in}}-y_1$$

(4)

To compute the pixel value $({I\prime }_{\mathrm{out}}\left[x_{\mathrm{out}},y_{\mathrm{out}}\right]$ in the resized image $I_i^{\prime }$, bilinear interpolation combines the values of the four closest pixels in the original image $I_i$. This process is explained by Equation (5):

$$I_{\mathrm{out}}\left[x_{\mathrm{out}},y_{\mathrm{out}}\right]=\left(1-\mathsf{\alpha }\right)\cdot \left(1-\mathsf{\beta }\right)\cdot I_{\mathrm{in}}\left[x_1,y_1\right]+\mathsf{\alpha }\cdot \left(1-\mathsf{\beta }\right)\cdot I_{\mathrm{in}}\left[x_2,y_1\right]+\left(1-\mathsf{\alpha }\right)\cdot \mathsf{\beta }\cdot I_{\mathrm{in}}\left[x_1,y_2\right]+\mathsf{\alpha }\cdot \mathsf{\beta }\cdot I_{\mathrm{in}}\left[x_2,y_2\right]$$

(5)

After processing all the images from the $D_{\mathrm{BrEaST}},D_{\mathrm{BUSI}},D_{\mathrm{Thammasat}},D_{\mathrm{HMSS}}$, the standardized images are saved in a separate dataset. This method guarantees that all the images are uniformly scaled to 512 × 512 pixels using bilinear interpolation. Algorithm 1 presents the complete preprocessing procedures used in our proposed technique in terms of pseudocode. Standardizing image dimensions is crucial in medical imaging because it enables uniform analysis and model training across different datasets. We improved the quality of the standardized images by using bilinear interpolation, which has detail-preserving features. This, in turn, provided precise and dependable medical image analysis, thereby enhancing the breast cancer diagnosis accuracy. These standardization and data preparation procedures were critically important for creating a high-quality and representative training set that unites a wide spectrum of ultrasound images of breast lesions. This approach ensures the high generalization ability of the developed models and increases the accuracy and efficiency of the ultrasound image classification.

Algorithm 1. Pseudocode for proposed preprocessing.

Input: Datasets $\left(D\right)\left[D_{\mathrm{BrEaST}},D_{\mathrm{BUSI}},D_{\mathrm{Thammasat}},D_{\mathrm{HMSS}}\right]$ Output: $D_{standardized}$ (standardized and uniformly resized images) # Step 1: Initialization Define $W_{target}$= 512 Define $H_{target}$= 512 $D_{standardized}$ = [] # Step 2: Process Each Dataset for each dataset D in $\left[D_{\mathrm{BrEaST}},D_{\mathrm{BUSI}},D_{\mathrm{Thammasat}},D_{\mathrm{HMSS}}\right]$:   for each image $I_i$ in dataset D:      # Step 2a: Crop Image Based on ROI      $x_{min}$ = calculate_$x_{min}$ ($I_i$)       $x_{max}$= calculate_$x_{max}$($I_i$)       $y_{min}$= calculate_$y_{min }$($I_i$)      $y_{max}$ = calculate_$y_{max}$ ($I_i$)       $I_i^{\mathrm{cropped}}$ = $I_i$[$y_{min}$: $y_{max}$, $x_{min}$: $x_{max}$]      # Step 2b: Resize Cropped Image to Uniform Dimensions      $W_{in}$ = width $\left(I_i^{\mathrm{cropped}}\right)$      $H_{in}$ = height $\left(I_i^{\mathrm{cropped}}\right)$      # Compute scaling factors      ${\mathrm{scale}}_w$ = $W_{in}$/$W_{target}$      ${\mathrm{scale}}_h$= $H_{in}$/$H_{target}$      # Initialize resized image I_i_prime      $I_i^{\prime }$ = $empt{y}_{image}$($W_{target}$, $H_{target}$)      # Perform bilinear interpolation for resizing      for $y_{out}$ from 0 to $H_{target}$− 1:         for $x_{out}$ from 0 to $W_{target}$− 1:          $x_{in}$ = ${\mathrm{scale}}_w\times x_{\mathrm{out}}$          $y_{\mathrm{in}}={\mathrm{scale}}_h\times y_{\mathrm{out}}$          $x_1$= floor($x_{in}$)         $y_1$ = floor($y_{in}$))         $x_2$ = min($x_1$ + 1, $W_{in}$ − 1)         $y_2$ = min($y_1$+ 1, $H_{in}$ − 1)         $\alpha =x_{\mathrm{in}}-x_1$         $\beta =y_{\mathrm{in}}-y_1$       $I_i^{\prime }\left[x_{\mathrm{out}},y_{\mathrm{out}}\right]=\left(1-\mathsf{\alpha }\right)\cdot \left(1-\mathsf{\beta }\right)\cdot I_i^{\mathrm{cropped}}\left[x_1,y_1\right]+\mathsf{\alpha }\cdot \left(1-\mathsf{\beta }\right)\cdot I_i^{\mathrm{cropped}}\left[x_2,y_1\right]+\left(1-\mathsf{\alpha }\right)\cdot \mathsf{\beta }\cdot I_i^{\mathrm{cropped}}\left[x_1,y_2\right]+\mathsf{\alpha }\cdot \mathsf{\beta }\cdot I_i^{\mathrm{cropped}}\left[x_2,y_2\right]$      # Step 2c: Store Standardized Image      $D_{standardized}$.append($I_i^{\prime }$) # Output the standardized and uniformly resized images return Dstandardized

3.3. Data Augmentation

We carefully addressed the possible class imbalance created by the use of synthetic data. We used class balancing strategies in our study, such as incorporating additional data augmentation (6 kinds) techniques to reduce any potential distortions. Furthermore, to make sure there were an equal number of classes within the synthetic data, we gathered multiple additional datasets (from four different sources and four types of datasets), including some that became readily accessible in 2024. This method enabled us to use the most recent and diverse data, considerably improving the quality and efficacy of the model training.

Data augmentation is a collection of techniques employed to generate additional data samples from an existing dataset [40]. The primary objective is to enrich the diversity and size of the dataset, thereby enabling ML/DL models to better learn from the available data. These techniques involve applying various transformations to the original images, such as random cropping, horizontal or vertical flipping, adding noise, blurring, random shifting, scaling, etc. During our training process, in addition to standardizing and cropping images, we integrated several data augmentation methods to augment the dataset and enhance the robustness of our models.

Before applying data augmentation, we partitioned the entire classification dataset into training and test datasets. The total dataset collected from all the sources comprised 3186 images for the classification of breast cancer in two categories: malignant and benign. Initially, the dataset was split into 80% (training data) and 20% (test data), resulting in 2548 images for training, with the remaining 638 images reserved for testing the models. Subsequently, we applied the following augmentation techniques exclusively to the training set (2548 images), while the test images were kept in reserve and not subjected to any processing techniques. Conventional augmentation such as horizontal flipping, vertical flipping, random shifts, scaling and rotations, perspective changes, and Gaussian noise addition have been applied to the real training images. These augmentation strategies were carefully selected to improve the model training and enhance the models’ ability to handle the diverse distortions often present in clinical settings. To provide a concise overview of the augmentation techniques employed, they have been organized in

Table 1. Various augmentation techniques utilized on the ultrasound images.

Augmentation Techniques	Description	Probability
Horizontal Flipping	Images are flipped horizontally	0.5
Vertical Flipping	Images are flipped vertically	0.5
Random Shifts, Scaling, and Rotations	Random shifts, scaling, and rotations applied to images	0.5
Perspective Changes	Perspective changes applied to images	0.5
Addition of Gaussian Noise	Gaussian noise added to images	0.7
Cutout Effect	Randomly masks out sections of the image	0.5

, while visual representations of the applied augmentation techniques can be found in Figure 3.

3.4. StyleGAN3

The StyleGAN3 architecture (Figure 4) is an advanced generative model that creates images of excellent quality by combining a number of carefully constructed layers and functions to improve the image fidelity, eliminate aliasing, and ensure the translation equivariance [43]. The architecture is made up of a mapping network, a synthesis network, layer operations, noise injection, and ToRGB layers, all of which are mathematically designed to help generate realistic images [44].

First, a latent vector $z\sim \mathcal{N}\left(0,I\right),$ derived from a Gaussian distribution, is transformed via the mapping network. The mapping network transforms z into an intermediate latent space (w) using an 8-layer multi-layer perceptron (MLP), given by Equation (6).

$$w=f\left(z\right)=MLP\left(z\right)$$

(6)

Next, using sinusoidal mappings, Fourier features are employed to encode high-frequency details (Equation (7)).

$$\varphi \left(z\right)=\left[\sin \left(\omega z\right),\cos \left(\omega z\right)\right]$$

(7)

where $\omega$ is a learnt frequency parameter. These modified features are passed into the synthesis network, which employs the features to create images by processing them through layers with increasing resolution [43,44].

The feature maps are then gradually passed through several layers via the synthesis network, each of which is impacted by the intermediate latent vector w, to gradually produce images. The Fourier features are first converted into baseline feature maps by the network’s Conv 1 × 1 layer, which also acts as a starting point for further feature extraction. The synthesis network is made up of numerous layers (L0 through L13), each intended to handle varying levels of picture detail. The coarse layers (L0 to L6) record the image’s main structure, including the fundamental forms and layouts. The network refines mid-level elements, such as complex frameworks like facial features and textures, as it moves through the middle layers (L7 to L11). Lastly, at the greatest possible quality of the image, the fine layers (L12 to L13) concentrate on catching minute features such as hair strands and skin textures [43]. Style modulation (Mod) is included into every layer, as expressed by Equation (8).

$$y=s_i\cdot x, s_i=\mathrm{Mod}\left(w\right)$$

(8)

This is followed by the demodulation process to regulate the feature magnitudes, as given in Equation (9).

$$\widehat{x}={\displaystyle \frac{y}{\sqrt{E\left[y^2\right]+ϵ}}}$$

(9)

After that, convolution blocks (Conv 3 × 3 or 1 × 1) are used to process the modulated and demodulated features. These blocks are essential for feature extraction and refinement at every resolution stage [44]. Moreover, the network makes use of exact upsampling and downsampling procedures, as given by Equation (10).

$$\mathrm{Downsample}\left(x\right)=x\downarrow s, \mathrm{Upsample}\left(x\right)=x\uparrow s$$

(10)

where a scaling factor is represented by $s$. The StyleGAN3 design is significantly impacted by noise inputs, which introduce stochastic variation at every layer. A noise is added, as expressed by Equation (11):

$$x_{\mathrm{out}}=x+b_i\cdot N_i$$

(11)

where the learned parameter weight is denoted by $b_i$ and noise vector is denoted by $(N_i\sim \mathcal{N}\left(0,I\right))$. The final feature maps are modulated to RGB using the ToRGB Layers, as given by Equation (12).

$$\mathrm{RGB}=Con{v}_{1x1}\left(\mathrm{Mod}\left(w\right)\right)$$

(12)

An exponential moving average (EMA) is utilized for smoother updates in order to stabilize the training, as given by Equation (13).

$${\theta }_{\mathrm{EMA}}=\alpha {\theta }_{\mathrm{EMA}}+\left(1-\alpha \right){\theta }_{\mathrm{current}}$$

(13)

StyleGAN3 can produce high-fidelity, alias-free images with realistic features and a consistent structure by combining these procedures. However, while we acknowledge the potential of other image-generating techniques, such diffusion models, we believe that GANs—specifically, StyleGAN3—may be more advantageous for enhancing medical datasets, particularly those employed in diagnostics. One major factor is the high complexity and computing needs involved with diffusion models. Compared to GANs, these models usually need more processing power and take longer to train, which makes them less useful in resource-constrained applications like hospital settings. Diffusion models also frequently offer less control over the generating process, which may result in inaccurate or distorted diagnostic characteristics. However, GANs such as StyleGAN3 provide more control over the material that is generated, guaranteeing that the pictures fulfill certain standards that are essential for medical applications.

3.5. Data Generation Using StyleGAN3

To enhance the quality and diversity of the training dataset, generative adversarial network technology (StyleGAN3) [44] was employed, configured to create ultrasound images of breast lesions. A crucial aspect of the generation process was the production of images with a resolution of 512 × 512 pixels. This size was chosen to preserve the high image detail and quality, which is critically important for ensuring the accuracy of the subsequent classification. The model was trained separately for each class of lesion, which allowed for the generation of category-specific images while maintaining their high resolution. The training process of the StyleGAN3 model continued until kimg = 1000 was reached, which provided a sufficient degree of training for generating high-quality 512 × 512 images. This resolution was strictly maintained throughout the entire generation process to ensure that all the generated images would have consistent sizes and the quality required for accurate and efficient classification.

(Explanation: kimg = 1000 means that the training process will continue until the model has seen 1,000,000 (1000 thousand) real images in total. The quality of the generated images was assessed using the Fréchet inception distance (FID) metric, which quantitatively measured the closeness of the distribution of the generated images to the original dataset. It is reported that the FID metric reached 22.9 for malignant lesions and 35.4 for benign lesions, indicating the high quality and realism of the images generated at a resolution of 512 × 512).

Following the completion of training of the StyleGAN3 model and achieving FID values of 22.9 for malignant lesions and 35.4 for benign lesions, a detailed visualization of the results was conducted to assess the quality of generation. For each class, 25 generated images were selected and visualized in a tile format, allowing for a clear evaluation of the diversity and realism of the synthetic images (Figure 5). This visualization serves as further confirmation of the effectiveness of using generative adversarial networks to enhance training datasets in medical diagnostics.

Synthetic Image Generation

Our approach leveraged the StyleGAN3 architecture to synthesize realistic ultrasound images from random noise vectors. The generator within the GAN model transforms random noise vectors, $z,$ sampled from a latent space into high-resolution ultrasound images, represented as $x,[z\sim \mathcal{N}\left(\mathrm{0,1}\right)]$. The core of the generator consists of stacked convolution blocks [45], which progressively enhance the resolution of the generated images. These blocks enable the generator to capture fine-grained details and larger-scale features simultaneously, as is expressed by Equation (14).

$$x=G\left(z\right)$$

(14)

The discriminator component of StyleGAN3 distinguishes between real $\left(x_{\mathrm{real}}\right)$ and generated $\left(x_{\mathrm{fake}}\right)$ ultrasound images. It achieves this through a series of convolutional blocks, as represented by Equation (15).

$$\begin{gathered} P_{\mathrm{real}}=D\left(x_{\mathrm{real}}\right) \\ {P}_{\mathrm{fake}}=D\left(x_{\mathrm{fake}}\right) \end{gathered}$$

(15)

The discriminator’s output, $P_{\mathrm{real}}\mathrm{a}\mathrm{n}\mathrm{d}{P}_{\mathrm{fake}},$ represents the probabilities that the input samples are real or fake, respectively, as described by Equation (16).

$$\begin{gathered} {\widehat{y}}_{real}=\sigma \left(P_{\mathrm{real}}\right) \\ {\widehat{y}}_{fake}=\sigma \left(P_{\mathrm{fake}}\right) \end{gathered}$$

(16)

By training the generator and discriminator adversarially, StyleGAN3 learns to generate ultrasound images that closely resemble real ones, offering a valuable tool for various applications in medical imaging and research.

3.6. EffiecientNetB7

Selection of the right classifier is very important for an accurate and efficient detection process. There are several deep-learning classifiers available for the classification of medical images, such as CNNs, LSTMs, AttentionNet, transformers, EfiicientNets and many more [46,47]. CNN are the most utilized models for the medical image classification task. To improve the accuracy of the models, they are scaled up; for example, Resnet-18 scaled up to ResNet-201 [48]. There are only three dimensions available to scale up the CNN models: width, depth, and image resolution. The CNN operation can be defined by Equation (17).

$$\begin{gathered} N={\odot }_{i=1....S} F_i^{L_i}\left(X_{<H_i, W_i, C_i>}\right) \\ N=F_k\odot ....F_1\odot F_1\left(X_1\right)={\odot }_{j=1...k}{F}_j\left(X_1\right) \\ {Y}_{ij}^l=\sum _{a=1}^{A^{l-1}}\sum _{b=1}^{B^{l-1}}{F}_i^l\cdot X_{a+\left(i-1\right)s_l,b+\left(j-1\right)s_l}^{l-1}+b_i^l \end{gathered}$$

(17)

Scaling up the dimensions of the models in width, depth, and resolution increases the accuracy, but for the bigger model, the accuracy gain diminishes. To overcome this, compound scaling is introduced. In this case, the model is scaled up in an efficient way in all three dimensions simultaneously, which will extract more complex features and fine details without increasing the computational cost [49,50]. The compound scaling is defined by Equation (18).

$$\begin{gathered} \mathit{depth}\left(d\right)={\alpha }^{\phi } \\ \mathit{width}\left(w\right)={\beta }^{\phi } \\ \mathit{resolution}\left(r\right)={\gamma }^{\phi } \\ s.t.\alpha .{\beta }^2.{\gamma }^2\approx 2 \\ \alpha \geqq 1,\beta \geqq 1,\gamma \geqq 1 \end{gathered}$$

(18)

Here, $\alpha ,\beta ,\gamma$ are the scaling coefficients, which can be determined through a grid search, and $\phi$ is a nonnegative scaling step that is user specific. The FLOPS of the CNN model is directly proportional to the depth, width2 and resolution2, so if we double the depth, the FLOPS will be double, but if we double the width or resolution, the FLOPS becomes 4 times greater. In EfficientNet, if we scale the model using the above equation, the FLOPS will be scaled up by ${\left(\alpha .{\beta }^2.{\gamma }^2\right)}^{\phi }$. So, our main objective is to optimize the FLOPS as well as the accuracy, as given by Equation (19) [50].

$$Accurcay\times {\left[{\displaystyle \frac{FLOPS}{T}}\right]}^w$$

(19)

where Accuracy is the model’s performance on the target task, FLOPS is the model’s computational cost, measured by floating-point operations, T is a target FLOPS budget, if specified, and w is a hyperparameter set to −0.07 (a negative value emphasizes accuracy). The objective function aims to maximize the product of the accuracy and a term that penalizes high FLOPS. The negative weight w of −0.07 emphasizes the importance of achieving high accuracy while still keeping the FLOPS under control.

The EfficientNet models come with 8 variants from EfficientNetB0–EfficientNetB7. In this work, we have utilized the EfficientNetB7 model, which best suited for our application (breast cancer detection task). Figure 6 illustrates an architectural view of the EfficientNet-B7, along with layer wise distribution. This model includes the mobile inverted residual block (MBConv), which is a building block of the EfficientNet model. The MBConv block utilizes the depthwise separable convolution (Depthwise Conv) block, along with squeeze and excitation (SE) block. The depthwise convolution block concentrates on extracting characteristics unique to each color channel in the image while collecting spatial information within each channel. It ignores the connections among these channels, though. Mathematically, this operation is represented in Equation (20).

$$Y_d\left(i,j,k\right)=\sum _{m=0}^{M-1}\sum _{n=0}^{N-1}X\left(i+m,j+n,k\right)\cdot K_d\left(m,n,k\right)$$

(20)

where $X$ represents the input image, $K_d$ represents the depthwise filter, $M$ and $N$ are the filter dimensions, and $Y_d\left(i,j,k\right)$ represents the output feature map at position $\left(i,j\right)$ for channel $k.$

The output of the depthwise convolutions is combined with the $(1\times 1)$ pointwise convolution, as defined by Equation (21).

$$Y_p\left(i,j,k\right)=\sum _{c=0}^{C-1}{W}_p\left(i,j,k,c\right)\cdot Y_d\left(i,j,c\right)$$

(21)

where the number of channels is denoted by $C,$ the $1\times 1$ convolution filter is represented by $W_p$, and the output following the pointwise convolution is given by $Y_p\left(i,j,k\right).$ Squeeze and excitation (SE) blocks are incorporated into the architecture to enhance the feature extraction by recalibrating the channel-wise feature response, as given in Equation (22).

$$S\left(x\right)=\mathsf{\sigma }\left(W_2\cdot \mathsf{\delta }\left(W_1\cdot {\displaystyle \frac{1}{H\cdot W}}\sum _{i=1}^H\sum _{j=1}^{W}x\left(i,j,:\right)\right)\right)$$

(22)

where $H \mathrm{a}\mathrm{n}\mathrm{d} W$ are the height and width of the feature maps, and $x(i, j, :)$ represents the feature map values at position $\left(i,j\right)$ through all the channels. $\delta$ is the ReLU activation function, which introduces non-linearity, $W_1$ and $W_2$ are the weights of the two fully connected (FC) layers, and σ is the sigmoid activation function. These blocks may restrict less informative channels and highlight more informative ones. By concentrating on the most pertinent information found in the data, this procedure improves the feature extraction. An architectural view of the blocks, (a) SE block, (b) MBConv1(3 × 3), (c) MBConv6 (3 × 3), and (d) MBConv6 (5 × 5), used in the EfiicientNet-B7 model is shown in Figure 7.

EfficientNet-B7’s architecture feeds the input image to the Conv 3 × 3 block, a fundamental conventional convolution block that extracts the crucial features from the input image. Following the first Conv 3 × 3 block, the structure comprises numerous blocks (blocks 1–7) that use MBConv blocks of variable numbers. Block 1 has three MBConv1 blocks, where each MBConv1 block is represented by Equation (23).

$$Y_{\mathrm{MBConv}1}\left(i,j,k\right)=\sum _{k=0}^{C-1}\left(\sum _{m=0}^2\sum _{n=0}^{2}X\left(i+m,j+n,k\right)\cdot K_d\left(m,n,k\right)\right)\cdot W_p\left(i,j,k,c\right)$$

(23)

Here, $\mathrm{DWConv}$ represents the depthwise convolution and $\mathrm{PWConv}$ denotes the pointwise convolution.

Blocks 2 and 3 employ seven MBConv6 (3 × 3) and MBConv6 (5 × 5) blocks, respectively. This design enables the model to capture features at various sizes within each channel, as formalized in Equation (24).

$$\begin{gathered} Y_{\mathrm{MBConv}6 (3\times 3)}\left(i,j,k\right)=\sum _{k=0}^{C-1}\left(\sum _{m=0}^2\sum _{n=0}^{2}X\left(i+m,j+n,k\right)\cdot K_d\left(m,n,k\right)\right)\cdot W_p\left(i,j,k,c\right) \\ {Y}_{\mathrm{MBConv}6 (5\times 5)}\left(i,j,k\right)=\sum _{k=0}^{C-1}\left(\sum _{m=0}^4\sum _{n=0}^{4}X\left(i+m,j+n,k\right)\cdot K_d\left(m,n,k\right)\right)\cdot W_p\left(i,j,k,c\right) \end{gathered}$$

(24)

Similarly, 10 MBConv6 (3 × 3) and MBConv6 (5 × 5) are utilized in blocks 4 and 5, respectively. Block 7 employs 13 MBConv6(5 × 5), followed by 4 MBConv6 (3 × 3) in block 6, as it proceeds through the feature map. The model is able to extract more complicated characteristics for precise breast cancer classification because of the hierarchical structure with increasing complexity in the MBConv blocks.

3.7. Evaluation Metrics

In our analysis, we employed some standard evaluation metrics, as defined by Equations (25)–(28).

$$\mathit{Accuracy}(\mathit{Acc})={\displaystyle \frac{TP+TN}{TP+TN+FP+FN}}$$

(25)

$$\mathit{Precision}(\mathit{Pre})={\displaystyle \frac{TP}{TP+FP}}$$

(26)

$$\mathit{Recall}(\mathit{Rec})={\displaystyle \frac{TP}{TP+FN}}$$

(27)

$$F1\text{-}\mathit{score} (F1)={\displaystyle \frac{2\ast Pre\ast Rec}{Pre+Rec}}$$

(28)

The foundation for analyzing classification models lies in the confusion matrix. This matrix methodically presents the quantity of accurate classifications (true positives—TPs, true negatives—TNs) and incorrect classifications (false positives—FPs, false negatives—FNs) for a given task. The confusion matrix holds specific significance in circumstances where missing positive cases carry considerable concerns [47]. A high recall value ensures the model minimizes the risk of dominating serious instances, even if it means accepting a higher rate of false positives. This trade-off is crucial, particularly when the cost of missing a positive case outweighs the cost of falsely identifying a negative one [51,52].

4. Results

We have performed two separate experiments for the detection of breast cancer utilizing ultrasound datasets from four sources. The first experiment was on the original or real dataset with simple augmentation and the second experiment with real and synthetic dataset generated using StyleGAN3, utilizing the EfficientNet-B7 model. For the classification of ultrasound images into malignant and benign lesions, we selected the EfficientNet-B7 architecture based on preliminary experiments, where this architecture demonstrated the best performance on our dataset. This choice was driven by the fact that the EfficientNet-B7 model offers an optimal balance between the number of parameters and computational efficiency, which is especially important when working with high-resolution medical images. During the experiments, it was found that models with fewer parameters, such as EfficientNet-B5, showed lower accuracy, indicating insufficient model complexity for successful training on this dataset. The initial dataset included 3186 ultrasound images collected from various public sources. From this pool, 20% (638 images) were set aside for testing using a fixed seed to ensure the consistency and reproducibility of the results. The remaining data were split into five folds for cross-validation, ensuring that each image would be used in the training and validation sets in different iterations of the cross-validation.

Both experiments were carried out on common hardware and software setups to examine the performance of both methodologies. The hardware used in the experiments was a computer system with 16 GB RAM, a 500 GB SSD hard drive, and 12 GB GPU support for smooth and quick training of the used model. The graphics processing unit (GPU) is especially important for sophisticated data processing and accelerating deep-learning training. In the software part, we used the Python 3.10 programming environment and the PyTorch Python library to create the DL model. We also utilized the same hyperparameters for both experiments: 50 epochs, Adam optimizer, 0.0001 learning rate, five-fold cross-validation training technique, and input image size 512 × 512.

4.1. Experimentation on Real Dataset

In the first experiment, we utilized the original 3186 ultrasound images, among which 80% (2548) images were used for training, along with some augmented images. In this experiment, we have used the real images with some augmentation applied, such as horizontal flipping, vertical flipping, random shifts, scaling, and rotations, perspective changes, and addition of Gaussian noise. We have utilized the common DL model EfficientNet-B7 for both the experiments, with 50 epochs as the input size as the images are of high quality, 512 × 512. The five-fold cross-validation strategy was utilized for training the utilized model, and for each fold, we have plotted the training history in terms of the training and validation accuracy and loss, respectively. Figure 8, Figure 9, Figure 10, Figure 11 and Figure 12 illustrate the training history curves for the first experiment on a real dataset. These curves provide the model training summary in terms of the loss and accuracy during five-fold cross-validation, where each fold utilizes a separate set of training and validation data.

The test result using the EfficientNet-B7 model in terms of the confusion matrix is presented in Figure 13 on the test dataset (638 US images). From the confusion matrix of the first experiment, we can observe that the model identifies 291 malignant and 275 benign classes and misclassifies 30 benign and 42 malignant classes. Also, the performance of each fold in terms of the evaluation metrics, accuracy, F1 score, precision and recall, for each fold is presented in

Table 2. Performance metrics of breast cancer detection on the test dataset of the first experiment.

Metric	Fold 0	Fold 1	Fold 2	Fold 3	Fold 4	Aggregated
Accuracy	0.8558	0.8621	0.8543	0.8668	0.8746	0.8871
F1 Score	0.8557	0.8621	0.8543	0.8667	0.8745	0.8872
Precision	0.856	0.8621	0.8569	0.8718	0.8748	0.8879
Recall	0.8558	0.8621	0.8542	0.8668	0.8746	0.8871

. The accuracy metrics among all five folds show reliable performance, where fold 0 achieves 0.8558 accuracy, whereas highest accuracy is achieved in fold 4 with 0.8746. The overall aggregated accuracy for this experiment is 88.71%, which shows the good reliability of the model for correctly classifying breast cancer from ultrasound images.

The precision score for this experiment varies from 0.8560 in fold 0 to 0.8748 in fold 4, and the aggregated precision value is 88.79%. This precision score shows that the model has a good capability for identifying positives cases (malignant cancer) for the detection of breast cancer. If we look at the recall values, it varies from 0.8542 for fold 2 to 0.8746 for fold 4. The aggregated recall value is 88.71%, which shows the good ability of the model to correctly classify the actual cancerous images. In terms of the F1 score metrics, the harmonic mean of the precision and recall varies from 0.8543 (fold 2) to 0.8745 (fold 4). The overall aggregated score is 88.72% in the first experiment using the proposed model, which shows the good ability to minimize the false negative and identify the true positive, along with the good predictions.

If we observe the overall performance of the model in the first experiment on a real dataset, firstly, it shows consistent performance with slight variations in terms of all the evaluation metrics. This proves that the model generalizes well across all five subsets of the data, which is very curial for the real-time application. Secondly, fold 4 provides the highest values for each metrics, accuracy of 0.8746, F1 of 0.8745, precision of 0.8748, and 0.8746, which show that the model performs optimally on this subset, providing strong classification ability and consistency across all metrics. Similarly, fold 2 shows the lowest performance, with an accuracy of 0.8543, an F1 Score of 0.8543, a precision of 0.8569, and a recall of 0.8542. These differences, although minor, indicate areas for potential model improvement, such as handling variations in data that are less represented in other folds. The overall aggregated score for all the metrics is 88.71% accuracy, 88.72% F1 score, 88.79% precision, and 88.71% recall, which show that model is reliable and effective in the detection of breast cancer from ultrasound images.

4.2. Experimentation on Combined Dataset (Real and Synthetic)

The second experiment was performed on the combined dataset, which included the real data and GAN-generated ultrasound images. The same model architecture, hyperparameters and all the other environments were utilized while working with the combined dataset. The main difference was in the expansion of the training set by including 10,000 generated images (5000 malignant and 5000 benign), created using the generative adversarial network StyleGAN3. These generated images, with a resolution of 512 × 512 pixels, were added to 2548 original images (after setting aside the test sample), creating a combined dataset of 12,548 images for training. Combining these augmentations with the diversity of features presented in the generated data enabled the creation of a powerful and comprehensive training set. The application of augmentations significantly expanded the model’s capabilities in image classification, increasing its accuracy and resistance to overfitting. In Figure 14, the distribution of each pathology type, malignant and benign, after the GAN augmentation is shown.

Training on the combined dataset was also conducted using five-fold cross-validation and similar augmentation methods as for the original dataset. Additionally, this approach allowed for the exploration of the impact of generated data on the model’s generalization ability and its accuracy. Special attention was paid to the analysis of the quality metrics, including the F1 score, to assess the effectiveness of the models trained on the expanded dataset. We have also illustrated the training history curves for the second experiment in Figure 15, Figure 16, Figure 17, Figure 18 and Figure 19 on the Real+GAN dataset.

The second experiment also employed the EfficientNet-B7 model on a Real+GAN augmented dataset, which also utilized the same test set of 638 ultrasound images. Figure 20 depicts the result in terms of the confusion matrix, which provides the model with enhanced performance for breast cancer diagnosis. It can be observed from the confusion matrix that the model correctly classifies 302 (true positive) malignant cases and 285 (true negative) benign cases. The number of false positives, or benign cases misclassified as malignant, has decreased to 20. This reduction represents a significant improvement in the model’s accuracy, demonstrating its improved ability to prevent false alarms in benign situations.

Similarly, the number of false negatives, or malignant cases misclassified as benign, was reduced to 31, demonstrating the model’s better recall in recognizing true malignant instances. In comparison to the confusion matrix from the first experiment, the second experiment’s matrix shows a considerable increase in all of the crucial areas: greater true positive and true negative counts, as well as lower false positive and negative counts. This suggests that the GAN-augmented dataset successfully improved the model’s capacity to generalize and reliably detect breast cancer in ultrasound images.

The second experiment, which used the EfficientNet-B7 model on a combined dataset improved with GAN-augmented images of excellent quality, showed a considerable improvement in performance across all the metrics. The performance of the second experiment in terms of all the evaluation metrics on the combined dataset is presented in

Table 3. Performance metrics of breast cancer detection on the test dataset in the second experiment.

Metric	Fold 0	Fold 1	Fold 2	Fold 3	Fold 4	Aggregated
Accuracy	0.8934	0.9044	0.895	0.8746	0.8715	0.9201
F1 Score	0.8934	0.9044	0.895	0.8746	0.8715	0.9201
Precision	0.8935	0.9061	0.8952	0.8748	0.8748	0.9207
Recall	0.8934	0.9044	0.895	0.8746	0.8715	0.9201

. The accuracy metrics’ values across the folds range from 0.8715 (fold 4) to 0.9044 (fold 1), where the aggregated accuracy is 92.01%. The introduction of GAN-augmented images obviously contributed to this improvement by providing the model with more high-quality data from which to train, hence improving its generalization capabilities. The precision score varies between 0.8748 (Fold 3) to 0.9061 (Fold 1), with an average precision of 92.07%, indicating that the model can properly detect the majority of malignant instances. High recall is vital for detecting most cancer cases and reducing the chance of missed diagnosis. Recall scores ranging from 0.8715 (Fold 4) to 0.9044 (Fold 1), with an aggregated recall of 92.01%, show that the model can accurately identify the majority of malignant cases. High recall is important for detecting the majority of cancer cases and reducing the possibility of missed diagnosis. The F1 scores, which balance precision and recall, were consistently high across all the folds, ranging from 0.8715 to 0.9044, with an average of 0.9201. The uniformity of the F1 score across various folds shows that the model performed well in both recognizing real positive events and limiting false positives. These metrics demonstrate that the GAN-augmented dataset improved the EfficientNet-B7 model’s ability to distinguish between benign and malignant instances, increasing the model’s sensitivity and reliability.

These metrics reveal that the GAN-augmented dataset enabled the EfficientNet-B7 model to better distinguish between benign and malignant cases, enhancing both the reliability and the sensitivity of the model. Figure 21, Figure 22, Figure 23, Figure 24 and Figure 25 provide a comparison of the performance over all the folds as well as aggregated results for all the metrics (accuracy, recall, precision, F1 score, and aggregate score). It is clear that in the second experiment utilizing the combined (Real+GAN) dataset, all the metric values increase or improve, with the exception of fold 4, where the first experiment scores on the real dataset are marginally higher than those on the combined dataset. The aggregated values across all the metrics—accuracy, precision, recall, and F-score—increase by 3.3%, 3.28%, 3.3%, and 3.29%, respectively, for the second experiment utilizing the combined dataset (Real+GAN). This finding emphasizes the overall improvement made by incorporating GAN-generated data into the training set.

4.3. Statistical Analysis

In addition to evaluating the Real and Real+GAN datasets using numerous metrics, we performed extensive statistical analyses to estimate the significance of the observed differences. More precisely, we used the t-test and the Wilcoxon signed-rank test to determine the statistical significance of the metrics across the two experimental scenarios. This section summarizes the analysis and findings.

4.3.1. T-Test Analysis

We conducted a t-test to evaluate if there was any significant variation between the metrics in the Real dataset and the Real+GAN dataset. To determine if the outcomes of two groups differ statistically from one another, the t-test is frequently used to compare the results.

Table 4. T-test results and corresponding hypothesis for the metrics.

Metric	Real+GAN Dataset	Real Dataset	t-Value	p-Value	Hypothesis Result
Accuracy	92.01	88.71	6.4	<0.0001	Reject H0; Significant
F1 Score	92.01	88.72	6.38	<0.0001	Reject H0; Significant
Precision	92.07	88.79	6.36	<0.0001	Reject H0; Significant
Recall	92.01	88.71	6.4	<0.0001	Reject H0; Significant

provides a summary of the t-test results and the corresponding hypothesis. The t-test produces t-values that are substantially larger than zero (~6.3) for all the metrics, and the associated p-values that are much lower (0.0001) than the standard significance level of 0.05. As a result, we must reject the null hypothesis (H0) for each metric, proving that there is a statistically significant difference within the experiments performed on Real dataset and the Real+GAN dataset.

4.3.2. Wilcoxon Signed-Rank Test Analysis

We also used the Wilcoxon signed-rank test to assess the significance of the variations between matched samples of both experiments on the Real and Real+GAN datasets.

Table 5. Wilcoxon signed-rank test results with the corresponding hypothesis for the metrics.

Metric	Real Dataset	Real+GAN Dataset	Absolute Difference	Rank	Wilcoxon Statistic	p-Value	Hypothesis Result
Accuracy	88.71	92.01	3.3	3.5	0	<0.0001	Reject H0; Significant
F1 Score	88.72	92.01	3.29	2	0	<0.0001	Reject H0; Significant
Precision	88.79	92.07	3.28	1	0	<0.0001	Reject H0; Significant
Recall	88.71	92.01	3.3	3.5	0	<0.0001	Reject H0; Significant

provides a summary of the Wilcoxon signed-rank test results with the corresponding hypothesis for each metric. The Wilcoxon test is a non-parametric approach that does not presuppose the normal distribution of the data, making it appropriate for instances in which the data do not fit the t-test assumptions. The results of the Wilcoxon signed-rank test indicate that every metric has a Wilcoxon statistic of 0, with p-values below 0.0001. The null hypothesis (H0) is rejected, and the statistical significance of the observed differences is confirmed, as this demonstrates a very significant difference between the experiments on both datasets.

The findings of the Wilcoxon signed-rank test and the t-test consistently demonstrate that there are notable distinctions between both experimentations on the Real and Real+GAN datasets for each metric that was evaluated. The statistical reliability of the observed gains (or changes) in the metrics with the inclusion of GAN data is confirmed by the substantial p-values derived from these tests, which rule out the possibility of random variation.

5. Discussion

In this section, we discuss and compare our proposed techniques with the state-of-the-art techniques for the detection of breast cancer. For this purpose, we have included several studies on breast cancer detection published between 2018 and 2024. Also, we have included several criteria, such as techniques utilized, datasets used, imaging modalities, and metrics (%). From

Table 6. State-of-the-art comparison of breast cancer detection techniques on imaging datasets ¹.

Author/Year	Literature	Technique	Dataset	Modality	Metrics (%)
(Sadad et al., 2018)	[14]	Decision Tree	MIAS, DDSM	MMI	Acc: 98.20 Sen: 100.00 Spe: 97.00
(Mughal et al., 2018)	[15]	BPNN	MIAS, DDSM	MMI	Acc: 98.50 Sen: 100 Spe: 95.0
(Vijayarajeswari et al., 2019)	[17]	SVM	MIAS	MMI	Acc: 94.00
(Kaur et al., 2019)	[18]	SVM, LDA	MIAS	MMI	Acc: 94.00
(Cai et al., 2020)	[22]	DCNN	Local, INbreast	MMI	93.7
(Vaka et al., 2020)	[23]	DNNS	Private Hospital	H&E	Acc: 97.21 Pre: 97.90 Rec: 97.01
(Ur Rehman et al., 2021)	[24]	FC-DSCNN	DDSM, PINUM	MMI	Acc: 90.00 Sen: 99.00 Spe: 82.00 F1: 85.00 Pre: 89.00 Rec: 82.00
(Hekal et al., 2021)	[53]	SVM	CBIS-DDSM	MMI	Acc: 94.00
(Ragab et al., 2022)	[25]	CSO-MLP	BUSI	US	Acc: 97.09 Sen: 95.54 Spe: 97.65 Pre: 94.76
(Kavitha et al., 2022)	[16]	BPNN	Mini-MIAS, DDSM	MMI	Acc: 98.50 Sen: 98.46 Spe: 99.08 F1: 98.91
(Sheeba et al., 2023)	[26]	TCL-RAM	Bisque, Break His	H&E	Acc: 97.00 Sen: 93.00 Spe: 94.00
(Yan et al., 2023)	[27]	Ensemble Classifier	DDSM, MIAS	MMI	Acc: 93.26 Pre: 90.40 Rec: 82.31 F1: 89.89 Spe: 93.20
(Bouzar-Benlabiod et al., 2023)	[30]	SE-ResNet-101 + CBR	CBIS-DDSM	MMI	Acc: 86.71 IoU: 64.00 F1: 75.00 Rec: 76.00 Pre: 81.00
(Huynh et al., 2023)	[29]	EfficientNet, ConvNeXt	Six Datasets	MMI	Acc: 92.00 Pre: 92.00 Rec: 92.00 F1: 97.00
(Kadadevarmath & Reddy, 2024)	[32]	DualNet-DL model	CBIS-DDSM, MIAS	MMI	Acc:94.29 Pre:98.32 Sen: 94.74 Spe: 94.74 F1: 95.79
(AlSalman et al., 2024)	[33]	Federated +DCNN	VINDR-MAMMO, CMMD, INBREAST	MMI	Acc:98.90
(Oyelade et al., 2024)	[31]	TwinCNN	MIAS and BreakHis	H&E, MMI	Acc: 97.7 (H&E) Acc:91.3 (MMI) Acc: 68.4 (Fused)
(Mahesh et al., 2024)	[21]	CNN	CBIS-DDSM, Mini-DDSM Breast Histopathology Images,	MMI, H&E	Acc:95.2
Proposed		EffiecientNet-B7	BrEaST BUSI Thammasat HMSS	Ultrasound	Acc: 92.01 Pre: 92.07 Rec: 92.01 F1: 92.01

¹ Abbreviations: BPNN, backpropagation neural network; BUSI; breast ultrasound image dataset; CSO-MLP, cat swarm optimization with the multilayer perceptron; DCNN, deep convolutional neural network; DDSM, digital database for screening mammography; DNNS, deep neural network with support value; FC-DSCNN, fully connected depth wise-separable convolutional neural network; H&E, hematoxylin and eosin (histopathology); LDA, linear discriminant analysis; MIAS, Mammographic Image Analysis Society; MMI, mammography imaging; SVM, support vector machine; TCL-RAM, transfer learning integrated with regional attention mechanism; US, ultrasound.

, we can observe that the authors have utilized several modalities, such as mammography imaging (MMI), histopathology (H&E), and ultrasound imaging (US), from various datasets. The most commonly utilized modality for the detection of breast cancer is mammography and the most commonly utilized datasets are the DDSM (Mini or CBIS) [14,15,16,21,24,27,30,32,53], which shows the acceptability of the DDSM datasets.

There are many types of techniques have been used by many authors, including machine-learning models such as SVM, BPNN, DT, MLP, etc.) and deep-learning models (CNN, DCNN, ResNet, EffiecientNet, etc.). The highest accuracy (98.90%) for the detection is achieved by the authors in [33], utilizing federated learning techniques in combination with the DCNN model on multiple datasets, VINDR-MAMMO, CMMD, INBREAST with the mammography imaging modality. Although the authors have achieved the highest accuracy among all the utilized studies, they have mainly focused on the accuracy metrics, which may not be the only criteria to decide the supremacy. The study in [29] used a maximum of six datasets utilizing EfficientNet and ConvNeXt models on MMI modality and achieved 92.00% accuracy, precision, recall and 97% F1 score. But this score is not on the combined dataset, as they have performed the experiment on the individual dataset. Some authors [21,29,33] have utilized more than two datasets, but they have performed the experiment of individual datasets.

The multiple modalities are also utilized in only [21,24], which achieved 95.2% accuracy and 97.7% accuracy respectively. But the issue is that the authors of [21] only provided the accuracy metrics, whereas the authors of [24] have achieved 97.7% accuracy with the H&E modality, but with the MMI and fused modalities, they achieved 91.3% and 68.4%, respectively.

In our study, we utilized four different datasets from four different sources and then generated 10,000 high-quality images using StyleGAN3, which has not been done by most of the researchers. Also, we have used ultrasound images that are very easily and quickly acquired around every corner of the world through non-invasive techniques. We have utilized the EfficientNet-B7 deep-learning model for the detection of breast cancer, which has less parameters as compared to many DL models, providing less computation complexity. Also, we used four types of metrics, Acc, pre, rec, and F1 score, with consistent values over 92%, which shows the effectiveness of our model. Also, five-fold cross-validation with high-quality images provides the better generativity of the dataset from a real-life perspective. Our proposed techniques demonstrate the impact of synthetically generated high-quality images using GAN over the conventional augmentation.

Overall, the performance comparison for the second experiment on the combined dataset clearly shows the benefits of using GAN-augmented images of excellent quality. The significant improvements in the accuracy, F1 score, precision, and recall demonstrate the model’s improved ability to accurately categorize breast cancer images. This enhanced performance is notably obvious in the lower number of false positives and false negatives, demonstrating GAN augmentation’s potential to improve diagnostic models for breast cancer detection. Comparing the two approaches, it can be noted that the inclusion of the generated data in the training set improved the generalization ability of the model and increased the classification accuracy. This demonstrates the value of generative adversarial networks in generating additional training data, especially when the original medical images are limited. An important aspect is not only the quantitative increase in the data set but also the quality of the generated images, confirmed by the FID metric, indicating a high degree of realism and closeness to the original images.

Although our research shows that incorporating synthetically created data in the training set has several benefits, it is crucial to be aware of any possible drawbacks. The possibility of overfitting is one issue, especially if the model begins to depend too much on patterns observed in the synthetic data that may not accurately represent actual situations. The possibility for artifacts or minor distortions to be introduced into the synthetic images that do not present in the original data is another drawback. These artifacts may accidentally influence the model’s learning process and impact its generalizability. To reduce these concerns, we applied rigorous quality assurance processes throughout the development of the synthetic images, and we made ensured that the FID metric was utilized to assess the synthetic images’ realism and similarity to the original dataset.

Even with these safety measures, using synthetic data in clinical applications needs to be performed with caution. To completely evaluate the robustness of our technique, further validation across several imaging modalities and testing on real-time clinical datasets are required. Furthermore, the identification of breast cancer using the proposed methodologies has not been verified in the real-word dataset, indicating another area for further research. Future research will solve these drawbacks and ensure that the proposed approaches are secure and efficient for clinical use.

6. Conclusions

In this work, we have performed the detection of breast cancer and demonstrated the impact of synthetically generated images using a GAN over the conventional augmentation technique. We have utilized the ultrasound imaging modality from four publicly available datasets, BrEaST, BUSI, Thammasat, and HMSS, from four different sources. A total of 3186 ultrasound images have been utilized with benign (1574) and malignant (1612) pathology types. To prove the impact of synthetically generated images, we have used the StyleGAN3 model to generate high-quality (512 × 512) 10,000 images. We have performed two separate experiments using a real dataset with conventional augmentation and a combined dataset (Real+GAN) utilizing the EfficientNet-B7 deep-learning model. The model is trained using five-fold cross-validation techniques and validated using the accuracy, precision, recall and F1 score metrics. The proposed methodology demonstrated that utilizing the EfficientNet-B7 model on the combined dataset provides over 92% metric values. It also demonstrated that training the model using a large dataset (Real+GAN) increases the model performance by over 3%.

In future work, we may utilize multimodal datasets to check the data generalizability of our proposed techniques across different data types. Also, we may test the proposed techniques on a real-time dataset for clinical application.