1. Introduction
Screening mammography is associated with not only decreased breast cancer mortality rates but also high breast cancer detection rates at an early stage. In particular, diagnosing breast cancer using this method may decrease the need for more radical surgical treatment and chemotherapy [
1]. Mammography is primarily used for specific and general screening and breast cancer diagnosis in the early stages. Digital breast tomosynthesis (DBT) is a promising tool in both areas.
The disadvantages of mammography are associated with the composite overlapping of fibroglandular tissues. In cancer diagnosis, dense fibroglandular tissues may cause a blurred visualization of a malignancy. Thus, the diagnostic value of the method decreases. In theory, DBT can compensate for the indicated limitations of mammography. In contrast to conventional mammography, which conveys radiography using tissue volume and builds an image from the absorbed X-rays from all tissues, DBT concentrates on a thin slice of tissue that is of interest while obscuring all other tissues.
Recent studies have shown increasing interest in deep learning models (DLMs). Their application in medical image reproduction is directly associated with that of basic models, such as generative adversarial networks (GANs). GANs bypass the uncompliant probabilistic assumptions using a standardized network for data distribution. This sampling network is used in an adversarial fashion, in which the data samples that are generated are as real as possible and a discriminator distinguishes real samples from the ones that were generated.
DBT studies can clearly identify masses and have an enhanced image quality in processes that use DLMs [
2,
3,
4,
5,
6,
7]. Regarding image processing with conditional GAN (cGAN or supervised image-to-image translation [pix2pix]) [
8], pix2pix, which aligns the object image with the reference with an adversarial network using a generator and discriminator, was beneficial for contrast structure (CS) improvement and noise reduction.
Deep learning was applied directly to image pre-reconstruction. First, a reconstruction algorithm (exact filtered backprojection: FBP or iterative) was developed to transform raw data (projection data) into a medical image. Considering the limitations of the reconstruction algorithm, DLM that used a low-quality medical image as the input and released the improved image as the output was applied. For example, the image had a higher resolution or a lower level of noise or fewer artifacts. This was referred to as the pre-reconstruction image-domain approach, which worked with the reconstructed image, not including the reconstruction pipeline. Thus, existing image-to-image deep neural networks (pix2pix), such as U-Net [
9], could be applied to address this issue.
The conventional approach for image processing and building is associated with challenges that limit the identification of masses and the accurate preservation of their normal structure [
10,
11,
12,
13,
14,
15,
16,
17]. As the level of noise in low-dose imaging increases, the detection of masses and the preservation of normal structures (e.g., structural distortion, image sharpness, and artifacts) occur simultaneously.
The image quality and radiation dosage of several DBT algorithms have been mathematically examined [
18,
19]. Previous studies have shown that iterative reconstruction may substantially lower the level of noise and radiation exposure. However, their analyses were limited, and they focused on a comparative assessment of existing methods. Liu et al. used a controlled image processing technique to train a neural network using a linear-output-layer backpropagation algorithm. By applying and evaluating low-dose images in a limited number of clinical cases, the proposed method was found to be a promising technology for low-dose reduction [
20]. Recently, Gao et al. revealed that diminishing noise in a deep convolutional neural network with adversarial training is helpful in improving microcalcification (MC) CS in DBT using in silico data as a learning set [
21]. Gomi et al. compared DBT images generated by applying pix2pix with the low-dose pre-reconstruction domain and conventional image processing methods. Results showed that pix2pix was useful in upgrading the quality of low-dose images. Nonetheless, the reconstruction algorithm was only FBP, and the evaluation cross-section was an in-focus plane with limited information [
22]. Among the studies that have used deep learning, only a few performed a quantitative assessment of image quality refinements, reduction of the MC dose, and identification of masses in diverse circumstances using automatic exposure control (AEC) as the comparative dose because of variations in breast thickness. In particular, because it can decrease the radiation dose and upgrade image quality, this logic (i.e., an image-to-image translation process, which allows the application of low-dose images to reference-dose images) can be used in DBT to upgrade image quality degradation under low doses.
A previous study investigated supervised learning approaches for DBT image reconstruction [
22]. However, it is impractical to use supervised models because of the absence of paired training data. For example, in noise reduction, artifacts and noise are derived from low-dose conditions. Therefore, capturing a pair of images is not appropriate, as one is low-dosed and the other is not under a low-dose condition, which is not supposed to yield practical results. In DBT, images are inherently noisy because of the diminished number of photons emitted by the flat-panel detector. Therefore, there are no available clean ground-truth images. Unsupervised learning approaches are used to address these scenarios. Considering low-dose image data, the models that could learn DBT image reconstruction without high-quality ground-truth data were evaluated. GANs could produce data that are statistically more comparable with the training samples. Considering image reconstruction, GANs can generate high-quality images without supervision. This is an interesting property when combined with a GAN-based method using residual neural network (ResNet) [
23] techniques, such as cycle-consistent GAN (cycleGAN) [
24], because it provides opportunities for teaching DBT image reconstruction from unpaired training samples.
The use of ResUNet, which is an improved version of U-Net, is useful for improving image identification accuracy and reducing artifacts [
25]. ResUNet is based on U-Net, with a residual mechanism added to the encoder and decoder. Therefore, compared to U-Net, it is possible to improve the identification accuracy of feature maps in each layer.
Based on recent studies, attention mechanisms have been adopted in deep learning to improve image recognition and classification accuracy [
26,
27,
28]. Attention calculates the dot product of the vector corresponding to the original feature map. Then, with respect to the generated coefficients and the feature map (part) of the next output layer, coefficients are made larger for areas with a strong association and smaller for areas with a weak association. Using this attention mechanism, a feature map that emphasizes the target region of interest can be generated. In a conventional convolutional neural network (CNN), learning ranges from simple patterns between pixels to complex patterns for input images. However, the attention mechanism realizes the learning of complex relationships from simple relationships among multiple feature maps. Therefore, by incorporating this attention mechanism into a part of the generator network, the identification accuracy of the feature map of each layer can improve and contribute to obtaining a better final target image. In addition, another study has reported that incorporating an additive attention gate (AG), which is a developed attention mechanism between U-Net system skip connections, is useful [
26].
In this study, by inserting residual, squeeze, and excitation (SE) [
29] into the bridge, the structural extraction of normal tissues and diseased areas improved. Incorporating an AG between the skip connections further improved the accuracy of separating normal and mass structures.
CycleGAN was selected as the base system in constructing a network for improvement. This is because the training data were unpaired samples, and a network comprising two pairs of generators and discriminators was required to improve the accuracy of the extraction of minute masses required for breast diagnosis. In addition, an encoder and a decoder structure, which allowed the flexible insertion of an AG into the process related to skip connection and SE in the bridge was preferred for the configuration of the generator. ResUNet, an optimal network based on U-Net, has a residual network configuration that improves the network’s ability to discrimination between features in each convolution layer, was consistent with this condition.
Based on this logic, to improve image quality and reduce distortion attributed to artifacts, we used ResUNet in a cycleGAN-based generator network to perform pre-reconstruction processing for projection data, inserted residual and SE in the bridge, and skipped the connection from encoder to decoder. Thus, a novel residual squeeze and excitation attention gate (rSEAG) that incorporates AG in between was proposed. The image reconstruction algorithm used FBP to calculate the exact solution of projection data.
To validate the usefulness of the proposed rSEAG, the conventional networks U-Net and ResUNet were compared. Moreover, the use of perception-based non-reference-type comprehensive image quality evaluation (PIQE) was examined [
30]. Subsequently, basic image features such as “Homogeneity” and “Contrast” were calculated via texture analysis [
31]. Finally, the evaluation was performed by the streak artifacts of the high-frequency components using a statistical model with a Gumbel distribution [
32].
4. Discussion
CycleGAN pre-reconstruction processing with rSEAG can improve the quality of DBT images under unsupervised clinical case conditions. In this study, the in-focus plane of cycleGAN pre-reconstruction processing with rSEAG improved image quality and reduced architectural distortion in the whole mass image. Further, cycleGAN pre-reconstruction processing with rSEAG could have contributed to ripple artifact reduction. Thus, this method could be a promising novel strategy for improving the separation accuracy between normal tissues and mass areas, unlike images subjected to conventional network architectures. Moreover, the flexibility of cycleGAN pre-reconstruction processing with rSEAG is associated with a better usability.
Based on the comparative study results, rSEAG could provide images featuring an integrated image quality balance between the in-focus plane in AEC (
Figure 11 and
Figure 12). The ability to provide accurate image information from both in-focus plane aspects under reference-dose imaging conditions can enhance the diagnostic value of the breast area in the future. Therefore, rSEAG allows us to proceed in a direction in which, unlike conventional filtering-based image processing, unknown degraded images can be restored to a state with standard image quality by utilizing existing learning data. We believe that the method presented in our study is among the measures that could improve image quality in DBT with reference to integrated image quality, though we acknowledge that our results are experimental and based on a limited number of cases.
There are several concerns regarding the use of DBT. For example, it is associated with increased radiation exposure and the potential risk of radiation-related malignancy. DBT vendors have used various approaches to address this issue. These approaches reduce the additional dose by making the examination at a single view. Considering that the DBT dose is twice that of the usual radiation exposure for mammography, performing a single DBT can decrease the cumulative dose. In terms of accuracy, a few studies have shown that single-view DBT is least compatible with two-view mammography [
42]. The use of different angular dose profiles produces high-quality projections. Nishikawa et al. revealed that a high-dose central projection can improve the speed and accuracy of MC detection while maintaining high mass detection rates in DBT reconstruction [
43]. However, other studies have shown a similar [
44] or inferior performance [
45] for MC detection (or specks simulating MCs) using this projection. Therefore, further investigations should be performed to determine whether these projections have clinical benefits.
CNN does not require input from hidden layers obtained from a specific processing layer. Thus, calculations can be parallelized. However, the association between tokens (vectors that compress the information of the whole image) located at distant locations is challenging to identify. Attention mechanisms have been proposed to address this issue. The similarity measure of the attention mechanism is determined by the inner product of vectors and the weighted sum using the inner product as a coefficient. By creating a processing system that focuses on areas of interest, such as mass areas, using a network that combines CNN and attention, such as the proposed rSEAG, the accuracy of identifying normal from mass tissues can improve. Hence, a system that reduces distortion caused by artifacts in the mass area can be useful.
Moreover, inserting an AG between the skip connections of ResUNet, which is the basis of this generator network, could improve image quality (especially by removing distortion and reducing artifacts caused by ripples). By concatenating encoder and decoder information, the features required for updating increase, and they are effective in improving identification accuracy. The effect of improving identification accuracy can be synergized by incorporating an AG with the utilization of the rich feature values that exist between skip connections in each layer. Instead of outputting each channel of the convolutional layer equally, addition of an SE to the generator bridge could cause the input conditioned by dynamics due to adaptive weighting, which improves the discrimination ability of the features. While the AG calculates the similarity between multiple features with improved discrimination ability via SE, the attention weight, which is a coefficient that indicates the association of the similarity with improved accuracy by the dot product and weighted sum, could also be calculated. This attention weight allows learning to focus on the mass region that deserves attention in the image. Therefore, rSEAG, which combines SE and AG, has reduced the structural distortion seen in normal and mass images and reduced ripple artifacts.
CycleGAN defines cycle consistency and adversarial losses. The cycle consistency loss is based on the difference between the original image and the original image transformed to another domain and then retransformed to the original domain. Meanwhile, the adversarial loss ensures that the image is real. Training an image-to-image transform framework is often challenging, because it requires perfectly matched images. These issues can be solved by CycleGAN by allowing transformations between domains that are not perfectly matched. Therefore, unlike conventional processing for correcting differences between images, an image that approximates a reference image with high precision and efficiency can be generated.
In FBP, the measurement process is characterized by the Radon transform [
46]. Moreover, the system’s analytical equation is solved before discretization of the solution. However, only approximate solutions are presented for incomplete data, and efforts have been made to identify appropriate approximations. In cycleGAN pre-reconstruction processing with FBP, effective CS preservation enabled the derivation of an appropriate approximate solution at the projection data level, thereby addressing the analytical solution for the inverse problem for DBT.
In medical images, the types, distribution, and density of breast glandular tissues and the corresponding overlaps of glandular density were inconsistent, which caused issues in the model optimization. Overcoming the effects of breast gland thickness and breast overlap owing to the use of more varied cases is a future challenge. In the next evaluation stage, to overcome these issues, learning and evaluation using clinical cases acquired at the projection data level can be used. For example, the use of samples with high and low noise levels at the projection data level was proposed. In this study, a training dataset that did not significantly differ between populations was used. A good performance was achieved on the test data because learning was performed in a state where there was no gap in the main distribution between the two domains. Further studies using more clinical data should be performed to better understand the association between learning and performance. Nevertheless, no studies have focused on improving the reproducibility or performance of unsupervised GANs. Therefore, studies that focus on evaluating the reproducibility and performance of unsupervised GANs are interesting.
This study had some limitations. One evaluation had an increased variation in the dataset used for testing. Therefore, it is important to evaluate cases of MCs and benign and malignant masses in patients with various breast types.