EEUR-Net: End-to-End Optimization of Under-Sampling and Reconstruction Network for 3D Magnetic Resonance Imaging

Abstract

It is time-consuming to acquire complete data by fully phase encoding in two orthogonal directions along with one frequency encoding direction. Under-sampling in the 3D k-space is promising in accelerating such 3D MRI process. Although 3D under-sampling can be conducted according to predefined probability density, the density-based method is not optimal. Because of the large amount of 3D data and computational cost, it is challenging to perform data-driven and learning-based 3D under-sampling and subsequent 3D reconstruction. To tackle this challenge, this paper proposes a deep neural network called EEUR-Net, realized by optimizing specific under-sampling patterns for the fully sampled 3D k-space data. Innovatively, our under-sampling algorithm employs an end-to-end deep learning approach to optimize phase encoding patterns and uses a 3D U-Net for image reconstruction of under-sampled data. Through end-to-end training, we obtain an optimized 3D under-sampling pattern, which significantly enhances the quality of the reconstructed image under the same acceleration factor. A series of experiments on a knee MRI dataset demonstrate that, in comparison to standard random uniform, radial, Poisson and equispaced Cartesian under-sampling schemes, our end-to-end learned under-sampling pattern considerably improves the reconstruction quality of under-sampled MRI images.

1. Introduction

Magnetic Resonance Imaging (MRI) is a prevalent technique in modern medical diagnostics. It offers clear and stable imaging, non-invasiveness, the absence of ionizing radiation, high tissue contrast, and high resolution [1]. However, traditional MRI is not without its limitations, including prolonged scanning times, leading to decreased patient throughput and a suboptimal patient experience [2]. Such constraints hinder the widespread use and further development of MRI equipment. Particularly, during 3D MRI scans, capturing a full 3D k-space data under inherent physical constraints is time-consuming [3]. Under-sampling the k-space data and reconstructing images from the subsampled data are crucial for fast MRI. Thus, accelerating the 3D MRI process is both urgent and challenging.

Existing fast MRI techniques can be broadly categorized into parallel imaging (PI) [4] and compressed sensing (CS) [5]. The former captures multiple anatomical views simultaneously, while the latter collects fewer samples than traditional methods. Parallel imaging utilizes phased-array coils to reduce measurements required for image reconstruction, thereby shortening the scan duration. In contrast, compressed sensing leverages the sparsity or compressibility of MRI data to achieve high-quality reconstructions with fewer samples, enabling faster imaging at reduced costs. Yet, these approaches have their limitations: PI is constrained by its reliance on spatially variant coil sensitivities, which limits the exploitation of data correlations [4], while CS depends on the sparsity of data, becoming less effective at higher acceleration rates [5]. Consequently, both techniques face challenges in efficiently handling high acceleration factors due to these inherent limitations.

MRI measurements represent spatial frequency transformation coefficients, also known as k-space. Images are computed by applying the inverse Fourier transform that maps k-space data to the spatial domain. Medical images typically exhibit significant spatial regularities. For instance, intensity values change smoothly in space, barring a few boundary voxels. This regularity induces redundancy in k-space, providing opportunities for sub-Nyquist sampling [6]. Several Cartesian and non-Cartesian under-sampling patterns have been proposed and were widely applied in practice, such as standard random uniform, radial, Poisson, or equispaced Cartesian under-sampling schemes. These k-space under-sampling strategies speed up the MRI process.

Image reconstruction algorithms play an indispensable role in fast MRI. These algorithms aim to reconstruct images from under-sampled k-space data. Many solutions have been proposed to address the inverse problem of recovering full k-space data from under-sampled measurements. These can be broadly classified into traditional optimization algorithms and deep learning-based methods. Traditional algorithms deploy mathematical techniques, like compressed sensing and iterative reconstruction, to harness the sparsity or structure of MRI data and recover lost information. Classic techniques like SENSE [7], SMASH [8], and GRAPPA [9] exploited correlations between k-space priors and imaging system sampling properties. Other methods include nonlinear optimization based on low-rank [10] and total variation [11]. While these traditional algorithms are mature and commercially implemented, they have limitations due to insufficient, inflexible priors and somewhat longer reconstruction times due to iterative computation [12].

In recent years, deep learning-based methods have emerged as promising alternatives for MRI reconstruction. Leveraging the potent representational capacity of deep neural networks, these methods directly learn the mapping between under-sampled k-space data and the corresponding image. Trained on vast datasets, deep learning models can capture complex image priors and produce high-fidelity reconstructions. End-to-end deep learning techniques excel at noise and artifact removal, with many methodologies based on GAN networks [13], RNN networks [14], and U-Net [15]. These models are trained on data to learn the mapping of sampled k-space measurements to image domain reconstructions. The aforementioned machine learning-based methods are typically optimized for given under-sampling patterns. There are also techniques that optimize under-sampling patterns for specified reconstruction methods. Reconstruction model performance largely depends on the under-sampling pattern, making a good pattern pivotal in MRI tasks.

In this paper, for 3D MRI tasks, we address the challenge of time-consuming data acquisition by focusing on optimizing under-sampling patterns in a data-driven manner. Recognizing the limitations of traditional density-based under-sampling methods due to extensive data volume and computational costs, our approach innovatively combines the optimization of under-sampling patterns with advanced 3D reconstruction. By employing an end-to-end deep learning strategy, our method not only refines the under-sampling pattern for enhanced image quality but also ensures efficiency in the MRI process.

Our contributions include:

• Inspired by the unique characteristics of 3D k-space, we design a novel 3D k-space under-sampling pattern. This pattern selectively under-samples in the two phase encoding directions while fully sampling in the frequency encoding direction, enabling the generation of an optimal under-sampling pattern specifically tailored for the training dataset.
• We propose an end-to-end 3D under-sampling and reconstruction network (EEUR-Net), where the integrated training process generates a learned under-sampling pattern and enhances reconstruction, significantly improving image quality.
• Experiments reveal that our network performs well, with the learned under-sampling pattern surpassing many established methods. Furthermore, the end-to-end three-dimensional under-sampling and reconstruction approach achieves more robust and accurate results in 3D MRI, demonstrating impressive performance on the Stanford University 3D FSE knee dataset.

2. Related Works

The related work will be presented from the following perspectives: First, we will discuss studies on under-sampling schemes. Subsequently, we will delve into research on MRI reconstruction based on deep learning techniques.

2.1. Studies on Undersampling Schemes

In MRI, by using an under-sampling strategy in the k-space domain, high-quality images can be reconstructed from a small number of samples. There are many insightful methods currently available. Fan et al. [16] explored how to recover high-quality images by sparse representation and optimization algorithms while reducing the number of samples. Compressed sensing MRI methods based on random under-sampling [5] examined the effects of different under-sampling patterns and reconstruction algorithms on the quality of reconstruction, and detailed quantitative analysis were carried out. Khare K et al. [17] discussed how to design effective sampling patterns and reconstruction algorithms to maintain image quality while reducing sampling time. Low-rank and sparse matrix decomposition [2] can also be used for sampling pattern optimization in MRI, using this decomposition method to recover high-quality images while reducing the quantity of sampling data. Convolutional Neural Networks (CNNs) have been successfully applied to optimize under-sampling patterns using two-dimensional MRI data [18], but this method was designed for two-dimensional MRI data. Therefore, in 3D MRI tasks, we should consider optimizing three-dimensional under-sampling patterns [19] to accelerate the process of MRI.

2.2. MR Image Reconstruction Using Deep Learning

In the existing research, many reconstruction methodologies make good use of deep learning. Ding et al. [18] presented a deep learning-based approach that enhances image reconstruction in accelerated MRI acquisition by refining the U-Net architecture to achieve better image resolution. Wang et al. [20] capitalized on deep learning techniques to expedite the process of 3D MRI. Their approach employs deep neural networks to learn and infer missing data, enabling the reconstruction of high-quality 3D images from partially sampled data. Zhang et al. [21] proposed to apply of two-dimensional super-resolution techniques for the super-resolution reconstruction of three-dimensional MRI images, boasting superiority in texture and frequency information over other super-resolution methods. Han et al. [22] introduced a deep learning method based on both ALOHA (annihilating filter-based low-rank Hankel matrix approach) and U-net, realizing a mapping from k-space to the image domain to expedite the reconstruction task in MRI image acquisition. Thus, for 3D MRI tasks, we contemplate employing a deep learning approach using a network model based on 3D U-Net to accomplish the reconstruction of three-dimensional MR images.

3. Methods

In the Methods section, we first present an under-sampling strategy specifically for the distinct characteristics of three-dimensional k-space data. Subsequently, we delve into our proposed EEUR-Net, which is integration of the learned optimization for the under-sampling scheme and the reconstruction process utilizing under-sampled data into a singular end-to-end deep learning framework. For the reconstruction of the under-sampled data, we use the 3D U-Net architecture.

3.1. Three-Dimensional k-Space Characteristics and Three-Dimensional Undersampling Scheme

In existing MRI scanning techniques, due to inherent physical constraints, obtaining fully sampled data sequentially in k-space requires a significant amount of time [3]. For 2D k-space scanning, the data to be collected in k-space can be represented as a data matrix: $\mathrm{K}\in \mathrm{C}^(\mathrm{N}\times \mathrm{F}\times \mathrm{P})$ [23]. For 3D k-space scanning, the data to be collected in k-space can be represented as a data matrix $\mathrm{K}\in \mathrm{C}^(\mathrm{N}\times \mathrm{F}\times \mathrm{P}1\times \mathrm{P}2)$ [24]. Here, N represents the number of receiving coil arrays, $\mathrm{F}$ is the number of frequency encoding steps, $\mathrm{P}$ denotes the quantity of phase encoding, and $\mathrm{P}1$, $\mathrm{P}2,\mathrm{r}\mathrm{e}\mathrm{s}\mathrm{p}\mathrm{e}\mathrm{c}\mathrm{t}\mathrm{i}\mathrm{v}\mathrm{e}\mathrm{l}\mathrm{y}$ represent the first and second phase encoding quantities in 3D scanning.

The mechanisms underlying 2D scanning and 3D scanning diverge markedly, as shown in Figure 1a. In 2D scanning, imaging is conducted slice-by-slice. Initially, a radiofrequency (RF) pulse is employed to selectively excite a specific slice, a process termed as “Slice Selection”. Subsequently, spatial encoding for each individual slice is achieved using frequency encoding gradients and phase encoding gradients, culminating in the final imaging objective. Conversely, 3D scanning omits the slice selection step, as shown in Figure 1b. Instead, a volumetric region is directly excited using an RF pulse. The spatial encoding for this volume is accomplished using three orthogonal encoding gradients: one frequency encoding gradient and two phase encoding gradients.

Distinctly different from 2D k-space data, 3D k-space data comprise three spatial frequency axes. This 3D space encompasses a comprehensive spatial frequency domain enriched with intricate and detailed structural information, which is advantageous for imaging outcomes. However, this advantage comes at the expense of an exponentially increased scanning duration due to the expanded volume of data. Through an in-depth analysis of k-space characteristics, we observed that, in terms of energy distribution, the center of 3D k-space data exhibits a spherical concentration, as opposed to the circular concentration evident in 2D k-space. This most pivotal information is centralized in the heart of the k-space, often termed the “DC component.” This central region is referred to as the “central k-space”, while the surrounding area is designated as the “peripheral k-space”. The central k-space encapsulates data regarding contrast and the overall image impression, whereas the peripheral k-space captures details and edges. Given these unique features of the 3D k-space, an effective strategy for reconstructing high-quality images involves dense sampling within the energy-concentrated central k-space and employing techniques such as compressed sensing or machine learning-based methods to under-sample the insufficiently sampled k-space, as illustrated in Figure 1c.

Our proposed under-sampling strategy capitalizes on the energy concentration in 3D k-space. For a given sparsity constraint, we opt for full sampling over a fixed-size central region and under-sample in other areas, maintaining the sparsity constraints. This approach facilitates under-sampling operations at a designated acceleration rate. To implement this strategy in our experiments, we chose to fully sample along the frequency-encoding direction and perform under-sample encoding on the first and second phase encoding directions. Viewed from a three-dimensional volumetric perspective, we conducted full sampling within the central cuboid region of the 3D k-space, as depicted below, while other areas are sampled based on the optimized under-sampling patterns, as shown in Figure 1d.

3.2. EEUR-Net

In this section, we delve deeply into the architecture and compositional structure of the framework utilized for learning the sampling patterns as well as the image reconstruction network. We further elucidate the associated mathematical principles and implementation details.

3.2.1. Overall Framework of EEUR-Net

The diagram of our proposed overall network framework is shown in Figure 2a, which can be viewed as a whole consisting of under-sampling pattern optimization network and an image reconstruction network, and using an end-to-end training deep learning framework. Our network flow can be explained as follows: first, we first obtained 3D K-space data from the 3D FSE Knees dataset. Then, the learned under-sampling pattern could be obtained by the under-sampling pattern optimization network, which in turn obtained the under-sampled k-space data. Following this, an inverse Fourier transform resulted in zero-filled images, which then underwent image reconstruction via the reconstruction network. In the final step, the loss function used structural similarity index (SSIM) was computed, and the relevant network parameters were updated. Through end-to-end training, we achieved an optimized learned under-sampling pattern and superior reconstruction outcomes.

The under-sampling pattern optimization network framework is depicted in Figure 2b. The flow of this network can be explained as follows: Firstly, we initialized the sampling pattern using relevant weight parameters. The 3D k-space encoding direction is composed of two phase encoding directions and one frequency encoding direction. Typically, under-sampling is applied to the two phase encoding directions, while the frequency encoding direction remains fully sampled. We input a fixed proportion of the central fully sampled region and, based on the given acceleration factor, initialized the phase encoding directions to a state of random uniform sampling. Additionally, independent random variables were present at each k-space location on the full-resolution grid. From the probability values of frequency encoding for each grid point, a ‘Probabilistic Mask’ was formed. A higher probability value indicated a greater likelihood that the particular point was selected for sampling. Subsequent binarization leads to the formation of a ‘Binary Mask’. A value of 1 (or 0) in the binary mask denotes that the corresponding k-space location is (or is not) sampled. Through end-to-end training, by calculating the loss function and updating the network’s relevant parameters, the probabilistic mask can be altered. This, in turn, modifies the binary mask, resulting in an optimized learned under-sampling pattern.

3.2.2. Related Mathematical Principle

In the mathematical model of sampling and reconstruction, a given under-sampling pattern and corresponding 3D k-space data can be expressed as obtaining a reconstructed image $\widehat{\mathrm{p}}$ by minimizing the following objective function:

$$\widehat{\mathrm{p}}=\arg \underset{\mathrm{p}}{\min }\sum _{\mathrm{j}}{‖\mathrm{AF}{\mathrm{S}}_{\mathrm{j}}\mathrm{p}-{\mathrm{k}}_{\mathrm{j}}‖}_2^2+\mathrm{R}\left(\mathrm{p}\right)$$

(1)

where $\mathrm{p}$ is the variable to optimize, and the MR image to be reconstructed is $\mathrm{p}*$, ${\mathrm{S}}_{\mathrm{j}}$ is the coil sensitivity map of the $\mathrm{j}$-th coil, $\mathrm{i}$ is the number of receiving coils, $\mathrm{F}$ is the three-dimensional Fourier transform, $A$ is the k-space under-sampling pattern, and ${\mathrm{k}}_{\mathrm{j}}$ is the k-space under-sampled data of the $\mathrm{j}$-th coil. $\mathrm{R}\left(\mathrm{p}\right)$ is the regularization term.

Equation (1) can also be implemented by a neural network, where we can learn a parameterized mapping to model the input ${\{\mathrm{k}}_{\mathrm{j}}\}$ to the output $\left\{\widehat{\mathrm{p}}\right\}$. We represent the above mapping as $\widehat{\mathrm{p}}=\mathrm{N}\mathrm{e}\mathrm{t}\left({\mathrm{k}}_{\mathrm{j}}\right)$ using a deep neural network.

$A$ describes an independent (binary) random variable $B$ at each k-space location on a full-resolution grid in the discrete Fourier domain. Thus, the probability mask $A$ forms an image of probability values within the k-space. A binary mask value of 1 (or 0) signifies that the corresponding k-space position is (or is not) sampled.

Our aim was to obtain the optimal under-sampling pattern $A$ with fixed under-sampling ratio α from K fully sampled data by retrospective under-sampling. The mathematical formulation of this problem is as follows:

$$\underset{\mathrm{A}}{\min }\frac{1}{\mathrm{N}}{\displaystyle {\sum }_{\mathrm{t}=1}^{\mathrm{N}}}\mathrm{Loss}\left({\mathrm{p}}_{\mathrm{t}}^{*},\widehat{{\mathrm{p}}_{\mathrm{t}}}\left(\mathrm{A}\right)\right),\widehat{{\mathrm{p}}_{\mathrm{t}}}\left(\mathrm{A}\right)=\mathrm{Net}\left(\right\{\mathrm{A}{\mathrm{p}}_{\mathrm{tj}}^{*}\left\}\right)$$

(2)

where ${\mathrm{p}}_{\mathrm{t}}^{*}$ is the $\mathrm{t}$-th MR image reconstructed from the fully sampled k-space data ${\{\mathrm{k}}_{\mathrm{ij}}\}$ by direct Fourier inverse transform, and the $\mathrm{Loss}\left(\right)$ function is defined as a loss function measuring the similarity between the reconstructed images, with the fully sampled data, and generates $A$ at a fixed acceleration factor α. $\mathrm{Net}\left(\right)$ is the anti-aliasing network. By implementing the above optimization problem through end-to-end training, we can obtain the optimal sampling mask. We implemented this using a deep neural network, which solved the learning problem by stochastic gradient descent.

3.2.3. Network Architecture of EEUR-Net

The network diagram of the EEUR-Net is illustrated in Figure 3, which provides a comprehensive depiction of the under-sampling pattern optimization network and the image reconstruction network encompassed within the EEUR-Net.

Figure 3a shows the implementation details of the under-sampling pattern optimization network. The network is capable of generating probability values at each k-space grid point and optimizing them, and then generating the optimized k-space under-sampling pattern. Specifically, the input layer generates the probability value at each grid point in both phase-encoding directions as a probabilistics mask through an initialized weight parameter, and this probabilistics mask is then passed through a fully connected layer with tunable parameters to optimize the probability values. This layer determines the importance of a single k-space data point, with a higher probability value indicating a higher probability that the point will be selected for under-sampling. Next is the binarization layer, which converts the continuous data from the fully connected layer into binarized data. The output layer then generates a 3D k-space under-sampling mask that is under-sampled in both phase encoding directions and remains fully sampled in the frequency encoding direction, resulting in the “Binary Mask” shown in the figure. The value of the binary mask is 1(0), which indicates yes (no) sampling at the corresponding encoding position. Multiplying the obtained 3D k-space under-sampled mask with the k-space data gives the under-sampled k-space data, which can be obtained as the zero-filled image as the input of the reconstruction network after the 3D inverse Fourier transform.

The reconstruction network architecture based on 3D U-Net is shown in Figure 3b, where each vertical blue line represents an image within the image domain, alongside the results of each step of the processing. The grey arrows denote the “Conv_Block3D” modules, further detailed in Figure 3c. Each module consists of a series of operations starting with a three-dimensional convolution, which expands the two-dimensional kernel of traditional convolutions into the third dimension, enabling the network to incorporate information from the depth of the input volume. Following the 3D convolution, a ReLU activation function introduces non-linearity, which is essential for the model to capture complex patterns within the data. Each layer of the encoding pathway in the reconstruction network includes a “Conv_Block3D” module, followed by a downsampling through “MaxPooling3D” operations. In the decoding pathway, upsampling techniques are employed to incrementally restore the spatial dimensions, achieved by “Upsampling3D” operations. The concatenation steps, symbolized by the yellow arrows, merge the upsampled features with the corresponding feature maps from the encoding pathway, allowing the network to preserve high-resolution features throughout the network. The 3D U-Net extends the conventional U-Net into the three-dimensional space, enhancing its capacity for analyzing volumetric images. Our model is designed to learn the optimal undersampling scheme within the three-dimensional k-space. The under-sampling mask is treated as a trainable parameter, which is updated during the training process to optimize the quality of the reconstructed image.

4. Experiments and Results

In this section, we provide a comprehensive overview of the datasets employed in the relevant experiments and delve into the specific implementation details. Furthermore, we present the experimental results, elucidating their significance in relation to established evaluation metrics.

4.1. Dataset

The dataset used in this study is Stanford Fully sampled 3D FSE Knees dataset [25]. The images in the dataset were collected from the knee joints of multiple patients and imaged using the Fast Spin Echo (FSE) sequence. These images were obtained by acquiring a large amount of k-space data in three directions: axial, coronal, and sagittal. Each image features high resolution and rich tissue contrast, including joint cartilage, ligaments, tendons, and surrounding tissues.

The dataset contains 19 volumes, collected using a 3T GE medical system scanner with an 8-channel phased-array coil. A notable characteristic of the Stanford Fully sampled 3D FSE Knees dataset is its complete sampling; that is, every k-space sampling point is measured, without involving any under-sampling techniques.

While there are many 2D MRI databases, 3D databases are scarce. This makes the dataset an ideal choice for evaluating and comparing different reconstruction algorithms, providing a benchmark to measure the reconstruction quality and accuracy of other methods. Utilizing this dataset, our study aims to investigate and develop end-to-end sampling and reconstruction methods for three-dimensional knee joint MRI. The availability of high-quality, fully sampled knee images allows us to explore these advanced algorithms that can effectively utilize the rich information contained within the complete k-space data. By evaluating these methods on the Stanford Fully sampled 3D FSE Knees dataset, we can assess their performance in terms of reconstruction effectiveness, evaluation metrics, and accurately capturing the complex anatomical structures within the knee joint.

4.2. Implementation Details

Our research was implemented in the PyTorch framework and trained on an NVIDIA Titan Xp GPU. We divided the Stanford Fully sampled 3D FSE Knees dataset, consisting of 19 volumes, into training, validation, and test sets with 14, 3, and 2 volumes, respectively. The network optimization was performed using the RMSProp optimizer. We trained the network for 50 epochs with a batch size of 1 and an initial learning rate of 0.001. At the 40th epoch, the learning rate was reduced to 0.0001. During training, we utilized the SSIM loss function. We used the validation set to monitor the model’s convergence by comparing training and validation loss. This helped us to identify and prevent overfitting

For evaluating the reconstruction results, we employed several quality metrics: structural similarity index (SSIM), peak signal to noise ratio (PSNR), and the normalized mean-squared error (NMSE). These metrics were used to assess the quality of the reconstructed images and provide quantitative measurements of their fidelity.

SSIM indicates the degree of similarity that exists between the under-sampled reconstructed images and fully sampled ground truth images. We used NMSE to represent the normalized mean square error existing between the under-sampled reconstructed images and the fully sampled ground truth images. We can use PSNR to describe the ratio between the maximum possible energy of the image’s intensity and the power of the noise.

4.3. Comparison with Other Methods

4.3.1. Visualization of Under-Sampling Patterns of Various Methods

In this study, to validate the efficacy of an end-to-end learning-based approach that optimizes both the under-sampling scheme and the reconstruction model, we conducted a comparative analysis with our method against several prevalent sampling techniques, all employing the same acceleration factor. These techniques included standard random uniform, radial, Poisson, and equispaced Cartesian under-sampling schemes.

Importantly, to ensure a consistent basis for comparison, all experiments utilized the 3D U-Net for image reconstruction of under-sampled data, maintaining uniformity in the reconstruction methodology across different under-sampling schemes.

Notably, standard random uniform sampling, equispaced Cartesian under-sampling, Poisson sampling, and the learned sampling pattern proposed in this paper all featured a fixed 32 × 26 fully sampled rectangular region at the center of the k-space, contributing to enhanced reconstruction performance.

Each comparison method employed the same training dataset during their training phase, and evaluations for the various techniques were also conducted on a same test set. For the aforementioned sampling techniques, under-sampling experiments were carried out at fourfold and eightfold acceleration rates. In both the first phase encoding and second phase encoding directions, the resultant under-sampling patterns can be visualized in Figure 4.

Figure 4 illustrates the under-sampling patterns obtained from various acceleration rates. Notably, the learned under-sampling pattern exhibits more sampling in the central region. This observation aligns well with our experimental expectations, since the center region contains more information that is beneficial to the reconstruction results.

In addition, Poisson sampling ranked second among the methods we compare. By looking at the results of the sampling patterns, we can find that both Poisson under-sampling and our EEUR-Net have a higher sampling density in the center region of k-space. Usually, the center region in 3D k-space contains more energy. In 3D k-space, k-space is a frequency domain representation in which the low-frequency components contain most of the contrast and structural information of the image, while the high-frequency components represent the edges and details of the image.

4.3.2. Quantitative Evaluation

To quantitatively evaluate the reconstruction performance, we compared the reconstructed images, resulting from different under-sampling methods, at 4× and 8× under-sampling rates, with full-sampled images. The comparison metrics include NMSE, PSNR, and SSIM.

The quantitative comparison results are shown in

Table 1. Evaluation metrics of various methods at acceleration factor (AF) = 4. The methods in the table used the same test set.

Method	AF	NMSE↓	PSNR↑	SSIM↑
Uniform	4	0.01712	36.44	0.9034
Radial	4	0.02377	34.68	0.8854
Equispaced	4	0.02198	35.32	0.8979
Poisson	4	0.01928	36.71	0.9123
EEUR-Net (Ours)	4	0.01013	38.65	0.9324

and

Table 2. Evaluation metrics of various methods at acceleration factor (AF) = 8. The methods in the table used the same test set.

Method	AF	NMSE↓	PSNR↑	SSIM↑
Uniform	8	0.0597	33.67	0.8896
Radial	8	0.07092	32.71	0.867
Equispaced	8	0.05505	33.45	0.8774
Poisson	8	0.4762	34.88	0.8921
EEUR-Net (Ours)	8	0.02484	36.67	0.9109

. Our primary focus was on the SSIM metric, a highly valued measure in the MRI field. SSIM closely approximates human visual perception and is therefore of great significance for clinical diagnosis.

In the experiments with 4× acceleration, our method achieved the best SSIM value of 0.9324. Compared to the baseline (random uniform sampling), our method increased the SSIM by 0.029 and improved the PSNR by 2.21 dB. For the 8× acceleration experiments, our method recorded an optimal SSIM of 0.9109, an improvement of 0.0213 over the baseline SSIM, and a PSNR enhancement of 3 dB.

From the above metrics, it is evident that the approach proposed in this paper delivers superior performance in terms of NMSE, PSNR, and SSIM. This highlights the efficacy of our proposed EEUR-Net, designed to simultaneously optimize the under-sampling scheme and the reconstruction process, in achieving optimal sampling patterns and reconstruction outcomes.

In addition, we carried out cross-validation by randomly dividing the data of 19 volumes three times and conducting experiments separately to average the obtained reconstruction result metrics. The average metrics of the reconstruction results could be obtained, as shown in

Table 3. Evaluation metrics of cross-validation at acceleration factor (AF) = 4, 8.

Method	AF	NMSE	PSNR	SSIM
EEUR-Net (Ours)	4	0.01361	37.86	0.9269
EEUR-Net (Ours)	8	0.02766	36.13	0.9041

. Figure 5 illustrates the under-sampling patterns obtained from various acceleration rates for the three sets of cross-validation. This analysis helps to assess the robustness of our method in generating effective under-sampling patterns and image reconstruction results. In summary, our method demonstrated excellent performance on the 3D FSE knee dataset.

4.3.3. Qualitative Evaluation

The resulting images from different sampling methods at 4× and 8× acceleration are illustrated in Figure 6. All experiments employed the 3D U-Net for the image reconstruction of under-sampled data.

From the image reconstruction results, we can see that the learned under-sampling pattern generated by this method (EEUR-Net) can produce higher-quality reconstruction than other widely used under-sampling patterns, with less aliasing artifacts than other methods, and the reconstruction results better preserved image details, with significant improvement in the visualization of the main structures of the knee joint.

In order to provide a more direct and visual observation of the effect of each network reconstruction, we displayed additional three sets of reconstruction results in Figure 7, all of which were reconstructed using the 3D U-Net. Examining the results of reconstructed images, it is evident that the results produced by our method (EEUR-Net) were superior, with the clearest reconstruction of image details and the least artifacts. This demonstrates that the learned under-sampling pattern can yield higher-quality reconstructions compared to other widely utilized under-sampling masks. The end-to-end optimization of under-sampling significantly contributes to the improvement in image reconstruction quality.

5. Conclusions

In this paper, we introduced the EEUR-Net, a network capable of generating specific 3D under-sampling patterns by optimizing phase encoding from a data-driven perspective, thus enhancing under-sampling efficiency in the 3D k-space. The EEUR-Net can be holistically viewed as a fusion of an under-sampling scheme optimization network and an image reconstruction network using an end-to-end deep learning framework. We derived optimized under-sampling patterns through the under-sampling scheme optimization network, and performed image reconstruction using the 3D U-Net network. This methodology, adopting an end-to-end learning strategy, concurrently optimizes both the under-sampling pattern and the reconstruction model, aiming for robust and precise sampling and reconstruction in 3D MRI.

6. Discussion

For tasks involving 3D magnetic resonance reconstruction, we embraced a data-driven approach, leveraging the unique properties of 3D k-space data, and proposed an integrated end-to-end under-sampling and reconstruction strategy. Joint training of the k-space under-sampling network and the reconstruction network allowed our method to efficiently generate optimized under-sampling patterns. Experiments on the 3D FSE Knees dataset demonstrated that our under-sampling patterns achieve superior reconstruction quality compared to other commonly used under-sampling masks, exhibiting exceptional results at 4× and 8× acceleration rates. Looking forward, we plan to explore deeper into the properties of k-space data to further enhance under-sampling efficiency. Additionally, the exploration of advanced data-driven under-sampling techniques [26], in conjunction with more sophisticated network architectures, presents an exciting avenue for faster and more accurate reconstruction performance in future research. The under-sampling in 3D k-space significantly accelerates the 3D MRI process. Future research can explore the integration with other fast MRI methods to harness their complementary advantages [27], further enhancing scan speeds [28], and advancing the miniaturization of MRI devices [29]. In the realm of MRI imaging, particularly with 3D imaging, addressing and resolving the issues in image quality enhancement is essential. This effort not only serves the goal of optimization but also significantly contributes to the enhancement of the whole healthcare system.