1. Introduction
Computed Tomography (CT) [
1] is diffusely known as an approach to exhibit precise details inside the scanned object [
2], thus is applied to a wide range of applications including clinical diagnosis, industrial inspection, material science and biomedicine [
3,
4]. In addition, the raging epidemic caused by the Corona Virus Disease 2019 (COVID-19) has made CT known to the public as an efficacious auxiliary technology. Nevertheless, the associated x-ray radiation dose brings potential risk of cancers [
5], which has drawn wide attention. Consequently, the demand of radiation dose reduction is becoming more and more acute under the principle of ALARA (as low as reasonably achievable) [
6,
7,
8,
9,
10].
Generally, Low-dose Computed Tomography (LDCT) can be realized through two strategies including current (or voltage) reduction [
11,
12] and projection reduction [
13,
14,
15]. The first strategy aims to lower the x-ray exposure in each view, while it greatly suffers from the increased noise in projections. Although the second strategy can avoid the above problem and realize the additional benefit of accelerated scanning and calculation, it gives rise to severe image quality deterioration of increased artifacts due to its lack of projections. In this paper, we will focus on obtaining high-quality CT images from limited-view CT with inadequate scanning angle.
Researchers have proposed various CT image reconstruction algorithms in the past few decades, but when it comes to LDCT reconstruction, the problem becomes challenging. Traditional analytical reconstruction algorithms, such as FBP [
16], have high requirements for data integrity. When the radiation dose is reduced, artifacts in reconstructed images will increase rapidly [
17]. Compared with analytical reconstruction algorithms, iterative reconstruction algorithms can obtain better performance, while suffering from higher complexity. Model-based iterative reconstruction (MBIR) algorithm [
18], combines the modeling of some key parameters to perform high-quality reconstruction of LDCT. Using image priors in MBIR can effectively improve the image reconstruction quality of LDCT scans [
14,
19], while still have the high computational complexity.
In addition, diverse regularization methods have played a crucial role in CT reconstruction, which is a typical inverse problem. The most prevailing regularization method is the total variation (TV) method [
20]. In the light of TV, researchers came up with more reconstruction methods, such as TV-POCS [
21], TGV [
22] and SART-TV [
13] which was proposed on the basis of SART [
23]. Those algorithms can suppress image artifacts to a certain extent so as to improve imaging quality. In addition, dictionary learning is often used as a regularizer in MBIR algorithms [
24,
25,
26,
27], and multiple dictionaries are beneficial to reducing artifacts caused by limited-view CT reconstruction.
With the development of computing power, deep learning-based methods [
28,
29,
30,
31,
32,
33,
34] have been applied to the restoration of LDCT reconstructed images in recent years. The methods can be roughly divided into the below three categories.
Image inpainting algorithms usually reconstruct the damaged Radon data into the damaged image with artifacts through regular methods, such as FBP, then reduce the artifacts and noises in the image domain. Lots of researchers are currently using convolutional neural network (CNN) and deep learning architecture to perform this procedure [
4,
35,
36,
37,
38,
39,
40,
41,
42,
43,
44]. Zhang et al. [
35] proposed a data-driven learning method based on deep CNN. RED-CNN [
4] combines the autoencoder, deconvolutional network and shortcut connections into the residual encoder-decoder CNN for LDCT imaging. Kang et al. [
36] applied deep CNN to the wavelet transform coefficients of LDCT images, used directional wavelet transform to extract the directional component of artifacts. Wang et al. [
39] developed a limited-angle translational CT (TCT) image reconstruction algorithm based on U-Net [
40]. Since Goodfellow et al. proposed Generative Adversarial Nets (GAN) [
42] in 2014, GAN has been widely used in various image processing tasks, including the post-processing of CT images. Xie et al. [
43] proposed an end-to-end conditional GAN with joint loss function, which can effectively remove artifacts.
Sinogram inpainting algorithms firstly restore the missing part in the Radon domain, then reconstruct it into the image domain to get the final result [
45,
46,
47,
48,
49]. Li et al. [
45] proposed an effective GAN-based repairing method named patch-GAN, which trains the network to learn the data distribution of the sinogram to restore the missing sinogram data. In another paper [
46], Li et al. proposed SI-GAN on the basis of [
37], using a joint loss function combining the Radon domain and the image domain to repair “ultra-limited-angle” sinogram. In 2019, Dai et al. [
47] proposed a limited-view cone-beam CT reconstruction algorithm. It slices the cone-beam projection data into the sequence of two-dimensional images, uses an autoencoder network to estimate the missing part, then stack them in order and finally use FDK [
50] for three-dimensional reconstruction. Anirudh et al. [
48] transformed the missing sinogram into a latent space through a fully convolutional one-dimensional CNN, then used GAN to complement the missing part. Dai et al. [
49] calculated the geometric image moment based on the projection-geometric moment transformation of the known Radon data, then estimated the projection-geometric moment transformation of the unknown Radon data based on the geometric image moment.
Sinogram inpainting and image refining algorithms firstly restore the missing part in the Radon domain, then reconstruct the full-view Radon data into the image domain so as to finely repair the image to obtain higher quality [
51,
52,
53,
54,
55]. In 2017, Hammernik et al. [
51] proposed a two-stage deep learning architecture, they first learn the compensation weights that account for the missing data in the projection domain, then they formulate the image restoration problem as a variational network to eliminate coherent streaking artifacts. Zhao et al. [
52] proposed a GAN-based sinogram inpainting network, which achieved unsupervised training in a sinogram-image-sinogram closed loop. Zhao et al. [
53] also proposed a two-stage method, firstly they use an interpolating convolutional network to obtain the full-view projection data, then use GAN to output high-quality CT images. In 2019, Lee et al. [
54] proposed a deep learning model based on fully convolutional network and wavelet transform. In the latest research, Zhang et al. [
55] proposed an end-to-end hybrid domain CNN (hdNet), which consists of a CNN operating in the sinogram domain, a domain transformation operation, and a CNN operating in the image domain.
However, we cannot help but notice that, when it comes to image restoration, all the methods above merely focus on a single CT image while neglecting the solid fact that the scanned object are often spatially continuous. On account of that, these obtained consecutive CT images are always highly correlative, which leads to copious spatial information hidden between them that is still largely left to be explored. Consequently, we propose a novel two-step cascaded model in the second stage which concentrates on groundbreakingly utilizing the strong spatial correlation between consecutive CT images. So as to break the limit of feature extraction in the two-dimensional space and dig deep into the three-dimensional spatial neighborhood.
These two domains are also combined in our method to amalgamate their respective strengths for high-quality CT reconstruction results, which leads to our proposed three-stage structure. Specifically, we firstly conduct data completion in the Radon domain to acquire the full-view CT data, and then reconstruct it into images through FBP. Subsequently, image restoration and artifacts removal are accomplished in a “coarse-to-fine” [
56] manner with the combination of stage two and stage three.
It is also worth mentioning that, unlike other current prevailing limited-view CT reconstruction methods [
39], we adopt FBP [
16] (and implement it on GPUs) instead of SART-TV [
13] to speed up the overall procedure. Besides, since our method actually consists of fully convolutional networks, it does not limit the resolution of input images, thus can be well generalized to various datasets. In our experiments, we compare our algorithm with other methods under four sorts of limited-view CT data, exhibiting its prominent performance and robustness.
The organization of this paper is as follows,
Section 2 presents our proposed method in detail,
Section 3 exhibits the experimental results and corresponding discussion, and conclusion is stated in
Section 4.
4. Conclusions
In order to obtain the ideal high-quality restoration results from the limited-view CT images that contains severe artifacts, we propose a hybrid-domain structure that efficaciously utilizes the spatial information between consecutive CT images, and utilizes the idea of “coarse to fine” to refine the image texture.
In the first stage, we establish an adversarial autoencoder to preliminarily complement the original limited-view Radon data. After converting the obtained full-view Radon data into images through FBP, and feed them into our proposed Spatial-AAE in stage two for artifacts removal based on spatial information. By now, we have managed to thoroughly eliminate the severe artifacts from the original limited-view CT images, while the image texture still needs to be further refined. Therefore, in the third stage, we propose the Refine-AAE to refine the image in the form of patches, so as to achieve the accurate restoration of the image texture.
For limited-view Radon data that lacks the rear 60 projection views, our method can increase its PSNR to 40.209, and SSIM to 0.943, not only largely improve the image quality compared to other current methods, but also precisely present the image texture. At the same time, our method can be well applied to other sorts of limited-view CT data with more serious artifacts in their imaging, demonstrating its remarkable robustness.