An Imaging Enhancement Method for a Terahertz Rotation Mirror Imaging System Based on a Scale-Recurrent Network

Abstract

The terahertz (THz) rotation mirror imaging system is an alternative to the THz array imaging system. A THz rotation mirror imaging system costs less than a THz array imaging system, while the imaging speed of a THz rotation mirror imaging system is much higher than the imaging speed of a THz raster-scan imaging system under the same hardware conditions. However, there is some distortion in the THz image from the THz rotation mirror imaging system. The distortion, which makes images from the THz rotation mirror imaging system difficult to identify, results from the imaging principle of the THz rotation mirror imaging system. In this article, a method based on the scale-recurrent network (SRN) is put in place to correct the distortion. A comparison between distorted THz images and corrected images shows that the proposed method significantly increases the structural similarity between the THz images and the samples.

1. Introduction

THz waves are electromagnetic waves ranging from 0.1 THz to 10 THz [1]. THz waves are nonionizing [2] and have the ability to penetrate foam [3], ceramic [4], and magnetic material [5] as well as polymer composites [6]. Therefore, the THz imaging technique has great application prospects for non-contact [7,8] and non-destructive [9,10] detection. In addition, the use of terahertz radiation has a very promising future in medical imaging due to its nonionizing properties [11].

In the last 20 years, research on THz imaging has been conducted on a large scale [12,13]. There are two types of THz imaging techniques. One is called the THz active imaging technique, which includes THz computed tomography (CT) [14], THz synthetic aperture radar (SAR) [15], terahertz frequency-modulated continuous wave (FMCW) radar [16], THz time-domain spectroscopy (TDS) [17] and THz scanning near-field microscopy (SNOM) [18], etc. The other is the THz passive imaging technique [19,20,21,22].

For the THz active imaging technique, the THz pixel-by-pixel scanning imaging method is the most frequently used imaging method because THz array components are expensive. The THz raster-scan imaging system is the most common THz pixel-by-pixel scanning imaging method. However, the THz raster-scan imaging method has always been criticized for its slow imaging speed. The application of the THz raster-scan imaging method is limited in some fields because of the THz raster-scan imaging method’s slow imaging speed; for example, in the field of THz biomedical imaging, it is hard to keep a creature at rest for a long time. The imaging process must be completed in a short time. Therefore, the THz array imaging system, which is expensive, is most widely used in the field of THz biomedical imaging [23,24].

The THz rotation mirror imaging system, first suggested by C. A. Weg et al. in 2009 [25], is a kind of THz pixel-by-pixel scanning imaging system. This system is offered as an alternative to the THz array imaging system [26]. A THz rotation mirror imaging system costs less than one tenth of a THz array imaging system. Additionally, the imaging speed of a THz rotation mirror imaging system is much higher than the imaging speed of a THz raster-scan imaging system under the same hardware conditions.

Though the THz rotation mirror imaging system has many advantages, there is one major drawback: distortion exists in the THz image from the THz rotation mirror imaging system. Depending on the imaging principle of the THz rotation mirror imaging system, the THz image is reconstructed by a series of triangles [27]. Therefore, the THz image from the THz rotation mirror imaging system consists of a series of triangles instead of squares. That is the main reason for the distortion of the THz image.

Many studies on distortion correction have been reported. Some of them focus on the distortion correction of diffusion magnetic resonance imaging (DMRI) [28], some of them focus on the distortion correction of radial lenses [29], some of them focus on the distortion correction of cameras [30], some of them focus on the distortion correction of scanning electron microscopy imaging [31], etc. However, few of them pay attention to the distortion correction of the THz rotation mirror imaging system.

The distortion correction of the THz rotation mirror imaging system can hardly be resolved by traditional methods because the distortion correction is an ill-posed problem. In this article, a method based on the scale-recurrent network (SRN) is introduced to correct the distortion of the THz rotation mirror imaging system, considering that the basic outlines of the sample in the THz image are unchanged. SRN, proposed by X. Tao et al. in 2018 [32], has the ability to learn image features in a coarse-to-fine way.

In this paper, a training set is built. The training set allows the proposed network to be trained well so that the distortion of the THz rotation mirror imaging system can be effectively corrected by the network.

According to the quantitative results, the method proposed in this article has the ability to increase the THz image quality effectively.

2. Theory

2.1. Imaging Principle of THz Rotation Mirror Imaging System

A schematic diagram of the THz rotation mirror imaging system is shown in Figure 1. According to Figure 1, passing through the shaping lens, the THz wave from the source then hits the rotation mirror. The rotation mirror deflects the converging wave and thus directs it to the sample. Penetrating through the sample, the THz wave is then focused on the detector by the collecting lens. The scanning is achieved by the rotation mirror rotating about two perpendicular axes. The rotation mirror is installed on a spinning axis. The tilt at an angle causes an ellipse-shaped scanning path of the focal spot in the object plane. A rotary stage turns the first axis in a direction perpendicular to it, thereby leading to a linear movement of the scanning ellipse. Figure 2a shows the scan path—i.e., the scan points from which the imaging data are received. The triangulation of the scan points based on the Delaunay algorithm [33] is presented in Figure 2b. The image can be obtained by filling every triangle with the average pixel value of its three vertexes.

2.2. Network Structure

The proposed network is expressed as:

$$Output=Ne{t}_{E-D}\left(Input,Ne{t}_{E-D}\left(Input\downarrow ,Ne{t}_{E-D}\left(Input\downarrow \downarrow ,Input\downarrow \downarrow ,h^1;{\theta }_1\right)\uparrow ,h^2;{\theta }_2\right)\uparrow ,h^3;{\theta }_3\right)$$

(1)

where Net_E-D is an Encoder-Decoder block with the training parameter θ, h is the hidden state feature of the Encoder-Decoder block, Input is the THz image, Output is the output of the network, ↓ is a downsampling operator mand ↑ is an upsampling operator. On the proposed scale-recurrent network, image restoration is carried out in a coarse-to-fine manner.

The proposed network structure is illustrated in Figure 3. As shown in Figure 3a, the overall structure is a scale-recurrent structure. The downsampling operation will halve the image size while the upsampling operation, realized by way of bilinear interpolation, will double the image size. The structure of the Encoder-Decoder block is shown in Figure 3b. The Encoder-Decoder Residual block contains an Input Residual block, two Encoder Residual blocks, two Decoder Residual blocks, and an Output Residual block. There are 64 filters and 128 filters in the two Encoder Residual blocks (Eblock) and there are 128 filters and 64 filters in the two Decoder Residual blocks (Dblock). The Eblock includes a convolution layer (Conv), and three Residual blocks. The stride for the convolution layer is two. It doubles the number of kernels of the previous layer. There are two convolution layers in a Residual block (Resblock). The Dblock contains three Residual blocks and one deconvolution layer (Deconv). The deconvolution layer will halve the number of kernels. The Input Residual block (Iblock), which produces a 32-channel feature map, contains three Residual blocks and one deconvolution layer. The Output Residual block (Oblock), which generates the output image, contains three Residual blocks and a convolution layer. In the network proposed in this article, the convolution kernels are 3 × 3 for every convolution layer.

The mean squared error (MSE) is adopted as the loss function of the proposed model. The loss function can be expressed as:

$$L_{MSE}(\theta )=\frac{1}{3N}{\displaystyle {\sum }_{k=1}^3{\displaystyle {\sum }_{i=1}^N\Vert I_{predict}\left(I_{in},\theta \right)-I_{s\tan dard}\Vert }}$$

(2)

where L_MSE denotes the MSE between the standard images and the predicted images, N denotes the number of training samples, θ denotes the weight of the network, I_predict is the output of the network, and I_standard is the standard image.

2.3. Training Set Creation

In Figure 4, the Z axis is the first spinning axis of the THz rotation mirror system. Normal1 is normal to the rotation mirror plane. Point O is the incident point of the incident light on the rotation mirror plane. The path of incidence is in the XZ plane. Normal2 is normal to the object plane. In spherical coordinates, let the unit vector of incident light be (1, θ, 0), the unit vector of Normal1 be (1, β, γ), the unit vector of Normal2 be (1, φ, 180°), and the distance between the point O and the object plane be d. Then in a Cartesian coordinate system of (1, 90°−φ, 0), axis Y and Normal2, the scanning spot (x’, y’) on the object plane is the point at which the vector (x’, y’, d) intersects the object plane and can be expressed as:

$$\{\begin{array}{l}x\prime =\frac{d({\kappa }_1\cos \phi +{\kappa }_2\sin \phi )}{{\kappa }_2\cos \phi -{\kappa }_1\sin \phi }\\ y\prime =\frac{d({\kappa }_2+\cos \theta )\tan \beta \sin \gamma }{{\kappa }_2\cos \phi -{\kappa }_1\sin \phi }\end{array}$$

(3)

where 𝜅₁, 𝜅₂ can be calculated as:

$$\{\begin{array}{l}{\kappa }_1=\tan \beta \cos \gamma \left(c+\cos \theta \right)-\sin \theta \\ {\kappa }_2=\frac{\cos \beta \left[2{\left(\sin \beta \cos \gamma \sin \theta +\cos \beta \cos \theta \right)}^2-1\right]+\cos \beta {\sin }^2\theta -\sin \beta \cos \gamma \sin \theta \cos \theta }{\sin \beta \cos \gamma +\cos \beta \sin \theta }\end{array}$$

(4)

In this paper, the θ is from 22° to 32° at 0.5° intervals, the γ is from 0° to 359.5° at 0.5° intervals, β is 5° and the φ = 50°−θ. Thus, the scanning spots on the sampled image can be determined.

Considering the image degradation, the sampled image can be obtained from the following equation:

$$I_{sampled}=\left(I_{s\tan dard}\otimes PSF\right)+n$$

(5)

where I_sampled is the sampled image, I_standard the standard image, PSF is the Gaussian point spread function (GPSF), n is the additive white Gaussian noise (AWGN), and ⨂ is the convolution operation. Because the radius of the spot on the object plane is about 6 mm, a 13 × 13 matrix is selected as the Gaussian point spread function PSF. Elements in the PSF can be calculated as:

$$PSF=\frac{1}{2\pi ·6^2}\exp \left(-\frac{\left(x^2+y^2\right)}{2·6^2}\right)$$

(6)

Acquiring all the scanning points on the sampled image, the simulation THz image can be reconstructed using the Delaunay algorithm.

3. Experimental Setup and Implementation Details

3.1. THz Rotation Mirror Imaging Experiment

A THz rotation mirror imaging system was built to carry out the imaging experiment. Figure 1 shows the schematic diagram of the experimental setup. A photograph of the THz rotation mirror imaging system is presented in Figure 5a. The source, produced by TeraSense, is a THz impact ionization avalanche transit-time (IMPATT) diode. The radiated power of the source is 2 milliwatts (mW) and the center frequency of the transmitted THz wave is 0.3 THz. The detector with a frequency response range from 0.1 to 1.15 THz is a high electron mobility field–effect transistor (FET). The shaping lens is a 2 inch convex lens and its focal length is 75 mm. The collecting lens is a 4 inch convex lens and its focal length is 100 mm. The reflecting surface of the rotation mirror is tilted at an angle of 5° against the rotation mirror’s fast rotation axis.

Three samples with metal layers in the shapes of three different letters were designed for the imaging experiment. The metal layers of the three samples are shown in Figure 5b and the common structure of them is shown in Figure 5c. As can be seen in Figure 5c, the sample consists of 2 pieces of cardboard with a metal layer hidden in between them.

3.2. THz Raster-Scan Imaging Experiment

The THz raster-scan imaging experiment was conducted with the purpose of acquisition of the standard image because there is little distortion in the THz image from the THz raster-scan imaging system. An experimental system for THz raster-scan imaging is illustrated in Figure 6. The source is a Gunn diode oscillator with 0.5 mW of radiated power and 0.28 THz of radiated frequency. The detector is the same as the one mentioned in Section 3.1. Every lens in this system is a convex lens with a 50 mm focal length and a 2 inch aperture. A sample is mounted on the stepper motor platform. The scanning is realized by a two-dimensional motion of the stepper motor platform.

3.3. Implementation Details of Network Training and Image Predicting

The training set includes 33,200 images gained by the translation or rotation of printed Chinese characters. The size of each of these images is 256 × 256. Each image, taken as the standard image, was converted into grayscale. The distorted image was obtained by the method set out in Section 2.3. Figure 7 shows some training image pairs from the training set.

When the network is in training, the initial learning rate is set to 2.7 × 10⁻⁶, the batch size is set to 16, and the minimal learning rate is set to 1 × 10⁻⁶. Additionally, the stochastic gradient descent (SGD) method [34], instead of the Adam optimization algorithm, is used to optimize the network’s loss function.

After training, the predictive network is gained. By smoothing the output of the foregoing predictive network with a 9 × 9 median filter and carrying out the histogram specification, the final corrected image is obtained.

4. Results and Discussion

Figure 8 presents the THz images of the samples obtained from the THz raster-scan imaging experiment (each image took more than 40 min). Comparing Figure 8 with Figure 5b, it can be seen that the THz images obtained from the THz raster-scan imaging experiment are almost identical to the samples themselves. Therefore, the images in Figure 8 can be regarded as the standard images, which are the reference to evaluate the image quality.

Figure 9 is the THz images of the samples acquired by the THz rotation mirror imaging system (the imaging time of each image is about 30 s). As illustrated in Figure 9, there are jagged distortions in the THz images, particularly around the contour.

In Figure 10, the THz images in Figure 9 have been corrected according to the method proposed in this article. Comparing Figure 9 with Figure 10, it can be seen that the twisted contour of sample 1′s THz image has been reverted to a straight one in Figure 10a; a curved and fuzzy part of sample 2′s THz image has been corrected in Figure 10b; and an indistinct part of sample 3′s THz image has become legible in Figure 10c.

For the purpose of assessing the effectiveness of the distortion correction quantitatively, structural similarity index measures (SSIM) [35], are introduced. The SSIM of two images can be calculated as:

$$SSIM(x,y)=\frac{\left(2{\mu }_x{\mu }_y{+\mathrm{c}}_1\right)\left(2{\sigma }_{xy}+c_2\right)}{\left({\mu }_x^2+{\mu }_y^2{+\mathrm{c}}_1\right)\left({\sigma }_x^2+{\sigma }_y^2{+\mathrm{c}}_2\right)}$$

(7)

where μ_x is the mean of image x, μ_y is the mean of image y, 𝜎_x is the variance of image x, 𝜎_y is the variance of image y, 𝜎_xy is the covariance of image x and image y, c₁ = (0.01L)² and c₂ = (0.03L)² are constants used to maintain stability, and L is the dynamic range of pixel values. In particular, the SSIM value is equal to 1 when the two images are identical.

Table 1. Quantitative results for original images and corrected images.

Name	Original Image	Corrected Image
	SSIM	SSIM
Sample 1	0.5101	0.7368
Sample 2	0.4637	0.6652
Sample 3	0.4971	0.6813

demonstrates the quantitative results for the original images and the corrected images. As can be seen in Table 1, the SSIM dramatically increases after distortion correction with the proposed method. This demonstrates that the proposed method based on SRN has the ability to solve the distortion problem of the terahertz rotation mirror imaging system effectively.

5. Conclusions

The scanning mode of the rotation mirror imaging system is circular scanning. When a THz rotation mirror imaging system scans the target, the stepper motor does not need to go through the motion-stop-motion process like the traditional raster-scanning imaging system. Therefore, the imaging speed of the THz rotation mirror imaging system is much higher. Considering that the cost and electronic complexity of a THz rotation mirror imaging system are far lower than those of a THz array scanning imaging system, the THz rotation mirror imaging system is an effective substitute for the THz array scanning imaging system. However, there are a lot of distortions in the imaging results of the THz rotation mirror imaging system due to the imaging principle of the THz rotation mirror imaging system. In this article, a method based on SRN is proposed to implement the imaging enhancement of the terahertz rotation mirror imaging system. In the proposed scale-recurrent network, image restoration is carried out in a coarse-to-fine manner. The imaging principle of the THz rotation mirror imaging system is analyzed. Then, a training set is created using the imaging principle of the THz rotation mirror imaging system. To verify the ability of the proposed method to restore the distorted THz image, an experiment was carried out. SSIM was adopted as the quantitative evaluation index for the distortion correction effect. According to the quantitative results, the SSIM between the standard image and the corrected image is much higher than the SSIM between the standard image and the original image. The above results indicate that the corrected results are much closer to reality. The proposed method based on SRN is an effective and practical method. In the future, the network could be further optimized to achieve a higher SSIM between the standard image and the corrected image. In the experiments of this paper, the imaging time of the THz rotating mirror imaging system is about 1/80 that of the raster-scanning system. The THz rotation mirror imaging system has the advantage of a fast imaging speed. As the THz image distortion is solved, the application of the THz rotation mirror imaging system will be further popularized.