Exploring Frontiers of 4D X-ray Tomography

Abstract

In the 4D world of three-dimensional (3D) space plus time that we live in, there is a vast blue ocean in the spatio-temporal domain of micrometers and milliseconds that has never been accessed even with the most advanced measurement technology, and it is expected to be full of various non-equilibrium phenomena. In this paper, we review recent advances in synchrotron hard X-ray tomography we have made that can be used to explore the 4D frontier.

1. Introduction

While telescopes and microscopes have been used for more than 400 years to unravel the mysteries of the universe and matter, the ability to take continuous images beyond the limits of the human eye is a relatively recent development, dating back only about 150 years. Since the advent of the charge-coupled device (CCD) image sensor in 1970 [1], digital cameras have gradually replaced film cameras, and this digitalization has dramatically accelerated the speed of continuous shooting. Recently, complementary metal-oxide-semiconductor (CMOS) cameras with a frame rate of over 1000 frames per second have been available at relatively low prices, and those with a frame rate of even over 100,000,000 frames per second have been developed in the most advanced fields [2].

However, such high-speed CMOS cameras can only capture visible light that is reflected from the surface of an object or transmitted through a transparent object. Many objects in nature are opaque, and it is not possible to observe high-speed phenomena occurring inside them using visible light. X-ray computed tomography (CT) and magnetic resonance imaging (MRI) can visualize the interior of objects in three dimensions, but they generally cannot capture images faster than the order of seconds.

In this paper, we will introduce X-ray CT with a temporal resolution in the order of milliseconds that we recently developed. The spatio-temporal domain described in this paper is an unexplored world, a blue ocean that is expected to be full of various non-equilibrium phenomena that have never been seen before. In 2020, we successfully demonstrated proof-of-concept X-ray CT that can capture a three-dimensional (3D) image of a sample in 1 ms without sample rotation by using a synchrotron radiation multi-beam optical system, and it is expected to be applied to various kinds of material and biological research in the future.

2. High-Speed X-ray CT

2.1. X-ray Imaging

X-rays have been used for imaging for more than 120 years since their discovery by Röntgen in 1895 [3]. The advantage of hard X-rays lies in their high penetration and straight propagation, both of which are due to the weak interaction of X-rays with matter. The former is due to the low absorption of X-rays, which makes it possible to observe the interior of an object. The latter is due to the fact that X-rays are only slightly refracted by matter, which not only facilitates the interpretation of projection images but also led to the success of CT in visualizing the interior of objects in three dimensions. X-ray imaging and CT are now widely used as indispensable tools in various fields ranging from medical diagnosis to non-destructive testing.

In general, improving the resolution is an eternal issue in imaging technology. We have to consider spatial resolution, which determines the size that can be spatially resolved, temporal resolution, which determines the sampling interval at which data can be collected, and density resolution, which determines how much difference in density can be distinguished. There is generally a trade-off relationship among them. For example, to improve the temporal resolution by a factor of 100 while keeping the sensitivity of the optical system constant, we must set the spatial resolution to 10 times lower by increasing the pixel area by a factor of 100. Likewise, if we want to increase the temporal resolution by a factor of 10 while maintaining the spatial resolution, we need to increase the sensitivity of the optical system by an absolute factor of 10.

Thus, to improve the temporal resolution while maintaining the spatial resolution, it is necessary to improve the sensitivity of the imaging system. Using the phase shift of X-rays caused by an object provides a solution to improving the sensitivity. In classical electromagnetism, X-rays are described as electromagnetic waves, i.e., transverse waves of electric and magnetic fields, characterized by phase, amplitude, frequency, and direction of polarization. When X-rays pass through an object, the intensity of the transmitted X-rays decreases due to the attenuation of the amplitude. A conventional X-ray image is an image of the contrast caused by this attenuation. In addition, when X-rays pass through an object, the phase of the X-rays also shifts, as shown in the left of Figure 1. Imaging carried out using the phase shift of X-rays is called “X-ray phase-contrast imaging” [4,5]. This phase shift can also be understood as the deformation of the wavefront caused by the refraction of X-rays by an object and is therefore also called “X-ray refraction-contrast imaging”.

In principle, the use of X-ray phase shifts can improve sensitivity by several orders of magnitude compared with imaging methods that use X-ray absorption. This is mainly due to the fact that the cross section of the elastic scattering of X-rays by electrons (Thomson scattering) is larger than that of photoelectric absorption. As shown in the right pad of Figure 1, the phase shift cross section is several orders of magnitude larger than the absorption cross section for a given energy, especially for light elements with small atomic numbers. By taking quantitative X-ray phase-contrast images from various directions, the so-called phase CT can be realized. In this case, what is reconstructed in the CT is a physical quantity corresponding to the deviation $\delta$ of the refractive index from 1 (refractive index decrement) for X-rays, which is proportional to the density of electrons per unit volume, except in special cases where the energy of X-rays used is close to an absorption edge. The refractive index decrement $\delta$ is about ${10}^{-5}$ to ${10}^{-6}$, which is so small that X-rays are hardly refracted, but it can be detected by special methods as described below. Note that the absorption of X-rays is described by the imaginary part of the complex refractive index, which is about ${10}^{-8}$ to ${10}^{-9}$ several orders of magnitude smaller.

Four typical optical systems that have been proposed as methods for realizing X-ray phase-contrast imaging are compared in Figure 2. The method in Figure 2a is called “crystal interferometry” [6,7,8,9], which uses a Mach–Zehnder interferometer using Bragg reflection by a single crystal. Figure 2b shows diffraction enhanced imaging (DEI) [10,11,12], a method using an analyzer crystal, which detects the slight refraction of X-rays by an object by utilizing the very small angular width of the Bragg reflection of a single crystal (about 10 $\mathsf{\mu }$rad). The imaging methods in Figure 2a,b require monochromatic X-rays, so the measurement time for obtaining a projection image is generally on the order of seconds even when highly brilliant synchrotron radiation X-rays are used. The methods in Figure 2c,d are called the “propagation-based method” [13,14,15,16] and “grating interferometry” [17,18,19,20,21,22,23,24], respectively, and can use X-rays with a wide energy bandwidth (continuous X-rays) and spherical waves, which means that high-intensity X-rays can be used. Therefore, they are suitable for imaging with a high temporal resolution. The propagation-based method can be realized with a very simple configuration consisting only of a sample and an image detector. Images with enhanced edges can be obtained on the basis of the interference between slightly refracted X-rays inside and outside the boundaries of the structures in a sample. However, the pseudo-homogeneous assumption [16] is required for the quantitative analysis of the phase shift. Grating interferometry, in comparison, allows for quantitative imaging without any special assumptions. Another advantage of grating interferometry over the propagation-based method is its multi-modality; three independent projection images, absorption, differential-phase, and visibility-contrast images [23,24] can be obtained, as shown in Figure 3.

2.2. X-ray CT

Since its development in the 1970s, X-ray CT has been used in a wide range of fields including medical diagnosis and non-destructive testing. Hounsfield and Cormack were awarded the Nobel Prize in Physiology or Medicine in 1979 for their work on X-ray CT. In CT, a projection of a physical quantity is obtained experimentally in a number of directions. Here, the projection is mathematically just a line integral along a certain direction. For example, the projection $q\left(x\right)$ of the physical quantity $p(x,y)$ in the direction parallel to the y-axis is given by

$$q\left(x\right)={\int }_{-\infty }^{\infty }p(x,y)dy.$$

(1)

Solving the inverse problem of obtaining a tomogram, $p(x,y)$, from a set of projections in many directions is called “tomographic reconstruction” [3]. We can obtain a three-dimensional image, $p(x,y,z)$, by stacking tomograms $p(x,y)$ in the z direction.

In the case of visible light transmitted through a transparent object, the deviation in the refractive index from 1 is generally large, so it is not easy to obtain a projection experimentally. However, in the case of X-rays, the interpretation of an experimentally acquired projection image becomes much easier since X-rays are only slightly refracted by materials.

In X-ray CT using absorption, which is historically the oldest, projections of the linear absorption coefficient are obtained experimentally from X-ray transmission, and a three-dimensional image of the linear absorption coefficient is reconstructed. In the case of X-ray CT using phase shift, projections of the refractive index decrement $\delta$ are obtained, and a three-dimensional image of $\delta$, which is approximately proportional to the number density of electrons, is reconstructed. Note that, for a synchrotron radiation beam, a tomographic reconstruction algorithm for a plane wave can be used because it can be approximated as a plane wave, while CT measurement using a laboratory X-ray source requires a tomographic reconstruction algorithm [25] for a cone beam for a large sample.

3. High-Speed X-ray CT

3.1. With Sample Rotation

The X-ray CT scanners commonly used in hospitals generally take several tens of seconds to several tens of minutes to take a 3D image. In the case of medical X-ray CT, an X-ray source and X-ray image detector are rotated around a target object in order to capture projection images from many directions. The speed of these rotations places a limit on the measurement time. Sub-second scan times have been achieved in state-of-the-art devices [26], but now, the intensity of the X-rays is an issue. A low-brilliance X-ray source based on bombarding a target material with accelerated electrons is commonly used in hospitals, and the melting of the target determines the intensity limit. Therefore, it is necessary for high-speed measurements to sacrifice the spatial resolution of the X-ray image detector to increase the number of counts for each pixel.

Time scales that can be perceived by the human eye have been the mainstream for not only X-ray CT but also other methods of non-destructive three-dimensional observation of the interior of objects opaque to visible light. Figure 4 outlines the temporal and spatial resolutions of various 3D visualization methods for samples opaque to visible light, including X-ray CT. It can be seen that the domain of millisecond- and micrometer-order spatio-temporal resolution introduced in this paper has never been explored. The pink zone in the figure shows, as an example, the spatial and temporal resolutions suitable for observing the movement of animals. This zone roughly represents kinetic speed determined by metabolic rate × biological time. The zone corresponding to the mechanical motion of artifacts (green zone) is one or two orders of magnitude higher than this. Many of the phenomena we come across in our daily lives occur in these zones, making them extremely important subjects of research in science, technology, and industry. Various applications are possible for measurement methods that can access these temporal and spatial scales, from academic research, such as observing phenomena inside living animals (e.g., insects), the failure process of materials, and the behavior of liquids, to industrial applications, such as developing intelligent materials, and dynamic biomimetics applications.

The trade-off between temporal and spatial resolutions (dashed line) is also shown in Figure 4. To increase the temporal resolution, it is generally sufficient to sacrifice the spatial resolution. In fact, temporal resolution on the order of milliseconds has been achieved for a spatial resolution reduced to the order of millimeters. It is worth noting, however, that the direction of this trade-off relationship is close to orthogonal to the pink and green zones. In other words, the smaller the observation object, the larger the demand for high-speed viewing, but improving both spatial and temporal resolutions is often technically challenging.

Thus, how can we realize high-speed X-ray CT that far exceeds the limits of what the human eye can perceive? It is impossible to rotate the X-ray source and the X-ray image detector of a hospital X-ray CT scanner at the speed that is needed. Additionally, the intensity of X-ray sources used at the laboratory level is insufficient.

Highly intense synchrotron radiation X-rays can solve this problem [27,28,29,30,31]. The authors succeeded in X-ray CT with an imaging time on the order of milliseconds by using a third-generation synchrotron radiation source and rotating samples at high speed [27,28,29]. The experiments were performed at the experimental station BL28B2 at SPring-8, a large synchrotron radiation facility. Third-generation synchrotron radiation sources such as SPring-8 are capable of producing much more brilliant X-rays (i.e., a high-intensity X-ray beam from a small light source) than laboratory X-ray sources. The small size of the light source allows us to use high-spatial-coherence X-rays, which is suitable for X-ray phase-contrast imaging. The advantage of this imaging is that it is more sensitive than X-ray absorption imaging as described in Section 2.1. With this imaging, the experimentally obtained refractive index decrement $\delta$ for X-rays is taken as the physical quantity $p(x,y)$ in Equation (1).

The experimental setup (side view) is shown in Figure 5. Here, we used X-ray grating interferometry. As described in Section 2.1, this method uses gratings for X-rays (the two gratings in Figure 5), and it has the advantage that a white synchrotron radiation beam with high intensity and a relatively large area can be used.

In grating interferometry [32], a self-imaging phenomenon called the Talbot effect [33,34] is used. The Talbot effect occurs when a periodic object such as a grating is illuminated by light with a spatial coherence length comparable to or larger than the period of the object. The self-images generated downstream of the object are deformed when light is refracted by a sample located in front of or behind the object. From this deformation, we can obtain an image of the change in the propagation direction due to the refraction by the sample, which corresponds to the differential phase image for the sample.

For the X-ray grating interferometer shown in Figure 5, we used a $\pi /2$-phase grating (G1) designed for an X-ray energy of 25 keV, whose lines are aligned in the horizontal direction. The grating was located at a distance of 44 m from the X-ray source with a full width at half maximum (FWHM) of 28.7 $\mathsf{\mu }$m, which, from the van-Cittert Zernike theorem [35], provides a spatial coherence length at G1 sufficiently larger than the period of G1 (5.3 $\mathsf{\mu }$m). Another 5.3 $\mathsf{\mu }$m-period grating (a 60 $\mathsf{\mu }$m-thick gold absorption grating (G2) was located on a self-image generated at a distance of 283 mm downstream of G1 to detect the deformation of the self-image as moiré fringes. Both gratings were made of gold on 200 $\mathsf{\mu }$m-thick Si substrates.

To achieve high-speed imaging, an indirect imaging-type X-ray detector was used that consists of a scintillator, a lens coupling system, and a high-speed camera. A scintillator is a material that converts X-rays into visible light [36]. The 40 $\mathsf{\mu }$m-thick single-crystal Ce:GAGG scintillator used in this experiment had a very short luminescence decay time of 53 ns and a large luminescence intensity [37]. As a high-speed camera, we used a CMOS camera (Photron FASTCAM Mini AX100), which is capable of capturing 1024 × 1024 full-frame images at a maximum of 4000 frames per second. The effective pixel size at the sample position was 9.9 $\mathsf{\mu }$m, and the spatial resolution was 21 $\mathsf{\mu }$m.

Figure 6 shows a 3D phase tomogram of a toothpick (piece of hardwood) as an example (3D CT reconstruction image of the refractive index decrement $\delta$) [29]. This 3D image was obtained on the basis of a fringe scanning method [38,39,40]: a three-step equal-sampling fringe scanning was performed by continuously moving G2 and rotating the sample. The speed of G2 was tuned so that it was moved by one period in the vertical direction while the sample was rotated through three turns. Since three projection images in the same projection angle are necessary for the three-step fringe scanning, 2.5 turns of the sample are required for CT reconstruction for the synchrotron beam. The sample was rotated at 33,850 rpm, so that the 3D image was captured in 4.43 ms. The projection images were captured at a frame rate of 127,500 frames per second (7.8 $\mathsf{\mu }$s per frame), and the 3D phase tomogram was obtained from 113 differential phase images by the filtered back-projection algorithm. Note that, for the projection images, the field of view was reduced to be 128 × 64 pixels at the frame rate because there is generally a trade-off between the frame rate and the field of view of a high speed camera. The specimen was not stained with a contrast agent, but high contrast images of the internal structure, such as canals and xylem fibers, were obtained by using the phase of the X-rays, even though the material is composed of light elements that absorb X-rays very little.

Recently, dynamic 4D observation has also become possible. Figure 7 demonstrates 4D X-ray CT with a temporal resolution of 10 ms for the process of breaking rubber while pulling it [41,42]. The upper left pad of Figure 7 shows a schematic illustration of the rotating device for the 4D X-ray CT. It comprises two coaxial inverter-driven motors, which can be electrically synchronized by a motor controller. The motors for rotation are mounted on two carriages on parallel stainless-steel rails and are uniaxially movable along the rails in the direction parallel to the rotation axis (in the horizontal direction). The rubber sample was bonded on top of the motor axes, as shown in the upper right pad of Figure 7, and stretched while rotating at high speed.

The indirect imaging-type X-ray detector with a 10 $\mathsf{\mu }$m-thick single crystal Ce:GAGG scintillator was employed for the 4D X-ray CT. Here, a CMOS camera with a higher frame rate (FASTCAM Nova S12 from Photron, 1024 × 1024 full frame at a maximum of 12,800 frames per second) was used for a high-speed camera.

The sample was located at a distance of 4.7 m upstream of the detector, and full-frame projection images (propagation-based phase-contrast images) were obtained at a frame rate of 12,800 frames per second. The sample was rotated at 3000 rpm, so that the number of projections for half turn of the sample was 128. The effective pixel size was 4.5 $\mathsf{\mu }$m at the sample position, which corresponds to the spatial resolution of the projection images. It was confirmed from the projection images, that the sample was stretched at a constant rate of 1226.8 $\mathsf{\mu }$m/s.

The lower pad of Figure 7 demonstrates a series of the reconstructed 3D images, which is the world’s fastest example of active observation of a micron-scale fracture process using X-ray CT, and it clearly shows the formation of voids inside the rubber. Note that this example is an absorption tomogram, but since the X-rays transmitted through the sample interfered with each other to produce an edge-enhanced projection image, the tomogram also has an enhanced boundary.

3.2. With Compressed Sensing

If we try to further shorten the acquisition time, the number of projections has to be reduced because the frame rate cannot be increased while maintaining the field of view. Figure 8a shows a tomogram of a polypropylene (PP) sphere with a diameter of 3/32 inches acquired at 2.0 ms. Here, the projection images were obtained by the same experimental setup as that of Figure 5, but the Fourier transform method [43] was used instead of the fringe scanning method. In the Fourier transform method, a differential phase image can be obtained from a projection image with fine moiré fringes, so that higher temporal resolution is realized than that of the fringe scaning method, although the spatial resolution becomes worse up to the period of the moiré fringes. We obtained the projection images at a frame rate of 37,500 frames per second with a field of view of 256 × 256 and rotated the sample at a speed of 29,605 rpm. To increase the field of view, the so-called offset CT was used: the two projections from opposite directions were combined to obtain a single projection image. Since only 76 projections for a half turn of the sample can be obtained, using the classical CT reconstruction algorithm (filtered back-projection algorithm) results in radial artifacts as shown in the left pad in the figure.

A compressed sensing-based CT reconstruction method has recently attracted attention as a solution to this problem [28,44,45,46,47,48,49]. In this method, images are recovered from a small number of observation data samples by using the inherent sparsity of images. Sparsity is often inherent in images obtained from the natural world. For example, if we carry out a wavelet transform of an image, extract only 10% of its components, and carry out an inverse transform, the resulting image will be almost indistinguishable from the original. In other words, even if the number of data samples is insufficient, the original image can be obtained by applying an appropriate linear transformation (called sparsify transformation) since the number of non-zero components to be recovered can be approximated to a small number.

The problem of compressed sensing can be mathematically generalized as follows. Suppose that an original physical quantity, $f\left(\mathbf{r}\right)$, results in the observed data $y_i$ ($i=1,\cdots ,M$), as expressed in the following equation.

$$y_i=\int f\left(\mathbf{r}\right){\psi }_i\left(\mathbf{r}\right)d\mathbf{r},$$

(2)

where the inner product with ${\psi }_i\left(\mathbf{r}\right)$ expresses the ith sampling. Additionally, suppose that the original physical quantity $f\left(\mathbf{r}\right)$ can be expanded by the basis ${\varphi }_j\left(\mathbf{r}\right)$ ($j=1,\dots ,N$) of the sparsify transformation as follows

$$f\left(\mathbf{r}\right)=\sum _{j=1}^N{x}_j{\varphi }_j\left(\mathbf{r}\right).$$

(3)

Substituting Equation (3) into Equation (2) and defining $A_{ij}\equiv \int {\psi }_i\left(\mathbf{r}\right){\varphi }_j\left(\mathbf{r}\right)d\mathbf{r}$, we obtain

$$y_i=\sum _{j=1}^N{A}_{ij}{x}_j,$$

(4)

that is, the vector $\mathbf{y}$ with components $y_i$ can be expressed by the product of the matrix $\mathbf{A}$ and the vector $\mathbf{x}$ with components of $A_{ij}$ and $x_j$, respectively. If $M=N$, we can find $\mathbf{x}$ by calculating the inverse; if $M>N$ and there is no $\mathbf{x}$ satisfying $\mathbf{y}=\mathbf{Ax}$, we can find the least-squares solution (the solution that minimizes (${\left(\Vert \mathbf{y}-\mathbf{A}\mathbf{x} {\Vert }_2\right)}^2$) by using the so-called Moore–Penrose pseudo-inverse matrix, for example. If $M<N$, the solution is indefinite and becomes an ill-posed problem. However, if the number K of non-zero components of $\mathbf{x}$ is small and $M\ge K$, then there is a possibility of finding the K non-zero components. The goal of compressed sensing is to solve the problem of recovering the sparse vector $\mathbf{x}$ from $\mathbf{y}$ under the constraint of $\mathbf{y}=\mathbf{A}\mathbf{x}$. The problem can be formulated as

$$\mathrm{minimize}{\Vert \mathbf{x} \Vert }_0\mathrm{subject}\mathrm{to}\mathbf{y}-\mathbf{A}\mathbf{x}=\mathbf{0},$$

(5)

where the $L_p$-norm ${\Vert \mathbf{x} \Vert }_p$ of $\mathbf{x}$ is defined by

$${\Vert \mathbf{x} \Vert }_p=\left\{\begin{array}{cc}{\left(\sum _{j=1}^N{\left|x_j\right|}^p\right)}^{\frac{1}{p}}\hfill & (p>0),\hfill \\ \underset{ϵ\to +0}{lim}{\left(\sum _{j=1}^N{\left|x_j\right|}^{ϵ}\right)}^{\frac{1}{ϵ}}\hfill & (p=0).\hfill \end{array}\right.$$

(6)

That is, ${\Vert \mathbf{x} \Vert }_0$ in Equation (5) is called the $L_0$-norm, which is defined as the number of non-zero components of vector $\mathbf{x}$. If we know where the non-zero components are, we can find $\mathbf{x}$ by extracting only the non-zero components and calculating the inverse matrix, but the difficulty of the problem lies in the fact that we do not know where the non-zero components are.

The problem setup in Equation (5) can be applied to X-ray CT. In this case, the vector of all the pixel values of the projection images acquired from various projection directions corresponds to the observed data $\mathbf{y}$, $f\left(\mathbf{r}\right)$ corresponds to a tomogram, and the transformation coefficients with the sparsify transformation correspond to $\mathbf{x}$. In the case where the observed data $\mathbf{y}$ contains random errors, the constraint in Equation (5) can be included as a penalty term, resulting in the following optimization problem.

$$\mathrm{minimize}\lambda {\Vert \mathbf{x} \Vert }_0+{{\Vert \mathbf{y}-\mathbf{A}\mathbf{x} \Vert }_2}^2,$$

(7)

where ${\Vert \mathbf{y}-\mathbf{A}\mathbf{x} \Vert }_2$ is the square root of the sum of the squared components of the vector $\mathbf{y}-\mathbf{Ax}$ (i.e., the distance in Euclidean space), called the “$L_2$ norm.” The hyperparameter $\lambda$ (>0) regulates the degree of sparsity; the larger $\lambda$, the stronger the degree of sparsity, and the closer the solution to that of Equation (5). The $L_0$-norm optimization problem in Equation (7) is known to be an NP-hard problem when M, N, and K are large, and it can be solved numerically by replacing it with the $L_1$-norm optimization problem as follows.

$$\mathrm{minimize}\lambda {\Vert \mathbf{x} \Vert }_1+{{\Vert \mathbf{y}-\mathbf{A}\mathbf{x} \Vert }_2}^2,$$

(8)

where the $L_1$-norm ${\Vert \mathbf{x} \Vert }_1$ is the sum of the absolute values of each component of the vector $\mathbf{x}$. Equation (8) can be interpreted as a formulation of the problem of finding the $\mathbf{x}$ that minimizes ${{\Vert \mathbf{y}-\mathbf{A}\mathbf{x} \Vert }_2}^2$ under the constraint that ${\Vert \mathbf{x} \Vert }_1$ is constant, using Lagrange’s undecided multiplier $\lambda$. Using the $L_1$ norm instead of the $L_0$ norm makes it easier to choose a sparse solution. The reason can be intuitively explained as shown in Figure 9. We consider a two-dimensional vector $\mathbf{x}$ (i.e., $N=2$) for simplicity. In the figure, a line with constant ${\Vert \mathbf{x} \Vert }_1$ and a contour line of ${{\Vert \mathbf{y}-\mathbf{A}\mathbf{x} \Vert }_2}^2$ are drawn. It can be seen that the solution that minimizes ${{\Vert \mathbf{y}-\mathbf{A}\mathbf{x} \Vert }_2}^2$ under the constraint of constant ${\Vert \mathbf{x} \Vert }_1$ is ${\mathbf{x}}_0$ on a coordinate axis in the figure, which is a sparse solution (the solution with $x_1=0$). Note that the optimization problem in Equation (7) is equivalent to the so-called sparse modeling, i.e., the method of estimating unknown parameters by adding a penalty term to estimate an original physical quantity under the assumption that the parameters are sparse. For image data such as CT image reconstruction, the total variation (TV) norm is often used as it has essentially the same effect as taking the $L_1$ norm after the sparsify transformation. The TV norm of image $\mathbf{p}$, ${\Vert \mathbf{p} \Vert }_{\mathrm{TV}}$, is defined by the following equation:

$${\Vert \mathbf{p} \Vert }_{\mathrm{TV}}\equiv \sum _k\sqrt{{\Delta h_k}^2+{\Delta v_k}^2}.$$

(9)

Here, $\Delta h_k$ and $\Delta v_k$ are the differences in pixel values between the kth pixel and its neighbors in the horizontal and vertical directions, respectively. This problem can be formulated as

$$\mathrm{minimize}\lambda {\Vert \mathbf{p} \Vert }_{\mathrm{TV}}+{{\Vert \mathbf{y}-\mathbf{B}\mathbf{f} \Vert }_2}^2,$$

(10)

where $\mathbf{B}$ is an operator representing the projection operation. Using the TV norm, we can obtain an image with a sparse number of edges (i.e., an image in which the edges are strongly preserved and the pixel values within the region surrounded by the edges are almost uniform).

Figure 8b shows the result of using one of the CT reconstruction algorithms based on such compressed sensing, and it can be seen that the image is free of the artifacts seen in the left panel.

3.3. Without Sample Rotation

The tomogram in Figure 7 was obtained by pulling the sample while rotating it at a high speed of 3000 rpm (50 Hz). To further increase the temporal resolution, the sample needs to be rotated at an even higher speed. However, when the rotation speed was increased by several times, the sample was deformed by centrifugal force, so no CT reconstruction image could be obtained. Rotating the specimen at high speed not only causes the centrifugal force to be large but also imposes various restrictions on such experiments, such as difficulty in controlling the environment of the specimen and inapplicability to fluid specimens. Therefore, the authors aimed to develop a method for realizing synchrotron radiation X-ray CT without the need to rotate samples.

Since it is not possible to rotate the synchrotron radiation source around a sample, the authors developed a multi-beam optical system for millisecond X-ray CT without sample rotation. There are three ways to change the propagation direction of X-rays: reflection, refraction, and diffraction, but the change in propagation direction that can be achieved by the first two methods is on the order of mrad. The authors selected diffraction (Bragg reflection at a single crystal with high perfection), which can change the propagation direction most significantly, due to the need to cover a wide angular range of projection directions in X-ray CT.

However, the resonance width of highly perfect Bragg reflection is very narrow (about 10 $\mathsf{\mu }$rad in terms of the angle of incidence) due to the dynamical diffraction effect of X-rays, which generally requires a very expensive, precise optical system. Therefore, the authors used white synchrotron radiation and aimed to precisely control multiple reflected beams at the same time by precisely curving a thin single-crystal wafer.

Figure 10a shows the principle of the multi-beaming technique we developed to realize X-ray CT without sample rotation. A 100 $\mathsf{\mu }$m-thick Si single crystal was etched into the shape shown in the upper figure of Figure 10a by microfabrication [50], and when it is bent, the small rectangular plates at the top (hereinafter called “blades”) point in different directions in accordance with the orientation of the lower part of the crystal. If the Si wafer is curved along a hyperbola, the reflected beams will converge toward the sample. As mentioned above, the resonance width of Bragg reflection is very narrow, so the energy bandwidth of the reflected X-rays is very small. Therefore, the transmitted X-rays can be reused. Figure 10b shows the hyperbolic-shape multi-beam optical element we developed [31]. The element can generate 32 beams covering a range of $\pm {70}^{\circ }$. Figure 10c is a 3D CT reconstruction image acquired with an exposure time of 1 ms using a 50-$\mathsf{\mu }$m diameter tungsten wire as a sample. The spatial resolution of the tomogram was 65 $\mathsf{\mu }$m. We used the state-of-the-art CT reconstruction algorithm based on compressed sensing described in Section 3.2.

4. Summary

In this paper, we introduced synchrotron radiation X-ray CT with a millisecond temporal resolution, which is far beyond the limit of what can be perceived by the human eye. Temporal resolutions of 2 to 10 ms have been achieved with sample rotation and of 1 ms without sample rotation (with synchrotron radiation multi-beam optics). The spatio-temporal resolutions we achieved recently are summarized in Figure 11, in comparison with those of other representative reports [30,51,52,53,54,55,56,57,58]. The multi-beam optics, together with the multi-beam X-ray image detector [59,60] developed by the authors, enables 4D observation of fluid samples and living animals on the millisecond order, and since various sample environments can be introduced, it is expected to have a ripple effect in a wide range of fields from basic research in materials and life sciences to industrial applications. Its potential applications include observing fracture processes of polymer materials and adhesive interfaces and dynamic biomimetics research conducted on the basis of the 4D observation of live insects. Our techniques can be used to explore the frontiers of the 4D spatio-temporal domain of $\mathsf{\mu }$m and ms orders and is expected to unveil mechanisms of various non-repeatable and non-equilibrium phenomena that have never been uncovered by other techniques.