On Ghost Imaging Studies for Information Optical Imaging

Abstract

Since the birth of information theory, to understand, study, and optimize optical imaging systems from the information–theoretic viewpoint has been an important research subfield of optical imaging, accompanied by a series of corresponding advances. However, since the “direct point-to-point” image information acquisition mode of traditional optical imaging systems, which directly performs one-to-one signal mapping from the object to the detection plane, lacks a “coding–decoding” operation on the image information, related studies based on information theory are more meaningful in the theoretical sense, while almost acting as icing on the cake for the optimization and design of practical systems and contributing little to substantive breakthroughs in further imaging capabilities. With breakthroughs in modern light-field modulation techniques as well as ghost imaging techniques, which establish point-to-point image signal reproduction based on high-order correlation of light fields, currently, it is able to encode the image information with controllable spatiotemporal light-field fluctuations during the ghost imaging process. Combined with modern digital photoelectric detection technologies, ghost imaging systems behave more in line with the modulation–demodulation information transmission mode compared to traditional optical imaging. This puts forward imperative demands and challenges for understanding and optimizing ghost imaging systems from the viewpoint of information theory, as well as bringing more development opportunities for the research field of information optical imaging. This article will briefly review the development of information optical imaging since the birth of information theory, overview its current research status by combining with latest related progresses in ghost imaging, and discuss the potential developing tendency of this research topic.

1. Introduction

“The fundamental problem of communication is that of reproducing at one point either exactly or approximately a message selected at another point”: this is the famous assertion of Claude Shannon, the main founder of information theory, in his paper “A mathematical theory of communication” [1]. Under the guidance of information theory, modern communication systems have been well developed [2], which essentially realize a “point-to-point” reproduction of temporal domain signals. Similarly, an optical imaging system can be understood as performing the “point-to-point” signal reproducing between the object space and the image space. The only difference between them is that communication is accomplished in the temporal domain while imaging focuses more on information in the spatial domain. Therefore, analyzing and optimizing optical imaging systems from the information–theoretic view have aroused wide research interests since the birth of information theory [3,4,5,6,7,8].

Imaging resolution is one of the most important indicators for an imaging system. From the viewpoint of information theory, several studies have been performed to give resolution descriptions of traditional imaging systems [3,9,10,11,12,13,14,15,16,17], including quantitative ones, such as the “information resolution distance” [11,12] based on analyzing the optical information capacity and the “statistical resolution” [16,17] based on the viewpoint of statistical signal estimation, which complement the classical diffraction-limited resolution description and have been applied to analyze the resolution limit in the microscopy system [18,19] and imaging through scattering media [20]. Generally, these studies focus on describing the resolution of image information in the two-dimensional (2-D) spatial domain. However, according to the optical Shannon formula, the information capacity of an optical system actually depends on multiple dimensions of light fields and is described as [3,4]

$$\mathrm{I}=N_{\mathrm{DOF}}{log}_2(1+\mathrm{SNR}),$$

(1)

where $\mathrm{SNR}$ is the signal-to-noise ratio of optical signals, and $N_{\mathrm{DOF}}$ is called the optical degree of freedom (DOF), which consists of multiple-dimensional (temporal, spatial, spectral, and polarization) components [4]

$$N_{\mathrm{DOF}}=N_t·N_s·N_c·N_{\psi }.$$

(2)

This dimension disagreement between resolution analysis and optical information is actually related to the imaging and detection mechanism of traditional imaging systems. Since the “direct point-to-point” signal mapping from the object to the detection plane is performed while current optical detectors are restricted to conduct 2-D planar intensity detection, the information of multiple dimensions (e.g., spectral, polarization, and temporal) of light is lost after the detection (essentially all degenerated into the 2-D spatial dimension). Hence, in those studies on both optical information capacity analysis [5,6,7] and the resolution description of traditional imaging systems, only the 2-D spatial DOF is involved, and the generalized resolution description in a higher-dimensional light-field domain as well as the influence of multiple-dimensional information on the 2-D spatial resolution are hard to analyze for traditional imaging systems.

Additionally, information theory has been used to analyze and design traditional optical imaging systems. Roughly from the information–theoretic perspective, studies in the field of image compression [21,22] have shown that for most images, the trivial image representation is a redundant one [23]. Since typical traditional imaging systems use the “direct point-to-point” imaging mode that directly performs one-to-one signal mapping from the object to the detection plane, corresponding detected signals actually contain considerable redundancy, which is unnecessary in high-SNR detection. On the other hand, for those non-imaging tasks, such as object tracking and classification, though a task-specific information measure was proposed [24], since the image information acquisition mode is largely fixed, the traditional imaging system design according to information theory could only perform the optimization of system parameters [25,26,27,28], rather than accordingly designing different optical transmission functions.

Different from those traditional imaging systems that realize the 2-D spatial domain “direct point-to-point” optical transmission function by using optical elements and are thus diffraction limited by the system aperture, the coherent diffraction imaging (CDI) technique [29] performs “direct point-to-point” information mapping in the Fourier domain by using light propagation without optical elements. Hence, diffraction imaging is rather suitable for imaging with light sources of wavebands where corresponding high-precision imaging elements are hard to manufacture, such as X-rays [30,31], and the resolution can theoretically reach the order of the wavelength. However, in practice, the resolution of this type of imaging mode is empirically proportional to the quarter root of the intensity of coherent radiation sources [32]. Since it is hard to obtain a highly bright coherent radiation source for X-ray and Fermions (such as neutrons and electrons), an extension on its ability of ultra-high resolution in practical applications is greatly limited. In addition, the information transmission mode in CDI is also largely fixed, thus it is hard to be optimized according to information theory to address issues such as weak signal processing in noisy circumstances for a higher resolution.

Through information–theoretic studies and analyses on traditional imaging systems, a series of theories, concepts, and methods has been developed, which promotes the integration of information theory and optical imaging techniques, and sets up the basic framework of information optical imaging.

However, limitations of traditional imaging systems due to the “direct point-to-point” image information acquisition mode are also revealed, including the following:

(1)

Multiple-dimensional light-field information is degenerated into the 2-D domain;

(2)

Resolution analysis is also restricted to the 2-D spatial domain;

(3)

Detection signals usually contain unnecessary redundant image information;

(4)

For specific tasks where the whole image information is not necessary, (over)sampling on the full-resolution image is still performed first;

(5)

For diffraction imaging systems, it is hard to achieve a resolution at the order of the wavelength without highly bright coherent sources since methods for weak signal detection developed in modern information science cannot be applied.

These significantly limit the design and development of further imaging capabilities from the perspective of information theory, and make it difficult to fully exploit the capability of the Shannon channel capacity for transmitting image information in an efficient and accurate way.

In the remaining parts of this review, we elaborate on how ghost imaging (GI) systems can break those limitations of traditional imaging systems from the viewpoint of information theory. In Section 2, the connection between GI and information optical imaging is illustrated by combining viewpoints of both optical coherence theory and information theory. In Section 3, specific GI studies with various extended imaging capabilities are reviewed. Section 4 gives a detailed prospect for future GI studies from our perspective. In Section 5, a brief conclusion is drawn.

2. Optical Imaging from the Viewpoint of Optical Coherence Theory and the Connection between Ghost Imaging and Information Optical Imaging

Generally speaking, the aim of an imaging system is to transmit the image information of imaging objects to realize spatial domain “point-to-point” signal reproduction, and can be mathematically considered as accomplishing the vector transform from the object Hilbert space to the image Hilbert space [33]. In an optical system, the image information is essentially carried by propagating light fields $E(r,t)$, which, according to the optical coherence theory [34], is a random process that can be well described by its statistical properties. Namely, the entirety of the information that it carries is contained in its joint moments of all orders (i.e., optical correlation functions of all orders)

$$\begin{array}{cc}\hfill & G^{\left(M,N\right)}\left(r_1,r_2,\cdots ,r_{M+N};t_1,t_2,\cdots ,t_{M+N}\right)\hfill \\ \hfill =& 〈\begin{array}{c}{E}^{*}\left(r_1,t_1\right)E^{*}\left(r_2,t_2\right)\cdots E^{*}\left(r_M,t_M\right)\hfill \\ \times E\left(r_{M+1},t_{M+1}\right)E\left(r_{M+2},t_{M+2}\right)\cdots E\left(r_{M+N},t_{M+N}\right)\hfill \end{array}〉\hfill \end{array}.$$

(3)

It is worth noting that the optical correlation function has different appellations and notations; for the case $M=N$ involving light fields at $2M$ different spatiotemporal positions, the correlation function $G^{(M,M)}$ is also referred to as being of order M rather than $2M$, and is written as $G^{\left(M\right)}$ instead. Below in this article, the appellation and notation corresponding to $G^{\left(M\right)}$ are applied. Since the coherent transfer function (CTF) of an imaging system relates detected light fields with light fields from the object [35], by properly designing paraxial-approximate CTFs of various imaging systems and combining with the coherence property of light sources, the desired image information can be retrieved with varied accuracy from the correlations of different orders of light fields at the detection plane. From this perspective, traditional imaging systems can be understood as acquiring the desired image information from the first-order correlation of light fields at the detection plane [36]. Specifically, in a typical camera system, a spatial point-like transfer function is designed under the paraxial approximation so that the light intensity (the first-order auto-correlation of light fields) recorded at the detection plane is

$$I_{\mathrm{t}}\left(r\right)=〈E_{\mathrm{t}}^{*}\left(r\right)E_{\mathrm{t}}\left(r\right)〉=h_{\mathrm{I}}\left(r\right)*T_{\mathrm{o}}\left(r\right),$$

(4)

and the point-to-point reproduction of the desired spatial image information $T_{\mathrm{o}}\left(r\right)$ is realized. In the holographic imaging system [37], the hologram generated by interfering the object beam and the reference beam is distributed as

$$\begin{array}{cc}\hfill I\left(r\right)=& 〈E_{\mathrm{r}}^{*}\left(r\right)E_{\mathrm{r}}\left(r\right)〉+〈E_{\mathrm{o}}^{*}\left(r\right)E_{\mathrm{o}}\left(r\right)〉\hfill \\ \hfill & +〈E_{\mathrm{r}}^{*}\left(r\right)E_{\mathrm{o}}\left(r\right)〉+〈E_{\mathrm{o}}^{*}\left(r\right)E_{\mathrm{r}}\left(r\right)〉\hfill \end{array},$$

(5)

and the complex transmittance information of the imaging object $t\left(r^{\prime }\right)$ is essentially contained in the first-order cross-correlation between the light fields of two beams, i.e., the last two terms of Equation (5), where $〈E_{\mathrm{r}}^{*}\left(r\right)E_{\mathrm{o}}\left(r\right)〉\propto \int t\left(r^{\prime }\right)exp\left[\mathrm{j}\frac{k}{2z}{\left(r^{\prime }-r\right)}^2\right]\mathrm{d}{r}^{\prime }$.

The image information can also be extracted from the high-order correlation of light fields [38]. In the famous Hanbury Brown-Twiss (HB-T) intensity interference experiment [39], light from the star is split into two beams and recorded by two separated detectors, and the desired Fourier-domain information of the distant star’s intensity distribution $I_{\mathrm{s}}\left(r_{\mathrm{s}}\right)$ is reproduced by the intensity correlation of the two beams (essentially the second-order auto-correlation of the star’s light fields)

$$\begin{array}{cc}\hfill & 〈E^{*}\left(r_1,t\right)E^{*}\left(r_2,t+\tau \right)E\left(r_1,t\right)E\left(r_2,t+\tau \right)〉\hfill \\ \hfill =& 〈E^{*}\left(r_1,t\right)E\left(r_1,t\right)〉〈E^{*}\left(r_2,t+\tau \right)E\left(r_2,t+\tau \right)〉\hfill \\ \hfill & +{\left|G^{\left(1\right)}\left(r_1,r_2;\tau \right)\right|}^2\hfill \end{array},$$

(6)

where ${\left|G^{\left(1\right)}\left(r_1,r_2;\tau \right)\right|}^2\propto {\left|\mathcal{F}\left\{I_{\mathrm{s}}\left(r_{\mathrm{s}}\right)\right\}\right|}^2$.

On the other hand, GI [40,41,42] (shown as Figure 1) is also a representative high-order correlation imaging technique that has been widely considered and applied. In GI, illumination light with spatiotemporal fluctuations is split into two beams, one of which passes through the object (called the object beam) and then is recorded by a detector as $I_{\mathrm{t}}\left(r_{\mathrm{t}}\right)$, while the other propagates without passing through the unknown object (called the reference beam) and is detected by another detector as $I_{\mathrm{r}}\left(r_{\mathrm{r}}\right)$. For the incoherent thermal source, the point-to-point image information reproduction can be achieved by the second-order intensity correlation between the two detection signals (essentially the second-order cross-correlation) [43]

$$\Delta G^{\left(2\right)}(r_{\mathrm{r}},r_{\mathrm{t}})=\langle \Delta I_{\mathrm{r}}\left(r_{\mathrm{r}}\right)\Delta I_{\mathrm{t}}\left(r_{\mathrm{t}}\right)\rangle \propto {\left|\int d^{2}r{h}_{\mathrm{r}}^{*}(r_{\mathrm{r}};r)h_{\mathrm{t}}(r_{\mathrm{t}};r)\right|}^2,$$

(7)

where $h_{\mathrm{r}}(r_{\mathrm{r}};r),h_{\mathrm{t}}(r_{\mathrm{t}},r)$ are CTFs of light fields in the reference and object paths, respectively (It is worth mentioning that, there exists another imaging scheme similar to GI that encodes the image information with light-field fluctuations, while retrieving the desired information via the first-order cross-correlation. This scheme is mostly used in microwave imaging and commonly referred to as “correlated imaging” [44] or “radar coincidence imaging” [45,46]). Several studies have shown that, for GI with thermal light fields, by designing proper CTFs, various types of point-to-point signal reproduction between the desired object space and the image space can be realized, resulting in different GI systems, such as spatial-domain GI [47] (related to but different from single-pixel imaging [48,49], which constructs mapping between the image space and the object space via first-order light-field correlation), diffraction GI [50], phase contrast GI [51,52] and holographic GI [53].

In practical optical imaging systems, commonly used optical detectors are mostly based on Mandel’s principle of photoelectric detection [54,55], which indicates that the mean number of radiated electrons is proportional to the integrated light intensity during the exposure time. Since the characteristic time of thermal light field fluctuations is usually much smaller than the integration time of the detector, the output signal of optical detectors is actually proportional to the average light intensity (the first-order auto-correlation). Hence, the first-order light-field correlation can be directly detected to immediately realize traditional imaging, while the high-order correlation of light fields cannot be directly measured so that the desired image information in imaging techniques based on high-order correlation is hard to directly record. Hence, in current GI studies, light fields with controllable macro fluctuations, such as pseudo-thermal light fields [56], are used to imitate the true thermal light-field fluctuations, so that the imitated fluctuations can be measured and used to calculate the correlation as

$$\frac{1}{m}\sum _{i=1}^m\Delta I_{\mathrm{r}}^{\left(i\right)}\left(r_{\mathrm{r}}\right){\Delta }_{\mathrm{t}}^{\left(i\right)}\left(r_{\mathrm{t}}\right).$$

(8)

Mathematically speaking, the calculated correlation under finite number of samplings is actually a kind of data mean rather than the ensemble average, and there will thus be significant deviation between them due to the finite number of practical measurements. Therefore, the retrieved image information will deviate from the true one and become degraded. To address this issue, prior information is considered during the image retrieval, and an optimization-type problem is applied [57]. Intuitively, in the case of a bright classical light source, by considering the underlying light-field propagation in the object path, the problem can be roughly expressed as

$$\underset{O}{min}{f}_{\mathrm{t}}\left[I_{\mathrm{t}}-{\left|T\left(E_{\mathrm{s}},O\right)\right|}^2\right]+{\lambda }_{\mathrm{t}}{C}_{\mathrm{t}}\left[O\right].$$

(9)

Here, $f_{\mathrm{t}}(·)$ means the noise effect on propagation and detection, $C_{\mathrm{t}}\left[O\right]$ denotes the prior information expressed as the constraint on the object’s image information O, and $T(E_{\mathrm{s}},O)$ is used to implicitly denote the light propagation in the object path. In GI, the ensemble correlation between two paths will ensure that these two detectors “see” the same “effective light source” $E_{\mathrm{s}}$, thus the unknown $E_{\mathrm{s}}$ in (9) can be obtained in principle by solving the following phase-retrieval-type [58,59,60] problem of the reference path:

$$\underset{E_{\mathrm{s}}}{min}{f}_{\mathrm{r}}\left[I_{\mathrm{r}}-{\left|R\left(E_{\mathrm{s}}\right)\right|}^2\right]+{\lambda }_{\mathrm{r}}{C}_{\mathrm{r}}\left[E_{\mathrm{r}}\right]$$

(10)

such that it is connected with $E_{\mathrm{r}}$ as $E_{\mathrm{s}}=g\left(E_{\mathrm{r}}\right)$, where $R(·)$ denotes the effective light propagation on the reference path, $f_{\mathrm{r}}(·)$ means the noise effect, and $C_{\mathrm{r}}\left[E_{\mathrm{r}}\right]$ is the constraint imposed on reference light fields $E_{\mathrm{r}}$. Substituting that into Equation (9), the optimization-type problem can be written as [61]

$$\underset{O}{min}{f}_{\mathrm{t}}\left[I_{\mathrm{t}}-{\left|TR\left(\sqrt{I_{\mathrm{r}}}{e}^{\mathrm{j}{\varphi }_{\mathrm{r}}},O\right)\right|}^2\right]+{\lambda }_{\mathrm{t}}{C}_{\mathrm{t}}\left[O\right],$$

(11)

where $TR(·)$ denotes the implicit relationship between the reference path and the object path, and $\left\{{\varphi }_{\mathrm{r}}\right\}$ denotes unknown phases of light fields at the reference detection plane. Here, since only the “intensity” correlation between $I_{\mathrm{t}}$ and $I_{\mathrm{r}}$ are necessary to retrieve the image information of the unknown object in GI, it is intuitively believed that, under the constraint of setup parameters in a GI system, Equation (11) should be insensitive to the accuracy of those light phases $\left\{{\varphi }_{\mathrm{r}}\right\}$ (this has been empirically and theoretically verified in spatial-domain GI and diffraction GI [61,62,63], but has not been strictly proved for all kinds of GI systems). Furthermore, since an imaging system is linear, Equation (11) can be written as a first-order linear detection model as $y=Ax+n$, where methods in information theory can be directly applied.

Considering that light propagation is reversible, from Equation (11), lights illuminating the object can be imagined to be emitted from the detection plane of the reference path [64] (shown as the imagined light path in Figure 1), thus the illuminated light-field pattern can be flexibly “effectively modulated” by non-locally designing the CTF of the reference path [65] to encode the image information of the object. From this perspective, GI systems are able to behave more in line with the communication system than traditional imaging systems, and are thus expected to overcome those limitations that traditional imaging systems have.

3. Ghost Imaging Studies for Information Optical Imaging

3.1. Mapping Higher-Dimensional Light-Field Information into Lower-Dimensional Domain

As demonstrated in (11), GI can modulate the object’s image information with controllable light fields and map it into lower detectable dimensions through encoding, thus making it possible to realize direct imaging in the high-dimensional light-field domain and enable further-developed imaging capabilities. Typical examples include the GI LiDAR and the GI camera.

The principle of GI LiDAR is shown in Figure 2. It was first made public in 2011 [66,67] and demonstrated to perform three-dimensional (3-D) imaging of natural scenes [68]. GI LiDAR uses light fields with designed spatiotemporal fluctuations, which are generated by modulating a pulse laser with a diffuser to illuminate the object, then acquires the return signal with a bucket detector and retrieves the 3-D spatial-depth image information by the second-order correlation between illuminated light fields and return signals as

$$〈\Delta I_{\mathrm{t}}\left(t_{\mathrm{d}}\right)\Delta I_{\mathrm{r}}\left(r_{\mathrm{r}}\right)〉\propto T\left(r_{\mathrm{r}},d\left(t_{\mathrm{d}}\right)\right),$$

(12)

where $t_{\mathrm{d}}$ is the relative time delay of the detection signal. During the detection process, the encoding is performed via the designed light fields to transform the 3-D information into a time-serial signal which is recorded by the bucket detector. Compared with traditional LiDAR systems which only perform encoding on the time–frequency dimension, GI LiDAR essentially extends the encoding domain to higher dimensions including both spatial and temporal domains. Hence, different from traditional LiDAR, which can only measure the distance and velocity, GI LiDAR can simultaneously perform the imaging of scenes of different distances as well as obtaining their depth and velocity information [69], which belong to higher dimensions.

For GI LiDAR with incoherent bucket detection, the phase information (which may include the vibration information) of the target is lost when recording. On the contrary, in those GI variants in microwave bands (e.g., the correlated imaging), the micro-vibration information of the target can be obtained when performing imaging. This is mainly owing to the utilization of the coherent detection method for signal recording since it is the first-order field correlation that is involved there [44,45,46]. Recently, with the development of coherent optical communication [70] and fiber optical time–frequency transmission [71], coherent detection methods for optical-waveband signals have been largely developed. Thus, they have also been introduced into GI LiDAR at optical wavebands, leading to coherent detection GI LiDAR [72,73,74] which has the ability to acquire the information in a higher dimension (e.g., the micro-Doppler vibration information) and enable a stronger anti-interference capability. In this scheme, a long-pulse laser is modulated in the spatial–temporal/spatial–frequency domains to illuminate the object, and the return signal is detected via a typical coherent detection approach. Then, by combining the second-order correlation with time–frequency analysis techniques, 3-D spatial-depth image information as well as the spatial distribution information of velocity and vibration of the target can be retrieved together.

Different from GI LiDAR with active illumination, the GI camera [75] is a passive (using natural lights only) GI scheme inspired by near-field diffraction speckle phenomena [76,77], proposed via utilizing the ergodicity of thermal light fields to extract image information from the second-order correlation in the spatial domain rather than the original temporal domain. The principle of a representative GI spectral camera is shown in Figure 3. The object illuminated by natural lights is firstly imaged by a front imaging module, then diffracted by a spatial random phase modulator (SRPM) and finally recorded by a detector as a pattern of intensity distribution $I_{\mathrm{t}}\left(r_{\mathrm{t}}\right)$. By the spatial–dimensional second-order correlation between this intensity distribution and the pre-measured impulse response $I_{\mathrm{r}}(r_{\mathrm{t}};r,k)$ corresponding to a monochromatic point source $\delta (r,k)$ where $k=1/\lambda$, the object’s spatial–spectral information $T(r,k)$ can be achieved, namely [75]

$$\Delta G^{\left(2\right)}(r,k)={〈\Delta I_{\mathrm{t}}\left(r_{\mathrm{t}}\right)\Delta I_{\mathrm{r}}\left(r_{\mathrm{t}};r,k\right)〉}_{r_{\mathrm{t}}}\propto T\left(r,k\right)\ast g^{\left(2\right)}\left(r,k\right)$$

(13)

where $g^{\left(2\right)}\left(r,k\right)$ is the normalized second-order correlation function that characterizes the resolving ability of high-dimensional light-field information. In addition, the GI camera that enables to realize a higher-dimensional spectral-polarization imaging is also demonstrated [78].

Unlike the typical traditional camera that only maps the object’s 2-D spatial information into the detection plane by a “direct point-to-point” mode, in the GI camera, the desired information in the high-dimensional light-field domain is encoded with randomly distributed speckle patterns and mapped into the 2-D detection plane. Hence, the GI camera is able to transmit high-dimensional image information with less degeneracy, namely, it can much more efficiently make use of the imaging channel capacity compared to the traditional camera. For example, besides the spectral imaging, the spectral-polarization GI camera has also been demonstrated [78]. Further, to improve the efficiency, the GI camera with the super-Rayleigh modulator has been proposed by designing the distribution of those encoded speckle patterns [79], which exhibits a better anti-noise imaging performance compared to the vanilla one. Additionally, the issue of extending the super-Rayleigh modulator to a broad spectral band for hyperspectral GI camera has been considered, and some progress has been made by combining with dispersion control techniques [80].

3.2. Resolution Analysis in the High-Dimensional Light-Field Domain

For the GI system, since it enables to map information of higher-dimensional light-field domains into lower dimensions and to achieve imaging of the high-dimensional image information via the second-order correlation, the description of its resolution should be reconsidered from a different perspective. Intuitively, it is the resolution in the high-dimensional light field domain rather than the 2-D spatial domain that would be limited by the optical diffraction effect. In this case, some analyses on the resolution of GI from the viewpoint of information theory have been conducted. For example, taking the GI camera as the research object, Tong et al. [63,81] proposed a criterion to measure its discernibility on high-dimensional light-field image information by combining with the compressed sensing (CS) theory [82,83]. Specifically, the relationship between the normalized second-order correlation in the high-dimensional light-field domain $g^{\left(2\right)}\left(\left|{\tau }_i-{\tau }_j\right|\right)$ ($\tau$ denotes a high-dimensional coordinate that could contain spatial, spectral and polarization dimensions) and the mutual coherence $\mu$ of system’s transmission matrix is first constructed as

$$\mu =\underset{i\ne j}{max}\left|g^{\left(2\right)}\left(\left|{\tau }_i-{\tau }_j\right|\right)\right|,$$

(14)

and the condition for resolving sparse point-like targets is given as the minimal distance of two points that establishes

$$g^{\left(2\right)}\left(\left|{\tau }_i-{\tau }_j\right|\right)<\frac{1}{2K-1}$$

(15)

by combining with the exact recovery condition [84], where K is the sparsity level of those point-like targets. Through this description, it can be roughly seen that multiple dimensions of light fields do interact with each other when considering the resolving ability. Since the resolution is fundamentally limited by the system’s transmission function which transfers the high-dimensional image information, it is possible to greatly increase the spatial resolving ability beyond the Rayleigh criterion by utilizing discrepancies of the imaging object in other dimensions [81]. Similar to the typical deconvolution operation [85,86] for traditional imaging that exploits the prior information of the system’s transmission function, it is possible to achieve spatial super-resolution imaging in GI. By performing regularized preconditioning operations [87,88,89] on both the detection signal as well as the transmission matrix and solving the preconditioned problem via optimization algorithms, it is experimentally demonstrated that a high-quality imaging result with resolution beyond the diffraction limit can be obtained, and further theoretical analysis from the perspective of Fourier frequency domain also verifies its capability [90].

3.3. Optimizing the Encoding Mode to Reduce Unnecessary Sampling Redundancy

The issue of unnecessary sampling redundancy of the traditional imaging mode is hard to address due to its “direct point-to-point” imaging architecture. In the GI system, however, since the encoding mode can be varied more flexibly during the practical imaging process, unnecessary redundancy in the detection signal can be largely reduced by designing light fields based on the information theory. From the theoretical perspective, several studies in this direction have been performed. In 2013, Li et al. [91] firstly tried to theoretically analyze the mutual information between the detection signal and the imaging object given the determined light fields in GI, and on this basis, gave the optimal parameter of Bernoulli light fields that maximizes the mutual information to perform the optimization. However, this research was restricted to designing light fields subject to some specific distributions. Besides ideas inspired by Shannon information theory, by combining the CS theory, light field optimization scheme via minimizing the mutual coherence between the sampling matrix (consisting of light field patterns) and the orthogonal sparsity basis was proposed [92] according to the incoherent sampling principle [93]. The effect of this scheme on improving the imaging quality was also demonstrated by practical experiments [92]. To further incorporate image statistics into the optimization, an optimization framework combined with dictionary learning [94,95] was proposed [96]. In this framework, an over-complete sparsity basis that describes the statistics of imaging objects was given via dictionary learning, and light fields were optimized by similarly solving a mutual coherence minimization problem through a concise method. A closed-form solution of the optimal light fields was theoretically derived and experimentally applied to further enhance the imaging quality. Briefly speaking, these studies have shown that optimizing light fields with the help of information theory can essentially increase image information transmission efficiency. In addition, some heuristic encoding mode optimization studies have been proposed, which adaptively adjust the encoding mode according to precedent-acquired information [97,98,99,100] to reduce the redundancy contained in subsequent samplings.

On the other hand, the information content that the GI system can transmit is a more fundamental problem which could inspire the encoding optimization. As aforementioned, mutual information between the detection signal and the imaging object given a determined type of light fields has been analyzed [91] and applied to light-field optimization. Besides the mutual information, Fisher information [101] has also been used to analyze the information that detection signals contain about the imaging object in the GI system [102]. In this study, it was shown that signals with larger fluctuations contain more information, which is consistent with several existing studies [103,104,105,106] and provides a potential idea for the encoding mode optimization.

Generally, by utilizing fundamental information–theoretic studies, it is possible to provide a solid foundation for those research topics of the GI system such as encoding light fields optimization, resolution analysis, and performance investigation. Hence, there would be significant demands on this research direction considering that existing studies on the information content analysis of the GI system are still rather limited.

3.4. Task-Oriented Gi System Design

In many practical scenarios, a high-quality image is not necessarily required; instead, it is sufficient to only have the information related to specific tasks [24,107]. Intuitively, this can be realized by designing the imaging system with the help of information theory. Traditional imaging systems, however, usually need to first perform high-quality imaging and then perform a specific analysis based on a sensing–storing–computing–integrated equipment due to their fixed image information transmission mode. Since the GI system can perform more flexible encoding than traditional imaging systems, it is rather suitable for such a task-oriented kind of application scenario so that the desired information can be acquired and retrieved without the need for performing high-quality imaging. Currently, several task-oriented GI studies have been performed, mainly realized by designing the encoding light-field patterns according to the desired task information and assisting with data-processing methods. They can be classified into several categories, including non-imaging object detection [108], non-imaging object classification [109,110], non-imaging object edge detection [111,112], and object tracking [113] as well as progressive imaging [114]. In brief, these studies largely utilize the flexibility of the information mapping mode of GI systems. However, they are mostly come up with from a heuristic perspective rather than a more theoretically solid one. Thus, for further task-oriented studies, GI systems may be combined with the task-specific information measure [24] to develop a more complete information–theoretic framework for designing systems to perform a specific task. Task-oriented GI systems should significantly reduce the excessive demand of the transmission and storage of redundant information, thereby saving much of the energy consumption of devices and leading to a more “green” task information acquisition mode for many application scenarios, such as automatic driving and the industrial internet of things.

3.5. X-ray Diffraction Gi

Different from GI LiDAR and the camera that performs imaging in the spatial domain, the diffraction GI [50,65] (shown as Figure 4), by designing the CTF, instead extracts image information of Fourier diffraction spectra via the second-order correlation

$$\Delta G^{\left(2\right)}(r_{\mathrm{r}},r_{\mathrm{t}})=〈\Delta I_{\mathrm{t}}\left(r_{\mathrm{t}}\right)\Delta I_{\mathrm{r}}\left(r_{\mathrm{r}}\right)〉\propto {\left.\mathcal{F}\left\{t\right\}\right|}_{\mathbf{f}=\frac{r_{\mathrm{t}}-r_{\mathrm{r}}}{\lambda z_2}},$$

(16)

thus enabling to analyze a non-crystalline object with an incoherent thermal source in principle. Hence, it has the potential to address the imperative demand of CDI on a high–bright coherent radiation source at the waveband of X-rays and Fermions. The first X-ray diffraction GI experiment was realized in 2016 [115]. Since the X-ray diffraction GI can acquire the Fourier spectra information at the Fresnel region rather than the distant Fraunhofer region with a thermal X-ray source, it provides the potential to realize miniaturized thermal X-ray microscopic equipment.

From the perspective of information theory, the diffraction GI essentially performs encoding on information in the Fourier domain, and this can be understood more clearly when performing analysis on the angular spectra [116]. By further combining with Equation (11), it can be found that more flexible encoding can be performed on the reference path [65] without increasing the radiation dose incident on the sample according to information theory, compared to the fixed-architecture traditional CDI scheme. For example, a non-local encoding method (adding masks at the reference path) for the X-ray diffraction GI scheme was experimentally demonstrated [117], which can significantly enhance the imaging quality without increasing the radiation dose on the sample. To achieve a high-quality decoding, a deep-learning-based Y-net has been proposed [118], whose structure is inspired by the form of Equations (9) and (10). Additionally, the imaging performance analysis and system optimization of the diffraction GI system have been performed with information–theoretic studies [63,102].

4. Outlook

The capability of GI systems on breaking those limitations of traditional imaging systems from the viewpoint of information theory has been well demonstrated. Nevertheless, there are still many studies needing to be performed on those typical topics to further develop this research field.

4.1. Further Utilizing High-Dimensional Light-Field Information Capacity

Currently, the imaging equipment based on GI LiDAR has demonstrated the ability to acquire additional vibration information during imaging by combining with the coherent detection method and the micro-Doppler effect [73], making it possible to additionally “see the sound” via an imaging system. Though this scheme is naturally suitable to detect and image scenes with vibration modes with discrete frequency spectra, it is still a challenging problem for it to perform imaging on scenes in transparent media that have vibration modes with 3-D vibrating directions or scenes with continuous spatial–temporal frequency spectra since these kinds of vibration mode and wave propagation essentially have a much more complicated structure [119,120].

For the GI camera, our short-term research focus will be theoretical and application studies on its ability for 2-D spatial super-resolution imaging achieved by exploiting the discrepancies of the imaging object in the high-dimensional light-field domain. Particularly, in the theoretical aspect, the first step is to further improve the quantitative description of the super-resolution ability on imaging objects which are apparently sparse in the high-dimensional light-field domain under the noisy case. To achieve this, one of potential approaches would be to incorporate the concept of statistical resolution [17,18] into the high-dimensional domain, while combining with the object’s prior information [121,122,123]. This statistical-based description is expected to be rather meaningful for the application of a GI camera in related scenarios, such as fluorescence microscopy [124] and astronomy observing [125,126]. Further, for spatial-domain continuous, non-sparse scenes commonly existing in the remote-sensing scenario, to quantitatively describe the super-resolution ability in the 2-D spatial domain resulting from discrepancies in the high-dimensional light-field domain is a very important problem that remains unresolved.

4.2. Task-Oriented System

Since GI systems can realize imaging directly in the high-dimensional light-field domain and have a higher image information acquisition efficiency, they are capable of demonstrating superiority in imaging applications for those time-varying dynamic scenes [127,128]. In such scenarios, possible tasks to be performed can be roughly divided into object tracking, object classification, and complete imaging [129,130]. Intuitively, sufficient information contents for those three kinds of tasks are varied, and actually ascending. As aforementioned, there have already been several studies on GI systems that aim at tasks of object classification and imaging. Then, in the next stage, one of the research topics worth paying attention to for task-oriented GI systems should be theories and methods to realize the ability of tracking tasks, especially the tracking-before-classification scheme [131,132,133] based on the GI framework.

On the other hand, in the task-oriented imaging scheme, there are commonly considerable demands on the confidence and uncertainty of the retrieved task information [134,135,136]. Considering that GI is an imaging mode that utilizes light-field spatiotemporal fluctuations to encode the image information so that various kinds of tasks can be performed accordingly, for this kind of imaging mode that has a “Codec” operation different from the traditional “direct point-to-point” imaging mode, estimation on the confidence and uncertainty of the retrieved information is particularly important for its practical applications, remaining unaddressed. Hence, to give quantitative descriptions of these based on methods in statistical theory and information theory, such as the bootstrapping method [137,138,139], Fisher information [101,136], and Bayesian Cramér–Rao bound [63,140,141], will also be one of the next expected areas of research focus.

4.3. X-ray Diffraction Gi

On the basis of the already-realized X-ray diffraction GI experiment, in the near future, the main goal of X-ray diffraction GI will be to demonstrate an imaging resolution up to the order of 10 nm on the table-top-X-ray-device-based experimental facility. This is expected to be realized by fully exploiting the ability of the non-local encoding scheme on greatly improving the imaging quality without increasing the radiation exposure of the sample. Next, it is expected that performing exploratory studies will challenge the possibility of realizing a higher, near-atom-scale resolution by conducting experiments on the X-ray free-electron laser system. These will put forward further demands on both the non-local encoding mode design as well as recovery algorithms. Furthermore, for diffraction GI, it would be rather fascinating to realize diffraction imaging with thermal Fermion sources.

Though compressed sensing [142,143] and deep learning [144,145,146,147] have both been applied to retrieve the image information in GI, it should still be an important research theme to adopt and develop befitting signal processing methods which can not only more efficiently utilize the detected signal and the prior information about the imaging scene by taking the advantage of GI, but also be more naturally compatible with task-oriented imaging mode as well as the quantum detection theory. The retrieval methods currently used in GI (including correlation-based, compressed sensing and deep learning algorithms) almost all perform information retrieval after obtaining the sampling data. However, this processing mode is not in line with the task-oriented imaging mode, since in many task-oriented scenes, it is required to judge in real time whether the already measured information is sufficient for the task or more samplings are needed. The confidence and uncertainty of the task information retrieved from these methods can hardly be immediately determined from a theoretical viewpoint. Nevertheless, for imaging systems based on light-field correlation (coincidence imaging, GI, etc.) that perform encoding on the image information via light-field fluctuations, it is possible to formulate the image information acquisition procedure into a (stochastic) dynamical system by arranging the measured encoded signals in a certain way. Then, Bayesian estimation methods for dynamical systems could be applied to obtain the probability distribution function (PDF) of the retrieved image information, resulting in a complete description about the retrieval accuracy of the imaging information [148,149,150]. By doing this, one of the advantages is that it is a recursive procedure, namely, the estimation of the image information as well as its uncertainty can be updated immediately by incorporating newly measured signals [151,152,153], and hence the sampling can be terminated once the desired information precision is achieved. This is more suitable for the task-oriented imaging mode compared with those “measurement-before-processing” modes, where oversampling or undersampling commonly exists. Additionally, since the PDF of the image information can be obtained, more complete information–theoretical description and analysis could be further performed. In addition, as a statistical method, it is also convenient to be combined with the quantum detection theory.

As a matter of fact, in those very early studies of GI, it was believed that GI is a pure quantum effect [154,155], until the demonstration of GI experiments with classical light sources [156,157]. Currently, existing studies in GI focus more on systems with classical sources since they have more general application prospects and can thus be further targeted at various specific imaging demands. Nonetheless, quantum GI techniques exploiting non-classical sources as well as the quantum description of the interaction between lights and objects may bring a wonderful sight that has never been seen before [158]. Specifically, GI using quantum sources has been demonstrated to have superiority in the case of a weak light to enable a higher SNR [157,159]. In current GI studies with classical source, the interaction between incident lights and objects is commonly modeled via a transmittance/reflectance function T. This description, however, may not be so accurate when performing microscopic imaging, especially the super-resolution imaging, which has been verified in related experiments [160,161]. In this case, a more detailed model may be obtained by considering the quantum description of the interaction between lights and objects. Therefore, uses of both non-classical sources and the quantum description are expected to largely develop the application scenarios of GI.

5. Conclusions

In summary, from the viewpoint of information transmission, imaging essentially performs the “point-to-point” signal reproduction of image information between the object space and the image space. From existing studies in the field of information optical imaging, it is clear that the “direct point-to-point” mode of traditional imaging systems has principle limitations in several aspects. Specifically, these aspects include fully utilizing the imaging channel capacity in order to obtain higher-dimensional image information with lower-dimensional detectors, increasing the 2-D spatial resolution by exploiting the discrepancy information of the high-dimensional light-field domain, performing compressed sensing without detecting unnecessary redundancy, and flexibly designing imaging systems according to specific tasks, as well as achieving diffraction imaging with resolution at the order of light wavelengths in the case of lacking highly bright coherent radiation sources. In contrast, GI studies for information optical imaging provide a different technical approach to break through these limitations, as well as promoting a deep integration between various kinds of imaging schemes with classical/non-classical sources and the modern information theory under the framework of optical coherence theory, and thus show a very broad developing prospect in the field of optical imaging.