Mid-Wave Infrared Snapshot Compressive Spectral Imager with Deep Infrared Denoising Prior

Abstract

Although various infrared imaging spectrometers have been studied, most of them are developed under the Nyquist sampling theorem, which severely burdens 3D data acquisition, storage, transmission, and processing, in terms of both hardware and software. Recently, computational imaging, which avoids direct imaging, has been investigated for its potential in the visible field. However, it has been rarely studied in the infrared domain, as it suffers from inconsistency in spectral response and reconstruction. To address this, we propose a novel mid-wave infrared snapshot compressive spectral imager (MWIR-SCSI). This design scheme provides a high degree of randomness in the measurement projection, which is more conducive to the reconstruction of image information and makes spectral correction implementable. Furthermore, leveraging the explainability of model-based algorithms and the high efficiency of deep learning algorithms, we designed a deep infrared denoising prior plug-in for the optimization algorithm to perform in terms of both imaging quality and reconstruction speed. The system calibration obtains 111 real coded masks, filling the gap between theory and practice. Experimental results on simulation datasets and real infrared scenarios prove the efficacy of the designed deep infrared denoising prior plug-in and the proposed acquisition architecture that acquires mid-infrared spectral images of 640 pixels × 512 pixels × 111 spectral channels at an acquisition frame rate of 50 fps.

1. Introduction

The mid-wavelength infrared (MWIR) region spanning 3.0–5.0 $\mathsf{\mu }$m contains the characteristic vibrational absorption bands of most molecules as well as the atmospheric transmission window and is, therefore, of critical importance in various applications. Meanwhile, spectral imaging is a technique used to acquire narrow-band spatial and spectral signatures from the scene, making identification and quantification easier. In recent years, the unification between them, the mid-wave infrared spectral imager, is a promising tool that displays unique and robust mid-infrared spectral fingerprints, which can be widely applied in remote sensing [1], planetary exploration [2], medical diagnosis [3,4], and industrial emissions monitoring [5].

Various techniques have been developed for MWIR spectral imagers, such as narrow-band filters [6,7], Fourier transform IR spectroscopy [8,9], acousto-optical tuning [10,11], and frequency up-conversion [4,12,13]. Nevertheless, these Nyquist-sampling-based methods acquire spectral images at a certain given sampling rate in spatial dimension $M\times N$ or spectral dimension L. This process requires the scanning of a large number of data cube slices that grows linearly in proportion to the desired spatio-spectral resolution $M\times N\times L$, which is inadequate for dynamic scenes and causes a heavy computing burden for storage, transmission, and processing. Additionally, the complexity and high cost associated with Nyquist sampling are prohibitive for various applications. To alleviate these issues, compressive sensing (CS) [14] provides a new approach for new data acquisition schemes in the imaging field. This theorem enables the reconstruction of sparse or compressible images from measurements requiring fewer samples than is needed for Nyquist sampling.

Most signals can be compressed during acquisition because they are sparse or can be represented sparsely [14]. At the same time, the sensing phase recovers the original signal from a small number of compressed signals. On this basis, compressive spectral imaging compresses the three-dimension ($x,y,\lambda$) data cube to a measurement on the low-dimension detector, and the reconstruction algorithms recover the three-dimension (3D) data cube from the measurement. In the compressed process, different wavelengths in the 3D data cube are modulated by different coded masks, and the coded signals are then integrated into the detector. Coded aperture snapshot spectral imaging (CASSI) [15] is based on the pioneering work that used a static coded aperture and two dispersers to implement compressive spectral imaging, known as DD-CASSI. Following this, single-disperser CASSI was invented [16] to achieve modulation with a single disperser. To supplement the coded information for reconstruction, multi-frame CASSI [17] (MS-CASSI) was proposed to use many different coded apertures to modulate. In addition, by replacing the traditional coded aperture with a colored one, Arce et al. [18] proposed colored coded aperture compressive spectral imaging (CC-CASSI) to extend the compressive capabilities. Lin et al. [19] designed spatial–spectral encoded compressive spectral imaging (SS-CASSI) to provide a higher degree of randomness in the measurement. To leverage the side information for facilitating reconstruction, dual-camera compressive spectral imaging (DCCHI) has been intensively studied [20,21,22,23]. Recently, motivated by the spectral variant responses of different media, more types of spectral imaging systems have also been explored [24,25,26].

Compared to mature compressive spectral imaging at visible wavelengths, there are a few studies on imaging at infrared wavelengths. Mahalanobis et al. [27] used the CS structure of mid-wave infrared wavelengths to prove that measurements obtained by a small infrared focal plane array (IRFPA), which can recover high spatial resolution information. Zhang et al. [28], to improve the reconstruction strategy, used an imaging calibration method coupled with a parallel computing accelerated reconstruction algorithm to speed up mid-wave infrared imaging. Wu et al. [29] researched system patterns for mid-wave infrared image super-resolution by IRFPA compressive sensing, described the generation process of the digital micro-mirrors device (DMD), and modified the block-based reconstruction algorithm. Further, Wu et al. [30] proposed calibration-based non-uniformity correction and the stray light correction method for mid-wave infrared imaging to improve image sharpness and reduce the blocky effect in the recovered images.

CS has paved the way for infrared imaging using a low-resolution detector that benefits from replacing an expensive IRFPA [31,32,33,34]. The spectral and spatial resolution is achieved using a fully integrated spectrometer and a DMD, respectively. However, to 3D reconstruct spectral images $M\times N\times L$, the scene needs to be sequentially encoded $\eta$ times ($\eta$ refers to the sampling rate) by DMD with size $M\times N$, which severely prolongs the data acquisition time. Instead of sequential acquisition, snapshot computational spectral imaging utilizes a coded aperture to implement static encoding, and a 2D detector is used to capture compressed data, which meets the demands of compressive sensing reconstruction [35]. In this way, Sullenberger et al. [36] proposed a long-wave infrared computational re-configurable imaging spectrometer, which adopts a dual-disperser re-imaging design with a static coded aperture. It realizes the diversity of coding via the device motion or IRFPA scanning. Nevertheless, this method depends on multiple exposures to capture enough measurement sequences for completely recovering spectral images, which hinders instantaneous imaging. Furthermore, spectral calibration is relatively complicated due to the re-imaging design of the system structure.

To address the above issues, a novel mid-wave infrared snapshot compressive spectral imager is proposed that utilizes an infrared static coded mask and a dispersion element to modulate the spatial and spectral information of scenes. A 2D, multiplexed projection of the 3D data cube representing the signal is captured by an IRFPA. Instead of coding the spectrum in a spatially uniform manner, e.g., CASSI [16], which places fundamental limits on the performance of the compressive reconstruction algorithms, a spatio-spectral encoded optical design is able to increase the degree of randomness to preserve more encoded information. Moreover, from a functional perspective, the snapshot structure reduces the difficulty of calibration and improves the acquisition speed with a single-shot capture procedure.

For our system, the infrared spectral images can be reconstructed from two-dimensional measurements by solving an $l_0$, $l_1$, and $l_p$ (0 < p < 1) relaxation optimization problem using prior knowledge [28,37,38]. However, these optimization-based algorithms have a low reconstruction speed and limited performance. Recently, deep learning approaches exhibit promising potential in resolving the reconstruction problem of visible snapshot compressive imaging for a rapid and raised effect [35,39,40,41,42,43,44,45,46]. However, these system-specific networks are hard to apply directly to our task due to the lack of infrared hyperspectral datasets for network training. Inspired by the plug-and-play (PnP) framework proposed for inverse problems [47], we designed an infrared denoising network as a prior for infrared spectral image reconstruction, combining the advantages of optimization-based and deep network algorithms. That is, an ingenious, quick, and high-accuracy algorithm is exploited to solve the reconstruction problem of our mid-wave infrared snapshot compressive spectral imager. In conclusion, our major contributions are as follows:

1. A novel mid-wave infrared snapshot compressive spectral imager that utilizes an infrared static coded mask and a dispersion element to modulate the spatial and spectral dimensions of scenes is presented. The spatio-spectral encoded optical design increases the degree of randomness to preserve more encoded information. Furthermore, the snapshot structure can reduce the difficulty of calibration and improve the acquisition speed with a single-shot capture procedure.

2. To integrate the best of traditional optimization-based and deep learning algorithms, an infrared denoising network as a prior for infrared spectral image reconstruction is designed to resolve the inverse problem of our mid-wave infrared snapshot compressive spectral imager.

3. Experimental results on both a simulation dataset and real infrared scenarios demonstrate the efficacy of the designed deep infrared denoising prior and the proposed acquisition architecture that acquires mid-infrared spectral images of 640 pixels × 512 pixels × 111 spectral channels at an acquisition frame rate of 50 fps.

2. A Compressive Spectral Imaging Model

The basic idea of compressive spectral imaging is to encode a 3D spatio-spectral data cube to a 2D detector forming a snapshot 2D measurement, as shown in Figure 1. The spectral scene is spatially encoded by the coded mask. The prism spectrally disperses the encoded scene. At last, a detector captures the spatial-spectral encoding scene.

For the spectral data cube of the scene $X\in {\mathbb{R}}^{n_x\times n_y\times B}$, where $n_x$, $n_y$, and B denote the width, height, and the number of spectral bands, respectively, and is compressed by B coded masks $A\in {\mathbb{R}}^{n_x\times n_y\times B}$ into the measurements $Y\in {\mathbb{R}}^{n_x\times n_y}$. We can mathematically form this process as

$$Y=\sum _{b=1}^B{A}_b\odot X_b+M$$

(1)

where b denotes the b-th coded mask and the corresponding spectral image. $M\in {\mathbb{R}}^{n_x\times n_y}$ denotes the measurement noise. ⊙ denotes the element-wise product. By vectorizing $y=vec\left(Y\right)\in {\mathbb{R}}^{n_x{n}_y}$, $x=vec\left(X\right)\in {\mathbb{R}}^{n_x{n}_{y}B}$, and $m=vec\left(M\right)\in {\mathbb{R}}^{n_x{n}_y}$, Equation (1) can be written in matrix-vector form as

$$y=Ax+m$$

(2)

After obtaining the measurement y, the spectral images can be recovered from reconstruction algorithms.

3. Optical Sensing and Reconstruction of the Proposed Architecture

A conceptual diagram of the MWIR-SCSI optical setup is presented in Figure 2. The scene’s infrared radiation is collected by an objective lens and then dispersed into the spectral plane by the prism. After a relay lens, a coded mask is mounted in front of the image plane, which allows the mask to modulate the spectral scene in both spatial and spectral dimensions. The encoded spectral image is focused onto the 2D IRFPA by the last relay lens. The captured 2D radiance field sampled by our system includes light multiplexed from the spatio-spectral encoded of the entire scene. The mathematical model of the sensing process is represented as follows:

The scene information of a 3D spectral clip in discrete form is denoted by Z$\left(i,j,\lambda \right)$, where $1\le i\le W$ and $1\le j\le H$ are the spatial coordinates, and $1\le \lambda \le \mathsf{\Omega }$ denotes the spectral channels. Therefore, we can write the IRFPA measurement obtained during the entire interval as follows:

$$g(i,j)=\sum _{\lambda =1}^{\mathsf{\Omega }}w\left(\lambda \right)O\left(i,j-\sigma \left(\lambda \right)\right)z\left(i,j-\sigma \left(\lambda \right),\lambda \right)$$

(3)

where $O(i,j)$ is the transfer function of the coded mask, $w\left(\lambda \right)$ is the point spread function (PSF) achieved by the optical system, and $\sigma \left(\lambda \right)$ denotes the dispersion function of the dispersion element. We can rewrite the formula of Equation (3), as follows:

$$G=\mathsf{\Phi }Z$$

(4)

where $G\in {\mathbb{R}}^{WH\times 1}$ is a representation of g by vectorization, and $Z={\left(z_1,z_2,\dots ,z_{\lambda }\right)}^{\mathsf{T}}\in {\mathbb{R}}^{WH\mathsf{\Omega }\times 1}$ denotes the vectorized representation of the original spectral images. Since the spectral images are dispersed to different positions on the coded aperture, the spectral images are encoded by different sensing matrices ${\phi }_k,k=1,\dots ,\lambda$. Thus, we let $\mathsf{\Phi }\in {\mathbb{R}}^{WH\times WH\mathsf{\Omega }}$ denote the sensing matrices of the coded spectral images, and it can be expressed as

$$\mathsf{\Phi }=\left[Diag\right({\phi }_1),Diag({\phi }_2),\dots ,Diag({\phi }_{\lambda }\left)\right]$$

(5)

where $Diag(·)$ is a diagonal matrix, and the vector ${\phi }_k$ forms diagonal elements. The entire sensing process is depicted in Figure 3.

Thus, the spectral images reconstruction is formulated as a recovery problem of a fitting data term with an additional regularization term:

$$\tilde{Z}=\underset{Z}{argmin}{∥G-\mathsf{\Phi }Z∥}_2^2+\rho \mathcal{K}\left(Z\right)$$

(6)

where $\rho$ denotes the Lagrange parameter that balances sparsity and the regularization term $\mathcal{K}\left(Z\right)$ during optimization. We further solve Equation (6) using the PnP algorithm with a deep infrared denoising prior, described in the next section.

4. A Deep Infrared Denoising Prior for Hyperspectral Image Reconstruction

In the optimization-based methods, the GAP-TV algorithm has shown impressive performance in compressive sensing reconstruction [48]. The generalized alternating projection (GAP) framework applies a faster denoiser and total variation (TV), but the denoiser cannot achieve high-quality results. Recently, deep learning approaches have been used to resolve recovery problems of visible snapshot compressive imaging for rapid and raised effects [35,39,40,41,42,44,45,46]. However, these system-specific networks cannot be directly applied to our task due to the lack of infrared hyperspectral datasets for network training. From the idea of the recent advance of the PnP image restoration, it has demonstrated that a proper denoiser can act as the image prior for optimization-based algorithms to solve the recovery problem of Equation (6). To leverage the advantage of GAP, a deep infrared denoising network is designed as the spatio-spectral prior and integrated into GAP for reconstruction. The GAP solution to the recovery problem of Equation (6) can be represented as

$$Z^{(t+1)}=v^{\left(t\right)}+{\mathsf{\Phi }}^{\mathsf{T}}{\left(\mathsf{\Phi }{\mathsf{\Phi }}^{\mathsf{T}}\right)}^{-1}(G-\mathsf{\Phi }{v}^{\left(t\right)})$$

(7)

$$v^{(k+1)}={\mathcal{D}}_{\sigma }\left(F^{(t+1)}\right)$$

(8)

where $\left(t\right)$ is the iteration number. The data is first projected on a linear manifold $G=\mathsf{\Phi }F$ in Equation (7), and the projected data are then denoised during iteration in Equation (8).

A great challenge in our task of using deep denoisers as a prior is that the contrast of the infrared image is low, various noise levels from different IRFPAs appear in the captured infrared image, and the inconspicuous features of the infrared image are difficult to extract. Therefore, the denoiser should be robust to adapt to different noise levels. We designed a deep infrared denoising network, as shown in Figure 4, and employed it in the denoising step of Equation (8), which can be formulated as

$${\mathcal{D}}_{\sigma }\left(Z^{(t+1)}\right)=\underset{v}{argmin}g\left(v\right)+\frac{1}{2}{∥Z^{(t+1)}-v∥}_2^2$$

(9)

where $g\left(v\right)$ is the regularization term. Given the noisy infrared image v, the clean infrared image can be obtained by our deep infrared denoising network.

To adapt the character of the infrared image, the noisy infrared image is down-sampled into four sub-images by convolution, as shown in Figure 4, and we then concatenate an estimated noise level map with noise deviation $\sigma$ of additive white Gaussian noise (AWGN) as an input of the network. A data cube of size $\frac{W}{2}\times \frac{H}{2}\times (4C+1)$ is then transported into a CNN with 14 convolution blocks ($[Conv+ReLU]\times 14$). The residual learning strategy is adopted for a faster convergence [49]. A convolutional kernel of size $3\times 3$ and zero padding are employed to maintain the feature map size during convolution. To produce the denoised infrared image with its original image size $W\times H$, the PixelShuffle operation [50,51] is applied to up-sample at the last step. Hereby, the denoised infrared image is obtained and then served for the next iteration. Our code is available at https://github.com/shuowenyang/GAP-DIDNet (accessed on 28 December 2022).

5. Simulation Results

5.1. Training Details of the Infrared Denoising Network

We used a cooled mid-wave infrared camera (TB-M640-CL) to obtain 100 infrared images of size $640\times 512$ (see Figure 5). We then cropped images into patches with a size of $64\times 64$, and applied data augmentation methods (flip, rotation, and scale) to increase the number of training samples. A similar operation was performed on another 20 images for testing. We added AWGN with a noise level of $\sigma \in$ [0, 75] to the original images to generate a noisy version. PyTorch [52] was used for the implementation, and ADAM [53] was chosen as the optimizer. The training epochs were set to 100 with a learning rate starting from ${10}^{-3}$, which decayed 10 times every 50 training epochs.

5.2. Algorithm Evaluation

We conducted simulation experiments to demonstrate the hardware principle and the proposed reconstruction algorithm. For the simulation data, we employed the publicly available datasets CAVE [54], KAIST [55], and Harvard [56] and transformed the images into grayscale images. We then generated measurements following our MWIR-SCSI framework using the shifting random binary mask. The proposed reconstruction algorithm competes with other popular methods, including TwIST [57], GAP-TV [48], GAP-3DTV [58], AutoEncoder [55], TSA-Net [59], and HDNet [60]. We assessed the performance of these competing methods by four metrics: peak signal-to-noise ratio (PSNR), structural similarity (SSIM) [61], the spectral angular mapper (SAM) [62], and running time.

The average PSNR, SSIM, and SAM on the entire CAVE and KAIST datasets constructed by the aforementioned algorithms are listed in

Table 1. Average PSNR and SSIM comparison of competing methods on the CAVE, KAIST, and Harvard datasets.

Datasets	Index	TwIST	GAP-TV	GAP-3DTV	AutoEncoder	TSA-Net	HDNet	Ours
CAVE	PSNR	23.74	29.07	29.15	32.46	26.10	32.18	33.03
	SSIM	0.8523	0.9219	0.8866	0.9235	0.8105	0.9024	0.9257
	SAM	16.4033	11.5969	14.7321	4.7991	15.8743	13.7483	11.1254
KAIST	PSNR	23.78	35.60	28.25	32.64	23.65	33.58	35.73
	SSIM	0.8623	0.9468	0.8708	0.9475	0.7910	0.9428	0.9494
	SAM	15.2222	6.0389	12.3403	3.0663	14.2571	7.9834	5.8549
Harvard	PSNR	22.84	30.23	28.19	31.84	23.28	32.02	32.73
	SSIM	0.8346	0.9204	0.8739	0.9214	0.8043	0.9312	0.9345
	SAM	16.3466	11.9845	14.8793	5.2893	14.9385	12.3812	10.4895

. We can observe that the PSNR and SSIM values of the optimization-based GAP algorithms are much higher than the CNN-based TSA-Net. The table also shows that the AutoEncoder method is outstanding, as it can outperform the other methods. It achieved the best results in the SAM. The reason for this may be that this algorithm learns nonlinear spectral representations from real-world hyperspectral datasets. As a comparison, the proposed algorithm achieves a further promotion in PSNR and SSIM and ranks second in terms of SAM. It can be observed that the GAP algorithms can achieve considerable results compared with the ordinary reconstruction methods TwIST and TSA-Net. This indicates that the PnP framework can perform substantially well in spectral image reconstruction. However, the traditional denoisers (TV and 3DTV) are insufficient. In terms of the performance comparison of our proposed algorithm and the learning-based algorithms, our algorithm is excellent in PSNR and SSIM and ranks second in terms of the SAM because our algorithm plugs in a deep denoiser, exploiting the advantages of both optimization-based algorithms and learning-based algorithms.

Further, to evaluate the fidelity of reconstructed spectral images, we tried to calculate the accuracy, precision, recall, and F1-score for verification. Because the original and reconstructed spectral images are grayscale instead of binary, we normalized the values between 0 and 1, and we then set the values of the images greater than 0.5 to 1 to calculate these indexes, as shown in

Table 2. Mean accuracy, precision, recall, and F1-score of the reconstructed spectral images of different algorithms.

Index	TwIST	GAP-TV	GAP-3DTV	AutoEncoder	TSA-Net	HDNet	Ours
Accuracy	0.4749	0.4821	0.4892	0.4938	0.4873	0.4645	0.4921
Precision	0.6085	0.5599	0.8334	0.7792	0.7694	0.7812	0.7799
Recall	0.3090	0.2818	0.6889	0.8528	0.8498	0.8752	0.8787
F1-score	0.3968	0.2644	0.7454	0.8139	0.8123	0.8203	0.8209

. The results show that the learning-based algorithms are capable of achieving a higher fidelity than the optimization-based ones. In particular, the proposed method produces the best results.

In terms of running time, the reconstruction of the CAVE data of spatial size $512\times 512$, the KAIST data of spatial size $256\times 256$, and the Harvard data of spatial size $1040\times 1392$ is shown in

Table 3. CPU and GPU execution time (second) comparison of different methods on the CAVE, KAIST, and Harvard datasets.

Algorithm	CAVE		KAIST		Harvard		Programming Language	Platform
	CPU	GPU	CPU	GPU	CPU	GPU
TwIST	441.4	-	111.8	-	1788.3	-	Matlab	Intel Core i3-6100 CPU
GAP-TV	49.3	-	12.7	-	210.5	-
GAP-3DTV	29.7	-	7.4	-	130.8	-
AutoEncoder	-	414.2	-	103.5	-	1639.5	Python + TensorFlow	NVIDIA GTX 1080Ti GPU
TSA-Net	-	48.6	-	12.4	-	201.4
HDNet	-	38.4	-	9.5	-	158.4	Python + Pytorch
Ours	-	26.8	-	6.7	-	124.9

for each method. The codes of the optimization-based methods were written using MATLAB and run on a CPU. The deep-learning-based methods were written in Python and run on a GPU. With the increase in image size, the time consumption also increases, but for deep learning algorithms, parallel operation can effectively save time. It is observed that the GAP-based methods have a speed advantage, but the reconstruction results of these algorithms are far from optimum. The AutoEncoder method requires a high amount of training time to obtain the spectral prior. The time-consuming TSA-Net benefits from deep learning. Although our strategy requires iterative processing during the estimation, it demonstrates an acceptable amount of time consumption for the network test on the GPU by combining the pre-trained deep denoiser.

To visualize the experimental results, the spatial details and spectral accuracy of the reconstructed results by different algorithms on images from the CAVE, KAIST, and Harvard datasets are compared in Figure 6, Figure 7 and Figure 8, respectively. The reconstructed frames of TwIST are not clean. GAP-TV can provide visually decent results, whereas GAP-3DTV suffers from blurry details. Although AutoEncoder, TSA-Net, and HDNet, as CNN-based methods, can recover most spatial details, some areas have blurred edges and lost detail, especially using the TSA-Net algorithm. By contrast, our method benefits from both the GAP and the deep denoiser, and thus displays sharper edges and higher visual quality. Significantly, the drawback of learning-based methods is even more obvious in large-size images because they rely heavily on training data, and more training parameters are required. Optimization-based algorithms can fill this gap. Thus, the combination of learning-based and optimization-based methods, i.e., the proposed method, can maintain performance well. Furthermore, the reconstructed spectral curves of the four special regions were plotted to compare the spectral accuracy, shown in Figure 9, Figure 10 and Figure 11. The reconstructed spectral curves of the proposed method have a higher consistency with the ground-truth spectra. The spectral curves of TwIST and GAP-3DTV deviate from the ground truth the most.

6. Experiment Results

6.1. MWIR Snapshot Compressive Spectral Imager Design

To experimentally evaluate the effectiveness of the proposed architecture, a proof-of-concept MWIR-SCSI system was established, as shown in Figure 12. An objective lens (focal length: 127 mm; diameter: 50 mm) was used to optically form the scene’s infrared radiation on the prism (customized ${\mathrm{AL}}_2{\mathrm{O}}_3$). The dispersed spectral scene was coded by a coded aperture, which is a lithographically patterned chrome-on-Ge mask. In our experiments, the mask we used was a Gaussian random pattern with the same resolution and pixel pitch as the IRFPA. Furthermore, each mask pixel was represented by a 1 × 1 window of IRFPA pixel subsets, so the maximum modulation resolution was 640 × 512. To flexibly adjust the mask-IRFPA distance, another relay lens (focal length 125 mm, f) was used to transfer the spatio-spectral modulation of the scene onto the IRFPA with a resolution 640 × 512 and a pixel size of 15 $\mathsf{\mu }$m.

6.2. Spatial, Temporal, and Spectral Resolution

In our MWIR-SCSI system, the spatial resolution and temporal resolution are determined by the IRFPA and its frame rate, respectively. Therefore, our MWIR-SCSI system can capture a maximum spatial resolution of 640 × 512 and a maximum temporal resolution of 50 fps. Both the dispersed spectrum width on the IRFPA plane and the IRFPA pixel size determine the spectral resolution. The calculation process of spectral resolution is as follows:

Figure 13a shows the optical path in the ${\mathrm{AL}}_2{\mathrm{O}}_3$ prism. The outgoing ray angle $\widehat{\gamma }$ is represented in Equation (10):

$$\widehat{\gamma }\left(c\right)=arcsin\left(\beta -arcsin\frac{sin\gamma }{n\left(c\right)}\right)$$

(10)

where $n(·)$ denotes the refractive index of the ${\mathrm{AL}}_2{\mathrm{O}}_3$ prism, with incident light wavelength c, $\beta$ indicates the angle of the ${\mathrm{AL}}_2{\mathrm{O}}_3$ prism, $\gamma$ denotes the angle of incidence, and y represents the position between the image plane and the prism. The spectral resolution can be calculated by Equation (11):

$$R_{spe}=\frac{\psi }{\tau }=\frac{f\left(tan\left(\widehat{\gamma }\left(c_e\right)-y\right)-tan\left(\widehat{\gamma }\left(c_s\right)-y\right)\right)}{\tau }$$

(11)

where $\psi$ denotes the dispersed spectrum width, $\tau$ denotes the IRFPA array size, and f denotes the imaging focal length. $c_e$ and $c_s$ are the maximum and minimum wavelengths in the angle of refraction of the outgoing rays, respectively. Figure 13b shows the spectral resolution $R_{spe}$ curve from 3.7 $\mathsf{\mu }$m to 4.8 $\mathsf{\mu }$m in MWIR-SCSI.

6.3. System Calibration

Equation (6) indicates the features of this coding pattern $\mathsf{\Phi }$ have a great influence on the fidelity of the estimated spectral images. In most simulation experiments, the coding pattern is regarded as a binary pattern, whereas the infrared radiation does not entirely penetrate the coded mask or reflect completely in the experiment. In addition, the response of the IRFPA to infrared radiation is inconsistent, which is also the case with other infrared optical components. As a result, the coding pattern used in the reconstruction cannot be regarded as a binary pattern but as a grayscale pattern.

Therefore, system calibration is necessary to correct this theoretical error to reconstruct accurate spectral images. The response of our system or the PSF to each pixel source of the different spectral scenes can be obtained to achieve this process. We simplify this operation by obtaining the experimental coded matrix $\mathsf{\Phi }$. Specifically, we uniformly illuminate the MWIR-SCSI system with each monochromatic wavelength of light in 10 nm increments from 3.7 $\mathsf{\mu }$m to 4.8 $\mathsf{\mu }$m and capture 111 monochrome images of the coded mask.

Some of the results are shown in Figure 14. We can see that from 3.7 $\mathsf{\mu }$m to 4.8 $\mathsf{\mu }$m, the coded mask shift with different displacements, as well as the dispersion at the shorter end of the spectrum, is less than that at the longer end, which demonstrates the non-linear spectral response of the entire system, covering the non-linearity of the prism dispersion. All of the mask images of 111 spectral channels contain the information that the reconstruction algorithm attempts to recover. The wavelength curve according to the position of the cross on the template images is shown in Figure 15. Essentially, this nonlinear curve discretely represents the dispersion coefficient of the proposed MWIR-SCSI system.

Similar to the process above, we set the values of the captured coded mask images greater than 0.5 to 1. We generated a 0–1 Gaussian random matrix to produce the coded aperture (Figure 12). Thus, the accuracy, precision, recall, and F1-score can be calculated, as shown in

Table 4. Accuracy, precision, recall, and F1-score results of the coded mask images of different wavelengths.

Index	3.7 $\mathsf{\mu }$m	3.8 $\mathsf{\mu }$m	3.9 $\mathsf{\mu }$m	4.0 $\mathsf{\mu }$m	4.1 $\mathsf{\mu }$m	4.2 $\mathsf{\mu }$m	4.3 $\mathsf{\mu }$m	4.4 $\mathsf{\mu }$m	4.5 $\mathsf{\mu }$m	4.6 $\mathsf{\mu }$m	4.7 $\mathsf{\mu }$m	4.8 $\mathsf{\mu }$m
Accuracy	0.8453	0.8343	0.7984	0.8394	0.8473	0.8743	0.8323	0.8423	0.85023	0.8446	0.8564	0.8395
Precision	0.6574	0.6473	0.6473	0.7073	0.6378	0.6874	0.6594	0.5894	0.6058	0.6128	0.6392	0.6483
Recall	0.6673	0.6534	0.6889	0.7183	0.6483	0.6984	0.6639	0.5984	0.6139	0.6229	0.6432	0.6558
F1-score	0.6704	0.6606	0.6912	0.7291	0.6503	0.7049	0.6784	0.6084	0.6294	0.6384	0.6593	0.6639

. These values are not very high, which indicates the theoretical and practical gap caused by the nonlinear response of the system.

6.4. Results and Discussion

To demonstrate the spectral imaging capability, our MWIR-SCSI system recorded measurement images at mid-wave infrared wavelengths from the sample combustion process (candle flame) at live video frame rates, as shown in Figure 16a. The partial results of 32 wavelength channels between 3.7 and 4.8 $\mathsf{\mu }$m using the proposed reconstruction method are shown in Figure 17. In order to demonstrate the hyperspectral image acquisition capability of the proposed MWIR-SCSI system for dynamic targets, we measured another candle flame in motion, as shown in Figure 16b. Similarly, the restored results by the proposed reconstruction method are shown in Figure 18. Our system and algorithm were able to obtain the candle flame details well and distinguish significant spectral differences. The longer the wavelength, the darker the outer flame, and the brighter the inner flame. This observation can be explained by the temperature variation—the temperature of the outer flame is higher than that of the inner flame, so more radiation energy is distributed over shorter wavelengths, while the radiation energy of the inner flame is higher over longer wavelengths. The spectral curves selected from the outer and inner flame are presented in Figure 16c. The strong ${\mathrm{CO}}_2$ absorption in the gap at 4.3 $\mathsf{\mu }$m is also clearly presented in the figure. This provides a reliable way to identify the chemical composition of different flames by imaging at different wavelengths.

In mid-wave infrared, we have reported another low-cost computational spectral imaging system [34], which leverages the DMD and spectrometer to collect measurement signals through multiple encoding. This strategy only conducts spatial coding, whereas the proposed MWIR-SCSI system simultaneously performs spatial and spectral coding.

Table 5. The major differences between the two computational mid-wave infrared spectral imagers.

Mode	Coding Scheme	Spatial Resolution (Pixels)	Spectral Resolution	Spectral Channel	Acquisition Time (Second)	Reconstructed Time (Second)	Cost
Single-pixel	spatial	64 × 48	2	100	4	2293	low
Snapshot	spatial & spectral	640 × 512	10	111	0.02	107	high

lists the significant differences between these devices. The MWIR-SCSI system makes it possible to achieve fast acquisition through single encoding. However, the main limitation of the MWIR-SCSI system is that traditional optimization algorithms require high amounts of time for reconstruction. Fortunately, the future development of deep learning methods can effectively address this problem.

7. Conclusions

In this work, we present a novel mid-wave infrared snapshot compressive spectral imager based on a prism and a coded aperture to produce spatio-spectral coded to the benefit of reconstruction. In addition, as an important step in the infrared imaging system, convenient and practical calibration methods are implemented in our system. We acquire 111 real coded masks in which the displacement reflects the non-linear spectral response of the entire system. To address inconsistencies in infrared image noise levels and missing datasets, the image reconstruction problem in our system is formulated as an inverse problem with the GAP plug-in deep infrared denoising network, ensuring smooth reconstructed images in spatial and spectral dimensions. Compared to traditional scanning architectures, the proposed MWIR-SCSI performs well in dynamic scenes because objects can be sparsely sampled at live video frame rates. Extensive results on a simulation dataset and real infrared scenarios obtained by our system have demonstrated the performance of both the proposed system and the method.