Low-Rank and Sparse Matrix Recovery for Hyperspectral Image Reconstruction Using Bayesian Learning

Abstract

In order to reduce the amount of hyperspectral imaging (HSI) data transmission required through hyperspectral remote sensing (HRS), we propose a structured low-rank and joint-sparse (L&S) data compression and reconstruction method. The proposed method exploits spatial and spectral correlations in HSI data using sparse Bayesian learning and compressive sensing (CS). By utilizing a simultaneously L&S data model, we employ the information of the principal components and Bayesian learning to reconstruct the hyperspectral images. The simulation results demonstrate that the proposed method is superior to LRMR and SS&LR methods in terms of reconstruction accuracy and computational burden under the same signal-to-noise tatio (SNR) and compression ratio.

1. Introduction

HSI is a collection of hundreds of images that are usually acquired simultaneously in narrow and adjacent spectral bands by airborne sensors [1] or spaceborne spectrometers [2]. HSI combines the traditional two-dimensional remote sensing imaging technology and the optical imaging technology of spectroscopy [3]. Moreover, HSI has achieved the effect of acquiring images and spectra of objects at the same time, which has set off a revolution in the field of remote sensing. In recent years, with the rapid development of precision agriculture, hyperspectral imaging (HSI) technology in hyperspectral remote sensing [4] has been widely used. In precision agriculture, hyperspectral images can be used to monitor drought and flooding in farmlands, pests and diseases, and crop growth, as well as to predict farmland yield. According to the estimated variability from HSI, precision agriculture can improve resource-use efficiency, productivity, quality, profitability, and sustainability of agricultural production, such as by using less irrigation water, fewer pesticides and fertilizers, etc. HSI is also used in many applications besides precision agriculture, including forestry monitoring, natural resource investigation, vegetation observation, food safety, and geological mapping.

However, the acquisition and processing of HSI require a high-sensitivity detector, ultra-long-distance transmission, great computing power, and a huge data storage capacity in order to deal with such huge amounts of data. The HSI data need to be transmitted to ground stations. In this regard, one of the obstacles that researchers have had to face in the past 20 years is that of finding a way to program hyperspectral satellites to reduce the HSI data. Fortunately, HSI data are low-rank [5] and sparse in the transform domain (discrete cosine transform (DCT) or wavelet transform) [6,7,8]. These kinds of data have the features of being low-rank and sparse, which means that the data are redundant, and a small number of datasets can be used to express the entirety of the data. So, we can use these features to reduce the amount of HSI data to be transmitted. This might be a good way to solve the problem.

Recently, many works have focused on modeling, compressing, and reconstructing HSI data using a structured framework. By dividing HSI into many blocks, Zhang et al. [5] introduced an HSI data reconstruction method based on low-rank matrix recovery (LRMR). In particular, for this kind of signal, many new compressive sensing (CS)-based methods [9,10,11] have been proposed [6,8,12,13,14,15,16,17,18,19]. Golbabaee et al. [6] simultaneously reconstructed HSI data with a low-rank and joint-sparse (L&S) structure by assuming that HSI data are low-rank and using a spatially joint-sparse wavelet representation. Zhang et al. [12] showed that the block sparse Bayesian learning (bSBL) algorithm has good recovery performance for data with a spatial block structure (such as an L&S structure). However, most of the existing research focused on low-rank structure reconstruction methods or HSI data denoising methods, and there are no methods for integrating the process of HSI acquisition and combining a sparse structure reconstruction method to reconstruct HSI data.

In this paper, a method based on the bSBL framework is proposed to compress, transmit, and reconstruct the entirety of HSI data. Here, each sub-matrix of multi-channel data is collected, compressed, and transmitted to the ground processing center through push-broom imaging. The proposed method not only combines the L&S structure priors on HSI data and the appropriate priors on hyperparameters, but also uses the information of the main components of the data. Firstly, we assume that the covariance matrix of the data is a diagonal matrix, and then obtain the initial value of the hyperparameter. Finally, an EM-like method is used to obtain the best reconstruction of the HSI data. The simulation results show that the proposed method has better performance than other existing methods.

We highlight the following main contributions:

(1)

The proposed method gives the structure of the covariance matrix of the L&S signals, models HSI data with the L&S structure, and utilizes the CS and Bayesian learning methods to compress and reconstruct HSI data.

(2)

In the reconstruction part, the proposed method makes use of the relationship between multiple dimensions of high-dimensional data and combines data reconstruction with HSI data acquisition. It can be used to realize the segmented acquisition, compression, and transmission of HSI data so as to reduce the amount of calculation and data transmission that must be performed in the satellite.

(3)

We demonstrate the superior performance of the proposed method in comparison with state-of-the-art alternatives by conducting experiments on both synthetic signals and real signals.

The rest of this paper is arranged as follows: Section 2 introduces two 2D reconstruction methods for HSI data. Section 3 introduces the L&S data compression and reconstruction method combined with the acquisition mode. Section 3.1 introduces a model of the problem, and Section 3.2 proposes the HSI reconstruction method based on push-broom imaging. Section 3.3 introduces the simulation results. Discussion is given in Section 4. Finally, this work is concluded in Section 5.

Notation: $p\left(\mathbf{A}\right)\sim \mathcal{N}(\mathbf{0},\mathsf{\Sigma })$ denotes the probability density function of $\mathbf{A}$, which follows a Gaussian distribution with mean $\mathbf{0}$ and variance $\mathsf{\Sigma }$. $\left|\mathbf{A}\right|$ denotes the determinant of $\mathbf{A}$. ${\parallel \mathbf{x}\parallel }_2$ denotes the ${\ell }_2$ norm of $\mathbf{x}$. $\mathrm{vec}\left[\mathbf{A}\right]$ denotes the vectorization of the matrix $\mathbf{A}$ formed by stacking its columns into a single column vector. ${\mathbf{A}}^{\top }$ denotes the transpose of $\mathbf{A}$. $\mathrm{tr}\left(\mathbf{A}\right)$ denotes the trace of $\mathbf{A}$.

2. Two-Dimensional Reconstruction Methods for HSI Data

As described in the previous section, this section will combine the fast method proposed in our previous work [20] to compress and reconstruct HSI data from two different two-dimensional (2D) slice methods and will provide two 2D reconstruction algorithms for three-dimensional (3D) HSI data. These two methods are the method of expanding 3D data HSI from different bands to 2D data by using a tensor and the method of slicing them into 2D data one by one according to the acquisition mode.

The HSI data experiment was carried out with the repetition of 100 real tests. Each experimental datum was selected from the Salinas Database  (http://www.ehu.eus/ccwintco/index.php/Hyperspectral_Remote_Sensing_Scenes#Salinas-A_scene (accessed on 10 November 2021)) dataset in a $30\times 50$ area, and 20 bands were used, that is, the data matrix was a $30\times 50\times 20$ 3D matrix with $N=30,Q=50,M=20$. The two 2D reconstruction methods are first introduced in the following sections. The core part of this section shows the direct application of the fast L&S-bSBL method [20]. The same abbreviations as in [20] are used here.

(1)

Estimation of $\mathbf{x}$.

After the Bayesian posterior probability is obtained with the Bayesian rule, the maximum a posteriori (MAP) is used to obtain the estimation of $\mathbf{x}$:

$$\begin{array}{cc}\hfill \widehat{\mathbf{x}}=\mathrm{vec}\left({\widehat{\mathbf{X}}}^{\top }\right)\triangleq {\mathbf{\mu }}_{\mathbf{x}}& ={(\lambda {\mathsf{\Sigma }}_0^{-1}+{\mathbf{H}}^{\top }\mathbf{H})}^{-1}\mathbf{H}{\mathsf{\Sigma }}_0\hfill \\ \hfill & ={\mathsf{\Sigma }}_0{\mathbf{H}}^{\top }{(\lambda \mathbf{I}+\mathbf{H}{\mathsf{\Sigma }}_0{\mathbf{H}}^{\top })}^{-1}\mathbf{y},\hfill \end{array}$$

(1)

where ${\mathsf{\Sigma }}_0$ is a block matrix of $\mathbf{x}$, and there are few elements that are non-zero. Moreover, the sparsity of blocks of $\widehat{\mathbf{x}}$ is determined by ${\gamma }_i{\gamma }_j$. When ${\gamma }_k=0$, the value of the kth related block in $\widehat{\mathbf{x}}$ is zero.

(2)

Estimation of $\lambda$.

In order to obtain $\lambda$, we simplify the expression of ${\mathsf{\Sigma }}_0$ as

$${\mathsf{\Sigma }}_0=\mathsf{\Gamma }\otimes \mathbf{B}.$$

(2)

After maximizing the logarithm of the joint probability of $\mathbf{x}$ and $\mathbf{y}$, we take the derivative of it with respect to $\lambda$:

$$\lambda \leftarrow \frac{\parallel \mathbf{y}-\mathbf{H}{\mathbf{\mu }}_{\mathbf{x}}{\parallel }_2^2+{\lambda }^{\left(pre\right)}\left[mn-\mathrm{tr}\left({\mathsf{\Sigma }}_{\mathbf{x}}{\mathsf{\Sigma }}_0^{-1}\right)\right]}{pn}.$$

(3)

(3)

Estimation of $\mathbf{B}$.

$$\begin{array}{cc}\hfill \mathbf{B}& =arg\underset{\mathbf{X}}{min}\mathrm{tr}\left[{\mathbf{B}}_0^{-1}\left({\mathbf{XX}}^{\top }+{\nabla }_{{\mathbf{B}}_0^{-1}}\right)\right]+mlog\left|{\mathbf{B}}_0\right|\hfill \\ \hfill & =\frac{1}{m}\left(\widehat{\mathbf{X}}{\widehat{\mathbf{X}}}^{\top }+{\nabla }_{{\mathbf{B}}_0^{-1}}\right).\hfill \end{array}$$

(4)

where

$${\mathbf{B}}_0\leftarrow \frac{1}{m}\sum _{i=1}^m\frac{\left({\mathsf{\Sigma }}_{\mathbf{x}}^i+{\mathbf{\mu }}_{\mathbf{x}}^i{\left({\mathbf{\mu }}_{\mathbf{x}}^i\right)}^{\top }\right)}{{\gamma }_i^2},$$

(5)

$${\nabla }_{{\mathbf{B}}^{-1}}=\sum _{i=1}^m\mathbf{B}-\mathbf{B}{\mathbf{H}}_i^{\top }{\left(\mathbf{H}{\mathsf{\Sigma }}_0{\mathbf{H}}^{\top }+\lambda \mathbf{I}\right)}^{-1}{\mathbf{H}}_i\mathbf{B}.$$

(6)

(4)

Estimation of $\mathsf{\Gamma }$.

The algorithm calculates the singular values of $\mathbf{X}$ through singular value decomposition (SVD) for each slice of the HSI data and sorts the singular values in descending order. Depending on the singular value of the sequence, we can obtain the value of the corresponding ${\gamma }_i$$(i=1,\cdots$, $m)$. The ${\gamma }_i$ corresponding to the larger singular value is 1, or is otherwise 0. For HSI data, a large singular value distribution can be obtained by analyzing only one segment of its 2D slice signal in advance. The expression of $\mathsf{\Gamma }$ is

$$\mathsf{\Gamma }=\left[\begin{array}{cccc}{\gamma }_1{\gamma }_1& {\gamma }_1{\gamma }_2& \cdots & {\gamma }_1{\gamma }_m\\ {\gamma }_2{\gamma }_1& {\gamma }_2{\gamma }_2& \cdots & {\gamma }_2{\gamma }_m\\ ⋮& ⋮& \ddots & ⋮\\ {\gamma }_m{\gamma }_1& {\gamma }_m{\gamma }_2& \cdots & {\gamma }_m{\gamma }_m\end{array}\right].$$

(7)

(5)

Estimation of ${\mathbf{S}}_{opt}$.

$${\mathbf{X}}^{opt}\leftarrow \mathbf{x}.$$

(8)

2.1. Two-Dimensional HSI Reconstruction Algorithm—L&S-bSBL (1)

Here, we consider a typical HSR scenario (push-broom imaging) in which there are M sensors with different wavelengths (called channels or bands; here, we have M bands) for acquiring HSI data $\mathcal{F}=[:,{\mathbf{F}}_q,:]\in {\mathbb{R}}^{MN\times Q},q\in \{1,2,\cdots$, $Q\}$ in an $N\times Q$ area following the Q-dimension, where ${\mathbf{F}}_q=[{\mathbf{f}}_1,\cdots$, ${\mathbf{f}}_M{]}^{\top }\in {\mathbb{R}}^{M\times N\times 1}$. Here, we simply express this as ${\mathbf{F}}_q=[{\mathbf{f}}_1,\cdots$, ${\mathbf{f}}_M{]}^{\top }\in {\mathbb{R}}^{M\times N}$. After all HSI data $\mathcal{F}$ are collected, the proposed algorithm performs the following steps on the HSI data, as shown in Figure 1.

(1)

According to the different bands M, we use the tensor to expand with ${\mathbf{F}}_{1}1,\cdots$, ${\mathbf{F}}_{m}1,\cdots$, ${\mathbf{F}}_{M}1$;

(2)

let ${\mathbf{F}}_{m}1,m\in \{1,2,\cdots$, $M\}$ turn into ${\mathbf{F}}_m,m\in \{1,2,\cdots$, $M\}$ by using the $\mathrm{vec}$ operator;

(3)

let $\mathbf{F}=[{\mathbf{F}}_1^{\top },{\mathbf{F}}_2^{\top },\cdots$, ${\mathbf{F}}_M^{\top }]$;

(4)

obtain the corresponding value of $\mathbf{F}$ in the DCT domain $\mathbf{X}\in {\mathbb{R}}^{M\times NQ}$.

Here, we first conduct a principle component analysis (PCA) on the data $\mathbf{F}$. As shown in Figure 2, we can obtain 2D HSI tensor expansion data $\mathbf{F}$ in the sparse domain, which is mainly distributed in the first two columns, as well as the corresponding ${\gamma }_1=1$ and ${\gamma }_2=1$; the other ${\gamma }_i,i\in \{2,3,\cdots$, $M\}$ is 0. Figure 3 shows a schematic diagram of the covariance matrix of x.

For the obtained $\mathbf{X}\in {\mathbb{R}}^{M\times NQ}$, the single measurement vector (SMV) model is used to solve the problem of data compression, transmission, and reconstruction. First, the acquired data $\mathbf{X}$ are columnarized into $\mathbf{x}$, encoded by the linear mapping matrix $\mathbf{H}$, and then transmitted to the ground receiving station through a wireless channel. Here, we assume that the encoded data will be superimposed with noise $\mathbf{e}$ in the process of channel transmission, and the data received at the ground receiving station are denoted as $\mathbf{y}$. Therefore, the mathematical model of the problem can be expressed as $\mathbf{y}=\mathbf{Hx}+\mathbf{e}$. The proposed algorithm uses the fast L&S-bSBL algorithm in Equations (1), (3), (4), (7) and (8) to iteratively reconstruct the original data $\mathbf{x}$; then, we obtain $\mathbf{F}$, $\mathcal{X}$. Most existing research regards the HSI signal as a low-rank signal to reconstruct, so the typical Bayesian low-rank reconstruction algorithm BARM [21] and the simultaneous low-rank and joint-sparse reconstruction algorithm SS&LR [22] are selected as the algorithms for comparison.

Figure 4 shows a comparison of the reconstruction performance (including mean squared error (MSE) vs. signal-to-noise ratio (SNR) and runtime vs. SNR) of all algorithms in reconstructing 2D unfolded HSI data. In Figure 4a, we observe that the proposed algorithm L&S-bSBL (1) outperforms all of the other methods because the data expansion method of the proposed algorithm increases the amount of relevant data. As shown in Figure 4b, the proposed algorithm uses less runtime than BARM and almost the same amount as SS&LR. Comprehensive consideration of the two figures shows that the proposed algorithm is better than the other two algorithms in terms of reconstruction performance and computational resource consumption.

2.2. Two-Dimensional HSI Reconstruction Algorithm—L&S-bSBL (2)

Here, we also consider a typical HSR scenario (push-broom imaging) in which there are M sensors with different wavelengths (called channels or bands; here, we have M bands) to acquire HSI data $\mathcal{F}=[:,{\mathbf{F}}_q,:]\in {\mathbb{R}}^{MN\times Q},q\in \{1,2,\cdots$, $Q\}$ in an $N\times Q$ area following the Q-dimension, where ${\mathbf{F}}_q=[{\mathbf{f}}_1,\cdots$, ${\mathbf{f}}_M{]}^{\top }\in {\mathbb{R}}^{M\times N\times 1}$. After all HSI data $\mathcal{F}$ are collected, the proposed algorithm performs the following steps on the HSI data, as shown in Figure 5.

(1)

Let slices ${\mathbf{F}}_q\in {\mathbb{R}}^{M\times N\times 1}$ and $q\in \{1,2,\cdots$, $Q\}$ as ${\mathbf{F}}_q\in {\mathbb{R}}^{M\times N}$;

(2)

obtain the corresponding value of ${\mathbf{F}}_q\in {\mathbb{R}}^{M\times N}$ in the DCT domain ${\mathbf{X}}_q$, $q\in \{1,2,\cdots$, $Q\}$;

(3)

let ${\mathbf{X}}_q$ turn into ${\mathbf{x}}_q$, $q\in \{1,2,\cdots$, $Q\}$ by using the vec operator.

Here, we first conduct a PCA on the data ${\mathbf{F}}_q$. As shown in Figure 6, we can obtain 2D HSI slice data ${\mathbf{F}}_q$ in the sparse domain, which is mainly distributed in the first two columns, as well as the corresponding ${\gamma }_1=1$ and ${\gamma }_2=1$; the other ${\gamma }_i,i\in \{2,3,\cdots$, $M\}$ is 0. The schematic diagram of the covariance matrix of $\mathbf{x}$ is similar to that shown in Figure 3.

For the obtained ${\mathbf{X}}_q\in {\mathbb{R}}^{M\times N}$, the SMV model is used to solve the problem of data compression, transmission, and reconstruction. Firstly, the acquired data ${\mathbf{X}}_q$ are columnarized into ${\mathbf{x}}_q$, encoded by the linear mapping matrix $\mathbf{H}$, and then transmitted to the ground receiving station through a wireless channel. Here, we assume that the encoded data will be superimposed with noise ${\mathbf{e}}_q$ in the process of channel transmission, and the data received at the ground receiving station are denoted as ${\mathbf{y}}_q$. Therefore, the mathematical model of the problem can be expressed as ${\mathbf{y}}_q=H{\mathbf{x}}_q+{\mathbf{e}}_q$. The proposed algorithm uses the fast L&S-bSBL algorithm in Equations (1), (3), (4), (7) and (8) to iteratively reconstruct the original data $\mathbf{x}$; then, we obtain $\mathbf{F}$, $\mathcal{X}$. Most existing research regards HSI signals as low-rank signals to recover, so BARM and SS&LR are selected as the algorithms for comparison.

Figure 7 shows a comparison of the reconstruction performance of all algorithms for all Q slices. In Figure 7a, we can see that the reconstruction of these slices is acceptable. However, L&S-bSBL (1) has better performance than L&S-bSBL (2) in reconstructing the same data, as shown in Figure 4. The reason is that L&S-bSBL (1) greatly utilizes the correlation between all data and improves the reconstruction performance, while L&S-bSBL (2) only uses the correlation within a certain band, ignoring the correlation between the bands at the same location.

Moreover, L&S-bSBL (2) only takes about 35 s compared with L&S-bSBL (1), which takes 150 s to recover the same data, as shown in Figure 4. This is because L&S-bSBL (1) has a large data dimension for a single operation and L&S-bSBL (2) has a small dimension, and the amount of calculation required for the calculation of the matrix inverse operation is not within an order of magnitude.

3. L&S Reconstruction Algorithm Combined with Acquisition Methods

3.1. Problem Formulation and Signal Model

We consider a typical HSR scenario (push-broom imaging) in which there are M sensors with different wavelengths (called channels or bands; here, we have M bands) to collect HSI data $\mathcal{F}=[:,{\mathbf{F}}_q,:]\in {\mathbb{R}}^{MN\times Q},q\in \{1,2,\cdots$, $Q\}$ in an $N\times Q$ area. Figure 8 shows that a spaceborne spectrometer (push-broom imaging) acquires multi-channel data ${\mathbf{F}}_q=[{\mathbf{f}}_1,\cdots$, ${\mathbf{f}}_M{]}^{\top }\in {\mathbb{R}}^{M\times N}$ with time synchronization among different regions following the Q-dimension, where ${\mathbf{f}}_m\in {\mathbb{R}}^{N\times 1},m\in \{1,2,\cdots$, $M\}$ stands for the data collected by the $m\mathrm{th}$ channel sensor and ${\mathbf{F}}_q$ is the spatially and temporally correlated data matrix. Then, ${\mathbf{F}}_q$ is encoded by linearly mixing with ${\Xi }_q$ and transmitted to a ground receiving station, denoted as ${\mathbf{Y}}_q$, after superimposing noise ${\mathbf{V}}_q$. Finally, a novel CS method is used to decode all ${\mathbf{F}}_q$ at the same time at the ground processing center.

Fortunately, HSI data are highly correlated with the locations and bands of their non-zero elements in a sparse domain. So, we can get an approximately L&S matrix ${\mathbf{X}}_q\in {\mathbb{R}}^{M\times N}$ from ${\mathbf{F}}_q={\mathsf{\Psi }}_q{\mathbf{X}}_q$, where ${\mathsf{\Psi }}_q\in {\mathbb{R}}^{M\times M}$ is a sparsifying basis (e.g., DCT matrix or wavelet matrix) [23], and $\mathcal{X}=[:,{\mathbf{X}}_q,:]\in {\mathbb{R}}^{MN\times Q}$. Thus, we have the following formulation:

$${\mathbf{Y}}_q={\mathsf{\Phi }}_q{\mathbf{X}}_q+{\mathbf{V}}_q,$$

(9)

where ${\mathsf{\Phi }}_q={\Xi }_q{\mathsf{\Psi }}_q$ is a known dictionary matrix. Here, this problem belongs to the multiple measurement vector (MMV) [24] problem.

Following our recent work [12], we now consider a bSBL framework [25] (i.e., L&S-bSBL) to reconstruct all ${\mathbf{X}}_q$. By letting ${\mathbf{y}}_q=\mathrm{vec}\left[{\mathbf{Y}}_q^{\top }\right]\in {\mathbb{R}}^{P\times 1}$, ${\mathbf{A}}_q={\mathsf{\Phi }}_q\otimes {\mathbf{I}}_n\in {\mathbb{R}}^{P\times NM}$, ${\mathbf{x}}_q=\mathrm{vec}\left[{\mathbf{X}}_q^{\top }\right]\in {\mathbb{R}}^{NM\times 1}$, ${\mathbf{v}}_q=\mathrm{vec}\left[{\mathbf{V}}_q^{\top }\right]\in {\mathbb{R}}^{P\times 1}$, after transforming the MMV problem to the block SMV [24] problem, we have

$${\mathbf{y}}_q={\mathbf{A}}_q{\mathbf{x}}_q+{\mathbf{v}}_q.$$

(10)

Then, the original problem becomes

$$\mathbf{y}=\mathbf{Ax}+\mathbf{v},$$

(11)

where $\mathbf{y}=[{\mathbf{y}}_1^{\top },\cdots$, ${\mathbf{y}}_Q^{\top }{]}^{\top }\in {\mathbb{R}}^{QP\times 1}$, $\mathbf{A}={\mathbf{I}}_Q\otimes {\mathbf{A}}_q(q\in \{1,2,\cdots$, $Q\left\}\right)\in {\mathbb{R}}^{QP\times QNM}$, and ${\mathbf{I}}_Q\in {\mathbb{R}}^{Q\times Q}$ denotes a $Q\times Q$ identity matrix. $\mathbf{x}=[{\mathbf{x}}_1^{\top },\cdots$, ${\mathbf{x}}_Q^{\top }{]}^{\top }\in {\mathbb{R}}^{QNM\times 1}$, ${\mathbf{x}}_q$ is the qth block in $\mathbf{x}$. The presence of K non-zero rows in ${\mathbf{X}}_q$ means that there are $K\times Q$ non-zero blocks in $\mathbf{x}$. Thus, $\mathbf{x}$ is a block sparse vector. $\mathbf{v}=[{\mathbf{v}}_1^{\top },\cdots$, ${\mathbf{v}}_Q^{\top }{]}^{\top }\in {\mathbb{R}}^{QP\times 1}$.

We assume that the noise elements ${\mathbf{v}}_q$, $q\in \{1,\cdots$, $Q\}$ follow an identical and independent distribution with $p\left({\mathbf{v}}_q\right)\sim \mathcal{N}(0,\lambda ),\forall q$. We define the Gaussian likelihood for problem (10) as

$$p\left(\mathbf{y}\right|\mathbf{x};\mathbf{A},\lambda )\sim {\mathcal{N}}_{\mathbf{y}}|\mathbf{x}(\mathbf{Ax},\lambda \mathbf{I})\propto exp[-\frac{1}{2\lambda }{\parallel \mathbf{A}\mathbf{x}-\mathbf{y}\parallel }_2^2],$$

(12)

and the prior of $\mathbf{x}$ is given by

$$p(\mathbf{x};{\gamma }_{qi},{\gamma }_{qj},{\mathbf{B}}_{qij},\forall q,i,j)\sim {\mathcal{N}}_{\mathbf{x}}(\mathbf{0},{\mathsf{\Sigma }}_0)\propto exp[-\frac{1}{2}{\mathbf{x}}^{\top }{\mathsf{\Sigma }}_0^{-1}\mathbf{x}],$$

(13)

where ${\mathbf{B}}_{qij}\in {\mathbb{R}}^{N\times N}$ is a covariance matrix between ${\mathbf{x}}_{qi}$ and ${\mathbf{x}}_{qj}$, $i,j=1,\cdots$, M. ${\mathsf{\Sigma }}_0={\mathbf{I}}_Q$$\otimes {\mathsf{\Sigma }}_q$, ${\mathsf{\Sigma }}_q={\mathsf{\Gamma }}_q\otimes {\mathbf{B}}_{qij}$, ${\mathsf{\Gamma }}_q={\mathsf{\Gamma }}_{q0}{\mathsf{\Gamma }}_{q0}^{\top }$. ${\mathsf{\Gamma }}_{q0}=[{\gamma }_{q1},\cdots$, ${\gamma }_{qM}{]}^{\top }$ is the sparsity pattern vector of ${\mathbf{X}}_q$, where the support indicates ${\gamma }_{qi}\in \{0,1\}$, $i=1,\cdots$, M.

Typically, we can obtain

$$\begin{array}{cc}\hfill & {\mathsf{\Sigma }}_0=\mathrm{diag}({\mathsf{\Sigma }}_1,\cdots ,{\mathsf{\Sigma }}_q,\cdots ,{\mathsf{\Sigma }}_Q),\hfill \\ \hfill & {\mathsf{\Sigma }}_q=\left[\begin{array}{cccc}{\gamma }_{q1}{\gamma }_{q1}{\mathbf{B}}_{q11}& {\gamma }_{q1}{\gamma }_{q2}{\mathbf{B}}_{q12}& \cdots & {\gamma }_{q1}{\gamma }_{qM}{\mathbf{B}}_{q1M}\\ {\gamma }_{q2}{\gamma }_{q1}{\mathbf{B}}_{q21}& {\gamma }_{q2}{\gamma }_{q2}{\mathbf{B}}_{q22}& \cdots & {\gamma }_{q2}{\gamma }_{qM}{\mathbf{B}}_{q2M}\\ ⋮& ⋮& \ddots & ⋮\\ {\gamma }_{qM}{\gamma }_{q1}{\mathbf{B}}_{qM1}& {\gamma }_M{\gamma }_2{\mathbf{B}}_{qM2}& \cdots & {\gamma }_{qM}{\gamma }_{qM}{\mathbf{B}}_{qMM}\end{array}\right].\hfill \end{array}$$

(14)

Without losing generality, we assume that ${\mathbf{X}}_q$ is a matrix that is low-rank and joint-sparse in columns. Figure 9 illustrates an example of ${\mathbf{X}}_q$ and the structure of the covariance matrix ${\mathsf{\Sigma }}_q$ of ${\mathbf{x}}_q$ with $M=4$. In the figure, we find that only the first two columns have values, so only ${\gamma }_{q1}=1$ and ${\gamma }_{q2}=1$. Thus, the parts related to ${\gamma }_{q1}$, ${\gamma }_{q2}$ in ${\mathsf{\Sigma }}_q$ are valuable.

3.2. Proposed Method

Following our last work [12], we obtain the posterior density of $\mathbf{x}$ with the Bayesian rule:

$$p\left(\mathbf{x}\right|\mathbf{y};\lambda ,{\gamma }_{q}i,{\gamma }_{q}j,{\mathbf{B}}_{qij},\forall q,i,j)\sim {\mathcal{N}}_{\mathbf{x}}({\mathbf{\mu }}_{\mathbf{x}},{\mathsf{\Sigma }}_{\mathbf{x}}),$$

(15)

where the mean ${\mathbf{\mu }}_{\mathbf{x}}$ and the covariance ${\mathsf{\Sigma }}_{\mathbf{x}}$ can be obtained by

$${\mathbf{\mu }}_{\mathbf{x}}=\frac{1}{\lambda }{\mathsf{\Sigma }}_{\mathbf{x}}{\mathbf{A}}^{\top }\mathbf{y},$$

(16)

$$\begin{array}{cc}\hfill {\mathsf{\Sigma }}_{\mathbf{x}}& ={({\mathsf{\Sigma }}_0^{-1}+\frac{1}{\lambda }{\mathbf{A}}^{\top }\mathbf{A})}^{-1}\hfill \\ \hfill & ={\mathsf{\Sigma }}_0-{\mathsf{\Sigma }}_0{\mathbf{A}}^{\top }{(\lambda \mathbf{I}+\mathbf{A}{\mathsf{\Sigma }}_0{\mathbf{A}}^{\top })}^{-1}\mathbf{A}{\mathsf{\Sigma }}_0.\hfill \end{array}$$

(17)

When all of the hyperparameters $\lambda ,{\gamma }_{qi},{\gamma }_{qj},{\mathbf{B}}_{qij},\forall q,i,j$ are given, the maximum a posteriori (MAP) estimate of $\mathbf{x}$ can be obtained by

$$\begin{array}{cc}\hfill \widehat{\mathbf{x}}=\mathrm{vec}\left[{\widehat{\mathcal{X}}}^{\top }\right]\triangleq {\mathbf{\mu }}_{\mathbf{x}}& ={(\lambda {\mathsf{\Sigma }}_0^{-1}+{\mathbf{A}}^{\top }\mathbf{A})}^{-1}\mathbf{A}{\mathsf{\Sigma }}_0\hfill \\ \hfill & ={\mathsf{\Sigma }}_0{\mathbf{A}}^{\top }{(\lambda \mathbf{I}+\mathbf{A}{\mathsf{\Sigma }}_0{\mathbf{A}}^{\top })}^{-1}\mathbf{y},\hfill \end{array}$$

(18)

where ${\mathsf{\Sigma }}_0$ is the approximate diagonal matrix obtained by Equation (14), with most block matrices being zeros. It is clear that ${\gamma }_{qi},{\gamma }_{qj},\forall q,i,j$ control the sparsity of the blocks of $\widehat{\mathbf{x}}$. When ${\gamma }_{qk}=0$, the associated $qk$th block in $\widehat{\mathbf{x}}$ becomes zero. In fact, ${\gamma }_{1i}=\cdots ={\gamma }_{qi}=\cdots ={\gamma }_{Qi}$.

Following the bSBL framework [25], to avoid overfitting, we use a common positive definite matrix $\mathsf{\Sigma }$ to model all of the covariance matrices ${\mathsf{\Sigma }}_q$. In ${\mathsf{\Sigma }}_q$, we also use a common positive definite matrix $\mathbf{B}$ instead of ${\mathbf{B}}_{qij}$ and we use a group of ${\gamma }_i\in \{0,1\}$ instead of all ${\gamma }_{qi}\in \{0,1\}$, $\mathsf{\Gamma }$ and instead of ${\mathsf{\Gamma }}_q$. From the previous analysis, we can find that the covariance matrices induce the spatiotemporal correlation in the prior density. Thus, Equation (14) can be written as

$${\mathsf{\Sigma }}_0={\mathbf{I}}_Q\otimes \mathsf{\Gamma }\otimes \mathbf{B}.$$

(19)

Using the Bayesian strategy, we maximize the marginal probability $p(\mathbf{x},\mathbf{y})$ of $\mathbf{x}$:

$$\underset{\mathbf{B}\in {\mathbf{N}}^{+},\mathsf{\Gamma }\ge 0}{max}\int p\left(\mathbf{y}\right|\mathbf{x};\mathbf{A},\lambda \left)p\right(\mathbf{x};\mathsf{\Gamma },\mathbf{B})d\mathbf{x},$$

(20)

which is equivalent to minimizing the cost function of $-2logp(\mathbf{y};\lambda ,\mathsf{\Gamma },\mathbf{B})$:

$$\mathcal{L}(\mathsf{\Gamma },\mathbf{B},\lambda )={\mathbf{y}}^{\top }{\mathsf{\Sigma }}_y^{-1}\mathbf{y}+log|{\mathsf{\Sigma }}_y|,$$

(21)

where ${\mathbf{N}}^{+}$ denotes a set of $N\times N$ positive definite matrices.

$${\mathsf{\Sigma }}_y=\mathbf{A}{\mathsf{\Sigma }}_0{\mathbf{A}}^{\top }+\lambda \mathbf{I},{\mathsf{\Sigma }}_0={\mathbf{I}}_Q\otimes \mathsf{\Gamma }\otimes \mathbf{B}.$$

(22)

Here, ${\mathsf{\Sigma }}_y$ denotes the covariance of $\mathbf{y}$ given $\mathsf{\Gamma }$ and $\mathbf{B}$.

Simply, $\mathsf{\Theta }\equiv \{\mathsf{\Gamma },\mathbf{B},\lambda \}$; thus, (21) turns into

$$\mathcal{L}\left(\mathsf{\Theta }\right)={\mathbf{y}}^{\top }{\mathsf{\Sigma }}_y^{-1}\mathbf{y}+log|{\mathsf{\Sigma }}_y|.$$

(23)

Firstly, $\mathbf{x}$ is treated as a hidden variable in the expectation maximization (EM) formulation that proceeds, and we maximize

$$\begin{array}{cc}\hfill \mathcal{Q}\left(\mathsf{\Theta }\right)=& E_{\mathbf{x}|\mathbf{y};{\mathsf{\Theta }}^{\left(pre\right)}}[log\left(p\left(\mathbf{y}\right|\mathbf{x};\lambda \left)p\right(\mathbf{x};\mathsf{\Gamma },\mathbf{B})\right)]\hfill \\ \hfill =& E_{\mathbf{x}|\mathbf{y};{\mathsf{\Theta }}^{\left(pre\right)}}[logp\left(\mathbf{y}\right|\mathbf{x};\lambda )]\hfill \\ \hfill & +E_{\mathbf{x}|\mathbf{y};{\mathsf{\Theta }}^{\left(pre\right)}}[logp(\mathbf{x};\mathsf{\Gamma },\mathbf{B})],\hfill \end{array}$$

(24)

where ${\mathsf{\Theta }}^{\left(pre\right)}$ denotes the hyperparameters that are estimated in the previous iteration.

To estimate $\lambda$—only the first term in the $\mathcal{Q}$ function is correlated with $\lambda$—it can be simplified as

$$\begin{array}{cc}\hfill \mathcal{Q}\left(\lambda \right)=& E_{\mathbf{y}|\mathbf{x};{\mathsf{\Theta }}^{\left(pre\right)}}\left[logp\left(\mathbf{y}\right|\mathbf{x};\lambda )\right]-\frac{QP}{2}log\lambda \hfill \\ \hfill & -\frac{1}{2\lambda }\left[\parallel \mathbf{y}-\mathbf{A}{\mathbf{\mu }}_{\mathbf{x}}{\parallel }_2^2+{\lambda }^{\left(pre\right)}[QNM-\mathrm{tr}\left({\mathsf{\Sigma }}_{\mathbf{x}}{\mathsf{\Sigma }}_0^{-1}\right)]\right],\hfill \end{array}$$

(25)

where ${\lambda }^{\left(pre\right)}$ denotes the estimation of $\lambda$ in the previous iteration. When we calculate the derivative of Equation (25) over $\lambda$ and set it equal to zero, $\lambda$ is obtained:

$$\lambda \leftarrow \frac{\parallel \mathbf{y}-\mathbf{A}{\mathbf{\mu }}_{\mathbf{x}}{\parallel }_2^2+{\lambda }^{\left(pre\right)}\left[QNM-\mathrm{tr}\left({\mathsf{\Sigma }}_{\mathbf{x}}{\mathsf{\Sigma }}_0^{-1}\right)\right]}{PN}.$$

(26)

To estimate $\mathsf{\Gamma }$ and $\mathbf{B}$, $\mathsf{\Gamma }=\mathrm{diag}({\gamma }_1^2,\cdots ,{\gamma }_M^2)$ is first assumed, where $\mathrm{diag}(·)$ denotes a diagonal matrix operator. Notice that only the second term in Equation (24) is related to $\mathsf{\Gamma }$ and $\mathbf{B}$. So, we can simplify the $\mathcal{Q}$ function (24) to

$$\mathcal{Q}(\mathsf{\Gamma },\mathbf{B})=E_{x|y;{\mathsf{\Theta }}^{\left(pre\right)}}[logp(\mathbf{x};\mathsf{\Gamma },\mathbf{B})],$$

(27)

Then, we have

$$\begin{array}{cc}\hfill \mathcal{Q}(\mathsf{\Gamma },\mathbf{B})\propto & -\frac{QN}{2}log\left(\right|\mathsf{\Gamma }\left|\right)-\frac{QM}{2}log\left(\right|\mathbf{B}\left|\right)\hfill \\ \hfill & -\frac{1}{2}\mathrm{tr}\left[({\mathsf{\Gamma }}^{-1}\otimes {\mathbf{B}}^{-1})({\mathsf{\Sigma }}_{\mathbf{x}}+{\mathbf{\mu }}_{\mathbf{x}}{\mathbf{\mu }}_{\mathbf{x}}^{\top })\right].\hfill \end{array}$$

(28)

To obtain the values of ${\gamma }_i$, we use the feature given in [20]. Specifically, the values of ${\gamma }_i$ corresponding to the larger singular values (the larger singular values are defined as singular values that are larger than 10% of the largest singular value) are 1; otherwise, they are 0. Figure 10 gives a schematic diagram of how ${\gamma }_i$ is obtained from HSI data. From Figure 10, we can find that only the first singular value is larger, so ${\gamma }_1=1$.

To estimate $\mathbf{B}$, ${\mathbf{\mu }}_{\mathbf{x}}$ and ${\mathsf{\Sigma }}_{\mathbf{x}}$ are plugged into Equation (28). So, we can obtain the gradients of Equation (28) over $\mathbf{B}$, and then we can obtain ${\mathbf{B}}^{\left(pre\right)}$.

$${\mathbf{B}}^{\left(pre\right)}\leftarrow \frac{1}{M}\sum _{i=1}^M\frac{\left({\mathsf{\Sigma }}_{\mathbf{x}}^i+{\mathbf{\mu }}_{\mathbf{x}}^i{\left({\mathbf{\mu }}_{\mathbf{x}}^i\right)}^{\top }\right)}{{\gamma }_i^2}.$$

(29)

Thus, we will get ${\mathsf{\Gamma }}^{\left(pre\right)}$. Using the same method, we can get ${\lambda }^{\left(pre\right)}$. Finally, we get ${\mathsf{\Theta }}^{\left(pre\right)}$. Here, ${\mathbf{A}}^{\left(pre\right)}$ denotes an initial value of $\mathbf{A}$.

In order to get a closed form of $\mathsf{\Theta }$, we employ standard upper bounds to solve Equation (21), which is known as a non-convex optimization problem leading to an EM-like algorithm. For the first and second terms of $\mathcal{L}(\mathsf{\Gamma },\mathbf{B})$, we compute their respective bounds.

For the first term in Equation (21), we can obtain

$${\mathbf{y}}^{\top }{\mathsf{\Sigma }}_y^{-1}\mathbf{y}\le \frac{1}{\lambda }{\parallel \mathbf{y}-\mathbf{Ax}\parallel }_2^2+{\mathbf{x}}^{\top }{\mathsf{\Sigma }}_0^{-1}\mathbf{x},$$

(30)

where equality is obtained when $\mathbf{x}$ satisfies Equation (18).

For the second term, we can obtain

$$\begin{array}{cc}\hfill \log |{\mathsf{\Sigma }}_y|& \equiv QM\log \left|\mathbf{B}\right|+\log |\lambda {\mathbf{A}}^{\top }\mathbf{A}+{\mathsf{\Sigma }}_0^{-1}|\hfill \\ \hfill & \le QM\log \left|\mathbf{B}\right|+\mathrm{tr}\left[{\mathbf{B}}^{-1}{\nabla }_{{\mathbf{B}}^{-1}}\right]+C,\hfill \end{array}$$

(31)

where, for the second term $\log |\lambda {\mathbf{A}}^{\top }\mathbf{A}+{\mathsf{\Sigma }}_0^{-1}|$, a first-order approximation is used to approximate it with a bias term C. The equality will hold when the gradient satisfies

$${\nabla }_{{\mathbf{B}}^{-1}}=\sum _{m=1}^M\mathbf{B}-\mathbf{B}{\mathbf{A}}_m^{\top }{(\mathbf{A}{\mathsf{\Sigma }}_0{\mathbf{A}}^{\top }+\lambda \mathbf{I})}^{-1}{\mathbf{A}}_m\mathbf{B},$$

(32)

where $\mathbf{A}=[{\mathbf{A}}_1,\cdots ,{\mathbf{A}}_M]$ and ${\mathbf{A}}_m\in {\mathbb{R}}^{QP\times QN}$. Finally, by using the upper bounds of Equations (30) and (31) and ${\nabla }_{{\mathbf{B}}^{-1}}$, we can obtain the optimal $\mathbf{B}$ in a closed form:

$${\mathbf{B}}^{opt}=arg\underset{\mathbf{x}}{min}{\mathbf{x}}^{\top }{\mathsf{\Sigma }}_0^{-1}\mathbf{x}+\mathrm{tr}\left[{\mathbf{B}}^{-1}{\nabla }_{{\mathbf{B}}^{-1}}\right]+QM\log \left|\mathbf{B}\right|.$$

(33)

Starting with $\mathbf{B}={\mathbf{B}}^{\left(pre\right)}$, we iteratively compute Equations (18), (32) and (33), then obtain an estimation for $\mathbf{B}$ and a corresponding estimation for $\mathbf{x}$ given by Equation (18). Here, the proposed method is outlined in Algorithm 1.

Algorithm 1 Proposed method

Source data analysis calculate singular values of ${\mathbf{X}}_q$ by using SVD; obtain ${\gamma }_i$, $i=1,\cdots ,M$ by using singular values; Input$\mathbf{y},\mathbf{A}$;  Output$\mathcal{X}$; Initialize    assume  $\mathsf{\Gamma }=\mathrm{diag}({\gamma }_1^2,\cdots ,{\gamma }_M^2)$; $iters=0,\delta ={10}^{-6}$; $\mathbf{max}\mathbf{iteration}\mathbf{number}=500$; Set $\lambda ,\mathbf{B}$ by $\lambda ={10}^{-10}$, $\mathbf{B}=ones(N,N)$; compute $\lambda ,\mathbf{B}$ from Equations (26) and (33); ${\mathsf{\Sigma }}_0\leftarrow {\mathbf{I}}_Q\otimes \mathsf{\Gamma }\otimes \mathbf{B}$; While$\parallel \mathcal{X}-\widehat{\mathcal{X}}{\parallel }_2^2\ge \delta$ compute $\widehat{\mathcal{X}}$ by Equation (18); compute ${\nabla }_{{\mathbf{B}}^{-1}}$ by Equation (32); compute ${\mathbf{B}}^{opt}$ by Equation (33); $iters=iters+1$; if  $iters\ge 500$        STOP; end if EndWhile Get the best ${\mathbf{B}}^{opt}$ and $\mathcal{X}$.

3.3. Simulation Experiments

In this section, we present the results with HSI data in order to compare the performance of the proposed method with that of the prior state-of-the-art LRMR [5] and SS&LR [22] methods. For HSI data, 100 continuous-time trials were run. In each trial, data of a $30\times 50$ two-dimensional area and 20 bands from the Salinas Database (available at http://www.ehu.eus/ccwintco/index.php/Hyperspectral_Remote_Sensing_Scenes#Salinas-A_scene (accessed on 10 November 2021)) and Indian Pines (available at http://www.ehu.eus/ccwintco/index.php/Hyperspectral_Remote_Sensing_Scenes#Indian_Pines (accessed on 10 November 2021)) were used. Thus, the data matrix was a $30\times 50\times 20$ matrix.

Figure 11 and Figure 12 plot the performance in terms of MSE versus SNR and runtime versus SNR, respectively. Here, $P=10$, so the ratio of compression is $P/NM=1/60$. In Figure 11a, we can observe that proposed method outperforms all of the other methods in terms of MSE; e.g., at $\mathrm{SNR}=20$ dB, we note that the proposed method achieves reconstruction gain at least 5 dB greater than those of the other methods. Figure 11b shows that the proposed method uses less runtime than the others. In Figure 12, we can observe that the effect of SS&LR is not good, but our proposed method works well and outperforms all of the other methods in terms of MSE; e.g., at $\mathrm{SNR}=15$ dB, we note that the proposed method achieves a reconstruction gain at least 6 dB greater than those of the other methods. The proposed method also uses less runtime than the others.

Figure 13 and Figure 14 compare all of the methods for different values of compression P. Figure 13a shows that most of the MSE curves fall as P increases. As the compression ratio $P/NM$ decreases (from $1/300$ to $1/60$), the reconstruction error becomes smaller, affecting the signal reconstruction performance. Fortunately, the proposed algorithm yields better performance in all cases and has a higher compression ratio with the same reconstruction performance. Figure 13b shows that the proposed method consumes less computation time at the same compression ratio. From Figure 14, we observe that the effects of SS&LR and LRMR are not good, but our proposed method outperforms all of the other methods in terms of MSE; e.g., at $P=6$, we note that the proposed method achieves a reconstruction gain at least 7 dB greater than those of the other methods. The proposed method also uses less runtime than the others.

Figure 15 shows some visual reconstruction results obtained with the three methods. As expected, the proposed method preserves more fine details and better visual results than the others.

Figure 16 only shows a performance comparison chart of the various algorithms from the perspective of data recovery, and it does not start from the perspective of data collection methods, as in the beginning of this section. It can be seen in Figure 16 that the proposed 3D joint reconstruction algorithm has better performance and less time consumption than the other algorithms. What needs to be explained here is that the three methods—the proposed 3D HSI method, L&S-bSBL (1) method, and L&S-bSBL (2) method—are, in terms of the compression ratio: $P/\left(NM\right)=1/60$, $P/\left(NQ\right)=1/150$, and $P/N=1/3$.

4. Discussion

From Section 3.3, the simulation results demonstrate that the proposed method is superior to LRMR and SS&LR methods in terms of reconstruction accuracy and computational burden under the same SNR and compression ratio. This is because LRMR only focuses on the reconstruction of low-rank signals, while SS&LR focuses on the reconstruction of low-rank and sparse signals, but it only uses the spatial relationship or interband relationship of HSI signals and does not combine the signal acquisition process.

Based on our present work, some issues that could be addressed in future research can be summarized as follows. One open problem is that of reducing, as much as possible, the computational complexity of the proposed method. In particular, unmanned aerial vehicles (UAVs) are used to monitor fields, such as in farmland monitoring, environmental monitoring, etc. While our approach has significant reconstruction performance, there is still a long way to go before we can apply it to future portable hardware. We can also take advantage of the properties of HSI data with higher-dimensional structures to design an effective Bayesian learning-based reconstruction method; namely, tensor models could be considered to compress and reconstruct HSI data. Furthermore, we may also consider the design of a sensor selection method for HSI acquisition that reduces the number of sensors in different locations that are needed so as to extend the life and energy savings of sensors.

5. Conclusions

In this work, based on the fast method proposed in our previous work, two methods for 2D compression and reconstruction of 3D HSI data are given. Then, starting from the HSI acquisition method, we analyzed the push-brooming acquisition of data slices in the hyperspectral image acquisition process, and by assuming that the HSI data met the L&S model, a joint CS compression and reconstruction method for spatial and spectral correlation was studied and an L&S structure was proposed. The proposed method combines data reconstruction with HSI data acquisition. The simulation results show that the proposed method has better performance than the other two existing methods and the two proposed 2D reconstruction methods.