Efficient Sparse Bayesian Learning Model for Image Reconstruction Based on Laplacian Hierarchical Priors and GAMP

Abstract

In this paper, we present a novel sparse Bayesian learning (SBL) method for image reconstruction. We integrate the generalized approximate message passing (GAMP) algorithm and Laplacian hierarchical priors (LHP) into a basic SBL model (called LHP-GAMP-SBL) to improve the reconstruction efficiency. In our SBL model, the GAMP structure is used to estimate the mean and variance without matrix inversion in the E-step, while LHP is used to update the hyperparameters in the M-step.The combination of these two structures further deepens the hierarchical structures of the model. The representation ability of the model is enhanced so that the reconstruction accuracy can be improved. Moreover, the introduction of LHP accelerates the convergence of GAMP, which shortens the reconstruction time of the model. Experimental results verify the effectiveness of our method.

1. Introduction

The Sparse Bayesian learning (SBL) model has been successfully applied to sparse signal recovery (SSR) or image recovery in various fields [1,2,3]. The essence of SSR is to restore the sparse signal $\mathit{x}\in {\mathbb{R}}^{N\times 1}$ from the $M<N$ noisy measurement vector $\mathit{y}\in {\mathbb{R}}^{M\times 1}$. In theory, the model is expressed as follows:

$$\mathit{y}=\mathcal{D}\mathit{x}+\mathit{e},$$

(1)

where $\mathcal{D}\in {\mathbb{R}}^{M\times N}$ is the known dictionary matrix and $\mathit{e}\in {\mathbb{R}}^{M\times 1}$ is the observation noise.

Accordingly, many methods [4,5,6,7] have been proposed to solve (1) that update parameters through the Expectation-Maximization (EM) [8] algorithm and achieve good reconstruction results. The EM algorithm has two steps. The first step is to calculate the expectation (E-step), which calculates the posterior distribution of hidden variables (the expectation of log-likelihood) based on current parameter estimates. The second step is maximization (M-step), which updates parameters by maximizing the expectation calculated in the E-step. To estimate mean and variance accurately (in the E-step) and hyperparameters (in the M-step), most SBL models involve operations of matrix inversion. Thus, the computational efficiencies of these methods may be inhibited.

To tackle this issue, multiple optimization strategies have been presented [9,10]. In [9], Shutin proposed a fast variational sparse Bayesian method (FV-SBL) to accelerate convergence by calculating the stationary points of variational iterative updates with noninformative hyperpriors to maximize the bound. Subsequently, Duan [10] presented a fast inverse-free SBL method (IF-SBL) based on maximizing the relaxed evidence lower bound (ELBO). The relaxed-ELBO is first obtained to avoid the operation of matrix inversion in the E-step. Then, the variational EM algorithm is used to maximize the relaxed-ELBO, which improves computational efficiency by using inverse-free matrix operations.

In recent studies [11,12,13,14], GAMP was used in SBL models to speed up the reconstruction. Based on the transformation of the estimation problem, Gaussian approximation, and central limit theorem, GAMP can effectively estimate parameters without matrix inversion in the E-step. In [11], GAMP was used to perform the inference through information transformation between nodes of the iterative probability graph model. It applies quadratic approximation and Taylor expansion to the cyclic confidence propagation with low computational complexity. Based on this, Zou [12] proposed combining GAMP with the SBL model (GAMP-SBL) and embedding this into the EM framework in place of the E-step, which significantly reduces the computational complexity and shortens the running time. Because the GAMP framework can adapt to any prior distribution for the unknown vectors, Al-Shoukairi [13] applied Gaussian scale mixture (GSM) priors to SBL sparse coefficients (GGAMP-SBL) to enhance the robustness of the model and adopted multiple damping factors to achieve fast convergence of GAMP, thereby improving the reconstruction efficiency. In [14], Dong formulated GAMP and variational Bayesian (VB) EM structure into an SBL model (VGAMP-SBL). By replacing the traditional EM model with a VB-EM model and using the GAMP structure to estimate the mean in the E-step, the interaction between data fluctuations and parameter fluctuations was weakened, thereby improving reconstruction performance.

However, multiple random variables in these GAMP-based SBL models may result in a fat-tailed distribution, which cannot be well fitted by the prior distribution of Gaussian mixture functions. Therefore, local optimization may occur during the iteration process, which means that more time is required to approximate hyperparameters in the M-step. In contrast, Laplace distribution has good robustness against a fat-tailed distribution [15]. It can easily extract samples far from the mean (abnormal samples). Thus, a Laplacian prior is more suitable for the reconstruction of natural images. In [16], Babacan integrated Laplace hierarchical priori (LHP) into the basic SBL frame to speed up reconstruction. LHP can improve the sparsity of unknown signals and obtain a lower reconstruction error.

Inspired by this, we integrate GAMP and LHP into the basic SBL model (LHP-GAMP-SBL) to improve the efficiency of image reconstruction, reduce computational complexity, shorten running time, and accelerate convergence. The main contributions are summarized as follows:

(1)

We propose an efficient sparse Bayesian learning model for image reconstruction based on Laplace hierarchical priors and GAMP. The GAMP structure is used to estimate the mean and variance in the E-step without matrix inversion. It outperforms several mainstream SBL image reconstruction models in terms of running time.

(2)

Sufficient damping factors are used to constrain the output to prevent the divergence of the GAMP during iterations.

(3)

In order to solve the problem that the prior distribution of the Gaussian mixture function in GAMP cannot fit a fat-tailed distribution well, we use the Laplace hierarchical prior (LHP) instead of the Gaussian prior to update the hyperparameters in the M-step, which improves the robustness of the model and speeds up the convergence of GAMP.

(4)

The combination of GAMP and LHP further deepens the hierarchical structure of the model and improves the sparsity of the model. The higher the sparsity of the model, the fewer non-zero elements it has, and the stronger the representation ability of the model (using as few non-zero elements as possible to represent the main characteristic information of the original signal). The reconstruction accuracy is improved without affecting the computational efficiency. A large number of experiments have shown that our LHP-GAMP-SBL model outperforms other mainstream SBL image reconstruction models in overall reconstruction performance.

2. Methods

2.1. Laplace Hierarchical SBL Model

The Laplace distribution has a peak at zero, so it is easier to produce sparse solutions, so that signal coefficients close to zero are preferred [15,16]. In contrast, the Gaussian distribution is smooth and has no peak at zero, so it is less capable of modeling sparsity. According to sparse representation (SR) theory, the purpose of SR is to represent the main characteristic information of the original signal with as few non-zero elements as possible, which can make it easier to further process the signal. When the sparsity of the model is higher, there are fewer non-zero elements. Therefore, the representation ability of the model will be enhanced, thereby improving reconstruction accuracy. In addition, the Laplace distribution is more robust to abnormal values because its tail is heavier than that of the Gaussian distribution [15]. This means that in the presence of noise or abnormal values, the reconstruction performance of the SBL model using the Laplace prior is better. Therefore, this paper will adopt the Laplace hierarchical prior model to improve reconstruction accuracy.

In (1), all unknown parameters are treated as random quantities with a specified probability distribution. The unknown signal $\mathit{x}$ is assigned a prior distribution $p\left(\mathit{x}\right|\lambda )$, and the observation $\mathit{y}$ is a random process with conditional distribution $p\left(\mathit{y}\right|\mathit{x},\theta )$, where $\theta =1/{\sigma }^2$ is the inverse noise variance. These distributions depend on the model parameters, $\lambda$ and $\theta$, which can also be assigned additional prior distributions. The SBL model defines the joint distribution of all parameters and observations as follows:

$$p(\mathit{x},\lambda ,\theta ,\mathit{y})=p\left(\mathit{y}\right|\mathit{x},\theta \left)p\right(\mathit{x}\left|\lambda \right)p\left(\lambda \right)p\left(\theta \right).$$

(2)

Let noise $\mathit{e}$ follow an independent Gaussian distribution with zero mean and ${\theta }^{-1}$ variance. Thus, one has:

$$p\left(\mathit{y}\right|\mathit{x},\theta )=\mathcal{N}\left(\mathit{y}\right|\mathcal{D}\mathit{x},{\theta }^{-1}).$$

(3)

Let a Gamma prior be placed on $\theta$, we have:

$$p\left(\theta \right|a^{\theta },b^{\theta })=\mathrm{\Gamma }\left(\theta \right|a^{\theta },b^{\theta })={\displaystyle \frac{{\left(b^{\theta }\right)}^{a^{\theta }}}{\mathrm{\Gamma }\left(a^{\theta }\right)}}{\theta }^{a^{\theta }-1}exp(-b^{\theta }\theta ).$$

(4)

where $\theta >0$ represents the hyperparameter, a is the scale parameter, and b is the shape parameter. The expectation of $\theta$ is given by the following equation.

$$\langle \theta \rangle ={\displaystyle \frac{a^{\theta }}{b^{\theta }}}.$$

(5)

where $\langle ·\rangle$ denotes the expectation. Similarly, based on the sparsity of $\mathit{x}$ in (1), $\mathit{x}$ is assumed to follow the distribution of Laplace priori, that is:

$$p\left(\mathit{x}\right|\lambda )={\displaystyle \frac{\lambda }{2}}exp(-{\displaystyle \frac{\lambda }{2}}{\parallel \mathit{x}\parallel }_1).$$

(6)

However, the likelihood function in (3) and the prior probability in (6) are not conjugate. When they are conjugate, the posterior distribution and the prior probability are the same distribution. A conjugate prior can simplify the calculation of the posterior distribution. Therefore, we adopt the hierarchical prior model [16]. The first layer of the model is:

$$p\left(\mathit{x}\right|\omega )=\prod _{i=1}^N\mathcal{N}\left({\mathit{x}}_i\right|0,{\omega }_i).$$

(7)

where $\omega =({\omega }_1,{\omega }_2,\dots ,{\omega }_N)$. The second layer of the hierarchical model (${\omega }_i$) is:

$$p\left({\omega }_i\right|\lambda )=\mathrm{\Gamma }\left({\omega }_i\right|1,\lambda /2)={\displaystyle \frac{\lambda }{2}}exp(-{\displaystyle \frac{\lambda {\omega }_i}{2}}),{\omega }_i,\lambda \ge 0.$$

(8)

Based on (7) and (8), the distribution of Laplacian priors can be expressed by:

$$\begin{array}{cc}\hfill p\left(\mathit{x}\right|\lambda )& =\int p\left(\mathit{x}\right|\omega )p\left(\omega \right|\lambda )d\omega =\prod _i\int p\left({\mathit{x}}_i\right|{\omega }_i)p\left({\omega }_i\right|\lambda )d{\omega }_i\hfill \\ \hfill & ={\displaystyle \frac{{\lambda }^{N/2}}{2^N}}exp(-\sqrt{\lambda }\sum _i|{\mathit{x}}_i\left|\right).\hfill \end{array}$$

(9)

Finally, the parameter $\lambda$ is calculated by the Gamma hyperprior method.

$$p\left(\lambda \right|\gamma )=\mathrm{\Gamma }\left(\lambda \right|{\displaystyle \frac{\gamma }{2}},{\displaystyle \frac{\gamma }{2}}).$$

(10)

2.2. Laplace Hierarchical Bayesian Inference

According to Bayesian inference based on the maximum likelihood method, the posterior distribution can be written as:

$$p(\mathit{x},\omega ,\lambda ,\theta |\mathit{y})={\displaystyle \frac{p\left(\mathit{y}\right|\mathit{x},\theta \left)p\right(\mathit{x}\left|\omega \right)p\left(\omega \right|\lambda \left)p\right(\lambda \left)p\right(\theta )}{p\left(\mathit{y}\right)}}.$$

(11)

where $p\left(\mathit{y}\right)$ represents the marginal likelihood function, which can be expressed as:

$$\begin{array}{cc}\hfill p\left(\mathit{y}\right)& =\int p(\mathit{y}|\mathit{x},\theta )p(\mathit{x}|\omega )p(\omega |\lambda )p(\lambda )p(\theta )d\mathit{x}\hfill \\ \hfill & =({\displaystyle \frac{1}{2\pi }}{)}^{N/2}{\left|\mathit{C}\right|}^{-1/2}exp(-{\displaystyle \frac{1}{2}}{\mathit{y}}^T{\mathit{C}}^{-1}\mathit{y})p\left(\omega \right|\lambda )p\left(\lambda \right)p\left(\theta \right).\hfill \end{array}$$

(12)

where $\mathit{C}=({\theta }^{-1}\mathit{I}+\mathcal{D}{\mathsf{\Lambda }}^{-1}{\mathcal{D}}^T)$ and $\mathsf{\Lambda }=diag(1/{\omega }_i)$, $\mathit{I}$ represent the $M\times M$ identity matrix.

According to the properties of the Gaussian function, it can be obtained that the posterior distribution also approximately obeys the Gaussian distribution. Its posterior mean (${\mathit{\mu }}_{\mathit{x}}$), variance (${\mathsf{\Sigma }}_{\mathit{x}}$) of $\mathit{x}$, and hyperparameters ($\omega$, $\lambda$, $\theta$) can be iteratively updated by the EM algorithm, as summarized below:

$${\mathit{\mu }}_{\mathit{x}}=\theta {\mathsf{\Sigma }}_{\mathit{x}}{\mathcal{D}}^T\mathit{y},{\mathsf{\Sigma }}_{\mathit{x}}={(\theta {\mathcal{D}}^T\mathcal{D}+\mathsf{\Lambda })}^{-1}.$$

(13)

$$\begin{array}{cc}\hfill {\omega }_i& ={\displaystyle \frac{\sqrt{1+4\lambda \langle {\mathit{x}}_i^2\rangle }-1}{2\lambda }},\lambda ={\displaystyle \frac{2(N-1)+\gamma }{{\sum }_i{\omega }_i+\gamma }},\hfill \\ \hfill \theta & ={\displaystyle \frac{N/2+a^{\theta }}{{\langle \parallel \mathit{y}-\mathcal{D}\mathit{x}\parallel }^2\rangle /2+b^{\theta }}}.\hfill \end{array}$$

(14)

In addition, we can also estimate $\gamma$ by maximizing the value of the logarithm of (12) with respect to $\gamma$. The solution formula is shown in (15):

$$\begin{array}{cc}\hfill lnp\left(\mathit{y}\right)& =-{\displaystyle \frac{1}{2}}ln\left|\mathit{C}\right|-{\displaystyle \frac{1}{2}}{\mathit{y}}^T{\mathit{C}}^{-1}\mathit{y}+Nln{\displaystyle \frac{\lambda }{2}}-{\displaystyle \frac{\lambda }{2}}\sum _i{\omega }_i+{\displaystyle \frac{\gamma }{2}}ln{\displaystyle \frac{\gamma }{2}}\hfill \\ \hfill & -ln\mathrm{\Gamma }\left({\displaystyle \frac{\gamma }{2}}\right)+({\displaystyle \frac{\gamma }{2}}-1)ln\lambda -{\displaystyle \frac{\gamma }{2}}\lambda +(a^{\theta }-1)log\theta -b^{\theta }\theta ,\hfill \\ \hfill & \Rightarrow ln{\displaystyle \frac{\gamma }{2}}+1-\psi \left({\displaystyle \frac{\gamma }{2}}\right)+ln\lambda -\lambda =0.\hfill \end{array}$$

(15)

where $\psi (·)$ is the digamma function, which is the derivative of gamma function $\mathrm{\Gamma }(·)$ with respect to the logarithm of the independent variable. In (15), $\psi \left({\displaystyle \frac{\gamma }{2}}\right)$ is the derivative of $ln\mathrm{\Gamma }\left({\displaystyle \frac{\gamma }{2}}\right)$ with respect to ${\displaystyle \frac{\gamma }{2}}$.

It can be seen that (15) is a nonlinear equation, which is complicated by the existence of the digamma function $\psi (·)$. The digamma function has no simple analytical form, so (15) cannot be solved directly by simple algebraic manipulation. In order to solve this equation, numerical methods are usually required.

Unlike the method in [16], we use two layers of hierarchical priors. There are two reasons for this. First, the use of the LHP structure in the SBL model can accelerate the convergence of GAMP. Second, there is no direct solution in (15), and the last layer contributes little to reducing the recovery error, while more layers require more computing resources. It can be seen that the update of the posterior mean and covariance in (13) involves hyperparameters such as $\theta ,\omega$, and $\lambda$, which need to be updated through (14). Therefore, through continuous iterative updates of (13) and (14), it will eventually converge to an estimate of $\mathit{x}$.

2.3. LHP-GAMP-SBL Model

The GAMP algorithm is based on the channel coding and message propagation principles of information theory. Through the transformation of the estimation problem, Gaussian approximation and the use of central limit theorem, it can effectively approximate the parameters ${\mathit{\mu }}_{\mathit{x}}$ and ${\mathsf{\Sigma }}_{\mathit{x}}$ solved in E-step of the SBL model shown in (13). In addition, it avoids the matrix inversion, greatly improving reconstruction efficiency.

In our LHP-GAMP-SBL model, GAMP is used to replace (13) in the E-step, while LHP is used to update (14) in the M-step. The structure of the LHP-GAMP-SBL model is shown in Figure 1.

In Figure 1, $\mathit{k}\in {\mathbb{R}}^{M\times 1}$ represents the linear transformation of n-dimensional variable $\mathit{x}\in {\mathbb{R}}^{N\times 1}$, i.e.,

$$\mathit{k}=\mathcal{D}\mathit{x}.$$

(16)

For the given prior distribution $p\left(\mathit{x}\right|\omega )$ and marginal likelihood function $p\left(\mathit{y}\right|\mathit{k})$, GAMP constructs iterative expressions by quadratic approximation and Taylor series expansions. The detailed derivation of GAMP can be found in [11]. Specifically, during the process of message propagation in Figure 1, the input function $G_{\mathit{k}}(\overline{\mathit{x}},{\mathit{v}}_{\overline{\mathit{k}}})$ and the output function $G_{\mathit{x}}(\overline{\mathit{x}},{\mathit{v}}_{\overline{\mathit{x}}})$ can be defined as:

$$\begin{array}{cc}\hfill {\left[G_{\mathit{k}}(\overline{\mathit{k}},{\mathit{v}}_{\overline{\mathit{k}}})\right]}_m& ={\displaystyle \frac{\int {\mathit{k}}_{m}p\left({\mathit{y}}_m\right|{\mathit{k}}_m)\mathcal{N}({\mathit{k}}_m;\frac{{\overline{\mathit{k}}}_m}{{\mathit{v}}_{{\overline{\mathit{k}}}_m}},\frac{1}{{\mathit{v}}_{{\overline{\mathit{k}}}_m}})d{\mathit{k}}_m}{\int p\left({\mathit{y}}_m\right|{\mathit{k}}_m)\mathcal{N}({\mathit{k}}_m;\frac{{\overline{\mathit{k}}}_m}{{\mathit{v}}_{{\overline{\mathit{k}}}_m}},\frac{1}{{\mathit{v}}_{{\overline{\mathit{k}}}_m}})d{\mathit{k}}_m}},\hfill \end{array}$$

(17)

$$\begin{array}{cc}\hfill G_{\mathit{x}}(\overline{\mathit{x}},{\mathit{v}}_{\overline{\mathit{x}}}){]}_n& ={\displaystyle \frac{\int {\mathit{x}}_{n}p\left({\mathit{x}}_n\right|{\omega }_n)\mathcal{N}({\mathit{x}}_n;{\overline{\mathit{x}}}_n,{\mathit{v}}_{{\overline{\mathit{x}}}_n})d{\mathit{x}}_n}{\int p\left({\mathit{x}}_n\right|{\omega }_n)\mathcal{N}({\mathit{x}}_n;{\overline{\mathit{x}}}_n,{\mathit{v}}_{{\overline{\mathit{x}}}_n})d{\mathit{x}}_n}}.\hfill \end{array}$$

(18)

where $m\in M$ represents the m-th atom of the input function, $n\in N$ represents the n-th atom of the output function, the intermediate variables $\overline{\mathit{x}}$ and $\overline{\mathit{k}}$ represent the approximation estimation of the sparse signal $\mathit{x}$ and the noise-free model $\mathit{k}$, respectively, and ${\mathit{v}}_{\overline{\mathit{x}}}$ and ${\mathit{v}}_{\overline{\mathit{x}}}$ correspond to variances of $\mathit{x}$ and $\mathit{k}$, respectively. Based on (3) and (7), the update rules of the input function and output function can be obtained by simplifying (17) and (18):

$$\begin{array}{cccc}\hfill G_{\mathit{k}}(\overline{\mathit{k}},{\mathit{v}}_{\overline{\mathit{k}}})& ={\displaystyle \frac{\overline{\mathit{k}}/{\mathit{v}}_{\overline{\mathit{k}}}-\mathit{y}}{1/\theta +1/{\mathit{v}}_{\overline{\mathit{k}}}}},\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hfill & \hfill G_{\mathit{k}}^{\prime }(\overline{\mathit{k}},{\mathit{v}}_{\overline{\mathit{k}}})& ={\displaystyle \frac{\theta }{\theta +{\mathit{v}}_{\overline{\mathit{k}}}}},\hfill \end{array}$$

(19)

$$\begin{array}{cccc}\hfill G_{\mathit{x}}(\overline{\mathit{x}},{\mathit{v}}_{\overline{\mathit{x}}})& ={\displaystyle \frac{\omega }{\omega +{\mathit{v}}_{\overline{\mathit{x}}}}}\overline{\mathit{x}},\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hfill & \hfill G_{\mathit{x}}^{\prime }(\overline{\mathit{x}},{\mathit{v}}_{\overline{\mathit{x}}})& ={\displaystyle \frac{\omega }{\omega +{\mathit{v}}_{\overline{\mathit{x}}}}}.\hfill \end{array}$$

(20)

Usually, the damped factors, ${\eta }_{\overline{\mathit{k}}},{\eta }_{\overline{\mathit{x}}}\in (0,1]$, are introduced to ensure the convergence in GAMP. The two damped factors correspond to two square roots in (21), respectively.

$$F({\eta }_{\overline{\mathit{k}}},{\eta }_{\overline{\mathit{x}}})={\displaystyle \frac{\alpha {\mathrm{\Omega }}_{max}}{{\mathrm{\Sigma }}_{\mathrm{\Omega }}}}{\eta }^2+\beta \eta +const,$$

(21)

where ${\mathrm{\Omega }}_{max}$ represents the maximum singular value of dictionary $\mathcal{D}$, and ${\mathrm{\Sigma }}_{\mathrm{\Omega }}$ represents the sum of singular values of $\mathcal{D}$. Both $\alpha$ and $\beta$ are constant coefficients. The damped factor ${\eta }_{\overline{\mathit{k}}}$ is used to constrain $\overline{\mathit{s}}$ to enhance convergence and make it easier to obtain the estimate of $\mathit{x}$ through continuous iterations. The damping factor ${\eta }_{\overline{\mathit{x}}}$ is used to control the sparse signal $\mathit{x}$.

Algorithm 1 shows the detailed process of the LHP-GAMP-SBL algorithm.

Algorithm 1: LHP-GAMP-SBL Algorithm

In Algorithm 1, $\left|\mathcal{D}\right|$ represents the component wise magnitude squared of D; the variable $\overline{\mathit{x}}$ represents the approximate estimate of the sparse signal $\mathit{x}$ with variance ${\mathit{v}}_{\overline{\mathit{x}}}$; and $\overline{\mathit{k}}$ represents the approximate estimate of the noise-free model $\mathit{k}=\mathcal{D}\mathit{x}$ with variance ${\mathit{v}}_{\overline{\mathit{k}}}$. The parameters $Ite{r}_{em}$ and ${\epsilon }_{em}$ represent the maximum number of iterations and the normalized tolerance parameter of the EM algorithm, respectively. $Ite{r}_{gamp}$ and ${\epsilon }_{gamp}$ represent the maximum number of iterations and the normalized tolerance parameter of the GAMP algorithm, respectively. These parameters are obtained through continuous optimization.

Overall, the computational complexities of IF-SBL, SBL-EM, and Laplace-SBL are, respectively, $\mathcal{O}\left(M{N}^2\right)$, $\mathcal{O}\left(N^3\right)$, and $\mathcal{O}\left(N^3\right)$. In contrast, GGAMP-SBL, VGAMP-SBL, and LHP-GAMP-SBL have the same computational complexity, $\mathcal{O}\left(MN\right)$. In most SBL models, mean and variance are estimated in the E-step, and hyperparameters are updated in the M-step.

Table 1. Computational complexity of E-step and M-step for different SBL models.

Methods	E-STEP	M-STEP
IF-SBL [10]	$\mathcal{O}\left(M{N}^2\right)$	$\mathcal{O}\left(MN\right)$
SBL-EM [3]	$\mathcal{O}\left(N^3\right)$	$\mathcal{O}\left(MN\right)$
Laplace-SBL [16]	$\mathcal{O}\left(N^3\right)$	$\mathcal{O}\left(N\right)$
GGAMP-SBL [13]	$\mathcal{O}\left(MN\right)$	$\mathcal{O}\left(MN\right)$
VGAMP-SBL [14]	$\mathcal{O}\left(MN\right)$	$\mathcal{O}\left(MN\right)$
LHP-GAMP-SBL	$\mathcal{O}\left(MN\right)$	$\mathcal{O}\left(N\right)$

lists respective computational complexities of the E-step and M-step for different SBL models. By using GAMP to estimate mean and variance in the E-step and using LHP to update hyperparameters in the M-step, our LHP-GAMP-SBL model decreases the computational complexity significantly.

3. Results and Discussion

The public DIOR [17] and DOTA [18] optical remote sensing image datasets and UC Merced Land Use(UCM) [19] remote sensing image datasets are used to verify the effectiveness of our method. In order to highlight different types of ground object information in remote sensing images and reduce the influence of factors such as atmosphere and illumination, the intensity information of each band was extracted in the experiment, and the amplitude map of the remote sensing image was generated to facilitate subsequent image reconstruction.

In our experiments, $\mathcal{D}$ is initialized as a Gaussian random matrix, $Ite{r}_{em}$ is set as 5000, $Ite{r}_{gamp}$∈$[5,20]$. ${\lambda }^0$, ${\epsilon }_{gamp}$, and ${\epsilon }_{em}$ are set as 1 × 10⁻⁶, 1 × 10⁻⁴, and 3 × 10⁻⁵, respectively. The value of hyperparameter $\theta$ is empirically set as 0.01${\parallel \mathit{y}\parallel }_2^2$. Five state-of-the-art methods, namely, SBL-EM [3], Laplace-SBL [16], IF-SBL [10], GGAMP-SBL [13], and VGAMP-SBL [14], are utilized for comparison. The parameters in these methods are selected for optimal results. Normalized root-mean-square error (ERROR) [13], reconstruction time (TIME), peak signal-to-noise ratio (PSNR), and structural similarity (SSIM) are used to measure the performance of the different methods. Error is defined by $Error=\parallel \overline{S}{-S\parallel }_F^2/{\parallel S\parallel }_F^2$, where $\overline{S}$ and S are estimated and real model parameters, respectively.

Forty images are selected randomly from each dataset (a total of 120 images) and used for experiments. These images are resized to $256\times 256$ in pixels. They are then sparsely processed by using wavelet transform before optimization. The number of compressed samples and wavelet coefficients are 10,672 and 16,320, respectively. The programs are written in Matlab2017b and run in Windows 10 on a Lenovo Legion R9000X 2021 laptop with an NVIDIA GeForce RTX 3060 Laptop GPU, AMD Ryzen 7 5800H CPU, and 16GB RAM.

For the different SBL models, the reconstructed images of the three datasets are shown in Figure 2. Figure 2a–c represent two examples of 40 experimental images in each of the three image datasets DIOR, DOTA, and UCM. From left to right, they represent the original image and the reconstructed images of IF-SBL, SBL-EM, Laplace-SBL, GGAMP-SBL, VGAMP-SBL, and the LHP-GAMP-SBL algorithm we proposed. The datas in brackets represent the reconstruction error (ERROR).

We can see that the three reconstruction methods based on GAMP have better results. The image reconstructed by the IF-SBL method is fuzzy, and it is difficult to extract valid information from the reconstructed image. To further compare the performance differences between different methods, we show the reconstructed ERROR and reconstructed TIME in detail in

Table 2. ERROR and TIME of different SBL models for the three datasets.

Methods	Error (×10−2)			TIME (s)
	A	B	C	A	B	C
IF-SBL [10]	16.54	17.51	20.55	458.2	449.2	431.2
SBL-EM [3]	10.66	12.28	12.89	978.2	962.4	1083.4
Laplace-SBL [16]	11.03	13.48	13.01	281.9	290.2	335.0
GGAMP-SBL [13]	9.49	11.87	11.53	1322.0	1414.6	1517.5
VGAMP-SBL [14]	9.04	11.56	10.78	578.1	764.5	600.1
LHP-GAMP-SBL	8.73	11.10	10.36	259.0	276.3	273.9

A (DIOR Dataset) B (DOTA Dataset) C (UCM Dataset).

. Note that each value in Table 2 denotes the average value of the results for 120 images. As we see, our LHP-GAMP-SBL model achieves better reconstruction accuracies than other methods. The average ERROR values for the three datasets are, respectively, 0.0873, 0.1110, and 0.1036, which are lower than these of the GGAMP-SBL and VGAMP-SBL methods. In terms of TIME, our method has a larger advantage than the other GAMP-based models. By using Laplace prior to estimate hyperparameters in the M-step, our model can reconstruct the images with fewer iterations. This improves the reconstruction efficiency significantly.

In addition, Table 2 shows that the IF-SBL method obtains the lowest reconstruction accuracies. The main reason is that IF-SBL fails to obtain the optimal ELBO because of the use of a smoothing function in complex images.

Figure 3 shows the ERROR, TIME, PSNR, and SSIM results of 20 images (selected randomly) for different methods. From Figure 3a, the three GAMP-based SBL models have better reconstruction accuracies than the Laplace-SBL model. The best results are recorded by our LHP-GAMP-SBL model. Figure 3b plots the running time of different methods. By using GAMP to estimate mean and variance in the E-step and by employing LHP to update hyperparameters in the M-step, our method obtains better reconstruction efficiency than the GGAMP-SBL and VGAMP-SBL models. From Figure 3c,d, it can be seen that our proposed algorithm results in higher PSNR as well as SSIM. Combined with Figure 4, we can see that an algorithm with a higher PSNR value can obtain a lower ERROR and a higher SSIM.

Figure 4 shows the average convergence processes of 20 images per iteration for different methods. Figure 4a shows that our LHP-GAMP-SBL method requires about 50 iterations to complete convergence and with low ERROR. In contrast, VGAMP-SBL and GGAMP-SBL require approximately 100 iterations to produce the desired results. Surprisingly, Laplace-SBL requires over 500 iterations to achieve image reconstruction. This is because the Laplace-SBL model employs a fast decision rule to determine whether hyperparameters in the M-step are updated or not. Accordingly, some small sparse solutions are discarded by a specific condition. This method achieves high reconstruction efficiency at the cost of restoration quality. This method still costs less time to terminate iterations, compared with the GGAMP-SBL and VGAMP-SBL methods. Figure 4b confirms this. From Table 1, we know that the three GAMP-based SBL models have the same computational complexities. Because VGAMP-SBL and LHP-GAMP-SBL have better sparsities in SBL models than GGAMP-SBL, thus, the former two methods require less computation resources. This means that the VGAMP-SBL and LHP-GAMP-SBL methods cost less running time per iteration, as shown in Figure 4b.

According to the experimental results, it can be seen that our LHP-GAMP-SBL model greatly reduces the computational complexity through the use of the GAMP structure, and gives the model better convergence through appropriate damping factors. Due to the introduction of the LHP structure, it has better sparsity compared to the existing methods and accelerates the convergence of GAMP. Therefore, the combination of the two structures results in a better reconstruction performance of the LHP-GAMP-SBL model.

It should be noted that our experiments are performed for optical images with wavelet transform before training, and the experimental results proved the effectiveness of the LHP-GAMP-SBL model. We also used SAR images for reconstruction, and the results show that the LHP-GAMP-SBL model is still effective for some sparse scenes.

4. Conclusions

This paper proposes an efficient SBL image reconstruction model based on Laplace hierarchical prior (LHP) and GAMP, called the LHP-GAMP-SBL model. We integrate the two structures of GAMP and LHP into the basic SBL model to improve the reconstruction efficiency. In our LHP-GAMP-SBL model, the GAMP structure is used to estimate mean and variance without matrix inversion in the E-step, while the LHP structure is used to update the hyperparameters in the M-step. In the LHP-GAMP-SBL model, the GAMP structure is used to estimate the posterior mean and variance in the E-step without matrix inversion, which greatly reduces the computational complexity, and introduces a damping factor to enhance convergence. The LHP structure is used to update the hyperparameters in the M-step, and the high sparsity of the Laplace prior is used to improve the reconstruction accuracy. In addition, the introduction of LHP accelerates the convergence of GAMP, thereby shortening the reconstruction time of the model. The combination of these two structures further deepens the hierarchical structure of the model, so that the overall reconstruction performance of the model is enhanced. All experimental results verify the effectiveness of our LHP-GAMP-SBL model.

Future work will focus on further improving the reconstruction accuracy by using convex optimization strategies instead of the EM algorithm to update the hyperparameters in the M-step or by combining the LHP-GAMP-SBL model with a neural network.