Automatic Tempered Posterior Distributions for Bayesian Inversion Problems

Abstract

We propose a novel adaptive importance sampling scheme for Bayesian inversion problems where the inference of the variables of interest and the power of the data noise are carried out using distinct (but interacting) methods. More specifically, we consider a Bayesian analysis for the variables of interest (i.e., the parameters of the model to invert), whereas we employ a maximum likelihood approach for the estimation of the noise power. The whole technique is implemented by means of an iterative procedure with alternating sampling and optimization steps. Moreover, the noise power is also used as a tempered parameter for the posterior distribution of the the variables of interest. Therefore, a sequence of tempered posterior densities is generated, where the tempered parameter is automatically selected according to the current estimate of the noise power. A complete Bayesian study over the model parameters and the scale parameter can also be performed. Numerical experiments show the benefits of the proposed approach.

1. Introduction

The estimation of unknown parameters from noisy observations is an essential problem in signal processing, statistics, and machine learning [1,2,3]. Within the Bayesian signal processing framework, these problems are addressed by constructing posterior probability distributions of the unknowns. Given the posterior, one often wants to make inference about the unknowns, e.g., if we are estimating parameters, finding the values that maximize their posterior, or the values that minimize some cost function given the uncertainty of the parameters. Unfortunately, obtaining closed-form solutions, usually expressed as integrals of the posterior, is infeasible in most practical applications. Therefore, developing approximate computational techniques, such as importance sampling and Markov chain Monte Carlo (MCMC) algorithms, is often required [4,5,6].

The so-called tempering of the posterior distributions is a well-known procedure for improving the performance of Monte Carlo (MC) algorithms [7,8,9,10]. Tempering is performed by modulating an artificial scale parameter or by sequentially including new data. There are several reasons for the improvement in performance: improving mixing, discovering modes, fostering the exploration of the inference space, etc. In the first iterations of the MC scheme, a posterior density with a bigger scale is considered. The artificial scale parameter (often called temperature) is reduced along the iterations, until considering the true posterior distribution. However, the user should select a temperature schedule, i.e., a decreasing rule for the scale parameter, which is usually chosen in an heuristic way [4,5]. In the literature, the tempering procedure has gained particular attention for the estimation of the marginal likelihood (also known as Bayesian model evidence) [9,11,12].

Furthermore, the joint inference of parameters (denoted as $\mathbf{\theta }$) of observation models, $\mathbf{f}\left(\mathbf{\theta }\right)$, and scale parameters of the likelihood function (that, in the scalar case, is usually denoted as $\sigma$) can be a hard task. Indeed, “wrong choices” of $\sigma$ values can easily jeopardize the sampling of $\mathbf{\theta }$. In this work, we introduce a procedure to tackle this problem.

To be specific, in this work, we design an adaptive importance sampling (AIS) scheme [13] for Bayesian inversion problems, where an automatic tempering procedure is implemented. We assume that the vector of observations $\mathbf{y}$ is obtained by a nonlinear transformation $\mathbf{f}\left(\mathbf{\theta }\right)$ of the variables of interest $\mathbf{\theta }$, perturbed by additive Gaussian noise with unknown power ${\sigma }^2$. The nonlinear mapping $\mathbf{f}\left(\mathbf{\theta }\right)$ usually represents a complex physical model, a computer code, etc. The resulting posterior densities are usually highly multimodal and complex distributions. Furthermore, the inference task in the joint space $[\mathbf{\theta },\sigma ]$ is particularly challenging. We introduce a split strategy to tackle this problem, involving an optimization approach over $\sigma$ and a sampling scheme for $\mathbf{\theta }$. More specifically, we design an iterative procedure where these two tasks are alternated. Additionally, the current maximum likelihood (ML) estimate of the noise power, ${\widehat{\sigma }}_{\mathtt{ML}}^2$, is employed as a tempering parameter, starting from high values and then “cooling down” according to the ML estimates at each iteration. Therefore, the proposed scheme deals with a sequence of tempered posteriors according to the current estimation ${\widehat{\sigma }}_{\mathtt{ML}}^2$. It is important to observe that, given a fixed vector $\mathbf{\theta }$, the ML estimator ${\widehat{\sigma }}_{\mathtt{ML}}^2$ can be obtained analytically.

Furthermore, the complete Bayesian analysis regarding the joint posterior of $\mathbf{\theta }$ and $\sigma$ is also possible (as discussed in Section 5). This is obtained by implementing a proper re-weighting of the samples generated by the proposed algorithm, called Automatic Tempering AIS (ATAIS), without any additional evaluations of the observation model. An approximation of the marginal posterior of $\sigma$ is provided as well. The advantages of the proposed scheme are shown in two numerical experiments, one of them considering a complex astronomical model.

2. Problem Statement

Let us denote the observed measurements as $\mathbf{y}={[y_1,\dots ,y_K]}^{\mathsf{\top }}\in {\mathbb{R}}^K$, and the variable of interest that we wish to infer as $\mathbf{\theta }={[{\theta }_1,\dots ,{\theta }_M]}^{\mathsf{\top }}\in \mathsf{\Theta }\subseteq {\mathbb{R}}^M$. Furthermore, let us assume the observation model

$$\mathbf{y}=\mathbf{f}\left(\mathbf{\theta }\right)+\mathbf{v},$$

(1)

where we have a nonlinear mapping,

$$\begin{array}{c}\hfill \mathbf{f}\left(\mathbf{\theta }\right)={[f_1\left(\mathbf{\theta }\right),\dots ,f_K\left(\mathbf{\theta }\right)]}^{\mathsf{\top }}:\mathsf{\Theta }\to {\mathbb{R}}^K\mathrm{with}\mathsf{\Theta }\subseteq {\mathbb{R}}^M,\end{array}$$

(2)

and a Gaussian perturbation noise,

$$\begin{array}{c}\hfill \mathbf{v}={[v_1,\dots ,v_K]}^{\mathsf{\top }}\sim \mathcal{N}\left(\mathbf{v}\right|\mathbf{0},{\sigma }^2{\mathbf{I}}_K),\end{array}$$

(3)

with $\sigma >0$, and ${\mathbf{I}}_K$ denotes the K-dimensional identity matrix. The model can be easily extended to a matrix of observations $\mathbf{Y}={[{\mathbf{y}}_1,\dots ,{\mathbf{y}}_K]}^{\mathsf{\top }}\in {\mathbb{R}}^{d_y\times K}$ instead of a vector, if the nonlinear mapping is of type $\mathbf{F}\left(\mathbf{\theta }\right)={[{\mathbf{f}}_1\left(\mathbf{\theta }\right),\dots ,{\mathbf{f}}_K\left(\mathbf{\theta }\right)]}^{\mathsf{\top }}:\mathsf{\Theta }\subseteq {\mathbb{R}}^M\to {\mathbb{R}}^{d_y\times K}$. The noise variance ${\sigma }^2$ is unknown, in general. The mapping $\mathbf{f}\left(\mathbf{\theta }\right)$ may be analytically unknown: the only assumption is that we are able to evaluate it pointwise. The likelihood function is

$$\begin{array}{ccc}\hfill \ell \left(\mathbf{y}\right|\mathbf{\theta },\sigma )& =& \frac{1}{{\left(2\pi {\sigma }^2\right)}^{K/2}}exp\left(-\frac{1}{2{\sigma }^2}\left|\right|\mathbf{y}-{\mathbf{f}\left(\mathbf{\theta }\right)\left|\right|}^2\right),\hfill \end{array}$$

(4)

$$\begin{array}{ccc}& =& \frac{1}{{\left(2\pi {\sigma }^2\right)}^{K/2}}exp\left(-\frac{1}{2{\sigma }^2}\sum _{k=1}^K{(y_k-f_k\left(\mathbf{\theta }\right))}^2\right).\hfill \end{array}$$

(5)

Note that we have two types of variables of interest: the vector $\mathbf{\theta }$ contains the parameters of the nonlinear mapping $\mathbf{f}\left(\mathbf{\theta }\right)$, whereas $\sigma$ is a scale parameter of the likelihood function.

Given the vector of measurements $\mathbf{y}$, we wish to make inferences regarding the hidden parameters $\mathbf{\theta }$ and the noise power ${\sigma }^2$, obtaining at least some point estimators $\widehat{\mathbf{\theta }}$ and ${\widehat{\sigma }}^2$. We are also interested in performing uncertainty and correlation analysis for the components of $\mathbf{\theta }$. Furthermore, we aim to perform model selection, i.e., to compare, select, or properly average different models.

Bayesian inference in the complete space. We consider independent prior densities $g_{\theta }\left(\mathbf{\theta }\right)$ and $g_{\sigma }\left(\sigma \right)$ over the unknowns. Therefore, the complete posterior density is

$$\begin{array}{c}\hfill p(\mathbf{\theta },\sigma |\mathbf{y})=\frac{1}{p\left(\mathbf{y}\right)}p(\mathbf{\theta },\sigma ,\mathbf{y})=\frac{1}{p\left(\mathbf{y}\right)}\ell \left(\mathbf{y}\right|\mathbf{\theta },\sigma )g_{\theta }\left(\mathbf{\theta }\right)g_{\sigma }\left(\sigma \right),\end{array}$$

(6)

The marginal likelihood is

$$\begin{array}{c}\hfill Z=p\left(\mathbf{y}\right)={\int }_{{\mathbb{R}}^{+}}{\int }_{\mathsf{\Theta }}\ell \left(\mathbf{y}\right|\mathbf{\theta },\sigma )g_{\theta }\left(\mathbf{\theta }\right)g_{\sigma }\left(\sigma \right)d\mathbf{\theta }d\sigma ,\end{array}$$

(7)

This quantity is often needed for model selection. While Z is generally unknown, we can usually evaluate pointwise the un-normalized posterior $\pi (\mathbf{\theta },\sigma |\mathbf{y})=\ell \left(\mathbf{y}\right|\mathbf{\theta },\sigma )g_{\theta }\left(\mathbf{\theta }\right)g_{\sigma }\left(\sigma \right)$, i.e., $p(\mathbf{\theta },\sigma |\mathbf{y})\propto \pi (\mathbf{\theta },\sigma \left|\mathbf{y}\right)$. More generally, the computation of integrals of the form

$$I\left(h\right)={\int }_{{\mathbb{R}}^{+}}{\int }_{\mathsf{\Theta }}h(\mathbf{\theta },\sigma )p(\mathbf{\theta },\sigma |\mathbf{y})d\mathbf{\theta }d\sigma ,$$

(8)

where $h:\mathsf{\Theta }\times {\mathbb{R}}^{+}\to \mathbb{R}$ is an integrable function, is usually required. We consider a Monte Carlo quadrature approach for approximating the integral above and, more generally, provide a particle approximation of the joint posterior $p(\mathbf{\theta },\sigma |\mathbf{y})$.

Main observation. Generating random samples from a complicated posterior in Equation (6) and efficiently computing the integrals as in Equations (7) and (8) is very often a hard task. Moreover, this task becomes more difficult when we try to perform a joint inference where scale parameters are involved, i.e., $\sigma$, and parameters of the nonlinearity, i.e., $\mathbf{\theta }$. Indeed, “wrong choices” of $\sigma$ values can easily jeopardize the sampling of $\mathbf{\theta }$. In the next section, we describe a strategy that we propose to tackle this problem. Before do so, however, we need to recall some additional definitions.

Conditional and marginal posteriors. In other to design efficient computational schemes, it is often useful to consider the conditional posteriors, for instance,

$$\begin{array}{cc}\hfill p\left(\mathbf{\theta }\right|\mathbf{y},\sigma )=\frac{p(\mathbf{\theta },\mathbf{y},\sigma )}{p(\mathbf{y},\sigma )}& =\frac{\ell \left(\mathbf{y}\right|\mathbf{\theta },\sigma )g_{\theta }\left(\mathbf{\theta }\right)g_{\sigma }\left(\sigma \right)}{p\left(\mathbf{y}\right|\sigma )g_{\sigma }\left(\sigma \right)},\hfill \\ \hfill & =\frac{\ell \left(\mathbf{y}\right|\sigma ,\mathbf{\theta })g_{\theta }\left(\mathbf{\theta }\right)}{p\left(\mathbf{y}\right|\sigma )}.\hfill \end{array}$$

(9)

In the next section, we will see that the idea underlying the proposed scheme is to split the space $[\mathbf{\theta },\sigma ]$, restricting the sampling problem only to $\mathbf{\theta }$ and considering an optimization problem with respect to $\sigma$. The conditional marginal likelihood is obtained by integrating out one of the two variables, e.g.,

$$\begin{array}{c}\begin{array}{c}\hfill Z\left(\sigma \right)=p\left(\mathbf{y}\right|\sigma )={\int }_{\mathbf{\theta }}\ell \left(\mathbf{y}\right|\mathbf{\theta },\sigma )g_{\theta }\left(\mathbf{\theta }\right)d\mathbf{\theta }.\end{array}\end{array}$$

(10)

The integral above cannot be computed analytically, in general. We can also consider marginal posteriors, for instance, the marginal posterior of $\sigma$ is

$$\begin{array}{c}\hfill p\left(\sigma \right|\mathbf{y})=\frac{p\left(\mathbf{y}\right|\sigma )g_{\sigma }\left(\sigma \right)}{p\left(\mathbf{y}\right)}=\frac{Z\left(\sigma \right)g_{\sigma }\left(\sigma \right)}{Z}.\end{array}$$

(11)

Note that the joint posterior in Equation (6) can be also written as

$$\begin{array}{cc}\hfill p(\mathbf{\theta },\sigma |\mathbf{y})& =p\left(\mathbf{\theta }\right|\mathbf{y},\sigma \left)p\right(\sigma \left|\mathbf{y}\right).\hfill \end{array}$$

(12)

Outline of the proposed approach. The underlying idea of this work is to divide the inference study in two parts. In the first part (Section 3 and Section 4), we focus on the study of the conditional posterior $p\left(\mathbf{\theta }\right|\mathbf{y},\sigma )$ given a fixed $\sigma$. Then, in the second part (Section 5), we also estimate the marginal posterior $p\left(\sigma \right|\mathbf{y})$. Finally, using (12), we can obtain a final approximation of the complete posterior $p(\mathbf{\theta },\sigma |\mathbf{y})$. Estimations of $Z\left(\sigma \right)$ and Z are also obtained.

3. Key Observations and Proposed Approach

3.1. Split Inference

In the first part of work, we assume a uniform proper (or improper) prior over $\mathbf{\theta }$, i.e., $g_{\theta }\left(\mathbf{\theta }\right)\propto 1$ in $\mathsf{\Theta }$. The possible use of a general choice of $g_{\theta }\left(\mathbf{\theta }\right)$ is discussed in Section 4.1. Let ${\mathbf{\theta }}_{\mathtt{MAP}}=arg{max}_{\mathbf{\theta }}p\left(\mathbf{\theta }\right|\mathbf{y},\sigma )$ denote the MAP estimator of $\mathbf{\theta }$. Generally, ${\mathbf{\theta }}_{\mathtt{MAP}}$ should be a function of $\sigma$, i.e., ${\mathbf{\theta }}_{\mathtt{MAP}}={\mathbf{\theta }}_{\mathtt{MAP}}\left(\sigma \right)$. However, due to the choice of likelihood function (and the uniform prior) considered in this paper, we have that

$$\begin{array}{cc}\hfill {\mathbf{\theta }}_{\mathtt{MAP}}& =arg\underset{\mathbf{\theta }}{max}logp\left(\mathbf{\theta }\right|\mathbf{y},\sigma ),\hfill \\ \hfill & =arg\underset{\mathbf{\theta }}{min}\left|\right|\mathbf{y}-{\mathbf{f}\left(\mathbf{\theta }\right)\left|\right|}^2,\mathrm{with}{g}_{\theta }\left(\mathbf{\theta }\right)\propto 1, \mathrm{for} \mathbf{\theta }\in \mathsf{\Theta },\hfill \end{array}$$

which does not depend on $\sigma$, i.e., we have that ${\mathbf{\theta }}_{\mathtt{MAP}}$ maximizes the conditional posterior $p\left(\mathbf{\theta }\right|\mathbf{y},\sigma )$ for any $\sigma$. See Appendix A for further details.

Furthermore, the variance of the conditional posterior $p\left(\mathbf{\theta }\right|\mathbf{y},\sigma )$ grows when $\sigma$ increases. In this sense, with larger $\sigma$, the density $p\left(\mathbf{\theta }\right|\mathbf{y},\sigma )$ is “broader”; thus, it is easier for Monte Carlo methods to explore the space (namely, we have a tempering effect). Based on these considerations, we can run Monte Carlo schemes (specifically IS algorithms) on $p\left(\mathbf{\theta }\right|\mathbf{y},{\sigma }_0)$ with a large value ${\sigma }_0$ for estimating ${\mathbf{\theta }}_{\mathtt{MAP}}$ more efficiently. Furthermore, apart from estimating ${\mathbf{\theta }}_{\mathtt{MAP}}$, we are also interested in studying the conditional posterior $p\left(\mathbf{\theta }\right|\mathbf{y},{\sigma }_{\mathtt{ML}})$, where

$${\sigma }_{\mathtt{ML}}=arg\underset{\sigma }{max}\ell \left(\mathbf{y}\right|{\mathbf{\theta }}_{\mathtt{MAP}},\sigma ).$$

The value ${\sigma }_{\mathtt{ML}}$ can be obtained in closed-form (see Appendix A). In fact, for any $\mathbf{\theta }$, we have

$$\begin{array}{ccc}\hfill \ell \left(\mathbf{y}\right|\mathbf{\theta },\sigma )& \propto & {\left(\frac{1}{{\sigma }^2}\right)}^{\frac{K}{2}}exp\left(-\frac{\left|\right|\mathbf{y}-{\mathbf{f}\left(\mathbf{\theta }\right)\left|\right|}^2}{2{\sigma }^2}\right),\hfill \end{array}$$

(13)

which has the form of an Inverse Gamma density for ${\sigma }^2$ and it has a unique mode at $\sqrt{\frac{1}{K}\left|\right|\mathbf{y}-{\mathbf{f}\left(\mathbf{\theta }\right)\left|\right|}^2}$, where $\mathbf{\theta }$ is a fixed. Therefore, finally we have

$${\sigma }_{\mathtt{ML}}=\sqrt{\frac{1}{K}\left|\right|\mathbf{y}-\mathbf{f}\left({\mathbf{\theta }}_{\mathtt{MAP}}\right){\left|\right|}^2}.$$

This can serve as a point estimator of the noise power in the system, and also as a threshold value to stop the tempering of the conditional posterior, as we show in the following section.

3.2. An Iterative Scheme

Consider that we start with a large value ${\sigma }_0$, which can be viewed as a coarse approximation of ${\sigma }_{\mathtt{ML}}$, so we denote it ${\sigma }_0={\widehat{\sigma }}_{\mathtt{ML}}^{\left(0\right)}$. Let ${\widehat{\mathbf{\theta }}}_{\mathtt{MAP}}^{\left(1\right)}$ denote an estimate of ${\mathbf{\theta }}_{\mathtt{MAP}}$ obtained by working w.r.t. $p\left(\mathbf{\theta }\right|\mathbf{y},{\widehat{\sigma }}_{\mathtt{ML}}^{\left(0\right)})$. We use this current estimation to obtain the next value of $\sigma$, i.e., ${\widehat{\sigma }}_{\mathtt{ML}}^{\left(1\right)}=\sqrt{\frac{1}{K}\left|\right|\mathbf{y}-\mathbf{f}\left({\widehat{\mathbf{\theta }}}_{\mathtt{MAP}}^{\left(1\right)}\right){\left|\right|}^2}$. In general, ${\widehat{\sigma }}_{\mathtt{ML}}^{\left(1\right)}$ is a better estimator of ${\sigma }_{\mathtt{ML}}$ than ${\widehat{\sigma }}_{\mathtt{ML}}^{\left(0\right)}$, as we have tried to evaluate of the smallest error between $\mathbf{f}\left(\mathbf{\theta }\right)$ and the data, $\mathbf{y}$, i.e., $\left|\right|\mathbf{y}-\mathbf{f}\left(\mathbf{\theta }\right)\left|\right|$, which is related to the power of the noise perturbation in the system. For instance, assuming zero noise, we would have $\left|\right|\mathbf{y}-\mathbf{f}\left({\mathbf{\theta }}_{\mathtt{MAP}}\right)\left|\right|=0$, recalling that $g_{\theta }\left(\mathbf{\theta }\right)\propto 1$ for $\mathbf{\theta }\in \mathsf{\Theta }$. We can iterate this procedure for $t=1,\cdots ,T$:

1

Estimate ${\widehat{\mathbf{\theta }}}_{\mathtt{MAP}}^{\left(t\right)}$ by Monte Carlo (e.g., an IS scheme) by approximately maximizing $p\left(\mathbf{\theta }\right|\mathbf{y},{\widehat{\sigma }}_{\mathtt{ML}}^{(t-1)})$.

2

Compute

$${\widehat{\sigma }}_{\mathtt{ML}}^{\left(t\right)}=\sqrt{\frac{1}{K}\left|\right|\mathbf{y}-\mathbf{f}\left({\widehat{\mathbf{\theta }}}_{\mathtt{MAP}}^{\left(t\right)}\right){\left|\right|}^2}.$$

(14)

With this iterative scheme, we have that ${\widehat{\sigma }}_{\mathtt{ML}}^{\left(T\right)}\to {\sigma }_{\mathtt{ML}}$ as T grows, thus we eventually perform IS with respect to the density of interest $p\left(\mathbf{\theta }\right|\mathbf{y},{\sigma }_{\mathtt{ML}})$. Furthermore, a non-increasing sequence of values ${\widehat{\sigma }}_{\mathtt{ML}}^{\left(0\right)}\ge {\widehat{\sigma }}_{\mathtt{ML}}^{\left(1\right)}\ge \cdots \ge {\widehat{\sigma }}_{\mathtt{ML}}^{\left(T\right)}$ is produced, which facilitates the estimation of ${\mathbf{\theta }}_{\mathtt{MAP}}$, and ensures the IS estimation of $p\left(\mathbf{\theta }\right|\mathbf{y},{\sigma }_{\mathtt{ML}})$ is performed efficiently by using the set of intermediate, tempered (i.e., wider) distributions $p\left(\mathbf{\theta }\right|\mathbf{y},{\widehat{\sigma }}_{\mathtt{ML}}^{\left(t\right)})$ for $t=0,1,\dots ,T$. Finally, a particle approximation of $p\left(\mathbf{\theta }\right|\mathbf{y},{\widehat{\sigma }}_{\mathtt{ML}}^{\left(T\right)})$ is obtained, i.e.,

$$p\left(\mathbf{\theta }\right|\mathbf{y},{\widehat{\sigma }}_{\mathtt{ML}}^{\left(T\right)})=\sum _{t=1}^T\sum _{n=1}^N{\tilde{w}}_t^{\left(n\right)}\delta (\mathbf{\theta }-{\mathbf{\theta }}_t^{\left(n\right)}),$$

where ${\sum }_{t=1}^T{\sum }_{n=1}^N{\tilde{w}}_t^{\left(n\right)}=1$. Note that ${\tilde{w}}_t^{\left(n\right)}$ are the final corrected weights obtained at the end of the algorithm (see Algorithm 1).

4. Automatic Tempering Adaptive Importance Sampling (ATAIS)

In this section, we describe an adaptive importance sampler with an automatic tempering approach which follows the procedure given above. At each iteration t of the algorithm, we have an ML approximation of $\sigma$, i.e., ${\widehat{\sigma }}_{\mathtt{ML}}^{(t-1)}$. Considering Equation (9), we define the un-normalized tempered conditional posterior at the t-th iteration,

$$\begin{array}{c}\hfill {\pi }_t\left(\mathbf{\theta }\right)=\ell \left(\mathbf{y}\right|\mathbf{\theta },{\widehat{\sigma }}_{\mathtt{ML}}^{(t-1)})g_{\theta }\left(\mathbf{\theta }\right),\end{array}$$

(15)

where we assume $g_{\theta }\left(\mathbf{\theta }\right)\propto 1$ in $\mathsf{\Theta }$. For other generic choice of $g_{\theta }\left(\mathbf{\theta }\right)$, see the discussion in Section 4.1. At each iteration, we consider $p\left(\mathbf{\theta }\right|\mathbf{y},{\widehat{\sigma }}_{\mathtt{ML}}^{(t-1)})\propto {\pi }_t\left(\mathbf{\theta }\right)$ as the target distribution. The dependence on the iteration t occurs because ${\widehat{\sigma }}_{\mathtt{ML}}^{\left(t\right)}$ varies with t. The ATAIS algorithm is outlined in Algorithm 1. The resulting scheme is an adaptive IS algorithm which combines sampling schemes and stochastic optimization. It is important to remark that if ${\widehat{\sigma }}_{\mathtt{ML}}^{\left(0\right)}$ is bigger than the true ML value, we generate a non-increasing sequence of ${\widehat{\sigma }}_{\mathtt{ML}}^{\left(t\right)}$, i.e., ${\widehat{\sigma }}_{\mathtt{ML}}^{\left(0\right)}\ge {\widehat{\sigma }}_{\mathtt{ML}}^{\left(1\right)}\ge \dots {\widehat{\sigma }}_{\mathtt{ML}}^{\left(t\right)}\ge {\widehat{\sigma }}_{\mathtt{ML}}^{(t+1)}$, etc. Note that this is true as we have assumed a uniform prior $g_{\theta }\left(\mathbf{\theta }\right)$. To see this, recall that ${\widehat{\sigma }}_{\mathtt{ML}}=\sqrt{\frac{1}{K}\left|\right|\mathbf{y}-\mathbf{f}\left({\widehat{\mathbf{\theta }}}_{\mathtt{MAP}}\right){\left|\right|}^2}$. Improving ${\widehat{\mathbf{\theta }}}_{\mathtt{MAP}}$ means that the squared error $\left|\right|\mathbf{y}-\mathbf{f}\left({\widehat{\mathbf{\theta }}}_{\mathtt{MAP}}\right){\left|\right|}^2$ is smaller, as shown in Equation (14), which implies that ${\widehat{\sigma }}_{\mathtt{ML}}$ always decreases (provided that we start with ${\widehat{\sigma }}_{\mathtt{ML}}>{\sigma }_{\mathtt{ML}}$).

IS steps. A set of N samples ${\left\{{\mathbf{\theta }}_t^{\left(n\right)}\right\}}_{n=1}^N$ are drawn from a (normalized) proposal density $q\left(\mathbf{\theta }\right|{\mathbf{\mu }}_t,{\mathsf{\Sigma }}_t)$ with mean ${\mathbf{\mu }}_t$ and a covariance matrix ${\mathsf{\Sigma }}_t$. An importance weight

$$w_t^{\left(n\right)}=\frac{{\pi }_t\left({\mathbf{\theta }}_t^{\left(n\right)}\right)}{q\left({\mathbf{\theta }}_t^{\left(n\right)}\right|{\mathbf{\mu }}_t,{\mathsf{\Sigma }}_t)},$$

is assigned to each sample.

Proposal adaptation. A particle estimation of the conditional MAP estimator of $\mathbf{\theta }$ is given by ${\widehat{\mathbf{\theta }}}_t=arg\underset{n}{max}{\pi }_t\left({\mathbf{\theta }}_t^{\left(n\right)}\right)$. The value of current MAP approximation ${\pi }_t\left({\widehat{\mathbf{\theta }}}_t\right)$ is then compared with the value of global MAP estimator obtained so far denoted as ${\pi }_{\mathtt{MAP}}$. If ${\pi }_t\left({\widehat{\mathbf{\theta }}}_t\right)\ge {\pi }_{\mathtt{MAP}}$, all the global MAP estimators are updated and the proposal pdf is moved at ${\widehat{\mathbf{\theta }}}_t$, i.e., we set

$$\begin{array}{c}\hfill {\widehat{\mathbf{\theta }}}_{\mathtt{MAP}}^{\left(t\right)}={\widehat{\mathbf{\theta }}}_t,{\pi }_{\mathtt{MAP}}={\pi }_t\left({\widehat{\mathbf{\theta }}}_t\right),{\mathbf{\mu }}_t={\widehat{\mathbf{\theta }}}_t.\end{array}$$

(16)

Algorithm 1: ATAIS: AIS with automatic tempering.

Initializations: Choose N, ${\mathbf{\mu }}_1$, ${\mathsf{\Sigma }}_1$, and obtain an initialization for ${\widehat{\sigma }}_{\mathtt{ML}}^{\left(0\right)}$, and set ${\pi }_{\mathtt{MAP}}=0$. For $t=1,\dots ,T$: (a) Sampling: i. Draw ${\mathbf{\theta }}_t^{\left(1\right)},\dots ,{\mathbf{\theta }}_t^{\left(N\right)}\sim q\left(\mathbf{\theta }\right|{\mathbf{\mu }}_t,{\mathsf{\Sigma }}_t)$. ii. Assign to each sample the weights $$\begin{array}{c}\hfill w_t^{\left(n\right)}=\frac{{\pi }_t\left({\mathbf{\theta }}_t^{\left(n\right)}\right)}{q\left({\mathbf{\theta }}_t^{\left(n\right)}\right|{\mathbf{\mu }}_t,{\mathsf{\Sigma }}_t)},n=1,\dots ,N.\end{array}$$ (17) (b) Current maximum estimations: i. Obtain ${\widehat{\mathbf{\theta }}}_t=arg\underset{n}{max}{\pi }_t\left({\mathbf{\theta }}_t^{\left(n\right)}\right)$, and compute ${\widehat{\mathbf{r}}}_t=\mathbf{f}\left({\widehat{\mathbf{\theta }}}_t\right)$ ii. Compute ${\widehat{\sigma }}_t=\sqrt{\frac{1}{K}\left|\right|\mathbf{y}-{\widehat{\mathbf{r}}}_t{\left|\right|}^2}$. (c) Global maximum estimations: i. If ${\widehat{\sigma }}_t\le {\widehat{\sigma }}_{\mathtt{ML}}^{(t-1)}$, then set ${\widehat{\sigma }}_{\mathtt{ML}}^{\left(t\right)}={\widehat{\sigma }}_t$. Otherwise, set ${\widehat{\sigma }}_{\mathtt{ML}}^{\left(t\right)}={\widehat{\sigma }}_{\mathtt{ML}}^{(t-1)}$. ii. If ${\pi }_t\left({\widehat{\mathbf{\theta }}}_t\right)\ge {\pi }_{\mathtt{MAP}}$, then set ${\widehat{\mathbf{\theta }}}_{\mathtt{MAP}}^{\left(t\right)}={\widehat{\mathbf{\theta }}}_t$ and ${\pi }_{\mathtt{MAP}}={\pi }_t\left({\widehat{\mathbf{\theta }}}_t\right)$. Otherwise, ${\widehat{\mathbf{\theta }}}_{\mathtt{MAP}}^{\left(t\right)}={\widehat{\mathbf{\theta }}}_{\mathtt{MAP}}^{(t-1)}$ and keep the value of ${\pi }_{\mathtt{MAP}}$. (d) Adaptation: Set $$\begin{array}{ccc}\hfill {\mathbf{\mu }}_t& =& {\widehat{\mathbf{\theta }}}_{\mathtt{MAP}}^{\left(t\right)},\hfill \end{array}$$ (18) $$\begin{array}{ccc}\hfill {\mathsf{\Sigma }}_t& =& \sum _{n=1}^N{\overline{w}}_t^{\left(n\right)}{({\mathbf{\theta }}_t^{\left(n\right)}-{\overline{\mathbf{\theta }}}_t)}^{\mathsf{\top }}({\mathbf{\theta }}_t^{\left(n\right)}-{\overline{\mathbf{\theta }}}_t)+ϵ{\mathbf{I}}_M,\hfill \end{array}$$ (19) where ${\overline{w}}_t^{\left(n\right)}\frac{w_t^{\left(n\right)}}{{\sum }_{i=1}^N{w}_t^{\left(i\right)}}$ are the normalized weights, ${\overline{\mathbf{\theta }}}_t={\sum }_{n=1}^N{\overline{w}}_t^{\left(n\right)}{\mathbf{\theta }}_t^{\left(n\right)}$ and $ϵ>0$ is a small scalar value . Output: Return the final estimators ${\widehat{\mathbf{\theta }}}_{\mathtt{MAP}}^{\left(T\right)}$, ${\widehat{\sigma }}_{\mathtt{ML}}^{\left(T\right)}$, and all the weighted samples $\{{\mathbf{\theta }}_t^{\left(n\right)},{\tilde{w}}_t^{\left(n\right)}\}$, for all t and n, with the corrected weights $${\tilde{w}}_t^{\left(n\right)}=w_t^{\left(n\right)}\frac{{\pi }_{T+1}\left({\mathbf{\theta }}_t^{\left(n\right)}\right)}{{\pi }_t\left({\mathbf{\theta }}_t^{\left(n\right)}\right)}.$$ (20)

Otherwise, we keep the previous values of ${\widehat{\mathbf{\theta }}}_{\mathtt{MAP}}^{\left(t\right)}={\widehat{\mathbf{\theta }}}_{\mathtt{MAP}}^{(t-1)}$, ${\pi }_{\mathtt{MAP}}$, and ${\mathbf{\mu }}_t={\mathbf{\mu }}_{t-1}$. The covariance matrix ${\mathsf{\Sigma }}_t$ is adapted by considering the empirical covariance of the weighted samples. Note that we set ${\mathbf{\mu }}_t={\widehat{\mathbf{\theta }}}_{\mathtt{MAP}}^{\left(t\right)}$ instead of using the empirical mean of the samples (as in other classical AIS schemes). This is because we have noticed that this choice provides better and more robust results, especially as the dimension of the problem grows.

Automatic tempering. As we showed in the previous section, the current ML estimator of $\sigma$ can be obtained analytically as

$${\widehat{\sigma }}_t=\sqrt{\frac{1}{K}\left|\right|\mathbf{y}-{\widehat{\mathbf{r}}}_t{\left|\right|}^2},$$

(21)

where ${\widehat{\mathbf{r}}}_t=\mathbf{f}\left({\widehat{\mathbf{\theta }}}_t\right)$. If the current ML estimator ${\widehat{\sigma }}_t$ is smaller than the current global one ${\widehat{\sigma }}_{\mathtt{ML}}^{(t-1)}$, i.e., ${\widehat{\sigma }}_t<{\widehat{\sigma }}_{\mathtt{ML}}^{(t-1)}$, then we update ${\widehat{\sigma }}_{\mathtt{ML}}^{\left(t\right)}={\widehat{\sigma }}_t$, Otherwise, we keep the value of ${\widehat{\sigma }}_{\mathtt{ML}}^{\left(t\right)}={\widehat{\sigma }}_{\mathtt{ML}}^{(t-1)}$. Actually, with a uniform prior $g_{\theta }\left(\mathbf{\theta }\right)$, every time that we update ${\widehat{\mathbf{\theta }}}_{\mathrm{MAP}}^{\left(t\right)}$, we also update ${\widehat{\sigma }}_{\mathtt{ML}}^{\left(t\right)}$ (see footnote in the previous page).

ATAIS outputs. After T iterations, a final correction of the weights is needed, i.e.,

$${\tilde{w}}_t^{\left(n\right)}=w_t^{\left(n\right)}\frac{{\pi }_{T+1}\left({\mathbf{\theta }}_t^{\left(n\right)}\right)}{{\pi }_t\left({\mathbf{\theta }}_t^{\left(n\right)}\right)},$$

(22)

in order to obtain a particle approximation of the measure of the final conditional posterior $p\left(\mathbf{\theta }\right|\mathbf{y},{\widehat{\sigma }}_{\mathtt{ML}}^{\left(T\right)})\propto {\pi }_{T+1}\left(\mathbf{\theta }\right)$. Thus, the algorithm returns the final estimators ${\widehat{\mathbf{\theta }}}_{\mathtt{MAP}}^{\left(T\right)}$, ${\widehat{\sigma }}_{\mathtt{ML}}^{\left(T\right)}$, and all the weighted samples $\{{\mathbf{\theta }}_t^{\left(n\right)},{\tilde{w}}_t^{\left(n\right)}\}$, for all $n=1,\dots ,N$ and $t=1,\dots ,T$. Other outputs can be obtained with a postprocessing of the weighted samples, as shown below. Note that Equation (22) does not require any additional evaluation of the model, and the error is $e_t^{\left(n\right)}=\left|\right|\mathbf{y}-\mathbf{f}\left({\mathbf{\theta }}_t^{\left(n\right)}\right){\left|\right|}^2$. Moreover, we can also use $e_t^{\left(n\right)}$ and $\left\{{\mathbf{\theta }}_t^{\left(n\right)}\right\}$ for building a particle approximation of any other conditional posterior $p\left(\mathbf{\theta }\right|\mathbf{y},\sigma )$. This allows the study of the marginal posterior $p\left(\sigma \right|\mathbf{y})$ and provides the complete Bayesian inference, as we show in the next section.

4.1. With a Generic Prior $g_{\theta }\left(\mathbf{\theta }\right)$

The ATAIS algorithm is based on the fact that ${\mathbf{\theta }}_{\mathtt{MAP}}$ does not depend on $\sigma$. This allows us to progressively estimate it by targeting the sequence of tempered posteriors ${\pi }_t\left(\mathbf{\theta }\right)\propto p\left(\mathbf{\theta }\right|\mathbf{y},{\widehat{\sigma }}_{\mathtt{ML}}^{\left(t\right)})$ that share all the same MAP. However, in the case $g_{\theta }\left(\mathbf{\theta }\right)$ is not uniform, we generally have one ${\mathbf{\theta }}_{\mathtt{MAP}}\left(\sigma \right)$ for each $p\left(\mathbf{\theta }\right|\mathbf{y},\sigma )$, and we could have that the sequence of ${\mathbf{\theta }}_{\mathtt{MAP}}\left({\widehat{\sigma }}_{\mathtt{ML}}^{\left(t\right)}\right)$ will not approach ${\mathbf{\theta }}_{\mathtt{MAP}}\left({\sigma }_{\mathtt{ML}}\right)$.

If the data are informative and the prior $g_{\theta }\left(\mathbf{\theta }\right)$ is chosen such it is vague with respect to the likelihood, the position of ${\mathbf{\theta }}_{\mathtt{MAP}}\left(\sigma \right)$ is not very sensitive to the value of $\sigma$. Namely, we have ${\mathbf{\theta }}_{\mathtt{MAP}}\left({\widehat{\sigma }}_{\mathtt{ML}}^{\left(1\right)}\right)\approx {\mathbf{\theta }}_{\mathtt{MAP}}\left({\widehat{\sigma }}_{\mathtt{ML}}^{\left(2\right)}\right)\approx \cdots \approx {\mathbf{\theta }}_{\mathtt{MAP}}\left({\sigma }_{\mathtt{ML}}\right)$, and thus our algorithm can be applied in this context. When the data are not informative, we should use an even more vague prior (i.e., wider than the likelihood function) in order to maintain the usefulness of the algorithm.

5. Complete Bayesian Inference with ATAIS

Let us assume we have a proper prior $g_{\theta }\left(\mathbf{\theta }\right)$ and we introduce another proper prior $g_{\sigma }\left(\sigma \right)$ for $\sigma$. The outputs of the ATAIS algorithm can serve to approximate the normalizing constant of the joint posterior $p(\mathbf{\theta },\sigma |\mathbf{y})\propto \ell \left(\mathbf{y}\right|\mathbf{\theta },\sigma )g_{\theta }\left(\mathbf{\theta }\right)g_{\sigma }\left(\sigma \right)$, i.e., the so-called marginal likelihood or Bayesian model evidence, given by

$$Z={\int }_{{\mathbb{R}}^{+}}{\int }_{\mathsf{\Theta }}\ell \left(\mathbf{y}\right|\mathbf{\theta },\sigma )g_{\theta }\left(\mathbf{\theta }\right)g_{\sigma }\left(\sigma \right)d\mathbf{\theta }d\sigma ={\int }_{{\mathbb{R}}^{+}}Z\left(\sigma \right)g_{\sigma }\left(\sigma \right)d\sigma ,$$

(23)

where we have denoted $Z\left(\sigma \right)={\int }_{\mathsf{\Theta }}\ell \left(\mathbf{y}\right|\mathbf{\theta },\sigma )g_{\theta }(\mathbf{\theta }d\mathbf{\theta }$, usually called conditional marginal likelihood. The quantity Z is useful for model selection purposes. Furthermore, a complete Bayesian study of the joint posterior $p(\mathbf{\theta },\sigma |\mathbf{y})$ can be provided as well.

Approximation of $Z\left(\sigma \right)=p\left(\mathbf{y}\right|\sigma )$. After the T iterations of ATAIS, we can also approximate the conditional marginal likelihood $Z\left(\sigma \right)=p\left(\mathbf{y}\right|\sigma )$ without additional evaluations of the target function. Indeed, saving the error values at each particle obtained for the computation of the likelihood function during ATAIS,

$$e_t^{\left(n\right)}=\left|\right|\mathbf{y}-\mathbf{f}\left({\mathbf{\theta }}_t^{\left(n\right)}\right){\left|\right|}^2,$$

We can compute the IS weights,

$${\rho }_t^{\left(n\right)}\left(\sigma \right)=\frac{\frac{1}{{\left(2\pi {\sigma }^2\right)}^{\frac{K}{2}}}exp\left(-\frac{e_t^{\left(n\right)}}{2{\sigma }^2}\right)g_{\theta }\left({\mathbf{\theta }}_t^{\left(n\right)}\right)}{q\left({\mathbf{\theta }}_t^{\left(n\right)}\right|{\mathbf{\mu }}_t,{\mathsf{\Sigma }}_t)},$$

(24)

for a generic value of $\sigma$ Thus, the IS estimator of the conditional marginal likelihood $Z\left(\sigma \right)$ is given by the arithmetic mean of the weights ${\rho }_t^{\left(n\right)}\left(\sigma \right)$,

$$\widehat{Z}\left(\sigma \right)=\widehat{p}\left(\mathbf{y}\right|\sigma )=\frac{1}{NT}\sum _{t=1}^T\sum _{n=1}^N{\rho }_t^{\left(n\right)}\left(\sigma \right).$$

(25)

Approximation of Z. Drawing ${\sigma }^{\left(r\right)}\sim g_{\sigma }\left(\sigma \right)$, for $r=1,\dots ,R$, (or considering a deterministic grid, e.g., as a Riemannian integration), we can approximate the global marginal likelihood Z by applying simple Monte Carlo to the integral in Equation (23),

$$\widehat{Z}=\frac{1}{R}\sum _{r=1}^R\widehat{Z}\left({\sigma }^{\left(r\right)}\right).$$

(26)

Approximation of $p\left(\sigma \right|\mathbf{y})$. An approximation of the marginal posterior $p\left(\sigma \right|\mathbf{y})=\frac{p\left(\mathbf{y}\right|\sigma )g_{\sigma }\left(\sigma \right)}{p\left(y\right)}$ can be also obtained as

$$\begin{array}{c}\hfill p\left(\sigma \right|\mathbf{y})\approx \widehat{p}\left(\sigma \right|\mathbf{y})=\frac{\widehat{Z}\left(\sigma \right)g_{\sigma }\left(\sigma \right)}{\widehat{Z}},\end{array}$$

(27)

which can be used to approximate, e.g., the MAP of $p\left(\sigma \right|\mathbf{y})$ by ${\sigma }_{\mathtt{MAP}-\mathtt{marg}}\approx arg{max}_{\sigma }\widehat{Z}\left(\sigma \right)g_{\sigma }\left(\sigma \right).$ Other different moments of $p\left(\sigma \right|\mathbf{y})$ can be computed by a deterministic quadrature (as the problem is now one-dimensional) or applying noisy Monte Carlo approaches.

Complete Bayesian analysis. We can approximate the integral of interest as

$$\begin{array}{cc}\hfill I& ={\int }_{{\mathbb{R}}^{+}}{\int }_{\mathsf{\Theta }}h(\mathbf{\theta },\sigma )p(\mathbf{\theta },\sigma |\mathbf{y})d\mathbf{\theta }d\sigma ,\hfill \end{array}$$

(28)

$$\begin{array}{cc}\hfill & ={\int }_{{\mathbb{R}}^{+}}{\int }_{\mathsf{\Theta }}h(\mathbf{\theta },\sigma )p\left(\mathbf{\theta }\right|\mathbf{y},\sigma )p\left(\sigma \right|\mathbf{y})d\mathbf{\theta }d\sigma \hfill \end{array}$$

(29)

$$\begin{array}{cc}\hfill & \approx \frac{1}{J}\sum _{j=1}^J\sum _{t=1}^T\sum _{n=1}^N{\overline{\rho }}_t^{\left(n\right)}\left({\sigma }^{\left(j\right)}\right)h({\mathbf{\theta }}_t^{\left(n\right)},{\sigma }^{\left(j\right)}),\hfill \end{array}$$

(30)

where

$$\begin{array}{c}\hfill {\overline{\rho }}_t^{\left(n\right)}\left({\sigma }^{\left(j\right)}\right)=\frac{{\rho }_t^{\left(n\right)}\left({\sigma }^{\left(j\right)}\right)}{{\sum }_{\tau =1}^T{\sum }_{i=1}^N{\rho }_{\tau }^{\left(i\right)}\left({\sigma }^{\left(j\right)}\right)},\end{array}$$

(31)

and ${\sigma }^{\left(j\right)}$ are generated by applying a noisy MCMC with invariant density $\widehat{p}\left(\sigma \right|\mathbf{y})\propto \widehat{Z}\left(\sigma \right)g_{\sigma }\left(\sigma \right)$. Note that the samples ${\mathbf{\theta }}_t^{\left(n\right)}$ do not depend on the index j (they do not change) as we are recycling the particles generated by ATAIS and reusing evaluations $e_t^{\left(n\right)}=\left|\right|\mathbf{y}-\mathbf{f}\left({\mathbf{\theta }}_t^{\left(n\right)}\right){\left|\right|}^2$.

6. Simulations

We test the proposed scheme in two numerical examples: The first numerical experiment is a simple bidimensional example (which is easy to be reproduced). The second experiment considers a real-world application, i.e., a radial velocity model of exoplanet systems which is often employed in astronomy applications (with a dimension of the inference problem of 6 and 11).

6.1. First Numerical Analysis

For the sake of simplicity, let us consider $\theta \in \mathbb{R}$ and an observation model given by the equation

$$y_k={\theta }^2+log\left(\right|sin\left(10\theta \right)\left|\right)+v_k,$$

so that $f\left(\theta \right)={\theta }^2+log\left(\right|sin\left(10\theta \right)\left|\right)$, and $v_k\sim \mathcal{N}(0,{\sigma }^2)$. We consider ${\theta }_{\mathtt{true}}=2.5$, and ${\sigma }_{\mathtt{true}}=4$. We generate $K=8$ observations from the model above. We also consider a uniform prior for $\theta$ in $(0,20]$. The conditional posterior $p\left(\theta \right|\mathbf{y},{\sigma }_{\mathtt{true}})$ is shown in Figure 1c. We can observe that $p\left(\theta \right|\mathbf{y},{\sigma }_{\mathtt{true}})$ is highly multimodal. Figure 1 also depicts the conditional posteriors $p\left(\theta \right|\mathbf{y},\sigma )$ with $\sigma \in \{10,20\}$. Considering also a uniform prior over $\sigma$ in $(0,20]$, we have also a bidimensional joint posterior over $[\theta ,\sigma ]$, which is depicted in Figure 2a.

In this bidimensional example, it is possible to obtain the ground-truths using an expensive thin grid. We show the ground-truths of the different pdfs in

Table 1. Summary of pdfs and ground-truths for the first numerical experiment.

Pdf	Expectation	Variance	MAP
$p\left(\mathbf{\theta }\right|\mathbf{y},{\sigma }_{\mathtt{ML}})$	2.48	0.11	2.56
$p\left(\sigma \right|\mathbf{y})$	4.32	2.43	3.46
$p\left(\mathbf{\theta }\right|\mathbf{y})$	2.46	0.18	2.56

. Moreover, the true value of the complete evidence $Z=p\left(\mathbf{y}\right)=1.5983\times {10}^{-9}$. As the prior over $\sigma$ is uniform, the maximum likelihood of $\sigma$ is ${\sigma }_{\mathtt{ML}}={\sigma }_{\mathtt{MAP}\text{-}\mathtt{joint}}=3.23$. The two marginal posteriors are shown in Figure 2b,c.

We apply ATAIS with the goal of estimating the expected value and the variance of the posterior density with respect to $\theta$. We consider a Gaussian proposal $q\left(\theta \right|{\mu }_t,{\lambda }_t)$ with ${\mu }_0=10$ and a starting variance of ${\lambda }_0=4$. Note that ${\mu }_0$ is located in a region that does not contain modes. We also start with ${\widehat{\sigma }}_{\mathtt{ML}}^{\left(0\right)}=20$ and ${\pi }_{\mathtt{MAP}}=0$ (initial conditions). The Mean Square Error (MSE) of ATAIS, averaged over 500 runs, in estimation of different moments and modes as function of N (and with $T=10$), is given in

Table 2. Mean Square Error (MSE) of ATAIS (averaged over 500 runs), in the estimation of the evidence, different moments and modes as function of N and $T=10$.

Value	$\mathit{N}=10$	$\mathit{N}=100$	$\mathit{N}=1000$	$\mathit{N}=5000$	Ground-Truths
$E\left[\theta \right|\mathbf{y},{\sigma }_{\mathtt{ML}}]$	0.0311	0.0098	0.0034	0.0024	2.48
$\mathrm{var}\left[\theta \right|\mathbf{y},{\sigma }_{\mathtt{ML}}]$	0.0474	0.0370	0.0298	0.0201	0.11
${\theta }_{\mathtt{MAP}}$	0.0410	0.0337	0.0285	0.0127	2.56
$E\left[\sigma \right|\mathbf{y}]$	0.9233	0.0785	0.0097	0.0023	4.32
$\mathrm{var}\left[\sigma \right|\mathbf{y}]$	6.1869	0.2640	0.0035	0.0010	2.43
${\sigma }_{\mathtt{MAP}\text{-}\mathtt{marg}}$	0.0056	0.0004	0.0001	$3\times {10}^{-5}$	3.46
${\sigma }_{\mathtt{ML}}$	$8\times {10}^{-5}$	$2\times {10}^{-5}$	$5\times {10}^{-7}$	$6\times {10}^{-9}$	3.23
$Z=p\left(\mathbf{y}\right)$	$2\times {10}^{-18}$	$1.8\times {10}^{-20}$	$1.4\times {10}^{-20}$	$3.6\times {10}^{-22}$	$1.6\times {10}^{-9}$

. The ML estimation ${\widehat{\sigma }}_{\mathtt{ML}}^{\left(t\right)}$, as function of the iteration t (with $N=5$) for different runs, is given in Figure 3a. The approximation of the marginal posterior $p\left(\sigma \right|\mathbf{y})$, denoted $\widehat{p}\left(\sigma \right|\mathbf{y})$, is obtained as in Equation (27) in one specific run, with different $N\in \{10,100,500\}$ and $T=10$. The approximations of the joint posterior $p(\mathbf{\theta },\sigma |\mathbf{y})$ and the marginal posterior $p\left(\mathbf{\theta }\right|\mathbf{y})$, obtained by resampling the particles according to the normalized weights in Equations (31) and (24), are shown in Figure 4, i.e., using a sampling importance resampling procedure. For more details, see in [14] and Chapter 24 in [15].

6.2. Radial Velocity Curves of Exoplanets and Binary Systems

In this example, we consider an application in an astronomical model. In recent years, the problem of revealing objects orbiting other stars has acquired large attention. Different techniques have been proposed to discover exo-objects but, nowadays, the radial velocity technique is still the most used [16,17,18,19]. The problem consists in fitting a model (the so-called radial velocity curve) to data acquired at different moments spanning during long time periods (up to years). The model is highly nonlinear, and it is costly in terms of computation time (especially for certain sets of parameters). Obtaining a value to compare to a single observation involves numerically integrating a differential equation in time or an iterative procedure for solving to a nonlinear equation. Typically, the iteration is performed until a threshold is reached or ${10}^6$ iterations are performed. The problem of radial velocity curve fitting is applied in several related applications.

Observation model—likelihood. When analyzing the radial velocity data of an exoplanetary system, it is commonly accepted that the wobbling of the star around the center of mass is caused by the sum of the gravitational force of each planet independently and that they do not interact with each other. Each planet follows a Keplerian orbit, and the radial velocity of the host star is given by

$$y_{r,t}=V_0+\sum _{i=1}^S{A}_i\left[cos\left(u_{i,t}+{\omega }_i\right)+e_{i}cos\left({\omega }_i\right)\right]+{\xi }_t,$$

(32)

with $t=1,\dots ,T$ and $r=1,\dots ,R$. In this equation, $A_i$ is the amplitude of the curve, $w_i$ is the argument of perigee, and $e_i$ is the eccentricity of the orbit of the i-th planet. The parameter $V_0$ represents the mean velocity, and it is common for all the planets. The number of objects in the system is S, which is considered to be known in this experiment (for the sake of simplicity). Both $y_{r,t}$ and $u_{i,t}$ depend on time t, and then ${\xi }_t$ is a Gaussian noise perturbation with variance ${\sigma }^2$. The likelihood function is defined by (32) and some indicator variables described below. The angle $u_{i,t}$ is the true anomaly of the planet i, and it can be determined from

$$\frac{d{u}_{i,t}}{dt}=\frac{2\pi }{P_i}\frac{{\left(1+e_{i}cos{u}_{i,t}\right)}^2}{{\left(1-e_i\right)}^{\frac{3}{2}}}$$

(33)

This equation has analytical solution. As a result, the true anomaly $u_t$ can be determined from the mean anomaly M. However, the analytical solution contains a nonlinear term that needs to be determined by iterating. First, we define the mean anomaly $M_{i,t}$ as

$$M_{i,t}=\frac{2\pi }{P_i}\left(t-{\tau }_i\right),$$

(34)

where ${\tau }_i$ is the time of periastron passage of the planet i and $P_i$ is the period of its orbit. Then, through the Kepler’s equation,

$$M_{i,t}=E_{i,t}-e_{i}sin{E}_{i,t},$$

(35)

where $E_{i,t}$ is the eccentric anomaly. Equation (35) has no analytic solution and it must be solved by an iterative procedure. A Newton–Raphson method is typically used to find the roots of this equation [20]. For certain sets of parameters, this iterative procedure can be particularly slow. We also have

$$tan\frac{u_{i,t}}{2}=\sqrt{\frac{1+e_i}{1-e_i}}tan\frac{E_{i,t}}{2},$$

(36)

The variable of interest $\mathbf{\theta }$ is then the vector

$$\mathbf{\theta }=[V_0,A_1,{\omega }_1,e_1,P_1,{\tau }_1,\dots ,A_S,{\omega }_S,e_S,P_S,{\tau }_S],$$

(37)

Then, for a single object (e.g., a planet or a natural satellite), the dimension of $\mathbf{\theta }$ is $M=5+1=6$, with two objects the dimension of $\mathbf{\theta }$ is $M=11$ etc.

This example consists in a synthetic radial velocity curve of a planetary system with one planet or two planets (i.e., $S=1$ or $S=2$). More specifically, we generate simulated data with a model with two planets. The orbital parameters of the planets are listed in

Table 3. Main orbital parameters of the two exoplanets in the simulation.

Parameter	Planet 1	Planet 2
P	15 d	115 d
A	25 m s${}^{-1}$	5 m s${}^{-1}$
e	0.1	0.0
$\omega$	0.61 rad	0.17 rad
$\tau$	3 d	24 d

, where P is the period of the orbit, A is the amplitude of the curve, e is the eccentricity of the orbit, $\omega$ is the argument of perigee, and $\tau$ is the last periastron passage. A mean velocity $V_0=5$ m s${}^{-1}$ is assumed. A Gaussian noise perturbation is added with a standard deviation $\sigma =3$ m s${}^{-1}$. To simulate observations, a total of $K=120$ data points are selected from three random time periods (and two planets in the system). Note that the amplitude of the radial velocity curve of the second planet is close to the noise level. We run ATAIS and a standard AIS scheme with the model with one planet and with the model with two planets. The purpose of this simulation is to check the ability of the method to detect the two planets (by approximating the model evidence).

We apply ATAIS and a standard AIS scheme [13] over the space $[\mathbf{\theta },\sigma ]$ for approximating the model evidence $Z=p\left(\mathbf{y}\right)$ (marginal likelihood) of both models (one planet or two planets) with the given data (generated considering two planets). Uniform priors are considered for each parameter: $P\in [0,365]$, $A\in [-20,20]$, $e\in [0,1]$, $\omega \in [0,2\pi ]$, and $\tau \in [0,50]$ (moreover, $\sigma \in [0,30]$ for the standard AIS scheme). The ATAIS algorithm and the standard AIS scheme have been run with $N={10}^6$ and $T=50$ iterations for both the model with one planet and the model with two planets. In both cases, we consider the same Gaussian proposal with a starting standard deviation of 5 for each component (note that the standard AIS scheme works in higher dimensional space due the inference over $\sigma$). To decide which model is more probable, the model evidence Z of each model is estimated. More specifically, we approximate the one-planet model ${\widehat{Z}}_1={\widehat{p}}_1\left(\mathbf{y}\right)$ and the two-planet model ${\widehat{Z}}_2={\widehat{p}}_2\left(\mathbf{y}\right)$ with the ATAIS algorithm and the standard AIS scheme. When ${\widehat{Z}}_1>{\widehat{Z}}_2$, we select the first model; otherwise, if ${\widehat{Z}}_1<{\widehat{Z}}_2$, we select the second one. The true model is the two-planet model, as the simulated data were generated from that model. After 500 independent runs, the percentage of correct detection of the true model for ATAIS is $\approx 98\%$, whereas with the standard AIS scheme is only $\approx 56\%$. This is due to the difficulty of making inference jointly over $[\mathbf{\theta },\sigma ]$. Let us denote the Bayesian factor as $B=Z_2/Z_1$. In ATAIS, the expected value of the ratio between the model evidences (averaged over the 500 runs) is $E\left[B\right]\approx 5·{10}^3$ with a relative variance of $\frac{E\left[{(B-E\left[B\right])}^2\right]}{E{\left[B\right]}^2}\approx 0.04$. In the case of the standard AIS, we have $E\left[B\right]\approx 16.32$ and $\frac{E\left[{(B-E\left[B\right])}^2\right]}{E{\left[B\right]}^2}\approx 0.15$. Therefore, for ATAIS, the model with two planets is clearly more probable than the model with one planet.

The fitted curves, corresponding to the vector of parameters ${\widehat{\mathbf{\theta }}}_{\mathtt{MAP}}$ obtained with ATAIS, are shown in Figure 5. From the figure, it is not clear which model better fits the simulated observations (blue points), although the model with two planets seems to better fit the observations in the time period from 200 to 300 days. The values of ${\widehat{\mathbf{\theta }}}_{\mathtt{MAP}}$, obtained in one specific run by ATAIS, are given in

Table 4. The value of ${\widehat{\mathbf{\theta }}}_{\mathtt{MAP}}$ and the variances of the marginal posteriors for the 2-planets model (with $K=120$ data points).

Parameter	Planet 1		Planet 2
	${\widehat{\mathit{\theta }}}_{\mathtt{MAP}}$	$\mathbf{Var}\left(\mathit{\theta }\right|\mathit{y})$	${\widehat{\mathit{\theta }}}_{\mathtt{MAP}}$ —Planet 2	$\mathbf{Var}\left(\mathit{\theta }\right|\mathit{y})$
P	14.99 d	0.18	110.39 d	11.28
K	23.78 m s${}^{-1}$	0.52	3.50 m s${}^{-1}$	0.44
e	0.05	0.047	0.00	0.003
$\omega$	7.69 rad	0.61	0.68 rad	0.82
$\tau$	6.8 d	0.76	7.96 d	20.31

. We notice that $\omega$ and $\tau$ are highly correlated and more iterations may be needed to obtain the actual global maximum, but the remaining parameters obtained from ${\widehat{\mathbf{\theta }}}_{\mathtt{MAP}}$ are similar to the simulated values. In addition, the amplitude of the curve of the second planet is close to the intensity of the noise, making it difficult to derive the best fit for that planet. Summarizing, our results show the method is able to discriminate between a model with one planet (with six dimensions of the inference problem) and a model with two planets (with 11 dimensions of the inference problem), for this particular simulation. Finally, the evolution of the automatic tempering parameter ${\widehat{\sigma }}_{\mathtt{ML}}^{\left(t\right)}$ is shown in Figure 6. The dashed line is the evolution of ${\widehat{\sigma }}_{\mathtt{ML}}^{\left(t\right)}$ for the single-planet model, whereas the continuous line is the evolution of ${\widehat{\sigma }}_{\mathtt{ML}}^{\left(t\right)}$ for the model with two planets. In this second model, the tempering parameter reaches a smaller value, as expected.

7. Conclusions

We have proposed a novel AIS scheme for Bayesian inversion problems where an automatic tempering procedure is implemented (called ATAIS). The inference of the variables of interest $\theta$ and the noise power ${\sigma }^2$ is divided. A sampling strategy is considered for $\theta$ and an optimization approach is employed for ${\sigma }^2$. Thus, ATAIS performs an iterative procedure, alternating sampling and optimization steps. Therefore, the proposed scheme deals with a sequence of tempered posteriors according to the current estimation of the noise power. We have also discussed the possibility of approximating the marginal posterior of $\sigma$ without additional evaluations of the complex model. Furthermore, the complete Bayesian analysis regarding the complete joint posterior is possible as discussed in Section 5, again without any additional evaluations of the likelihood function.

Several simulations are provided and the application to a sophisticated astronomical model has been considered, where the number of planets in the system is detected by the analysis of the marginal likelihood. The results show the benefits of the proposed scheme. For instance, in the astronomical example, the percentage of correct detection of the true model obtained by ATAIS is ≈$98\%$, whereas with the standard AIS scheme is only ≈$56\%$. As future research, we plan to extend the ATAIS scheme in order to deal with an observation model with correlated noise perturbations (for instance, using a Gaussian Process). Moreover, the use of parallel AIS schemes (or MCMC algorithms) will be also considered. A combination of parallel MCMC chains and AIS schemes can be found in the so-called layered AIS method and other similar approaches [21,22]. This idea seems particularly interesting for improving the inference with radial velocity models.