Geometric Ergodicity of the Random Walk Metropolis with Position-Dependent Proposal Covariance

Abstract

We consider a Metropolis–Hastings method with proposal $\mathcal{N}(x,hG{\left(x\right)}^{-1})$, where x is the current state, and study its ergodicity properties. We show that suitable choices of $G\left(x\right)$ can change these ergodicity properties compared to the Random Walk Metropolis case $\mathcal{N}(x,h\Sigma )$, either for better or worse. We find that if the proposal variance is allowed to grow unboundedly in the tails of the distribution then geometric ergodicity can be established when the target distribution for the algorithm has tails that are heavier than exponential, in contrast to the Random Walk Metropolis case, but that the growth rate must be carefully controlled to prevent the rejection rate approaching unity. We also illustrate that a judicious choice of $G\left(x\right)$ can result in a geometrically ergodic chain when probability concentrates on an ever narrower ridge in the tails, something that is again not true for the Random Walk Metropolis.

1. Introduction

Markov chain Monte Carlo (MCMC) methods are techniques for estimating expectations with respect to some probability measure $\pi (·)$, which need not be normalised. This is done by sampling a Markov chain which has limiting distribution $\pi (·)$, and computing empirical averages. A popular form of MCMC is the Metropolis–Hastings algorithm [1,2], where at each time step a ‘proposed’ move is drawn from some candidate distribution, and then accepted with some probability, otherwise the chain stays at the current point. Interest lies in finding choices of candidate distribution that will produce sensible estimators for expectations with respect to $\pi (·)$.

The quality of these estimators can be assessed in many different ways, but a common approach is to understand conditions on $\pi (·)$ that will result in a chain which converges to its limiting distribution at a geometric rate. If such a rate can be established, then a Central Limit Theorem will exist for expectations of functionals with finite second absolute moment under $\pi (·)$ if the chain is reversible.

A simple yet often effective choice is a symmetric candidate distribution centred at the current point in the chain (with a fixed variance), resulting in the Random Walk Metropolis (RWM) (e.g., [3]). The convergence properties of a chain produced by the RWM are well-studied. In one dimension, essentially convergence is geometric if $\pi \left(x\right)$ decays at an exponential or faster rate in the tails [4], while in higher dimensions an additional curvature condition is required [5]. Slower rates of convergence have also been established in the case of heavier tails [6].

Recently, some MCMC methods were proposed which generalise the RWM, whereby proposals are still centred at the current point x and symmetric, but the variance changes with x [7,8,9,10,11]. An extension to infinite-dimensional Hilbert spaces is also suggested in Reference [12]. The motivation is that the chain can become more ‘local’, perhaps making larger jumps when out in the tails, or mimicking the local dependence structure of $\pi (·)$ to propose more intelligent moves. Designing MCMC methods of this nature is particularly relevant for modern Bayesian inference problems, where posterior distributions are often high dimensional and exhibit nonlinear correlations [13]. We term this approach the Position-dependent Random Walk Metropolis (PDRWM), although technically this is a misnomer, since proposals are no longer random walks. Other choices of candidate distribution designed with distributions that exhibit nonlinear correlations were introduced in Reference [13]. Although powerful, these require derivative information for $log\pi \left(x\right)$, something which can be unavailable in modern inference problems (e.g., [14]). We note that no such information is required for the PDRWM, as shown by the particular cases suggested in References [7,8,9,10,11]. However, there are relations between the approaches, to the extent that understanding how the properties of the PDRWM differ from the standard RWM should also aid understanding of the methods introduced in Reference [13].

In this article, we consider the convergence rate of a Markov chain generated by the PDRWM to its limiting distribution. Our main interest lies in whether this generalisation can change these ergodicity properties compared to the standard RWM with fixed covariance. We focus on the case in which the candidate distribution is Gaussian, and illustrate that such changes can occur in several different ways, either for better or worse. Our aim is not to give a complete characterisation of the approach, but rather to illustrate the possibilities through carefully chosen examples, which are known to be indicative of more general behaviour.

In Section 2 necessary concepts about Markov chains are briefly reviewed, before the PDRWM is introduced in Section 3. Some results in the one-dimensional case are given in Section 4, before a higher-dimensional model problem is examined in Section 5. Throughout $\pi (·)$ denotes a probability measure (we use the terms probability measure and distribution synonymously), and $\pi \left(x\right)$ its density with respect to Lebesgue measure $dx$.

Since an early version of this work appeared online, some contributions to the literature were made that are worthy of mention. A Markov kernel constructed as a state-dependent mixture is introduced in Reference [15] and its properties are studied in some cases that are similar in spirit to the model problem of Section 5. An algorithm called Directional Metropolis–Hastings, which encompasses a specific instance of the PDRWM, is introduced and studied in Reference [16], and a modification of the same idea is used to develop the Hop kernel within the Hug and Hop algorithm of Reference [17]. Kamatani considers an algorithm designed for the infinite-dimensional setting in Reference [18] of a similar design to that discussed in Reference [12] and studies the ergodicity properties.

2. Markov Chains and Geometric Ergodicity

We will work on the Borel space $(\mathcal{X},\mathcal{B})$, with $\mathcal{X}\subset {\mathbb{R}}^d$ for some $d\ge 1$, so that each $X_t\in \mathcal{X}$ for a discrete-time Markov chain ${\left\{X_t\right\}}_{t\ge 0}$ with time-homogeneous transition kernel $P:\mathcal{X}\times \mathcal{B}\to [0,1]$, where $P(x,A)=\mathbb{P}[X_{i+1}\in A|X_i=x]$ and $P^n(x,A)$ is defined similarly for $X_{i+n}$. All chains we consider will have invariant distribution $\pi (·)$, and be both $\pi$-irreducible and aperiodic, meaning $\pi (·)$ is the limiting distribution from $\pi$-almost any starting point [19]. We use $|·|$ to denote the Euclidean norm.

In Markov chain Monte Carlo the objective is to construct estimators of ${\mathbb{E}}_{\pi }\left[f\right]$, for some $f:\mathcal{X}\to \mathbb{R}$, by computing

$${\widehat{f}}_n=\frac{1}{n}\sum _{i=1}^{n}f\left(X_i\right),X_i\sim P^i(x_0,·).$$

If $\pi (·)$ is the limiting distribution for the chain then P will be ergodic, meaning ${\widehat{f}}_n\stackrel{a.s.}{\to }{\mathbb{E}}_{\pi }\left[f\right]$ from $\pi$-almost any starting point. For finite n the quality of ${\widehat{f}}_n$ intuitively depends on how quickly $P^n(x,·)$ approaches $\pi (·)$. We call the chain geometrically ergodic if

$$\parallel P^n{(x,·)-\pi (·)\parallel }_{TV}\le M\left(x\right){\rho }^n,$$

(1)

from $\pi$-almost any $x\in \mathcal{X}$, for some $M>0$ and $\rho <1$, where ${\parallel \mu (·)-\nu (·)\parallel }_{TV}:={sup}_{A\in \mathcal{B}}|\mu \left(A\right)-\nu \left(B\right)|$ is the total variation distance between distributions $\mu (·)$ and $\nu (·)$ [19].

For $\pi$-reversible Markov chains geometric ergodicity implies that if ${\mathbb{E}}_{\pi }\left[f^2\right]<\infty$ for some $f:\mathcal{X}\to \mathbb{R}$, then

$$\sqrt{n}\left({\widehat{f}}_n-{\mathbb{E}}_{\pi }\left[f\right]\right)\stackrel{d}{\to }\mathcal{N}\left(0,v(P,f)\right),$$

(2)

for some asymptotic variance $v(P,f)$ [20]. Equation (2) enables the construction of asymptotic confidence intervals for ${\widehat{f}}_n$.

In practice, geometric ergodicity does not guarantee that ${\widehat{f}}_n$ will be a sensible estimator, as $M\left(x\right)$ can be arbitrarily large if the chain is initialised far from the typical set under $\pi (·)$, and $\rho$ may be very close to 1. However, chains which are not geometrically ergodic can often either get ‘stuck’ for a long time in low-probability regions or fail to explore the entire distribution adequately, sometimes in ways that are difficult to diagnose using standard MCMC diagnostics.

Establishing Geometric Ergodicity

It is shown in Chapter 15 of Reference [21] that Equation (1) is equivalent to the condition that there exists a Lyapunov function $V:\mathcal{X}\to [1,\infty )$ and some $\lambda <1,b<\infty$ such that

$$PV\left(x\right)\le \lambda V\left(x\right)+b{\mathbb{I}}_C\left(x\right),$$

(3)

where $PV\left(x\right):=\int V\left(y\right)P(x,dy)$. The set $C\subset \mathcal{X}$ must be small, meaning that for some $m\in \mathbb{N}$, $\epsilon >0$ and probability measure $\nu (·)$

$$P^m(x,A)\ge \epsilon \nu \left(A\right),$$

(4)

for any $x\in C$ and $A\in \mathcal{B}$. Equations (3) and (4) are referred to as drift and minorisation conditions. Intuitively, C can be thought of as the centre of the space, and Equation (3) ensures that some one dimensional projection of ${\left\{X_t\right\}}_{t\ge 0}$ drifts towards C at a geometric rate when outside. In fact, Equation (3) is sufficient for the return time distribution to C to have geometric tails [21]. Once in C, (4) ensures that with some probability the chain forgets its past and hence regenerates. This regeneration allows the chain to couple with another initialised from $\pi (·)$, giving a bound on the total variation distance through the coupling inequality (e.g., [19]). More intuition is given in Reference [22].

Transition kernels considered here will be of the Metropolis–Hastings type, given by

$$P(x,dy)=\alpha (x,y)Q(x,dy)+r\left(x\right){\delta }_x\left(dy\right),$$

(5)

where $Q(x,dy)=q\left(y\right|x)dy$ is some candidate kernel, $\alpha$ is called the acceptance rate and $r\left(x\right)=1-\int \alpha (x,y)Q(x,dy)$. Here we choose

$$\alpha (x,y)=1\wedge \frac{\pi \left(y\right)q\left(x\right|y)}{\pi \left(x\right)q\left(y\right|x)},$$

(6)

where $a\wedge b$ denotes the minimum of a and b. This choice implies that P satisfies detailed balance for $\pi (·)$ [23], and hence the chain is $\pi$-reversible (note that other choices for $\alpha$ can result in non-reversible chains, see Reference [24] for details).

Roberts and Tweedie [5], following on from Reference [21], introduced the following regularity conditions.

Theorem 1.

(Roberts and Tweedie). Suppose that $\pi \left(x\right)$ is bounded away from 0 and ∞ on compact sets, and there exists ${\delta }_q>0$ and ${\epsilon }_q>0$ such that for every x

$$|x-y|\le {\delta }_q\Rightarrow q\left(y\right|x)\ge {\epsilon }_q.$$

Then the chain with kernel (5) is ${\mu }^{Leb}$-irreducible and aperiodic, and every nonempty compact set is small.

For the choices of Q considered in this article these conditions hold, and we will restrict ourselves to forms of $\pi \left(x\right)$ for which the same is true (apart from a specific case in Section 5). Under Theorem 1 then (1) only holds if a Lyapunov function $V:\mathcal{X}\to [1,\infty ]$ with ${\mathbb{E}}_{\pi }\left[V\right]<\infty$ exists such that

$$\underset{\left|x\right|\to \infty }{lim\; sup}\frac{PV\left(x\right)}{V\left(x\right)}<1.$$

(7)

when P is of the Metropolis–Hastings type, (7) can be written

$$\underset{\left|x\right|\to \infty }{lim\; sup}\int \left[\frac{V\left(y\right)}{V\left(x\right)}-1\right]\alpha (x,y)Q(x,dy)<0.$$

(8)

In this case, a simple criterion for lack of geometric ergodicity is

$$\underset{\left|x\right|\to \infty }{lim\; sup}r\left(x\right)=1.$$

(9)

Intuitively this implies that the chain is likely to get ‘stuck’ in the tails of a distribution for large periods.

Jarner and Tweedie [25] introduce a necessary condition for geometric ergodicity through a tightness condition.

Theorem 2.

(Jarner and Tweedie). If for any $\epsilon >0$ there is a $\delta >0$ such that for all $x\in \mathcal{X}$

$$P(x,B_{\delta }\left(x\right))>1-\epsilon ,$$

where $B_{\delta }\left(x\right):=\{y\in \mathcal{X}:d(x,y)<\delta \}$, then a necessary condition for P to produce a geometrically ergodic chain is that for some $s>0$

$$\int e^{s\left|x\right|}\pi \left(dx\right)<\infty .$$

The result highlights that when $\pi (·)$ is heavy-tailed the chain must be able to make very large moves and still be capable of returning to the centre quickly for (1) to hold.

3. Position-Dependent Random Walk Metropolis

In the RWM, $Q(x,dy)=q(y-x)dy$ with $q(y-x)=q(x-y)$, meaning (6) reduces to $\alpha (x,y)=1\wedge \pi \left(y\right)/\pi \left(x\right)$. A common choice is $Q(x,·)=\mathcal{N}(x,h\Sigma )$, with $\Sigma$ chosen to mimic the global covariance structure of $\pi (·)$ [3]. Various results exist concerning the optimal choice of h in a given setting (e.g., [26]). It is straightforward to see that Theorem 2 holds here, so that the tails of $\pi \left(x\right)$ must be uniformly exponential or lighter for geometric ergodicity. In one dimension this is in fact a sufficient condition [4], while for higher dimensions additional conditions are required [5]. We return to this case in Section 5.

In the PDRWM $Q(x,·)=\mathcal{N}(x,hG{\left(x\right)}^{-1})$, so (6) becomes

$$\alpha (x,y)=1\wedge \frac{{\pi \left(y\right)\left|G\left(y\right)\right|}^{\frac{1}{2}}}{{\pi \left(x\right)\left|G\left(x\right)\right|}^{\frac{1}{2}}}exp\left(-\frac{1}{2}{(x-y)}^T[G\left(y\right)-G\left(x\right)](x-y)\right).$$

The motivation for designing such an algorithm is that proposals are more able to reflect the local dependence structure of $\pi (·)$. In some cases this dependence may vary greatly in different parts of the state-space, making a global choice of $\Sigma$ ineffective [9].

Readers familiar with differential geometry will recognise the volume element ${\left|G\left(x\right)\right|}^{1/2}dx$ and the linear approximations to the distance between x and y taken at each point through $G\left(x\right)$ and $G\left(y\right)$ if $\mathcal{X}$ is viewed as a Riemannian manifold with metric G. We do not explore these observations further here, but the interested reader is referred to Reference [27] for more discussion.

The choice of $G\left(x\right)$ is an obvious question. In fact, specific variants of this method have appeared on many occasions in the literature, some of which we now summarise.

• Tempered Langevin diffusions [8] $G\left(x\right)=\pi \left(x\right)I$. The authors highlight that the diffusion with dynamics $d{X}_t={\pi }^{-\frac{1}{2}}\left(X_t\right)d{W}_t$ has invariant distribution $\pi (·)$, motivating the choice. The method was shown to perform well for a bi-modal $\pi \left(x\right)$, as larger jumps are proposed in the low density region between the two modes.
• State-dependent Metropolis [7] $G\left(x\right)={(1+|x\left|\right)}^{-b}$. Here the intuition is simply that $b>0$ means larger jumps will be made in the tails. In one dimension the authors compare the expected squared jumping distance $\mathbb{E}\left[{(X_{i+1}-X_i)}^2\right]$ empirically for chains exploring a $\mathcal{N}(0,1)$ target distribution, choosing b adaptively, and found $b\approx 1.6$ to be optimal.
• Regional adaptive Metropolis–Hastings [7,11]. $G{\left(x\right)}^{-1}={\sum }_{i=1}^m\mathbb{I}(x\in {\mathcal{X}}_i){\Sigma }_i$. In this case the state-space is partitioned into ${\mathcal{X}}_1\cup ...\cup {\mathcal{X}}_m$, and a different proposal covariance ${\Sigma }_i$ is learned adaptively in each region $1\le i\le m$. An extension which allows for some errors in choosing an appropriate partition is discussed in [11]
• Localised Random Walk Metropolis [10]. $G{\left(x\right)}^{-1}={\sum }_{k=1}^m{\check{q}}_{\theta }\left(k\right|x){\Sigma }_k$. Here ${\check{q}}_{\theta }\left(k\right|x)$ are weights based on approximating $\pi \left(x\right)$ with some mixture of Normal/Student’s t distributions, using the approach suggested in Reference [28]. At each iteration of the algorithm a mixture component k is sampled from ${\check{q}}_{\theta }(·|x)$, and the covariance ${\Sigma }_k$ is used for the proposal $Q(x,dy)$.
• Kernel adaptive Metropolis–Hastings [9]. $G{\left(x\right)}^{-1}={\gamma }^{2}I+{\nu }^2{M}_{x}H{M}_x^T$, where $M_x=2[{\nabla }_{x}k(z_1,x),...,{\nabla }_{x}k(z_n,x)]$ for some kernel function k and n past samples $\{z_1,...,z_n\}$, $H=I-(1/n){\mathbf{1}}_{n\times n}$ is a centering matrix (the $n\times n$ matrix ${\mathbf{1}}_{n\times n}$ has 1 as each element), and $\gamma$, $\nu$ are tuning parameters. The approach is based on performing nonlinear principal components analysis on past samples from the chain to learn a local covariance. Illustrative examples for the case of a Gaussian kernel show that $M_{x}H{M}_x^T$ acts as a weighted empirical covariance of samples z, with larger weights given to the $z_i$ which are closer to x [9].

The latter cases also motivate any choice of the form

$$G{\left(x\right)}^{-1}=\sum _{i=1}^{n}w(x,z_i){(z_i-x)}^T(z_i-x)$$

for some past samples $\{z_1,...,z_n\}$ and weight function $w:\mathcal{X}\times \mathcal{X}\to [0,\infty )$ with ${\sum }_{i}w(x,z_i)=1$ that decays as $|x-z_i|$ grows, which would also mimic the local curvature of $\pi (·)$ (taking care to appropriately regularise and diminish adaptation so as to preserve ergodicity, as outlined in Reference [10]).

Some of the above schemes are examples of adaptive MCMC, in which a candidate from among a family of Markov kernels $\{P_{\theta }:\theta \in \mathsf{\Theta }\}$ is selected by learning the parameter $\theta \in \mathsf{\Theta }$ during the simulation [10]. Additional conditions on the adaptation process (i.e., the manner in which $\theta$ is learned) are required to establish ergodicity results for the resulting stochastic processes. We consider the decisions on how to learn $\theta$ appropriately to be a separate problem and beyond the scope of the present work, and instead focus attention on establishing geometric ergodicity of the base kernels $P_{\theta }$ for any fixed $\theta \in \mathsf{\Theta }$. We note that this is typically a pre-requisite for establishing convergence properties of any adaptive MCMC method [10].

4. Results in One Dimension

Here we consider two different general scenarios as $\left|x\right|\to \infty$, i) $G\left(x\right)$ is bounded above and below, and ii) $G\left(x\right)\to 0$ at some specified rate. Of course there is also the possibility that $G\left(x\right)\to \infty$, though intuitively this would result in chains that spend a long time in the tails of a distribution, so we do not consider it (if $G\left(x\right)\to \infty$ then chains will in fact exhibit the negligible moves property studied in Reference [29]). Proofs to Propositions in Section 4 and Section 5 can be found in Appendix A.

We begin with a result that emphasizes that a growing variance is a necessary requirement for geometric ergodicity in the heavy-tailed case.

Proposition 1.

If $G\left(x\right)\ge {\sigma }^{-2}$ for some ${\sigma }^{-2}>0$, then unless $\int e^{\eta \left|x\right|}\pi \left(dx\right)<\infty$ for some $\eta >0$ the PDRWM cannot produce a geometrically ergodic Markov chain.

The above is a simple extension of a result that is well-known in the RWM case. Essentially the tails of the distribution should be exponential or lighter to ensure fast convergence. This motivates consideration of three different types of behaviour for the tails of $\pi (·)$.

Assumption 1.

The density $\pi \left(x\right)$ satisfies one of the following tail conditions for all $y,x\in \mathcal{X}$ such that $\left|y\right|>\left|x\right|>t$, for some finite $t>0$.

• $\pi \left(y\right)/\pi \left(x\right)\le exp\{-a(\left|y\right|-\left|x\right|\left)\right\}$ for some $a>0$
• $\pi \left(y\right)/\pi \left(x\right)\le {exp\{-a(\left|y\right|}^{\beta }-{\left|x\right|}^{\beta }\left)\right\}$ for some $a>0$ and $\beta \in (0,1)$
• $\pi \left(y\right)/\pi \left(x\right)\le {\left(\left|x\right|/\left|y\right|\right)}^p$ for some $p>1$.

Naturally Assumption 1 implies 2 and Assumption 2 implies 3. If Assumption 1 is not satisfied then $\pi (·)$ is generally called heavy-tailed. When $\pi \left(x\right)$ satisfies Assumption 2 or 3 but not 1, then the RWM typically fails to produce a geometrically ergodic chain [4]. We show in the sequel, however, that this is not always the case for the PDRWM. We assume the below assumptions for $G\left(x\right)$ to hold throughout this section.

Assumption 2.

The function $G:\mathcal{X}\to (0,\infty )$ is bounded above by some ${\sigma }_b^{-2}<\infty$ for all $x\in \mathcal{X}$, and bounded below for all $x\in \mathcal{X}$ with $\left|x\right|<t$, for some $t>0$.

The heavy-tailed case is known to be a challenging scenario, but the RWM will produce a geometrically ergodic Markov chain if $\pi \left(x\right)$ is log-concave. Next we extend this result to the case of sub-quadratic variance growth in the tails.

Proposition 2.

If $\exists r<\infty$ such that $G\left(x\right)\propto {\left|x\right|}^{-\gamma }$ whenever $\left|x\right|>r$, then the PDRWM will produce a geometrically ergodic chain in both of the following cases:

• $\pi \left(x\right)$ satisfies Assumption 1 and $\gamma \in [0,2)$
• $\pi \left(x\right)$ satisfies Assumption 2 for some $\beta \in (0,1)$ and $\gamma \in \left(2\right(1-\beta ),2)$

The second part of Proposition 2 is not true for the RWM, for which Assumption 2 alone is not sufficient for geometric ergodicity [4].

We do not provide a complete proof that the PDRWM will not produce a geometrically ergodic chain when only Assumption 3 holds and $G\left(x\right)\propto {\left|x\right|}^{-\gamma }$ for some $\gamma <2$, but do show informally that this will be the case. Assuming that in the tails $\pi \left(x\right)\propto {\left|x\right|}^{-p}$ for some $p>1$ then for large x

$$\alpha (x,x+c{x}^{\gamma /2})=1\wedge {\left(\frac{x}{x+c{x}^{\gamma /2}}\right)}^{p+\gamma /2}exp\left(-\frac{c^2{x}^{\gamma }}{2h}\left[\frac{1}{{(x+c{x}^{\gamma /2})}^{\gamma }}-\frac{1}{x^{\gamma }}\right]\right).$$

(10)

The first expression on the right hand side converges to 1 as $x\to \infty$, which is akin to the case of fixed proposal covariance. The second term will be larger than one for $c>0$ and less than one for $c<0$. So the algorithm will exhibit the same ‘random walk in the tails’ behaviour which is often characteristic of the RWM in this scenario, meaning that the acceptance rate fails to enforce a geometric drift back into the centre of the space.

When $\gamma =2$ the above intuition will not necessarily hold, as the terms in Equation (10) will be roughly constant with x. When only Assumption 3 holds, it is, therefore, tempting to make the choice $G\left(x\right)=x^{-2}$ for $\left|x\right|>r$. Informally we can see that such behaviour may lead to a favourable algorithm if a small enough h is chosen. For any fixed $x>r$ a typical proposal will now take the form $y=(1+\xi \sqrt{h})x$, where $\xi \sim N(0,1)$. It therefore holds that

$$y=e^{\xi \sqrt{h}}x+r(x,h,\xi ),$$

(11)

where for any fixed x and $\xi$ the term $r(x,h,\xi )/\sqrt{h}\to 0$ as $h\to 0$. The first term on the right-hand side of Equation (11) corresponds to the proposal of the multiplicative Random Walk Metropolis, which is known to be geometrically ergodic under Assumption 3 (e.g., [3]), as this equates to taking a logarithmic transformation of x, which ‘lightens’ the tails of the target density to the point where it becomes log-concave. So in practice we can expect good performance from this choice of $G\left(x\right)$. The above intuition does not, however, provide enough to establish geometric ergodicity, as the final term on the right-hand side of (11) grows unboundedly with x for any fixed choice of h. The difference between the acceptance rates of the multiplicative Random Walk Metropolis and the PDRWM with $G\left(x\right)=x^{-2}$ will be the exponential term in Equation (10). This will instead become polynomial by letting the proposal noise $\xi$ follow a distribution with polynomial tails (e.g., student’s t), which is known to be a favourable strategy for the RWM when only Assumption 3 holds [6]. One can see that if the heaviness of the proposal distribution is carefully chosen then the acceptance rate may well enforce a geometric drift into the centre of the space, though for brevity we restrict attention to Gaussian proposals in this article.

The final result of this section provides a note of warning that lack of care in choosing $G\left(x\right)$ can have severe consequences for the method.

Proposition 3.

If $G\left(x\right)x^2\to 0$ as $\left|x\right|\to \infty$, then the PDRWM will not produce a geometrically ergodic Markov chain.

The intuition for this result is straightforward when explained. In the tails, typically $|y-x|$ will be the same order of magnitude as $\sqrt{G{\left(x\right)}^{-1}}$, meaning $|y-x|/\left|x\right|$ grows arbitrarily large as $\left|x\right|$ grows. As such, proposals will ‘overshoot’ the typical set of the distribution, sending the sampler further out into the tails, and will therefore almost always be rejected. The result can be related superficially to a lack of geometric ergodicity for Metropolis–Hastings algorithms in which the proposal mean is comprised of the current state translated by a drift function (often based in $\nabla log\pi \left(x\right)$) when this drift function grows faster than linearly with $\left|x\right|$ (e.g., [30,31]).

5. A Higher-Dimensional Case Study

An easy criticism of the above analysis is that the one-dimensional scenario is sometimes not indicative of the more general behaviour of a method. We note, however, that typically the geometric convergence properties of Metropolis–Hastings algorithms do carry over somewhat naturally to more than one dimension when $\pi (·)$ is suitably regular (e.g., [5,32]). Because of this we expect that the growth conditions specified above could be supplanted onto the determinant of $G\left(x\right)$ when the dimension is greater than one (leaving the details of this argument for future work).

A key difference in the higher-dimensional setting is that $G\left(x\right)$ now dictates both the size and direction of proposals. In the case $G{\left(x\right)}^{-1}=\Sigma$, some additional regularity conditions on $\pi \left(x\right)$ are required for geometric ergodicity in more than one dimension, outlined in References [5,32]. An example is also given in Reference [5] of the simple two-dimensional density $\pi (x,y)\propto exp(-x^2-y^2-x^2{y}^2)$, which fails to meet these criteria. The difficult models are those for which probability concentrates on a ridge in the tails, which becomes ever narrower as $\left|x\right|$ increases. In this instance, proposals from the RWM are less and less likely to be accepted as $\left|x\right|$ grows. Another well-known example of this phenomenon is the funnel distribution introduced in Reference [33].

To explore the behaviour of the PDRWM in this setting, we design a model problem, the staircase distribution, with density

$$\mathfrak{s}\left(x\right)\propto 3^{-\lfloor x_2\rfloor }{\mathbb{I}}_R\left(x\right),R:=\{y\in {\mathbb{R}}^2;y_2\ge 1,|y_1|\le 3^{1-\lfloor y_2\rfloor }\},$$

(12)

where $\lfloor z\rfloor$ denotes the integer part of $z>0$. Graphically the density is a sequence of cuboids on the upper-half plane of ${\mathbb{R}}^2$ (starting at $y_2=1$), each centred on the vertical axis, with each successive cuboid one third of the width and height of the previous. The density resembles an ever narrowing staircase, as shown in Figure 1.

We denote by $Q_R$ the proposal kernel associated with the Random Walk Metropolis algorithm with fixed covariance $h\Sigma$. In fact, the specific choice of h and $\Sigma$ does not matter provided that the result is positive-definite. For the PDRWM we denote by $Q_P$ the proposal kernel with covariance matrix

$$hG{\left(x\right)}^{-1}=\left(\begin{array}{cc}{3}^{-2\lfloor x_2\rfloor }& 0\\ 0& 1\end{array}\right),$$

which will naturally adapt the scale of the first coordinate to the width of the ridge.

Proposition 4.

The Metropolis–Hastings algorithm with proposal $Q_R$ does not produce a geometrically ergodic Markov chain when $\pi \left(x\right)=\mathfrak{s}\left(x\right)$.

The design of the PDRWM proposal kernel $Q_P$ in this instance is such that the proposal covariance reduces at the same rate as the width of the stairs, therefore naturally adapting the proposal to the width of the ridge on which the density concentrates. This state-dependent adaptation results in a geometrically ergodic chain, as shown in the below result.

Proposition 5.

The Metropolis–Hastings algorithm with proposal $Q_P$ produces a geometrically ergodic Markov chain when $\pi \left(x\right)=\mathfrak{s}\left(x\right)$.

6. Discussion

In this paper we have analysed the ergodic behaviour of a Metropolis–Hastings method with proposal kernel $Q(x,·)=\mathcal{N}(x,hG{\left(x\right)}^{-1})$. In one dimension we have characterised the behaviour in terms of growth conditions on $G{\left(x\right)}^{-1}$ and tail conditions on the target distribution, and in higher dimensions a carefully constructed model problem is discussed. The fundamental question of interest was whether generalising an existing Metropolis–Hastings method by allowing the proposal covariance to change with position can alter the ergodicity properties of the sampler. We can confirm that this is indeed possible, either for the better or worse, depending on the choice of covariance. The take home points for practitioners are (i) lack of sufficient care in the design of $G\left(x\right)$ can have severe consequences (as in Proposition 3), and (ii) careful choice of $G\left(x\right)$ can have much more beneficial ones, perhaps the most surprising of which are in the higher-dimensional setting, as shown in Section 5.

We feel that such results can also offer insight into similar generalisations of different Metropolis–Hastings algorithms (e.g., [13,34]). For example, it seems intuitive that any method in which the variance grows at a faster than quadratic rate in the tails is unlikely to produce a geometrically ergodic chain. There are connections between the PDRWM and some extensions of the Metropolis-adjusted Langevin algorithm [34], the ergodicity properties of which are discussed in Reference [35]. The key difference between the schemes is the inclusion of the drift term $G{\left(x\right)}^{-1}\nabla log\pi \left(x\right)/2$ in the latter. It is this term which in the main governs the behaviour of the sampler, which is why the behaviour of the PDRWM is different to this scheme. Markov processes are also used in a wide variety of application areas beyond the design of Metropolis–Hastings algorithms (e.g., [36]), and we hope that some of the results established in the present work prove to be beneficial in some of these other settings.

We can apply these results to the specific variants discussed in Section 3. Provided that sensible choices of regions/weights are made and that an adaptation scheme which obeys the diminishing adaptation criterion is employed, the Regional adaptive Metropolis–Hastings, Locally weighted Metropolis and Kernel-adaptive Metropolis–Hastings samplers should all satisfy $G\left(x\right)\to \Sigma$ as $\left|x\right|\to \infty$, meaning they can be expected to inherit the ergodicity properties of the standard RWM (the behaviour in the centre of the space, however, will likely be different). In the State-dependent Metropolis method provided $b<2$ the sampler should also behave reasonably. Whether or not a large enough value of b would be found by a particular adaptation rule is not entirely clear, and this could be an interesting direction of further study. The Tempered Langevin diffusion scheme, however, will fail to produce a geometrically ergodic Markov chain whenever the tails of $\pi \left(x\right)$ are lighter than that of a Cauchy distribution. To allow reasonable tail exploration when this is the case, two pragmatic options would be to upper bound $G{\left(x\right)}^{-1}$ manually or use this scheme in conjunction with another, as there is evidence that the sampler can perform favourably when exploring the centre of a distribution [8]. None of the specific variants discussed here are able to mimic the local curvature of the $\pi \left(x\right)$ in the tails, so as to enjoy the favourable behaviour exemplified in Proposition 5. This is possible using Hessian information as in Reference [13], but should also be possible in some cases using appropriate surrogates.