Interacting Particle Solutions of Fokker–Planck Equations Through Gradient–Log–Density Estimation

Abstract

Fokker–Planck equations are extensively employed in various scientific fields as they characterise the behaviour of stochastic systems at the level of probability density functions. Although broadly used, they allow for analytical treatment only in limited settings, and often it is inevitable to resort to numerical solutions. Here, we develop a computational approach for simulating the time evolution of Fokker–Planck solutions in terms of a mean field limit of an interacting particle system. The interactions between particles are determined by the gradient of the logarithm of the particle density, approximated here by a novel statistical estimator. The performance of our method shows promising results, with more accurate and less fluctuating statistics compared to direct stochastic simulations of comparable particle number. Taken together, our framework allows for effortless and reliable particle-based simulations of Fokker–Planck equations in low and moderate dimensions. The proposed gradient–log–density estimator is also of independent interest, for example, in the context of optimal control.

1. Introduction

Many biological and physical systems are characterised by the presence of stochastic forces that influence their dynamics. These forces may be attributed either to intrinsic or extrinsic sources [1,2,3], i.e., arising either from random fluctuations of constituent subsystems [4,5], or from fluctuating interactions with the environment [6,7].

Often, deterministic analysis of these systems suffices to describe their macroscopic behaviour, and the fluctuations contribute only negligible perturbations around the deterministic dynamics. In systems biology, for example, rate equations describing mean concentrations of considered system’s species have provided a useful description of chemical reaction networks’ dynamics, and have enabled answering invaluable questions pertaining chemical systems comprising large numbers of molecules [8].

However, in several settings, the effect of stochastic forces influences considerably the resulting system’s behaviour by qualitatively altering its evolution. In those settings, random fluctuations have to be accounted for, and thus stochastic analysis becomes essential [9,10]. Phenomena such as stochastic resonance [11,12], noise induced transitions [13,14,15], and stochastic synchronisation, to name a few, are prevalent in many physical systems, and highlight the importance of considering fluctuating forces in the analysis of a system’s behaviour. Manifestations of these phenomena abound in nature and have been encountered in genetics [16], neuroscience [17,18], climate science [12,19,20] and other fields.

Indispensable tools for the analysis of stochastic systems are Kolmogorov equations, and in particular the Fokker–Planck equation (FPE) [21,22]. FPEs characterise the evolution of the probability density functions (PDF) for the state variables of dynamical systems described by stochastic differential equations (SDE). SDEs commonly arise in modeling random effects in systems with continuous state variables [23], or after diffusion approximation of Master equations for systems involving discrete state transitions in time [8,24,25]. The associated FPEs have been widely used for modelling stochastic phenomena in various fields, such as, for example, in physics, finance, biology, neuroscience, traffic flow [26].

Yet, although commonly used, explicit closed-form solutions of FPEs are rarely available [27], especially in settings where the underlying dynamics is nonlinear. In particular, exact analytic solutions may be obtained only for a restricted class of systems following linear dynamics perturbed by white Gaussian noise, and for some nonlinear Hamiltonian systems [21,28]. Further systems that admit analytical treatment independent of system dimension are those with discrete state transitions approximated via van Kampen expansion (linear noise approximation), resulting thus in linear SDEs with time dependent coefficients [9].

Existing numerical approaches for computing Fokker–Planck solutions may be grouped into three broad categories: grid-based, semi-analytical, and sample-based methods. The first category comprises mainly finite difference and finite element methods [29,30,31,32]. These frameworks, based on integration of FPEs employing numerical solvers for partial differential equations, entail computationally demanding calculations with inherent finite spatial resolution [33].

Conversely, semi-analytical approaches try to reduce the number of required computations by assuming conditional Gaussian structures [34], or by employing cumulant neglect closures [35], statistical linearisation [36,37], or stochastic averaging [38]. Although efficient in the settings they are devised for, their applicability is limited since the resulting solutions are imprecise or unstable in certain settings.

On the other hand, in the sample-based category, Monte Carlo methods resort to stochastic integration of a large number of independent stochastic trajectories that as an ensemble represent the probability density [39,40]. These methods are appropriate for computing unbiased estimates of exact expectations from empirical averages. Nevertheless, as we show in the following, cumulants of resulting distributions exhibit strong temporal fluctuations when the number of simulated trajectories is not sufficiently large.

Surprisingly, there is an alternative sample-based approach built on deterministic particle dynamics. In this setting, the particles are not independent, but they rather interact via an (approximated) probability density, and the FPE describes the mean field limit when their number grows to infinity. This approach introduces a bias in the approximated expectations, but significantly reduces the variance for a given particle number.

Recent research, see e.g., [41,42,43,44], has focused on particle methods for models of thermal equilibrium, where the stationary density is known analytically. For these models, interacting particle methods have found interesting new applications in the field of probabilistic Bayesian inference: by treating the Bayesian posterior probability density as the stationary density of a FPE, the particle dynamics provides posterior samples in the long time limit. For this approach, the particle dynamics are constructed by exploiting the gradient structure of the probability flow of the FPE. This involves the relative entropy distance to the equilibrium density as a Lyapunov function. Unfortunately, this structure does not apply to general FPEs in non–equilibrium settings, where the stationary density is usually unknown.

In this article, we introduce a framework for interacting particle systems that may be applied to general types of Fokker–Planck equations. Our approach is based on the fact that the instantaneous effective force on a particle due to diffusion is proportional to the gradient of the logarithm of the exact probability density (GLD). Rather than computing a differentiable estimate of this density (say by a kernel density estimator), we estimate the GLD directly without requiring knowledge of a stationary density. Therefore, we introduce an approximation to the effective force acting on each particle, which becomes exact in the large particle number limit given the consistency of the estimator.

Our approach is motivated by recent developments in the field of machine learning, where GLD estimators have been studied independently and are used to fit probabilistic models to data. An application of these techniques to particle approximations for FPE is, to our knowledge, new. (The approach in [45] uses a GLD estimator different from ours for particle dynamics but with a probability flow towards equilibrium which is not given by a standard FPE.) Furthermore, our method provides straightforward approximations of entropy production rates, which are of primary importance in non–equilibrium statistical physics [46].

This article is organised as follows: Section 2 describes the deterministic particle formulation of the Fokker–Planck equation. Section 3 shows how a gradient of the logarithm of a density may be represented as the solution of a variational problem, while in Section 4 we discuss an empirical approximation of the gradient-log-density. In Section 5, we introduce function classes for which the variational problem may be solved explicitly, while in Section 6 we compare the temporal derivative of empirical expectations based on the particle dynamics with exact results derived from the Fokker–Planck equation. Section 7 is devoted to the class of equilibrium Fokker–Planck equations, where we discuss relations to Stein Variational Gradient Descent and other particle approximations of Fokker–Planck solutions. In Section 8, we show how our method may be extended to general diffusion processes with state-dependent diffusion, while Section 9 discusses how our framework may be employed to simulate second order Langevin dynamics. In Section 10 we demonstrate various aspects of our method by simulating Fokker–Planck solutions for different dynamical models. Finally, we conclude with a discussion and an outlook in Section 11.

2. Deterministic Particle Dynamics for Fokker–Planck Equations

We consider Fokker–Planck equations of the type

$$\frac{\partial p_t\left(x\right)}{\partial t}=-\nabla ·\left[f\left(x\right)p_t\left(x\right)-\frac{{\sigma }^2}{2}\nabla p_t\left(x\right)\right].$$

(1)

Given an initial condition $p_0\left(x\right)$, Equation (1) describes the temporal development of the density $p_t\left(x\right)$ for the random variable $X\left(t\right)\in R^d$ following the stochastic differential equation

$$dX\left(t\right)=f\left(X\right(t\left)\right)dt+\sigma dB\left(t\right).$$

(2)

In Equation (2), $f\left(x\right)\in R^d$ denotes the drift function characterising the deterministic part of the driving force, while $dB\left(t\right)\in R^d$ represents the differential of a vector of independent Wiener processes capturing stochastic, Gaussian white noise excitations. For the moment, we restrict ourselves to state independent and diagonal diffusion matrices, i.e., diffusion matrices independent of $X\left(t\right)$ (additive noise) with diagonal elements ${\sigma }^2$ characterising the noise amplitude in each dimension. Extensions to more general settings are deferred to Section 8.

We may rewrite the FPE Equation (1) in the form of a Liouville equation

$$\frac{\partial p_t\left(x\right)}{\partial t}=-\nabla ·\left[g(x,t)p_t\left(x\right)\right]$$

(3)

for the deterministic dynamical system

$$\frac{dX}{dt}=g(X,t),X\left(0\right)\sim p_0\left(x\right),$$

(4)

(dropping the time argument in $X\left(t\right)$ for simplicity) with velocity field

$$g(x,t)=f\left(x\right)-\frac{{\sigma }^2}{2}\nabla ln{p}_t\left(x\right).$$

(5)

Hence, by evolving an ensemble of N independent realisations of Equation (4) (to be called ’particles’ in the following) according to

$$\frac{d{X}_i}{dt}=g(X_i,t),i=1,\dots ,N{X}_i\left(0\right)\sim p_0\left(x\right),$$

(6)

we obtain an empirical approximation to the density $p_t\left(x\right)$.

Since the only source of randomness in Equation (4) can be attributed to the initial conditions $X_i\left(0\right)$, averages computed from the particle approximation (Equation (6)) are expected to have smaller variance compared to N independent realisations of the SDE (Equation (2)). Unfortunately, this approach requires perfect knowledge of the unknown instantaneous density $p_t\left(x\right)$ (Equation (5)), which is actually the unknown quantity of interest.

Here, we circumvent this issue by introducing statistical estimators for the term $\nabla ln{p}_t\left(x\right)$, computed from the entire ensemble $\left(X_1\left(t\right)\right),\dots ,X_N\left(t\right))$ of particles at time t. Although this additional approximation introduces interactions among the particles via the estimator, for sufficiently large particle number N, fluctuations of the estimator are expected to be negligible and the limiting dynamics should converge to its mean field limit (Equation (4)) provided the estimator is asymptotically consistent. Thus, rather than computing a differentiable approximation to $p_t\left(x\right)$ from the particles, e.g., by a kernel density estimator, we show in the following section, how the function $\nabla ln{p}_t\left(x\right)$ may be directly estimated from samples of $p_t\left(x\right)$.

3. Variational Representation of Gradient–Log–Densities

To construct a gradient–log–density (GLD) estimator we rely on a variational representation introduced by Hyvärinen in his score–matching approach for the estimation of non–normalised statistical models [47]. We favoured this approach over other estimators [48,49] due to its flexibility to adapt to different function classes chosen to approximate the GLD.

Here, we use a slightly more general representation compared to [47] allowing for an extra arbitrary reference function $r\left(x\right)=(r^{\left(1\right)}\left(x\right),\dots ,r^{\left(d\right)}\left(x\right))$ such that the component $\alpha$ of the gradient is represented as

$${\partial }_{\alpha }lnp\left(x\right)=r^{\left(\alpha \right)}\left(x\right)+arg\underset{\varphi }{min}{\mathcal{L}}_{\alpha }^r[\varphi ,p]\left(x\right),$$

(7)

where ${\partial }_{\alpha }\doteq \frac{\partial }{\partial x^{\left(\alpha \right)}}$ stands for the partial derivative with respect to coordinate $\alpha$ of the vector $x\equiv (x^{\left(1\right)},\dots x^{\left(d\right)})$.

The cost function is defined as an expectation with respect to the density $p\left(x\right)$ by

$${\mathcal{L}}_{\alpha }^r[\varphi ,p]=\int p\left(x\right)\left({\varphi }^2\left(x\right)+2r^{\left(\alpha \right)}\left(x\right)\varphi \left(x\right)+2{\partial }_{\alpha }\varphi \left(x\right)\right)dx,$$

(8)

with $dx$ representing the volume element in $R^d$. To obtain this relation, we use integration by parts (assuming appropriate behaviour of densities and $\varphi$ at boundaries), and get

$$\begin{array}{c}\hfill {\mathcal{L}}_{\alpha }^r[\varphi ,p]=\int p\left(x\right){\left(\varphi \left(x\right)+r^{\left(\alpha \right)}\left(x\right)-{\partial }_{\alpha }lnp\left(x\right)\right)}^{2}dx-\int p\left(x\right){\left({\partial }_{\alpha }lnp\left(x\right)-r^{\left(\alpha \right)}\left(x\right)\right)}^{2}dx.\end{array}$$

(9)

Minimisation with respect to $\varphi$ yields Equation (7).

4. Gradient–Log–Density Estimator

To transform the variational formulation into a GLD estimator based on N sample points $(X_1,\dots ,X_N)$, we replace the density $p\left(x\right)$ in Equation (8) by the empirical distribution ${\widehat{p}}_t\left(x\right)=\frac{1}{N}{\sum }_{i=1}^N\delta (x-X_i\left(t\right))$, i.e.,

$${\mathcal{L}}_{\alpha }^r[\varphi ,p_t]\approx {\mathcal{L}}_{\alpha }^r[\varphi ,{\widehat{p}}_t]=\frac{1}{N}\sum _{i=1}^N\left({\varphi }^2\left(X_i\right)+2r^{\left(\alpha \right)}\left(X_i\right)\varphi \left(X_i\right)+2{\partial }_{\alpha }\varphi \left(X_i\right)\right),$$

(10)

and

$${\partial }_{\alpha }ln{p}_t\left(x\right)\approx r^{\left(\alpha \right)}\left(x\right)+arg\underset{\varphi \in \mathcal{F}}{min}{\mathcal{L}}_{\alpha }^r[\varphi ,{\widehat{p}}_t]\left(x\right),$$

(11)

where $\mathcal{F}$ is an appropriately chosen family of functions with controllable complexity. By introducing the estimator of Equation (11) in Equation (6), we obtain a particle representation for the Fokker–Planck equation

$$\frac{d{X}_i^{\left(\alpha \right)}}{dt}=f^{\left(\alpha \right)}\left(X_i\right)-\frac{{\sigma }^2}{2}\left(r^{\left(\alpha \right)}\left(X_i\right)+arg\underset{\varphi \in \mathcal{F}}{min}{\mathcal{L}}_{\alpha }^r[\varphi ,{\widehat{p}}_t]\left(X_i\right)\right),$$

(12)

for $i=1,\dots ,N\mathrm{and}\alpha =1,\dots ,d$, with

$${\widehat{p}}_t\left(x\right)=\frac{1}{N}\sum _{i=1}^N\delta (x-X_i),X_i\left(0\right)\sim p_0\left(x\right).$$

Although in this article, we use reference functions $r(·)\equiv 0$ for all simulated examples, the choice $r\left(x\right)=\frac{2}{{\sigma }^2}f\left(x\right)$, which cancels the first two terms in Equation (12), leads to interesting relations with other particle approaches for simulating Fokker–Planck solutions for equilibrium systems (c.f. Section 7) and impacts on the numerical approximation of $\varphi$ (c.f. Section 5). In particular, when considering Brownian dynamics, where $f\left(x\right)=-\nabla U\left(x\right)$, the choice $r\left(x\right)=-\frac{2}{{\sigma }^2}\nabla U\left(x\right)$ leads to $\varphi \left(x\right)=0$ once the system has reached thermal equilibrium. More generally speaking, one may choose $r\left(x\right)=\frac{2}{{\sigma }^2}\nabla ln{p}_{*}$, where $p_{*}$ is an appropriate reference measure such as the equilibrium measure of the underlying stochastic process.

Estimating the Entropy Rate

Interestingly, the variational approach provides us with a simple, built in method for computing the entropy rate (temporal change of entropy) of the stochastic process (Equation (2)).

Using the FPE (1) and integration by parts, one can derive the well-known relation, see e.g., [50],

$$-\frac{d}{dt}\int p_t\left(x\right)ln{p}_t\left(x\right)dx=\frac{{\sigma }^2}{2}\sum _{\alpha =1}^d\int p_t\left(x\right){\left({\partial }_{\alpha }ln{p}_t\left(x\right)\right)}^{2}dx+\int p_t\left(x\right)\nabla ·f\left(x\right)dx.$$

(13)

The first term on the right hand side is usually called entropy production, whereas the second term corresponds to the entropy flux. In the stationary state, the total entropy rate vanishes. For equilibrium dynamics, both terms vanish individually at stationarity. This should be compared to the minimum of the cost function (Equation (9)), which for $r\equiv 0$, equals

$$\underset{\varphi }{min}{\mathcal{L}}_{\alpha }^0[\varphi ,p_t]=-\int p_t\left(x\right){\left({\partial }_{\alpha }ln{p}_t\left(x\right)\right)}^{2}dx.$$

(14)

Thus, we obtain the estimator

$$-\frac{d}{dt}\int p_t\left(x\right)ln{p}_t\left(x\right)dx\approx -\frac{{\sigma }^2}{2}\sum _{\alpha =1}^d\underset{\varphi }{min}{\mathcal{L}}_{\alpha }^0[\varphi ,{\widehat{p}}_t]+\frac{1}{N}\sum _{i=1}^N\nabla ·f\left(X_i\right).$$

(15)

We will later see for the case of equilibrium dynamics that a similar method may be employed to approximate the relative entropy distance to the equilibrium density.

5. Function Classes

In the following, we discuss choices for families of functions $\mathcal{F}$ leading to explicit, closed form solutions for estimators.

5.1. Linear Models

A simple possibility is to choose linearly parametrised functions of the form

$$\varphi \left(x\right)=\sum _{k=1}^m{a}_k{\varphi }_k\left(x\right),$$

(16)

where the ${\varphi }_k\left(x\right)$ are appropriate basis functions, e.g., polynomials, radial basis functions or trigonometric functions. For this linear parametrisation, the empirical cost (Equation (10)) is quadratic in the parameters $a_k$ and can be minimised explicitly. A straightforward computation from Equation (12) shows that

$$\begin{array}{c}\hfill \frac{d{X}_i}{dt}=f\left(X_i\right)-\frac{{\sigma }^2}{2}r\left(X_i\right)+\frac{{\sigma }^2}{2}\sum _{k,j=1}^m{\left(C^{-1}\right)}_{kj}{\varphi }_k\left(X_i\right)\sum _{l=1}^N\left\{\nabla {\varphi }_j\left(X_l\right)+{\varphi }_j\left(X_l\right)r\left(X_l\right)\right\},\end{array}$$

(17)

with ${\displaystyle C_{kl}=\sum _{i=1}^N{\varphi }_k\left(X_i\right){\varphi }_l\left(X_i\right)}$.

Here, we require the number of samples to be greater than the number of employed basis functions, i.e., $N\ge m+1$, to have a non–singular matrix C. This restriction may be lifted by adding a penalty to the empirical cost (Equation (10)) for regularisation, similar to ridge regression estimators. Equation (17) is independent of the reference function r, when r belongs to the linear span of the selected basis functions. However, this model class with a finite parameter number has limited complexity.

When the particle number N grows large, we expect convergence of the dynamics to a mean field limit which would be given by

$$\begin{array}{c}\hfill \frac{d{X}_i}{dt}=f\left(X_i\right)-\frac{{\sigma }^2}{2}r\left(X_i\right)+\frac{{\sigma }^2}{2}\sum _{k,j=1}^m{\left({\mathcal{C}}^{-1}\right)}_{kj}{\varphi }_k\left(X_i\right){〈\nabla {\varphi }_j\left(x\right)+{\varphi }_j\left(x\right)r\left(x\right)〉}_{q_t},\end{array}$$

where the brackets ${\langle ·\rangle }_{q_t}$ denote expectation with respect to the limiting density $q_t\left(x\right)$ and ${\displaystyle {\mathcal{C}}_{kl}={\langle {\varphi }_k\left(X_i\right){\varphi }_l\left(X_i\right)\rangle }_{q_t}}$. Since the linear model class (Equation (16)) exhibits limited complexity for fixed m, we do not expect the approximated solution $q_t\left(x\right)$ to equal the exact solution $p_t\left(x\right)$ of the FPE. Nevertheless, for rare cases, where both $\nabla ln{p}_t\left(x\right)$ and also $r\left(x\right)$ are linear combinations of the employed basis functions for all times t, $q_t\left(x\right)$ would provide an exact solution. For example, in a setting with linear drift function $f\left(x\right)=-\gamma x$, reference function $r\left(x\right)=0$, and dimensionality $d=1$, a basis consisting of a constant ${\varphi }_1\left(x\right)=1$ and a linear function ${\varphi }_2\left(x\right)=x$, would be able to perfectly represent the GLD of the Gaussian density $p_t\left(x\right)$.

5.2. Kernel Approaches

Here, we consider a family $\mathcal{F}$ of functions for which the effective number of parameters to be computed is not fixed beforehand, but rather increases with the sample number N: a reproducing kernel Hilbert space (RKHS) of functions defined by a positive definite (Mercer) kernel $K(·,·)$. Statistical models based on such function spaces have played a prominent role in the field of machine learning in recent years [51].

A common, kernel-based approach to regularise the minimisation of empirical cost functions is via penalisation using the RKHS norm ${\parallel ·\parallel }_{\mathrm{RKHS}}$ of functions in $\mathcal{F}$. This can also be understood as a penalised version of a linear model (Equation (16)) with infinitely many feature functions ${\varphi }_k$. For so-called universal kernels [52], this unbounded complexity suggests that we could expect asymptotic convergence of the GLD estimator (see [53] for related results) and a corresponding convergence of the particle model to the FPE as its mean field limit. However, a rigorous proof may not be trivial, since particles in our setting are not independent.

The explicit form of the kernel-based approximation is given by

$${\partial }_{\alpha }lnp\left(x\right)\approx r^{\left(\alpha \right)}\left(x\right)+arg\underset{\varphi \in \mathcal{F}}{min}\left\{{\mathcal{L}}_{\alpha }^r[\varphi ,\widehat{p}]+\frac{\lambda }{N}{\parallel \varphi \parallel }_{\mathrm{RKHS}}^2\right\}\left(x\right),$$

(18)

where the parameter $\lambda$ controls the strength of the penalisation. Again, this optimisation problem can be solved in closed form in terms of matrix inverses. One can prove a representer theorem which states that the minimiser $\varphi \left(x\right)$ in Equation (18) is a linear combination of kernel functions evaluated at the sample points $X_i$, i.e.,

$$\varphi \left(x\right)=\sum _{i=1}^N{a}_{i}K(x,X_i).$$

(19)

For such functions, the RKHS norm is given by

$${\parallel \varphi \parallel }_{\mathrm{RKHS}}^2=\sum _{i,j=1}^N{a}_i{a}_{j}K(X_i,X_j).$$

(20)

Hence, this representation leads again to a quadratic form in the N coefficients.

The solution of the minimisation problem is given by

$$a_j=-\sum _{k=1}^N{\left({(K^2+\lambda K)}^{-1}\right)}_{jk}\sum _{l=1}^N\left\{{\partial }_{{\alpha }_l}K(X_l,X_k)+K(X_l,X_k)r^{\left(\alpha \right)}\left(X_l\right)\right\},$$

(21)

where $K_{ij}\doteq K(X_i,X_j)$. Similar approaches for kernel-based GLD estimators have been discussed in [48,49]. For $r(·)=0$, Equation (21) agrees with the GLD estimator of [48] derived by inverting Stein’s equation, or by minimising the Kernelised Stein discrepancy.

The resulting particle dynamics is given by

$$\frac{d{X}_i}{dt}=f\left(X_i\right)-\frac{{\sigma }^2}{2}r\left(X_i\right)+\frac{{\sigma }^2}{2}\sum _{k=1}^N{\left({(K+\lambda I)}^{-1}\right)}_{ik}\sum _{l=1}^N\left\{{\nabla }_{l}K(X_l,X_k)+K(X_l,X_k)r\left(X_l\right)\right\}.$$

(22)

Please note that here the inverse matrix also depends on the particles $X_k$.

We may simplify Equation (22) by adding and subtracting a term $\lambda {\delta }_{kl}r\left(X_l\right)$ in the summation over l, with ${\delta }_{kl}$ denoting the Kronecker delta. This yields

$$\frac{d{X}_i}{dt}=f\left(X_i\right)+\frac{{\sigma }^2}{2}\sum _{k=1}^N{\left({(K+\lambda I)}^{-1}\right)}_{ik}\left\{\sum _{l=1}^N{\nabla }_{l}K(X_l,X_k)-\lambda r\left(X_k\right)\right\}.$$

(23)

In the limit of small $\lambda$, the right hand side becomes independent of the reference function r.

In the present article, we employ Gaussian radial basis function (RBF) kernels given by

$$K(x,x^{\prime })=exp\left[-\frac{1}{2l^2}{\parallel x-x^{\prime }\parallel }^2\right],$$

(24)

with a length scale l. A different possibility would be given by kernels with a finite dimensional feature representation

$$K(x,x^{\prime })=\sum _{j=1}^m{\varphi }_j\left(x\right){\varphi }_j\left(x^{\prime }\right),$$

(25)

which may also be interpreted as a linear model as in Equation (16) with a $L_2$ penalty on the unknown coefficients.

5.3. A Sparse Kernel Approximation

Since the inversions of the $N\times N$ matrices in Equation (22) have to be performed at each step of a time discretised ODE system (Equation (22)), for large particle number N, the cubic complexity becomes too computationally demanding. Hence, here, we resort to a well established approximation in machine learning to overcome this issue, by applying a sparse approximation to the optimisation problem of Equation (18), see e.g., [54]. In particular, we introduce a smaller set of $M\ll N$inducing points${\left\{z_k\right\}}_{k=1}^M$ that need not necessarily be a subset of the N particles. We then minimise the penalised cost function (Equation (18)) in the finite dimensional family of functions

$$\varphi \left(x\right)=\sum _{i=1}^M{a}_{i}K(x,z_i).$$

(26)

This may also be understood as a special linear parametric approximation. To keep matrices well conditioned, in practice we add a small ’jitter’ term to Equation (18), i.e., we use

$${\lambda \parallel \varphi \parallel }_{\mathrm{RKHS}}^2+ϵ{\parallel \varphi \parallel }_2^2,$$

(27)

as the total penalty. In the limit $\lambda ,ϵ\to 0$, this representation reduces to an approximation of the form of Equation (16) with M basis functions $K(·,z_l)$ for $l=1,\dots ,M$.

By introducing the matrices

$$K_{kl}^{zz}\doteq K(z_k,z_l)+ϵ{\delta }_{kl},K_{ij}^{xz}\doteq K(X_i,z_j),$$

(28)

and

$$A\doteq K^{xz}{\left[(\lambda +ϵ)I+{\left(K^{zz}\right)}^{-1}{\left(K^{xz}\right)}^{\top }\left(K^{xz}\right)\right]}^{-1}{\left(K^{zz}\right)}^{-1},$$

(29)

we replace the particle dynamics of Equation (22) by

$$\frac{d{X}_i}{dt}=f\left(X_i\right)-\frac{{\sigma }^2}{2}r\left(X_i\right)+\frac{{\sigma }^2}{2}\sum _k{A}_{ik}\sum _l\left\{{\nabla }_{l}K(X_l,z_k)+K(X_l,z_k)r\left(X_l\right)\right\}.$$

(30)

Hence, for this approximation we have to invert only $M\times M$ matrices. For fixed M, the complexity of the GLD estimator is limited. Results for log–density estimators in machine learning (obtained for independent data) indicate that for a moderate growth of the number of inducing points M with the number of particles N, similar approximation rates may be obtained as for full kernel approaches.

6. A Note on Expectations

In this section, we present a preliminary discussion of the quality of the particle method to approximate expectations of scalar functions h of the random variable $X\left(t\right)$. We concentrate on the temporal development of $h\left(X\right(t\left)\right)$. While it would be important to obtain an estimate of the approximation error over time, we will defer such an analysis to future publications and only concentrate on a result for the first time derivative of expectations, i.e., the evolution over infinitesimal times.

Using the FPE (Equation (1)) and integrations by parts, one derives the exact result

$$\frac{d\langle h\left(X\right)\rangle }{dt}=\langle L_{x}h\left(X\right)\rangle ,$$

(31)

where $\langle ·\rangle$ denotes the expectation with respect to $p_t\left(x\right)$ and the operator $L_x$ is defined as

$$L_x\doteq f\left(X\right)·\nabla +\frac{{\sigma }^2}{2}{\nabla }^2.$$

(32)

In Equation (32), $L_x$ denotes the generator of the diffusion process defined by the corresponding stochastic differential equation (Equation (2)). To obtain a related result for the particle dynamics and the empirical expectations denoted by ${\langle ·\rangle }_{{\widehat{p}}_t}$, we employ the relation

$$\frac{d{\langle h\left(X\right)\rangle }_{\widehat{p}t}}{dt}={\langle \nabla h\left(X\right)·\frac{dX}{dt}\rangle }_{\widehat{p}t}=\frac{1}{N}\sum _{i=1}^N\nabla h\left(X_i\right)·\frac{d{X}_i}{dt},$$

(33)

where we have used the chain rule for the time derivative.

We will focus initially on the dynamics based on basis functions. Expressing the sum over $X_l$ in Equation (17) as an expectation, we obtain

$$\frac{d{X}_i}{dt}=f\left(X_i\right)-\frac{{\sigma }^2}{2}r\left(X_i\right)+\frac{N{\sigma }^2}{2}\sum _{k,j=1}^m{\left(C^{-1}\right)}_{kj}{\varphi }_k\left(X_i\right){〈\left(r\left(X\right)+\nabla \right){\varphi }_j\left(X\right)〉}_{\widehat{p}t}.$$

(34)

Hence, by inserting Equation (34) into Equation (33) and adding and subtracting the term $\frac{{\sigma }^2}{2}{\langle {\nabla }^{2}h\left(X\right)\rangle }_{\widehat{p}t}$ we obtain

$$\frac{d{\langle h\left(X\right)\rangle }_{\widehat{p}t}}{dt}={\langle L_{x}h\left(X\right)\rangle }_{\widehat{p}t}+\Delta ,$$

(35)

where the remainder term is given by

$$\Delta =\frac{{\sigma }^2}{2}{\langle \left(r\left(X\right)+\nabla \right)·\left(\widehat{\nabla }h\left(x\right)-\nabla h\left(x\right)\right)\rangle }_{\widehat{p}t}.$$

(36)

For the approximation of the vectorial function $\nabla h\left(x\right)$ we have

$$\widehat{\nabla }h\left(x\right)=\sum _{j,k=1}^m{\varphi }_j\left(x\right){\left(C^{-1}\right)}_{jk}\sum _{i=1}^N{\varphi }_k\left(X_i\right)\nabla h\left(X_i\right).$$

(37)

A simple comparison shows that each component of the vector $\widehat{\nabla }h\left(x\right)$ can be written as the minimiser of ${\sum }_{l=1}^N{\left(\Phi \left(X_l\right)-{\partial }_{\alpha }h\left(X_l\right)\right)}^2$ where $\Phi \left(x\right)$ is a linear combination of basis functions. Hence, $\widehat{\nabla }h\left(x\right)$ equals the best approximation of the vectorial function $\nabla h\left(x\right)$ based on the ’data’ $\nabla h\left(X_l\right)$ using regression with basis functions. Thus, if $\nabla h\left(x\right)$ is well approximated by basis functions, the remainder $\Delta$ is small. If indeed $\nabla h\left(x\right)={\sum }_{n=1}^M{c}_n{\varphi }_n\left(x\right)$, for some $c_n\in R^d$, the remainder term vanishes, $\Delta =0$. By its similarity to the finite basis function model, this result should also be valid for the sparse kernel dynamics of Equation (30), when the penalty $\lambda$ is small. One might conjecture that the temporal development of expectations for reasonably smooth functions might be faithfully represented by the particle dynamics. This conjecture is supported by our numerical results.

A similar result also holds for the dynamics of Equation (22). In this case, the function $\widehat{\nabla }h\left(x\right)$ in the remainder $\Delta$ (Equation (36)) is given by

$$\widehat{\nabla }h\left(x\right)=\sum _{j,k=1}^{N}K(x,X_j){\left({(K+\lambda I)}^{-1}\right)}_{jk}\nabla h\left(X_k\right),$$

(38)

which has also an interpretation as an approximation of $\nabla h\left(x\right)$ by regularised kernel regression.

7. Equilibrium Dynamics

An important class of stochastic dynamical systems describe thermal equilibrium, for which the drift function f is the negative gradient of a potential U, while the limiting equilibrium density $p_{\infty }$ is explicitly given by a Gibbs distribution:

$$\begin{array}{c}\hfill f\left(x\right)=-\nabla U\left(x\right)\end{array}$$

(39)

$$\begin{array}{c}\hfill \nabla ln{p}_{\infty }\left(x\right)=\frac{2}{{\sigma }^2}f\left(x\right).\end{array}$$

(40)

For this class of models, our method provides a simple and built-in estimator for the relative entropy between the instantaneous and the equilibrium density, $p_t$ and $p_{\infty }$ respectively. As we discuss here, our framework may also be related to two other particle approaches that converge to the (approximate) equilibrium density.

7.1. Relative Entropy

The relative entropy or Kullback–Leibler divergence is defined as

$$D\left(p_t\right|p_{\infty })\doteq \int p_t\left(x\right)ln\frac{p_t\left(x\right)}{p_{\infty }\left(x\right)}dx.$$

(41)

Following a similar calculation that led to Equation (13), we obtain

$$\begin{array}{c}\hfill \frac{d}{dt}D\left(p_t\right|p_{\infty })=-\frac{{\sigma }^2}{2}\int p_t\left(x\right){∥\nabla ln{p}_t\left(x\right)-\nabla ln{p}_{\infty }\left(x\right)∥}^{2}dx=-\frac{2}{{\sigma }^2}\int p_t\left(x\right){\parallel g(x,t)\parallel }^{2}dx,\end{array}$$

(42)

where $g(x,t)$ indicates the velocity field of the particle system defined in Equation (4). The first equality holds for arbitrary drift functions. To obtain the second equality, we have inserted the explicit result for $p_{\infty }$.

Hence, we may compute the relative entropy at any time T as a time integral

$$D\left(p_T\right|p_{\infty })=D\left(p_0\right|p_{\infty })-\frac{2}{{\sigma }^2}{\int }_0^T\left\{\int p_t\left(x\right){\parallel g(x,t)\parallel }^{2}dx\right\}dt,$$

(43)

where the inner expectation is easily approximated by our particle algorithm. This result shows that the exact velocity field $g(x,t)$ converges to 0 for $t\to \infty$, and one expects particles to also converge to fixed points. For other non–equilibrium systems, asymptotic fixed points are, however, the exception.

7.2. Relation to Stein Variational Gradient Descent

Recently, Stein variational gradient descent (SVGD), a kernel-based particle algorithm, has attracted considerable attention in the machine learning community [55,56]. The algorithm is designed to provide approximate samples from a given density $p_{\infty }$ as the asymptotic fixed points of a deterministic particle system. Setting $-ln{p}_{\infty }\left(x\right)=U\left(x\right)+const$, SVGD is based on the dynamics

$$\frac{d{X}_i}{dt}=\sum _l\left\{-K(X_i,X_l)\nabla U\left(X_l\right)+{\nabla }_{l}K(X_i,X_l)\right\}.$$

(44)

This may be compared to our approximate FPE dynamics (Equation (22)) for the equilibrium case by setting ${\sigma }^2=2$ and $r\left(x\right)=f\left(x\right)=-\nabla U\left(x\right)$. For this setting, both algorithms have in fact, the same conditions

$$\sum _l\left\{-K(X_i,X_l)\nabla U\left(X_l\right)+{\nabla }_{l}K(X_i,X_l)\right\}=0,$$

(45)

for the ’equilibrium’ fixed points. See [44] for a discussion of these fixed points for different kernel functions. However, both dynamics differ for finite times t, where a single time step of SVGD is computationally simpler, being free of the matrix inversion required by our framework. The mean field limit $N\to \infty$ of Equation (44) differs from the FPE limit, and the resulting partial differential equation is nonlinear [57]. Nevertheless, it is possible to interpolate between the two particle dynamics. In fact, in the limit of a large regularisation parameter $\lambda \to \infty$, the inverse matrix in Equation (22) becomes diagonal, i.e., ${(K+\lambda I)}^{-1}\simeq \frac{1}{\lambda }I$, and we recover SVGD (Equation (44)) by introducing a rescaled time $\tau \doteq t/\lambda$. This result could be of practical importance when the goal is to approximate the stationary distribution, irrespective of the finite-time dynamics. The SVGD combines faster matrix operations with slower relaxation times to equilibrium compared to the FPE dynamics. It would be interesting to see if an optimal computational speed of a particle algorithm might be achieved at some intermediate regularisation parameter $\lambda$.

7.3. Relation to Geometric Formulation of FPE Flow

Following Otto [58] and Villani [59], the FPE for the equilibrium case can be viewed as a gradient flow on the manifold of probability densities with respect to the Wasserstein metric. This formulation can be used to define an implicit Euler time discretisation method for the dynamics of the density $p_t$. For small times $\delta t$ (and ${\sigma }^2=2$) this is given by the variational problem

$$p_{t+\delta t}=arg\underset{p}{inf}\left(W_2^2(p,p_t)+\delta tD(p\parallel p_{\infty })\right)$$

(46)

in terms of the Kullback–Leibler divergence and the $L_2$Wasserstein distance${\mathcal{W}}_2$. The latter gives the minimum of $\langle \parallel X-X\left(t\right){\parallel }^2\rangle$ for two random variables $X\left(t\right)$ and X, where the expectation is over the joint distribution with fixed marginals $p_t$ and p. Using the dual formulation for a regularised Wasserstein distance, approximate numerical algorithms for solving Equation (46) have been developed by [60] and by [61] with applications to simulations of FPE.

We show in the following that Equation (46) may be cast into a form closely related to our variational formulation (Equation (7)) for $r\left(x\right)=-\nabla U\left(x\right)$. Assuming that X and $X\left(t\right)$ are related through deterministic (transport) mappings of the form

$$X=X\left(t\right)+\delta t\nabla \psi \left(X\right(t\left)\right),$$

(47)

we may represent the Wasserstein distance in terms of $\psi$ and the variational problem in Equation (46) may be reformulated as

$${\psi }^{*}=arg\underset{\nabla \psi }{min}\frac{\delta t^2}{2}\int {\parallel \nabla \psi \left(x\right)\parallel }^2{p}_t\left(x\right)dx+\delta tD(p_{t+dt}\parallel p_{\infty }).$$

(48)

where

$$p_{t+\delta t}\left(x\right)=p_t\left(x\right)-\delta t\nabla ·\left(p_t\left(x\right)\nabla \psi \left(x\right)\right)+O\left(\delta t^2\right).$$

(49)

To proceed, we expand the relative entropy in Equation (48) to first order in $\delta t$, inserting the representation of Equation (49) for $p_{t+\delta t}\left(x\right)$, thereby obtaining

$$\begin{array}{cc}\hfill & \frac{\delta t}{2}\int {\parallel \nabla \psi \left(x\right)\parallel }^2{p}_t\left(x\right)dx+D(p_{t+\delta t}\parallel p_{\infty })=D(p_t\parallel p_{\infty })+\hfill \end{array}$$

(50a)

$$\begin{array}{cc}\hfill & +\frac{\delta t}{2}\left(\int p_t\left(x\right)\left\{{\parallel \nabla \psi \left(x\right)\parallel }^2-2{\nabla }^2\psi \left(x\right)+2\nabla U\left(x\right)·\nabla \psi \left(x\right)\right\}dx\right)+O\left(\delta t^2\right).\hfill \end{array}$$

(50b)

Minimisation ignoring the $O\left(\delta t^2\right)$ terms (employing integration by parts) yields

$$\nabla {\psi }^{*}\left(x\right)=-\nabla U\left(x\right)-\nabla ln{p}_t\left(x\right),$$

(51)

which is related to our cost function (Equation (8)) if we formally identify $\varphi \left(x\right)=-{\partial }_{\alpha }\psi \left(x\right)$. More precisely, by replacing $p_t$ by samples, the empirical cost function may be regularised with a RKHS norm penalty resulting in a nonparametric estimator for unnormalised log–density ${\psi }^{*}\left(x\right)=-ln{p}_t\left(x\right)-U\left(x\right)+const$, as shown in [62]. One could use this estimator as an alternative to our approach. This would lead to a simultaneous estimate of all components of the GLD. In our approach, each of the d components of the gradient is computed individually. In this way, we avoid additional second derivatives of kernels, which would increase the dimensionality of the resulting matrices.

8. Extension to General Diffusion Processes

The Fokker–Planck equations for an SDE with arbitrary drift $f\left(x\right)$ and general, state-dependent diffusion matrix $D\left(x\right)$ is given by

$$\frac{\partial p_t\left(x\right)}{\partial t}=\nabla ·\left[-f\left(x\right)p_t\left(x\right)+\frac{1}{2}\nabla ·\left(D\left(x\right)p_t\left(x\right)\right)\right].$$

(52)

This may again be written in the form of a Liouville equation (Equation (3)) where the effective force term equals

$$g(x,t)=f\left(x\right)-\frac{1}{2}\nabla ·D\left(x\right)-\frac{1}{2}D\left(x\right)\nabla ln{p}_t\left(x\right).$$

(53)

9. Second Order Langevin Dynamics (Kramer’s Equation)

For second-order Langevin equations, the system state comprises positions $X\in R^d$ and velocities $V\in R^d$ following the coupled SDE

$$\begin{array}{cc}\hfill dX& =Vdt\hfill \end{array}$$

(54)

$$\begin{array}{cc}\hfill dV& =\left(-\gamma V+f\left(X\right)\right)dt+\sigma d{B}_t.\hfill \end{array}$$

(55)

In Equation (54), the dynamics describe the effect of a friction force, $\gamma V$, an external force, $f\left(X\right)$, and a fluctuating force, where $\gamma$ denotes the dissipation constant. In this setting, the effective deterministic ODE system is given by

$$\begin{array}{cc}\hfill \frac{dX}{dt}& =V\hfill \\ \hfill \frac{dV}{dt}& =-\gamma V+f\left(X\right)-\frac{{\sigma }^2}{2}{\nabla }_{v}ln{p}_t(X,V).\hfill \end{array}$$

(56)

Considering here the equilibrium case, we set $f\left(x\right)=-\nabla U\left(x\right)$ for which the stationary density equals

$$ln{p}_{\infty }(X,V)=-\beta \left(\frac{{\parallel V\parallel }^2}{2}+U\left(X\right)\right)\equiv -\beta H(X,V),$$

(57)

where $\beta =\frac{2\gamma }{{\sigma }^2}$ and $H(x,v)=\frac{{\parallel V\parallel }^2}{2}+U\left(x\right)$ denotes the Hamiltonian function. Inserting $p_{\infty }$ into Equation (56), we find that for $t\to \infty$, the damping and the density-dependent part of the force cancel and we are left with pure Hamiltonian dynamics

$$\begin{array}{cc}\hfill \frac{dX}{dt}& =V\hfill \\ \hfill \frac{dV}{dt}& =-\nabla U\left(X\right),\hfill \end{array}$$

(58)

for which all particles become completely decoupled, with each one conserving energy separately. Of course, this result also precludes fixed point solutions to the particle dynamics. However, this limiting dynamics captured by Equation (58) assumes the mean field limit $N\to \infty$ together with a consistent estimate of the GLD before taking the limit $t\to \infty$. For GLD estimators at finite N, we expect reasonably small stationary fluctuations of individual particle energies, which were also evident in our numerical experiments.

The exact asymptotic behaviour is also reflected in the expression for the change of the relative entropy for Kramer’s equation. Similar to Equation (42) we obtain

$$\begin{array}{cc}\hfill \frac{d}{dt}D\left(p_t\right|p_{\infty })& =-\frac{{\sigma }^2}{2}\int p_t(x,v){∥{\nabla }_{v}ln{p}_t(x,v)-{\nabla }_{v}ln{p}_{\infty }(x,v)∥}^{2}dxdv\hfill \\ & =-\frac{2}{{\sigma }^2}\int p_t(x,v){\parallel \gamma v+\frac{{\sigma }^2}{2}{\nabla }_{v}lnp(x,v)\parallel }^{2}dxdv.\hfill \end{array}$$

(59)

When the system approaches equilibrium, both terms in the norm cancel out and the entropy production rate converges to 0.

10. Simulating Accurate Fokker–Planck Solutions for Model Systems

To demonstrate the accuracy of our approach, we simulated solutions of FPEs for a range of model systems and compared the results with those obtained from direct stochastic simulations (Monte Carlo sampling) with the same particle number, as well as with analytic solutions where relevant. We tested our framework on systems with diverse degrees of nonlinearity and dimensionality, as well as with various types of noise (additive/multiplicative). We quantified the accuracy of transient and steady state solutions resulting from our method in terms of 1-Wasserstein distance [59] and Kullback–Leibler (KL) divergence (Appendix C and Appendix D), along with the squared error of distances between distribution cumulants. For evaluating particle solutions for nonlinear processes, where analytical solutions of the Fokker–Planck equation are intractable, we simulated a very large number ($N^{\infty }$) of stochastic trajectories that we considered to be ground truth Fokker–Planck solutions. We employed an Euler–Maruyama and forward Euler integration scheme of constant step size $dt={10}^{-3}$ for stochastic and deterministic simulations respectively. We provide a description of the employed algorithm along with analysis of its computational complexity in Appendix H, while further numerical experiments on the influence of hyperparameter values on the performance of the estimator are provided in Appendix G and Appendix F.

10.1. Linear Conservative System with Additive Noise

For a two dimensional Ornstein-Uhlenbeck process (Appendix A.1) transient and stationary densities evolved through deterministic particle simulations (D) consistently outperformed their stochastic counterparts (S) comprising the same number of particles in terms accuracy in approximating the underlying density (Figure 1). In particular, comparing the 1-Wasserstein distance between samples from analytically derived densities ($P_t^A$) (Appendix B)—considered here to reflect the ground truth—and the deterministically (D) or stochastically (S) evolved densities ($P_t^N$), ${\mathcal{W}}_1(P_t^A,P_t^N)$, we observed smaller Wasserstein distances to ground truth for densities evolved according to our deterministic particle dynamics, both for transient (Figure 1a) and stationary (Figure 1c) solutions. Specifically, we quantified the transient deviation of simulated densities from ground truth by the average temporal 1-Wasserstein distance, ${〈{\mathcal{W}}_1(P_t^A,P_t^N)〉}_t$ (Appendix D). For small particle number, deterministically evolved interacting particle trajectories represented more reliably the evolution of the true probability density compared to independent stochastic ones, as portrayed by smaller average Wasserstein distances. For increasing particle number, the accuracy of the simulated solutions with the two approaches converged. Yet, while for $N=2500$ particles the stochastically evolved densities suggest on average (over trials) comparable approximation precision with their deterministic counterparts, the deterministically evolved densities more reliably delivered densities of a certain accuracy, as proclaimed by the smaller dispersion of Wasserstein distances among different realisations (Figure 1a,c).

Likewise, we observed similar results when comparing only the stationary distributions, ${\mathcal{W}}_1(P_{\infty }^A,P_{\infty }^N)$ (Figure 1c). While for small particle number, the interacting particle system more accurately captured the underlying limiting distribution, for increasing particle number the accuracy of both approaches converged, with our method consistently delivering more reliable approximations among individual repetitions.

Moreover, densities evolved with our deterministic framework exhibited less fluctuating cumulant trajectories in time, compared to their stochastic counterparts (Figure 2c). In particular, even for limited particle number, cumulants calculated over deterministically evolved particles progressed smoothly in time, while substantially more particles for the stochastic simulations were required for the same temporal cumulant smoothness. To further quantify the transient accuracy of Fokker–Planck solutions computed with our method, we compared the average transient discrepancy between the first two analytic cumulants ($m_t$ and $C_t$) to those estimated from the particles (${\widehat{m}}_t$ and ${\widehat{C}}_t$), $\langle \parallel {\widehat{m}}_t-m_t{{\parallel }_2\rangle }_t$ (Figure 1b) and $\langle \parallel {\widehat{C}}_t-C_t{{\parallel }_F\rangle }_t$ (Figure 1d), where ${\parallel ·\parallel }_F$ stands for the Frobenious norm (Appendix E). In line with our previous results, our deterministic framework delivered considerably more accurate transient cumulants when compared to stochastic simulations, with more consistent results among individual realisations, denoted by smaller dispersion of average cumulant differences. (Notice the logarithmic y-axis scale in Figure 1b,d. Error bars for the stochastic solutions were in fact larger than those for the deterministic solutions on a linear scale.)

Interestingly, the number of sparse points M employed in the gradient–log–density estimation had only minor influence on the quality of the solution (Figure 1a,c). This hints to substantially low computational demands for obtaining accurate Fokker–Planck solutions, since our method is computationally limited by the inversion of the $M\times M$ matrix in Equation (29).

10.2. Bi-Stable Nonlinear System with Additive Noise

For nonlinear processes, since the transient solution of the FPE is analytically intractable, we compared the transient and stationary densities estimated by our method with those returned from stochastic simulations of $N^{\infty }$ = 26,000 particles, and contrasted them against stochastic simulations with the same particle number.

For a system with bi-modal stationary distribution (Appendix A.2), the resulting particle densities from our deterministic framework closely agreed with those arising from the stochastic system with $N^{\infty }$ = 26,000 particles (Figure 3a). In particular, deterministically evolved distributions respected the symmetry of the underlying double–well potential, while the stochastic system failed to accurately capture the potential symmetric structure Figure 3a (iii).

Systematic comparisons of the 1-Wasserstein distance between deterministic and stochastic N particle simulations with the “$N^{\infty }$” stochastic simulation comprising $N^{\infty }=26,000$ particles revealed that our approach efficiently captured the underlying PDF already with $N=500$ particles (Figure 3c,d). For increasing particle number, the stationary solutions of both systems converged to the “$N^{\infty }$” one. However, we observed a systematically increasing approximation accuracy delivered from the deterministic simulations compared to their stochastic counterparts.

It is noteworthy that, on average, deterministic simulations of $N=500$ particles conveyed a better approximation of the underlying transient PDF compared to stochastic simulations of $N=2500$ particles (Figure 3c).

Interestingly, for small particle number, the number of employed inducing points M did not significantly influence the accuracy of the approximated solution. However for increasing particle number, enlarging the set of inducing points contributed to more accurate approximation of Fokker–Planck equation solutions (Figure 3c), with the trade off of additional computational cost.

Similar to the Ornstein Uhlenbeck process (Section 10.1), comparing cumulant trajectories computed from both the deterministic and stochastic particle systems revealed less fluctuating cumulant evolution for densities evolved with our deterministic framework also in this nonlinear setting (Figure 3b).

10.3. Nonlinear System Perturbed by Multiplicative Noise

To assess the accuracy of our framework on general diffusion processes perturbed by state-dependent (multiplicative) noise, we simulated a bi-stable system with dynamics governed by Equation (A3) with diffusion function $D\left(x\right)=si{n}^2\left(x\right)$ according to Equation (53). Similarly, in this setting, deterministic particle distributions delivered a closer approximation of the underlying density when compared to direct stochastic simulations. In particular, we found that in this setting, deterministically evolved distributions more accurately captured the tails of the underlying distribution, mediated here by stochastic simulations of $N^{\infty }$ = 35,000 particles (Figure 4a,b).

Similar to the previously examined settings, the deterministic framework delivered more reliable and smooth trajectories for the marginal statistics of the underlying distribution (Figure 4c).

Comparing the temporal average and stationary 1-Wasserstein distance (Figure 4d,f) between the optimal stochastic distributions and the deterministic and stochastic particle distributions of size N, we found that the deterministic system delivered consistently more accurate approximations, as portrayed by smaller 1-Wasserstein distances.

Interestingly, we found that for deterministic particle simulations, the number of employed sparse points in the gradient–log–density estimation mediated a moderate approximation improvement for small system sizes, while for systems comprising more than $N=2000$ particles, the number of sparse points had minimal or no influence on the accuracy of the resulting distribution (Figure 4e,g).

10.4. Performance in Higher Dimensions

To quantify the scaling and performance of the proposed framework for increasing system dimension, we systematically compared simulated densities with analytically calculated ones for Ornstein–Uhlenbeck processes of dimension $D=\{2,3,4,5\}$ following the dynamics of Equation (A4) for inducing point number $M=100$ (Figure 5) and $M=200$ (Figure 6). To evaluate simulated Fokker–Planck solutions, we calculated the Kullback–Leibler divergence between analytically evolved densities (Appendix B) and particle densities. We employed the closed-form equation for estimating KL divergence between two Gaussian distributions (Appendix C) for empirically estimated mean, ${\widehat{m}}_t$, and covariance, ${\widehat{C}}_t$, for particle distributions.

For all dimensionalities, the deterministic particle solutions approximated transient and stationary densities remarkably accurately with Kullback–Leibler divergence between the simulated and analytically derived densities below ${10}^{-2}$ for all dimensions, both for transient and stationary solutions (Figure 5a,d and Figure 6a,d). In fact, the deterministic particle solutions delivered more precise approximations of the underlying densities compared to direct stochastic simulations of the same particle number. Remarkably, even for processes of dimension $D=5$, deterministically evolved solutions mediated through $N=500$ particles resulted in approximately the same KL divergence of stochastic particle solutions of $N=6500$ particles.

Our deterministic particle method delivered consistently better approximations of the mean of the underlying densities compared to stochastic particle simulations (Figure 5b,e). Specifically, estimations of the stationary mean of the underlying distributions were more than two orders of magnitude accurate that their stochastically approximated counterparts already for small particle number (Figure 5e).

Yet, the accuracy of our deterministic framework deteriorated for increasing dimension (Figure 5a,d). More precisely, while for low dimensionalities the covariance matrices of the underlying densities were accurately captured by deterministically evolved particles, for increasing system dimension approximations of covariance matrices became progressively worse. Yet, even for systems of dimension $D=5$, covariance matrices computed from deterministically simulated solutions of $N=500$ particles were at the same order of magnitude as accurate as covariances delivered by stochastic particle simulations of size $N=6500$.

However, comparing the resulting performance of solutions delivered by employing different number of inducing points (Figure 5 and Figure 6) reveals that for increasing dimension more inducing points are required to attain accurate FPE solutions. In particular, both transient and stationary KL divergences to ground truth improved remarkably for dimensions $D=4$ and $D=5$ by employing $M=200$ inducing points in the gradient–log–density estimation (Figure 5a,d and Figure 6a,d). In more detail, for nearly all dimensions, the estimation of the covariance of the underlying distribution improved considerably, both for transients and stationary solutions (Figure 5c,f and Figure 6c,f), while only the stationary mean of dimension $D=4$ showed significant improvement (Figure 5b,e and Figure 6b,e). Figure 6c,f reveals that by increasing the number of inducing points our framework is able to capture more effectively the spread of the underlying distribution, clearly surpassing in approximation accuracy solutions mediated by stochastic particle simulations.

10.5. Second order Langevin Systems

To demonstrate the performance of our framework for simulating solutions of the FPEs for second order Langevin systems as described in Section 9, we incorporated our method in a symplectic Verlet integrator (Equations (A10)–(A12)) simulating the second-order dynamics captured by Equation (56) for a linear $f\left(x\right)=-4x$ and a nonlinear, $f\left(x\right)=-4x^3+4x$, drift function (Equation (A10)), and compared the results with stochastic simulations integrated by a semi-symplectic framework [63]. In agreement with previous results, cumulant trajectories evolved smoother in time for deterministic particle simulations when compared to their stochastic counterparts (Figure 7a and Figure 8c). Stationary densities closely matched analytically derived ones (see Equation (A7)) (purple contour lines in Figure 7b and Figure 8b), while transient densities captured the fine details of simulated stochastic particle densities comprising $N^{\infty }=20,000$ (Figure 8a).

Furthermore, the symplectic integration contributed to the preservation of energy levels for each particle after the system reached equilibrium (Figure 7e and Figure 8f), which was also evident when observing individual particle trajectories in the state space (Figure 7c,d and Figure 8d,e).

As already conveyed in Section 9, the velocity term and the gradient–log–density term canceled out in the long time limit (Figure 7f and Figure 8g) for each particle individually, while the average kinetic energy in equilibrium exactly resorted to the value dictated by the fluctuation–dissipation relation and the equipartition of energy property, i.e., ${\langle {\mathcal{K}}^{\left(i\right)}\rangle }_N=\frac{{\sigma }^2}{2\gamma }$ (Figure 7g and Figure 8h).

10.6. Nonconservative Chaotic System with Additive Noise (Lorenz Attractor)

As a final assessment of our framework for simulating accurate solutions of Fokker–Planck equations, we employed a Lorenz attractor model with parameters rendering the dynamics chaotic, perturbed by moderate additive Gaussian noise (Equation (A13)). By comparing stochastic simulations of $N^{\infty }$ = 150,000 particles and deterministic and stochastic simulations of $N=4000$ particles (Figure 9), we observed that the deterministic framework more precisely captured finer details of the underlying distribution (Figure 9a), represented here by the $N^{\infty }$ stochastic simulation. While both stochastic and deterministic simulations capture the overall butterfly profile of the Lorenz attractor, the deterministic system indeed delivered a closer match to the underlying distribution.

Similar to the previously examined models, cumulant trajectories computed from deterministically evolved particles show closer agreement with those computed from the $N^{\infty }$ stochastic system, compared to the stochastic system comprising N particles (Figure 9b). In particular, cumulants for the x and y states exhibited high temporal fluctuations when computed from stochastically evolved distributions, while our framework conveyed more accurate cumulant trajectories, closer to those delivered by the $N^{\infty }$ stochastic system.

11. Discussion and Outlook

We presented a particle method for simulating solutions of FPEs governing the temporal evolution of the probability density for stochastic dynamical systems of the diffusion type. By reframing the FPE in a Liouville form, we obtained an effective dynamics in terms of independent deterministic particle trajectories. Unfortunately, this formulation requires the knowledge of the gradient of the logarithm of the instantaneous probability density of the system state, which is actually the unknown quantity of interest. We circumvented this complication by introducing statistical estimators for the gradient–log–density based on a variational formulation. To combine high flexibility of estimators with computational efficiency, we employed kernel-based estimation together with an additional sparse approximation. For the case of equilibrium systems, we related our framework to Stein Variational Gradient Descent, a particle-based dynamics to approximate the stationary density, and to a geometric formulation of Fokker–Planck dynamics. We further discussed extensions of our method to settings with multiplicative noise and to second order Langevin dynamics.

To demonstrate the performance of our framework, we provided detailed tests and comparisons with stochastic simulations and analytic solutions (when possible). We demonstrated the accuracy of our method on conservative and non-conservative model systems with different dimensionalities. In particular, we found that our framework outperforms stochastic simulations both in linear and nonlinear settings by delivering more accurate densities for small particle number when the dimensionality is small enough. For increasing particle number, the accuracy of both approaches converges. Yet, our deterministic framework consistently delivered results with smaller variance among individual repetitions. Furthermore, we showed that our method, even for small particle numbers, exhibits low-order cumulant trajectories with significantly less temporal fluctuations when compared against to stochastic simulations of the same particle number.

We envisage several ways to improve and extend our method. There is room for enhancement by optimising hyperparameters of our algorithm such as inducing point position and kernel length scale. Current grid-based and uniform random selection of inducing point position may contribute to the deterioration of solution accuracy in higher dimensions. Other methods, such as subsampling or clustering of particle positions may lead to further improvements. On the other hand, a hyperparameter update may not be at all necessary at each time step in certain settings, such that a further speedup of our algorithm could be achieved.

The implementation of our method depends on the function class chosen to represent the estimator. In this paper we have focused on linear representations, leading to simple closed form expressions. It would be interesting to see if other, nonlinear parametric models, such as neural networks, (see e.g., [64]) could be employed to represent estimators. While in this setting, there would be no closed-form solutions, the small changes in estimates between successive time steps suggest that only a few updates of numerical optimisation may be necessary at each step. Moreover, the ability of neural networks to automatically learn relevant features from data might help to improve performance for higher dimensional problems when particle motion is typically restricted on lower dimensional submanifolds.

From a theoretical point of view, rigorous results on the accuracy of the particle approximation would be important. These would depend on the speed of convergence of estimators towards exact gradients of log–densities. However, to obtain such results may not be easy. While rates of convergence for kernel-based estimators have been studied in the literature, the methods for proofs usually rely on the independence of samples and would not necessarily apply to the case of interacting particles.

We have so far addressed only the forward simulation of FPEs. However, preliminary results indicate that related techniques may be applied to particle based simulations for smoothing (forward–backward) and related control problems for diffusion processes [65]. Such problems involve computations of an effective, controlled drift function in terms of gradient–log–densities. We defer further details and discussions on subsequent publications on the topic.

Taken together, the main advantage of our framework is its minimal requirement in simulated particle trajectories for attaining reliable Fokker–Planck solutions with smoothly evolving transient statistics. Moreover, our proposed method is nearly effortless to set up when compared to classical grid-based FPE solvers, while it delivers more reliable results than direct stochastic simulations.