Efficient Solution of Fokker–Planck Equations in Two Dimensions

Abstract

Finite element analysis (FEA) of the Fokker–Planck equation governing the nonstationary joint probability density function of the responses of a dynamical system produces a large set of ordinary differential equations, and computations become impractical for systems with as few as four states. Nonetheless, FEA remains of interest for small systems—for example, for the generation of baseline performance data and reference solutions for the evaluation of machine learning-based methods. We examine the effectiveness of two techniques which, while they are well established, have not to our knowledge been applied to this problem previously: reduction of the equations onto a smaller basis comprising selected eigenvectors of one of the coefficient matrices, and splitting of the other coefficient matrix. The reduction was only moderately effective, requiring a much larger basis than was expected and producing solutions with clear artifacts. Operator splitting, however, performed very well. While the methods can be combined, our results indicate that splitting alone is an effective and generally preferable approach.

1. Introduction

The Fokker–Planck (FP) equation, also known as the forward Kolmogorov equation, is a deterministic partial equation (PDE) whose solution gives the joint probability density function (pdf) of the responses (states) of a dynamical system subjected to random excitation. While analytical solutions are available for linear systems of any dimension [1,2], nonlinear dynamical systems naturally give rise to more challenging FP equations, for which analysis may be limited to the stationary pdf [3] when it can be carried out at all. When the FP equation is discretized, for example, by using finite elements in a system’s phase space, the resulting matrix equation grows much faster than the dimension of the original dynamical system, and computations quickly become intractable. In the following, we try to address this problem for small (but nontrivial) systems.

The first attempts at the application of finite element analysis (FEA) in numerical treatment of the FP equation date back several decades. Bergman and Heinrich [4] first proposed Petrov–Galerkin FEA to solve the backward Kolmogorov equation (the formal adjoint of the FP equation), and solved the first passage problem for a second-order linear system driven by additive Gaussian white noise. Langley [5] used a finite element formulation to study the stationary random vibration of 2-D nonlinear systems. Spencer and Bergman [6] used a Bubnov–Galerkin FEA method to solve the FP equations of linear, Duffing, and Van der Pol oscillators, and studied the corresponding first passage problems. El-Gebeily and Shabaik [7] reported a straightforward analysis of the transient problem. Masud and Bergman [8] presented a multi-scale FEA formulation, applied it to calculate the solution of the FP equation, and examined its efficacy through selected applications. Floris [9] used spline interpolation in FEA of the FP equation, Kumar et al. [10] considered systems subjected to colored non-Gaussian noise, and Král and Náprstek [11] adopted simplex elements to treat higher-dimensional problems. Further algorithmic refinements and additional applications may be found in Refs. [12,13,14,15,16,17].

All of these works, however, suffer from the same fundamental limitation. When the FE method is used to solve the FP equation, as the dimension of the state space increases, the number of equations to be solved increases exponentially. The resulting huge amount of computation required brings great challenges. In this paper, we consider two approaches to overcoming this difficulty—namely, reducing the dimension of the FE equation by projecting it onto a smaller basis, eliminating one coefficient matrix in the process; and splitting the remaining matrix to enable efficient time marching of the nonstationary solution. Numerous examples of related work on deterministic problems may be found in the literature, for example, in computational fluid dynamics [18,19,20,21] and in nearfield acoustic holography [22]. The novelty of this work lies in the application of these techniques to the FP equation, and in the results obtained for specific choices of basis and splitting scheme.

Dimension reduction has been gradually applied in the solution of FP equations. Lötstedt and Ferm [23] obtained results for a problem involving multiple chemical species by deriving PDEs to approximate the first moments for some of the species and coupling these to the FP equations for the species not so reduced. Leonenko and Phillips [24] used spectral discretization and a high-order reduced-basis approximation in an application to polymer dynamics, and Er [25] proposed a subspace method to solve for the response probability density functions of large-scale nonlinear stochastic dynamical systems. Chen and Lin [26] and Chen and Rui [27] obtained the generalized probability density evolution equation by using decoupled physical equations. Zhu [28] studied the stationary probability density function of a multi-degree-of-freedom nonlinear system subjected to external, independent Poisson white noise, reducing the high-dimensional generalized FP equation to a low-dimensional equation, and Liang et al. [29] improved on this approach. In contrast to most of these studies, the reduction method used herein does not depend on the physics of the problem considered.

Independent of the dimension of the problem, operator splitting provides a numerical method to compute the solution to matrix differential equations that can be more efficient than straightforward integration using the original coefficient matrices. Operator splitting methods have been known for more than a century, starting with the scheme introduced by Lie in the mid-1870s [30]. Usadi and Dawson [31] developed alternating direction implicit (ADI) methods. In the mid-1970s, close relationships between the augmented Lagrangian methods of Hestenes [32] and Powell [33,34] and ADI methods were identified, which inspired the alternating direction method of multipliers (ADMM). Based on a symmetrization principle, Strang [35] introduced a second-order variant of the Lie scheme, which was motivated by a need for the accurate solution of hyperbolic problems. This second-order scheme is unconditionally stable, which makes it popular in computing solutions to PDEs. Speth et al. [36] further developed balanced splitting and rebalanced splitting methods; compared with earlier splitting methods, these have higher accuracy and can correctly capture steady-state behavior. Goldstein and Osher [37] discussed an algorithm that is equivalent to the ADMM, and this method has been used in many image-processing applications.

Operator splitting has attracted attention for its computational efficiency in the solution of FP equations. Zorzano et al. [38] combined an ADI scheme with the finite difference method to solve FP problems in particle accelerators. For studying the influence of noise in the particle dynamics in a storage ring, a method based on operator splitting was developed by Mais and Zorzano [39], which can also be used in stochastic beam-dynamics problems in accelerators. Zorzano et al. [40] provided a robust finite difference scheme with operator splitting (ADI) for the FP equation in two spatial variables and time.

Because of the rapid growth of the FE equation with the increasing dimension of the underlying dynamical system, it now seems unlikely that this approach to the solution of the FP equation will be extended beyond the three or four states that have represented a practical limit for several years. However, systems of this size do have practical applications, such as design optimization and nonlinear filtering [41], and so there is some value in pursuing their efficient solution. In addition, we are motivated in part by the goal of providing baseline timing and solution results for the FEA of such problems as the field shifts increasingly toward the routine use of alternative formulations, such as physics-informed neural networks (PINNs) [42,43,44], which are very promising for problems of high dimension but whose numerical properties have not yet been fully tested.

Our interest is thus pragmatic. The finite element analysis of the FP equation is largely a mature subject, with some gaps where known numerical techniques may be applied. We try to provide practical guidance for the use of two of these, model reduction and operator splitting. While we focus on 2-D problems herein, our approach is directly extensible to larger systems, which have structurally similar coefficient matrices; however, both programming complexity and storage requirements grow very quickly, and the efficiency gains we achieve are not enough to offset this. Rather than trying to compete with methods such as the PINN solvers mentioned above (a rough computation based on estimating the numbers of floating-point operations needed suggests that they overtake FEA by the time a system has grown to four states), we seek to improve the efficiency of the solution of a class of problems that may serve as well-understood test cases. For the same reason, we believe these results will be of value to those studying FEA or other techniques [45,46,47] with an eye primarily to performance and algorithmic properties other than speed.

In the following sections, the FP equation is used to formulate a PDE for the joint pdf of the displacement and velocity of a single-degree-of-freedom (SDOF) oscillator, which is then discretized in the phase plane. The resulting large matrix ordinary differential equation has constant coefficient matrices, and the properties of their eigenvectors for use in a reduced basis are elucidated. One of the coefficient matrices may be usefully split into the sum of two matrices with better stability characteristics, which makes feasible the use of a state transition matrix to advance the solution by a fixed step size. The performance of these techniques is demonstrated through numerical examples where the nonstationary random responses of linear and nonlinear SDOF oscillators are computed. Recommendations based on the findings from these applications are summarized in the results and conclusion.

2. Problem Formulation

Consider an SDOF system (an oscillator) governed by the stochastic differential equations

$$\begin{array}{cc}\hfill {\dot{X}}_1& =g_1\left(\mathbf{X}\right),\hfill \end{array}$$

(1a)

$$\begin{array}{cc}\hfill {\dot{X}}_2& =g_2\left(\mathbf{X}\right)+w\left(t\right),\hfill \end{array}$$

(1b)

where the state vector is $\mathbf{X}={\left[X\dot{X}\right]}^{\mathrm{T}}={\left[X_1{X}_2\right]}^{\mathrm{T}}$ and the overdot denotes differentiation with respect to time. The excitation $w\left(t\right)$ is Gaussian white noise, which is fully defined by its first two moments,

$$E\left[w\left(t\right)\right]=0\mathrm{and}E\left[w\left(t\right)w(t+t^{\prime })\right]=2D\delta \left(t^{\prime }\right),$$

(2)

where $\delta \left(·\right)$ is the Dirac delta function and $D/\pi$ is the magnitude of the constant two-sided spectral density function of the excitation. Equation (1) describes a Markov process, and its behavior is completely determined by the transition probability density function $p(\mathbf{x},t|{\mathbf{x}}_0)$, which satisfies the (deterministic) Fokker–Planck equation

$${\displaystyle \frac{\partial p}{\partial t}}=D{\displaystyle \frac{{\partial }^{2}p}{\partial x_2^2}}-{\displaystyle \frac{\partial }{\partial x_1}}\left[g_1\left(\mathbf{x}\right)p\right]-{\displaystyle \frac{\partial }{\partial x_2}}\left[g_2\left(\mathbf{x}\right)p\right]$$

(3)

and the initial condition ${lim}_{t\to 0}p(\mathbf{x},t|{\mathbf{x}}_0)=\delta (\mathbf{x}-{\mathbf{x}}_0)$. Homogeneous boundary conditions are imposed where $p\to 0$ as $\left|x_1\right|,\left|x_2\right|\to \infty$. The joint probability density function of the response is related to the transition probability density by

$$f(\mathbf{x},t)={\int }_{-\infty }^{\infty }{\int }_{-\infty }^{\infty }p(\mathbf{x},t|{\mathbf{x}}_0)f\left({\mathbf{x}}_0\right)d{\mathbf{x}}_0,$$

(4)

where for convenience the initial condition may be taken in the form of a binormal distribution. Substituting this function into the FP equation and integrating term-wise gives

$${\displaystyle \frac{\partial f}{\partial t}}=D{\displaystyle \frac{{\partial }^{2}f}{\partial x_2^2}}-{\displaystyle \frac{\partial }{\partial x_1}}\left(g_{1}f\right)-{\displaystyle \frac{\partial }{\partial x_2}}\left(g_{2}f\right).$$

(5)

The desired joint probability density function $f(\mathbf{x},t)$ satisfies this PDE subject to the initial conditions on f and the boundary conditions (which are the same as those on p).

A Galerkin finite element method is used where within a single element with domain ${\Omega }_e$, the probability density function is interpolated according to

$$f(\mathbf{x},t)=\sum _{r=1}^n{N}_r\left(\mathbf{x}\right)f_r^e\left(t\right),$$

(6)

where the $N_r\left(\mathbf{x}\right)$, $r=1,2,\dots ,n$, are bilinear trial functions of class $C^0$, n is the number of nodes in a single element, and $f_r^e$ are the nodal values of the density. Substituting Equation (6) into the weak form of Equation (5) leads to

$$\sum _{s=1}^n{m}_{rs}{\dot{f}}_s^e+\sum _{s=1}^n{k}_{rs}{f}_s^e=0,r=1,\dots ,n,$$

(7)

where the element matrices are given by

$$m_{rs}={\int \int }_{{\Omega }_e}{N}_r{N}_{s}d{x}_{1}d{x}_2$$

(8a)

and

$$k_{rs}={\int \int }_{{\Omega }_e}\left[D{\displaystyle \frac{\partial N_r}{\partial x_2}}{\displaystyle \frac{\partial N_s}{\partial x_2}}\right.\left.-N_s\left(g_1{\displaystyle \frac{\partial N_r}{\partial x_1}}+g_2{\displaystyle \frac{\partial N_r}{\partial x_2}}\right)\right]d{x}_{1}d{x}_2.$$

(8b)

Assembling the global coefficient matrices by standard procedures produces

$$\mathbf{M}\dot{\mathbf{f}}\left(t\right)+\mathbf{K}\mathbf{f}\left(t\right)=\mathbf{0},$$

(9)

where $\mathbf{f}\left(t\right)$ is the global vector of nodal values of the joint probability density function and $\mathbf{M}$ and $\mathbf{K}$ are constant, square matrices.

We note some properties of the coefficient matrices that are relevant to the discussion to follow. Both $\mathbf{M}$ and $\mathbf{K}$ are real and sparse. The matrix $\mathbf{M}$ is symmetric and positive definite, while $\mathbf{K}$ is nonsymmetric and has 0 as an eigenvalue with multiplicity one; the corresponding eigenvector, scaled so that

$${\int }_{\Omega }fd\Omega =1,$$

(10)

is the solution to

$$\mathbf{K}\mathbf{f}=\mathbf{0},$$

(11)

and gives the stationary joint pdf of the displacement and velocity responses. (Such normalization is not needed when the transient solution is computed starting from initial conditions given in the form of a pdf.) The matrix $\mathbf{M}$ is independent of the oscillator model and the excitation intensity, while $\mathbf{K}$ depends on both.

Experience has shown that for typical linear, Duffing, and Van der Pol oscillators, an effective finite element discretization can be achieved with about 100 elements in each dimension of a uniform rectangular mesh covering $\Omega$ [6]. This leads to a state vector $\mathbf{f}$ of dimension ${101}^2$ = 10,201. After imposition of homogeneous essential boundary conditions at the nodes on the boundary $\partial \Omega$, this is reduced to 9801.

3. Model Reduction

Given the matrices $\mathbf{M}$ and $\mathbf{K}$, we can pose three eigenproblems whose solutions may be useful in reducing the dimension of Equation (9) and thus lowering the computational cost of its solution. In this section, we review some familiar results from linear algebra and establish a framework for the systematic selection of the eigenvectors to be included in a reduced basis [48]. We also discuss an ad hoc technique for improving satisfaction of the boundary conditions in the reduced problem.

3.1. Eigenproblems in the Coefficient Matrices

The simplest eigenvalue problem we will consider is

$$\mathbf{M}\mathit{\varphi }=\lambda \mathit{\varphi }.$$

(12)

The $n\times n$ matrix $\mathbf{M}$ is real, symmetric, and positive definite, with eigenvalues ${\lambda }_i$, $i=1,\dots ,n$, where n is on the order of ${10}^4$ in the problems considered herein. Consequently, the eigenvectors ${\mathit{\varphi }}_i$ are orthogonal, both simply and with respect to $\mathbf{M}$, and can be normalized so that

$${\mathit{\varphi }}_i^{\mathrm{T}}\mathbf{M}{\mathit{\varphi }}_j={\delta }_{ij}.$$

(13)

If we define a transformation matrix using a subset of the eigenvectors,

$$\mathbf{S}=\left[\cdots {\mathit{\varphi }}_i\cdots \right],$$

(14)

and let

$$\mathbf{f}=\mathbf{S}\mathit{\eta },$$

(15)

we obtain

$$\dot{\mathit{\eta }}+\widehat{\mathbf{K}}\mathit{\eta }=\mathbf{0},$$

(16)

where

$$\widehat{\mathbf{K}}={\mathbf{S}}^{\mathrm{T}}\mathbf{K}\mathbf{S}.$$

(17)

If we have used $m<n$ eigenvectors in defining the transformation, the matrix $\mathbf{S}$ is not square, but $n\times m$; as a result, $\mathit{\eta }$ is of length m, and $\widehat{\mathbf{K}}$ is $m\times m$. The dimension of the problem has been reduced from n to m.

The situation is somewhat different if we instead choose to solve

$$\mathbf{K}\mathit{\varphi }=\lambda \mathit{\varphi }.$$

(18)

Because $\mathbf{K}$ is real but not symmetric, its eigenvalues may not all be real. We do again expect an eigenvalue of 0 and its (real) eigenvector giving the stationary pdf. (Computationally, it is usually necessary to accept the eigenvalue that is smallest in modulus as “zero”, but we have encountered no cases in which this was problematic.) The other eigenvectors may be complex, so we need to use the Hermitian (conjugate) transpose; otherwise, the foregoing calculations are essentially unchanged. In general, the eigenvectors of $\mathbf{K}$ are not orthogonal. Because neither of the transformed coefficient matrices $\widehat{\mathbf{M}}$ and $\widehat{\mathbf{K}}$ can be expected to be diagonal while they both will likely be complex, it is not clear that this offers any advantage over the original DE with real coefficient matrices. By defining the transformation of coordinates using a matrix $\mathbf{S}$ containing a subset of the eigenvectors of $\mathbf{K}$, it is again possible to obtain a reduced problem. However, the real-valued reduced problem of the previous subsection, constructed using eigenvectors of $\mathbf{M}$, seems preferable.

The solution of the generalized eigenproblem

$$\mathbf{K}\mathit{\varphi }=\lambda \mathbf{M}\mathit{\varphi }$$

(19)

will produce n eigenvalues and eigenvectors, generally complex-valued. We once again expect an eigenvalue of zero, with its eigenvector giving the stationary solution. The eigenvectors have properties similar to those of $\mathbf{K}$ above, and are not orthogonal with respect to either matrix. The reduced coefficient matrices, while they may be structurally simpler than $\mathbf{M}$ and $\mathbf{K}$, are again both complex.

Subsets of vectors from any of the three problems will work in practice. Because neither of the transformed coefficient matrices is diagonal, and because many standard differential equation solvers require that the resulting complex-valued problem be recast as a real-valued problem of double the dimension, there is little attraction to using eigenvectors of $\mathbf{K}$. The same observations can be made about the use of vectors from the generalized eigenproblem, although there it might be hoped that these drawbacks would be offset by the greater effectiveness of a smaller basis. However, experience has shown that neither set of complex eigenvectors offers any significant advantage over the real eigenvectors obtained from the regular eigenproblem in $\mathbf{M}$, particularly regarding the size of the basis required to achieve a given accuracy. Furthermore, the use of eigenvectors of $\mathbf{M}$ allows the diagonalization of one coefficient matrix, and, with proper normalization, $\widehat{\mathbf{M}}=\mathbf{I}$. Therefore, we choose to draw our reduced basis from the eigenvectors of matrix $\mathbf{M}$.

3.2. Selection of Basis Vectors

The questions immediately arise of how to choose m and which m eigenvectors are to be included in $\mathbf{S}$. Some guidance may be had if a vector $\mathbf{u}$ representing a meaningful pdf is available—for example, results from a past calculation. The projections of this vector onto the eigenvectors are given by

$${\beta }_i={\displaystyle \frac{{\mathit{\varphi }}_i^{\mathrm{T}}\mathbf{u}}{{\mathit{\varphi }}_i^{\mathrm{T}}{\mathit{\varphi }}_i}},i=1,\dots ,n,$$

(20)

when ${\mathit{\varphi }}_i$ is real, and by a similar expression involving the conjugate transpose when the eigenvector is complex. These coefficients can be sorted by absolute value and examined for abrupt changes indicating groups of eigenvectors of similar importance in the new basis. If there are no obvious groupings of the ${\beta }_i$, the results will at least show which ${\mathit{\varphi }}_i$ should be chosen first. Naturally, the value of this ranking depends on the accuracy of $\mathbf{u}$ in representing the solution over time. It may be desirable to construct a composite, artificial “solution” from snapshots taken from a previously computed time series, if one is available. This technique is demonstrated in the examples of a later section.

3.3. Limitations of Solution on Reduced Basis

When computing a nonstationary solution of the FP equation on a reduced basis, it is frequently observed that while the overall character of the pdf obtained at any time step is correct, there are small errors in both the peak values and the tails of the distribution. The latter often manifest as large values near the boundaries, and in combination with the homogeneous boundary conditions lead to large gradients in the pdf and so encourage diffusion of probability mass over the boundaries. This leads to errors that grow over time and ultimately to an underestimated, if not qualitatively wrong, stationary solution. Enlarging the computational domain, refining the mesh, or both have little effect on this phenomenon; a nearly complete basis is often needed if the solution is to satisfy the boundary conditions over time. This significantly limits the dimension reduction that can be employed.

This deterioration of the pdf can be ameliorated by introducing into the original PDE artificial diffusive terms that oppose this loss over the boundaries. Specifically, we consider a modified equation in the form

$${\displaystyle \frac{\partial f}{\partial t}}=D{\displaystyle \frac{{\partial }^{2}f}{\partial x_2^2}}+{\alpha }_1{\displaystyle \frac{{\partial }^{2}f}{\partial x_1^2}}+{\alpha }_2{\displaystyle \frac{{\partial }^{2}f}{\partial x_2^2}}-{\displaystyle \frac{\partial }{\partial x_1}}\left[g_{1}f\right]-{\displaystyle \frac{\partial }{\partial x_2}}\left[g_{2}f\right].$$

(21)

For simplicity, the coefficients ${\alpha }_1$ and ${\alpha }_2$ are taken to be nonzero only in rectangular regions of widths $b_1$ and $b_2$ near $\partial \Omega$, starting at zero on the inner edges of these strips and growing quadratically to values of $d_1$ and $d_2$ on the boundaries. These terms are easily discretized in the finite element formulation of the problem. The effects of the parameters controlling the subdomains (“boundary layers”) where these artificial terms exist and the strength of the resulting inward diffusion are examined in the examples to follow.

4. Operator Splitting

If the convection and diffusion terms in the original equation are separated, perhaps in the expectation that their physics may develop on different time scales, the terms in the element $\mathbf{k}$ matrix can be grouped and that matrix can be divided into

$${\mathbf{k}}_1=\underset{{\Omega }_e}{\int \int }D{\displaystyle \frac{\partial \mathbf{N}}{\partial x_2}}{\displaystyle \frac{\partial {\mathbf{N}}^{\mathrm{T}}}{\partial x_2}}d{x}_{1}d{x}_2$$

(22a)

and

$${\mathbf{k}}_2=-\underset{{\Omega }_e}{\int \int }\left(g_1{\displaystyle \frac{\partial \mathbf{N}}{\partial x_1}}+g_2{\displaystyle \frac{\partial \mathbf{N}}{\partial x_2}}\right){\mathbf{N}}^{\mathrm{T}}d{x}_{1}d{x}_2.$$

(22b)

Obviously, this splitting satisfies $\mathbf{k}={\mathbf{k}}_1+{\mathbf{k}}_2$. Then, global matrices ${\mathbf{K}}_1$ and ${\mathbf{K}}_2$ can be assembled as the previous $\mathbf{K}$ matrix was and transformed to obtain ${\widehat{\mathbf{K}}}_1={\mathbf{S}}^{\mathrm{T}}{\mathbf{K}}_1\mathbf{S}$ and ${\widehat{\mathbf{K}}}_2={\mathbf{S}}^{\mathrm{T}}{\mathbf{K}}_2\mathbf{S}$.

For the system of ordinary differential equations of the form $\dot{\mathit{\eta }}+\widehat{\mathbf{K}}\mathit{\eta }=\mathbf{0}$ obtained using $\mathbf{S}=\left[\begin{array}{ccc}{\mathit{\varphi }}_1& \dots & {\mathit{\varphi }}_n\end{array}\right]$ (so that $\widehat{\mathbf{M}}=\mathbf{I}$), we begin with the simple first-order splitting method and

$$\dot{\mathit{\eta }}+({\widehat{\mathbf{K}}}_1+{\widehat{\mathbf{K}}}_2)\mathit{\eta }=\mathbf{0},$$

(23)

and separate the original equation into two parts. Over a time step ${\Delta }_t$, we solve the two equations

$$\dot{\mathbf{q}}+{\widehat{\mathbf{K}}}_1\mathbf{q}=\mathbf{0},\dot{\mathbf{p}}+{\widehat{\mathbf{K}}}_2\mathbf{p}=\mathbf{0}$$

(24)

separately, using the solution of the first as the initial conditions for the second, and then combine the two separate solutions to form a solution to the original Equation (23) [35]. Using the state transition matrices associated with Equation (24), we can advance the complete solution from time step n to step $n+1$ by computing

$${\mathbf{q}}_{n+1}=e^{-{\mathbf{K}}_1{\Delta }_t}{\mathbf{p}}_n,{\mathbf{p}}_{n+1}=e^{-{\mathbf{K}}_2{\Delta }_t}{\mathbf{q}}_{n+1}.$$

(25)

Combining these equations for a fixed time step ${\Delta }_t$, the result

$${\mathit{\eta }}_{n+1}=e^{-{\widehat{\mathbf{K}}}_2{\Delta }_t}{e}^{-{\widehat{\mathbf{K}}}_1{\Delta }_t}{\mathit{\eta }}_n$$

(26)

is obtained, which may be used to efficiently advance the solution over any number of steps by simply multiplying the previous value by a constant matrix. By comparison to the Taylor series for the exact solution of a canonical equation, it can be shown that this splitting method is first-order accurate ($\mathcal{O}\left({\Delta }_t\right)$) even when the matrices ${\mathbf{K}}_1$ and ${\mathbf{K}}_2$ do not commute under multiplication [49].

This scheme is both simple and stable once the matrix exponentials are computed and multiplied to form the single-state transition matrix needed to advance the solution over a fixed step. It is of course possible to choose a time step so large that the exponentiation (carried out in floating-point arithmetic) fails, but our experience has been that using time steps small enough to resolve the dynamics of interest in a system usually avoids any such difficulties.

Symmetric splitting, as introduced by Strang [35], can achieve second-order accuracy. Here, $\mathbf{K}$ is split into

$$\mathbf{K}=\left(\frac{1}{2}{\mathbf{K}}_1+{\mathbf{K}}_2\right)+\frac{1}{2}{\mathbf{K}}_1.$$

(27)

Examining the Taylor series shows that

$$e^{{\displaystyle \frac{1}{2}}{\mathbf{K}}_1{\Delta }_t}{e}^{{\mathbf{K}}_2}{e}^{{\displaystyle \frac{1}{2}}{\mathbf{K}}_1{\Delta }_t}-e^{({\mathbf{K}}_1+{\mathbf{K}}_2){\Delta }_t}=\mathbf{C}{\Delta }_t^3+O\left({\Delta }_t^4\right),$$

(28)

where

$$\mathbf{C}=\left[[{\mathbf{K}}_1,{\mathbf{K}}_2],{\mathbf{K}}_1\right]+2\left[[{\mathbf{K}}_1,{\mathbf{K}}_2],{\mathbf{K}}_2\right]$$

(29)

and where $[\mathbf{A},\mathbf{B}]$ denotes the matrix commutator $\mathbf{A}\mathbf{B}-\mathbf{B}\mathbf{A}$. The total number of time steps is $T/{\Delta }_t$, so the global error is $\mathcal{O}\left({\Delta }_t^2\right)$.

5. Numerical Examples

The example problems of this section are taken from Spencer and Bergman [6] and comprise two SDOF oscillators, one linear and one nonlinear (a Duffing oscillator with negative linear stiffness but positive cubic stiffness). The notation adopted here is typical of that used to describe vibrating mechanical systems, but these equations of motion have analogs in many areas of science and engineering. Initial conditions are specified as a binormal distribution. For each example, we give results from reduced problems, from operator splitting, and from these techniques used in combination. The solution of the full FE problem by an adaptive Runge–Kutta (RK) integrator is used as a reference except in those cases where an exact solution is available. (Specifically, we used the function solve_ivp from the SciPy Python library (v. 1.8.0), which implements the RK5(4) algorithm of Dormand and Prince [50].) This routine was also used to integrate those reduced problems for which operator splitting was not employed. Time-dependent results are usually reported here in terms of the nondimensional time $\tau ={\omega }_{0}t/2\pi$ (i.e., in units of the (linearized) natural period of the system being considered).

Basis vectors were selected from eigenvectors of $\mathbf{M}$ with the aid of a composite vector $\mathbf{u}$ constructed by superposing twenty snapshots from a full RK solution, equally spaced in time from the relevant initial conditions to apparent stationarity, according to the magnitudes of the projections of this vector onto the eigenvectors. This allows the estimation of the minimum basis size required for the solution of the nonstationary problem, but it is obviously not a technique that will be generally available. Because the required bases were found to be quite large, half or more of the original problem dimension in the Duffing example, we have not investigated alternative means of selecting the best basis vectors; however, we note that the eigenvectors selected on the basis of the projection coefficients obtained using $\mathbf{u}$ and the eigenvectors corresponding to the largest eigenvalues of $\mathbf{M}$ were generally strongly correlated. This suggests that, in the typical case where a composite solution vector is not available, including in the basis, those eigenvectors with the largest eigenvalues will generally give the best results. This choice, and the required size of the basis, will be influenced by the degree to which the nonstationary solution is localized, and thus by the initial conditions. The initial mean values used in our examples are rather large, which makes for interesting but challenging examples.

We summarize here the steps followed in each example when either Runge–Kutta integration or first-order operator splitting was used. The steps for second-order splitting are very similar.

• Form the finite element global coefficient matrices $\mathbf{M}$ and $\mathbf{K}$. If operator splitting will be used, form ${\mathbf{K}}_1$ and ${\mathbf{K}}_2$ in lieu of $\mathbf{K}$. If basis reduction will be used, optionally include in $\mathbf{K}$ or ${\mathbf{K}}_1$ additional terms representing artificial diffusion.
• Solve the eigenproblem $\mathbf{M}\mathit{\varphi }=\lambda \mathit{\varphi }$ and normalize the resulting vectors with respect to $\mathbf{M}$. (Hence, $\widehat{\mathbf{M}}=\mathbf{I}$.)
• Select the eigenvectors to be used to transform the problem (which may be all of them) and form $\mathbf{S}$.
• Compute $\widehat{\mathbf{K}}$ or ${\widehat{\mathbf{K}}}_1$ and ${\widehat{\mathbf{K}}}_2$, as appropriate.
• Starting from the given initial conditions, march the solution in time (typically to stationarity).

  (a)

  If using a Runge–Kutta integrator, solve $\dot{\mathit{\eta }}=-\widehat{\mathbf{K}}\mathit{\eta }$ directly. If the output is in the form of a sparse structure, evaluate the solution at the desired times (e.g., on uniform steps).

  (b)

  If using operator splitting, choose a time step ${\Delta }_t$, evaluate the necessary matrix exponentials, and form the corresponding state transition matrix (cf. Equation (26)). Advance the solution by forming a sequence of matrix–vector products.

Computations were performed on a typical x86 desktop computer using programs written in Python and taking advantage of its compiled numerical libraries, with the exception of the eigensolutions, which were found using Matlab (v. R2021b) running on the same hardware. The reported run times (normalized by the time required for the computation of the RK reference solution) include any transformations associated with problem reduction and recovery of the full solution vector.

5.1. Linear Oscillator

The FP equation for a linear system has an analytical solution [1,2], and it is frequently used as a benchmark problem for testing numerical methods [4,6,8]. We consider the well-known SDOF linear oscillator (LO)

$$\begin{array}{cc}\hfill g_1(x_1,x_2)& =x_2,\hfill \end{array}$$

(30a)

$$\begin{array}{cc}\hfill g_2(x_1,x_2)& =-2\zeta {\omega }_0{x}_2-{\omega }_0^2{x}_1,\hfill \end{array}$$

(30b)

with the parameters $\zeta =0.05$ and ${\omega }_0=1$, the excitation $D=0.1$, and the initial conditions $({\mu }_{10},{\mu }_{20})=(5,5)$ and $({\sigma }_{10}^2,{\sigma }_{20}^2)=(1/9,1/9)$. The domain for computation is square, $-10\le x_1,x_2\le 10$. The FE mesh is uniform, with an element size of 0.2 in both the $x_1$ and $x_2$ directions.

We begin with the FEA represented by Equations (8) and (9), applied to the state Equation (30). Using a composite solution vector $\mathbf{u}$ composed of snapshots taken from a reference Runge–Kutta solution of the full FE equations, the coefficients resulting from the projection of this vector onto all the eigenvectors of $\mathbf{M}$ are plotted in Figure 1, where they have been sorted from large to small. The norm of the error resulting when $\mathbf{u}$ is approximated with an increasing number of eigenvectors follows a similar trend.

When artificial diffusive terms were used in the solution of this problem on reduced bases, parameters were chosen after a limited amount of experimentation. In the $x_1$ direction, the parameters $b_1=2.4$ and $d_1=1.6$ were selected, such that the added terms are nonzero only for $\left|x_1\right|>7.6$ and rise to a maximum value of 1.6 on the boundary of $\Omega$. Likewise, in the $x_2$ direction, with $b_2=2.5$ and $d_2=1.3$, the boundary terms enter where $\left|x_2\right|>7.5$, with coefficients that increase to 1.3 on $\partial \Omega$. While an effort was made to minimize the effect of these terms on the original PDE (beyond stabilizing clearly erroneous solutions), the results obtained were not particularly sensitive to the values chosen.

Figure 2 compares solutions computed using a reduced basis of 2000 eigenvectors, with and without artificial diffusive boundary terms, to the exact solution. The basis vectors were selected according to the projection coefficient magnitudes in Figure 1. Figure 2a shows the evolution with time of the nonstationary pdf at the origin of the phase plane. Both reduced solutions exhibit initial large errors because of the distortion of the initial conditions caused by the transformation onto the smaller basis. Both then track the exact solution until approximately $\tau =6$, when the reduced solution computed without added boundary terms begins to visibly exceed the reference curve. This trend continues, and would eventually result in divergence rather than a stationary solution. In contrast, the numerical solution computed on the same basis but with diffusive terms introduced near the boundary continues to agree with the exact analytical result, exceeding it only slightly as $\tau$ approaches 20. A cross section of the stationary pdf along the displacement axis of the phase plane is plotted in Figure 2b, where it may be seen that the artificial diffusive boundary terms both improve the accuracy of the peak value and reduce spurious oscillations in the tails of the distribution. However, both reduced solutions do oscillate rather than going smoothly to zero in the tails, and consequently, they take on physically meaningless negative values at some points.

Figure 3 and Figure 4 reveal the effect of basis size, depicting the same responses discussed above, this time computed using a basis of 1000, 1500, or 2000 eigenvectors. All of the reduced problems were solved using artificial diffusive terms to improve the solutions. Collectively, these figures suggest that the use of fewer than 2000 basis vectors will result in spurious, sometimes negative numerical solutions, if not outright divergence. This is an important and somewhat surprising result: despite the simplicity and smoothness of the solution, the minimum useful basis is approximately 20% of the size of the original problem.

We now return to consideration of the full-dimensional problem and examine the use of operator splitting in its solution, without reduction. When the results of operator splitting are compared to the exact solution, no significant differences are found except that the splitting solution very slightly underestimates the peak value of the stationary pdf. The nonstationary and stationary numerical solutions are practically indistinguishable from the exact ones, and so are not shown here.

In combination with operator splitting, larger and smaller reduced bases, of sizes 1000 and 3000, were used without adding boundary terms. Trends and errors similar to those seen in the earlier reduced formulations for this system were found. We conclude that the use of a basis consisting of 2000 eigenvectors of the $\mathbf{M}$ matrix along with splitting the $\mathbf{K}$ matrix according to convective and diffusive terms produces reasonably accurate results for this problem.

Table 1. Integration times for various methods of solution for the linear oscillator ($\tau$ = 0:0.01:4). The size of the basis used in each method is shown in parentheses. ABT denotes the use of artificial boundary terms. Run times are normalized with respect to the Runge–Kutta solution of the full FE problem.

Method	Time, %
RK (9801)	100.00
RK (3000)	15.90
RK (2000)	11.98
RK (3000) and ABT	17.17
RK (2000) and ABT	12.70
1st-order splitting (9801)	17.36
1st-order splitting (3000)	10.07
1st-order splitting (2000)	9.69
1st-order splitting (3000) and ABT	10.16
1st-order splitting (2000) and ABT	9.81

shows the run times of the different calculation methods applied to this example. Compared with Runge–Kutta integration, operator splitting and solving the problem using a state transition matrix can obviously accelerate the process. The more basis vectors used, naturally, the longer any solver will take. Solving the equation with artificial boundary terms is more time-consuming, but because of the higher accuracy, the run time cost is generally worthwhile. In many cases, the use of only operator splitting, but not reduction, may result in acceptable run times, obviating any need for artificial diffusive terms.

5.2. Duffing Oscillator

The governing equation of the nonlinear Duffing oscillator is, in state form,

$$\begin{array}{cc}\hfill g_1(x_1,x_2)& =x_2,\hfill \end{array}$$

(31)

$$\begin{array}{cc}\hfill g_2(x_1,x_2)& =-2\zeta {\omega }_0{x}_2-{\omega }_0^2{x}_1(\gamma +ϵx_1^2),\hfill \end{array}$$

(32)

with $\zeta =0.2$, ${\omega }_0=1$, $\gamma =-1$, $ϵ=0.1$, and $D=0.4$. (Like the previous example, this system is taken from [6].) These parameters are those of a system with negative linear stiffness but positive cubic stiffness, resulting in a bimodal stationary pdf that is doubly symmetric about the origin of the phase plane and approximately Gaussian in $x_2$. The nonlinearity thus controls the stationary response despite the small numerical value of $ϵ$. The initial conditions were taken to be $({\mu }_{10},{\mu }_{20})=(0,10)$ and $({\sigma }_{10}^2,{\sigma }_{20}^2)=(1/2,1/2)$, and the FE domain was $-15\le x_1\le 15$, $-20\le x_2\le 20$, with 100 elements used in each dimension.

Guided by the projection coefficients of a composite vector $\mathbf{u}$ and some experimentation, we determined that somewhat more than 4000 basis vectors were needed to capture the dynamics of this problem in reduced form. The diffusive boundary layer coefficients $b_1=8$ and $d_1=4.6$ and $b_2=13$ and $d_2=0.1$ were then chosen for the $x_1$ and $x_2$ directions, respectively.

The discretized equation for the joint pdf of the Duffing oscillator was reduced onto a basis of 4500 eigenvectors, with and without introducing artificial boundary terms in the PDE. The nonstationary response in Figure 5a is shown for the point $(3.3,0)$, which is the approximate location in the phase plane of one of the two peaks of the bimodal stationary distribution. Both the full and reduced problems were integrated here by a standard Runge–Kutta routine. When artificial diffusive boundary terms are included in the formulation, the solution of the resulting reduced problem approximates well the solution of the full problem, after a transient error attributed to the inability of the reduced basis to represent the given initial conditions accurately. Without added boundary terms, however, the solution does not approach the correct stationary value, and begins to decrease from it as probability mass is lost over the edges of the FE domain. Figure 5b, a section of the stationary pdf along the $x_1$ axis, shows spurious oscillations in the tails of the pdf. (The exact stationary solution shown there is from Caughey [3].) Both reduced problems underestimate the pdf in the valley between the peaks of the stationary distribution. While the inclusion of diffusive boundary terms reduces some of the extraneous ripples, it must be said that these results are not as clean as those obtained for the linear oscillator, despite the use of a much larger reduced basis (here, almost 46% of the size of the full problem).

When the results of first- and second-order splitting are compared to those of a Runge–Kutta solution of the unsplit problem, second-order splitting is found to be more accurate, but both solutions from splitting are found to agree quite well with the RK solution during the entire response, including at stationarity.

Responses found using a combination of basis reduction and operator splitting, with and without artificial diffusive boundary terms, followed the trends observed for the linear oscillator. Even when the basis was enlarged to 5000 eigenvectors, the solutions found without boundary terms deviated significantly from the reference RK integration of the unreduced system. With boundary terms, good results were obtained with the previous basis size of 4500, albeit with the same sorts of errors observed when these equations were earlier solved without splitting.

The times required to solve this example as described above are shown in

Table 2. Integration times for various methods of solution for the Duffing oscillator ($\tau$ = 0:0.01:4). The size of the basis used in each method is shown in parentheses. ABT denotes the use of artificial boundary terms. Run times are normalized with respect to the Runge–Kutta solution of the full FE problem.

Method	Time, %
RK (9801)	100.00
RK (4500)	19.84
RK (4500) and ABT	19.76
1st-order splitting (9801)	1.57
2nd-order splitting (9801)	1.61
1st-order splitting (4500)	1.00
1st-order splitting (4500) and ABT	1.03

. Because of the large basis required, the savings compared to the full-dimensional problem are not as dramatic as for the linear oscillator, but they are still significant. Operator splitting and advancing the solution using a state transition matrix, however, yields great improvement in computational efficiency compared to straightforward time integration using a Runge–Kutta algorithm. Further savings can be achieved by combining dimension reduction with operator splitting, but in view of the errors that result and the practical requirement for adding artificial diffusive terms to the PDE, this approach is relatively unattractive in this example.

6. Results

We have investigated two techniques to improve the speed of solution of Fokker–Planck equations as discretized using finite element analysis in the phase plane. In the first of these, reduced bases were constructed from selected eigenvectors of the coefficient matrices $\mathbf{M}$ and $\mathbf{K}$ appearing in the resulting first-order differential equation. Basis vectors obtained by solving a regular eigenproblem in $\mathbf{M}$ were attractive because they were real-valued, could be used to diagonalize $\mathbf{M}$, and were independent of the parameters of the underlying dynamical system and the intensity of the excitation. The size of the basis required in order for the reduced problem to capture well the dynamics of the full problem was typically one-fifth to one-half that of the original system. Considering that the number of elements of the reduced coefficient matrices goes roughly as the square of this ratio, this represents an attractive savings in storage and computation; however, it does not always enable the drastic improvement in computational efficiency sometimes associated with a reduced-order model. In addition, even when a large basis was used, the reduced problem was found to be susceptible to spurious oscillations in the tails of the computed pdf, and to loss of probability mass over the edges of the computational domain. These phenomena were effectively addressed by the introduction into the original PDE of artificial diffusive terms near the boundaries of the domain. While greater gains were achieved in the linear example, in the more representative, nonlinear Duffing oscillator example, the typical reduction in run time, compared to integration by a conventional adaptive Runge–Kutta algorithm, was approximately 80%. This is certainly significant; however, we must reiterate that the reduced bases were selected with the benefit of the composite solution vector $\mathbf{u}$, and that even a modest reduction in the size of the basis used was found to require the introduction of artificial diffusive terms to stabilize the solution near the boundaries of the computational domain. In view of these limitations, we cannot enthusiastically recommend the use of this reduction.

The second numerical method applied herein to the same discretized PDE was operator splitting, where the singular coefficient matrix $\mathbf{K}$ was written as the sum of two nonsingular matrices. This allowed the rapid computation of the joint response pdf with a uniform time step using the constant state transition matrix given by the product of the exponentials of the two component matrices. Both first- and second-order-accurate splitting schemes were considered. It was observed that, although the cost of second-order methods was not much greater, first-order splitting was usually sufficiently accurate while being simpler to program, and that separating the FE coefficient matrix $\mathbf{K}$ according to the physical source of the terms in the original PDE (i.e., convection or diffusion) was effective in the linear and Duffing oscillator examples studied. The resultant saving in computational effort was comparable to that obtained with basis reduction for the linear oscillator example, a little more than 80%, but savings of over 98% were observed when operator splitting was used to compute the joint pdf of a Duffing oscillator.

When these methods were combined, by using operator splitting to solve equations reduced on a basis of selected eigenvectors, the advantages of both techniques were retained. Run times for the solution phase of nonstationary problems were commonly 1% to 10% of those required for Runge–Kutta integration. This improvement was achieved with little cost in accuracy, although the caveats stated above regarding basis reduction apply here as well.

7. Conclusions

The class of dynamical systems for which the solution of the FP equation by FEA is tractable is small, effectively limited to those with only a few states. While the methods reported here will not, alone, allow the treatment of new, larger systems, they do represent a significant advance in the speed of the computation of the nonstationary pdfs of realistic, complex small systems. It is expected that these results will be useful to anyone needing to accelerate such calculations in a particular application or to test an alternative numerical approach.