Rational Approximation Method for Stiff Initial Value Problems

Abstract

While purely numerical methods for solving ordinary differential equations (ODE), e.g., Runge–Kutta methods, are easy to implement, solvers that utilize analytical derivations of the right-hand side of the ODE, such as the Taylor series method, outperform them in many cases. Nevertheless, the Taylor series method is not well-suited for stiff problems since it is explicit and not A-stable. In our paper, we present a numerical-analytical method based on the rational approximation of the ODE solution, which is naturally A- and $A\left(\alpha \right)$-stable. We describe the rational approximation method and consider issues of order, stability, and adaptive step control. Finally, through examples, we prove the superior performance of the rational approximation method when solving highly stiff problems, comparing it with the Taylor series and Runge–Kutta methods of the same accuracy order.

1. Introduction

The basic principle in designing numerical methods for solving the initial value problem

$$\dot{y}=f(t,y),y\left(t_0\right)=y_0$$

(1)

is that the numerical method must fit the Taylor series expansion of the solution in a given point with the desired accuracy [1]. The Taylor series can directly be used for numerical integration as well, and a number of algorithms and software have been developed based on Taylor series expansion [2,3,4,5].

In real-life applications, the one-step Taylor series method outperforms purely numerical one-step algorithms such as the Dormand–Prince and Gragg–Bulirsch–Stör methods when extended data types are used and small error tolerances are required. Its high computational efficiency on conventional data types can be achieved by using automatic differentiation for computing the derivatives in the Taylor expansion [3,6].

The main drawback of the explicit Taylor series method is that it can fail on stiff ODEs, i.e., lose stability when the step size is not appropriate or may select small step sizes in adaptive time step implementation, being guided more by stability requirements rather than accuracy limits, which results in performance loss. The formal definition of stiffness can be found in [7], and a vital feature of the numerical method suitable for efficient solving stiff equations is A-stability and its generalizations, e.g., $A\left(\alpha \right)$-stability [8]. The Taylor series method is not A-stable. Still, its stability region grows with its order [9], which may increase the performance of high-order Taylor series methods on moderately stiff problems. Recently, a modification of the Taylor series method for semi-linear problems called the exponential Taylor series method was proposed [10]. This idea can be used to derive Taylor-like methods, which are A-stable and L-stable [11].

In general, the Taylor series approximation can be considered a variant of the solution’s polynomial approximation with respect to time step h.

However, the Taylor series is not the only variant of the function decomposition that allows for constructing integration schemes. Several other examples include the Haar wavelet decomposition and hybrid function decomposition for integer-order ODEs [12], and fractional-order general Lagrange scaling functions for fractional-order ODEs [13]. It is also well known that, in some practical cases, using rational approximation instead of polynomial approximation is more suitable, e.g., when the approximated function contains poles. L. Wuytack was among the first researchers to notice it in application to numerical integration and proposed the second-order numerical method based on the Padé approximation for solving problems with singularities [14]. Later works, e.g., [15], developed this idea. It is also notable that there is a well-known relation between multi-step methods and the Padé approximation [16]. The Padé approximation of a matrix exponential in domain-specific implementations of the exponential Taylor method can be highly efficient [17].

An attempt to analyze the applicability of the Padé type rational approximation methods to the one-dimensional stiff problem was made by M. Gadella and L. Lara [18]. However, the high potential of the rational approximation methods in solving stiff multidimensional problems has still not been investigated.

This paper aims to fill this gap by introducing a family of A- and $A\left(\alpha \right)$-stable rational approximation methods and by studying their performance on stiff problems. The paper is organized as follows. In Section 2, we introduce the theoretical background of the proposed rational approximation method. Section 3 presents the stability analysis and proves that the proposed method is A- or $A\left(\alpha \right)$-stable depending on the order of accuracy. In Section 4, we introduce an adaptive step control technique for this method. In Section 5, we present the experimental results. Section 6 gives the conclusions.

The following special notation is accepted in the paper:

• ∘ denotes the Hadamard (element-wise) multiplication;
• ⊘ denotes an arbitrary binary vector operation.

2. Basics of Rational Approximation in Numerical Integration

For a given initial value problem (1), where y is a vector-valued phase variable and t is a scalar (time), the approximate solution obtained in the next time point is denoted as $y_{n+1}\approx y\left(t_{n+1}\right)$. Denote the time step $h=t_{n+1}-t_n$. Then, consider the Taylor series that approximates the solution:

$$y_{n+1}=\sum _{i=0}^{\infty }\frac{y_n^{\left(i\right)}}{i!}{h}^i.$$

(2)

Consider a rational approximation of $y_{n+1}$ as a fraction of two polynomials $P_L\left(h\right)$ and $Q_M\left(h\right)$:

$$\begin{array}{cc}\hfill P_L\left(h\right)& =p_0+p_{1}h+...+p_L{h}^L,\hfill \\ \hfill Q_M\left(h\right)& =q_0+q_{1}h+...+q_M{h}^M,\hfill \end{array}$$

where L and M are powers of these polynomials. The rational approximation of the solution reads:

$$y_{n+1}=\frac{P_L\left(h\right)}{Q_M\left(h\right)}.$$

(3)

A number of reliable algorithms for finding the rational approximant (3) exist [19]. Here, we use the most straightforward approach. Let $M+L=p$, where p is a certain natural number. Assuming that (2) and (3) must give the same result up to the error term $O\left(h^{p+1}\right)$, it follows that

$$\sum _{i=0}^{\infty }\frac{y^{\left(i\right)}}{i!}{h}^i-\frac{P_L\left(h\right)}{Q_M\left(h\right)}=O\left(h^{p+1}\right),$$

(4)

which results in a system of equations:

$$\begin{array}{c}\hfill \left\{\begin{array}{c}{p}_0=y_n{q}_0,\hfill \\ p_1=h{y^{\prime }}_n{q}_0+y_n{q}_1,\hfill \\ p_2=h^2\frac{y_n^{″}}{2}{q}_0+h{y}_n^{\prime }{q}_1+y_n{q}_2,\hfill \\ ⋮\hfill \\ p_L=h^L\frac{y_n^{\left(L\right)}}{L!}{q}_0+h^{L-1}\frac{y_n^{(L-1)}}{(L-1)!}{q}_1+\cdots +y_n{q}_L,\hfill \\ 0=h^{L+1}\frac{y_n^{(L+1)}}{(L+1)!}{q}_0+h^L\frac{y_n^{\left(L\right)}}{L!}{q}_1+\cdots +h^{L-M+1}\frac{y_n^{(L-M+1)}}{(L-M+1)!}{q}_M,\hfill \\ ⋮\hfill \\ 0=h^{L+M}\frac{y_n^{(L+M)}}{(L+M)!}{q}_0+h^{L+M-1}\frac{y_n^{(L+M-1)}}{(L+M-1)!}{q}_1+\cdots +h^L\frac{y_n^{\left(L\right)}}{L!}{q}_M.\hfill \end{array}\right.\end{array}$$

(5)

One can notice that this system is complete only if $L+M=p$, which is exactly the case of the Padé rational approximation. Similarly, the Padé approximant corresponds to the rational approximation of the highest possible order of accuracy. Usually, the notation $R_{L/M}$ or $[L/M]$ is used to denote the powers of polynomials in the Padé approximant with the power of the numerator L and the power of the denominator M. Otherwise, if $L+M-p=r$, then $r>0$, the last r equations from (5) should be omitted, and the r coefficients in P and Q remain free. This option can be used to obtain rational approximants with some desired properties, but their order of accuracy is lower than that with the Padé approximant.

2.1. Padé Approximation

The second-order numerical integration method proposed by L. Wuytack is the $[1/1]$ Padé approximation similar to the type in (3). After simplification, it reads:

$$y_{n+1}=y_n+\frac{h{f}^2(t_n,y_n)}{f(t_n,y_n)-\frac{h}{2}{f}^{\prime }(t_n,y_n)},$$

(6)

where $f^{\prime }(t,y)$ is a time derivative of $f(t,y)$.

The fourth-order Padé numerical method of type [2/2] can be obtained from (5) in a similar way [18]. After simplification, it can be presented as follows:

$$\begin{array}{cc}\hfill y_{n+1}& =y_n+\hfill \\ & +\frac{(24f_n^{″}{f}_n^2-36{f_n^{\prime }}^2{f}_n)h+(24f_n^{″}{f}_n^{\prime }{f}_n-6f_n^{\left(3\right)}{f}_n^2-18{f_n^{\prime }}^3)h^2}{-36{f_n^{\prime }}^2+24f_n^{″}{f}_n+(12f_n^{″}{f}_n^{\prime }-6f_n^{\left(3\right)}{f}_n)h+(-4{f_n^{″}}^2+3f_n^{\left(3\right)}{f}_n^{\prime })h^2},\hfill \end{array}$$

(7)

where $f_n^{\left(q\right)}=f^{\left(q\right)}(t_n,y_n)$ for more compact notation.

Two problems are evident when working with numerical methods such as (7). First, this formula can be straightforwardly implemented in a one-dimensional case, but when $y_n$ and $f_n$ are vectors, some special techniques are needed to perform the division of two vector values, appearing in the numerator and denominator of (7). The classical definition of vector space does not include vector division, but some authors [14,20,21] propose using element-wise operation for this purpose:

$$\frac{\mathbf{v}}{\mathbf{w}}={\left(\frac{v_1}{w_1}\cdots \frac{v_i}{w_i}\cdots \frac{v_m}{w_m}\right)}^{\top }.$$

(8)

Alas, there is no proof that such an operation can preserve A- and $A\left(\alpha \right)$-stability even when the formula is A- or $A\left(\alpha \right)$-stable for a one-dimensional problem [22]. Moreover, it was shown that this approach might notably affect the dynamical properties of the simulated system [21].

The second problem of the Padé approximant is that, starting with the [2/2] formula, it becomes cumbersome and difficult to derive, implement, and compute.

2.2. Proposed Approximation

Reducing the rational approximation order may result in more feasible types of approximation. In this paper, we focus on a class of methods given by $[p-1]/[p-1]$ approximants for even p and $\left[p\right]/[p-1]$ for odd p. The method of order 2 of this class is (6), similar to the Padé approximant. The Methods of orders 3–6 are given in

Table 1. RA methods of orders 3–6.

p	Formula
3	$$y_{n+1}=y_n+\frac{h{f}_n^2+\frac{h^3}{12}\left(4f_n^{″}{f}_n-{f_n^{\prime }}^2\right)}{f_n-\frac{h}{2}{f}_n^{\prime }+\frac{h^2}{6}{f}_n^{″}}$$
4	$$y_{n+1}=y_n+\frac{h{f}_n^2+\frac{h^3}{12}\left(4f_n^{″}{f}_n-3{f_n^{\prime }}^2\right)}{f_n-\frac{h}{2}{f}_n^{\prime }+\frac{h^2}{6}{f}_n^{″}-\frac{h^3}{24}{f}_n^{\left(3\right)}}$$
5	$$y_{n+1}=y_n+\frac{h{f}_n^2+\frac{h^3}{12}\left(4f_n^{″}{f}_n-3{f_n^{\prime }}^2\right)+\frac{h^5}{360}\left(6f_n^{\left(4\right)}{f}_n+10{f_n^{″}}^2-15f_n^{\left(3\right)}{f}_n^{\prime }\right)}{f_n-\frac{h}{2}{f}_n^{\prime }+\frac{h^2}{6}{f}_n^{″}-\frac{h^3}{24}{f}_n^{\left(3\right)}+\frac{h^4}{120}{f}_n^{\left(4\right)}}$$
6	$$y_{n+1}=y_n+\frac{h{f}_n^2+\frac{h^3}{12}\left(4f_n^{″}{f}_n-3{f_n^{\prime }}^2\right)+\frac{h^5}{360}\left(6f_n^{\left(4\right)}{f}_n+10{f_n^{″}}^2-15f_n^{\left(3\right)}{f}_n^{\prime }\right)}{f_n-\frac{h}{2}{f}_n^{\prime }+\frac{h^2}{6}{f}_n^{″}-\frac{h^3}{24}{f}_n^{\left(3\right)}+\frac{h^4}{120}{f}_n^{\left(4\right)}-\frac{h^5}{720}{f}_n^{\left(5\right)}}$$

. These formulas were obtained from the equations in (5) with an additional condition: the denominator of these rational approximants Q should represent the Taylor series for $\left(y(t_n-h)-y\left(t_n\right)\right)/h$:

$$Q=f(t_n,y_n)-\frac{h}{2}{f}^{\prime }(t_n,y_n)+\frac{h^2}{6}{f}^{″}(t_n,y_n)+\cdots$$

Denoting the numerator P, one can notice that, in such types of approximation, the numerator is always a sum of two terms:

$$P=y_n{\Phi }_{-h}(t_n,y_n)+h{\Theta }_h(t_n,y_n),$$

where ${\Phi }_{-h}(t_n,y_n)=Q$ and ${\Theta }_h(t_n,y_n)$ is another function of $t_n$ and $y_n$. Therefore, we can write the general form of these type methods similar to conventional one-step methods:

$$y_{n+1}=y_n+\frac{h{\Theta }_h(t_n,y_n)}{{\Phi }_{-h}(t_n,y_n)}.$$

(9)

One obvious advantage of the proposed rational approximation (RA) method is that they have a more straightforward formulation than Padé approximation methods. Another advantage is that they not only can be A- or $A\left(\alpha \right)$-stable but also can guarantee stability preservation in the multidimensional case, which is considered in the next sections.

2.3. Multidimensional Case

Following [4], we rewrite the non-autonomous ODE (1) in a form of an autonomous ODE:

$$\begin{array}{cc}\hfill Y\left(t\right)& ={\left(t,y_1,\cdots ,y_m\right)}^{\top },\hfill \\ \hfill F\left(y\right)& ={\left(1,f_1\left(Y\right),\cdots ,f_m\left(Y\right)\right)}^{\top },\hfill \end{array}$$

so Equation (1) becomes

$$Y^{\prime }\left(t\right)=F\left(Y\left(t\right)\right).$$

(10)

The first derivatives of $F\left(Y\right)$ are as follows:

$$\begin{array}{cc}\hfill F^{\prime }& =JF,\hfill \\ \hfill F^{″}& =JJF+\frac{\partial J}{\partial Y}FF,\hfill \\ \hfill F^{‴}& =\frac{{\partial }^{2}J}{\partial Y^2}FFF+\frac{\partial J}{\partial Y}JFF+2\frac{\partial J}{\partial Y}FJF+J\frac{\partial J}{\partial Y}FF+JJJF,\hfill \\ \hfill & \cdots ,\hfill \end{array}$$

where $F=F\left(Y\right)$ and $J=\partial F/\partial Y$ is the Jacobian matrix. Higher-order derivatives are given in Appendix A.

Furthermore, we show that the RA method can be written in a matrix form:

$$M_{den}\Delta Y_{n+1}=M_{num}F\left(Y_n\right),$$

(11)

where $\Delta Y_{n+1}=Y_{n+1}-Y_n$, and $M_{num}$ and $M_{den}$ are $d\times d$ matrices, where d is the system dimension after writing it as an autonomous system: $d=m$ if (1) is autonomous and $d=m+1$ if (1) is non-autonomous. This allows us to solve (11) via the Gauss method in a manner similar to implicit methods performed for a single iteration of the Newton–Raphson procedure, which is important for stability.

First, we prove the following lemma.

Lemma 1.

The i-th derivative of$F\left(Y\right)$can be represented as$M_{i}F\left(Y\right)$, where$M_i$is a matrix,$i\in {\mathbb{Z}}_{+}$.

Proof. 

Due to the rules of differentiation,

$$\begin{array}{cc}\hfill F^{\prime }\left(Y\left(t\right)\right)& =\frac{\partial F\left(Y\right)}{\partial Y\left(t\right)}\frac{dY\left(t\right)}{dt}.\hfill \end{array}$$

(12)

Substituting (10) into (12) leads to

$$\begin{array}{c}\hfill F^{\prime }\left(Y\left(t\right)\right)=\frac{\partial F\left(Y\right)}{\partial Y\left(t\right)}F\left(Y\left(t\right)\right),\end{array}$$

(13)

which already reads as $M_1\left(Y\right)F\left(Y\right)$, and after differentiating,

$$\begin{array}{c}\hfill \frac{d}{dt}\left(M_1\left(Y\right)F\left(Y\right)\right)=\left(\frac{\partial M_1\left(Y\right)}{\partial Y\left(t\right)}F\left(Y\right)\right)F\left(Y\right)+M_1^2\left(Y\right)F\left(Y\right).\end{array}$$

(14)

Collecting terms by $F\left(Y\right)$ in (14) as $M_2=\left(\left(\frac{\partial M_1\left(Y\right)}{\partial Y\left(t\right)}F\left(Y\right)\right)F\left(Y\right)+M_1^2\left(Y\right)\right)$, we obtain

$$\begin{array}{c}\hfill \frac{d}{dt}\left(M_1\left(Y\right)F\left(Y\right)\right)=M_2\left(Y\right)F\left(Y\right),\end{array}$$

(15)

which is similar to (12) up to the index by M. Continuing this process, we obtain

$$\begin{array}{c}\hfill \frac{d}{dt}\left(M_i\left(Y\right)F\left(Y\right)\right)=M_{i+1}\left(Y\right)F\left(Y\right),\end{array}$$

(16)

which proves the lemma. □

Then, the following theorem can be proven.

Theorem 1.

RA method of order p

$$y_{n+1}=y_n+\frac{h{f}_n^2+\frac{h^3}{12}\left(-4f_n^{″}{f}_n+3{f_n^{″}}^2\right)+\cdots }{f_n-\frac{h}{2}{f}_n^{\prime }+\frac{h^2}{6}{f}_n^{″}+\cdots },$$

(17)

for the multidimensional ODE reads in a matrix form as

$$\Delta Y_{n+1}={\left(M_{den}\right)}^{-1}{M}_{num}F\left(Y_n\right),$$

(18)

where$M_{num}$and$M_{den}$are matrices.

Proof of Theorem 1.

Using the notation in (10) and results of Lemma 1, we rewrite (9) for the one-dimensional case as follows:

$$Y_{n+1}=Y_n+\frac{M_{num}F\left(Y_n\right)F\left(Y_n\right)}{M_{den}F\left(Y_n\right)},$$

(19)

where $M_{num}$ collects all terms by $F\left(Y_n\right)$ in the numerator and $M_{den}$ collects all terms by $F\left(Y_n\right)$ in the denominator. Equation (19) in the multidimensional case reads

$$\Delta Y_{n+1}={\left(M_{den}F\left(Y_n\right)\right)}^{-1}{M}_{num}F\left(Y_n\right)\circ F\left(Y_n\right),$$

(20)

where $\Delta Y_{n+1}=Y_{n+1}-Y_n$ and ∘ denotes the Hadamard (element-wise) multiplication. From (20), implying $\left|\right|F\left(Y_n\right)\left|\right|\ne 0$ and using the commutativity of the Hadamard product, we obtain the following the formula:

$$\Delta Y_{n+1}={\left(M_{den}\right)}^{-1}{M}_{num}F\left(Y_n\right),$$

(21)

which proves Theorem 1. □

The second-order RA method (6) in matrix form is as follows:

$$\Delta Y_{n+1}={\left(I-\frac{h}{2}J\left(Y_n\right)\right)}^{-1}hF\left(Y_n\right),$$

(22)

where I is the identity matrix, which gives the linearly implicit midpoint method, also known under the abbreviation LIMP [8].

Denote

$$\begin{array}{cc}\hfill F_1& =J,\hfill \\ \hfill F_2& =\frac{\partial J}{\partial Y}F+J^2,\hfill \\ \hfill F_3& =\left(\frac{{\partial }^{2}J}{\partial Y\partial Y}F\right)F+\frac{\partial J}{\partial Y}JF+2\left(\frac{\partial J}{\partial Y}F\right)J+J\left(\frac{\partial J}{\partial Y}F\right)+J^3.\hfill \end{array}$$

The RA method of order 4 is as follows:

$$\begin{array}{c}\hfill \Delta Y_{n+1}={\left(I-\frac{h}{2}{F}_1+\frac{h^2}{6}{F}_2-\frac{h^3}{24}{F}_3\right)}^{-1}\left(I+\frac{h}{2}\left(-\frac{1}{4}{F}_1^2+\frac{1}{3}{F}_2\right)\right)hF,\end{array}$$

(23)

Similarly, formulae for methods of higher order can be derived. From (23), one can see that one step of the fourth-order method is not that much more complicated than the fourth-order Taylor series method:

$$\begin{array}{c}\hfill \Delta Y_{n+1}=hF+\frac{h^2}{2}{F}_{1}F+\frac{h^3}{6}{F}_{2}F+\frac{h^4}{24}{F}_{3}F\end{array}$$

(24)

in terms of matrix operations except the matrix inverse, which resembles conventional implementations of implicit methods. This operation preserves the A-stability of the method, which is not inherent to implementations with element-wise vector division; see Theorem 2.

2.4. Other Issues of Implementation

One of the most complicated problems in the implementation of RA method is finding high-order partial derivatives of F since they are multidimensional matrices. Indeed,

$$J=\frac{\partial F}{\partial Y}$$

is a two-dimensional matrix, while

$$\dot{J}=\frac{\partial J}{\partial Y} \mathrm{and} \ddot{J}=\frac{{\partial }^{2}J}{\partial Y\partial Y}$$

are three-dimensional and four-dimensional matrices, respectively. However, in practical computations, we only need to compute products $\dot{J}F$ and $\left(\ddot{J}F\right)F$, which are two-dimensional matrices. Using manual or automatic matrix differentiation allows for finding and preliminarily implementing formulas for these matrices.

3. Stability Analysis

This section analyzes the stability of RA methods, proving that using a matrix inverse instead of element-wise division preserves the $A\left(\alpha \right)$-stability and plot stability regions of these methods.

First, consider the stability functions of $[p-1]/[p-1]$ RA methods. The linear test problem for stability analysis is as follows:

$$Y^{\prime }\left(t\right)=\lambda Y\left(t\right),$$

(25)

Denote $z=h\lambda$, and, since the denominator of RA methods is a Taylor series for $\Delta Y_{n-1}/h$, in terms of z,

$$Q\left(z\right)=1-\frac{z}{2}+\frac{z^2}{6}-\frac{z^3}{24}+\cdots \pm \frac{z^{p-1}}{p!}.$$

(26)

Solving the system (5) for $Q\left(z\right)$ and the problem (25), we obtain

$$P\left(z\right)=\left(\begin{array}{cccccc}1& & & & & \\ 1& 1& & & & \\ \frac{1}{2}& 1& 1& & & \\ \frac{1}{6}& \frac{1}{2}& 1& 1& & \\ & & \cdots & & \ddots & \\ \frac{1}{p!}& \cdots & \frac{1}{6}& \frac{1}{2}& 1& 1\end{array}\right)\left(\begin{array}{c}1\\ -\frac{1}{2}\\ \frac{1}{6}\\ ⋮\\ \pm \frac{1}{p!}\\ 0\end{array}\right)\circ \left(\begin{array}{c}1\\ z\\ z^2\\ ⋮\\ z^{p-1}\\ 0\end{array}\right),$$

(27)

where zero elements in the matrix are left blank. From (27), the stability functions of RA methods are

$$R\left(z\right)=\frac{1+\frac{z}{2}+\frac{z^2}{6}+\frac{z^3}{24}+\cdots +\frac{z^{p-1}}{p!}}{1-\frac{z}{2}+\frac{z^2}{6}-\frac{z^3}{24}+\cdots -\frac{z^{p-1}}{p!}}$$

(28)

for even p and

$$R\left(z\right)=\frac{1+\frac{z}{2}+\frac{z^2}{6}+\frac{z^3}{24}+\cdots -\frac{z^{p-1}}{p!}+\frac{2z^p}{(p+1)!}}{1-\frac{z}{2}+\frac{z^2}{6}-\frac{z^3}{24}+\cdots +\frac{z^{p-1}}{p!}}$$

(29)

for odd p. Even-order RA methods are A-stable for orders 2 and 4, and $A\left(\alpha \right)$-stable for the higher orders, and odd-order RA methods are not $A\left(\alpha \right)$-stable; see the stability regions of the order 2–7 RA methods in Figure 1.

The essential issue in RA method implementation is using the matrix inverse (18) instead of element-wise operations. Due to Theorem 2, this preserves $A\left(\alpha \right)$-stability.

Theorem 2.

For the$A\left(\alpha \right)$-stable rational approximation method,

$$Y_{n+1}=\mathbf{v}\left(Y_n\right)\oslash \mathbf{w}\left(Y_n\right),\mathbf{v}=PF\circ F,\mathbf{w}=QF,$$

(30)

where$\mathbf{v}$and$\mathbf{w}$are vectors, P and Q are matrices, and ⊘ is a certain binary operator:

1. 

Treating ⊘ as an element-wise division (8) may lead to the loss of $A\left(\alpha \right)$-stability for a certain $Y_n$, i.e., for a multidimensional linear test problem:

$$Y^{\prime }=AY,$$

(31)

where A is a matrix, $\exists \{Y_n,A,h\}:Y_{n+1}=R{Y}_n,max\left|\mathrm{eig}\left(R\right)\right|\ge 1$.

2. 

Treating ⊘ as a matrix inverse-based operation (18) preserves $A\left(\alpha \right)$-stability for any $Y_n$.

Proof. 

Consider the n-dimensional test problem in (31). For the method in (30), its matrix form reads:

$$Y_{n+1}=(P{Y}_n\circ Y_n)\oslash \left(Q{Y}_n\right),$$

and from (28)

$$P=1+\frac{hA}{2}+\frac{h^2{A}^2}{6}+\cdots +\frac{h^{p-1}{A}^{p-1}}{p!}$$

and

$$Q=1-\frac{hA}{2}+\frac{h^2{A}^2}{6}-\cdots -\frac{h^{p-1}{A}^{p-1}}{p!}.$$

The stability function can be found depending on the operator ⊘:

1.

If ⊘ is an element-wise division, the stability function for each dimension is found independently as

$$R_i(hA,Y_n)=\frac{u_i}{v_i},u=(P{Y}_n\circ Y_n),v=\left(Q{Y}_n\right).$$

Let us give one example of how the stability may be violated. Let $Y_n={(1100\cdots 0)}^{\top }$, and A is a matrix of a stable system in the Jordan normal form:

$$A=\left(\begin{array}{ccc}\sigma & \omega & \\ -\omega & \sigma & \\ & & \ddots \end{array}\right),$$

where $\sigma <0$ and $\omega >0$ are real numbers. In this example,

$$R_1(hA,Y_n)=\frac{1+h\frac{\sigma }{2}+h\frac{\omega }{2}+h^2\frac{{\sigma }^2-{\omega }^2}{6}+h^2\frac{2\sigma \omega }{6}+\cdots }{1-\frac{h\sigma }{2}-\frac{h\omega }{2}+h^2\frac{{\sigma }^2-{\omega }^2}{6}+h^2\frac{2\sigma \omega }{6}-\cdots }.$$

Let h be small enough to neglect terms by $h^2$ and its higher powers because 

$$R_1(hA,Y_n)=\frac{1+h\frac{\sigma }{2}+h\frac{\omega }{2}+\mathcal{O}\left(h^2\right)}{1-\frac{h\sigma }{2}-\frac{h\omega }{2}+\mathcal{O}\left(h^2\right)},$$

and denote $h\frac{\sigma }{2}=-a$, $h\frac{\omega }{2}=b$. Then,

$$R_1(hA,Y_n)=\frac{1-a+b}{1+a-b},R_2(hA,Y_n)=\frac{1-a-b}{1+a+b}.$$

The stability loss means elongation of the vector $Y_{n+1}$, i.e., $\left|\right|Y_{n+1}{\left|\right|}_2^2=R_1^2+R_2^2>\left|\right|Y_n{\left|\right|}_2^2=2$. However, for all positive a and b, the condition $\left|\right|Y_{n+1}{\left|\right||}_2^2>2$ is satisfied. This example explicitly shows that element-wise division may violate $A\left(\alpha \right)$-stability for an arbitrary stable system and initial conditions.

2.

If ⊘ is the matrix inverse, the stability function is

$$R\left(hA\right)=Q^{-1}P,$$

which does not depend on $Y_n$. Therefore, $A\left(\alpha \right)$-stability is preserved in the multidimensional case similar to the other one-step methods [1].

 □

4. Adaptive Step Control

Adaptive step control may sufficiently increase the performance of the integration program. In order to control the step size, error estimation is required. RA methods provide three possible error estimation strategies:

1.

The error term for the method of order p can be found similarly to the Taylor method:

$$er{r}_{n+1}=h^p\frac{F^{(p-1)}\left(Y_{n+1}\right)}{p!},$$

(32)

which is practically applicable since $F^{(p-1)}\left(Y_{n+1}\right)$ is already computed.

2.

The error term for the method of order p can be found from the previous order method. Let $p-1$ be odd, then the $(p-1)$-order solution is

$${\tilde{Y}}_{n+1}=Y_n+Q_{p-1}^{-1}\left(Y_n\right)h\Theta \left(Y_n\right),$$

and the p-order solution is

$$Y_{n+1}=Y_n+{\left(Q_{p-1}-h^p\frac{F^{(p-1)}\left(Y_n\right)}{p!}\right)}^{-1}\left(Y_n\right)h\Theta \left(Y_n\right),$$

since $\Theta \left(Y_n\right)$ is similar for orders p and $p-1$, from where

$$er{r}_{n+1}=Y_{n+1}-{\tilde{Y}}_{n+1}.$$

(33)

3.

The error term for the method of order p can be found using the numerator alternation technique. Let the p-order solution be

$$Y_{n+1}=Y_n+Q^{-1}\left(Y_n\right)h\Theta \left(Y_n\right),$$

then the $p-1$-order solution is

$${\tilde{Y}}_{n+1}=Y_n+Q^{-1}\left(Y_n\right)\left(h\Theta \left(Y_n\right)+c{h}^p{F}^{(p-1)}\left(Y_n\right)\right),$$

(34)

where c is a constant. We propose to use $c=\frac{1}{p!}$. The error is then found using the formula (33). The technique (34) is less computationally expensive since the matrix Q is similar for both ${\tilde{Y}}_{n+1}$ and $Y_{n+1}$, which allows for performing its $LU$-decomposition and for speeding up the second matrix inverse.

Furthermore, we used the error estimator based on Equation (34), which provided slightly better results in our preliminary tests.

In order to provide smoother step size control, we use a step control algorithm developed by Söderlind [23]:

$$h_{n+1}=\gamma h_n{\left(\frac{\xi tol}{\left|\right|er{r}_{n+1}\left|\right|}\right)}^{b_1}{\left(\frac{\xi tol}{\left|\right|er{r}_n\left|\right|}\right)}^{b_2}{\left(\frac{\xi tol}{\left|\right|er{r}_{n-1}\left|\right|}\right)}^{b_3}{\left(\frac{h_n}{h_{n-1}}\right)}^{-a_2}{\left(\frac{h_{n-1}}{h_{n-2}}\right)}^{-a_3},$$

where a and b are parameter vectors, $\gamma =0.99,\xi =0.9$ are safety factors, and $tol$ is the tolerance chosen to satisfy the condition:

$$tol=max(reltol·|\left|Y_{n+1}\right||,abstol),$$

where $reltol$ is the relative tolerance and $abstol$ is the absolute tolerance.

The best results were achieved using

$$\begin{array}{cc}\hfill b& = \left(\begin{array}{ccc}\frac{1}{4k}& \frac{1}{2k}& \frac{1}{4k}\end{array}\right),\hfill \\ \hfill a& = \left(\begin{array}{ccc}1& 0& 0\end{array}\right),\hfill \end{array}$$

where k is chosen proportional to the method order. In our practical tests, we use $k=4p$ for better damping.

5. Experimental Results

In this section, the implementation of the fourth-order rational approximation method with its third-order counterpart (34) for determining the error is considered. The resulting method is denoted as RA4(3). The order 2 RA method is identical to the widely known linearly implicit midpoint method, of which the properties have been investigated in [8]; therefore, we omit its analysis. The sixth-order method RA6(5) and the higher-order methods can be efficient only with extremely high tolerance settings, which require extended data types due to the complexity of the higher derivatives in matrix form (see Appendix A), so their properties are to be investigated in the future. Nevertheless, fourth-order methods, including the popular Runge–Kutta 4 method, are widely used in engineering and scientific computations and are among the most common methods [1].

We compare the RA4(3) methods with the Taylor 4(3) method and two Runge–Kutta methods. The first Runge–Kutta method under investigation is the explicit one-step 4(3) method described in [24]. Precisely, it is the classical Runge–Kutta 4 method with an embedded counterpart of order 3. This method is further denoted as ERK4(3). The second Runge–Kutta method is the implicit LobattoIIIC 4(3) method, which is L-stable and B-stable and was considered by some authors to be an excellent method for stiff problems [25]. Meanwhile, it is a fully implicit RK method and, therefore, a computationally expensive one. To make it more efficient in our study, we use the conventional technique of “Jacobian freezing” in the Newton method iterations; see [8].

To obtain a reference solution, we used the Dormand–Prince 8(7) method with high accuracy settings and extended data type [1].

We considered four stiff test problems in this study: the Van der Pol oscillator problem (VDPL), High Irradiance Responses (HIRES) of the photomorphogenesis problem [1], the Riccati four-dimensional equation problem (RICCATI) [26], and a modified FitzHugh–Nagumo–Rinzel neuron model (FNR) from [27]. Our first example shows how different methods deal with the step size and how numerical differentiation affects step size dynamics. The second example studies the performance of the methods.

5.1. Time Domain Analysis and Issues of Step Control

This subsection considers several test problems and how different methods behave with step-size controllers on them. The Van der Pol (VDPL) test problem is given by the following equations:

$$\begin{array}{c}\hfill \left\{\begin{array}{cc}\dot{y_1}\hfill & =y_2,\hfill \\ \dot{y_2}\hfill & =\mu (1-y_1^2)y_2-y_1,\hfill \end{array}\right.\end{array}$$

(35)

where $\mu$ is the nonlinearity parameter: the higher its value, the stiffer the problem. In our experiments, we use the value $\mu =1000$ and experiment time $t_{max}=2000$. We used the following settings: $h_{min}={10}^{-10},h_{max}=10,abstol={10}^{-11},reltol={10}^{-8}$. Figure 2 depicts the obtained results in the time domain.

The time-domain solution is given on a logarithmic scale of its absolute value. The plot at the bottom shows that the largest steps are selected by the L-stable Lobatto IIIC method. The RA method generally follows it, while the step size strategies of the Taylor and ERK methods are quite similar.

Figure 3 shows the difference in step selection strategies between the method with element-wise division (8) and the method with matrix inverse. This example compromises the element-wise division applied to the RA method. The element-wise division in the Van der Pol oscillator leads to a notable accuracy loss (see upper plot) and reduces the efficiency of the method because it becomes unable to select large step sizes in the same way that a truly implicit method can. The Taylor method stepping strategy is given for comparison.

The stiff Riccati equation test problem (RICCATI) is described by the following equations:

$$\left\{\begin{array}{c}\dot{y_1}=\mathrm{10,000}-y_1^2-y_3{y}_2,\hfill \\ \dot{y_2}=-y_2(y_1+y_4),\hfill \\ \dot{y_3}=-y_3(y_1+y_4),\hfill \\ \dot{y_4}=\mathrm{10,000}-y_4^2-y_3{y}_2,\hfill \end{array}\right.$$

(36)

with initial value $y\left(t_0\right)={\left(0010\right)}^{\top }$ and simulation time $t_{max}=3$. The solver settings used for time-domain test were $h_{min}={10}^{-10},h_{max}=100,abstol={10}^{-10},reltol={10}^{-5}$, Figure 4 depicts the obtained results in the time domain.

The High Irradiance Responses test problem (HIRES) is described by the following equations:

$$\left\{\begin{array}{c}\dot{y_1}=-1.71y_1+0.43y_2+8.32y_3+0.0007,\hfill \\ \dot{y_2}=1.71y_1-8.75y_2,\hfill \\ \dot{y_3}=-10.03y_3+0.43y_4+0.035y_5,\hfill \\ \dot{y_4}=8.32y_2+1.71y_3-1.12y_4,\hfill \\ \dot{y_5}=-1.745y_5+0.43y_6+0.43y_7,\hfill \\ \dot{y_6}=-280y_6{y}_8+0.69y_4+1.71y_5-0.43y_6+0.69y_7,\hfill \\ \dot{y_7}=280y_6{y}_8-1.81y_7,\hfill \\ \dot{y_8}=-280y_6{y}_8+1.81y_7,\hfill \end{array}\right.$$

(37)

with initial value $y\left(t_0\right)={\left(10000000.0057\right)}^{\top }$ and simulation time $t_{max}=100$. The solver settings were $h_{min}={10}^{-10},h_{max}=100,abstol={10}^{-10},reltol={10}^{-5}$, Figure 5 depicts the obtained results in the time domain.

The sine-excited FitzHugh–Nagumo–Rinzel neuron test problem (FHN) is described by the following equations:

$$\left\{\begin{array}{c}\dot{y_1}=y_2-y_1^3/3-y_2-y_3+y_5,\hfill \\ \dot{y_2}=a_1(a_2+y_1-a_3{y}_2),\hfill \\ \dot{y_3}=a_4(a_5-y_3-y_1),\hfill \\ \dot{y_4}=A{y}_5,\hfill \\ \dot{y_5}=-{\omega }^2{y}_4,\hfill \end{array}\right.$$

(38)

where the parameter vector is $a={(0.50.70.80.002-1)}^{\top }$, the sine perturbation function amplitude is $A=0.083$, the radial frequency is $\omega =0.00003·2\pi$, the initial value is $y\left(t_0\right)={\left(00001\right)}^{\top }$, and the simulation time is $t_{max}$ = 70,000.

The solver settings were $h_{min}={10}^{-10},h_{max}=100,abstol={10}^{-10},reltol={10}^{-8}$, Figure 6 shows the behavior of the obtained solution.

In all examples, the LobattoIIIC method can work with the largest step sizes, while the Taylor and ERK solvers select small step sizes, restricted by stability requirements.

5.2. Performance Analysis

In this subsection, we study the performance of the methods on four described problems. The results are given in Figure 7. Four considered test problems highlighted the higher performance of the proposed RA method. Explicit methods are poorly suited for this set of problems, as seen from the plots: their computational time is almost independent of accuracy. Thus, their step controllers restrict step sizes on all investigated tolerances to maintain the stability.

All of the experiments were carried out in a MATLAB R2018b environment, on a computer with the 64-bit Win10 operational system, Intel Core i5-7300HQ 2.5 GHz CPU, and 16 GB RAM. The relative tolerance value was altered within the interval $[{10}^{-9},{10}^{-1}]$, and the absolute tolerance was chosen as $abstol={10}^{-5}·reltol$. The double data type was used.

6. Conclusions and Discussion

In this paper, we proposed a prospective type of rational approximation method for solving stiff ODEs. This method has a strong relation to the Taylor expansion and is easy to derive and implement. The A- and $A\left(\alpha \right)$-stability of this method is proven, and most notably, the method is A-stable in the case of orders 2 and 4. Compared with the Taylor method, and explicit and implicit Runge–Kutta methods, it has the best performance when solving stiff test problems, as our experiments with 4(3) order methods on double data type revealed.

Similar to the Taylor series methods, the proposed method requires algebraic differentiation to achieve the best performance. It is of great interest to test the performance of the proposed method on extended data types and high orders, which will be the topic of our future work.

The question arises of whether we found the “best” rational approximation for numerical integration or whether there exist other types of approximation that can achieve better performance. It is also of interest to derive a Runge–Kutta-like formula for this method, especially for the matrix denominator, which may considerably increase the method performance.

The proposed numerical method is also prospective for solving problems with singularities, especially poles in the denominator of the derivative function.

RA methods are potentially suitable for solving DAEs, which will be a subject of our future investigation.