Enhancing Accuracy of Runge–Kutta-Type Collocation Methods for Solving ODEs

Abstract

In this paper, a new class of Runge–Kutta-type collocation methods for the numerical integration of ordinary differential equations (ODEs) is presented. Its derivation is based on the integral form of the differential equation. The approach enables enhancing the accuracy of the established collocation Runge–Kutta methods while retaining the same number of stages. We demonstrate that, with the proposed approach, the Gauss–Legendre and Lobatto IIIA methods can be derived and that their accuracy can be improved for the same number of method coefficients. We expressed the methods in the form of tables similar to Butcher tableaus. The performance of the new methods is investigated on some well-known stiff, oscillatory, and nonlinear ODEs from the literature.

1. Introduction

A special type of initial value problems, treated in this work, involves highly oscillatory systems and problems known as stiff systems. In the field of numerical analysis, scientific computing, and engineering, many such problems exhibit these properties [1,2,3]. Ordinary differential equations (ODEs) describing these systems are ill-conditioned in a computational sense and challenging to solve [1]. For such problems, implicit Runge–Kutta (RK) methods are a convenient and frequent choice. The methods gained prominence after Butcher showed that the s-stage fully implicit Gaussian RK method of order 2s is A-stable for all s [4].

A subset of implicit RK methods involves collocation methods. They yield a continuous approximation which makes them suitable for problems where globally continuously differentiable functions are required [5]. Despite being known for decades, the methods are still under continuous development. Publications examine the application of the methods to new problems, the derivation of new strategies, or the development of new procedures for solving nonlinear equations [3]. Three basic types of implicit RK methods, each based on different Gauss–Legendre-type quadrature formulae, are Gauss–Legendre quadrature formulae, Gauss–Radau quadrature formulae, and Gauss–Lobatto quadrature formulae [6].

Perez-Rodriguez et al. [7] applied an implicit RK method of Radau IIA type to semi-discretized, multidimensional partial differential equations (PDEs) of advection–diffusion–reaction type. To mitigate the stiffness of these systems, a special iteration technique was applied to solve the resulting implicit relations encountered in the integration steps. Ramos and Garcia-Rubio [8] constructed a new family of implicit methods on the basis of the s-degree Chebyshev approximation using the Chebyshev–Gauss–Lobatto points. Similarly, Vigo-Aguiar and Ramos [9], in addition to constructing an A-stable iRK method, proposed an embedding technique for changing the step size using the Runge–Kutta method and a linear multistep one. Teh and Yaacob [10] recently developed several implicit RK methods, two of them being Gauss–Kronrod methods with five stages and order 8 accuracy. These are based on five-point Gauss–Kronrod quadrature formula. They also developed four implicit RK methods on the basis of Gauss–Kronrod–Radau quadrature formulae [11] and four seven-stage 10th-order implicit RK methods using seven-point Gauss–Kronrod–Lobatto quadrature formula [6]. Liu and Sun [12] studied the Lobatto III-type RK methods. They extended the existing Lobatto methods using the W-transformation and constructed general Lobatto III methods with three parameters of stages two to four. Implicit RK methods play a vital role in solving systems of differential algebraic equations (DAEs) [13]. For example, Jay [14] analyzed the application of RK methods to DAEs of index-2 in Hessenberg form and (see [15]) applied the s-stage Lobatto IIIA–Lobatto IIIB method as a partitioned RK method to integrate Hamiltonian systems with holonomic constraints. A particularly important type of implicit RK methods for stiff and DAE problems involves the Radau IIA methods, implemented in the well-known code for stiff and DAE systems, RADAU5 [16].

In this work, we present a new class of collocation methods for numerical integration of ODEs by basing the derivation on their integral form. The approach resembles the well-known Galerkin method, which is based on the so-called weak formulation of the governing differential equations; nevertheless, we could not find any references involving applying the presented idea to initial value problems and deriving collocation methods. With the approach, we present how the accuracy of RK-type collocation methods (more specifically, the Gauss–Legendre and Lobatto IIIA method) can be increased while retaining the number of method coefficients. In addition to proposing some method instances of the new class of methods, we also demonstrate how the Gauss–Legendre and Lobatto IIIA methods can also be derived with the proposed approach.

In this paper, some theoretical preliminaries are given first (Section 2), followed by a general derivation of the method (Section 3). Particular method families are then presented in Section 4 where their accuracy and linear stability are also investigated. The performance of methods is analyzed using several numerical examples in Section 5. The discussion and conclusion then follow in Section 6 and Section 7.

2. Theoretical Preliminaries

Consider the following initial value problem:

$$y^{\prime }\left(x\right)=f\left(x,y\left(x\right)\right),$$

(1)

with $y\left(x_n\right)=y_0$, $y\in {\mathbb{R}}^d$, for which we seek an approximate solution in the interval $\left[x_n,x_n+h\right]$, with $h$ as the step size. Collocation methods utilize a polynomial, say $u$, of degree $s$ to approximate the solution $y$. The polynomial needs to satisfy the initial condition $u\left(x_n\right)=y_0$ and the differential equation $u^{\prime }\left(x_i\right)=k_i=f\left(x_i,u\left(x_i\right)\right)$ at collocation points $x_i=x_n+c_{i}h$ for $i=1,\dots ,s$, with $0\le c_1<c_2<\cdots <c_s\le 1$. The numerical solution of the collocation method is defined by $y_1=u\left(x_n+h\right)$.

The general class of RK methods is defined as follows [17]:

$$y_1=y_0+h{{\displaystyle \sum }}_{i=1}^s{k}_i{b}_i,k_i=f\left(x_n+c_{i}h,y_0+h{{\displaystyle \sum }}_{j=1}^r{k}_j{a}_{ij}\right),$$

(2)

where a fully implicit RK method is obtained when the upper limit r in the sum defining the $k_i$ is equal to s. The collocation method is equivalent to the s-stage RK method, Equation (2), with the following coefficients:

$$a_{ij}={{\displaystyle \int }}_0^{c_i}{l}_j\left(\tau \right)\mathrm{d}\tau ,b_i={{\displaystyle \int }}_0^1{l}_j\left(\tau \right)\mathrm{d}\tau ,$$

(3)

where $l_j\left(\tau \right)$ is the Lagrange polynomial $l_j\left(\tau \right)={\prod }_{l\ne j}\left(\tau -c_l\right)/\left(c_j-c_l\right),j=1,\dots ,s$ [18]. The choice of coefficients $a_{ij}$ and $b_i$, and points $c_1,\dots ,c_s$ defines a method instance. This work limits the arbitrary choice of $c_i$ points to either the roots of Legendre polynomials or a combination of Legendre polynomials of different stages. RK method subclasses based on these collocation points are Gauss–Legendre, Radau, and Lobatto [19]. The Gauss–Legendre quadrature employs a shifted Legendre polynomial of degree $s$ on the interval $\left[0,1\right]$.

$$P_s^{*}\left(\tau \right)=\frac{{\mathrm{d}}^s}{\mathrm{d}{\tau }^s}\left({\tau }^s{\left(\tau -1\right)}^s\right).$$

(4)

The method subclass accuracy is of order 2$s$ and its stability function is the diagonal ($s$,$s$)-Padé approximation [20]. The “Radau I quadrature formula” defines $c_i$ as the zeros of a polynomial $P_s^{*}\left(\tau \right)+P_{s-1}^{*}\left(\tau \right)$, yielding $c_1=0$, while the “Radau II quadrature formula” yields $c_s=1$ on the basis of the polynomial $P_s^{*}\left(\tau \right)-P_{s-1}^{*}\left(\tau \right)$. The “Lobatto III quadrature formula” with both $c_1=0$ and $c_s=1$ originates from the polynomial $P_s^{*}\left(\tau \right)-P_{s-2}^{*}\left(\tau \right)$. The accuracy of the $s$-stage Radau IA and Radau IIA methods is of order $2s-1$, and their stability function is the ($s-1$,$s$) subdiagonal Padé approximation. The accuracy of the Lobatto III class of method is of order $2s-2$, with the stability function being the diagonal ($s-1$,$s-1$) Padé approximation for Lobato IIIA and IIIB and ($s-2$,$s$) for Lobatto IIIC [20].

In contrast to classical collocation methods, the novel method class is derived from the integral form of Equation (1).

$${{\displaystyle \int }}_{x_n}^{x_n+h}{y}^{\prime }\left(x\right)v\left(x\right)\mathrm{d}x={{\displaystyle \int }}_{x_n}^{x_n+h}f\left(x,y\left(x\right)\right)v\left(x\right)\mathrm{d}x,$$

(5)

where $v\left(x\right)$ is the test function, arbitrary in general. The usage of the integral approach is, for instance, widely employed in finite element formulation. Its property is a functional approximation of the observed (primary) variable which needs to satisfy the governing (partial) differential equation in an integral fashion. The approximation function, denoted as $u^{\prime }$, is assumed to approximate the solution of the differential equation $y^{\prime }$. The system of equations determining the approximation polynomial $u^{\prime }$ is constructed by substituting different test functions $v\left(x\right)$ in Equation (5) and imposing the initial condition.

A new class of methods follows from Equation (5) by, firstly, employing different approximation polynomial functions for $u^{\prime }$. Functions are defined on the basis of roots of the Legendre polynomial $P_s^{*}\left(\tau \right)$ or its combinations (recall: $P_s^{*}\left(\tau \right)+P_{s-1}^{*}\left(\tau \right)$, $P_s^{*}\left(\tau \right)-P_{s-1}^{*}\left(\tau \right)$ or $P_s^{*}\left(\tau \right)-P_{s-2}^{*}\left(\tau \right)$). The choice of $u^{\prime }$ then determines the quadrature rule in integrating the left-hand side of Equation (5). Secondly, different quadrature formulae are employed for the integration of the right-hand side of Equation (5). This integration can be conducted by any quadrature rule (Gauss–Legendre, Radau I, Radau II, or Lobatto), independent of the choice for the left-hand side.

3. General Derivation of the Method

Let us introduce in Equation (5) a new dimensionless variable $\tau \in \left[0,1\right]$, where $x=x_n+\tau h$. Equation (5) now yields

$${{\displaystyle \int }}_0^1{y}^{\prime }\left(x_n+\tau h\right)v\left(\tau \right)\mathrm{d}\tau ={{\displaystyle \int }}_0^{1}f\left(x_n+\tau h,y\left(x_n+\tau h\right)\right)v\left(\tau \right)\mathrm{d}\tau .$$

(6)

For the approximate solution of $y^{\prime }$, we consider a collocation polynomial $u^{\prime }$ of stage $s$.

$$y^{\prime }\left(x_n+\tau h\right)\approx u^{\prime }\left(x_n+\tau h\right)={{\displaystyle \sum }}_{j=1}^s{k}_j{l}_j\left(\tau \right),$$

(7)

where $k_j$ represents discrete values of $u^{\prime }\left(x_n+\tau h\right)$ at collocation points $c_j$, $k_j=u^{\prime }\left(x_n+c_{j}h\right)$, and $l_j\left(\tau \right)$ represents the Lagrange polynomials.

The collocation points $c_j$, $j=1,\dots ,s$ can generally be chosen arbitrarily. As discussed in the preliminaries, this work limits their choice to the roots of $P_s^{*}\left(\tau \right)$ or any of its combinations. We continue without setting their numerical values until the next section where certain options are investigated.

By considering Equation (6) in Equation (5), we obtain

$${{\displaystyle \int }}_0^1{{\displaystyle \sum }}_{j=1}^s{k}_j{l}_j\left(\tau \right)v\left(\tau \right)\mathrm{d}\tau ={{\displaystyle \int }}_0^{1}f\left(x_n+\tau h,y_0+h{{\displaystyle \sum }}_{j=1}^s{k}_j{{\displaystyle \int }}_0^{\tau }{l}_j\left(\tau \right)\mathrm{d}\tau \right)v\left(\tau \right)\mathrm{d}\tau ,$$

(8)

substituting the integrated approximation function.

$$u\left(x_n+\tau h\right)=y_0+h{{\displaystyle \sum }}_{j=1}^s{k}_j{{\displaystyle \int }}_0^{\tau }{l}_j\left(\tau \right)\mathrm{d}\tau .$$

(9)

Unknowns in Equation (8) are $k_j$, $j=1,\dots ,s$, the discrete values of the approximation function $u^{\prime }$. Equations that determine the coefficients are generated by substituting different test functions in Equation (8). For the test function, we choose the Lagrange polynomials of degree $s$ (as is common in finite element method).

$$v_i\left(\tau \right)={\displaystyle \prod }_{l\ne i}\frac{\tau -{\zeta }_l}{{\zeta }_i-{\zeta }_l},i=1,\dots ,s,$$

(10)

where $s$ denotes the number of equations instantiated from Equation (8). Points ${\zeta }_i$ in Equation (10) are chosen as the roots of the Lobatto polynomial $P_s^{*}\left(\tau \right)-P_{s-2}^{*}\left(\tau \right)$, for the property ${\zeta }_1=0,{\zeta }_s=1$.

The left-hand side of Equation (8) assumes the form of the standard quadrature formula,

$${{\displaystyle \sum }}_{j=1}^s{k}_j{p}_{ij}\mathrm{where}{p}_{ij}={{\displaystyle \int }}_0^1{l}_j\left(\tau \right)v_i\left(\tau \right)\mathrm{d}\tau ,i=1,\dots ,s,$$

(11)

with $l_j\left(\tau \right)$ and $v_i\left(\tau \right)$ defined by Equations (7) and (10), respectively.

The right-hand side of Equation (8) is approximated as

$$f\left(x_n+\tau h,u\left(x_n+\tau h\right)\right)\approx {{\displaystyle \sum }}_{j=1}^{\hat{s}}{\hat{F}}_j{\hat{l}}_j\left(\tau \right),$$

(12)

where ${\hat{F}}_j$ represents discrete values of $f\left(x_n+\tau h,u\left(x_n+\tau h\right)\right)$ at collocation points ${\hat{c}}_j$, ${\hat{F}}_j=f\left(x_n+{\hat{c}}_{j}h,u\left(x_n+{\hat{c}}_{j}h\right)\right)$. Functions ${\hat{l}}_j\left(\tau \right)$ are the Lagrange polynomials.

$${\hat{l}}_j\left(\tau \right)={\displaystyle \prod }_{l\ne j}\frac{\tau -{\hat{c}}_l}{{\hat{c}}_j-{\hat{c}}_l},j=1,\dots ,\hat{s},$$

(13)

where collocation points are ${\hat{c}}_j$, $j=1,\dots ,\hat{s}$. As in the case of collocation points $c_j$, the numerical values of ${\hat{c}}_j$ are arbitrary in general and there is no requirement for ${\hat{c}}_j=c_j$. In this work, we limit their values to the roots of $P_{\hat{s}}^{*}\left(\tau \right)$ and some combinations, which we analyze in the next section.

The right-hand side of Equation (8) can thus be expressed as

$${{\displaystyle \int }}_0^1{{\displaystyle \sum }}_{j=1}^{\hat{s}}{\hat{F}}_j{\hat{l}}_j\left(\tau \right)v\left(\tau \right)\mathrm{d}\tau ={{\displaystyle \int }}_0^1{{\displaystyle \sum }}_{j=1}^{\hat{s}}f\left(x_n+{\hat{c}}_{j}h,y_0+h{{\displaystyle \sum }}_{m=1}^s{k}_m{{\displaystyle \int }}_0^{{\hat{c}}_j}{l}_m\left(\tau \right)\mathrm{d}\tau \right){\hat{l}}_j\left(\tau \right)v\left(\tau \right)\mathrm{d}\tau .$$

(14)

By employing Equations (6) and (10), the right-hand side of Equation (8) finally becomes

$$\begin{gathered} {{\displaystyle \sum }}_{j=1}^{\hat{s}}f\left(x_n+{\hat{c}}_{j}h,y_0+h{{\displaystyle \sum }}_{m=1}^s{k}_m{a}_{jm}\right)q_{ij},i=1,\dots ,s\mathrm{where} \\ {q}_{ij}={{\displaystyle \int }}_0^1{\hat{l}}_j\left(\tau \right)v_i\left(\tau \right)\mathrm{d}\tau ,a_{jm}={{\displaystyle \int }}_0^{{\hat{c}}_j}{l}_m\left(\tau \right)\mathrm{d}\tau ,i=1,\dots ,s,j=1,\dots ,\hat{s}, \end{gathered}$$

(15)

with ${\hat{l}}_j\left(\tau \right)$ and $v_i\left(\tau \right)$ defined by Equations (13) and (10), respectively.

The final form of Equation (8) is given as

$${{\displaystyle \sum }}_{j=1}^s{k}_j{p}_{ij}={{\displaystyle \sum }}_{j=1}^{\hat{s}}f\left(x_n+{\hat{c}}_{j}h,y_0+h{{\displaystyle \sum }}_{m=1}^s{k}_m{a}_{jm}\right)q_{ij},i=1,\dots ,s,$$

(16)

with the coefficients specifying a method instance summarized as follows:

$$\begin{array}{l}{p}_{ij}={{\displaystyle \int }}_0^1{l}_j\left(\tau \right)v_i\left(\tau \right)\mathrm{d}\tau ,i=j=1,\dots ,s\\ q_{ij}={{\displaystyle \int }}_0^1{\hat{l}}_j\left(\tau \right)v_i\left(\tau \right)\mathrm{d}\tau ,i=1,\dots ,s;j=1,\dots ,\hat{s}\\ a_{jm}={{\displaystyle \int }}_0^{{\hat{c}}_j}{l}_m\left(\tau \right)\mathrm{d}\tau ,j=1,\dots ,\hat{s};m=1,\dots ,s\\ b_j={{\displaystyle \int }}_0^1{l}_j\left(\tau \right)\mathrm{d}\tau ,j=1,\dots ,s\end{array}\}.$$

(17)

Equation (16) determines the discrete values of the approximation function $k_j$, $j=1,\dots ,s$ and can be linear or nonlinear. The end-point of the interval $y\left(x_n+h\right)$ is approximated as

$$y\left(x_n+h\right)\approx u\left(x_n+h\right)=y_0+h{{\displaystyle \sum }}_{j=1}^s{k}_j{{\displaystyle \int }}_0^1{l}_j\left(\tau \right)\mathrm{d}\tau =y_0+h{{\displaystyle \sum }}_{j=1}^s{k}_j{b}_j,$$

(18)

with coefficients $b_j$ defined with the last line in Equation (17). The numerical values of the coefficients summarized in Equation (17) are found after specifying the collocation points $c_j$, $j=1,\dots ,s$, and ${\hat{c}}_j$, $j=1,\dots ,\hat{s}$ (Equations (7) and (13), respectively).

4. Particular Method Families: Their Accuracy and Stability

The behavior of a method instance is determined by the choice of the collocation points $c_j$, $j=1,\dots ,s$, and ${\hat{c}}_j$, $j=1,\dots ,\hat{s}$, arbitrary in general. In this work, we investigate the option of selecting the Gauss–Legendre and Lobatto points. We denote method instances according to the quadrature formula used, e.g., $\mathrm{G}s|\mathrm{G}\hat{s}$ employs $s$-stage Gauss points for $c_j$, $j=1,\dots ,s$, and $\hat{s}$-stage Gauss points for ${\hat{c}}_j$, $j=1,\dots ,\hat{s}$.

The coefficients of a method instance, Equation (17), are listed in the manner of Butcher [21] as follows:

$$\begin{array}{ll}{\left[p\right]}_{s\times s}& {\left[q\right]}_{s\times \hat{s}}\\ {\left[a\right]}_{\hat{s}\times s}& \\ {\left\{b\right\}}_s& {\left\{\hat{c}\right\}}_{\hat{s}}\end{array}$$

Method families that we introduce in this work are presented in

Table 1. The proposed method families.

Method Family	Polynomials Defining the Collocation Points 1:
	${\mathit{c}}_{\mathit{j}}$$,\mathit{j}=1,\dots ,\mathit{s}$	${\hat{\mathit{c}}}_{\mathit{j}}$$,\mathit{j}=1,\dots ,\hat{\mathit{s}}$
Gs|Gs	$P_s^{*}\left(\tau \right)$	$P_{\hat{s}}^{*}\left(\tau \right)$$;\hat{s}=s$
Gs|Gs+1	$P_s^{*}\left(\tau \right)$	$P_{\hat{s}}^{*}\left(\tau \right)$$;\hat{s}=s+1$
Ls|Ls	$P_s^{*}\left(\tau \right)-P_{s-2}^{*}\left(\tau \right)$	$P_s^{*}\left(\tau \right)-P_{s-2}^{*}\left(\tau \right)$$;\hat{s}=s$
Ls|Ls+1	$P_s^{*}\left(\tau \right)-P_{s-2}^{*}\left(\tau \right)$	$P_s^{*}\left(\tau \right)-P_{s-2}^{*}\left(\tau \right)$$;\hat{s}=s+1$
Ls|Gs+1	$P_s^{*}\left(\tau \right)-P_{s-2}^{*}\left(\tau \right)$	$P_{\hat{s}}^{*}\left(\tau \right)$$;\hat{s}=s+1$
eLs|Gs	$P_s^{*}\left(\tau \right)-P_{s-2}^{*}\left(\tau \right)$	$P_{\hat{s}}^{*}\left(\tau \right)$$;\hat{s}=s$
eLs|Gs+1	$P_s^{*}\left(\tau \right)-P_{s-2}^{*}\left(\tau \right)$	$P_{\hat{s}}^{*}\left(\tau \right)$$;\hat{s}=s+1$

¹ Collocation points coincide with the roots of the listed polynomials, where $P_s^{*}\left(\tau \right)$ is the shifted Legendre polynomial of degree s, Equation (3).

. Methods are chosen due to their favorable accuracy (e.g., Gs|Gs+1 and Ls|Gs+1) or to demonstrate the option of enhanced integration accuracy of the right-hand side of the ODEs (for instance, by comparing methods Gs|Gs and Gs|Gs+1 or methods Ls|Ls and Ls|Ls+1). Method families of interest are also eLs|Gs and eLs|Gs+1. Their definition only differs in the computation of the first coefficient $k_1$, i.e., explicitly by using the initial condition $y_0$. Their labels include an “e” before the quadrature rule designation.

The coefficient arrays of the methods in Table 1, obtained for s = 2, 3, 4, are reported in the Supplementary Materials (in the form of analytical expressions), whereas Appendix A provides tableaus for methods Gs|Gs, Ls|Ls, and Ls|Ls+1. For convenience in practical usage, we provide the numerical values of coefficients in Appendix B. In Section 4.1, we prove that the Gs|Gs, and Ls|Ls method families are identical to the Gauss–Legendre and the Lobatto IIIA methods, respectively. The accuracy and linear stability of the methods are analyzed in Section 4.2.

4.1. Gauss–Legendre and Lobatto IIIA Method Families as Subclasses of the Novel Method Class

Let us show that (a) the Gs|Gs method family is identical to the Gauss–Legendre method and (b) the Ls|Ls method family is identical to the Lobatto IIIA method.

Proof. 

For $c_j={\hat{c}}_j$, $j=1,\dots ,s$, ($s=\hat{s}$), the coefficients $p_{ij}=q_{ij}$ in Equation (16) can be factored out, yielding

$${\displaystyle {\sum }_{j=1}^s}\left(k_j-f\left(x_n+c_{j}h,y_0+h{\displaystyle {\sum }_{m=1}^s}{k}_m{a}_{jm}\right)\right)p_{ij}=0,i=1,\dots ,s,$$

(19)

exploiting $q_{ij}=p_{ij}$, $i=j=1,\dots ,s$. The system of Equation (19) is equivalent to the classical RK system in Equation (2) if and only if the coefficients $a_{jm}$ and $c_j$ in Equation (19) are identical to $a_{ij}$ and $c_j$ of a RK method, and the matrix $P=\left[p_{ij}\right]$ is invertible.

Statement (a) is verified by reviewing the definition of the $a_{jm}$ coefficients (see Equation (17)). These can be defined in the exact same manner in the Gauss–Legendre and Lobatto IIIA methods (see Equation (2)) [18]. To verify statement (b), review the definition of $p_{ij}$ in Equation (17). In this case, the matrix $P$ is a square matrix of size $s\times s$. The nonzero determinant is a consequence of linearly independent equations (i.e., matrix rows), generated from test functions $v_i$. These are, specifically, the Lagrange polynomials of degree $s$, Equation (10), with points ${\zeta }_i$ as the roots of the Legendre polynomial. This is a sufficient condition for the matrix $P$ to be invertible.

4.2. Accuracy and Linear Stability

The accuracy of the methods is investigated by integrating the following two test equations:

$$\begin{gathered} \mathrm{Test}\mathrm{equation}\mathrm{A}\{\begin{array}{l}{y}^{\prime }\left(x\right)=\lambda y\left(x\right),y\left(x_n\right)=y_0,\\ y_e\left(x\right)=y_0{e}^{x\lambda }\end{array} \\ \mathrm{Test}\mathrm{equation}\mathrm{B}\{\begin{array}{l}{y}^{\prime }\left(x\right)=\lambda \cos \left(x\lambda \right)+x,y\left(x_n\right)=y_0,\\ y_e\left(x\right)=y_0+\sin \left(x\lambda \right)+x^2/2\end{array} \end{gathered}$$

(20)

where $y_e\left(x\right)$ is the exact solution. Equation A is a standard test equation for investigating the accuracy and linear stability of methods. Test equation B is proposed to demonstrate the impact of using different quadrature rules for the integration of the right-hand side of ODEs.

The accuracies are determined by applying the methods to both test equations. The approximate and the exact solutions are expanded into a Taylor series about $x_n$ and subtracted at $x_n+h$, yielding an error of the approximate solution. The stability analysis was performed using test equation A according to the established procedure. The same procedure was conducted for the Gauss–Legendre, Lobatto IIIA, and Lobatto IIIC method [13].

The results of accuracy and stability analysis are given in

Table 2. The accuracy orders of the proposed methods, Lobatto IIIA, Lobatto IIIC [22], and Gauss–Legendre [21] methods when applied to the test equations in Equation (20). Linear stability (integrating test equation A in Equation (20)) is reported in the form of Padé approximation [20].

Method		Ls|Ls 1 Gs|Ls	Gs|Gs 2 Ls|Gs Gs|Ls+1 Ls|Ls+1	Gs|Gs+1 Ls|Gs+1	Lobatto IIIC	eLs|Gs	eLs|Gs+1
No. of coefficients to be determined		s	s	s	s	s − 1	s − 1
Order of accuracy	Test equation A	2s − 2	2s	2s	2s − 2	2s − 1	2s − 1
	Test equation B	2s − 2	2s	2s + 2	2s − 2	2s	2s + 2
Linear stability provided as Padé approximation [20]		(s − 1, s − 1)	(s, s)	(s, s)	(s − 2, s)	(s, s − 1)

¹ Lobatto IIIA method. ² Gauss–Legendre method.

. The methods were grouped as follows by matching behavior:

• Ls|Ls and Gs|Ls (first column in the table);
• Gs|Gs, Ls|Gs, Gs|Ls+1, and Ls|Ls+1 (the second column);
• Gs|Gs+1 and Ls|Gs+1 (the third column).

The results suggest that the behavior of a method is primarily determined by the integration rule of the right-hand side of the ODE, and that taking Gauss points of $s$ stages is equivalent to using Lobatto points with $s+1$ stages.

The effect of using a higher integration rule for the right-hand side of ODEs is evident upon comparing the accuracy of Ls|Ls vs. Ls|Ls+1, Gs|Ls vs. Gs|Ls+1, and Gs|Gs vs. Gs|Gs+1 methods (Table 2). Their accuracy was improved by merely enhancing the integration of the right-hand side of the ODE while retaining the number of coefficients. This improvement in accuracy cannot be general since it was not observed in Ls|Gs vs. Ls|Gs+1, and Gs|Gs vs. Gs|Gs+1 for test equation A (Equation (20)).

The stability of the methods denoted with “e” (where the first coefficient is computed explicitly, eLs|Gs and eLs|Gs+1 in Table 2), is reduced to a (s, s − 1)-Padé approximation. This is the cost of having a smaller system of equations for computing the method’s coefficients, having the size of s − 1 × s − 1. Unlike the other proposed method families, “e”-methods are not A-stable. However, an interesting feature of these methods is their accuracy. Their accuracy order was 2s − 1 in test equation A but increased to 2s for eLs|Gs and even 2s + 2 for eLs|Gs+1 in test equation B.

5. Numerical Examples

We applied the novel and the established methods to several test problems and compared the results, finding that the novel methods offer either comparable or significantly improved performance. In this work, four examples are presented. Example 1 is proposed by the authors, while the remaining examples are well-known stiff and/or highly nonlinear test problems from the literature. In Example 1 (dynamic problem of a mass–spring system), we investigate the application of the methods to problems involving high-frequency oscillations. The behavior of the method with respect to the high initial stiffness of the system was analyzed in Example 2. In Example 3, the methods were examined for problems where the Lobatto IIIC methods might pose stability issues, and, in Example 4, the methods were investigated for a highly nonlinear system.

The examples are integrated by Gs|Gs, Gs|Gs+1, Ls|Ls+1, Ls|Gs, and Ls|Gs+1 for stages $s=3,4$, which have orders of accuracy 4 and 6, respectively (Table 2). For comparison, we also analyzed the examples with the Lobatto IIIC method of the same orders of accuracy, 4 and 6.

Analyses were performed in the fixed step size environment for different step sizes $h$. The approximations were compared to the exact solutions by computing the error as the discrete $L_2$ norm, defined for the variable $y$ as

$$N_{2}y=\sqrt{{\displaystyle {\sum }_{i=1}^N}{\left[y_e\left(x_i\right)-y\left(x_i\right)\right]}^2},$$

(21)

where $y$ and $y_e$ are the approximate and the exact solutions, respectively. $N$ is the number of intervals and $x_i$ represents the discrete values of the independent variable. All computations were conducted in Wolfram Mathematica 11. Nonlinear systems of equations were solved using the Newton–Raphson method with relative error tolerance 10⁻¹⁵.

5.1. Example 1

Mass–spring system:

$$\{\begin{array}{l}{y}_1^{\prime }=y_2,y_1\left(0\right)=1.1,\\ y_2^{\prime }=10-100y_1-10000\sin \left(20\pi x\right),y_2\left(0\right)=1\\ x\in \left[0,5\right]\end{array},$$

(22)

with the exact solution for $y_1$ (representing the displacement of the mass) being

$$\begin{array}{ll}{y}_1\left(x\right)=& (-1+4{\pi }^2+10\left(-1+4{\pi }^2\right)\cos \left(10x\right)\\ & +\left(-1+4\left(-500+\pi \right)\pi \right)\sin \left(10x\right)+1000\sin \left(20\pi x\right))/\left(40{\pi }^2-10\right).\end{array}$$

The differential equations describe the dynamic behavior of a mass–spring system. The mass is subject to a harmonic excitation and is oscillating about its static equilibrium. The particular excitation used adds high-frequency oscillations to the natural response of the mass, requiring a small step size and making the problem computationally demanding. The approximation errors of different methods for the $y_1$ variable are reported in Figure 1 with squares and circles for methods of order 4 and 6, respectively. In the same manner, the results for methods eLs|Gs and eLs|Gs+1 are denoted by triangles.

In Figure 1, Ls|Gs and Gs|Gs yielded the same error and, likewise, the Gs|Gs+1 and Ls|Gs+1 pair, as observed for all analyzed cases. The results demonstrate that the behavior of the new method seems to be determined by the integration rule on the right-hand side of the ODE. Gs|Gs+1 and Ls|Gs+1 also performed significantly better than the other methods. Ls|Ls+1 performed slightly better than Lobatto IIIC. A significant finding is that the usage of a different quadrature rule for the right-hand side of the ODE improved the accuracy of a numerical method.

Methods eLs|Gs and eLs|Gs+1 produced equivalent results in the analyzed example, as evident in Figure 1. Although they are of higher order (5 and 7) than the rest, they employ the same number of coefficients per variable (except Lobatto IIIC, which has a coefficient more than Gs|Gs at a given order of accuracy). For example, Gs|Gs of order 4 (G2|G2) employs two coefficients per variable, matching eLs|Gs at order 5; G3|G3 is of order 6 and employs three coefficients per variable and so does eL4|G4 at order 7. Although these methods share the same number of coefficients per step, a significant advantage of the “e” methods is observed.

Methods eLs|Gs and eLs|Gs+1 were applied only to Example 1. They require a very small step size for solving the remaining problems because they are not A-stable.

5.2. Example 2

The problem proposed by Frank and van der Houwen [23],

$$\{\begin{array}{l}{y}_1^{\prime }=-1002y_1+1000y_2^2,y_1\left(0\right)=1\\ y_2^{\prime }=y_1-y_2\left(1+y_2\right),y_2\left(0\right)=1\\ x\in \left[0,5\right]\end{array},$$

(23)

with exact solution $y_1\left(x\right)=e^{-2x}$, $y_2\left(x\right)=e^{-x}$ is a common choice for benchmarking [2,24]. Its intentionally designed stiffness is confirmed by computing the Jacobian of the right-hand side at the initial point and determining the eigenvalues ${\lambda }_1=-1004$, ${\lambda }_2=-1.00199$ [2]. The computed errors of different methods for the $y_1$ variable are reported in Figure 2 with squares and circles for methods of order 4 and 6, respectively.

In Figure 2, Ls|Ls+1 outperformed the rest. The problem also seems to be suitable for the Lobatto IIIC, which performed better than Gs|Gs, Ls|Gs, Gs|Gs+1, and Ls|Gs+1. The latter two outperformed the Gs|Gs. However, while Ls|Gs, Gs|Gs+1, Ls|Gs+1, and Gs|Gs have the same number of unknown coefficients per step, Lobatto IIIC has a coefficient more per variable.

5.3. Example 3

A problem presented by Vigo-Aguiar et al. [2], and Frank and van der Houwen [23]:

$$\{\begin{array}{l}{y}_1^{\prime }=-0.04y_1+{10}^4{y}_2{y}_3-0.96e^{-x},y_1\left(0\right)=1\\ y_2^{\prime }=0.04y_1-{10}^4{y}_2{y}_3-3\times {10}^7{y}_2^2-0.04e^{-x},y_2\left(0\right)=0\\ y_3^{\prime }=3\times {10}^7{y}_2^2+e^{-x},y_3\left(0\right)=0\\ x\in \left[0,5\right]\end{array},$$

(24)

with the exact solution $y_1\left(x\right)=e^{-x}$, $y_2\left(t\right)=0,$ and $y_3\left(t\right)=1-e^{-x}$. The same highly stiff Jacobian matrix is computed as in the famous Robertson problem (describing the kinetics of three species), only altered by the nonhomogeneous terms [23]. The computed errors for the $y_1$ variable are reported in Figure 3 with squares and circles for methods of order 4 and 6, respectively.

In this problem (see Figure 3), Lobatto IIIC was unstable at larger step sizes. Among the methods of order 4, L2|G3 and G2|G3 performed best, while L3|L4 seemed to be the most appropriate order 6 method.

5.4. Example 4

The problem presented by Vigo-Aguiar et al. [2], and Frank and van der Houwen [23]:

$$\{\begin{array}{l}{y}_1^{\prime }=-{10}^3\left(y_1^3{y}_2^6-{\cos }^3\left(t\right){\sin }^6\left(t\right)\right)-\sin \left(t\right),y_1\left(0\right)=1\\ y_2^{\prime }=-{10}^3\left(y_2^5{y}_3^4-{\sin }^9\left(t\right)\right)+\cos \left(t\right),y_2\left(0\right)=0\\ y_3^{\prime }=-{10}^3\left(y_1^2{y}_3^3-{\cos }^2\left(t\right){\sin }^3\left(t\right)\right)+\cos \left(t\right),y_3\left(0\right)=0\\ x\in \left[0,5\right]\end{array},$$

(25)

with exact solution $y_1\left(t\right)=\cos \left(t\right)$, $y_2\left(t\right)=y_3\left(t\right)=\sin \left(t\right)$. This is a more computationally intensive test problem due to strong nonlinearity. The approximation errors of different methods for the $y_1$ variable are reported in Figure 4 with squares and circles for methods of order 4 and 6, respectively. The best was Ls|Ls+1, followed by Ls|Gs+1 and Gs|Gs+1, and then Gs|Gs.

6. Discussion

The results show that changing the quadrature rule for the left-hand side of differential Equation (4) had a minuscule effect on the results. However, increasing the order on the right-hand side, i.e., methods #s|#s+1, improved the accuracy at the same number of coefficients. This was demonstrated on test equations analytically, as well as on some numerical examples.

In contrast to the classical derivation of collocation RK-type methods, the new method class is derived from the integral form of the ODE. This approach allows integrating the left-hand and the right-hand side separately, enabling us to employ different quadrature rules for each, resulting in a multitude of method families. Proof is provided that using Gauss quadrature on both ODE sides results in the Gauss–Legendre method and, likewise, Lobatto quadrature returns the Lobatto IIIA method.

By examining the computational cost of the analyzed methods for the presented examples (data not shown), the Gauss–Legendre method somewhat outperformed the proposed methods whereas the Lobatto method was the slowest. However, despite producing computationally somewhat inefficient results, we still believe that the presented approach contributes to our understanding of RK-type collocation methods; it shows how their accuracy can be enhanced (for the same number of method’s coefficients) and presents a new way of constructing them.

The proposed method class was found to be especially suitable for (high-frequency) oscillatory and stiff problems. We compared certain novel subclasses to implicit RK-type methods, i.e., the established collocation methods with the same framework and purpose, and we demonstrated the advantage of the new methods. Certain extensions and alternatives could outperform the proposed approach: the embedded formula collocation method [20,25], multivalue methods [26], parallel time integration methods [27], variable step size methods [28], block methods [29,30,31,32], etc. We focused on the popular RK-type methods to avoid an extensive comparison to all the possible alternatives. Tables similar to Butcher tableaus are provided for convenience and comparison to the RK methods. The behavior of methods was in this work demonstrated on some of the well-known ODEs from literature. These include stiff problems, as well as a nonlinear and highly oscillatory system. Systems where the accuracy of methods can behave differently, such as chaotic-type problems [33] or DAE systems [14], are beyond the scope of this work.

Although the implicit RK methods are known to be computationally expensive, much effort has been devoted to developing effective numerical algorithms and efficient computer software packages [3]. Well-known solvers based on the implicit RK approach are RADAU, STRIDE, DIRK, and SDIRK, popular for integrating stiff systems. Implicit RK methods can employ a much larger step size than the explicit ones. The relatively small interval of absolute stability makes the explicit RK methods unsuitable for stiff initial value problems [6]. The presented approach allows improving the implicit RK by increasing the integration accuracy of the right-hand side of the ODE at no additional computational cost.

7. Conclusions

A new class of methods for numerical integration of ODEs was proposed. The methods allow the usage of differing quadrature rules for the integration of the right-hand side of ODEs while using the same approximation function for the solution of the ODE. We demonstrated how the accuracy of Gauss–Legendre and Lobatto IIIA methods, which can also be derived from the proposed approach, can be increased at the same number of method stages.

In engineering and natural sciences, implicit RK-type methods are still widely used, especially for solving stiff systems [34,35]. As researchers are familiar with Butcher tableaus, the existing program codes can be enhanced in accuracy according to the proposed approach by forming an altered set of equations which is a rather small modification that retains the number of unknowns.

More methods of the novel class remain to be analyzed. For the sake of clarity, in this work, we only analyzed the Gauss–Legendre and Lobatto collocation points. Several more options are available, such as the Radau I, Radau II, or the so-called Gauss–Lobatto Chebyshev points. These would yield different method instances, further extending the investigation. The optimal choice for the quadrature on the right-hand side still remains to be determined.