Tensor Network Space-Time Spectral Collocation Method for Time-Dependent Convection-Diffusion-Reaction Equations

Abstract

Emerging tensor network techniques for solutions of partial differential equations (PDEs), known for their ability to break the curse of dimensionality, deliver new mathematical methods for ultra-fast numerical solutions of high-dimensional problems. Here, we introduce a Tensor Train (TT) Chebyshev spectral collocation method, in both space and time, for the solution of the time-dependent convection-diffusion-reaction (CDR) equation with inhomogeneous boundary conditions, in Cartesian geometry. Previous methods for numerical solution of time-dependent PDEs often used finite difference for time, and a spectral scheme for the spatial dimensions, which led to a slow linear convergence. Spectral collocation space-time methods show exponential convergence; however, for realistic problems they need to solve large four-dimensional systems. We overcome this difficulty by using a TT approach, as its complexity only grows linearly with the number of dimensions. We show that our TT space-time Chebyshev spectral collocation method converges exponentially, when the solution of the CDR is smooth, and demonstrate that it leads to a very high compression of linear operators from terabytes to kilobytes in TT-format, and a speedup of tens of thousands of times when compared to a full-grid space-time spectral method. These advantages allow us to obtain the solutions at much higher resolutions.

1. Introduction

Solving realistic problems often require numerical solutions of high-dimensional partial differential equations (PDEs). The discretization of these PDEs leads to a steep rise in the computational complexity in terms of storage and number of arithmetic operations with each added dimension, often rendering traditional numerical approaches impractical. This phenomenon, known as “the curse of dimensionality”, cf. [1], represents a formidable barrier to multidimensional numerical analysis, one that persists even in the era of exascale high-performance computing.

In recent years, tensor networks (TNs) [2] have come to the forefront as a promising strategy to counteract or circumvent the curse of dimensionality. TNs restructure high-dimensional data into networks of lower-dimensional tensors, enabling the division of complex data into more manageable subsets. Tensor factorizations such as the Canonical Polyadic decomposition, Hierarchical Tucker decomposition [3], and Tensor Train decomposition [4] are examples of tensor networks. Originally devised within the realm of theoretical physics, these methods now showcase their potential for accurate and efficient numerical solutions to high-dimensional PDEs.

Through a process we term tensorization, tensor networks offer an innovative means to efficiently represent the key elements of numerical algorithms for PDEs: grid functions that capture function values at grid nodes, and discrete operators approximating differential operators. This methodology has been applied successfully to a variety of challenging equations, from the quantum mechanical to the classical continuum [5,6,7,8,9,10,11,12,13,14].

The present work considers the time-dependent convection-diffusion-reaction (CDR) equation, a model equation of critical importance across a broad spectrum of physical and engineering systems. The CDR equation enables quantitative descriptions of heat transfer, mass transfer, fluid dynamics, and chemical interactions in complex settings from microscale processing to atmospheric modeling. Numerical methods for the multidimensional CDR equation also constitute a major research area given the equation’s utility spanning such a vast range of transport phenomena critical to climate modeling, energy systems, biomedical systems, materials synthesis, and related domains central to technology innovation [15,16,17]. Another approach to solving CDR equations is to use graph neural networks and deep neural networks [18,19,20].

Classical numerical methods, like finite differences and finite volumes, necessitate extremely fine grids for precision, leading to voluminous linear systems that are challenging to solve, particularly when the problem spans four dimensions (one temporal and three spatial). This is why spectral collocation methods are considered, due to their greater efficiency.

The spectral collocation method approximates the solution to convection-diffusion-reaction (CDR) equations through the use of high-degree polynomial interpolants, e.g., Legendre and Chebyshev. These interpolants are evaluated at specific collocation points chosen within the domain. Assuming that the ground-truth solution is regular enough, one of the most notable advantages of this approach is the exponential convergence, where the interpolation error decreases exponentially as the degree of the polynomial increases. As a computational method for solving partial differential equations (PDEs), spectral collocation offers several significant benefits, as highlighted by Funaro [21]: (i), the method achieves a high level of precision due to its inherent exponential convergence as the number of degrees of freedom grows; (ii), it accurately represents solutions that exhibit complex spatial variations by employing smooth and continuous basis functions; (iii) the basis functions are globally defined over the entire domain, making the spectral collocation method particularly suitable at handling problems with non-local interactions and dependencies.

Yet, the standard procedure for time-dependent PDEs, which couples spectral discretization of spatial derivatives with low-order temporal derivatives, often results in a temporal error that overshadows the spatial precision. Ideally, the accuracy of time integration should align with that of the spatial spectral approximation. Recent advancements in this area aim to overcome the time-stepping limitations by integrating space-time spectral collocation methods, which have shown exponential convergence in both spatial and temporal dimensions for sufficiently smooth heat equation solutions [22].

Despite these advances, the curse of dimensionality remains a significant challenge for space-time spectral methods applied to CDR equations, as computational complexity grows exponentially with problem size. In this study, we explore the discretization of the CDR equation in both space and time, employing a space-time operator scheme that simultaneously addresses spatial and temporal dimensions. This approach is particularly effective for phenomena with rapid dynamic changes where spatial and temporal variations are closely linked [23]. Although the space-time method traditionally faces the drawback of increased computational cost and memory requirements, we demonstrate that tensor network techniques effectively overcome these challenges.

We utilize spectral collocation discretization within this space-time framework, expanding the PDE solution in terms of a set of global basis functions. For the non-periodic CDR equation solution, we employ the Chebyshev polynomials as our basis functions. The Chebyshev collocation method requires regular grids, that is, it works mainly on Cartesian grids that are tensor products of 1D domain partitions, which is suitable to our tensor network formats.

The outline of the paper is as follows. In Section 2.2, we review some basic concepts for space-time collocation methods, introduce the discretization and its matrix formulation that leads to a linear system. In Section 3, we review the tensor notations and definitions of the TT-format, and the TT-matrix format for linear operators. In Section 3.4, we present our TT design of the numerical solution of the CDR equation and introduce our algorithms in tensor train format. In Section 4, we present our numerical results and assess the performance of our method. In Section 5, we offer our final remarks and discuss possible future work. For completeness, part of the details of the CDR tensorization approach are in Appendix A and Appendix B.

2. Mathematical Model and Numerical Discretization

2.1. Convection-Diffusion-Reaction Equation

We are interested in the following CDR problem:

Find $u(t,\mathbf{x})$ such that

$$\begin{array}{cc}\hfill {\displaystyle \frac{\partial u}{\partial t}}-\kappa (t,\mathbf{x})\Delta u+\mathbf{b}(t,\mathbf{x})·\nabla u+c(t,\mathbf{x})u& =f(t,\mathbf{x})\phantom{g(t,\mathbf{x})h\left(\mathbf{x}\right)}\mathrm{in}(0,T]\times \Omega ,\hfill \\ \hfill u& =g(t,\mathbf{x})\phantom{f(t,\mathbf{x})h\left(\mathbf{x}\right)}\mathrm{on}[0,T]\times \partial \Omega ,\hfill \\ \hfill u(0,\mathbf{x})& =h\left(\mathbf{x}\right)\phantom{f(t,\mathbf{x})g(t,\mathbf{x})}\mathrm{in}\Omega ,\hfill \end{array}$$

(1)

where $\kappa (t,\mathbf{x})$, $\mathbf{b}(t,\mathbf{x})$, and $c(t,\mathbf{x})$ are the diffusion, convection, and reaction coefficients, respectively. Furthermore, let $\Omega \subset {\mathbb{R}}^3$ be a three-dimensional, open parallelepiped domain with boundary $\partial \Omega$; $h\left(\mathbf{x}\right)$ is the initial condition, (IC), and $g(t,\mathbf{x})$ are the boundary conditions (BCs). Here, we denote the 3D vectors, such as $\mathbf{x}=(x,y,z)$, and matrices in bold font.

2.2. Chebyshev Collocation Method

We expand the approximate solution of the CDR equation on the set of orthogonal Chebyshev polynomials $T_k\left(x\right)$ = cos(k*arcos(x)) as global basis functions. Then, we enforce the CDR equation at a discrete set of points within the domain, known as collocation points, leading to a system of equations for the unknown expansion coefficients that we can solve. The collocation points of the Chebyshev grid are defined as in [24], and form the so-called collocation grid, cf. [21]. To this end, we need expressions for the derivatives of the approximate solution on the Chebyshev grid. The derivative expansion coefficients with respect to the same set of Chebyshev polynomials are determined by multiplying the matrix representation of the partial differential operators and the expansion coefficients of the numerical solution. In the rest of this section, we construct the time derivative, $\partial /\partial t$, the gradient, ∇, and the Laplacian, $\Delta$. To simplify such matrix representations, we modify the Chebyshev polynomials (following Ref. [21]), and construct a new set of Nth degree polynomials $l_j$, $0\le j\le N$, with respect to the collocation points, $x_i$, $0\le i\le N$, that satisfy:

$$l_j\left(x_i\right)=\left\{\begin{array}{c}1\mathrm{if}i=j,\hfill \\ 0\mathrm{if}i\ne j.\hfill \end{array}\right.$$

(2)

Using the new polynomials $l_j$, the discrete CDR solution becomes

$$u_h\left(x\right):=\sum _{j=0}^{N}u\left(x_j\right)l_j\left(x\right).$$

(3)

We construct all the spatial partial differential operators using the matrix representation of the single, one-dimensional derivative, ${\mathbf{S}}_x$, which reads as

$${\left({\displaystyle \frac{\partial }{\partial x}}\right)}_{ij}\to {\left({\mathbf{S}}_x\right)}_{ij}:={\displaystyle \frac{d}{dx}}{l}_j\left(x\right){|}_{x_i}.$$

(4)

The size of the matrix ${\mathbf{S}}_x$ is $(N+1)\times (N+1)$. We obtain the second-order derivative matrix, ${\mathbf{S}}_{xx}$, by matrix multiplication of the first-order derivatives as,

$${\left({\mathbf{S}}_{xx}\right)}_{ij}=\sum _{s=0}^N{\left({\mathbf{S}}_x\right)}_{is}{\left({\mathbf{S}}_x\right)}_{sj}.$$

(5)

For completeness, we refer to [21] for the derivation of this expression. Chebyshev polynomials and Legendre polynomials are both widely used basis functions in spectral collocation method. Here we focus on Chebyshev polynomials for clarity, but our scheme also provides exponential convergence for the choice of Legendre basis functions.

2.2.1. Matrix Form of the Discrete CDR Equation

First, we introduce the space-time matrix operators: ${\displaystyle \frac{\partial }{\partial t}}\to$${\mathbf{A}}_t$; $\kappa (t,\mathbf{x})\Delta$ →${\mathbf{A}}_D$; $\mathbf{b}(t,\mathbf{x})·\nabla$ → ${\mathbf{A}}_C$; and $c(t,\mathbf{x})$→ ${\mathbf{A}}_R$, respectively. Upon employing these matrices, Equation (1) results in the following linear system:

$$({\mathbf{A}}_t+{\mathbf{A}}_D+{\mathbf{A}}_C+{\mathbf{A}}_R)\mathbf{U}=\mathbf{F},$$

(6)

where U and F are the vectors corresponding to the solution, $u_h(t,\mathbf{x})$, and the loading term $f(t,\mathbf{x})$, respectively. The matrices ${\mathbf{A}}_t$, ${\mathbf{A}}_D$, ${\mathbf{A}}_C$, and ${\mathbf{A}}_R$ are of size ${(N+1)}^4\times {(N+1)}^4$ for the Nth order spectral collocation method, while U, F are column vectors of size ${(N+1)}^4\times 1$, and are designed to incorporate the boundary conditions in Equation (1).

2.2.2. Time Discretization Using Finite Differences and Chebyshev Grids

Here, we display two strategies to discretize the first-order derivative with respect to the time variable. First, we present the well-known temporal finite difference approach with the implicit backward Euler method [25], and second, we introduce a temporal spectral collocation discretization on a temporal Chebyshev grid. In both strategies, the space operators are discretized on a spatial Chebyshev grid.

• Finite difference approach: With $N+1$ time points, $t_0,t_1,\dots ,t_N$, such that $t_0=0$ and $t_N=T$ are the initial and final time points, the length of the time step is $\Delta t=(T-T_0)/N$. We emphasize that the backward Euler method is unconditionally stable, and thus the stability is independent of the size of the time step $\Delta t$ [25]. To consider the finite difference approach, we need to represent space and time separately. For this purpose we introduce a separate representation of the column vector, U, as a column vector with $(N+1)$ components, each of size ${(N+1)}^3\times 1$: ${\mathbf{U}}^T=[{\widehat{\mathbf{U}}}_0^T,\dots ,{\widehat{\mathbf{U}}}_N^T]$. Each component ${\widehat{\mathbf{U}}}_k$, represents the spatial part of the solution at time point $t_k$. Let ${\widehat{\mathbf{F}}}_k$ represent the load vector at time point $t_k$. In the temporal finite difference approach, at each time step, we have to solve the following linear system,

  $$\left({\mathbf{I}}_{\mathrm{space}}+\Delta t{\mathbf{S}}_{\mathrm{space}}\right){\widehat{\mathbf{U}}}_{k+1}={\widehat{\mathbf{U}}}_k+\Delta t{\widehat{\mathbf{F}}}_{k+1},k=0,\dots ,N.$$

  (7)
  Here, ${\mathbf{I}}_{\mathrm{space}}$ is the space-identity matrix of size ${(N+1)}^3\times {(N+1)}^3$, and, ${\mathbf{S}}_{\mathrm{space}}$ = ${\mathbf{A}}_D$ + ${\mathbf{A}}_C$ + ${\mathbf{A}}_R$, is the space spectral matrix which is positive definite. Hence, we have, $\det ({\mathbf{I}}_{\mathrm{space}}+\Delta t{\mathbf{S}}_{\mathrm{space}})\ne 0$, and the linear system Equation (7) has a unique solution. We can unite the space and time operators and solve Equation (7) in a single step. First, we rewrite Equation (7) as follows:

  $${\displaystyle \frac{1}{\Delta t}}{\widehat{\mathbf{U}}}_{k+1}-{\displaystyle \frac{1}{\Delta t}}{\widehat{\mathbf{U}}}_k+{\mathbf{S}}_{\mathrm{space}}{\widehat{\mathbf{U}}}_{k+1}={\widehat{\mathbf{F}}}_{k+1}.$$

  (8)
  Then, we define the time derivatives matrix, T, consistent with the backward Euler scheme,

  $$\mathbf{T}={\displaystyle \frac{1}{\Delta t}}{\left(\begin{array}{cccccc}1& 0& 0& \cdots & 0& 0\\ -1& 1& 0& \cdots & 0& 0\\ 0& -1& 1& \dots & 0& 0\\ ⋮& ⋮& ⋮& \ddots & ⋮& ⋮\\ 0& 0& 0& \cdots & -1& 1\end{array}\right)}_{(N+1)\times (N+1)}.$$

  (9)
  By employing T, ${\mathbf{S}}_{\mathrm{space}}$, and the Kronecker product, ⊗, we can construct the matricization of the space-time operator, $\mathbf{T}\otimes {\mathbf{I}}_{\mathrm{space}}+{\mathbf{I}}_t\otimes {\mathbf{S}}_{\mathrm{space}}$, where ${\mathbf{I}}_t$ is the time-identity matrix of size $(N+1)\times (N+1)$. Hence, the linear system, in the finite differences approach is:

  $$(\mathbf{T}\otimes {\mathbf{I}}_{\mathrm{space}}+{\mathbf{I}}_t\otimes {\mathbf{S}}_{\mathrm{space}})\mathbf{U}=\mathbf{F}.$$

  (10)
  We note that in Ref. [26], the authors employed the same finite difference technique to solve the heat equation in TT format. The authors explored this technique to approximate a high-dimensional PDE such as Fokker–Planck in Ref. [26]. Further, in [12], the authors approximated the Vlasov–Maxwell equation using a semi-implicit finite difference method in QTT format. In [27], the authors explored the Boltzmann–BGK equation using the Crank–Nicolson Leap Frog (CNLF) scheme in TT format. However, these schemes primarily suffer from a low convergence order. In what follows, we study the tensor network space-time spectral collocation methods to derive a high-order scheme.
• Time discretization on a Chebyshev grid: When the order of the Chebyshev polynomials, $N\to \infty$, the PDE approximate solution, $u_h(t_n,\mathbf{x})$, at time $t=t_n$ converges exponentially in space, as $N^{-\left|\alpha \right|}$, as shown in Appendix A.4 in [21], and linearly with the time step $\Delta t$,

  $$\parallel u(t_n,\mathbf{x})-u_h(t_n,\mathbf{x}){\parallel }_{L^2(\Omega )}\le (\Delta t+N^{-\left|\alpha \right|})\parallel {\displaystyle \frac{d^{\left|\alpha \right|}u(t_n,\mathbf{x})}{d{\mathbf{x}}^{\left|\alpha \right|}}}{\parallel }_{L^2(0,T;L^2(\Omega ))},$$

  (11)

  where ${\displaystyle \frac{d^{\left|\alpha \right|}u(t,\mathbf{x})}{d{\mathbf{x}}^{\left|\alpha \right|}}}\in L^2(\Omega )$, and $u(t,\mathbf{x})\in C^0(\Omega )$ for all $t\in (0,T)$. Here, $\alpha =({\alpha }_1,{\alpha }_2,{\alpha }_3)$ is a multi-index, which characterizes the smoothness of the CDR solution in space by $\left|\alpha \right|:={\alpha }_1+{\alpha }_2+{\alpha }_3$. It can be seen that even for higher-order Chebyshev polynomials, the global error is dominated by the error introduced by the temporal scheme that remains linear, which is the primary drawback of the temporal finite difference approach. To recover global exponential convergence, we apply the Chebyshev spectral collocation method for the discretization of both space and time variables. Further, the space-time spectral collocation methods are more suitable for parallel computation [28]. To accomplish this, we construct a single one-dimensional differential operator ${\displaystyle \frac{\partial }{\partial t}}$ in matrix form, ${\left({\mathbf{S}}_t\right)}_{ij}$, on a temporal Chebyshev grid, with collocation points $t_0,t_1\dots ,t_N$, as follows (see Equation (4)).

  $${\left({\mathbf{S}}_t\right)}_{ij}={\displaystyle \frac{d{l}_j\left(t\right)}{dt}}{|}_{t_i},0\le i,j\le N.$$

  (12)
  Hence, the linear system in the space-time collocation method is

  $$({\mathbf{S}}_t\otimes {\mathbf{I}}_{\mathrm{space}}+{\mathbf{I}}_t\otimes {\mathbf{S}}_{\mathrm{space}})\mathbf{U}=\mathbf{F},$$

  (13)

  where (see Equation (6)) ${\mathbf{A}}_t={\mathbf{S}}_t\otimes {\mathbf{I}}_{\mathrm{space}}$. To construct the space-time operator of the CDR equation on the Chebyshev grid, we also need to construct the space part, ${\mathbf{S}}_{\mathrm{space}}$.

2.2.3. Space Discretization on Chebyshev Grids

• Diffusionoperator on a Chebyshev grid: In this section, we focus on the matricization of the diffusion term, $\kappa (t,\mathbf{x})\Delta$→${\mathbf{A}}_D$. The diffusion operator $\Delta$ is constructed as follows:

  $$\begin{array}{c}\hfill \Delta ={\mathbf{I}}_t\otimes {\mathbf{S}}_{xx}\otimes {\mathbf{I}}_y\otimes {\mathbf{I}}_z+{\mathbf{I}}_t\otimes {\mathbf{I}}_x\otimes {\mathbf{S}}_{yy}\otimes {\mathbf{I}}_z+{\mathbf{I}}_t\otimes {\mathbf{I}}_x\otimes {\mathbf{I}}_y\otimes {\mathbf{S}}_{zz}.\end{array}$$

  (14)
  Then, the function $\kappa (t,\mathbf{x})$ is incorporated to form the diffusion term ${\mathbf{A}}_D$

  $${\mathbf{A}}_D=diag\left(\mathit{K}\right)\Delta ,$$

  (15)

  where $diag(\dots )$ denotes a diagonal matrix, and $\mathit{K}$ is a vector of size ${(N+1)}^4$ containing the evaluation of $\kappa (t,\mathbf{x})$ on the Chebyshev space-time grid.
• Discretization of the convection term on a Chebyshev grid: Here, we focus on the matricization of the convection term, $\mathbf{b}(t,\mathbf{x})·\nabla \to {\mathbf{A}}_C$, with the convective function, $\mathbf{b}(t,x,y,z)$, which we assume in the form

  $$\mathbf{b}(t,x,y,z)=\left[b^x(t,x,y,z)b^y(t,x,y,z)b^z(t,x,y,z)\right].$$

  (16)
  Then, we construct the convection term ${\mathbf{A}}_C$

  $$\begin{array}{cc}\hfill {\mathbf{A}}_C=& diag\left({\mathbf{B}}^x\right)\left({\mathbf{I}}_t\otimes {\mathbf{S}}_x\otimes {\mathbf{I}}_y\otimes {\mathbf{I}}_z\right)+\dots \hfill \\ \hfill & diag\left({\mathbf{B}}^y\right)\left({\mathbf{I}}_t\otimes {\mathbf{I}}_x\otimes {\mathbf{S}}_y\otimes {\mathbf{I}}_z\right)+\dots \hfill \\ \hfill & diag\left({\mathbf{B}}^z\right)\left({\mathbf{I}}_t\otimes {\mathbf{I}}_x\otimes {\mathbf{I}}_y\otimes {\mathbf{S}}_z\right),\hfill \end{array}$$

  (17)

  where ${\mathbf{B}}^x,{\mathbf{B}}^y$, and ${\mathbf{B}}^z$ are vectors of size ${(N+1)}^4$ containing the evaluation of the functions $b^x,b^y$, and $b^z$ on the Chebyshev space-time grid.
• Discretization of the reaction term on a Chebyshev grid: Here, we focus on the matricization of the reaction term, $c(t,\mathbf{x})$→${\mathbf{A}}_R$, which is given by ${\mathbf{A}}_R=diag\left(\mathbf{C}\right){\mathbf{I}}_t\otimes {\mathbf{I}}_z\otimes {\mathbf{I}}_y\otimes {\mathbf{I}}_z$, where C is the evaluation of the function $c(t,\mathbf{x})$ on the Chebyshev space-time grid.

In this article, we consider a time-dependent convection diffusion problem (1) with values of the diffusion that are relatively large such that the numerical solution of this model problem are stable and free of spurious oscillations. However, when the diffusion coefficient is very small compared to convective coefficients, the numerical scheme is unstable and produces non-physical oscillations. To reduce these oscillations, one needs to compute the solution on upwind grid as highlighted in Chapter 2 of Ref. [21].

2.2.4. Initial and Boundary Conditions on Space-Time Chebyshev Grids

So far, we have not incorporated the boundary conditions (BCs) and the initial condition (IC), given in (1), into the linear system $\mathbf{A}\mathbf{U}=\mathbf{F}$ of Equation (13). In the space-time method, we consider the IC equivalent to the BCs. The nodes of the Chebyshev grid are split into two parts: $\left(i\right)$ BC/IC nodes, and $\left(ii\right)$ interior nodes, where the solution is unknown. Let $i^{\mathrm{Bd}}$ and $i^{\mathrm{Int}}$ be the set of multi-indices, introduced in [29], of the BC/IC and interior nodes, respectively.

Boundary and initial conditions are imposed by explicitly enforcing the BC/IC nodes to be equal to the BCs, $g(t,x)$, or to the IC, $h\left(x\right)$, and then reducing the linear system for all nodes into a smaller system only for the interior nodes,

$${\mathbf{A}}^{\mathrm{Int}}{\mathbf{U}}^{\mathrm{Int}}={\mathbf{F}}^{\mathrm{Int}}-{\mathbf{F}}^{\mathrm{Bd}}.$$

(18)

To make it clear, we consider below a simple example with $N=4$ collocation points in two dimensions $(t,x)$. The set of Chebyshev nodes can be denoted by multi-indices as $\mathbf{U}:=\{u_1,u_2,\dots ,u_{16}\}$ (Figure 1). After imposing the BC/IC conditions, the linear system with BC/IC nodes, $\mathbf{A}\mathbf{U}=\mathbf{F}$, is

$$\begin{array}{cc}\hfill & u_1=g(t_0,x_0)\hfill \\ \hfill & ⋮\hfill \\ \hfill & u_4=g(t_3,x_0)\hfill \\ \hfill & u_5=h\left(x_1\right),\hfill \\ \hfill & A_{6,1}{u}_1+A_{6,2}{u}_2+\dots +A_{6,15}{u}_{15}+A_{6,16}{u}_{16}=F_6\hfill \\ \hfill & ⋮\hfill \\ \hfill & A_{8,1}{u}_1+A_{8,2}{u}_2+\dots +A_{8,15}{u}_{15}+A_{8,16}{u}_{16}=F_8\hfill \\ \hfill & u_9=h\left(x_2\right)\hfill \\ \hfill & A_{10,1}{u}_1+A_{10,2}{u}_2+\dots +A_{10,15}{u}_{15}+A_{10,16}{u}_{16}=F_{10}\hfill \\ \hfill & ⋮\hfill \\ \hfill & A_{12,1}{u}_1+A_{12,2}{u}_2+\dots +A_{12,15}{u}_{15}+A_{12,16}{u}_{16}=F_{12}\hfill \\ \hfill & ⋮\hfill \\ \hfill & u_{13}=g(t_0,x_3)\hfill \\ \hfill & ⋮\hfill \\ \hfill & u_{16}=g(t_3,x_3).\hfill \end{array}$$

(19)

The unknown values $(u_6,u_7,u_8,u_{10},u_{11},u_{12})$ associated to the six interior nodes satisfy the reduced linear system whose equations read as

$$\begin{array}{ll}{A}_{l,6}{u}_6+A_{l,7}{u}_7+A_{l,8}{u}_8+A_{l,10}{u}_{10}& +A_{l,11}{u}_{11}+A_{l,12}{u}_{12}\\ & =F_l-A_{l,1}g(t_0,x_0)-A_{l,2}g(t_1,x_0)-\dots ,\end{array}$$

(20)

with $l\in \{6,7,8,10,11,12\}$, where we transfer the values of BC/IC nodes to the right-hand side.

Here, we consider Dirichlet boundary condition [30]; however, real-world applications generally appear with more complex boundary conditions such as Neumann boundary conditions or mixed boundary conditions. Chapter 3 of Reference [21] contains a discussion of the spectral collocation discretization for the elliptic problem with Neumann and mixed boundary conditions. Exploring spectral collocation methods in TT format for a real-world application such as the Allen–Cahn equation, Navier–Stokes or Fokker–Planck equation is a new and exciting direction of research in the TT literature.

3. Tensor Networks

In this section, we introduce the TT format, the specific tensor network we used in this work, the representation of linear operators in TT-matrix format, and the cross-interpolation method. All these methods are fundamental in the tensorization of our spectral collocation discretization of the CDR equation. For a more comprehensive understanding of notation and concepts, we refer the reader to Refs. [3,4,29], which provide detailed explanations.

3.1. Tensor Train

The TT format, introduced by Oseledets in 2011 [4], represents a sequential chain of matrix products involving both two-dimensional matrices and three-dimensional tensors, referred to as TT cores. We can visualize this chain as in Figure 2. Given that tensors in our formulation are at most four dimensional (one temporal and three spatial dimensions), we consider the tensor train format in the context of 4D tensors. Specifically, the TT approximation ${\mathcal{X}}^{TT}$ of a four-dimensional tensor $\mathcal{X}$ is a tensor with elements

$$\begin{array}{c}\hfill {\mathcal{X}}^{TT}(i_1,i_2,i_3,i_4)=\sum _{{\alpha }_1=1}^{r_1}\sum _{{\alpha }_2=1}^{r_2}\sum _{{\alpha }_3=1}^{r_3}{\mathcal{G}}_1(1,i_1,{\alpha }_1){\mathcal{G}}_2({\alpha }_1,i_2,{\alpha }_2){\mathcal{G}}_3({\alpha }_2,i_3,{\alpha }_3){\mathcal{G}}_4({\alpha }_3,i_4,1)+\epsilon ,\end{array}$$

(21)

where the error, $\epsilon$, is a tensor with the same dimensions as $\mathcal{X}$, and the elements of the array $\mathbf{r}=[r_1,r_2,r_3]$ are the TT ranks and quantify the compression effectiveness of the TT approximation. Since each TT core, ${\mathcal{G}}_p\left(i_k\right)$, only depends on a single index of the full tensor $\mathcal{X}$, e.g., $i_k$, the TT format effectively embodies a discrete separation of variables [2].

In Figure 2, we show a four-dimensional array $\mathcal{X}(t,x,y,z)$, decomposed in TT format.

3.2. Linear Operators in TT-Matrix Format

Suppose that the approximate solution of the CDR equation is a 4D tensor $\mathcal{U}$; then, the linear operator $\mathcal{A}$ acting on that solution is represented as an 8D tensor. The transformation $\mathcal{A}\mathcal{U}$ is defined as:

$$\begin{array}{c}\hfill \left(\mathcal{A}\mathcal{U}\right)(i_1,i_2,i_3,i_4)=\sum _{j_1,j_2,j_3,j_4}\mathcal{A}(i_1,j_1,\dots ,i_4,j_4)\mathcal{U}(j_1,\dots ,j_4).\end{array}$$

The tensor $\mathcal{A}$ and the matrix operator A, defined in Equation (18), are related as:

$$\mathcal{A}(i_1,j_1,\dots ,i_4,j_4)=\mathbf{A}(i_1{i}_2{i}_3{i}_4,j_1{j}_2{j}_3{j}_4).$$

(22)

Thus, we can construct the tensor $\mathcal{A}$ by suitably reshaping and permuting the dimensions of the matrix A. The linear operator, $\mathcal{A}$, can be further represented in a variant of TT format, called TT matrix, cf. [14]. The component-wise TT matrix ${\mathcal{A}}^{TT}$ is defined as:

$${\mathcal{A}}^{TT}(i_1,j_1,\dots ,i_4,j_4)=\sum _{{\alpha }_1,{\alpha }_2,{\alpha }_3}{\mathcal{G}}_1\left(1,(i_1,j_1),{\alpha }_1\right)\dots {\mathcal{G}}_4\left({\alpha }_3,(i_4,j_4),1\right),$$

(23)

where ${\mathcal{G}}_k$ are 4D TT cores.

Figure 3 shows the process of transforming a matrix operator A to its tensor format, $\mathcal{A}$, and finally, to its TT-matrix format, ${\mathcal{A}}^{TT}$.

We can further simplify the TT-matrix representations of the matrix A if it is a Kronecker product of matrices, i.e., $\mathbf{A}={\mathbf{A}}_1\otimes {\mathbf{A}}_2\otimes {\mathbf{A}}_3\otimes {\mathbf{A}}_4$. Based on the relationship defined in Equation (22), the tensor $\mathcal{A}$ can be constructed using tensor product as $\mathcal{A}={\mathbf{A}}_1\circ {\mathbf{A}}_2\circ {\mathbf{A}}_3\circ {\mathbf{A}}_4$. This implies the internal ranks of the TT format of $\mathcal{A}$ in (23) are all equal to 1. In such a case, all summations in Equation (23) reduce to a sequence of single matrix–matrix multiplications, and the TT format of $\mathcal{A}$ becomes the tensor product of d matrices:

$$\begin{array}{cc}\hfill {\mathcal{A}}^{TT}& ={\mathbf{A}}_1\circ {\mathbf{A}}_2\circ \dots \circ {\mathbf{A}}_d.\hfill \end{array}$$

(24)

This specific structure appears quite often in the matrix discretization and is exploited in the tensorization to construct an efficient TT format.

3.3. TT Cross-Interpolation

The original TT algorithm is based on consecutive applications of singular value decomposition (SVD) on unfoldings of a tensor [4]. Although known for its efficiency, the TT algorithm requires access to the full tensor, which is impractical and even impossible for extra-large tensors. To address this challenge, the cross-interpolation algorithm, TT-cross, was developed [31]. The idea behind TT-cross is essentially to replace the SVD in the TT algorithm with an approximate version of the skeleton/CUR decomposition [32,33]. The CUR decomposition approximates a matrix by selecting a few of its columns $\mathbf{C}$, a few of its rows $\mathbf{R}$, and a matrix $\mathbf{U}$ that connects them, as shown in Figure 4. Mathematically, the CUR decomposition finds an approximation for a matrix $\mathbf{A}$, as $\mathbf{A}\approx \mathbf{C}\mathbf{U}\mathbf{R}$. The TT-cross algorithm, utilizes the maximum volume principle (maxvol algorithm) [34,35] to determine $\mathbf{U}$. The maxvol algorithm chooses a few columns, $\mathbf{C}$ and rows, $\mathbf{R}$, of $\mathbf{A}$, such that, the intersection matrix ${\mathbf{U}}^{-1}$ has maximum volume [36].

TT-cross-interpolation and its versions can be seen as a heuristic generalization of CUR to tensors [37,38]. TT-cross utilizes the maximum volume algorithm iteratively, often beginning with a few randomly chosen fibers, to select an optimal number of specific tensor fibers that capture essential information of the tensor [39]. These fibers are used to construct a lower-rank TT representation. The naive generalization of CUR was proven to be expensive, which led to the development of various heuristic optimization techniques, such as, TT-ALS [40], DMRG [36,41], and AMEN [42].

To solve the CDR equation, we use TT-cross to build the TT format directly from the coefficient functions $\kappa (t,\mathbf{x})$, $\mathbf{b}(t,\mathbf{x})$, $c(t,\mathbf{x})$, boundary conditions, initial conditions, and loading functions.

3.4. Tensorization

The space-time discretization produces a linear system for all interior nodes as specified in Equation (18). Here, we simplify the notation, and refer to this equation as $\mathbf{A}\mathbf{U}=\mathbf{F}-{\mathbf{F}}^{\mathrm{Bd}}$.

Tensorization is a process of building the TT format of all components of this linear system

$$\mathbf{A}\mathbf{U}=\mathbf{F}-{\mathbf{F}}^{\mathrm{Bd}}\to {\mathcal{A}}^{TT}{\mathcal{U}}^{TT}={\mathcal{F}}^{TT}-{\mathcal{F}}^{\mathrm{Bd},TT},$$

(25)

where ${\mathcal{A}}^{TT}={\mathcal{S}}_t^{TT}+{\mathcal{A}}_D^{TT}+{\mathcal{A}}_C^{TT}+{\mathcal{A}}_R^{TT}$.

In the matrix form, the operator $\mathbf{A}={\mathbf{A}}_t+{\mathbf{A}}_D+{\mathbf{A}}_C+{\mathbf{A}}_R$ is designed to act on the vectorized solution U. In the full tensor format, the solution is kept in its original tensor form $\mathcal{U}$, which is a 4D tensor. Consequently, all the operators ${\mathcal{A}}_t,{\mathcal{A}}_D,{\mathcal{A}}_C,\mathrm{and}{\mathcal{A}}_R$ are 8D tensors. Lastly, $\mathcal{F}$ and ${\mathcal{F}}^{\mathrm{Bd}}$ are both 4D tensors. Here, given that these operators in their matrix form have a Kronecker product structure, their TT format can be constructed by using component matrices as TT cores. To construct the tensor format of the operators acting on the interior nodes, we need to define some sets of indices:

$$\begin{array}{cc}\hfill & {\mathcal{I}}_t=1:N\mathrm{index}\mathrm{set}\mathrm{for}\mathrm{time}\mathrm{variable},\hfill \\ \hfill & {\mathcal{I}}_s=1:(N-1)\mathrm{index}\mathrm{set}\mathrm{for}\mathrm{space}\mathrm{variable}.\hfill \end{array}$$

• TT-matrix time operator, ${\mathcal{A}}_t^{TT}$: The temporal operator in TT-matrix format acting only on the interior nodes is constructed as:

  $${\mathcal{A}}_t^{TT}={\mathbf{S}}_t({\mathcal{I}}_t,{\mathcal{I}}_t)\circ {\mathbf{I}}_{N-1}\circ {\mathbf{I}}_{N-1}\circ {\mathbf{I}}_{N-1},$$

  (26)

  where ${\mathbf{I}}_{N-1}$ is the identity matrix of size $(N-1)\times (N-1)$.
• TT-matrix diffusion operator, ${\mathcal{A}}_D^{TT}$: The Laplace operator in TT-matrix format is constructed as:

  $$\begin{array}{cc}\hfill {\Delta }^{TT}={\mathbf{I}}_N\circ {\mathbf{S}}_{xx}({\mathcal{I}}_s,{\mathcal{I}}_s)\circ {\mathbf{I}}_{N-1}\circ {\mathbf{I}}_{N-1}+& {\mathbf{I}}_N\circ {\mathbf{I}}_{N-1}\circ {\mathbf{S}}_{yy}({\mathcal{I}}_s,{\mathcal{I}}_s)\circ {\mathbf{I}}_{N-1}\hfill \\ \hfill +& {\mathbf{I}}_N\circ {\mathbf{I}}_{N-1}\circ {\mathbf{I}}_{N-1}\circ {\mathbf{S}}_{zz}({\mathcal{I}}_s,{\mathcal{I}}_s).\hfill \end{array}$$

  (27)
  Hence, the Laplace operator is a sum of three linear operators in TT-matrix formats, which again is a TT matrix. Further, we need to format the diffusion coefficient $\kappa (t,\mathbf{x})$ on the Chebyshev grid of interior nodes in TT format. To transform $\kappa (t,\mathbf{x})\to {\mathcal{K}}^{TT}$, we apply the cross-interpolation technique described in Section 3.3. Finally, we need to transform ${\mathcal{K}}^{TT}$ in TT-matrix format to be able to multiply it with ${\Delta }^{TT}$. The transformation to TT → TT-matrix is given in Algorithm 1.
• TT-matrix convection operator, ${\mathcal{A}}_C^{TT}$: The convection operator in TT-matrix format is constructed as:

  $$\begin{array}{c}\hfill {\nabla }_x^{TT}={\mathbf{I}}^N\circ {\mathbf{S}}_x({\mathcal{I}}_s,{\mathcal{I}}_s)\circ {\mathbf{I}}^{N-1}\circ {\mathbf{I}}^{N-1},\\ \hfill {\nabla }_y^{TT}={\mathbf{I}}^N\circ {\mathbf{I}}^{N-1}\circ {\mathbf{S}}_y({\mathcal{I}}_s,{\mathcal{I}}_s)\circ {\mathbf{I}}^{N-1},\\ \hfill {\nabla }_z^{TT}={\mathbf{I}}^N\circ {\mathbf{I}}^{N-1}\circ {\mathbf{I}}^{N-1}\circ {\mathbf{S}}_z({\mathcal{I}}_s,{\mathcal{I}}_s).\end{array}$$

  (28)

Next, the tensor operators ${\mathcal{B}}_x^{op,TT}$, ${\mathcal{B}}_y^{op,TT}$ and ${\mathcal{B}}_z^{op,TT}$ are computed from the functions $b^x,b^y$ and $b^z$ in the same way as being computed with $\kappa (t,\mathbf{x})$. Then, the TT-matrix format of the convection operator is constructed as:

$${\mathcal{A}}_C^{TT}={\mathcal{B}}_x^{op,TT}{\nabla }_x^{TT}+{\mathcal{B}}_y^{op,TT}{\nabla }_y^{TT}+{\mathcal{B}}_z^{op,TT}{\nabla }_z^{TT}.$$

(29)

Algorithm 1: Reformat TT, ${\mathcal{K}}^{TT}$, in TT-matrix format, ${\mathcal{K}}^{op,TT}$.

• TT-matrix reaction operator, ${\mathcal{A}}_R^{TT}$: The reaction operator ${\mathcal{A}}_R^{TT}$ basically is the function $c(t,\mathbf{x})$ being converted to an operator in the same way as with other coefficient functions.
  Then, the TT-matrix format of the operator $\mathcal{A}$ is constructed as

  $${\mathcal{A}}^{TT}={\mathcal{A}}_t^{TT}+{\mathcal{A}}_D^{TT}+{\mathcal{A}}_C^{TT}+{\mathcal{A}}_R^{TT}.$$
• TT-matrix loading tensor, ${\mathcal{F}}^{TT}$: the TT-matrix loading tensor ${\mathcal{F}}^{TT}$ is constructed by applying the cross-interpolation on the function $f(t,\mathbf{x})$ on the grid of interior nodes.
• TT boundary tensor, ${\mathcal{F}}^{Bd,TT}$: The boundary tensor ${\mathcal{F}}^{Bd}$ is used to incorporate the information about boundary and initial conditions into the linear system of the interior nodes. Only in the case of homogeneous BCs and a zero IC is this tensor a zero tensor. Basically, the idea is to construct an operator ${\mathcal{A}}^{map}$, similar to $\mathcal{A}$. The difference is that ${\mathcal{A}}^{map}$ will be a mapping from all nodes to only unknown nodes. The size of the operator ${\mathcal{A}}^{map}$ is $N\times (N+1)\times (N-1)\times (N+1)\times (N-1)\times (N+1)\times (N-1)\times (N+1)$. Then, the boundary tensor ${\mathcal{F}}^{Bd}$ is computed by applying ${\mathcal{A}}^{map}$ to a tensor ${\mathcal{G}}^{Bd}$ containing only the information from the BCs/IC. The details about constructing ${\mathcal{A}}^{map,TT}$ and ${\mathcal{G}}^{Bd,TT}$ are included in Appendix B.

At this point, we have completed constructing the TT format of the linear system ${\mathcal{A}}^{TT}{\mathcal{U}}^{TT}={\mathcal{F}}^{TT}-{\mathcal{F}}^{Bd,TT}$. To solve the TT linear system by optimization techniques, we used the routines amen_cross, amen_solve and amen_mm form the MATLAB TT-Toolbox [43].

We primarily explored the space-time spectral collocation methods for time-dependent CDR equation in full-grid and TT format. Our scheme includes the discretization of space and time variables (13) and corresponding TT formats of stationary and non-stationary terms are discussed in Section 3. We emphasize that our scheme can also be employed to approximate general elliptic problem by simple modification of the space-time full-grid scheme. In fact, one needs to consider the discretization of diffusion, convection, reaction terms, and the right-hand side loading term. Further, we highlight that the tensor network space-time spectral collocation scheme for the nonlinear time-dependent CDR equation is not straightforward and needs the Newton or fixed-point scheme to solve the system of nonlinear equations. By using a TT-truncation technique [44], we bounded the rank of the TT scheme for nonlinear problems.

4. Numerical Results

We investigated the computational efficiency and potential low-rank structures of the space-time solver by implementing the space-time operator in the tensor train format. To validate the algorithm presented earlier and assess its performance, we conducted several numerical experiments using MATLAB R2022b on a 2019 Macbook Pro 16 equipped with a 2.4 GHz 8-core i9 CPU. All our implementations relied on the MATLAB TT-Toolbox [43]. In the first example, we considered a manufactured solution of the time-dependent CDR equation with constant coefficients. The second example involved the space-time solution operator with non-constant coefficients. Finally, the third example showcased the behavior of the methods applied to the CDR with a non-smooth manufactured solution.

For all the examples, we compared the TT format of the space-time operator with the finite difference–finite difference (FD-FD) method and the spectral–spectral (SP-SP) collocation method. Additionally, we compared the TT format of SP-SP with the full grid formulation of the SP-SP method. To quantify the compression achieved, we introduced the compression ratio CR, defined as:

$$CR:={\displaystyle \frac{Mem\left({\mathcal{X}}^{TT}\right)}{Mem\left(\mathcal{X}\right)}},$$

(30)

where $Mem\left({\mathcal{X}}^{TT}\right)$ and $Mem\left(\mathcal{X}\right)$ are the total memory required for storing the scheme’s unknowns in the TT format, e.g., ${\mathcal{X}}^{TT}$ and the full tensor format, e.g., $\mathcal{X}$. We show that the TT format may achieve significant compression of spectral-collocation differential operators from terabyte sizes to megabytes and even kilobytes when solving the CDR equation on very fine meshes. Such a compression especially reduces the memory requirements for solving the linear system that results from the CDR equation discretization.

4.1. Test 1: Manufactured Solution with Constant Coefficients

In this first test, we studied the convergence behavior of the benchmark problem described in Section 2.1 on the computational domain $[0,1]\times {[-50,50]}^3$. We considered constant coefficients, specifically, $\kappa (t,\mathbf{x})=1$, $\mathbf{b}(t,\mathbf{x})=(1,1,1)$, and $c(t,\mathbf{x})=1$. The right-hand side force function $f(t,\mathbf{x})$ and the boundary conditions were determined in accordance with the exact solution $u(t,x,y,z):=sin\left(2\pi \right(t+x+y+z\left)\right)$.

Figure 5 presents the results of this test.The left panel displays the relative error curve in the $L^2$ norm, demonstrating the exponential convergence of the SP-SP schemes. The middle panel depicts the elapsed time in seconds, indicating that the TT format enables solutions with much higher resolution compared to the full grid format. The right panel illustrates the compression ratio of the solution. We plotted all curves against the number of points per dimension.

In Figure 5, we compare the SP-SP method in the full-grid format and the SP-SP method in TT format. In the left panel, we plotted the $L^2$ relative norms computed using the full-grid and TT format against the number of nodes per dimension. The TT format provided the same level of accuracy as the full-grid scheme. However, the TT format allows us to handle large data sets, unlike the full-grid approach, which is limited to small grid data. This characteristic ensures that the TT format utilizes significantly less memory than the full-grid approach, making handling large data sets practical. Furthermore, the numerical results demonstrated the well-known exponential convergence of the spectral method. In the full-grid computation, we achieved an accuracy of approximately ${10}^{-3}$, while in the TT format, we attained an accuracy of approximately ${10}^{-12}$ with only a minimal increase in grid data. At 64 nodes, the accuracy did not improve further as it reached the truncation tolerance of the TT format.

In the middle panel, we compare the elapsed time (in seconds) required for both the TT format and full-grid approaches. In the TT format, the elapsed time to compute the space-time operator increased linearly while preserving the invariance of the variables. On the other hand, in the full-grid approach, the elapsed time increased exponentially. This behavior aligned with the theoretical prediction, as the complexity grows linearly. For instance, with 16 nodes per dimension, the time taken in the TT format (represented by the third cross point) was 1400 times faster than in the full-grid format (represented by the circled point).

Finally, we emphasize that this speed-up would further improve for larger grids. For example, based on the extrapolated elapsed time for the full-grid scheme, we expect the TT method to be approximately $9\times {10}^5$ times faster with 64 nodes per dimension.

Another interesting aspect is the compression ratio, which we defined in Equation (30). In the right panel of Figure 5, we present the plot of the compression ratio versus the number of nodes per dimension, indicated by the curve. The compression ratio exponentially decreased as the number of nodes per dimension increased. At 64 nodes per dimension, we achieved a compression ratio that saved approximately ${10}^4$ orders of magnitude in storage. This remarkable reduction in storage requirements is further illustrated in

Table 1. Storage cost comparison of $\mathcal{A}$ between full-format and TT-format formulations. At 64 points, the TT format allows a compression from 1640 Tb to 363 Kb, which is about ten orders of magnitude.

Number of Points per Dimension	Size of $\mathcal{A}$ in TB	Size of ${\mathcal{A}}^{\mathit{TT}}$ in KB	Compression Ratio
8	1.66 × 10−5	3.58	2.00 × 10−4
16	1.23 × 10−2	18.8	1.42 × 10−6
32	5.10	85.3	1.56 × 10−8
64	1.64 × 103	3.63 × 10−2	2.06 × 10−10

, where we showcase the compression of the aggregated operator $\mathcal{A}$, which is the memory bottleneck of the full-grid scheme. The TT format allowed the full utilization of terabyte-sized operators while utilizing only kilobytes of storage. These results demonstrate the computational advantages of the SP-SP scheme in TT format compared to existing techniques.

Next, we aimed to compare the performance of the SP-SP scheme with the FD-FD scheme, both in TT format. The space-time FD-FD scheme was initially proposed by Dolgov et al. [26]. In Figure 6, we compare the SP-SP scheme in TT format and the FD-FD scheme in TT format. The left panel shows the $L^2$ relative error norms computed by both schemes in TT format. As the SP-SP scheme converged exponentially, it provided significantly more accurate results than the FD-FD scheme, which converged linearly. For instance, at 64 nodes per dimension, the approximation error of the SP-SP scheme was approximately ${10}^{-11}$, while the approximation error of the FD-FD scheme was only of the order of ${10}^{-3}$. This fact showed that our TT-based spectral-collocation method was approximately ${10}^{-8}$ times more accurate than the FD-FD-based for the same grid data.

In the middle panel, we plotted the elapsed time (in seconds) against the number of nodes per dimension. At 64 nodes per dimension, the SP-SP scheme took roughly twice as long as the FD-FD scheme. This indicated that the SP-SP scheme was slower by a factor of two compared to the FD-FD scheme. However, in the right panel, we observed that the SP-SP scheme computed highly accurate solutions relative to the consumed time, in contrast to the FD-FD scheme. Specifically, the SP-SP scheme required 10 seconds to compute a solution accurate to the order of ${10}^{-12}$, whereas the FD-FD scheme computed a solution accurate to the order of ${10}^{-3}$ within the same time frame.

In conclusion, we found that the SP-SP scheme computed highly accurate approximate solutions compared to existing techniques, such as the FD-FD scheme.

4.2. Test 2: Manufactured Solution with Variable Coefficients

In this test case, we examined the same model problem as above with the variable coefficients $\kappa (t,\mathbf{x})=exp(-t^2)$, $\mathbf{b}(t,\mathbf{x})=(sin(2\pi x),cos(2\pi y),sin(2\pi z\left)\right)$, and $c(t,\mathbf{x})=cos\left(2\pi \right(t+x+y+z\left)\right)$.

In Figure 7, we compare the TT format and the full-grid approaches, where we used SP-SP in both cases. The TT approximation achieved the same level of accuracy as the full-grid computation. The left panel plot illustrates the exponential convergence observed in both approaches. The error at 64 nodes did not improve further because it reached the truncation tolerance of the TT format.

In the middle panel, we compare the elapsed time (in seconds) required for both approaches. The time taken by the full grid approach increased exponentially, while it increased linearly in the TT format. We conclude that the TT format provides approximately a 40-fold speed-up compared to the full-grid computation at a grid size of 16 nodes per dimension, and an estimated $5\times {10}^4$ times faster speed at 64 nodes based on the extrapolated full-grid time. It is worth noting that the full-grid computation was unable to handle the grid data with 32 nodes per dimension due to memory limitations.

In the third panel, we show the compression ratio of the TT format against the number of nodes per dimension. As expected, the compression ratio decreased exponentially. It is significant to highlight that the SP-SP scheme with the TT format achieved storage savings of approximately ${10}^3$ orders of magnitude at 64 nodes per dimension. Additionally,

Table 2. Storage Cost Comparison of $\mathcal{A}$ between full-size and TT-format. At 64 points, the TT-format allows compression from 1640 TBs to about 1.81 MBs, which is about nine orders of magnitude.

Number of Points per Dimension	Size of $\mathcal{A}$ in TB	Size of ${\mathcal{A}}^{\mathit{TT}}$ in KB	Compress Ratio
8	1.66 × 10−5	17.4	9.73 × 10−4
16	1.23 × 10−2	93.0	7.03 × 10−6
32	5.10	4.24 × 102	7.75 × 10−8
64	1.64 × 103	1.81 × 103	1.03 × 10−9

showcases the aggregated operator $\mathcal{A}$’s compression, demonstrating that the TT format allowed full access to terabyte-sized operators using only kilobytes of storage. For example, at 64 nodes, the TT format only required 1.81 MB of storage to store a full-grid operator with a size of 1640 terabytes.

In Figure 8, we present a comparison between the FD-FD scheme in TT format and the SP-SP scheme in TT format. The left panel displays the plot of $L^2$ relative errors against the number of nodes per dimension. The SP-SP scheme with TT format achieved an approximation of the numerical solution that was approximately accurate to the order of ${10}^{-12}$, while the FD-FD scheme was only accurate to the order of ${10}^{-3}$ at 64 nodes per dimension.

In the middle panel, we plotted the elapsed times for both methods against the number of nodes per dimension. Both methods consumed almost the same amount of time for all grid data.

In the right panel, we plotted the errors computed by both methods against the elapsed time. The SP-SP scheme generated a highly accurate numerical solution compared to the FD-FD scheme. Specifically, the SP-SP scheme achieved an accuracy of approximately ${10}^{-12}$, while the FD-FD scheme only approximated the solution to the order of ${10}^{-3}$ within the same time frame. Furthermore, the accuracy increased exponentially in the case of the SP-SP method.

4.3. Test 3: Non-Smooth Solution

The primary objective of this example was to validate the suitability of the tensor-train space-time spectral collocation method for solving the time-dependent CDR equation with less regular solutions. To investigate this aspect, we examined the behavior of the methods on a benchmark problem with the constant coefficients $\kappa (t,\mathbf{x})=1$, $b(t,\mathbf{x})=(1,1,1)$, and $c(t,\mathbf{x})=1$, and the non-smooth exact solution $u(t,x,y,z):=sin\left(\pi t\right)(sin(\pi x)sin(\pi y)$$sin\left(\pi z\right)+x^2\left|x\right|)$. As in the previous cases, the computational domain was $[0,1]\times {[-1,1]}^3$.

The exact solution u satisfied ${\displaystyle \frac{d^{2}u(t,\mathbf{x})}{d{\mathbf{x}}^2}}\in H^{\alpha }(0,T;L^2(\Omega ))$ and ${\displaystyle \frac{d^{2}u(t,\mathbf{x})}{d{\mathbf{x}}^2}}\in C^0(\Omega )$ for all $t\in (0,T)$ and $\alpha >0$. Accordingly, Equation (11) implied that the discrete approximation should converge quadratically.

Indeed, we clearly observed such a convergence order in the left panel of Figure 9. Due to the lower regularity of the exact solution, a highly refined mesh was required to achieve high accuracy, which was only feasible with the TT format.

In the middle panel, the plot shows that the SP-SP method was approximately 3500 times faster than the full-grid SP-SP method using 16 nodes, and about $1\times {10}^7$ times faster using 128 nodes, this latter value being extrapolated on the full grid. The compression ratio was approximately $5\times {10}^{-4}$. Since the coefficient functions were the same as in test case 1, the compression of the operators in this test case was similar to the ones shown in Table 1.

Figure 10 compares the TT FD-FD and TT SP-SP methods. The TT SP-SP method exhibited a linear convergence compared to TT FD-FD with smaller approximation errors. Furthermore, as the grid size increased, the TT SP-SP method required more elapsed time. The right panel, which plots the error versus elapsed time, clearly shows that the TT SP-SP method remained more efficient than the TT FD-FD method for this test case with a less regular solution.

In Section 4.3, we investigated a test case with a less regular exact solution. Since the convergence order of the spectral collocation method depends on the regularity of the exact solution (11), our scheme lost the exponential order of convergence and produced a numerical solution at the same order as schemes such as traditional schemes. In the space-time discretization, we treated the nodes corresponding to the initial time point as Dirichlet boundary nodes. The residual did not converge to zero as the polynomial degree tended to ∞ near the boundary. By using the bubble function approach from Chapter 6 of Ref. [21], an a posteriori error analysis could be a suitable choice to recover the higher-order convergence of the numerical scheme.

In this article, we primarily focused on the full-grid and TT scheme for the four-dimensional time-dependent CDR equation, which consists of three space variables x, y, and z and one time variable t. However, the extension of this technique to d dimensions is straightforward. In particular, if $\Omega \subset {\mathbb{R}}^d$, the matricization of all operators can easily be derived.

5. Conclusions

In this work, we introduced a method called the TT spectral collocation space-time (TT-SCST) approach for solving time-dependent convection-diffusion-reaction equations. Time was considered as an additional dimension, and the spectral collocation technique was applied in both space and time dimensions. This space-time approach demonstrated an exponential convergence property for smooth functions. However, it involved solving a large linear system for solutions at all time points. Utilizing the tensorization process, we converted the linear system into TT format and subsequently solved the TT linear system. Our numerical experiments validated that the TT-SCST method achieved significant compression of terabyte-sized matrices to kilobytes, leading to a computational speedup of tens of thousands of times, while maintaining the same level of accuracy as the full-grid space-time scheme.