Multi-Step Physics-Informed Deep Operator Neural Network for Directly Solving Partial Differential Equations

Abstract

This paper establishes a method for solving partial differential equations using a multi-step physics-informed deep operator neural network. The network is trained by embedding physics-informed constraints. Different from traditional neural networks for solving partial differential equations, the proposed method uses a deep neural operator network to indirectly construct the mapping relationship between the variable functions and solution functions. This approach makes full use of the hidden information between the variable functions and independent variables. The process whereby the model captures incredibly complex and highly nonlinear relationships is simplified, thereby making network learning easier and enhancing the extraction of information about the independent variables in partial differential systems. In terms of solving partial differential equations, we verify that the multi-step physics-informed deep operator neural network markedly improves the solution accuracy compared with a traditional physics-informed deep neural operator network, especially when the problem involves complex physical phenomena with large gradient changes.

1. Introduction

Machine learning, especially deep learning, has received increasing attention due to its breakthrough performance in various cognitive applications. In recent years, the modeling of partial differential equations has been subjected to neural networks (NNs), which can be regarded as universal approximators of nonlinear functions. On this basis, Raissi et al. [1] proposed physics-informed neural networks (PINNs), which changed the conventional method of modeling physical systems. When dealing with physical systems, PINNs utilize available measurement data to obtain a parameterized partial differential Equation (PDE), which provides additional information for NN training [1,2,3,4,5].

PINNs can solve difficult problems in which local boundary conditions are missing, such as the thermal boundary conditions in heat transfer problems [6], or that rely on small displacements [7] to detect holes and defects in materials when performing inference tasks. Recently, many researchers have used PINNs in the field of fluid dynamics. In applied research, Raissi et al. [8] developed hidden fluid dynamics using PINNs to directly extract the velocity and pressure field of a fluid flow from images. Jin et al. [9] used PINNs to precisely code the Navier–Stokes governing equations into deep NNs, overcoming the challenges faced by traditional numerical discrete solutions with incompressible laminar and turbulent flows, such as strong grid dependence, absence of boundary conditions, and the high computational cost of inverse problems. Moreover, using velocity–pressure and vorticity–velocity forms, Navier–Stokes flow networks were developed. Zhu et al. [10] applied PINNs in the modeling of three-dimensional metal processing and required only a small amount of labeled data to learn the dynamic changes in temperature and melt distribution. Song et al. [11] presented a Physics-Informed Neural Networks (PINNs) framework for identifying constitutive parameters in soft materials, achieving under 5% error in complex geometries and noisy conditions. Their approach leverages multi-modal synthetic datasets to ensure robust performance across various testing scenarios. For more research findings, please refer to reference [12]. Jagtap et al. [13] used adaptive activation functions to approximate the solutions of linear and nonlinear partial differential equations. This method has high solution accuracy and fast convergence speed, especially in the early training stage. Xiang et al. [14] observed that the loss function composed of a weighted combination of multiple loss sub-functions plays an important role in training PINNs. Therefore, they proposed an adaptive loss function method based on maximum likelihood estimation, which automatically assigns the weight of the loss by updating the noise parameters at each iteration step. Li et al. [15] proposed the use of the minimax algorithm to adaptively adjust the weights of the loss function based on this foundation. Yu et al. [16] proposed a gradient-enhanced PINN that embeds the gradient information of PDE residuals into the loss function when training PINN models and proved the effectiveness of this approach in forward and reverse PDE problems. To improve the modeling efficiency, Jagtap et al. [17] proposed a conservative PINN over discrete domains based on the nonlinear conservation law. This enables the solution process of PINNs to be parallelized, thus enhancing the solution efficiency. Jagtap et al. [18] then generalized this method to adapt to problems with various complex computational domains. Although PINNs have achieved remarkable effectiveness, they rely on specific boundary conditions, initial conditions, and special source terms during the training stage. This approach requires extensive expertise and incurs significant costs in the inference process. The application of transfer learning under specific conditions can alleviate this limitation [19,20]; although, this often leads to strong targeting of new target domains and a loss of generalization reasoning capability. Thus, PINNs cannot produce inferences in real time under different boundaries, initial conditions, and loads. If a cross-scenario prediction method with generalized reasoning capabilities could be established, the time cost would be almost negligible compared with traditional numerical solvers (such as CFD or solid mechanics simulators).

To establish the generalization reasoning capability of neural networks, Lu et al. [21] proposed the deep operator network (DeepONet). This network is based on the universal approximation theorem of operators [21,22], which has been theoretically proven. DeepONet predefines the input space with input generalization ability before training, such that the trained model can generalize predictions on the input space and achieve the generalization reasoning capability. Additionally, DeepONet is unaffected by the curse of dimensionality in the input space [23], which has great application value in engineering problems. Jin et al. [24] employed a machine learning framework with deep neural operators (DeepONet) to map microstructures to mechanical responses in metamaterials, achieving prediction accuracies within 5–10% from sparse data. This approach marks a significant advance in the inverse design of materials with complex, nonlinear properties. The cost of DeepONet is that it requires numerous expensive experiments or high-fidelity simulation datasets to express the input space, making it difficult for DeepONet to solve PDEs containing variable functions. Moreover, fitting the output function in DeepONet is not guaranteed to satisfy the deep PDEs and only provides a rough approximation of the target solution operator. Thus, DeepONet lacks interpretability at the physical level.

DeepONet and PINNs are complementary approaches. Therefore, Wang et al. [25] combined DeepONet with a PINN to give PIDeepONet, which makes use of NNs and solution function spaces to directly establish mapping relationships. This requires the network to capture very complex and highly nonlinear relationships [26]. Hence, the network learning process can be difficult and false model convergence may occur, meaning that accurate prediction results are not guaranteed.

This article establishes a new method for directly solving arbitrary PDEs. The proposed method does not rely on data and has strong predictive generalization capabilities and high accuracy. Inspired by the latent variable model [27], our approach breaks the direct mapping relationship between PIDeepONet and the solution function space and establishes an intermediate variable model. The model uses latent variables to encode the information bottleneck between the inputs and output and incorporates a multi-step deep network structure. Essentially, our proposed multi-step physics-informed deep operator neural network (MulSPIDeepONet) for solving PDEs combines the advantages of PINNs and DeepONet.

This paper describes the novel NN structure of MulSPIDeepONet, which simplifies the mapping process between variable functions and solution functions to achieve enhanced interpretability and generalization capabilities. The architecture of MulSPIDeepONet can be described as follows. First, the independent variables of the variable function and the solution function are represented by two subnetworks, and we obtain the transition operator of the solution function through the dot product operation. Second, the transition operator of the solution function provides the input to the transition subnetwork. Third, to enhance the information fusion between the network output and the trunk network, we take the dot product of the transition subnetwork and the trunk network to obtain the final solution function. Finally, physics-informed constraints are applied as additional penalty terms in the loss function. MulSPIDeepONet can be easily extended to solve any PDE.

To verify the effectiveness of the MulSPIDeepONet model, we perform data-independent modeling to solve three PDEs. Numerical simulation results demonstrate that the proposed MulSPIDeepONet achieves significantly improved model accuracy and generalization capability compared with the baseline PIDeepONet in solving these PDEs. The effect is especially significant when the model is applied to solve PDEs characterized by large gradient changes and discontinuities.

1.1. Our Contributions

Overall, we make the following three contributions:

• We outline approximation methods for nonlinear functions and nonlinear operators, along with the PIDeepONet, which we consider as the baseline method in this paper.
• We introduce a new structure named MulSPIDeepONet. In this architecture, the outputs from the trunk and branch networks are first merged via a dot product operation to form a transitional operator. This operator serves as an intermediate variable for the solution operator mapping and is used as the input to a transitional subnetwork. Following this, the input function and its associated trunk network are linked with the transitional subnetwork, establishing a connection through the dot product operation. Finally, the output from the transitional subnetwork is combined with that of the trunk network through a dot product operation to produce the final network output. The proposed MulSPIDeepONet framework is designed to be easily adaptable for solving arbitrary PDEs.
• The numerical results from three example PDEs indicate that the proposed MulSPIDeepONet achieves approximately 2–3× improvement in accuracy over the baseline PIDeepONet, particularly in areas with significant gradient changes or discontinuities in the equations.

1.2. Roadmap

The remainder of this paper is organized as follows. In Section 2, we introduce approximation methods for nonlinear functions and nonlinear operators and describe the DeepONet neural operator and physics-informed DeepONet. Finally, the proposed MulSPIDeepONet architecture is presented. We solve three PDEs to verify the effectiveness of the proposed method in Section 3. In Section 4, we summarize the results of this study and provide some ideas for future work.

2. Method

2.1. General Nonlinear Functions and Nonlinear Operator Approximations

In this section, we briefly review nonlinear functions and operators based on NNs and emphasize several key principles of general approximation. Consider the continuous function $f\left(u\right)$ defined on U, where $U\subseteq C[a,b]$, and a bounded function $\sigma$ representing the sigmoid function or some other nonlinear function. For $u\in U$, $f\left(u\right)$ can be approximated as follows [28]:

$$\left|f\left(u\right)-\sum _{i=1}^N{c}_i\sigma \left(\sum _{j=1}^m{\epsilon }_{i,j}u\left(x_j\right)+{\theta }_i\right)\right|<\delta ,$$

(1)

where $\delta$ is a given error range, $c_i,{\epsilon }_{i,j}$, and ${\theta }_i$ are real numbers, $u\left(x_j\right)$ is the j-th value of the input function $u\left(x\right)$ at the m points $\left\{x_1,x_2,\cdots ,x_m\right\}$, N is the number of neurons in the hidden layer, and N is a hyperparameter of the NN. Equation (1) is known as the universal approximation theorem for general linear and nonlinear functions.

Chen et al. [22] further expanded their work on universal approximation theorems for general linear and nonlinear functions to consider dynamical systems. These systems are described by PDEs and are called operators. Specifically, the mapping $G\left(u\right)\left(y\right)$ from function $u\left(x\right)$ to function $G\left(y\right)$ is approximated by the following expression [29]:

$$\left|G\left(u\right)\left(y\right)-\sum _{k=1}^N\left[\sum _{\mathrm{i}=1}^{\mathrm{M}}{c_i}^{k}g\left(\sum _{j=1}^{m_k}{\epsilon }_{i,j}^{k}u\left(x_j\right)+{\theta }_i^k\right)\right]g\left(\sum _{l=1}^n{w}_{k,l}{y}_l+{\zeta }_k\right)\right|<\delta ,$$

(2)

where ${c_i}^k$, ${\epsilon }_{i,j}^k$, ${\theta }_i^k$, $w_{k,l}$, $m_k$, n, and ${\zeta }_k$ are all constants, $i=1,2,\cdots M$, $k=1,2,\cdots N$, $l=1,2,\cdots N$, $x_j$ is the j-th value of the independent variable x at the m points $\left\{x_1,x_2,\cdots ,x_m\right\}$ of the input function, and $y_l$ is the l-th value of the independent variable y at the n points $\left\{y_1,y_2,\cdots ,y_n\right\}$ of the input function. G is a nonlinear continuous operator and g is a nonlinear activation function, such as the sigmoid function or ReLU. N and M are hyperparameters of the NN.

2.2. DeepONet and PIDeepONet Architectures

Inspired by the nonlinear operator approximation theorem, Lu et al. [21] recently proposed the DeepONet architecture. In this architecture, they named the network structures related to the input function u and the output variable y as the branch and trunk networks, respectively. Therefore, the general nonlinear operator approximate expression can be written as follows:

$$\left|\mathrm{G}\left(u\right)\left(y\right)-\sum _{k=1}^N\underset{\mathrm{Branch}\mathrm{Net}}{\underbrace{\left[\sum _{i=1}^M{c_i}^{k}g\left(\sum _{j=1}^{m_k}{\epsilon }_{i,j}^{k}u\left(x_j\right)+{\theta }_i^k\right)\right]}}\underset{\mathrm{Trunk}\mathrm{Net}}{\underbrace{g\left(\sum _{l=1}^n{w}_{k,l}{y}_l+{\zeta }_k\right)}}\right|<\delta .$$

(3)

The subexpressions related to the trunk and branch networks are explicitly divided into stacked and unstacked components according to whether the branch network has multiple subnetworks. Essentially, the unstacked type merges multiple subbranches of the network into one branch (as demonstrated in Figure 1).

Lu et al. also proposed a generalized approximation theorem for operators based on the unstacked DeepONet architecture [21], which is expressed as follows:

$$\left|\mathrm{G}\left(u\right)\left(y\right)-〈\underset{\mathrm{Branch}\mathrm{Net}}{\underbrace{g\left(u\left(x_1\right),u\left(x_2\right),\cdots ,u\left(x_m\right)\right)}},\underset{\mathrm{Trunk}\mathrm{Net}}{\underbrace{\mathrm{f}\left(\mathrm{y}\right)}}〉\right|<\delta .$$

(4)

Wang et al. [25] were inspired by PINNs and the DeepONet deep neural operator. They combined the advantages of DeepONet with those of PINNs to produce PIDeepONet. This architecture embeds the explicit formula of the PDE system into the network loss function as an additional physical constraint through automatic differentiation technology, thereby greatly reducing the need for training data. The network architecture of PIDeepONet is shown in Figure 2. The biggest difference between PIDeepONet and DeepONet is that the network output uses automatic differentiation to formulate an appropriate regularization mechanism, so that the target output function satisfies any given differential constraint. Therefore, the loss function of PIDeepONet is written as follows:

$$L\left(\theta \right)=L_{operator}\left(\theta \right)+L_{physics}\left(\theta \right),$$

(5)

where

$$L_{physics}\left(\theta \right)=\frac{1}{NQ}\sum _{i=1}^N\sum _{j=1}^Q{\left|\mathcal{N}(u^i,G_{\theta }\left(u^i\right)\left(y_{u,j}^i\right))\right|}^2,$$

(6)

and

$$L_{operator}\left(\theta \right)=\frac{1}{NP}\sum _{i=1}^N\sum _{j=1}^P{\left|G_{\theta }\left(u^i\right)\left(y_{u,j}^i\right)-G\left(u^i\right)\left(y_{u,j}^i\right)\right|}^2,$$

(7)

in which $\theta$ represents the set of all trainable weights and bias parameters in the trunk and branch networks, ${\left\{u^i\right\}}_{i=1}^N$ represents the N independent input functions for sampling each $u^i$, and ${\left\{y_{u,j}^i\right\}}_{j=1}^P$ are P positions determined by data observations, initial conditions, or boundary conditions. Additionally, ${\left\{y_{u,j}^i\right\}}_{j=1}^Q$ is a set of collocation points that can be randomly sampled in the domain. Therefore, $L_{operator}\left(\theta \right)$ is used to fit the available solution measurements, while $L_{physics}\left(\theta \right)$ imposes the basic PDE constraints.

We propose the MulSPIDeepONet architecture, which is based on the general nonlinear operator approximation theorem and incorporates the PIDeepONet structure.

2.3. MulSPIDeepONet Architecture

We now describe the MulSPIDeepONet structure, which considers the indirect mapping between the input function and the output function. Through physical constraints, this architecture achieves a better ability to solve arbitrary PDEs with relatively little training data.

MulSPIDeepONet is based on the advanced network architecture of DeepONet, which consists of a branch network, trunk network, and independent transition subnetwork. An independent transition subnetwork is established to learn the transition operator, thus enhancing the information utilization between the variable function and the independent variable of the solution function. This subnetwork also improves the feature recognition of the network with regard to the independent variable.

Our model uses the DeepONet architecture [21] and considers the form of general parametric PDEs. MulSPIDeepONet can be expressed as follows:

$$\left|F\left(u\right)\left(y\right)\left(G\right)-\sum _{s=1}^U\underset{\mathrm{Trunk}\mathrm{Net}}{\underbrace{g\left(\sum _{l=1}^n{w}_{s,l}{y}_l+{\zeta }_s\right)}}\underset{\mathrm{Transition}\mathrm{SubNet}}{\underbrace{g\left(\sum _{q=1}^d{p}_{q,s}G{(u,y)}_q+{\varsigma }_q\right)}}\right|<\delta ,$$

(8)

where $G(u,y)$ is the output of DeepONet:

$$G(u,y)=\sum _{k=1}^N\underset{\mathrm{Branch}\mathrm{Net}}{\underbrace{\left[\sum _{i=1}^M{c_i}^{k}g\left(\sum _{j=1}^{m_k}{\epsilon }_{i,j}^{k}u\left(x_j\right)+{\theta }_i^k\right)\right]}}\underset{\mathrm{Trunk}\mathrm{Net}}{\underbrace{g\left(\sum _{l=1}^n{w}_{k,l}{y}_l+{\zeta }_k\right)}},$$

(9)

in which ${c_i}^k$, ${\epsilon }_{i,j}^k$, ${\theta }_i^k$, $w_{k,l}$, ${\zeta }_k$, $p_{q,s}$, and ${\varsigma }_q$ are the weights and biases of the corresponding network, which are constants. N, M, and U are network hyperparameters. Therefore, MulSPIDeepONet can be expressed as follows:

$$\left|\left.F\left(u\right)\left(y\right)\left(G\right)-〈FNN\left(〈\underset{\mathrm{branch}}{\underbrace{g\left(u\left(x_1\right),u\left(x_2\right),\dots ,u\left(x_m\right)\right)}},\underset{\mathrm{trunk}}{\underbrace{f\left(y\right)}}〉\right)\underset{\mathrm{trunk}}{\underbrace{f\left(y\right)}}〉\right|\right.<\delta ,$$

(10)

where $F\left(u\right)\left(y\right)\left(G\right)$ is the final network output, and $FNN$ represents the transition subnetwork. The main purposes are to simplify the mapping of the network to the solution function and reduce the difficulty of capturing the complex and highly nonlinear relationship. The MulSPIDeepONet network architecture is depicted in Figure 3.

MulSPIDeepONet adopts an unstacked DeepONet structure. First, the outputs from the trunk and branch networks are merged through the dot product operation to obtain the transition operator, which is used as the intermediate variable for mapping the solution operator and forms the input to the transition subnetwork. Through the transition subnetwork, the characteristics of the branch and trunk networks are fused to enhance the utilization of hidden information between the two networks, thereby establishing an indirect mapping relationship with the solution function.

The accuracy of the PINN model is closely related to the physical constraints between the independent variables of the solution function. The key is to enhance the feature fusion of the solution function and the independent variables, as this has a great impact on improving the accuracy of the network model. To this end, the proposed architecture connects the input function and the trunk network with the transition subnetwork, and forms an association through the dot product operation. Finally, the outputs of the transition subnetwork and the trunk network are combined through the dot product operation to obtain the final network output. This improves the physics-informed fusion of the independent variables of the solution function and expands the recognition ability of the physical laws. In this way, MulSPIDeepONet captures regional information with complex physical changes more accurately. Furthermore, the network output embeds physics-informed constraints through automatic differentiation technology, with these physical constraints forming part of the loss function. Therefore, the proposed method not only achieves data-independent training but also explains the output results at the physical level.

MulSPIDeepONet uses automatic differentiation to formulate an appropriate regularization mechanism for the output of the network. This guarantees that the target output function satisfies any given differential constraint. Thus, the loss function is defined as follows:

$$L\left(\theta \right)=L_{operator}\left(\theta \right)+L_{physics}\left(\theta \right),$$

(11)

where

$$L_{physics}\left(\theta \right)=\frac{1}{NQ}\sum _{i=1}^N\sum _{j=1}^Q{\left|\mathcal{N}(u^i,F_{\theta }\left(u^i\right)\left(y_{u,j}^i\right)\left(G_{u,y,j}^i\right))\right|}^2,$$

(12)

and

$$L_{operator}\left(\theta \right)=\frac{1}{NP}\sum _{i=1}^N\sum _{j=1}^P{\left|F_{\theta }\left(u^i\right)\left(y_{u,j}^i\right)\left(G_{u,y,j}^i\right)-F\left(u^i\right)\left(y_{u,j}^i\right)\left(G_{u,y,j}^i\right)\right|}^2,$$

(13)

in which $\theta$ represents the set of all trainable weights and bias parameters in the trunk network, branch network, and transition subnetwork. The variables ${\left\{u^i\right\}}_{i=1}^N$, ${\left\{y_{u,j}^i\right\}}_{j=1}^P$, and ${\left\{y_{u,j}^i\right\}}_{j=1}^Q$ are the same as in Equations (6) and (7). $L_{operator}\left(\theta \right)$ is used to fit the available solution measurements, while $L_{physics}\left(\theta \right)$ imposes the basic PDE constraints.

3. Experimental Verification

This section presents the modeling results of MulSPIDeepONet. We verify the effectiveness of the proposed method by solving three different PDE systems and compare the results with those from the baseline NN model. To ensure fairness, all networks in the following examples have a similar number of trainable parameters or weights, facilitating a more accurate comparison of the generalization performance of the model. Additionally, fixed random seeds are used to minimize the effects of randomness.

The baseline model in this paper is the classic PIDeepONet architecture. All test results are obtained using an NVIDIA 3060 Ti GPU based on the open-source Jax deep learning framework, and the ground truth results of the PDEs are obtained by analytical numerical simulations. To comprehensively evaluate the fitting ability of the constructed model in solving PDEs, we select several performance indicators. These indicators are the mean absolute error (MAE), relative mean absolute error (RMAE), relative root mean square error (rRMSE), and L2 norm error:

$$MAE=\frac{1}{n}\sum _{i=1}^n\left|y_i^{true}-y_i^{predict}\right|,$$

(14)

$$RMAE=\frac{1}{n}\sum _{i=1}^n\left|\frac{y_i^{true}-y_i^{predict}}{y_i^{true}}\right|,$$

(15)

$$rRMSE=\frac{\sqrt{\frac{1}{n}{\displaystyle \sum _{i=1}^n}{\left(y_i^{true}-y_i^{predict}\right)}^2}}{y^{average}},$$

(16)

$$L2norm\mathrm{error}=\sqrt{\frac{1}{n}\sum _{i=1}^n{\left(y_i^{true}-y_i^{predict}\right)}^2},$$

(17)

where n is the number of cells in the computational grid.

3.1. Advection Equation

We first study the ability of PIDeepONet and MulSPIDeepONet to process advection equations. The physical process of such PDEs in fast flowing media (such as a gas or liquid) is somewhat complex, so the solutions to numerical simulations become complicated. In particular, sharp gradients or non-physical oscillations of the solution may occur [30]. The traditional reduced-order model also faces significant challenges in obtaining a solution [31]. Therefore, this paper considers a linear advection partial differential system with simple hyperbolic variable coefficients:

$$\frac{\partial s}{\partial t}+u\left(x\right)\frac{\partial s}{\partial x}=0,(x,t)\in (0,1)\times (0,1).$$

(18)

The initial boundary conditions are as follows:

$$\mathrm{s}(x,0)=sin\left(\pi \mathrm{x}\right),$$

$$\mathrm{s}(0,t)=sin\left(\frac{\pi }{2}\mathrm{t}\right),$$

where $u\left(x\right)$ is a variable function. To make the input function $u\left(x\right)$ strictly positive, we let $\mathrm{u}\left(x\right)=\nu \left(x\right)-{min}_x\nu \left(x\right)+1,$ where $\nu \left(x\right)$ is sampled from a Gaussian random field. The data used to train the MulSPIDeepONet model are obtained by sampling the random function $\nu \left(x\right)$ generated by the Gaussian random field [25]. The generalization performance of the model is better verified by evaluating the prediction accuracy using the variable function $u\left(x\right)$, which is not included in the training process. The training goal is to learn the solution operator that maps the variable coefficient (variable function) $u\left(x\right)$ to the solution $s(x,t)$.

The model learning rate is set as 0.001, and the Adam optimizer is used. The training set includes 1000 groups of training data. The specific network size settings are listed in

Table 1. Network settings.

Network Name	Subnetwork Categories	Network Setting
PIDeepONet	Trunk Net	[2,100,100,100,100,100,100]
	Branch Net	[100,100,100,100,100,100,100]
MulSPIDeepONet	Trunk Net	[2,100,100,100,100]
	Branch Net	[100,100,100,100,100]
	Transition SubNet	[1,100,100,100,100,100]

, and the network settings determine the depth of each network and the number of neurons in each layer. The results for the residual loss (Loss-Res) and boundary loss (Loss-Bcs) during the training process are shown in Figure 4.

It can be seen from Figure 5 that PIDeepONet trained by physics-informed constraints is in good agreement with the reference PDE solution derived under the conditions specified in Figure 6, but the accuracy needs to be improved when dealing with PDE systems with rigid, turbulent, or chaotic dynamics [25]. For example, in this test case, the PIDeepONet model produces obvious errors in areas with large gradient changes (such as the area in the red triangle in Figure 5), and the average absolute error is 0.0218. Using the proposed method, the average absolute error of the test results is 0.0083 in the area where the gradient changes significantly. More comparative results are given in

Table 2. Test results for solving linear advection equations. MulSPIDeepONet is superior to the baseline model in all indicators.

Test	Model	MAE	rMAE	rRMSE
Test1	PIDeepONet	0.0218	0.118	0.091
	MulSPIDeepONet	0.0083	0.004	0.039

, and additional experimental results are shown in Figure 7 and Figure 8, where Figure 7 and Figure 8 correspond to the solutions under the functions presented in Figure 9 and Figure 10, respectively.

In Figure 7, the average absolute error between the prediction results of PIDeepONet and the analytical solution in the region with large gradient changes is 0.0337, while the average absolute error between the prediction results of MulSPIDeepONet and the analytical solution in this region is 0.0108. In Figure 8, the average absolute error between the prediction results of PIDeepONet and the analytical solution in the region with large gradient changes is 0.0307, while the average absolute error between MulSPIDeepONet and the analytical solution in this area is 0.0070. More comparative indicators are presented in

Table 3. Test results for solving linear advection equations. MulSPIDeepONet is superior to the baseline model in all indicators.

Test	Model	MAE	rMAE	rRMSE
Test2	PIDeepONet	0.0337	0.117	0.111
	MulSPIDeepONet	0.0108	0.038	0.033
Test3	PIDeepONet	0.0307	0.177	0.167
	MulSPIDeepONet	0.0070	0.040	0.039

.

The MulSPIDeepONet architecture represents an improvement over the PIDeepONet method. First, by sharing the independent variable information in the variable function and the solution function, and then applying feature fusion between these two, an indirect mapping of the solution function is efficiently achieved. This method simplifies the task of the network in capturing extremely complex and highly nonlinear relationships, which not only simplifies the learning process of the network, but also improves the ability to identify and analyze complex physical phenomena. In dealing with large gradient changes, as are often encountered in physical problems, the experimental results show that the proposed method effectively captures and reveals the complex physical processes behind the phenomenon.

Through comparisons and analysis of the solution results, we find that the MulSPIDeepONet architecture proposed in this paper has an obvious effect in reducing the error in large-gradient regions. Compared with the baseline model, the error is reduced by a factor of three, which reflects the superior generalization performance of MulSPIDeepONet compared with the baseline model. This improvement provides a new perspective and solution for dealing with complex physical problems in regions with large gradient changes. In general, the MulSPIDeepONet architecture has unique advantages and higher-level analysis capabilities in the face of complex physical scenes with large gradient changes. Thus, it provides a new approach for deep learning in the field of physical problems.

3.2. One-Dimensional Parametric Euler Equation

The conservation of mass, momentum, and energy in a compressible, inviscid flow can be described by the Euler equation. The conservative form of the Euler equation can be written as follows [32]:

$${\partial }_{\mathrm{t}}U+\nabla ·f\left(U\right)=0,x\in \Omega \subset R^d,d=1,2,t\in (0,T].$$

(19)

The one-dimensional Euler equation has the following form:

$$\mathrm{U}=\left(\begin{array}{c}\rho \hfill \\ \rho u\hfill \\ \rho E\hfill \end{array}\right),f\left(U\right)=\left(\begin{array}{c}\rho u\hfill \\ \rho u^2+p\hfill \\ u(\rho E+p)\hfill \end{array}\right),$$

(20)

where $\rho$ is the density, p is the pressure, u is the velocity, and E is the total energy. To close the equation, a state equation describing the relationship between pressure and energy is introduced. In this paper, we consider the state equation for an ideal gas:

$$p=(\gamma -1)\left(\rho E-\frac{1}{2}\rho \left|\right|\mathrm{u}|{|}^2\right),$$

(21)

where $\gamma =1.4$ is the adiabatic coefficient.

In this class of PDE, even if the initial conditions are smooth, the solutions will be discontinuous at some finite time [33]. In such a case, it is difficult to obtain analytical solutions. Thus, numerical methods are widely used to find approximate solutions to such equations. The goal is to perform non-oscillatory reconstruction around the discontinuities. Therefore, this example aims to verify the ability of PIDeepONet and MulSPIDeepONet to deal with one-dimensional Euler equations with moving contact discontinuities [34].

In this example, the range of the computational domain x is [0, 1], the initial discontinuity is at $x=0.5$, and the left and right sides of the discontinuity can be described by

$$({\rho }_L,u_L,p_L)=(a,0.1,1.0),({\rho }_R,u_R,p_R)=(1.0,0.1,1.0),$$

(22)

where a is a variable parameter. To reduce the computational overhead, the value range for the training process is $a\in [1.2,5.4]$. We use Dirichlet boundary conditions and specify the exact solution as follows:

$$\rho (x,t)=\left\{\begin{array}{c}a,x<0.5+0.1t,\hfill \\ 1.0,x>0.5+0.1t,\hfill \end{array}\right.u(x,t)=0.1,p(x,t)=1.0.$$

(23)

Sampling different points on a hypercube within the range of a, we obtain sufficient training data. The prediction accuracy is represented by the variable parameter, which is not used during the model training. The training goal is to learn the solution operator that maps the variable parameter a to the solution of Equation (19). Sixty randomly distributed boundary points are used for training. The number of initial training points is 60 and the number of function training points is 1000. The distribution of training points under each set of training parameters is shown in Figure 11.

In this example, the learning rate is set to 0.005, and the Adam optimizer is used to train the model for 60,000 steps. The exponential decay learning strategy is adopted. The learning rate attenuates every 3000 steps, and the attenuation rate is 0.95. The training set includes 50 sets of training data randomly sampled in the range $a\in [1.2,5.4]$. The specific network size settings are listed in

Table 4. Network settings.

Network Name	Subnetwork Categories	Network Setting
PIDeepONet	Trunk Net	[2,200,200,200,200,200,100]
	Branch Net	[1,200,200,200,200,200,300]
MulSPIDeepONet	Trunk Net	[2,200,200,200,200,100]
	Branch Net	[1,200,200,200,200,200,300]
	Transition SubNet	[3,100,100,300]

, which displays the depth of each network and the number of neurons in each layer. The results for the Loss during the training process are shown in Figure 12.

Figure 13 shows the density and velocity curves of Equation (19) solved by PIDeepONet and MulSPIDeepONet at $t=2.0$ for $a=4.4$ without training. The experimental results show that the velocity curves of PIDeepONet and MulSPIDeepONet are almost consistent with the analytical solution, but the density curve given by PIDeepONet at the discontinuity has obvious errors. In contrast, MulSPIDeepONet makes full use of the information fusion between the trunk network and the branch network and enhances the feature fusion between the final output of the network and the independent variables of the solution function. Thus, it captures the occurrence of discontinuities more accurately, and the density curve in the discontinuous area exhibits better fitting performance.

Figure 14 shows the changes in the density field with time when $a=4.4$. Panels (c) and (d) indicate that the proposed method is closer to the analytical solution at the discontinuity, demonstrating that MulSPIDeepONet has better processing ability in the face of strong discontinuities.

Figure 15 shows the predicted density field given by PIDeepONet and MulSPIDeepONet when $a=5.0$ without training. The average absolute error between the PIDeepONet results and the analytical solution at the discontinuity (area framed by the red rectangle) is 0.7726. It is obvious that PIDeepONet dissipates significantly at the discontinuity. However, the average absolute error between the MulSPIDeepONet results and the analytical solution in this region is just 0.1918, which is very impressive. For more test error results, see

Table 5. Test results for solving one-dimensional parametric Euler equation. MulSPIDeepONet is superior to the baseline model in all indicators.

a	Model	MAE	rMAE	rRMSE
$a=4.4$	PIDeepONet	0.6567	0.2432	0.3177
	MulSPIDeepONet	0.1704	0.0631	0.1583
$a=5.0$	PIDeepONet	0.7726	0.2575	0.3308
	MulSPIDeepONet	0.1918	0.0639	0.1650

.

The experimental results show that PIDeepONet produces obvious errors at the discontinuity. MulSPIDeepONet provides better compensation for the dissipation at the discontinuity, resulting in predictions that are about three times better than PIDeepONet and improved generalization ability. This experimental result confirms that MulSPIDeepONet can effectively capture and reveal the processes behind complex physical phenomena with moving contact discontinuities. This is an important achievement for theoretical research.

3.3. Eikonal Equation

This example aims to highlight the ability of PIDeepONet and MulSPIDeepONet to handle different input functions in two dimensions. We consider the following two-dimensional eikonal equation:

$$\begin{array}{c}{∥\nabla \mathrm{s}\left(\mathrm{x}\right)∥}_2=1,\hfill \\ \mathrm{s}\left(\mathrm{x}\right)=0,\mathrm{x}\in \partial \Omega ,\hfill \end{array}$$

(24)

where $\mathrm{x}=(x,y)\in R^2$ denotes two-dimensional space coordinates, and $\Omega$ is an open domain with a piecewise smooth boundary $\partial \Omega$. The solution to Equation (24) is a signed function measuring the distance from the point in $\Omega$ to the nearest point on the boundary $\partial \Omega$. For example,

$$\mathrm{s}\left(\mathrm{x}\right)=\left\{\begin{array}{c}\mathrm{d}(\mathrm{x},\partial \Omega ),\mathrm{x}\in \Omega ,\hfill \\ -\mathrm{d}(\mathrm{x},\partial \Omega ),\mathrm{x}\in {\Omega }^C,\hfill \end{array}\right.$$

(25)

where $\mathrm{d}(·,·)$ is defined as the distance function. The purpose of this example is to explore an effective method for mapping the closed curve F to its corresponding signed distance function while satisfying the eikonal Equation [25]. In this paper, circular boundaries with different radii centered on the origin are considered to fully explore the characteristics of the mapping process. The training data are a random selection of 1000 circular boundaries of different radii taken from a uniform distribution, and the test data are the circular boundaries of different radii that are not involved in the training process. In this example, the learning rate is set as 0.001, and the Adam optimizer is used. The specific network size is listed in

Table 6. Network settings.

Network Name	Subnetwork Categories	Network Setting
PIDeepONet	Trunk Net	[2,50,50,50,50,50,50]
	Branch Net	[200,50,50,50,50,50,50]
MulSPIDeepONet	Trunk Net	[2,50,50,50,50,50]
	Branch Net	[200,50,50,50,50,50]
	Transition SubNet	[1,50,50,50]

. The results for the residual loss (Loss-Res) and boundary loss (Loss-Bcs) during the training process are shown in Figure 16.

The test results are displayed in Figure 17, indicating that PIDeepONet achieves good consistency between the exact value and predicted value of the signed distance function. The L2 norm error of the test result given by the proposed method is 0.001904, whereas that of the baseline model is 0.002975. The error is significantly reduced by using MulSPIDeepONet. Additional test results and error metrics are presented in Figure 18 and

Table 7. Test results for solving Eikonal equations. MulSPIDeepONet is superior to the baseline model in all indicators.

	Model	L2 Norm Error	MAE
Radiu 1	PIDeepONet	0.00313	0.00125
	MulSPIDeepONet	0.00153	0.00064
Radiu 2	PIDeepONet	0.00523	0.00122
	MulSPIDeepONet	0.00378	0.00083

, illustrating that the model proposed in this study outperforms the baseline model.

This experimental result again proves that, through its network enhancement strategy, the proposed method has an improved ability to identify the independent variable characteristics of the solution function by integrating the characteristics of the trunk network and the branch network. This unique network architecture and feature processing strategy give MulSPIDeepONet better generalization ability and solving effect in dealing with PDEs, which verifies the effectiveness of the network structure design and feature fusion strategy proposed in this work.

In this section, three different PDE systems have been solved and analyzed. The proposed MulSPIDeepONet indirectly constructs the mapping relationship between the variable function and the solution function and enhances the feature recognition of the solution function to the independent variables. Thus, it has stronger ability to solve PDEs, especially systems with large gradient changes, moving contact discontinuities, and large changes in physical characteristics.

The reasons are as follows. First, the baseline model completely separates the trunk network and the branch network, such that the information between the independent variables of the variable function and the solution function cannot be fully utilized. This results in the loss of available information. In contrast, the proposed method accurately captures the trend of changes in physical characteristics. Second, MulSPIDeepONet enhances the identification of the independent variable information in the solution function, which helps prevent the loss of information as the network layer deepens during training. Third, the potential variables are used to encode the information bottleneck between input and output. Therefore, this network structure exhibits unique advantages in capturing complex physical phenomena, dealing with regions with drastic gradient changes, and accurately reflecting discontinuities in the system.

4. Conclusions and Prospects

This paper has proposed the MulSPIDeepONet architecture. This structure changes the direct mapping method between the variable function and the solution function in classical network models. Instead, our method uses an indirect mapping to find the solution function. This simplifies the solution process when faced with complex, highly nonlinear relationships, and accelerates the network learning process. By enhancing the close relationship between the independent variables and the solution function, the feature recognition ability of the model is dramatically improved at the physical level, which enhances the ability to solve PDE systems.

The results of three example PDEs show that MulSPIDeepONet has stronger generalization ability than the classical PIDeepONet model. In the solution of PDEs, the prediction accuracy of MulSPIDeepONet is nearly three times greater than that of PIDeepONet, especially in the presence of large gradients and strong discontinuities. Therefore, the network structure proposed in this paper offers unique advantages, not only capturing complex physical phenomena but also handling regions with dramatic gradient changes. Moreover, it can accurately reflect the discontinuities within a system.

Future work will focus on the following aspects. First, we will introduce additional indirect explicit constraint information into the transition subnetwork architecture, such as the explicit constraint information in the Green’s function. Second, we will apply this model to more complex PDE scenarios, such as turbulent flows and ultra-high-speed flow field predictions, to verify the advantages of the proposed approach.