Hybrid Neural Networks for Solving Fully Coupled, High-Dimensional Forward–Backward Stochastic Differential Equations

Abstract

The theory of forward–backward stochastic differential equations occupies an important position in stochastic analysis and practical applications. However, the numerical solution of forward–backward stochastic differential equations, especially for high-dimensional cases, has stagnated. The development of deep learning provides ideas for its high-dimensional solution. In this paper, our focus lies on the fully coupled forward–backward stochastic differential equation. We design a neural network structure tailored to the characteristics of the equation and develop a hybrid BiGRU model for solving it. We introduce the time dimension based on the sequence nature after discretizing the FBSDE. By considering the interactions between preceding and succeeding time steps, we construct the BiGRU hybrid model. This enables us to effectively capture both long- and short-term dependencies, thus mitigating issues such as gradient vanishing and explosion. Residual learning is introduced within the neural network at each time step; the structure of the loss function is adjusted according to the properties of the equation. The model established above can effectively solve fully coupled forward–backward stochastic differential equations, effectively avoiding the effects of dimensional catastrophe, gradient vanishing, and gradient explosion problems, with higher accuracy, stronger stability, and stronger model interpretability.

1. Introduction

The concept of stochastic calculus and related theories was introduced in 1942 by the Japanese mathematician Itô, who later introduced stochastic differential equations (SDEs). The linear backward stochastic differential equation (BSDE) was first introduced in 1973 [1], and since then, the BSDE has long been used as a concomitant equation for stochastic optimal control problems, and the related theory has been continuously developed. In 1990, Pardoux et al. studied general nonlinear BSDE and proved the existence and uniqueness of solutions to such equations, laying the foundation for the development of the theory of forward–backward stochastic differential equation (FBSDE) [2]. Kohlmann et al. [3] attempted to explore the relationship between BSDEs and stochastic control by interpreting BSDEs as certain stochastic optimization problems. The study of forward–backward stochastic differential equations (FBSDEs) originates from the optimal control problem, and the main objective is to study how to reach the desired goal under the given conditions. Pardoux et al. [2] proposed the general stochastic maximum principle, which, under certain assumptions, defines a Hamiltonian function of the following form:

$$H(x,y,z)=\langle y,h(x,v)\rangle +\langle z,\mu (x,v)\rangle +L(x,v)$$

(1)

Here, the dyadic variables, $\left(y_t,z_t\right)$, and the state of the system corresponding to the optimal control, $x_t$, satisfy a stochastic Hamiltonian system of the following form:

$$\left\{\begin{array}{c}\mathrm{d}{x}_t=H_y\left(x_t,y_t,z_t\right)\mathrm{d}t+H_z\left(x_t,y_t,z_t\right)\mathrm{d}{B}_t,\hfill \\ -\mathrm{d}{y}_t=H_x\left(x_t,y_t,z_t\right)\mathrm{d}t-z_t\mathrm{d}{B}_t,\hfill \\ x_0=a,y_T=h_x\left(x_T\right).\hfill \end{array}\right.$$

(2)

The stochastic system consists of a class of fully coupled forward–backward stochastic differential equations and the study of its solvability is of great significance to the stochastic optimal control theory. Antonelli et al. [4] were the first to investigate the existence and uniqueness of FBSDE; since then, Ma et al. [5,6,7,8] have launched a detailed demonstration of the existence and uniqueness of the solution of FBSDE in different forms and dimensions. Zhen et al. [9] proved the existence and uniqueness of fully coupled FBSDE solutions. Now, the application of FBSDE theory in the direction of stochastic optimal control [10,11], financial mathematics [12], partial differential equation theory [13,14], and other directions is constantly deepening, and it has become an important branch of the stochastic analysis field. However, for the FBSDE solution, we usually cannot obtain an explicit solution; therefore, the study of numerical solutions plays a significant role in the development of the related theory.

Nowadays, the main numerical solution methods for FBSDE are as follows:

(1) We use the relationship between the FBSDE and the PDE solution, i.e., the Feynman–Kac formula, to find a numerical solution to the PDE, and thus, a corresponding numerical solution to the FBSDE [5];

(2) Local linearized solution of BSDE using the Picard iterative method [15];

(3) By approximating the Brownian motion and the BSDE generator, we approximate solution of BSDE and, thus, solve the corresponding FBSDE [16];

(4) According to the structure of the FBSDE, we combine the theories of stochastic analysis and scientific computation and use the differential method or integral method for numerical solving [17,18].

The above methods are limited to low-dimensional FBSDEs; for high-dimensional solutions, they inevitably encounter the “dimensional catastrophe” problem [19], i.e., in high-dimensional cases, computational complexity increases exponentially, which will undoubtedly restrict the applicability of FBSDE scenarios. In recent years, deep learning has been applied in different fields, providing new ideas for numerical approximations of high-dimensional functions.

The development of deep learning provides new ideas for numerical solutions of equations. Deep learning has been widely used in various fields in recent years, including computer vision, natural language processing, speech recognition, and recommender systems. At the heart of deep learning is a neural network, a mathematical model consisting of multiple neurons, or nodes, which are interconnected by weights. The universal approximation theorem [20] shows that given an infinite number of neurons in a neural network, it is possible to represent any complex continuous function exactly. In 2017, Han et al. [21] proposed the DBSDE method for solving high-dimensional partial differential equations (PDEs) or backward stochastic differential equations (BSDEs). Ref. [22] extends the DBSDE method to solve high-dimensional FBSDEs by constructing three different network structures based on various feedback forms.

This paper focuses on applying neural networks to the numerical solution computation of high-dimensional fully coupled FBSDEs. The main ideas are as follows:

(1) Constructing neural networks to solve fully coupled, high-dimensional forward–backward stochastic differential equations to effectively avoid falling into the “dimensional catastrophe” problem;

(2) Determining an equivalent stochastic optimal control process with the stochastic process $X(·)$ as the feedback and stochastic process $\left(Y\right(·),Z(·\left)\right)$ as the control pairs; it involves constructing the neural network model to estimate $\left(Y\right(·),Z(·\left)\right)$ at the same time and modifying the structure of the loss function to adapt to the different situations effectively.

(3) Constructing a neural network according to the nature of the FBSDE, the time dimension is introduced, the long- and short-term dependence between sequences is considered, as well as the influence of previous and subsequent time steps. Moreover, the hybrid BiGRU model is constructed, which can reduce error, improve computational accuracy and model interpretability, and at the same time, avoid problems such as “gradient disappearance”, “gradient explosion”, “gradient explosion”, and so on;

(4) Adjusting the internal structure of the neural network involves using a new weight initialization method, considering the gradient relationship between adjacent time steps, adjusting the activation function types, using the residual learning network, and discussing the data normalization process. Through the above adjustments, the accuracy of the model as well as the convergence speed are further improved.

This article is structured as follows:

Section 2 briefly introduces the theory of forward–backward stochastic differential equations and neural networks, briefly reviews the deep learning approach to solving the numerical solution of the BSDE (DBSDE method), and designs a hybrid BiGRU model to solve the equations. Section 3 designs and conducts numerical experiments to compare the experimental results of different models. Section 4 evaluates the models based on the experimental results and theoretical derivations.

2. Materials and Methods

This chapter introduces the basic concepts needed for the article, including the forward–backward stochastic differential equation, deep learning, and neural networks. It provides a brief review of the DBSDE method. We construct a hybrid BiGRU model based on the above study and explain the rationale for the modeling.

2.1. Materials

2.1.1. Forward–Backward Stochastic Differential Equation

In this section, from the structure of the solution and the structure of the neural network, we study the fully coupled forward–backward stochastic differential equation (FBSDE) in the high-dimensional case of numerical solutions.

A fully coupled forward–backward stochastic differential equation defined on $(\Omega ,\mathcal{F},\mathbb{F},P)$ has the following general form:

$$\left\{\begin{array}{c}{X}_t=X_0+{\int }_0^{t}b(s,X_s,Y_s,Z_s)ds+{\int }_0^t\sigma (s,X_s,Y_s,Z_s){dB}_s,\hfill \\ Y_t=\xi +{\int }_t^{T}f(s,X_s,Y_s,Z_s)ds-{\int }_t^T{Z}_s{dB}_s\hfill \end{array}\right.$$

(3)

Where $t\in [0,t]$, $X_0\in {\mathcal{F}}_0$ denotes the initial conditions for the forward stochastic differential equation (SDE), and $\xi \in {\mathcal{F}}_T$ represents the terminal conditions for the backward stochastic differential equation (BSDE). b and $\sigma$ denote the SDE drift coefficient and diffusion coefficient, and f denotes the generator of the BSDE. When $b(·)$ and $\sigma (·)$ do not depend on the solutions ($Y_t$ and $Z_t$) of the BSDE, the above FBSDE simplifies to the following:

$$\left\{\begin{array}{c}{X}_t=X_0+{\int }_0^{t}b(s,X_s)ds+{\int }_0^t\sigma (s,X_s){dB}_s,\hfill \\ Y_t=\xi +{\int }_t^{T}f(s,X_s,Y_s,Z_s)ds-{\int }_t^T{Z}_s{dB}_s\hfill \end{array}\right.$$

(4)

Where $t\in [0,T]$, at this point, the equation is an uncoupled FBSDE.

Assumption 1. 

(1) $A(t,v)$ is consistently Lipschitz continuous with respect to v.

(2) $g\left(x\right)$ on $x\in {\mathbb{R}}^n$ is consistently Lipschitz continuous.

Assumption 2. 

$$\begin{array}{cc}\hfill \langle A(t,v)-A(t,\overline{v}),v-\overline{v}\rangle & \le -{\beta }_1{\left|G\widehat{x}\right|}^2-{\beta }_2\left({\left|G^T\widehat{y}\right|}^2+{\left|G^T\widehat{z}\right|}^2\right),\hfill \\ \hfill \langle g\left(x\right)-g\left(\overline{x}\right),G(x-\overline{x})\rangle & \ge {\mu }_1{\left|G\widehat{x}\right|}^2,\hfill \end{array}$$

where ${\beta }_1,{\beta }_2$ and ${\mu }_1$ are given non-negative constants and satisfy ${\beta }_1+{\beta }_2>0,{\mu }_1+{\beta }_2>0$. Furthermore, when $m>n$, there are ${\beta }_1>0,{\mu }_1>0$.

$$\forall v=(x,y,z),\overline{v}=(\overline{x},\overline{y},\overline{z}),\widehat{x}=x-\overline{x},\widehat{y}=y-\overline{y},\widehat{z}=z-\overline{z},$$

where G is given a full rank matrix, and we define it as follows:

$$v=\left(\begin{array}{c}x\hfill \\ y\hfill \\ z\hfill \end{array}\right),A(t,v)=\left(\begin{array}{c}-G^{T}f\\ Gb\\ G\sigma \end{array}\right)(t,v),$$

$$G\sigma =\left(G{\sigma }_1,\cdots ,G{\sigma }_d\right)$$

When Assumption 1 and Assumption 2 hold, then there exists a unique adapted solution $\left(X\right(·),Y(·),Z(·\left)\right)$ of the FBSDE, and the proof of the theorem is given in [6,7].

2.1.2. Deep Learning and Neural Networks

Deep learning is a machine learning method that attempts to automatically learn and extract information from data by mimicking the workings of the human brain to be able to process and analyze complex data. Neural networks can have multiple layers, each with a set of neurons, where the input layer receives data, the output layer produces results, and the middle layer performs information processing. Commonly used neural network models [23] include feedforward neural networks (FNNs), convolutional neural networks (CNNs), recurrent neural networks (RNNs), generative adversarial networks (GANs), etc. At the current stage, deep learning is still rapidly evolving, and based on the models of the neural networks mentioned above, new models are emerging for different problems.

Recurrent neural networks (RNNs) can be specialized to process sequential data and, unlike traditional feedforward neural networks, have a recurrent structure that allows them to process variable-length input data and maintain memory while processing sequential data. Although RNNs have been widely used, there are still some problems, such as the long-term dependency problem and the difficulty for neural networks to capture long-distance dependencies. This problem can be solved efficiently in the gated recurrent unit (GRU) [24], a model that performs well in solving the gradient vanishing problem and the long-term dependency problem, which can be used to solve long sequence modeling and processing problems. The structure is shown in Figure 1:

BiGRU (bidirectional GRU) is a variant of recurrent neural networks (RNNs) with update gates and reset gates, which control the passing and forgetting of information and help to solve the gradient vanishing problem. It uses two separate GRU layers, one that handles forward (from the beginning to the end of the sequence) inputs and another that handles reverse (from the end to the beginning of the sequence) inputs, so more contextual information can be captured.

2.1.3. The DBSDE Method and Its Generalization

Neural networks rely on combinations of simple functions and have been shown to have good performance and accuracy for high-dimensional problems. Among them, the deep learning-based backward stochastic differential equation method (DBSDE) [21] is used to solve the partial differential equation and forward–backward stochastic differential equation. The method treats the BSDE corresponding to the PDE as a stochastic control problem by means of the Feynman–Kac formulation and discretizes time; the gradient of the solution at individual time steps is used as a strategy function to solve the loss function using gradient descent. The posterior estimation of the DBSDE method was further investigated by Han et al. [25]. Sirignano et al. [26] proposed a DGM method based on deep learning; Huré et al. [27] proposed the inverse dynamic programming method based on deep learning, constructed two formats, DBDP1 and DBDP2, according to the different ways of processing the solution, and provided the corresponding convergence analysis.

The main idea of the DBSDE method for solving high-dimensional PDE or BSDE is as follows:

For the following BSDE:

$$\left\{\begin{array}{c}-d{Y}_t=f\left(t,X_t,Y_t,Z_t\right)dt-Z_{t}d{B}_t\hfill \\ Y_T=g\left(X_T\right)\hfill \end{array}\right.$$

Where $X(·),Z(·)\in M_{\mathcal{F}}^2\left(0,T;{\mathbb{R}}^n\right)$, $Y(·)\in M_{\mathcal{F}}^2(0,T;\mathbb{R})$, $B_t$ is an n-dimensional standard Brownian motion. According to the nonlinear $Feynman--Kac$ formula, the BSDE is associated with a PDE, and the solution $(Y_t,Z_t)$ of the BSDE and the solution $u(t,x)$ of the PDE satisfy the following relation:

$$\left\{\begin{array}{c}{Y}_t=u\left(t,X_t\right),\hfill \\ Z_t=\sigma \left(t,X_t\right)\nabla u\left(t,X_t\right).\hfill \end{array}\right.$$

The BSDE is transformed into a stochastic optimal control problem, whose state equation is the following SDE (assuming that the initial moment is 0).

$$\left\{\begin{array}{c}d{X}_t=b\left(t,X_t\right)dt+\sigma \left(t,X_t\right)d{B}_t\hfill \\ -d{Y}_t=f\left(t,X_t,Y_t,Z_t\right)dt-Z_{t}d{B}_t\hfill \\ X_0=x,Y_0=Y_0\hfill \end{array}\right.$$

Consider the following equivalence control problem:

$$\underset{{\left\{Y_0,Z_t\right\}}_{0\le t\le T}}{inf}\mathbb{E}\left[{\left|Y_T-g\left(X_T\right)\right|}^2\right]$$

Here, the process $Z(·)$ and the initial state $Y_0$ are treated as stochastic controls. By discretizing the forward SDE, the corresponding Euler–Maruyama series can be obtained, as follows:

$$\left\{\begin{array}{c}{X}_{t_{i+1}}^{\pi }=X_{t_i}^{\pi }+b\left(t,X_{t_i}^{\pi }\right)\left(t_{i+1}-t_i\right)+\sigma \left(t,X_{t_i}^{\pi }\right)\left(B_{t_{i+1}}-B_{t_i}\right),\hfill \\ Y_{t_{i+1}}^{\pi }=Y_{t_i}^{\pi }-f\left(t,X_{t_i}^{\pi },Y_{t_i}^{\pi },Z_{t_i}^{\pi }\right)\left(t_{i+1}-t_i\right)+Z_{t_i}^{\pi }\left(B_{t_{i+1}}-B_{t_i}\right),\hfill \\ X_{t_0}^{\pi }=x,Y_{t_0}^{\pi }=Y_0\hfill \end{array}\right.$$

Where $\pi :0=t_0<t_1<\cdots <t_N=T$ is a division of the interval $[0,T]$. $Y_0,Z_0$ are two trainable initial parameters, and a feedforward neural network is constructed to approximate the control $Z_{t_i}^{\pi }$:

$$Z_{t_i}^{\pi }\simeq {\mathcal{NN}}_i\left(X_{t_i}^{\pi };{\theta }_i\right),1\le i\le N-1$$

where ${\theta }_i$ denotes the training parameters at moment $t_i$.

Remark, Neural network approximation refers to the process of using neural networks to approximate complex functions or data. Specifically, neural networks effectively approximate complex nonlinear functions by learning the mapping relationship between inputs and outputs. In this process, the parameters of the neural network are adjusted to minimize the difference between the predicted output and the actual output, enabling the neural network to produce prediction results close to the true output for a given input. The DBSDE method proposed in the references employs neural networks to approximate $Z_{t_i}^{\pi }$.

We define the loss function of the neural network as follows:

$$\mathrm{loss}=\frac{1}{M}\sum _{m=1}^M{\left|Y_{t_N}^{\pi ,m}-g\left(X_{t_N}^{\pi ,m}\right)\right|}^2$$

where M denotes the number of orbital samples for Brownian motion.

The neural network structure is shown in Figure 2; hereafter, and in later drawings, $X_{t_i},Y_{t_i},Z_{t_i}$ is abbreviated as $X_i,Y_i,Z_i$.

Comparing the numerical results of the DBSDE method with the explicit solutions of the equation, it was found that the DBSDE method is not only effective at approximating the explicit solutions of the equations in the low-dimensional case but also in the high-dimensional case; the results are still satisfactory.

Reference [22] generalizes the above DBSDE method to the numerical solution problem of high-dimensional FBSDE. And the algorithms are designed according to three different forms of state feedback with relatively good numerical results. However, for different equations, the three algorithms do not have the same effect, and the structures of the FBSDE and neural network are not considered in the design of the algorithms, reducing the scope of the use of the algorithms and efficiency. This paper is based on the DBSDE method.

2.2. Construction of Hybrid Models

For the following fully coupled FBSDE:

$$\left\{\begin{array}{c}d{X}_t=b\left(t,X_t,Y_t,Z_t\right)dt+\sigma \left(t,X_t,Y_t,Z_t\right)d{B}_t\hfill \\ -d{Y}_t=f\left(t,X_t,Y_t,Z_t\right)dt-Z_{t}d{B}_t\hfill \\ X_0=x,Y_T=g\left(X_T\right)\hfill \end{array}\right.$$

(5)

Where $t\in [0,T]$,$x\in {\mathcal{F}}_0$ is the initial condition for this FBSDE, $X(·),Y(·),Z(·)$ are ${\mathcal{F}}_t$ adapted, and stochastic processes take values of ${\mathbb{R}}^n,{\mathbb{R}}^m,{\mathbb{R}}^m,{\mathbb{R}}^{m\times d}$, respectively.

$$\begin{array}{c}b:\Omega \times [0,T]\times {\mathbb{R}}^n\times {\mathbb{R}}^m\times {\mathbb{R}}^{m\times d}\to {\mathbb{R}}^n\hfill \\ \sigma :\Omega \times [0,T]\times {\mathbb{R}}^n\times {\mathbb{R}}^m\times {\mathbb{R}}^{m\times d}\to {\mathbb{R}}^{n\times d}\hfill \\ f:\Omega \times [0,T]\times {\mathbb{R}}^n\times {\mathbb{R}}^m\times {\mathbb{R}}^{m\times d}\to {\mathbb{R}}^m\hfill \\ g:\Omega \times [0,T]\times {\mathbb{R}}^n\to {\mathbb{R}}^m\hfill \end{array}$$

are given continuous deterministic functions. b and $\sigma$ are the coefficients of the drift term and diffusion term, respectively, and f is the generator.

2.2.1. Equivalent Stochastic Optimal Control

In this section, the FBSDE is solved numerically from the perspective of stochastic optimal control. In the DBSDE method, $Z_t$ is regarded as a control process with states $X_t,Y_t$ as feedback variables, whereas in the actual computation, there are similar convergence results when $Z_t$ is regarded as a control process with state $X_t$ as a feedback variable. In this case, the stochastic optimal control problem is focused solely on the terminal time state, which is effective in certain specific cases. For instance, in the context of option theory, European options only require considering the state at maturity. The equivalent stochastic optimal control problem proposed in our paper not only considers the terminal time state but also considers the state at each time, adjusting between the two parts using the parameter $\lambda$. This effectively expands the model’s applicability; for example, when $\lambda =0$, only the terminal time scenario is considered; when $\lambda =1$, the terminal time and process time are given equal weight, aligning with the financial scenario where option holders have the right to exercise the option at any time. Other scenarios can also be dynamically adjusted through $\lambda$.

Therefore, in this paper, we consider $X(·)$ as the state process and $Y(·)$, $Z(·)$ as the control process; we obtain the equivalent stochastic optimal control process. We consider the following SDE:

$$\left\{\begin{array}{c}d{X}_t=b\left(t,X_t,Y_t,Z_t\right)dt+\sigma \left(t,X_t,Y_t,Z_t\right)d{B}_t\hfill \\ -d{Y}_t=f\left(t,X_t,Y_t,Z_t\right)dt-Z_{t}d{B}_t\hfill \\ X_0=x,Y_0=u_0\hfill \end{array}\right.$$

(6)

Where $t\in [0,T]$; the control $Y(·)$ of the SDE is denoted as $u(·)$ in order to distinguish it from the Y of the backward equation; and $\left(X_0=x,Y_0=u_0\right)$ is the initial state of the equation. Consider the following equivalent stochastic optimal control process:

$$\underset{{\left\{u_t,Z_t\right\}}_{0\le t\le T}}{inf}\mathbb{E}\left[{\left|Y_T-g\left(X_T\right)\right|}^2+\lambda {\int }_0^T{\left|Y_t-u_t\right|}^2\mathrm{d}t\right]$$

(7)

Where $\left(u\right(·),Z(·\left)\right)$ is the ${\mathcal{F}}_t-$ adapted control process with $X(·)$ as the feedback, and $\lambda \ge 0$ is the parameter used to adjust the weighting of two of the above stochastic optimal control processes. In particular, when $\lambda =0$, the stochastic control problem (7) takes $(Y_0,Z(·))$ as the control pair and $\left(X\right(·),Y(·\left)\right)$ as the feedback.

Further, the Euler–Maruyama discrete form of the state equation SDE is obtained as follows:

$$\left\{\begin{array}{c}{X}_{t_{i+1}}^{\pi }=X_{t_i}^{\pi }+b\left(t_i,X_{t_i}^{\pi },u_{t_i}^{\pi },Z_{t_i}^{\pi }\right)\Delta t_i+\sigma \left(t_i,X_{t_i}^{\pi },u_{t_i}^{\pi },Z_{t_i}^{\pi }\right)\Delta B_{t_i},\hfill \\ Y_{t_{i+1}}^{\pi }=Y_{t_i}^{\pi }-f\left(t_i,X_{t_i}^{\pi },Y_{t_i}^{\pi },Z_{t_i}^{\pi }\right)\Delta t_i+Z_{t_i}^{\pi }\Delta B_{t_i},\hfill \\ X_0^{\pi }=x,Y_0^{\pi }=u_0^{\pi }\hfill \end{array}\right.$$

Lemma 1. 

Solving the fully coupled FBSDE (5) is equivalent to solving the stochastic control problem (7), i.e., the stochastic control process (7) is the equivalent stochastic optimal control process of the FBSDE (7).

Proof. 

Let $\left(u\right(·),Z(·)$ be an optimal control pair for a stochastic control process (7), i.e.,

$$\underset{{\left\{u_t,Z_t\right\}}_{0\le t\le T}}{inf}\mathbb{E}\left[{\left|Y_T-g\left(X_T\right)\right|}^2+\lambda {\int }_0^T{\left|Y_t-u_t\right|}^2\mathrm{d}t\right]=0$$

then the terminal conditions, $Y_T=g\left(X_T\right),Y_t=u_t$, are satisfied at this point. Thus, the state Equation (6) can be written in the corresponding BSDE form, i.e., $\left(X\right(·),Y(·),Z(·\left)\right)$ is the solution of the FBSDE (5). Since Assumptions 1 and 2 hold, $\left(X\right(·),Y(·),Z(·\left)\right)$ is the unique solution to the FBSDE (5).

Conversely, a solution for FBSDE (5) exists and is unique when Assumptions 1 and 2 hold. Let the solution at this point be $(X^{*}(·),Y^{*}(·),Z^{*}(·))$,$(u^{*}(·),Z^{*}(·))$ as a control pair of a stochastic control process (7), and we clearly have the following:

$$\underset{{\left\{u_t,Z_t\right\}}_{0\le t\le T}}{inf}\mathbb{E}\left[{\left|Y_T-g\left(X_T\right)\right|}^2+\lambda {\int }_0^T{\left|Y_t-u_t\right|}^2\mathrm{d}t\right]=0$$

That is, stochastic optimal control pairs exist. □

Kohlmann et al. studied the relationship between FBSDE and stochastic optimal control [3], and through the proof of the lemma, it is not difficult to see that under the conditions of Assumptions 1 and 2, solving the FBSDE (5) is equivalent to solving the stochastic optimal control problem (7). If the coefficients $b,\sigma ,f,g$ in the FBSDE (5) satisfy weaker conditions, we can still obtain that the FBSDE (5) is solvable if and only if the stochastic optimal control problem (7) is solvable [28]. In this case, we can still obtain the solution of the FBSDE by solving the stochastic optimal control problem, but the solution of the FBSDE may not be unique. Therefore, the subsequent design of the cost function is also reasonable.

2.2.2. Neural Network Architecture

Both the DBSDE method and its generalization currently use standard fully connected neural networks, while the use of different neural network architectures will likely improve results depending on the type of problem to be addressed. For example, convolutional neural networks are widely used in computer vision and image generation, while recurrent neural networks are more common in natural language processing. Analyzing the issue of solving the high-dimensional fully coupled FBSDE, the sequence data are obtained after discretizing the equations, and it is easy to find that there are dependencies between the sequence time steps. Consider that the output of the previous time step of the SDE affects the input of the next step and that the output of the subsequent time step of the BSDE also affects the input of the previous time step. In this paper, inspired by the DBSDE method, we innovate from the overall architecture of the neural network and the internal structure of each time step of the neural network, which is used to perform a fully coupled, high-dimensional FBSDE solution.

(1) Loss function design.

Equivalent stochastic optimal control process (7). According to the implicit function existence theorem, we have the following:

$$\left\{\begin{array}{c}{u}_{t_i}^{\pi }={\phi }_i^u\left(X_{t_i}^{\pi }\right),\hfill \\ Z_{t_i}^{\pi }={\phi }_i^z\left(X_{t_i}^{\pi }\right).\hfill \end{array}\right.$$

We construct two neural networks to fit $u_{t_i}^{\pi },Z_{t_i}^{\pi }$, i.e.,

$$\left\{\begin{array}{c}{u}_{t_i}^{\pi }\simeq \mathcal{N}{\mathcal{N}}_u\left(X_{t_i}^{\pi };{\theta }_i^u\right),\hfill \\ Z_{t_i}^{\pi }\simeq \mathcal{N}{\mathcal{N}}_z\left(X_{t_i}^{\pi };{\theta }_i^z\right),\hfill \end{array}\right.$$

${\theta }_i^u$ and ${\theta }_i^z$ denote the trainable parameters of the two neural networks at the moment $t_i$, respectively.

We define the loss function of the neural network as follows:

$$\mathrm{loss}=\frac{1}{M}\sum _{m=1}^M\left[{\left|Y_T^{\pi ,m}-g\left(X_T^{\pi ,m}\right)\right|}^2+\lambda \sum _{i=0}^{N-1}{\left|Y_{t_i}^{\pi ,m}-u_{t_i}^{\pi ,m}\right|}^2\Delta t_i\right]$$

Where ${\left|Y_T^{\pi ,m}-g\left(X_T^{\pi ,m}\right)\right|}^2$ denotes loss at the endpoint, ${\left|Y_{t_i}^{\pi ,m}-u_{t_i}^{\pi ,m}\right|}^2\Delta t_i$ denotes loss at each time step, the parameter $\lambda$ effectively adjusts the proportion occupied by the endpoint error and the continuous time process error.

(2) Incorporation of the time dimension.

According to the nature of the sequence after FBSDE discretization, incorporating the time dimension in the neural network can effectively deal with the non-stationary problem and increase the generalization ability of the model.

(3) Based on the BiGRU model.

Due to the Eulerian discretization error, the objective $g\left(X_T\right)$ cannot be reached exactly, and the loss function cannot go exactly to zero. If the neural network’s output is permitted to depend not just on the variable realizations at moment t, but also on the long-term and short-term dependencies, it may be possible to offset the discretization error and find some strategy with a loss function smaller than that of a simple fully connected network. The LSTM and GRU models can effectively address the problem of short-term and long-term sequence dependencies, avoid the problems of “gradient disappearance” and “gradient explosion”, reduce the instability of numerical computation, and improve the convergence of the algorithm. Compared with the LSTM model, the GRU model can effectively limit the number of network parameters that need to be estimated when the temporal discretization increases, requiring a smaller amount of data without significant performance degradation, so the GRU model is used in this paper.

FBSDE is concerned with how to achieve the desired goal under the given conditions, and its solution requires finding the process that satisfies the beginning and end conditions of the time; therefore, when the neural network is constructed, the effects of the previous and subsequent time steps are considered at the same time, and a bidirectional GRU (BiGRU) model is used. Two separate GRU layers process the forward input and the reverse input, respectively, while capturing information related to the previous and subsequent time steps.

2.2.3. Internal Structure of Neural Networks

Analyze the internal structure of the DBSDE method and attempt to improve the algorithm in the following aspects to improve the convergence efficiency, accuracy, and stability of the model.

(1) Weight initialization.

Zero initialization makes it difficult to train the network, while random initialization has the issue of “backpropagation gradient Vanishing” or “forward propagation information flow vanishing”. The original method lacks a connection between the gradients of adjacent time steps, which is obviously not in line with the nature of FBSDE. In this paper, we attempt to initialize the neural network weights and biases to the weights and biases after processing the previous time step.

(2) Batch normalization (BN) processing.

BN processing not only accelerates the convergence speed of the model but also effectively alleviates the problem of dispersed feature distribution in the deep neural network, to avoid the emergence of “gradient explosion” and other problems that cannot be trained, so as to make the training model more stable. Note that BN processing is more applicable to convolutional neural networks, and is difficult to implement in recurrent neural networks (even if applicable, it would only be able to process static signals, not share statistical information about movements over time). The DBSDE method uses feedforward neural networks, so adding a normalization layer (BN layer) before the input of each layer can improve the speed of convergence. At the same time, our experiments have shown that omitting batch normalization in the last layer can effectively improve the accuracy of the results, so it is treated in this way.

(3) Adjust the activation function type.

The original method uses ReLU as the activation function, which has the advantages of unilateral inhibition, relatively wide excitation boundaries, and sparse activation properties. However, its output is not “0-centered”, and “neuron death” may occur as training progresses. In this regard, this paper attempts to adopt ELU as the activation function with an output mean close to 0, thus speeding up the learning rate while avoiding the problem of vanishing gradients through positive identification. Because of the above properties of ELU as an activation function, the batch normalization of data can be appropriately relaxed to reduce the amount of computation.

(4) ELU is used as an activation function while residual learning is incorporated.

Analyzing the batch normalization process and adjusting the activation function type reveal that the results obtained by the two methods are similar, but adjusting the activation function type requires fewer parameters of the neural network, and the computational amount is smaller. Therefore, a residual learning network is added to the ELU as an activation function. The largest difference between the residual network and the previous network is the extra identity shortcut branch, which creates identity shortcut connections between several hidden layers. And because of the existence of this one branch, it allows the network to propagate in reverse; the loss can be passed through this shortcut to pass the gradient directly to the network further ahead, thus slowing down the problem of network degradation and enhancing convergence speed. Other than that, no new parameters are added to the residual network, only an extra step of addition.

(5) Optimizer selection.

For the parameters in the neural network, Adam was chosen as the optimizer to iteratively update the network parameters.

The internal restructuring of the neural network for a single time step is shown in Figure 3:

Figure 3a illustrates the inclusion of batch normalization with ReLU activation for each input layer. However, experiments showed that omitting batch normalization in the final layer yielded better outcomes, as depicted in Figure 3b. Figure 3c represents the modification of the activation function in Figure 3b to ELU and the elimination of batch normalization. Figure 3d showcases the incorporation of residual learning to the structure outlined in Figure 3c.

2.2.4. Construction of the Model

In this section, we construct a hybrid BiGRU model and compare it with the model extended from the DBSDE method.

(1) Extension of the DBSDE method.

The equivalent stochastic optimal control process is still used as proposed in the DBSDE method, with $(X_t,Y_t)$ denoting the feedback variables and $Z_t$ denoting the control, i.e.,

$$\underset{{\left\{Z_t\right\}}_{0\le t\le T}}{inf}\mathbb{E}\left[{\left|Y_T-g\left(X_T\right)\right|}^2\right]$$

A feedforward neural network is used to fit $Z_t$, i.e.,

$$Z_{t_i}^{\pi }\simeq {\mathcal{NN}}_i\left(X_{t_i}^{\pi };{\theta }_i\right),1\le i\le N-1$$

Batch normalization is performed before each layer of input.

The neural network structure of the Extension of the DBSDE method is shown in Figure 4:

In Figure 4, the red line represents the neural network fitting process and the black line represents the data flow.

The pseudocode for the model is shown in

Table 1. Code 1.

Using $({\mathit{X}}_{\mathit{t}},{\mathit{Y}}_{\mathit{t}})$ as the feedback variable
Inputs: initial state x, Brownian motion increment $\Delta B_{t_i}$, learning rate $\eta$ etc.;
Output:$Y_0$
1. $X_0\leftarrow x$
2. $Y_0\leftarrow Y_0$
3. for $i=0,1,...,N-1$ do
4.    $Z_i\leftarrow {\mathcal{NN}}_z\left(X_i,\Delta B_{t_i};{\theta }_i^z\right)$
5.    $X_{i+1}\leftarrow X_i+b(t_i,X_i,Y_i,Z_i)\Delta t+\sigma (t_i,X_i,Y_i,Z_i)\Delta B_t$
6.     $Y_{i+1}\leftarrow Y_i-f(t_i,X_i,Y_i,Z_i)\Delta t+Z_i\Delta B_t$
7. end for
8. $loss\leftarrow \frac{1}{M}{\sum }_{m=1}^M{\left|Y_N-g\left(X_N\right)\right|}^2$
9. $\theta \leftarrow Adam\left(\theta ,\nabla loss\right)$
10: $Y_0=Adam\left(Y_0,\nabla \mathrm{loss}\right)$
11. end for

where M is the number of tracks sampled, variables such as $Y_0$ can be initialized, and so on.

:

(2) Hybrid BiGRU Model

Using $(X_t,Y_t)$ as the feedback variables, we fit the variable $Z_t$ with the BiGRU model; the neural network structure is shown in Figure 5, as follows:

In Figure 5, the red line represents the neural network fitting process and the black line represents the data flow.

The equivalent stochastic optimal control process uses the (7) scheme proposed in this paper. Using $X_t$ as the feedback variable, we construct two different feedforward neural networks to fit the variables, $u_t,Z_t$ respectively, i.e.,

$$\left\{\begin{array}{c}{u}_{t_i}^{\pi }={\phi }_i^u\left(X_{t_i}^{\pi }\right),\hfill \\ Z_{t_i}^{\pi }={\phi }_i^z\left(X_{t_i}^{\pi }\right).\hfill \end{array}\right.$$

The neural network structure is shown in Figure 6, as follows:

In Figure 6, the red line represents the neural network fitting process and the black line represents the data flow.

Combining the above two adjustments, a hybrid BiGRU model is constructed by using $X_t$ as the feedback variable and fitting $u_t$ and $Z_t$ using the BiGRU model, respectively, i.e.,

$u_i:={\mathcal{NN}}_u\left(X_i,\Delta B_{t_i};{\theta }_i^u\right)$

$Z_i:={\mathcal{NN}}_z\left(X_i,\Delta B_{t_i};{\theta }_i^z\right)$

The neural network structure is shown in Figure 7, where the red arrows indicate the neural network fitting process, the black arrows indicate the data flow, the green squares indicate the neural network fitting variables, and the blue squares indicate the structure of BiGRU.

The above network structure can likewise be optimized for a single time step using a similar approach to Figure 3.

The pseudocode for the hybrid BiGRU model is shown in

Table 2. Code 2.

Using ${\mathit{X}}_{\mathit{t}}$ as the feedback variable
Inputs: initial state x, Brownian motion increment $\Delta B_{t_i}$, loss weights $\lambda$, learning rate $\eta$ etc.
Output:$Y_0$
1. $X_0\leftarrow x$
2. $Y_0\leftarrow {\mathcal{NN}}_u\left(X_0;\Delta B_{t_i;{\theta }_0}\right)$
3. $\mathrm{losst}\leftarrow 0$
4. for $i=0,1,...,N-1$ do
5.     $Z_i\leftarrow {\mathcal{NN}}_z\left(X_i,\Delta B_{t_i;{\theta }_i^z}\right)$
6.     $u_i\leftarrow {\mathcal{NN}}_u\left(X_i,\Delta B_{t_i;{\theta }_i^u}\right)$
7.    if $i=0$ then
8.       $Y_i\leftarrow u_i$
9.     else
10.      $Y_{i+1}\leftarrow Y_i-f(t_i,X_i,Y_i,Z_i)\Delta t+Z_i\Delta B_t$
11.      $X_{i+1}\leftarrow X_i+b(t_i,X_i,u_i,Z_i)\Delta t+\sigma (t_i,X_i,u_i,Z_i)\Delta B_t$
12.      $losst\leftarrow losst+{\left|Y_i-u_i\right|}^2$
13. end for
15. $Y_N\leftarrow Y_{N-1}-f(t_{N-1},X_{N-1},Y_{N-1},Z_{N-1})\delta t+Z_{N-1}\delta B_t$
16. $lossT\leftarrow \frac{1}{M}{\sum }_{m=1}^M{\left|Y_N-g\left(X_N\right)\right|}^2$
17. $\mathrm{loss}\leftarrow \mathrm{lossT}+\lambda \mathrm{losst}$
18. $\theta \leftarrow \mathrm{Adam}(\theta ,\nabla \mathrm{loss})$
19. end for

, as follows:

3. Results

In this section, two equations are selected for experimentation.

The neural network has two hidden layers, and the learning rate is adjusted using an adaptive strategy to dynamically improve training effectiveness and stability. The weights of the neural network model are initialized using the Xavier method, and to avoid falling into local optimal solutions, $Y_0$ is initialized. A batch size of $M=300$ is chosen to balance learning efficiency and algorithmic speed, and the number of discrete time steps is $N=100$. The experiments were conducted on a Lenovo Xiaoxin Pro 14ACH 2021x computer, manufactured in Beijing, China. This computer is equipped with an Intel i5 CPU and 16 GB of RAM. The experiments required a setup with Python 3.8 and TensorFlow 2.8 environment.

We evaluate the experimental results and corresponding loss functions of the model using the following metrics:

(1) Relative error on $Y_0$:

$$\mathrm{Relative}\mathrm{error}\mathrm{on}{Y}_0=\frac{\left|Y_{0,\mathrm{est}}-Y_{0,\mathrm{real}}\right|}{\left|Y_{0,\mathrm{real}}\right|}$$

This paper uses relative error to quantify the difference between the true value and the predicted value of $Y_0$. A smaller relative error indicates a closer approximation to the true value, while a larger relative error suggests a greater disparity between the estimated and true values.

(2) Number of iterations to convergence. We recorded the number of iterations required for convergence based on the experimental results obtained to evaluate the training efficiency and performance of the neural network. A smaller number of iterations implies that the network can converge more quickly, reducing training time and improving efficiency. Additionally, the speed of convergence also reflects the extent to which the neural network fits the training data. Generally, faster convergence may indicate better model fitting.

On the one hand, for the hybrid BiGRU model, the following four lifting strategies are validated against the extension of the DBSDE method to prove the effectiveness of the lifting strategies.

• Strategy 1: Using $(X_t,Y_t)$ as the feedback variables, we use a feedforward neural network, adjust the activation function type from ReLU to ELU without batch normalization, and fit the variable $Z_t$;
• Strategy 2: Taking $(X_t,Y_t)$ as the feedback variables, we use the feedforward neural network, adjust the activation function type from ReLU to ELU without batch normalization, and also add residual learning to fit the variable $Z_t$;
• Strategy 3: Using $(X_t,Y_t)$ as the feedback variables, we fit the variable $Z_t$ with the BiGRU model;
• Strategy 4: Using $X_t$ as the feedback variable, we construct two different feedforward neural networks to fit the variables $u_t,Z_t$, respectively.

On the other hand, the experimental results of the extended model based on the DBSDE method are then compared with those of the hybrid BiGRU model, and conclusions are drawn.

3.1. Experiment 1

Suppose $t\in [0,T]$, the initial state is $x=\left(x_1,\cdots ,x_n\right)\in {\mathbb{R}}^d$; $y,z$ take the values of $\mathbb{R},{\mathbb{R}}^d$, respectively, and functions $b,\sigma ,f,g$ are satisfied, as follows:

$$\begin{array}{cc}\hfill b(t,x,y,z)& =0,\hfill \\ \hfill \sigma (t,x,y,z)& =0.25diag\left(x\right),\hfill \\ \hfill f(t,x,y,z)& =0.25\times \left(y-\frac{2+0.{25}^2\times d}{2\times 0.{25}^2\times d}\right)\left(\sum _{i=1}^d{z}_i\right),\hfill \\ \hfill g\left(x\right)& =\frac{exp\left(T+{\sum }_{i=1}^d{x}_i\right)}{1+exp\left(T+{\sum }_{i=1}^d{x}_i\right)},\hfill \end{array}$$

where $diag\left(x\right)$ means that the ith element is the diagonal matrix of $x_i$.

The analytical solution to this FBSDE is

$$Y\left(t,X_t\right)=\frac{exp\left(t+{\sum }_{i=1}^d{X}_{it}\right)}{1+exp\left(t+{\sum }_{i=1}^d{X}_{it}\right)}$$

where $X_t=\left(X_{1t},X_{2t},\cdots ,X_{dt}\right)$.

• the FBSDE set $d=100,T=0.5,x=0.{01}_d$, where ${\mathbf{1}}_d$ is a d-dimensional vector with all elements having a value of one. The explicit solution of $Y_0=0.5$ is a baseline for comparing our models.

3.1.1. Validation of Enhancement Strategies

The values for $d=100$ are calculated as shown in

Table 3. Extension of the DBSDE method and 4 strategies.

Strategy	Numerical Solution	Relative Error	Nb Iterations to cv.
Extension of DBSDE	0.52037	0.04074	$1.38\times {10}^4$
Strategy 1	0.53249	0.06498	$1.25\times {10}^4$
Strategy 2	0.48203	0.03594	$1.27\times {10}^4$
Strategy 3	0.49785	0.00430	$1.15\times {10}^4$
Strategy 4	0.50271	0.00542	$1.33\times {10}^4$

.

Very clearly, the above model and corresponding enhancement strategies are valid. In this case, the relative error significantly decreases for strategies 3 and 4, with a relative error of 0.04074 for the DBSDE extension method, while the relative errors for Strategy 3 and Strategy 4 are 0.00430 and 0.00542, respectively. Regarding the number of iterations required for convergence, the DBSDE extension method requires $1.38\times {10}^4$ iterations, while strategy 3 only requires $1.15\times {10}^4$ iterations. The other three enhancement strategies show varying degrees of decrease in the number of iterations required for convergence.

The trend plot of the results for $Y_0$ is shown in Figure 8:

The trend graph for loss is shown in Figure 9:

From the results of $Y_0$ and loss, it is easy to see that the four strategies show a clear downward trend and tend to be stable, with no overfitting, underfitting, or instability. They can achieve convergence in 14,000 iterations, and the results are valid.

It can be observed from Table 3 and Figure 8 and Figure 9 that all four strategies are effective. The convergence speeds of strategy 2, strategy 3, and strategy 4 are significantly better than that of strategy 1. This indicates that the strategies proposed in this paper, which involve using the equivalent stochastic optimal control problem to construct two separate neural networks for fitting $Y_t$ and $Z_t$, substituting the BiGRU model for the feedforward neural network, and employing residual learning, can be applied effectively in the hybrid BiGRU model.

3.1.2. Results of the Hybrid BiGRU Model and the Extension of the DBSDE Method

From the numerical results in

Table 4. Extension of the DBSDE method and hybrid BiGRU model.

Model	Numerical Solution	Relative Error	Nb Iterations to cv.
Extension of DBSDE	0.52037	0.04074	$1.38\times {10}^4$
Hybrid BiGRU model	0.49762	0.00476	$1.05\times {10}^4$

, it can be seen that both models are valid, but the relative error of the hybrid BiGRU model is significantly smaller, the relative error for the DBSDE extension method is 0.04074, while the relative error for the hybrid BiGRU model is only 0.00476. Additionally, the number of iterations required for convergence has decreased from the original $1.38\times {10}^4$ to $1.05\times {10}^4$.

The trend plot of the results for $Y_0$ is shown in Figure 10:

The trend plot of the results for loss is shown in Figure 11:

From the results of $Y_0$ and loss, it can be seen that both models converge effectively, but the hybrid BiGRU model has relatively smaller losses and converges more efficiently. Meanwhile, from the results, it can be seen that a reasonable initialization of $Y_0$ can effectively improve the efficiency of convergence and avoid falling into local optimal solutions. Since the Brownian motion data can be generated by the random wandering model, the effect of outliers can be temporarily disregarded.

3.2. Experiment 2

Assuming $t\in [0,T],x=\left(x_1,\cdots ,x_d\right)\in {\mathbb{R}}^d,y\in \mathbb{R},z\in {\mathbb{R}}^d$, consider the following fully coupled FBSDE:

$$\left\{\begin{array}{c}d{X}_{i,t}=dexp\left(-\frac{1}{d}\sum _{i=1}^d{X}_{i,t}\right)Z_{i,t}d{B}_{i,t},\hfill \\ -d{Y}_t=-exp\left(-\frac{1}{d}\sum _{i=1}^d{X}_{i,t}\right)\left(\sum _{i=1}^d{Z}_{i,t}^2\right)dt-\sum _{i=1}^d{Z}_{i,t}d{B}_{i,t},\hfill \\ X_0=x,Y_T=exp\left(\frac{1}{d}\sum _{i=1}^d{X}_{i,T}\right),\hfill \end{array}\right.$$

The analytical solution to this FBSDE is as follows:

$$Y(t,x)=exp\left(\frac{1}{d}\sum _{i=1}^d{X}_{i,t}\right)$$

Let $T=0.1,X_0={\mathbf{1}}_d$ and $d=10,d=50,d=100$, then the analytical solution of $Y_0$ is $Y_0\approx 2.7183$.

3.2.1. Validation of Enhancement Strategies

The numerical calculations for $d=100$ are as follows in

Table 5. Extension of the DBSDE method and 4 strategies.

Strategy	Numerical Solution	Relative Error	Nb Iterations to cv.
Extension of DBSDE	2.53740	0.066549	$1.25\times {10}^4$
Strategy 1	2.72427	0.002189	$1.27\times {10}^4$
Strategy 2	2.72198	0.001346	$1.11\times {10}^4$
Strategy 3	2.72229	0.001460	$1.12\times {10}^4$
Strategy 4	2.71323	0.001870	$1.28\times {10}^4$

:

The numerical results above illustrate that the relative errors of the four strategies are significantly smaller, The relative error for the DBSDE extension method is 0.066549, while the relative errors for all four enhancement strategies are no higher than 0.0022. In particular, the relative error for strategy 2 is only 0.001346, and for strategy 3, it is only 0.001460. There is a significant decrease in the relative error. Both the DBSDE extension method and several enhancement strategies exhibit high convergence rates. The number of iterations required for convergence is $1.25\times {10}^4$ for the DBSDE extension method, $1.11\times {10}^4$ for strategy 2, and $1.12\times {10}^4$ for strategy 3. This represents a slight decrease compared to the DBSDE extension method, while the decrease in the number of iterations for the other two strategies is not significant.

The trend graphs for $Y_0$ are shown in Figure 12:

The trend graphs for loss are shown in Figure 13:

From the trend graphs of $Y_0$ as well as loss, it can be seen that the numerical results of the four strategies tend to be stable. The loss function exhibits a significant downward trend, and there is no problem with underfitting or overfitting. Additionally, the loss is smaller when strategy 2, strategy 3, and strategy 4 converge. Combined with the results of relative error, it shows that selecting strategy 2, strategy 3, and strategy 4 to construct the hybrid BiGRU model is reasonable.

3.2.2. Results of the Hybrid BiGRU Model and Extension of the DBSDE Method

The numerical results of the two models for $d=100$ are shown in

Table 6. Extension of the DBSDE method and hybrid BiGRU model.

Model	Numerical Solution	Relative Error	Nb Iterations to cv.
Extension of DBSDE	2.53740	0.066549	$1.25\times {10}^4$
Hybrid BiGRU	2.71602	0.000850	$1.09\times {10}^4$

:

From the above numerical results, compared to the relative error of 0.06549 for the DBSDE extension method, the relative error for the hybrid BiGRU model is significantly lower at 0.000850, indicating a substantial reduction. Additionally, in terms of the number of iterations required for convergence, the hybrid BiGRU model shows a decrease from the original of $1.25\times {10}^4$ to $1.09\times {10}^4$, demonstrating a notable improvement. Overall, the hybrid BiGRU model performs well.

The trend graph for $Y_0$ is shown in Figure 14:

The trend graph for loss is shown in Figure 15:

From the plots of $Y_0$ as well as the trend change of loss, it can be seen that both models can converge effectively. However, the hybrid BiGRU model achieves significantly higher accuracy in predicting $Y_0$, and the value of the loss function at convergence is also significantly smaller. Combined with the results in Table 6, it can be concluded that the hybrid BiGRU model demonstrates relatively better performance in the experiment.

In order to compare the performance effects of different algorithms in different dimensions, we additionally select the dimensions of $d=10,d=50$, and $d=100$ for comparison, and the results are shown in

Table 7. $Y_0$ of different dimensions.

Algorithm	$\mathit{d}=10$	$\mathit{d}=50$	$\mathit{d}=100$
Extension of DBSDE	2.71659	2.72246	2.72427
Hybrid BiGRU	2.71843	2.71948	2.71602

:

A comprehensive analysis of the above results shows that both algorithms are effective in different dimensions.

In low dimensions, the results obtained by the hybrid BiGRU model and the extension of the DBSDE method do not differ much. When $d=10$, the solutions obtained by the two methods are 2.71659 and 2.71843, with relative errors of $6.29\times {10}^{-4}$ and $4.78\times {10}^{-5}$, respectively. However, when $d=100$, the numerical solutions obtained by the two methods are 2.72427 and 2.71602, with relative errors of $2.196\times {10}^{-3}$ and $8.388\times {10}^{-4}$, respectively. In high dimensions, the results of the hybrid BiGRU model are relatively better than those of the extension of the DBSDE method, indicating that the performance of the hybrid BiGRU model is more robust in high dimensions.

In addition, the error in equations is influenced by T, when T increases, the relative error of the algorithm may become larger. At this point, the hybrid BiGRU model will have relatively stable results due to its consideration of time dependence.

4. Discussion

In this paper, the hybrid BiGRU model is proposed to solve the high-dimensional fully coupled FBSDE. The constructed equivalent stochastic optimal process (7) can be simultaneously applied to different feedback forms by parameter tuning, which greatly improves the applicability range of the model. The hybrid BiGRU model is constructed by deeply analyzing the nature of the FBSDE, incorporating the time dimension, and taking into account the interactions between the time steps before and after the equations, as well as the long- and short-term dependencies. The internal structure of the individual time steps of the model is adjusted to further improve the stability of the model operation. At the same time, the hybrid BiGRU model will not be affected by the problems of “dimensional catastrophe”, “gradient vanishing” and “gradient explosion”. We attempted to extend the DBSDE method to the domain of fully coupled FBSDE high-dimensional solutions and refined the original model. Regarding the neural network solution of the FBSDE equation problem, we considered not only the general approximation theorem of neural networks, utilizing feedforward neural networks for solving, but also aimed to customize neural networks based on the equation’s inherent properties and structure.

In terms of the numerical experiments as a whole, first, we verify the effectiveness of the four lifting strategies, indicating that the lifting strategies in the BiGRU model are effective. Then, when comparing the performance of the two models, it can be seen that both models are effective in the high-dimensional case. However, the hybrid BiGRU model exhibits better accuracy, convergence speed, and robustness. Meanwhile, the hybrid BiGRU model has better performance when the dimension is elevated or T is increased. Regarding the influence of T on different methods, we will further investigate in our future work.

In future work, based on the property that the hybrid BiGRU model proposed in this paper can effectively deal with long- and short-term dependencies, we will attempt to extend the model to high-dimensional solutions of non-Markov-driven forward-inverse stochastic differential equations and explore the impact of the long-range memory on non-Markov-driven forward–backward stochastic differential equations. Meanwhile, from the perspective of time dependence, models used for solving high-dimensional PDEs continue to evolve, such as the DGM method, which has achieved good results in high-dimensional numerical PDE solving. In future work, we will consider extending neural network methods, such as DGM, for solving PDEs to the high-dimensional solution of FBSDEs. We will provide a rationale for the use of neural networks and compare the results obtained, continuously enriching the methods for neural network-based solving of high-dimensional FBSDE problems.