Mathematical Modeling on a Physics-Informed Radial Basis Function Network

Abstract

The article is devoted to approximate methods for solving differential equations. An approach based on neural networks with radial basis functions is presented. Neural network training algorithms adapted to radial basis function networks are proposed, in particular adaptations of the Nesterov and Levenberg-Marquardt algorithms. The effectiveness of the proposed algorithms is demonstrated for solving model problems of function approximation, differential equations, direct and inverse boundary value problems, and modeling processes in piecewise homogeneous media.

1. Introduction

Partial differential equations (PDEs) describe many physical, biological, and economic processes. The study of models of such processes requires solving boundary value problems. Traditionally used to solve such problems, the finite difference, finite element, and finite volume methods require the construction of meshes, which for real problems is a complex and time-consuming task, and the solution of ill-conditioned systems of high-dimensional mesh equations. Particularly difficult is the solution of inverse boundary value problems when, based on the results of measurements of certain characteristics of an object or process at a limited number of points, it is necessary to calculate the missing parameters of the mathematical description of the object or process. Such problems are mathematically incorrect.

It is well known that boundary value problems for PDEs can be successfully solved using neural networks [1,2]. The theoretical basis for solving PDEs in neural networks consists of two propositions:

-

Using neural networks as universal function approximators [3,4,5]. A neural network approximates an unknown solution to a problem.

-

Using a variational approach to solving boundary value problems. This means that the solution to the boundary value problem is found by minimizing the error functional of the neural network. The neural network’s error functional (often called a loss function) uses the residuals of the approximate solution at some set of test points within, at the boundary of the solution domain, and possibly at additional condition points. As a rule, test points are located in an arbitrary manner (although a grid can be used). Therefore, the solution to a PDE in neural networks is a meshless approximate analytical solution to the PDE, since the solution to the problem is a function (which is almost impossible to represent analytically due to complexity) determined by the architecture and parameters of the neural network. Because PINNs include a mathematical model, the solution results are interpretable.

Increased interest in the application of neural networks to solving PDEs began with the publication of works [6,7,8]. A new class of PDE solvers has been formed—physics-informed neural networks (PINNs), which include a mathematical model in the structure of the neural network. Training of such networks is based on minimizing residuals calculated by substituting an approximate solution in a limited set of test points in the PDE, boundary conditions, and, possibly, additional conditions. That is, the network is trained not on known examples but on the satisfaction of the solution generated by the network with physical laws. Using a PINN, you can solve direct and inverse boundary value problems. When solving direct problems, training data is not used to train a neural network, so there is no problem of overfitting [9], which is a great difficulty for machine learning models. Interest in PINNs is largely due to the popularity of freely available machine learning libraries, such as TensorFlow and PyTorch, which implement automatic differentiation [10], a very important step for solving PINNs.

A PINN is a new direction and is in its infancy. Many problems in the theory and practical application of PINNs have not yet been resolved. One of the important problems hindering the use of neural networks for solving boundary value problems is the long training time of neural networks, which is due to the use of first-order gradient algorithms in modern deep neural networks. For traditional problems solved using neural networks, such as pattern recognition, the network is trained once. The recognition process using a trained network is fast. Solving each boundary value problem requires new network training. Therefore, when solving PDEs with neural networks, training time becomes a critical factor. The solution is to adapt fast second-order algorithms for training neural networks to solve PDEs.

Neural networks can approximate continuous functions as accurately as desired. In the case of modeling processes in inhomogeneous multicomponent media, the solution at the boundary of the interface of piecewise homogeneous media suffers a discontinuity. The need to model processes in heterogeneous media often arises when solving very important applied problems, for example, modeling processes in layered composite materials, modeling oil reservoirs, and modeling groundwater filtration. Neural networks have practically never been used to solve problems with heterogeneous media.

Algorithms for training neural networks when solving inverse boundary value problems, including inverse coefficient problems, especially for piecewise homogeneous media, have been poorly studied. Such problems arise when detecting anomalies in the environment, for example, in medicine.

Most modern PINNs are fully connected deep networks. The use of radial basis function networks as PINNs has great promise [2,11,12,13,14]. Radial basis function networks allow for solving not only direct but also inverse problems [15,16,17]. Such networks are simpler than fully connected ones since they contain only two layers and are easier to train. Second-order gradient learning algorithms have been developed for such networks [13,18,19,20]. The results of comparing fully connected networks and radial basis function networks [21] when solving PDEs showed the advantage of radial basis function networks in terms of training time. Therefore, radial basis function networks designed to solve PDEs are becoming popular. A new type of neural network is being formed—physics-informed radial basis networks (PIRBNs) [12,22,23,24].

Radial basis function networks can be seen as an extension of E.J. Kansa’s method [25,26]. It is a collocation method that uses radial basis functions as basis functions [27]. The Kansa method is meshless, does not require mesh generation like finite difference, finite element, and finite volume methods, and is as accurate as these methods. The Kansa method is inferior in accuracy to spectral methods but does not have restrictions on the shape of the solution region, like these methods. It is superior in accuracy to pseudospectral methods [28], although it should be noted that pseudospectral methods use radial basis functions as basis functions [29]. Radial basis functions depend only on the distance between the input value and the center of the function, so E.J.’s method is applicable to high-dimensional problems.

The difference between radial basis function networks and the method of E.J. Kansa consists of using a number of basis functions that are not equal to the number of collocation points and determining the expansion coefficients (weights of the neural network) during the network training process rather than solving a system of linear algebraic equations. The authors’ approach to training networks of radial basis functions is distinguished by tuning not only the weights but also the parameters of the radial basis functions during the network training process. This approach reduces training time and increases solution accuracy.

The authors of this work adapted second-order gradient algorithms for training PIRBNs [13,18], proposed an approach to solving PDEs in PIRBNs for piecewise homogeneous media [30], and solved coefficient inverse problems in PIRBNs, including problems for piecewise homogeneous media [24]. In the above works, most of the experiments were performed in MATLAB with the preliminary obtaining of analytical expressions for the gradient components of the error function based on the network parameters and the Jacobian matrix. Analytical calculation of derivatives in these expressions is not theoretically difficult but is labor-intensive and prone to errors. Currently, the authors have developed programs for implementing PIRBNs in Python using the automatic differentiation functions of the TensorFlow library.

The purpose of this work is to summarize the authors’ experience in the field of PIRBNs and present the results of using automatic differentiation in PIRBN training.

2. Materials and Methods

The output of a PIRBN is described by the following expression:

$$u\left(x\right)={\displaystyle \sum _{k=1}^{n_{RBF}}{w}_k{\phi }_k\left(x\right)}$$

where $n_{RBF}$ is the number of radial basis functions, $w_k$ is the weight of the $k$th neuron, and ${\phi }_k\left(x\right)$ is the value of the radial basis function at the point $x$.

In this work, the Gaussian function was used:

$$\phi \left(x\right)=\exp \left(-\frac{{‖x-c‖}^2}{2a^2}\right)$$

where $c$ is the function center and $a$ is the form parameter (width).

For radial basis function networks, it is possible to analytically calculate the components of the gradient vector of the error functional and the elements of the Jacobian matrix. Using MATLAB, a preliminary analytical calculation of derivatives was carried out.

We will present the PIRBN parameters as a single vector of network parameters. For input vectors of a network with a dimension equal to two, we have:

$$\theta ={\left[w_{1 }, w_2, \dots , w_{n_{RBF}}, c_{11}, c_{21},\dots , c_{n_{RBF}1}, c_{12}, c_{22},\dots , c_{n_{RBF}2}, a_1, a_2, \dots , a_{n_{RBF}}\right]}^{\mathrm{T}},$$

where $w_j$ is the weight, $j=1, 2, 3, \dots , n_{RBF}$, $n_{RBF}$ is the number of radial basis functions, $c_{j1}$ and $c_{j2}$ are center coordinates, and $a_j$ is the width.

By representing the network parameters as a single vector, it is possible to adjust both the network weights, which are linearly included in the error functional, and the parameters of the radial basis functions, which are nonlinearly included in the error functional.

Let us consider the problem of approximating a function of two variables. To train the network, the Nesterov and Levenberg-Marquardt algorithms adapted by the authors were used. The error functionality for this task has the form:

$$L=\frac{1}{2}{\displaystyle \sum _{j=1}^n{e}_j^2}=\frac{1}{2}{\displaystyle \sum _{j=1}^n{\left(u\left(x_j\right)-T_j\right)}^2},$$

where $e_j$ is the residual at the $j$th test point, $n$ is the number of test points, $x_j$ is the coordinate vector of the $j$th test point, $u(x)$ is the network output for the $j$th test point, and $T_j$ is the known value of the function at the $j$th test point.

In Nesterov’s method [31], the vector of network parameters at each iteration is adjusted according to the following formulas:

$$\begin{gathered} {\theta }^{\left(k+1\right)}={\theta }^{\left(k\right)}+\Delta {\theta }^{\left(k+1\right)}, \\ \Delta {\theta }^{\left(k+1\right)}=\alpha \Delta {\theta }^{\left(k\right)}-\eta \nabla L\left({\theta }^{\left(k\right)}+\alpha \Delta {\theta }^{\left(k\right)}\right), \end{gathered}$$

where $\alpha$ and $\eta$ are selected coefficients and $\nabla L\left({\theta }^{\left(k\right)}+\alpha \Delta {\theta }^{\left(k\right)}\right)$ is the error functional gradient vector.

Nesterov’s method is a development of the gradient descent method with momentum. But in Nesterov’s method, the gradient of a pre-corrected vector of parameters is calculated, which provides a higher speed of convergence.

The gradient components of the functional have the following form:

$$\begin{gathered} \frac{\partial I}{\partial w_i^{}}=\frac{\partial }{\partial w_i^{}}\left[\frac{1}{2}{\displaystyle \sum _{j=1}^n{\left(u\left(x_j\right)-T_j\right)}^2}\right]={\displaystyle \sum _{j=1}^n\left(u\left(x_j\right)-T_j\right)}{\phi }_i\left(x_j\right). \\ \frac{\partial I}{\partial c_{i1}}=w_i{\displaystyle \sum _{j=1}^n\left(u\left(x_j\right)-T_j\right)}\cdot {\phi }_i\left(x_j\right)\cdot \frac{x_{j1}-c_{i1}}{a_i^2}. \\ \frac{\partial I}{\partial c_{i2}}=w_i{\displaystyle \sum _{j=1}^n\left(u\left(x_j\right)-T_j\right)}\cdot {\phi }_i\left(x_j\right)\cdot \frac{x_{j2}-c_{i2}}{a_i^2}. \\ \frac{\partial I}{\partial a_i^{}}=\frac{\partial }{\partial a_i^{}}\left[\frac{1}{2}{\displaystyle \sum _{j=1}^n{\left(u\left(x_j\right)-T_j\right)}^2}\right]=w_i{\displaystyle \sum _{j=1}^n\left(u\left(x_j\right)-T_j\right)}\cdot {\phi }_i\left(x_j\right)\cdot \frac{{‖x_j-c_i‖}^2}{a_i^3}. \end{gathered}$$

The Levenberg-Marquardt algorithm is an adaptation for training PIRBNs of the well-known second-order gradient optimization algorithm [32]. Second-order gradient algorithms have a significantly higher convergence rate than first-order algorithms but are more labor-intensive. Second-order gradient algorithms have not become widespread in deep neural networks. The simple architecture of the PIRBN allows this algorithm to be used for network training. In this algorithm, the correction $\Delta {\Theta }^{\left(k\right)}$ of the vector of network parameters is obtained as a result of solving a system of linear algebraic equations.

$$\left(J_{k-1}^{\mathrm{T}}{J}_{k-1}+{\mu }_{k}E\right)\Delta {\theta }^{\left(k\right)}=-\nabla L_{k-1},$$

(1)

where $J_{k-1}$ is the Jacobian matrix calculated from the values of network parameters in iteration $k-1$, $E$ is the identity matrix, and ${\mu }_k$ is the regularization parameter.

To approximate a function of two variables, we present the Jacobian matrix in block form $J=\left[\begin{array}{cccc}{J}_w¦& J_{c_1}¦& J_{c_2}¦& J_a\end{array}\right]$, where

$$\begin{gathered} J_w=\left[\begin{array}{cccc}\frac{\partial e_1}{\partial w_1}& \frac{\partial e_1}{\partial w_2}& \dots & \frac{\partial e_1}{\partial w_{n_{RBF}}}\\ \frac{\partial e_2}{\partial w_1}& \frac{\partial e_2}{\partial w_2}& \dots & \frac{\partial e_2}{\partial w_{n_{RBF}}}\\ \dots & \dots & \dots & \dots \\ \frac{\partial e_n}{\partial w_1}& \frac{\partial e_n}{\partial w_2}& \dots & \frac{\partial e_n}{\partial w_{n_{RBF}}}\end{array}\right],\hspace{1em}{J}_{c_1}=\left[\begin{array}{ccc}\frac{\partial e_1}{\partial c_{11}}& \cdots & \frac{\partial e_1}{\partial c_{n_{RBF}1}}\\ \frac{\partial e_2}{\partial c_{11}}& \cdots & \frac{\partial e_2}{\partial c_{n_{RBF}1}}\\ \cdots & \cdots & \cdots \\ \frac{\partial e_n}{\partial c_{11}}& \cdots & \frac{\partial e_n}{\partial c_{n_{RBF}1}}\end{array}\right], \\ {J}_{c_2}=\left[\begin{array}{ccc}\frac{\partial e_1}{\partial c_{12}}& \cdots & \frac{\partial e_1}{\partial c_{n_{RBF}2}}\\ \frac{\partial e_2}{\partial c_{12}}& \cdots & \frac{\partial e_2}{\partial c_{n_{RBF}2}}\\ \cdots & \cdots & \cdots \\ \frac{\partial e_n}{\partial c_{12}}& \cdots & \frac{\partial e_n}{\partial c_{n_{RBF}2}}\end{array}\right],\hspace{1em}{J}_a=\left[\begin{array}{ccc}\frac{\partial e_1}{\partial a_1}& \cdots & \frac{\partial e_1}{\partial a_{n_{RBF}}}\\ \frac{\partial e_2}{\partial a_1}& \cdots & \frac{\partial e_2}{\partial a_{n_{RBF}}}\\ \cdots & \cdots & \cdots \\ \frac{\partial e_n}{\partial a_1}& \cdots & \frac{\partial e_n}{\partial a_{n_{RBF}}}\end{array}\right]. \end{gathered}$$

The analytically calculated elements of the Jacobian matrix have the following form:

$$\begin{gathered} \frac{\partial e_i}{\partial w_j}=\frac{\partial }{\partial w_j}\left[u\left(x_i\right)-T_i\right]={\phi }_j\left(x_i\right), \\ \frac{\partial e_i}{\partial c_{j1}^{}}=w_j\cdot {\phi }_j\left(x_i\right)\cdot \frac{x_{i1}-c_{j1}}{a_j^2}, \frac{\partial e_i}{\partial c_{j2}^{}}=w_j\cdot {\phi }_j\left(x_i\right)\cdot \frac{x_{i2}-c_{j2}}{a_j^2}, \\ \frac{\partial e_i}{\partial a_j^{}}=\frac{\partial }{\partial a_j^{}}\left[u\left(x_i\right)-T_i\right]=w_j{\phi }_j\left(x_i\right)\cdot \frac{{‖x_i-c_j‖}^2}{a_j^3}. \end{gathered}$$

Let us consider the solution of the PDE in the PIRBN using the example of solving the two-dimensional Poisson equation.

$$\frac{{\partial }^{2}u}{\partial x_1^2}+\frac{{\partial }^{2}u}{\partial x_2^2}=f\left(x_1,x_2\right),\hspace{1em}\left(x_1,x_2\right)\in \Omega ,$$

(2)

$$u=p\left(x_1,x_2\right),\left(x_1,x_2\right)\in \partial \Omega ,$$

(3)

where $\partial \Omega$ is the region border and $f$ and $p$ are known functions.

$$f\left(x_1,x_2\right)=\sin \left(\pi x_1\right)\cdot \sin \left(\pi x_2\right), p\left(x_1,x_2\right)=0$$

(4)

PIRBN training consists of minimizing the error functional, which is the sum of squared residuals at the internal and boundary test points.

$$L=\frac{1}{2}\left[{\displaystyle \sum _{i=1}^N{\left(\frac{{\partial }^2{u}_i}{\partial x_1^2}+\frac{{\partial }^2{u}_i}{\partial x_2^2}-f_i\right)}^2}+\lambda \cdot {\displaystyle \sum _{j=1}^K{\left(u_j-p_j\right)}^2}\right],$$

where $N$ is the number of test points located inside the solution area, $K$ is the number of test points on the boundary of the solution area, and $\lambda$ is the penalty multiplier.

To train a PIRBN, similarly to the function approximation problem, the components of the gradient vector of the error functional and the elements of the Jacobian matrix can be calculated analytically.

At each iteration of the Levenberg-Marquardt method, it is necessary to solve system (1). Matrix $J_{k-1}^{\mathrm{T}}{J}_{k-1}+{\mu }_{k}E$ of this system is symmetric, positive-definite, and ill-conditioned. Various methods can be used to solve the system. For example, you can apply the direct Cholesky method. To solve high-dimensional systems, it is advisable to use iterative methods, for example, the conjugate gradient method. The complexity of solving systems of linear algebraic equations using direct methods is of the order of $O\left(n^3\right)$, where $n$ is the number of equations in the system and $O\left(n^2\right)$ is the complexity of one iteration of iterative algorithms. Therefore, fast iterative methods have an advantage over direct methods when solving large systems of linear algebraic equations with low accuracy. Our experiments have shown that in the Levenberg-Marquardt method, it is sufficient to solve system (1) up to the relative norm of the residual ${10}^{-2}-{10}^{-3}$, which requires 3–4 iterations for the conjugate gradient method. In addition, Krylov subspace methods, which include the conjugate gradient method, are numerically stable [33], which weakens the influence of poor matrix conditioning. The biconjugate gradient stabilized algorithm has even greater numerical stability [33].

Let us consider solving problems for a piecewise homogeneous medium using the example of a model problem with two regions of the medium.

$$\frac{\partial }{\partial x}\left({\sigma }_i\left(x,y\right)\frac{\partial u}{\partial x}\right)+\frac{\partial }{\partial y}\left({\sigma }_i\left(x,y\right)\frac{\partial u}{\partial y}\right)=f\left(x,y\right),\hspace{1em}\left(x,y\right)\in \Omega ,\hspace{1em}i=1, 2,$$

(5)

$$u(x,y)=p(x,y),(x,y)\in \partial \Omega ,$$

(6)

where $f=\sin (2\pi x_1)\cdot \sin (\pi x_2)$, $p=0$, and properties of media are described by functions ${\sigma }_i$.

The solution region has dimensions $1\times 1$ and is divided at $x=0.5$ into two subregions. At the interface between media s, the interface conditions must be met.

$${\left.u_1\right|}_S={\left.u_2\right|}_S, {\left.{\sigma }_1\frac{d{u}_1}{dx}\right|}_S={\sigma }_2{\left.\frac{d{u}_2}{dx}\right|}_S$$

(7)

In [30], it was proposed to solve problems (5)–(7) in the PIRBN iteratively. Each iteration consists of several steps, the number of which is determined by the number of subareas. At each step, the problem is solved for the corresponding subdomain with the conjugation conditions on the boundary of the subdomain obtained from the approximation of the solution at the previous iteration. The error functional for each subdomain is the sum of squared residuals within the subdomain, in the boundary conditions, and in the conjugation conditions. The condition for the end of the iterative process is small values of the norms of residuals within each subdomain, in the boundary conditions, and in the conjugation conditions.

The inverse coefficient problem of determining the properties of a piecewise homogeneous medium can be solved on radial basis function networks only approximately. It is not possible to accurately determine the position of the boundaries of subregions with different properties of the medium using meshless methods. Let us solve the inverse problem approximately using a continuous function $k\left(x\right)$, describing the environment as follows:

$$\frac{\partial }{\partial x_1}\left(k\left(x\right)\frac{\partial u}{\partial x_1}\right)+\frac{\partial }{\partial x_2}\left(k\left(x\right)\frac{\partial u}{\partial x_2}\right)=f\left(x\right),\hspace{1em}x\in \mathsf{\Omega }$$

(8)

with boundary conditions $Bu(x)=p(x),\hspace{1em} x\in \partial \mathsf{\Omega }$.

At a set of points $z\in Z, Z\subset \mathsf{\Omega }\cup \partial \mathsf{\Omega }$, the solution $u(z)=\psi \left(z\right)$ of the problem for a piecewise homogeneous medium is known and measured with a known error.

Each iteration of solution (8) consists of two steps. In the first step, one iteration of training radial basis function networks approximating function $k\left(x\right)$ is performed.

$$k_{RBF}(x)={\displaystyle \sum }_{m=1}^{M_k}{w}_m^k{\phi }_m^k(x).$$

(9)

At the second step, one iteration of training a network is performed that approximates solution $u$ of problem (8), in which function $k\left(x\right)$ is approximated by the first network:

$$u_{RBF}(x)={\displaystyle \sum }_{m=1}^{M_u}{w}_m^u{\phi }^u(x).$$

(10)

The task error function has the following form:

$$L=\frac{1}{2}{\displaystyle \sum }_{i=1}^N{\left[Au\left(x_i, k\left(x_i\right)\right)-f\left(x_i\right)\right]}^2+\frac{{\lambda }_B}{2}{\displaystyle \sum }_{j=1}^K{\left[Bu(x_j)-p(x_j)\right]}^2+\frac{{\lambda }_D}{2}{\displaystyle \sum }_{m=1}^S{\left[u(x_m)-\psi (x_m)\right]}^2,$$

(11)

where $Au\left(x_i, k\left(x_i\right)\right)$ is the differential operator of the problem (8), $N$, $K$, and $S$ are the number of test points inside the area, on the border, and points of additional conditions, respectively, and ${\lambda }_B, {\lambda }_D$ are the penalty multipliers.

Substituting expressions (9) and (10) into the error functional (11), the components of the gradient vectors and elements of the Jacobian matrices for training networks (9) and (10) are calculated.

Regularization of an ill-posed inverse problem is carried out by the iterative regularization method [34]. To do this, training the networks continues until ${\displaystyle {\displaystyle \sum }_{m=1}^S}{\left[u(x_m)-\psi (x_m)\right]}^2>S{\delta }^2$, where $\delta$ is the known absolute error of the solution at the points of additional conditions.

Analytical calculation of the derivatives of the error functional does not present any fundamental difficulties but is cumbersome and fraught with errors. Therefore, programs in Python were developed that implement PIRBNs, which use the automatic differentiation functions of the TensorFlow library. No graphics accelerator was used. In the future, it is planned to develop custom TensorFlow extensions to implement PIRBNs.

3. Results

The experiments were carried out on a computer running Windows 11 with an Intel(R) Core(TM) i5-12400F 2.50 GHz processor and 16.0 GB of RAM.

Function approximation using a PIRBN was tested in MATLAB using the Franke function [35], which is widely used for testing approximators, as an example (Figure 1a).

$$\begin{array}{l}f\left(x,y\right)=0.75\exp \left(-\frac{{\left(9x-2\right)}^2}{4}-\frac{{\left(9y-2\right)}^2}{4}\right)+0.75\exp \left(-\frac{{\left(9x+1\right)}^2}{49}-\frac{9y+1}{10}\right)+\\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}+0.5\exp \left(-\frac{{\left(9x-7\right)}^2}{4}-\frac{{\left(9y-3\right)}^2}{4}\right)-0.2\exp \left(-{\left(9x-4\right)}^2-{\left(9x-7\right)}^2\right).\end{array}$$

The problem was solved in the $\left(x=0\dots 1, y=0\dots 1\right)$ region using 100 randomly located sample points and 16 radial basis functions. The network was trained using the Levenberg-Marquardt method with an analytical calculation of derivatives.

The network was trained to a root mean square residual of ${10}^{-6}$ in 15 iterations. Figure 1b shows the approximated Franke function. Figure 2 shows radial basis function networks during initialization and after training the network. The diameters of the circles conventionally indicate the shape parameter, and the fill color conventionally displays the weights associated with the radial basis functions.

First-order gradient algorithms, including Nesterov’s method, did not allow the network to be trained to approximate the Franke function. Using Nesterov’s algorithm, it was not possible to achieve a root mean square error less than ${10}^{-1}$. At the same time, Nesterov’s method allows one to well approximate the smooth functions of one variable. Thus, when approximating a sinusoid, a root mean square error of ${10}^{-6}$ was achieved in 70 iterations.

Model problems (2)–(4) were solved using a PIRBN, trained by the Nesterov and Levenberg-Marquardt methods using automatic differentiation. The problem was solved in a unit square using 100 test points located inside the solution area and 40 test points located on the boundary of the area. The network contained 64 Gaussian functions.

On the network trained by the Nesterov method, in 100 iterations, it was not possible to achieve a residual RMSE value even equal to ${10}^{-3}$ (Figure 3a).

On the network trained by the Levenberg-Marquardt method, in 100 iterations, an RMSE residual value of ${10}^{-6}$ was achieved (Figure 4a). The plot of the network solution (Figure 4b) is almost identical to the analytical PDE solution.

Thus, the adapted Levenberg-Marquardt learning algorithm with both analytical calculation of derivatives and numerical differentiation allows training radial basis function networks for solving approximation problems and solving PDEs.

Before solving the inverse problem for a piecewise homogeneous environment, the direct problem for such an environment was first solved on a network trained in MATLAB using the Levenberg-Marquardt algorithm. The problem was solved for constant values of functions $k_1=2$ and $k_2=5$, describing two environments. In each of the two areas, 64 Gaussian functions were used. For each subregion, 60 internal and border test points were used. There were 20 test points along the media interface. The network was trained for 1500 iterations to an RMSE residual value of ${10}^{-12}$.

From the solution of the direct problem, a solution was taken at 40 points located on the grid. These values are taken as additional conditions. To complicate the problem, additional conditions were not taken at the interface between the media. The radial basis function networks for solving the inverse problem contained the same number of basis functions and were trained on the same number of test points as the networks for solving the direct problem.

The results of solving the inverse problem are shown in Figure 5.

As can be seen in Figure 5a, it was possible to well restore the solution to the direct problem. The calculated values of the environment function show the nature of changes in the properties of the environment.

4. Discussion

Although the developed algorithms have been studied on model problems, it can be expected that they will work well, at least for the same classes of problems as the considered model problems.

It is more difficult to theoretically substantiate the convergence of the developed algorithms. These issues are poorly developed for neural networks and are of interest for future research. Research into other learning algorithms is also planned.

In the practical area, it is planned to develop custom extensions of libraries for working with neural networks that implement the proposed and newly developed algorithms. This will expand the ability to solve real problems and the access of other researchers to test the proposed algorithms.