Deep Theory of Functional Connections: A New Method for Estimating the Solutions of Partial Differential Equations

Round 1

Reviewer 1 Report

The paper is based on references [8] (Mortari, D. The Theory of Connections: Connecting Points.) and [12] (Mortari, D.; Leake, C. The Multivariate Theory of Connections) as far as modeling the solution with the proper boundary properties is concerned.

The PDE solution procedure follows closely reference [5] (Lagaris, I.E.; Likas, A.; Fotiadis, D.I. Artificial neural networks for solving ordinary and partial differential equations).

The article under review contains an application of the ideas in [5,12] on a number of test problems.

There are many papers, even books (An Introduction to Neural Network Methods for Differential Equations, by Neha Yadav, Manoj Kumar & Anupam Yadav) that illustrate similar ideas to solve PDEs. So novelty is a weak point.

What I would consider as a novel element would be treating PDEs with interface conditions that may involve discontinuities. For example, a simple heat conduction problem between two touching slabs of different thermal conductivities.  There, at the interface where the slabs touch each other, the temperature is continuous but its derivative is not.

Neural networks are continuous functions and so are their derivatives. This has not been worked out (as far as I know) and would contribute to the novelty of the article.

One other issue is the multiple local minima of the “error” (or “loss”) function. There is no discussion regarding the decision criteria for which one to choose.

All the local optimization methods (such as ADAM, BFGS, etc) return only a local minimum. Among several “low” local minima, the generalization performance may be quite different.

Is there a technique used, such that the optimization is guided toward a network that generalizes well, or many trials are performed and the best network is retained?

The paper is well organized and I consider it quite illustrative.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report

The manuscript presents a new methodology to estimate the solutions of partial differential equations (PDEs) by combining neural networks with the Theory of Functional Connections (TFC). The authors present a fairly original method to solve (or rather estimate) numerically the PDE solution, by minimizing the numerical residue of the solution. Although they are not the first to use neural networks to solve PDE (see for example reference [Sirignano & Spiliopoulos, 2018]), there are few and recent works related to this topic, and the originality of the present work is to combine these neural networks with the TFC method which is a method used into unconstrained optimization.

The method is well explained and pedagogically exemplified, however a more pronounced study of the effectiveness of the method, whether theoretical or numerical, is missing. At least one convergence result of the method could be shown (as shown for example in reference [Sirignano & Spiliopoulos, 2018]). If the authors do not feel qualified to give a theoretical result such as a convergence or error estimation theorem, they should at least do a comparative study with the more classical methods such as FEM, and a numerical study of the stability of the method.

In short, this article is suitable for publication in MAKE, after adding these numerical studies that are recommended in this review,

 

Author Response

Please see the attachment.

Author Response File: Author Response.pdf