Abstract
In this article, we study the multivariate quantitative smooth approximation under differentiation of functions. The approximators here are multivariate neural network operators activated by the symmetrized and perturbed hyperbolic tangent activation function. All domains used here are infinite. The multivariate neural network operators are of quasi-interpolation type: the basic type, the Kantorovich type, and the quadrature type. We give pointwise and uniform multivariate approximations with rates. We finish with illustrations.
Keywords:
symmetrized and perturbed hyperbolic tangent activation function; multivariate quasi-interpolation neural network operators; multivariate quantitative approximations MSC:
41A17; 41A25; 41A36
1. Introduction
The author of [1,2] (see Chapters 2–5) was the first to establish neural network approximation to continuous functions with rates by very specific neural network operators of Cardaliaguet–Euvrard and “squashing” types, by using the modulus of continuity of the engaged function or its high order derivative, and producing very tight Jackson type inequalities. He treats both the univariate and multivariate cases. These operators “bell-shaped” and “squashing” functions are assumed to be of compact support.
Further, the author, inspired by [3], continued his studies on neural network approximation by introducing and using the proper quasi-interpolation operators of sigmoidal and hyperbolic tangent types which resulted in the complete monographs [4,5], by treating both the univariate and multivariate cases. He included also the corresponding fractional cases.
The author here performs symmetrized and perturbed hyperbolic tangent activated multivariate neural network approximation to differentiated functions from into , .
We present real multivariate quasi-interpolation quantitative approximations. We derive very tight Jackson type multivariate inequalities.
Real feed-forward neural networks (FNNs) with one hidden layer, the ones we use here are mathematically expressed by
where for , are the thresholds, are the connection weights, are the coefficients, is the inner product of and x, and is the activation function of the network. About neural networks in general read [6,7,8].
Recent developments in neural network approximation are [9,10,11,12,13,14,15,16,17,18].
2. Basics
Initially we follow [5], pp. 455–460.
Our perturbed hyperbolic tangent activation function is
Above is the parameter and q is the deformation coefficient.
For more read Chapter 18 of [5]: “q-deformed and -Parametrized Hyperbolic Tangent based Banach space Valued Ordinary and Fractional Neural Network Approximation”.
Chapters 17 and 18 of [5] motivate our current work.
The proposed “symmetrization method” aims to use half data feed to our multivariate neural networks.
We will employ the following density function
; .
So that
is an even function, symmetric with respect to the y-axis.
By (18.18) of [5], we have
sharing the same maximum at symmetric points.
By Theorem 18.1, p. 458 of [5], we have that
Consequently, we derive that
By Theorem 18.2, p. 459 of [5], we have that
so that
therefore is a density function.
By Theorem 18.3, p. 459 of [5], we have:
Let , and with ; . Then
where
Similarly, we get that
Consequently we obtain that
where
Remark 1.
We introduce
It has the properties:
(i)
(ii)
where , ∀ hence
(iii)
∀ , and
(iv)
that is Z is a multivariate density function.
Here, denote , , also set , upon the multivariate context,
(v)
where ,
Theorem 1.
Let , and with . It holds
Proof.
The condition , implies that there exists at least one , where
Indeed, it is
for some
Let , that is . Using the mean value theorem we have that
for some
Hence
That is
Similarly, it holds
and hence
We also have that ()
Hence it is , so that and
Next, we observe that
We have proved that
□
We need
Definition 1.
The modulus of continuity here is defined by
where is a bounded and continuous function, denoted by , . Similarly is defined for (uniformly continuous functions). We have that , iff as
Notation 1.
Let , . Here, denotes a partial derivative of f, , , and , where . We write also and we say it is of order l.
We denote
Call also
where is the supremum norm.
Under differentiation the speed of convergence of our neural network operators improves a lot, that is the aim of this work.
Next, we describe our neural network operators.
Definition 2.
Let , we define
, ∀, , the multivariate quasi-interpolation neural network operator.
Also for we define the multivariate Kantorovich type neural network operator
, ∀
Again for we define the multivariate neural network operator of quadrature type , , as follows. Let , , , such that
and
where
We put
Remark 2.
We notice that
Thus, it holds
Motivation here comes from the following.
Here, is an arbitrary Banach space.
Theorem 2.
([5], p. 58). Let (continuous and bounded X-valued functions from ), , , , with , is defined similarly as in Definition 1. Then
(i)
(ii)
Given that , we obtain , uniformly.
The speed of convergence above is
The space denotes the uniformly continuous functions. The operator is defined similarly to (36).
3. Main Results
Next, we study the approximation properties of , , and neural network operators.
Theorem 3.
Let , , , , with , for all , and , where . Then
(1)
(2) assume that , for all , , we have
with the high speed of convergence
(3)
and
(4)
We have that , as , pointwise and uniformly.
Proof.
Consider , ; . Then
for all
We have the multivariate Taylor’s formula
Notice . Also for we have
Furthermore
So, we treat
Thus, we have for , , that
where
We see that
Notice here that
Conclusion: When , we proved that
We proved in general that
Next we see that, let
then
We have proved that
Consequently, it holds
Next, we estimate
We have proved that
for
The proof of the theorem is now completed. □
We continue with the following result.
Theorem 4.
Assume all the conditions are the same as in Theorem 3. Then
(1)
(2) assume that , for all , , we have
with the high speed of convergence
(3)
and
(4)
We have that , as , pointwise and uniformly.
Proof.
It holds that
where
We see that
Notice that
Let
Here, we consider ,
We further see that
Conclusion: When , we proved that
We proved in general that
Next, we see that
So, when , we get
And, in general it holds
Furthermore, it holds
Call
Finally, we estimate
We have found that
for
The proof of the theorem now it is completed. □
Theorem 5.
Assume all the conditions are the same as in Theorem 3. Then
(1)
(2) assume that , for all , , we have
with the high speed of convergence
(3)
and
(4)
We have that , as , pointwise and uniformly.
Proof.
We have that
where
We see that
Notice that
We further see that
Conclusion: When , we proved that
We have extablished in general
Next, we observe that
So, when , we get
And, it holds in general
Furthermore, it holds
Call
We derive
And, furthermore we get that
As in the proof of Theorem 4, we obtain
Consequently it holds
Finally, we estimate
for
The theorem is proved. □
4. Illustrations for m = 1
We present
Corollary 1.
Let , , , , with , for all , and , where . Then
(1)
(2) assume that , for all , we have
at the high speed of convergence
(3)
(4)
We have that , as , pointwise and uniformly.
Proof.
By Theorem 3. □
We finish with
Corollary 2.
Assume all the conditions are the same as in Corollary 1. Then
(1)
(2) assume that , for all , we have
with the high speed of convergence
(3)
and
(4)
We have that , , as , pointwise and uniformly.
Proof.
By Theorems 4 and 5. □
5. Conclusions
Here, we presented multivariate neural network approximation over infinite domains under differentiation of functions. The activation function was the symmetrized and perturbed hyperbolic tangent function. The rates of convergence were higher and the data needed to feed the neural network were half due to symmetry.
Funding
This research received no external funding.
Data Availability Statement
No new data were created or analyzed in this study.
Conflicts of Interest
The author declares no conflicts of interest.
References
- Anastassiou, G.A. Rate of convergence of some neural network operators to the unit-univariate case. J. Math. Anal. Appl. 1997, 212, 237–262. [Google Scholar] [CrossRef]
- Anastassiou, G.A. Quantitative Approximations; Chapman & Hall/CRC: Boca Raton, NY, USA, 2001. [Google Scholar]
- Chen, Z.; Cao, F. The approximation operators with sigmoidal functions. Comput. Math. Appl. 2009, 58, 758–765. [Google Scholar] [CrossRef]
- Anastassiou, G.A. Inteligent Systems II: Complete Approximation by Neural Network Operators; Springer: Heidelberg, Germany; New York, NY, USA, 2016. [Google Scholar]
- Anastassiou, G.A. Parametrized, Deformed and General Neural Networks; Springer: Heidelberg, Germany; New York, NY, USA, 2023. [Google Scholar]
- Haykin, S. Neural Networks: A Comprehensive Foundation, 2nd ed.; Prentice Hall: New York, NY, USA, 1998. [Google Scholar]
- McCulloch, W.; Pitts, W. A logical calculus of the ideas immanent in nervous activity. Bull. Math. Biophys. 1943, 7, 115–133. [Google Scholar] [CrossRef]
- Mitchell, T.M. Machine Learning; WCB-McGraw-Hill: New York, NY, USA, 1997. [Google Scholar]
- Yu, D.; Cao, F. Construction and approximation rate for feedforward neural network operators with sigmoidal functions. J. Comput. Appl. Math. 2025, 453, 116150. [Google Scholar] [CrossRef]
- Cen, S.; Jin, B.; Quan, Q.; Zhou, Z. Hybrid neural-network FEM approximation of diffusion coefficient in elliptic and parabolic problems. IMA J. Numer. Anal. 2024, 44, 3059–3093. [Google Scholar] [CrossRef]
- Coroianu, L.; Costarelli, D.; Natale, M.; Pantiş, A. The approximation capabilities of Durrmeyer-type neural network operators. J. Appl. Math. Comput. 2024, 70, 4581–4599. [Google Scholar] [CrossRef]
- Warin, X. The GroupMax neural network approximation of convex functions. IEEE Trans. Neural Netw. Learn. Syst. 2024, 35, 11608–11612. [Google Scholar] [CrossRef] [PubMed]
- Fabra, A.; Guasch, O.; Baiges, J.; Codina, R. Approximation of acoustic black holes with finite element mixed formulations and artificial neural network correction terms. Finite Elem. Anal. Des. 2024, 241, 104236. [Google Scholar] [CrossRef]
- Grohs, P.; Voigtlaender, F. Proof of the theory-to-practice gap in deep learning via sampling complexity bounds for neural network approximation spaces. Found. Comput. Math. 2024, 24, 1085–1143. [Google Scholar] [CrossRef]
- Basteri, A.; Trevisan, D. Quantitative Gaussian approximation of randomly initialized deep neural networks. Mach. Learn. 2024, 113, 6373–6393. [Google Scholar] [CrossRef]
- De Ryck, T.; Mishra, S. Error analysis for deep neural network approximations of parametric hyperbolic conservation laws. Math. Comp. 2024, 93, 2643–2677. [Google Scholar] [CrossRef]
- Liu, J.; Zhang, B.; Lai, Y.; Fang, L. Hull form optimization research based on multi-precision back-propagation neural network approximation model. Int. J. Numer. Methods Fluid 2024, 96, 1445–1460. [Google Scholar] [CrossRef]
- Yoo, J.; Kim, J.; Gim, M.; Lee, H. Error estimates of physics-informed neural networks for initial value problems. J. Korean Soc. Ind. Appl. Math. 2024, 28, 33–58. [Google Scholar]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).