Abstract
This article is based on chapters 9 and 19 of the new neural network approximation monograph written by the first author. We use the approximation properties coming from the parametrized and deformed neural networks based on the parametrized error and q-deformed and -parametrized half-hyperbolic tangent activation functions. So, we implement a univariate theory on a compact interval that is ordinary and fractional. The result is the quantitative approximation of Brownian motion on simple graphs: in particular over a system S of semiaxes emanating from a common origin radially arranged and a particle moving randomly on S. We produce a large variety of Jackson-type inequalities, calculating the degree of approximation of the engaged neural network operators to a general expectation function of this kind of Brownian motion. We finish with a detailed list of approximation applications related to the expectation of important functions of this Brownian motion. The differentiability of our functions is taken into account, producing higher speeds of approximation.
Keywords:
neural network operators; Brownian motion on simple graphs; expectation; quantitative approximation MSC:
26A33; 41A17; 41A25; 60G15; 60G22
1. Introduction
The first author in [1,2], see Section 2, Section 3, Section 4 and Section 5, was the first researcher to derive quantitative neural network approximation to continuous functions by precisely defined neural network operators of Cardaliaguet–Euvrard and ‘Squasing’ types, by using the modulus of continuity of the engaged function or its high-order derivative and obtaining almost attained Jackson-type inequalities. He took care of both the univariate and multivariate cases. Defining these operators as ’bell-shaped’ and ’squashing’ functions is supposed to provide compact support.
Furthermore, the first author (motivated by [3]) continued his studies on neural networks approximation by introducing and using the appropriate quasi-interpolation operators of sigmoidal and hyperbolic tangent type, which resulted in [4], by treating both the univariate and multivariate cases. He dealt also with the corresponding fractional cases [5,6,7]. The authors also are motivated by the seminal works [8,9,10,11,12,13].
In [14,15], the first author extended his studies for Banach space-valued functions for activation functions induced by the parametrized error and q-deformed and -parametrized half-hyperbolic tangent sigmoid functions. The authors, motivated by [16], created neural network quantitative approximations to Brownian motion over a simple graph of a system of semiaxes.
They obtained a collection of Jackson-type inequalities, calculating the error of approximation to a general expectation function of this Brownian motion and its derivative. They present ordinary and fractional calculus results. They finish with a plethora of interesting applications.
2. Basics
2.1. About the Parametrized (Gauss) Error Special Activation Function
Here, we follow [17].
We consider here the parametrized (Gauss) error special activation function
which is a sigmoidal-type function and a strictly increasing function.
Of special interest in neural network theory is when ; see Section 1—Introduction.
It has the basic properties
and
We consider the function
and we notice that
Thus, is an even function.
Since , then , and , all .
We see
Let ; then, we have that
proving , for . That is, is strictly decreasing on , and it is strictly increasing on , and
Clearly, the x-axis is the horizontal asymptote of .
Concluding, is a bell symmetric function with maximum
Theorem 1.
It holds
We have
Theorem 2.
We have that
Hence, is a density function on We need
Theorem 3.
Let , and with , . It holds
with
Denote by the integral part and by the ceiling of a number.
Furthermore, we need
Theorem 4.
Let , and so that . Then,
Remark 1.
As in [18], we have that
Note 1.
For large enough n, we always obtain . Also, , iff . As in [18], we obtain that
Definition 1.
Let and . We introduce and define the X-valued linear neural network operators
Clearly here, . We study here the pointwise and uniform convergence of to with rates.
For convenience, also, we call
that is
So that
Consequently, we derive
That is
We will estimate the right-hand side of (19).
For that, we need, for , the first modulus of continuity
The fact is equivalent to ; see [19].
We present a series of real-valued neural network approximations to a function given with rates.
We first give
Theorem 5.
Let , , , Then,
(i)
and
(ii)
We notice , pointwise and uniformly.
The speed of convergence is
We need
Definition 2
([20]). Let , ; ( is the ceiling of the number), . We assume that . We call the left Caputo fractional derivative of order α:
If , we set the ordinary real-valued derivative (defined similar to numerical one, see [21], p. 83), and set . See also [22,23,24,25].
By [20], exists almost everywhere in and .
If , then by [20], hence
We mention the following.
Lemma 1
([19]). Let , , , and . Then, .
We also mention the following.
Definition 3
([26]). Let , , . We assume that , where . We call the right Caputo fractional derivative of order α:
We observe that for , and
By [26], exists almost everywhere on and .
If , and by [26], hence
See also [27].
We need
Lemma 2
([19]). Let , , , , . Then, .
We present the following real-valued fractional approximation result by -based neural networks.
Theorem 6.
Let , , , , Then,
(i)
and
(ii)
When , we derive
Corollary 1.
Let , , , , Then,
(i)
and
(ii)
2.2. About q-Deformed and -Parametrized Half-Hyperbolic Tangent Function
All the next background comes from [28].
Here, we describe the properties of the activation function
where
We have that
and
hence
It is
and
Furthermore
therefore, is striclty increasing. Moreover, in case of , we have that is strictly concave up, with
And in case of , we have that is strictly concave down.
Clearly, is a shifted sigmoid function with , and , ∀ (a semi-odd function); see also [28].
We consider the function
; . Notice that , so the x-axis is horizontal asymptote. We have that
which is a deformed symmetry.
Next, we have that
is striclty increasing over and it is strictly decreasing over
Moreover, is concave down over , and it is strictly concave down over
Consequently, has a bell-type shape over
Of course, it holds Thus, at , we have the maximum value of , which is
We mention
Theorem 7
([29]). We have that
It follows
Theorem 8
([29]). It holds
So that is a density function on
We need the following result,
Theorem 9
([29]). Let , and with ; . Then,
where
Let be the ceiling of the number, and let be the integral part of the number.
We mention the following result:
Theorem 10
([29]). Let and so that . For , we consider the number with and . Then,
We also mention
Remark 2
([29]). (i) We have that
where
(ii) Let . For large n, we always have . Also, , if . In general, it holds
We need
Definition 4.
Let and . We introduce and define the real-valued linear neural network operators
Clearly,
We study here the pointwise and uniform convergence of to with rates.
For convenience, also we call
That is
So that
Consequently, we derive that
where as in (41). We will estimate the right-hand side of the last quantity.
We present a set of real-valued neural network approximations to a function given with rates.
Theorem 11.
Let , , , , Then,
and
(ii)
We observe that , pointwise and uniformly.
Next, we present the following.
Theorem 12.
Let , , , Then,
(i)
and
(ii)
When , we derive
Corollary 2.
Let , , , Then,
(i)
and
(ii)
3. Combine 2.1 and 2.2
Let with , . Let also .
For the next theorems, we call
Also, we set
Furthermore, we set
We present the following.
Theorem 13.
Let , , , , Then, for ,
(i)
and
(ii)
We observe that , pointwise and uniformly.
Proof.
From Theorems 5 and 11. □
Next, we present
Theorem 14.
Let , , , Then, for
(i)
and
(ii)
Proof.
From Theorems 6 and 12. □
When , we derive
Corollary 3.
Let , , , Then, for
(i)
and
(ii)
4. About Random Motion on Simple Graphs
Here, we follow [16].
Suppose we have a system of S semiaxes with a common origin radially arranged and a particle moving randomly on S. Possible applications include the spread of toxic particles in a system of channels or vessels or the propagation of information in networks.
The mathematical model is the following: Let S be the set consisting of n semiaxes with a common origin 0 and let be the Brownian motion process on S: namely, the diffusion process on S whose infinitesimal generator L is
where
together with the continuity conditions (a total of equations),
and the so-called “Kirchoff condition”
This is a Walsh-type Brownian motion (see [30]).
The process has a standard Brownian motion on each of the semiaxes and, when it hits 0, it continues its motion on the j-th semiaxis, with probability
For each semiaxis , it is convenient to use the coordinate Notice that if is a function on S, then its j-th component, , is a function on thus, .
The transition density of is
and
We need the following result.
Theorem 15.
Let where with fixed. We consider the function which is bounded on , i.e., there exists such that for every and it is Lebesgue measurable on Let also be the standard Brownian motion on each of the semiaxes as described above. Here, is fixed on semiaxes, . We consider the related expected value function
The function is continuous in t and differentiable.
Proof.
First, we observe that for and with
Also, for and , it is
It is enough to prove that
is continuous in .
We have that
Thus,
Furthermore, as , with we obtain
By the dominating convergence theorem and thus, is continuous in consequently, the function
is continuous in t. □
We also need the next theorem.
Theorem 16.
Let where with are fixed. We consider function which is bounded on and Lebesgue measurable on Let also be the standard Brownian motion on each of the semiaxes as described above. Here, is fixed on semiaxes, . Then, the related expected value function
is differentiable in and
which is continuous in t.
Proof.
First, we observe that for and with ,
Also, for and , it is
Furthermore, for
for every
and
for every
So, and are bounded with respect to t. The bounds are integrable with respect to and , respectively.
We have
We apply differentiation under the integral sign.
We notice that
and
Therefore, there exists
which is continuous in t (same proof as in Theorem 15). □
5. Main Results
We present the following general approximation results of Brownian motion on simple graphs.
Theorem 17.
We consider function which is bounded on and Lebesgue measurable on Let also be the related expected value function.
If , , , where with then for
(i)
and
(ii)
We observe that , pointwise and uniformly.
Proof.
From Theorem 13. □
Next, we present
Theorem 18.
We consider function which is bounded on and Lebesgue measurable on Let also be the related expected value function.
If , , where with and Then, for
(i)
and
(ii)
Proof.
From Theorem 14. □
When , we derive
Corollary 4.
We consider function which is bounded on and Lebesgue measurable on Let also be the related expected value function.
If , and Then, for ,
(i)
and
(ii)
Proof.
From Corollary 3. □
We continue with
Theorem 19.
We consider function which is bounded on and Lebesgue measurable on Let also be the related expected value function.
If , , , where with then for ,
(i)
and
(ii)
We observe that , pointwise and uniformly.
Proof.
From Theorem 13. □
6. Applications
Let a function which is bounded on , where with and is Lebesgue measurable on . For the Brownian Motion on simple graphs , we will use the following notations
and
We can apply our main results to the function . Consider the function , where for every . Let also be the Brownian motion on simple graphs. Then, the expectation
is continuous in t.
Moreover,
Corollary 5.
Let , , , where with then, for and
(i)
and
(ii)
We observe that , pointwise and uniformly.
Proof.
From Theorems 17 and 19. □
Next, we present
Corollary 6.
Let , , where with and Then, for
(i)
and
(ii)
Proof.
From Theorem 18. □
When , we derive
Corollary 7.
Let , and Then, for ,
(i)
and
(ii)
Proof.
From Corollary 4. □
For the next application, we consider the function , where for every . Let also be the Brownian motion on simple graphs. Then, the expectation
is continuous in t.
Moreover,
Corollary 8.
Let , , , where with then, for and
(i)
and
(ii)
We observe that , pointwise and uniformly.
Proof.
From Theorems 17 and 19. □
Next, we present
Corollary 9.
Let , , where with and Then, for ,
(i)
and
(ii)
Proof.
From Theorem 18. □
When , we derive
Corollary 10.
Let , and Then, for ,
(i)
and
(ii)
Proof.
From Corollary 4. □
Let us consider now the function , where for every . Let also be the Brownian motion on simple graphs. Then, the expectation
is continuous in t.
Moreover,
Corollary 11.
Let , , , where with then, for and ,
(i)
and
(ii)
We observe that , pointwise and uniformly.
Proof.
From Theorems 17 and 19. □
Next, we present
Corollary 12.
Let , , where with and Then, for ,
(i)
and
(ii)
Proof.
From Theorem 18. □
When , we derive
Corollary 13.
Let , and Then, for ,
(i)
and
(ii)
Proof.
From Corollary 4. □
In the following, we consider the function , where for every . Let also be the Brownian motion on simple graphs. Then, the expectation
is continuous in t.
Moreover,
Corollary 14.
Let , , , where with then, for and ,
(i)
and
(ii)
We observe that , pointwise and uniformly.
Proof.
From Theorems 17 and 19. □
Next, we present
Corollary 15.
Let , , where with and Then, for ,
(i)
and
(ii)
Proof.
From Theorem 18. □
When , we derive
Corollary 16.
Let , and Then, for ,
(i)
and
(ii)
Proof.
From Corollary 4. □
Let the generalized logistic sigmoid function , where for every . Let also be the Brownian motion on simple graphs. Then, the expectation
is continuous in t.
Moreover,
Corollary 17.
Let , , , where with then, for and ,
(i)
and
(ii)
We observe that , pointwise and uniformly.
Proof.
From Theorems 17 and 19. □
Next, we present
Corollary 18.
Let , , where with and Then, for ,
(i)
and
(ii)
Proof.
From Theorem 18. □
When , we derive
Corollary 19.
Let , and Then, for ,
(i)
and
(ii)
Proof.
From Corollary 4. □
When , we have the usual logistic sigmoid function.
For the last application, we consider the Gompertz function , where for every . The Gompertz function is also a sigmoid function which describes growth as being slowest at the start and end of a given time period. Let also be the Brownian motion on simple graphs. Then, the expectation
is continuous in t.
Moreover,
Corollary 20.
Let , , , where with then for and ,
(i)
and
(ii)
We observe that , pointwise and uniformly.
Proof.
From Theorems 17 and 19. □
Next, we present
Corollary 21.
Let , , where with and Then, for ,
(i)
and
(ii)
Proof.
From Theorem 18. □
When , we derive
Corollary 22.
Let , and Then, for ,
(i)
and
(ii)
Proof.
From Corollary 4. □
7. Conclusions
Here, we employ two important parametrized and deformed activation function neural network approximators with their establish approximation properties. The parametrized activation functions kill far fewer neurons than the original ones. The asymmetry of the brain is best described by deformed activation functions. We derive quantitative stochastic approximations to Brownian motion over a set of semiaxes emanating from a fixed point. We finish with a very wide variety of interesting applications. This article is intended for interested mathematicians, probabilists and engineers.
Author Contributions
Conceptualization, G.A.A.; Validation, D.K. All authors have read and agreed to the published version of the manuscript.
Funding
This research received no external funding.
Data Availability Statement
Not applicable.
Acknowledgments
The authors would like to thank Vassilis G Papanicolaou of the National Technical University of Athens for having fruitful discussions during the course of this research; also, the authors would like to thank the referees for constructive comments.
Conflicts of Interest
The authors declare no conflict of interest.
References
- Anastassiou, G.A. Rate of convergence of some neural network operators to the unit-univariate case. J. Math. Anal. Appl. 1997, 212, 237–262. [Google Scholar] [CrossRef]
- Anastassiou, G.A. Quantitative Approximations; Chapman and Hall/CRC Boca Raton: New York, NY, USA, 2001. [Google Scholar]
- Chen, Z.; Cao, F. The approximation operators with sigmoidal functions. Comput. Math. Appl. 2009, 58, 75–765. [Google Scholar] [CrossRef]
- Anastassiou, G.A. Intelligent Systems: Approximation by Artificial Neural Networks, Intelligent Systems Reference Library; Springer: Heidelberg/Berlin, Germany, 2011; Volume 19. [Google Scholar]
- Anastassiou, G.A. Intelligent Systems II: Complete Approximation by Neural Network Operators; Springer: Heidelberg/Berlin, Germany; New York, NY, USA, 2016. [Google Scholar]
- Anastassiou, G.A. Nonlinearity: Ordinary and Fractional Approximations by Sublinear and Max-Product Operators; Springer: Heidelberg/Berlin, Germany; New York, NY, USA, 2018. [Google Scholar]
- Anastassiou, G.A. Intelligent Computations: Abstract Fractional Calculus, Inequalities, Approximations; Springer: Heidelberg/Berlin, Germany; New York, NY, USA, 2018. [Google Scholar]
- Barron, A.R. Approximation and estimation bounds for Artificial neural networks. Mach. Learn. 1994, 14, 115–133. [Google Scholar] [CrossRef]
- Cybenko, G. Approximation by superpositions of a sigmoidal functon. Math. Control Signals Syst. 1989, 2, 303–314. [Google Scholar] [CrossRef]
- Funahashi, K.I. On the approximate realization of continuous mappings by neural networks. Neural Netw. 1989, 2, 183–192. [Google Scholar] [CrossRef]
- Guliyev, N.; Ismailov, V. On the approximation by single hidden layer feedforward neural networks with fixed weights. Neural Netw. 2018, 98, 296–304. [Google Scholar] [CrossRef] [PubMed]
- Hornik, K.; Stinchcombe, M.; White, H. Multilayer feedforward networks are universal approximators. Neural Netw. 1989, 2, 359–366. [Google Scholar] [CrossRef]
- Pinkus, A. Approximation theory of the MLP model in neural networks. Acta Numer. 1999, 8, 143–195. [Google Scholar] [CrossRef]
- Anastassiou, G.A. Parametrized, Deformed and General Neural Networks; Springer: Heidelberg/Berlin, Germany; New York, NY, USA, 2023. [Google Scholar]
- Anastassiou, G.A. Banach Space Valued Neural Network; Springer: Heidelberg/Berlin, Germany; New York, NY, USA, 2023. [Google Scholar]
- Papanicolaou, V.G.; Papageorgiou, E.G.; Lepipas, D.C. Random motion on simple graph. Methodol. Comput. Appl. Probab. 2012, 14, 285–297. [Google Scholar] [CrossRef]
- Anastassiou, G.A. Parametrized Error Function Based Banach Space Valued Univariate Neural Network Approximation; Chapter 9, Parametrized, Deformed and General Neural Networks; Springer: Heidelberg/Berlin, Germany; New York, NY, USA, 2023. [Google Scholar]
- Anastassiou, G.A. General sigmoid based Banach space valued neural network approximation. J. Comput. Anal. Appl. 2023, 31, 520–534. [Google Scholar]
- Anastassiou, G.A. Vector fractional Korovkin type approximations. Dyn. Syst. Appl. 2017, 26, 81–104. [Google Scholar]
- Anastassiou, G.A. A strong Fractional Calculus Theory for Banach space valued functions. Nonlinear Funct. Anal. Appl. (Korea) 2017, 22, 495–524. [Google Scholar]
- Shilov, G.E. Elementary Functional Analysis; Dover Publications, Inc.: New York, NY, USA, 1996. [Google Scholar]
- Diethelm, K. The Analysis of Fractional Diferential Equations, Lecture Notes in Mathematics 2004; Springer: Heidelberg/Berlin, Germany, 2010. [Google Scholar]
- El-sayed, A.M.A.; Geber, M. On the finite Caputo and finite Riesz derivatives. Electron. J. Theor. Phys. 2006, 3, 81–95. [Google Scholar]
- Frederico, G.S.; Torres, D.F.M. Fractional Optimal Control in the sense of Caputo and the fractional Noether’s theorem. Int. Math. Forum 2008, 3, 479–493. [Google Scholar]
- Samko, S.G.; Kilbas, A.A.; Marichev, O.I. Fractional Integrals and Derivatives, Theory and Applications; Gordon and Breach: Amsterdam, The Netherlands, 1993; [English translation from the Russian, Integrals and Derivatives of Fractional Order and Some of Their Applications (Nauka i Tekhnika, Minsk, 1987)]. [Google Scholar]
- Anastassiou, G.A. Strong Right Fractional Calculus for Banach space valued functions. Rev. Proyecciones 2017, 36, 149–186. [Google Scholar] [CrossRef]
- Anastassiou, G.A. On Right Fractional Calculus. Chaos Solitons Fractals 2009, 42, 365–376. [Google Scholar] [CrossRef]
- Anastassiou, G.A. q-Deformed and β-Parametrized Half Hyperbolic Tangent Based Banach Space Valued Ordinary and Fractional Neural Network Approximation; Chapter 20, Parametrized, Deformed and General Neural Networks; Springer: Heidelberg/Berlin, Germany; New York, NY, USA, 2023. [Google Scholar]
- Anastassiou, G.A. q-Deformed and Parametrized Half Hyperbolic Tangent Based Banach Space Valued Multivariate Multi Layer Neural Network Approximations; Chapter 19, Parametrized, Deformed and General Neural Networks; Springer: Heidelberg/Berlin, Germany; New York, NY, USA, 2023. [Google Scholar]
- Barlow, M.T.; Pitman, J.W.; Yor, M. On Walsh’s Brownian motions. Sem. prob. XXIII. In Lecture Notes in Mathematics; Springer: Heidelberg/Berlin, Germany; New York, NY, USA, 1989; Volume 1372, pp. 275–293. [Google Scholar]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).