1. Introduction
The classic theory of univariate approximation by singular integral operators is well-documented in the monograph [
1]. The corresponding multivariate theory is also extensively presented in the monograph [
2]. Earlier important works that motivate us are [
3,
4,
5]. So, inspired by all of the above we attempt to expose a new theory: the multivariate approximation by activated singular integral operators. In this case, the kernel comes from a multivariate neural network activation function, and the parametrized and deformed hyperbolic tangent function. In recent intense mathematical activity, neural networks play a leading role in solving numerically univariate and multivariate differential equations, so multivariate activated singular integrals will play a central role.
What is surprising here is the use of the reverse process from applied mathematics to theoretical ones, which is very rare.
So, here we present multivariate pointwise and uniform and
,
approximations. We continue by detailing the global smoothness preservation property by our operators. We conclude with the simultaneous treatment of all of these. This is a seminal article, as it is the first of its kind. Other inspiring sources have been the articles [
6,
7,
8,
9,
10,
11,
12,
13,
14,
15].
2. Background
(I) Upon uniform approximation, see [
2], Ch. 1.
Here,
,
; we define
and
Let be a probability Borel measure on , , ,
We now define the multiple smooth singular integral operators
where
,
,
,
is a Borel measurable function, and also
is a bounded sequence of positive real numbers.
The operators are not in general positive, and they preserve the constant functions in N variables.
We need
Definition 1. Let , the space of all bounded and continuous functions on . Then, the rth multivariate modulus of smoothness of f is given bywhere is the sup-norm andLet and let . Assume that all partial derivatives of f of order m are bounded, i.e.,for all i We need
Theorem 1. Let , , . Assume for all
Let be a Borel probability measure on , for , bounded sequence.
Assume that for all , we have For , and , , , call Then
- (i)
- (ii)
Given that , as and is uniformly bounded, then we obtain with rates. - (iii)
It holds also thatgiven that for all , Furthermore, as when , assuming that , while is uniformly bounded, we concludewith rates.
Theorem 2. Let , . Thenunder the assumptionAs and , given that are uniformly bounded and f is uniformly continuous, we obtainwith rates. (II) On
,
, approximation, see [
2], Ch. 2.
Here, we deal with , , with , , where denotes the mixed partial , , i ,
We need
Definition 2. We callLet , the modulus of smoothness of order r is given by. The following comes from ([
16], Ch. 25).
Theorem 3. Let , , , with , Let . Here, is a Borel probability measure on for , bounded sequence. Assume for all , that we haveFor and , , callThenAs and , by (22), we obtain with rates. We also obtain, by (22),given that , Assuming that , as , we obtain , that is the unit operator, in norm, with rates.
Inequality (
22) provides a correction in the constants of the inequality in (Theorem 4 in [
2], p. 25).
In particular, we have
Theorem 4. Let ; ; . Assume probability Borel measures on , and bounded. Also supposeThenAs , when , we obtain , i.e., , the unit operator, in norm. Next, we mention
Theorem 5. Let , , with , Here, is a Borel probability measure on for , is a bounded sequence. Assume for all , that we haveFor and , , callThenAs , we obtain with rates. From (28), we obtaingiven that , As , assuming and , we obtain , that is in norm, with rates.
Theorem 6. Let , Assume probability Borel measures on , and bounded. Also supposeThenAs , we obtain in norm. (III) Global smoothness and simultaneous approximation ([
2], Ch. 3).
We mention the following general global smoothness preservation result
Theorem 7. We suppose , ∀. Let , , .
- (i)
Assume . Then - (ii)
Assume Then - (iii)
Assume , Then
We need
Theorem 8. Let , . Here is a Borel probability measure on , a bounded sequence. Let , , Here, , , is -integrable with reference to s, for There exist -integrable functions () on , such that∀. Then, both of the following exist, and We detail the simultaneous global smoothness results.
Theorem 9. Let and assumptions of Theorem 8 are valid. Here, , ().
- (i)
Assume . Then - (ii)
Additionally, assume Then - (iii)
Additionally, assume , Then
Next comes simultaneous approximation.
Theorem 10. Let , . The assumptions of Theorem 8 are valid. Call . Assume andfor all , i , where is a Borel probability measure on , for , bounded sequence. For and , , callThen Theorem 11. Let , (functions l-times continuously differentiable and bounded). The assumptions of Theorem 8 are valid. Call . AssumeThen We mention more simultaneous approximation results.
Theorem 12. Let The assumptions of Theorem 8 are valid. Call . Let , , , and Here is a Borel probability measure on for , bounded sequence. Assume for all , , we haveFor , and , , , callThen Theorem 13. Let The assumptions of Theorem 8 are valid. Call . Let , ; Assume probability Borel measures on and bounded. Also supposeThen Theorem 14. Let The assumptions of Theorem 8 are valid. Call . Let , Assume probability Borel measures on and bounded. Also supposeThen The last supporting result is as follows:
Theorem 15. Let , . The assumptions of Theorem 8 are valid. Call . Let , . Here is a Borel probability measure on for , is a bounded sequence. Assume for all , , we haveFor , and , , , callThen 3. About the -Deformed and -Parametrized
Hyperbolic Tangent Function
We consider the activation function
and we mention some of its related properties; most of these come from ([
17], ch. 17).
Let the activation function
It is
and
with
We consider the function
;
. We have
, so the
x-axis is the horizontal asymptote.
It holds
and
That is
an even function.
Next we follow [
17], pp. 432–433.
Let , ;
From (
57), we have
That is
So, the last is
.
We have
that is
is a multivariate density function.
Let
, then
so that
is a multivariate density function on
, and let
Clearly,
is a Borel probability measure in
.
We have that
∀
Adding the above, we obtain
that is
∀
So,
is a symmetric function over
,
, and
∀
Therefore, we obtain
∀
Furthermore, it holds that
for any
,
so that
.
Furthemore, (
) we have
∀
,
Above and are univariate density functions.
Furthermore, is also a density function on , and is a density function on
So, we can rewrite
∀
,
In
Section 4, all proofs are based on our “Symmetrization Technique” in estimating integrals, as you will see.
4. Auxiliary Essential Results
We need the following:
Theorem 16. Let , , ; , ; ;
Thenwhere above is the incomplete upper gamma function. Proof. We observe that
(by [
19], p. 348)
□
We continue with
Theorem 17. Let , , , . Then It follows
Theorem 18. All as in Theorem 16, . Then We continue with
Theorem 19. Let , , , , . Then The last supporting result for uses the following:
Theorem 20. Let , , , ,
5. Main Results
We construct
Definition 3. Let be a Borel measurable function, we follow (1)–(5). We define The above multiple smooth singular integral operators are a special case of (
5).
In this section, we study the approximation properties of , .
We present
Theorem 21. Let , , , . Assume for all
Let , , and the Borel probability measure on denoted by , such that For all , , we denote For , and , , , call Then
- (i)
- (ii)
Given that , as , we have with rates. - (iii)
It also holds thatgiven that for all , . Also, it holds thatwith rates, as .
Proof. By Theorems 1, 16, 20. □
We continue with
Theorem 22. Let , . ThenwhereClearly, we havewith rates, given that f is uniformly continuous. Proof. By Theorem 2 and Theorem 17. □
Next comes , approximation.
Theorem 23. Let , , , with , Let . Let , , and the Borel probability measure on , denoted by , such that For all , , we denote this byHere, is as in (95). ThenAs and , by (103), we obtain with rates. By (103), furthermore, we obtaingiven that , Indeed, here, , that is the unit operator, in norm, with rates.
Proof. By Theorems 3, 18, 20. □
We especially have
Theorem 24. Let ; ; ; , . SetThenAs , when , we obtain , that is , the unit operator, in norm. Proof. By Theorems 4, 19. □
The approximation follows.
Theorem 25. Let , , with , Here , , and the Borel probability measure on , denoted by :The is as in (94), and , as in (95). ThenAs , we find with rates. From (107), we derivegiven that , As , and , we derive that , that is in norm, with rates.
Proof. By Theorems 5, 16, 20. □
Theorem 26. Let , , Here, is as in (76). ThenAs , we obtain in norm. Proof. By Theorems 6, 17. □
We continue with global smoothness preservation.
Theorem 27. We suppose , ∀. Let , , .
- (i)
Assume . Then - (ii)
Assume Then - (iii)
Assume , Then
Here ,
Let the Borel probability measure on
, denoted by
:
We detail the simultaneous smoothness results.
Theorem 28. Let , and the assumptions of Theorem 8 regarding are valid. Here, , ().
- (i)
Assume . Then - (ii)
Additionally, assume Then - (iii)
Additionally, assume , Then
Simultaneous approximation follows.
Theorem 29. Let , . The assumptions of Theorem 8 are valid with respect to , , . Call , assume
Here, , , is as in (95); and is as in (71). Proof. By Theorems 8, 10, 16, 20. □
Theorem 30. Let , . The assumptions of Theorem 8 are valid for , , . Call . Here, is as in (76). Proof. By Theorems 8, 11, 17. □
The , simultaneous approximation follows:
Theorem 31. Let The assumptions of Theorem 8 are valid regarding , , . Call . Let , , , and Here, is as in (81), and , is as in (95). Proof. By Theorems 8, 12, 18, 20. □
Theorem 32. Let The assumptions of Theorem 8 are valid with respect to , , . Call . Let , ; Here, is as in (86). Then Proof. By Theorems 8, 13, 19. □
We conclude with simultaneous approximation.
Theorem 33. Let The assumptions of Theorem 8 are valid for , , . Call . Let , Here is as in (76). Then Proof. By Theorems 8, 14, 17. □
Theorem 34. Let , . The assumptions of Theorem 8 are valid for , , . Call . Let , . Here , , is as in (71), and is as in (95), Proof. By Theorems 8, 15, 16, 20. □
6. Conclusions
Here we presented the novel idea of going from the neural networks main tools, the activation functions, to multivariate singular integral approximation. This is a rare case of employing applied mathematics to theoretical ones.