Approximation by the Extended Neural Network Operators of Kantorovich Type

Abstract

Based on the idea of integral averaging and function extension, an extended Kantorovich-type neural network operator is constructed, and its error estimate of approximating continuous functions is obtained by using the modulus of continuity. Furthermore, by introducing the normalization factor, the approximation property of the new version of the extended Kantorovich-type neural network (normalized extended Kantorovich-type neural network) operator is obtained in $L_p[-1,1]$. The numerical examples show that this newly proposed neural network operator has a better approximation performance than the classical one, especially at the endpoints of a compact interval.

1. Introduction

Neural networks are broadly applied in various applications, such as visual recognition, healthcare, astronomical physics, geology, cybersecurity, and many more. As the most widely used neural networks, feedforward neural networks (FNNs) have been thoroughly studied because of their universal approximation capabilities. Theoretically, any continuous function on a compact set can be approximated by FNNs to an arbitrary desired degree provided that the number of neurons is sufficiently large. Further, some upper bounds of the approximation error for FNNs in the uniform metric and $L_p$ metric have been studied in [1,2,3,4,5], and so on.

FNNs with one hidden layer can be mathematically expressed as

$$N_{n,\sigma }\left(\mathbf{x}\right)=\sum _{k=1}^n{c}_k\sigma ({\mathbf{a}}_k·\mathbf{x}+b_k),\mathbf{x}=(x_1,x_2,\cdots ,x_s)\in {\mathbb{R}}^s,s\in \mathbb{N},$$

where for $1\le k\le n,b_k\in \mathbb{R}$ are the thresholds, ${\mathbf{a}}_k=(a_{k1},a_{k2},\cdots ,a_{ks})\in {\mathbb{R}}^s$ are the connection weights, $c_k\in \mathbb{R}$ are the coefficients, ${\mathbf{a}}_k·\mathbf{x}$ is the inner product of ${\mathbf{a}}_k$ and $\mathbf{x}$, and $\sigma$ is the activation function.

NN operators and their approximation properties have attracted a lot of attention since the 1990s. Cardaliaguet and Euvrard [6] first introduced NN operators to approximate the unit operator. Since then, different types of NN operators were constructed, and their approximation properties were widely investigated. A lot of impressed results concerning the convergence of NN operators as well as the complexity of approximation have been achieved ([6,7,8,9,10,11,12,13,14,15,16,17], etc.). Traditional operators provide references for NN operators in many aspects, such as the construction, the way of discussing approximation properties, and the practical applications. They are both related and different in approximation performance and practical applications. Taking the classical Bernstein operator as an example, it is usually used to approximate continuous functions, while NN operators can be utilized to approximate a broad class of functions, such as integrable functions; Bernstein operators are valuable tools in computer-aided geometric design, while NN operators are used for a wider range of applications, such as machine learning. Moreover, different from classical operators, one remarkable feature of NN operators is that they are nonlinear. It is worth mentioning that the main advantage of using NN operators as approximation tools is that the NN operators can be viewed as FNNs with multiple layers, while all the components in the NN operators are known, such as the coefficients, the weights, and the thresholds, in order to approximate the target function. The constructions of NN operators and related discussions have formed an important part of the fundamental theories of artificial neural networks, which were introduced in order to simulate human brain activities.

Next, we are going to review some NN operators and their approximation properties. Let $C[a,b]$ be the space of continuous functions on $[a,b]$. The approximation properties of NN operator $G_n(f,x):[a,b]\to \mathbb{R}$,

$$\begin{array}{c}\hfill G_n(f,x)=\frac{{\displaystyle \sum _{k=\lceil na\rceil }^{\lfloor nb\rfloor }f\left(\frac{k}{n}\right)·{\varphi }_{\sigma }(nx-k)}}{{\displaystyle \sum _{k=\lceil na\rceil }^{\lfloor nb\rfloor }{\varphi }_{\sigma }(nx-k)}},x\in [a,b]\end{array}$$

has been studied in [12,18,19], where ${\varphi }_{\sigma }$ will be defined in Section 2, $f\in C[a,b]$, equipped with the uniform norm ${\displaystyle {\parallel f\parallel }_{\infty }=\underset{a\le x\le b}{max}\left|f\left(x\right)\right|}$. The symbol $\lfloor x\rfloor$ denotes the greatest integer not exceeding x, while $\lceil x\rceil$ denotes the smallest integer greater than or equal to x.

In applications, the values of $f\left(k/n\right)$ would have some errors due to the time-jitter or offset of the input signals, while more information is usually known around a point than precisely at that point. In approximation theory, constructing a Kantorovich-type operator to reduce the “time-jitter or offset“ errors is a well-known method. This kind of operator is also very useful in areas such as signal processing. To achieve this, we replace single function values $f(k/n)$ by an average of f on a small interval around $k/n$, namely the mean, ${\displaystyle n{\int }_{\frac{k}{n}}^{\frac{k+1}{n}}f\left(u\right)\mathrm{d}u}$.

In [13], the authors used a similar idea and constructed a Kantorovich-type neural network operator $F_n(f,\mathrm{x}):\mathcal{R}\to \mathbb{R}$ in ${\mathbb{R}}^d,d\in \mathbb{Z}$, and the related estimates are given therein. If we consider the one-dimension case of $F_n\left(f\right)$, then the NN operator degenerates to $N_n(f,x):[-1,1]\to \mathbb{R}$,

$$\begin{array}{c}\hfill N_n(f,x)=\frac{{\displaystyle \sum _{k=-n}^{n-1}n{\int }_{\frac{k}{n}}^{\frac{k+1}{n}}f\left(u\right)\mathrm{d}u·{\varphi }_{\sigma }(nx-k)}}{{\displaystyle \sum _{k=-n}^{n-1}{\varphi }_{\sigma }(nx-k)}},x\in [-1,1].\end{array}$$

It was proved in [13] that $N_n(f,x)$ convergences to $f\left(x\right)$ in various functional spaces.

In [9], the authors extended the continuous function f on $[-1,1]$ to the function

$$f_{1,e}\left(x\right)=\left\{\begin{array}{cc}f\left(x\right),\hfill & x\in [-1,1]\hfill \\ f\left(\mathrm{sgn}x\right),\hfill & 1<\left|x\right|<2,\hfill \end{array}\right.$$

(1)

and discovered that, after constructing Kantorovich-type operators, the approximation rate of the constructed NN operator to the target function was significantly improved.

A natural question is, what happens if we combine the idea of function extension with that of integral averaging? In this paper, we investigate the approximation effects, including the convergence properties and quantitative estimates, of our newly constructed NN operator, called the extended Kantorovich-type NN operator, regarding this question. A numerical example shows that this type of neural network operator has a better approximation performance than $N_n(f,x)$, especially at the endpoints of a compact interval.

The remaining part of this paper is organized as follows. In Section 2, we propose two new types of NN operators, the extended Kantorovich-type NN operator (EKNNO) and the normalized extended Kantorovich-type NN operator (NEKNNO), and investigate their basic properties and the operator-dependent activation function. In Section 3, we establish the approximation theorems of the EKNNO and NEKNNO for continuous functions, including the convergence theorems and quantitative estimates, using the modulus of continuity. This section also demonstrates a numerical example and its results, which verify the validity of the theoretical results and the potential superiority of these operators. In Section 4, we further establish and prove the approximation properties of the EKNNO in the Lebesgue space, as well as the convergence results and the approximation rate of the NEKNNO. The conclusions are included in Section 5.

2. Extended Neural Network Operators of Kantorovich Type

In this section, we explain how to construct two extended neural network operators of a Kantorovich type—EKNNO and NEKNNO.

The activation function $\sigma$ plays a significant role in the approximation properties of neural networks. In many fundamental NN models, the activation function $\sigma$ is usually taken to be a sigmoid function ([20]), which is defined below.

A function $\sigma \left(x\right):\mathbb{R}\to \mathbb{R}$ is called sigmoid ([20]) if

$$\underset{x\to -\infty }{lim}\sigma \left(x\right)=0,\underset{x\to +\infty }{lim}\sigma \left(x\right)=1.$$

For example, the well-known Logistic function is a typical sigmoid type, which is defined by ${\displaystyle \sigma \left(x\right)=\frac{1}{1+e^{-x}}}$.

Next, we write ${\varphi }_{\sigma }\left(x\right)$, a combination of the translations of $\sigma$ that was first proposed by Chen and Cao [8]:

$${\varphi }_{\sigma }\left(x\right):=\frac{1}{2}\left[\sigma (x+1)-\sigma (x-1)\right],x\in \mathbb{R}.$$

(2)

We assume in this paper that the sigmoid-type function is nondecreasing on $\mathbb{R}$, such that $\sigma \left(4\right)>\sigma \left(2\right)$, and it is centrosymmetric with respect to the point $(0,\sigma (0\left)\right)$. For example, the Logistic function is exactly the sigmoid-type function that satisfies these conditions, i.e.,

$$\underset{x\to +\infty }{lim}{\varphi }_{\sigma }\left(x\right)=\underset{x\to -\infty }{lim}{\varphi }_{\sigma }\left(x\right)=0.$$

The function ${\varphi }_{\sigma }\left(x\right)$, which is often called the “bell-shaped function” (a name given by Cardaliaguet and Euvrard [6]), has been discussed in many papers. It has some important properties, and we cite part of them from [11] here, which are related to the research presented in this paper.

$$\begin{array}{cc}& \left(\mathrm{i}\right)\forall x\in \mathbb{R},{\varphi }_{\sigma }\left(x\right)\ge 0,\mathrm{specifically},{\varphi }_{\sigma }\left(3\right)>0;\hfill \\ & \left(\mathrm{ii}\right){\varphi }_{\sigma }\left(x\right)\mathrm{is}\mathrm{an}\mathrm{even}\mathrm{function}\mathrm{on}\mathbb{R};\hfill \\ & \left(\mathrm{iii}\right){\varphi }_{\sigma }\left(x\right)\mathrm{is}\mathrm{nondecreasing}\mathrm{for}x<0,\mathrm{and}{\varphi }_{\sigma }\left(x\right)\mathrm{is}\mathrm{nonincreasing}\mathrm{for}x\ge 0.\hfill \\ & \mathrm{Moreover},{\varphi }_{\sigma }\left(x\right)\le {\varphi }_{\sigma }\left(0\right),x\in \mathbb{R}.\hfill \end{array}$$

Using the definition of the Fourier transformation ([21]) and the Poisson summation formula ([9,22]), we have Theorem 1.

Theorem 1. 

Assume that  $\sigma \left(x\right)$  is  $\mathrm{sigmoid}$ type and centrosymmetric with respect to the point $(0,\sigma (0\left)\right)$, ${\varphi }_{\sigma }\left(x\right)$ is defined by (2). If there exist constants $C>0$ and $\delta >1$ such that

$${\displaystyle |{\varphi }_{\sigma }{\left(x\right)|\le C(1+\left|x\right|)}^{-1-\delta },|{\widehat{\varphi }}_{\sigma }{\left(x\right)|\le C(1+\left|x\right|)}^{-1-\delta },x\in \mathbb{R},{\int }_{-\infty }^{+\infty }{\varphi }_{\sigma }\left(x\right)\mathrm{d}x=1,}$$

then we have

$$\sum _{k=-\infty }^{+\infty }{\varphi }_{\sigma }(x-k)=1+2\sum _{k=1}^{+\infty }{\widehat{\varphi }}_{\sigma }\left(k\right)cos2k\pi x,$$

(3)

where ${\widehat{\varphi }}_{\sigma }\left(x\right)$ is the Fourier transformation of ${\varphi }_{\sigma }\left(x\right)$.

Throughout the paper, C refers to a positive constant whose value may vary under different circumstances.

Remark 1. 

If $\sigma \left(x\right)$ is taken as the $\mathrm{Logistic}$ function in Theorem 1, it was proved in [8,22] that for all $k\in \mathbb{Z}-\left\{0\right\}$, we have ${\widehat{\varphi }}_{\sigma }\left(k\right)=0$, that is, Equality (3) can be simply written as ${\displaystyle \sum _{k=-\infty }^{+\infty }{\varphi }_{\sigma }(x-k)=1}$.

Inspired by the idea of integral averaging and function extension, we propose the first new operator—the extended Kantorovich-type NN operator (EKNNO)—as follows.

Definition 1. 

Let $A>0$, ${\varphi }_{\sigma }\left(x\right)$ is defined by (2). Denote

$${\varphi }_{\sigma ,A}\left(x\right)=\frac{1}{A}{\varphi }_{\sigma }\left(\frac{x}{A}\right).$$

(4)

Assume $f\in C[-1,1]$, and $f_{1,e}$ is defined as in (1). Then, EKNNO is defined by

$$K_{n,A}^{\ast }(f,x)=\sum _{k=-2n}^{2n-1}n{\int }_{\frac{k}{n}}^{\frac{k+1}{n}}{f}_{1,e}\left(u\right)\mathrm{d}u·{\varphi }_{\sigma ,A}(nx-k).$$

The EKNNO $K_{n,A}^{\ast }(f,x)$ has several key characteristics. First, we replace $f(k/n)$ by the integral averaging $n{\int }_{\frac{k}{n}}^{\frac{k+1}{n}}{f}_{1,e}\left(u\right)\mathrm{d}u$ to remove the effect of time-jitter. Second, we extend the traditional function f defined on interval $[-1,1]$ to $f_{1,e}$ on $(-2,2)$ to have better approximation abilities, especially around the endpoints $-1,1$. Further, we introduce a parameter A into the activation function ${\varphi }_{\sigma ,A}(nx-k)$, which serves as a flexible quantity used to fine tune the approximation ability of the operator.

Based on Definition 1, by introducing the normalization factor, we further construct the normalized version of the extended NN operator (NEKNNO) as follows.

Definition 2. 

Assume that $f_{1,e}\left(u\right)$ is given by (1), and ${\varphi }_{\sigma ,A}\left(x\right)$ is defined by (4). Then, we define

$$\begin{array}{c}\hfill K_{n,A}(f,x)=\frac{{\displaystyle \sum _{k=-2n}^{2n-1}n{\int }_{\frac{k}{n}}^{\frac{k+1}{n}}{f}_{1,e}\left(u\right)\mathrm{d}u·{\varphi }_{\sigma ,A}(nx-k)}}{{\displaystyle \sum _{k=-2n}^{2n-1}{\varphi }_{\sigma ,A}(nx-k)}}.\end{array}$$

Normalization allows us to further discuss the approximation performance of NN operators in the integrable function space.

Compared with $K_{n,A}^{\ast }(f,x)$, $K_{n,A}(f,x)$ has the following properties.

Theorem 2. 

Assume $K_{n,A}(f,x)$ is defined as in Definition 2. Then, for $f,g\in C[-1,1]$, we have

(I) $\left|K_{n,A}(f,x)-K_{n,A}(g,x)\right|\le K_{n,A}\left(\right|f-g|,x),x\in [-1,1]$.

(II) Denote $\mathbf{1}$ the unitary constant function on $[-1,1]$, i.e., $g=\mathbf{1}\iff g\left(t\right)\equiv 1,t\in [-1,1]$. Then,

$$\begin{array}{c}\hfill K_{n,A}(\mathbf{1},x)\equiv 1,x\in [-1,1].\end{array}$$

Proof. 

For arbitrary $x\in [-1,1]$, we have

$$f\left(x\right)=f\left(x\right)-g\left(x\right)+g\left(x\right)\le \left|f\right(x)-g(x\left)\right|+g\left(x\right),$$

(5)

similarly

$$g\left(x\right)\le \left|g\right(x)-f(x\left)\right|+f\left(x\right).$$

(6)

Notice that for arbitrary $x\in [-1,1]$, if $f\left(x\right)\le g\left(x\right)$, then

$$K_{n,A}(f,x)\le K_{n,A}(g,x),$$

(7)

$$K_{n,A}(f+g,x)\le K_{n,A}(f,x)+K_{n,A}(g,x).$$

(8)

By (5)–(8), we have

$$\left\{\begin{array}{c}{K}_{n,A}(f,x)\le K_{n,A}\left(\right|f-g|,x)+K_{n,A}(g,x)\hfill \\ K_{n,A}(g,x)\le K_{n,A}\left(\right|g-f|,x)+K_{n,A}(f,x)\hfill \end{array}\right.$$

$$\Rightarrow \left|K_{n,A}(f,x)-K_{n,A}(g,x)\right|\le K_{n,A}\left(\right|f-g|,x).$$

The Proof of (II) is straightforward because

$$K_{n,A}(\mathbf{1},x)=\frac{{\displaystyle \sum _{k=-2n}^{2n-1}n{\int }_{\frac{k}{n}}^{\frac{k+1}{n}}1\mathrm{d}u·{\varphi }_{\sigma ,A}(nx-k)}}{{\displaystyle \sum _{k=-2n}^{2n-1}{\varphi }_{\sigma ,A}(nx-k)}}=1.$$

Theorem 2 is proved. □

To obtain the error estimates of the operators $K_{n,A}(f,x)$ and $K_{n,A}^{\ast }(f,x)$, we need the following lemmas.

Lemma 1. 

Let ${\varphi }_{\sigma ,A}\left(x\right)$ be defined as in (4). Then, for any $\gamma >0,k\in \mathbb{Z}$,

$$\underset{n\to +\infty }{lim}\sum _{|x-k|>\gamma n}{\varphi }_{\sigma ,A}(x-k)=0$$

holds uniformly for $x\in \mathbb{R}$.

Proof. 

For any $x\in \mathbb{R}$, set $x_0=x-\lfloor x\rfloor$, and then $0\le x_0<1$. Further, for any $k\in \mathbb{Z}$, denote $\overline{k}=k-\lfloor x\rfloor$. Then, for sufficiently large $n\in \mathbb{N}$, by the definition of ${\varphi }_{\sigma ,A}\left(x\right)$ and Theorem 1, we have

$$\begin{array}{cc}\hfill \sum _{|x-k|>\gamma n}{\varphi }_{\sigma ,A}(x-k)& =\sum _{|x_0-\overline{k}|>\gamma n}{\varphi }_{\sigma ,A}(x_0-\overline{k})\hfill \\ & \le \sum _{|\overline{k}|>\gamma n-1}{\varphi }_{\sigma ,A}(x_0-\overline{k})\hfill \\ & \le C\underset{x_0\in [0,1)}{sup}\sum _{|\overline{k}|>\gamma n-1}{|x_0-\overline{k}|}^{-1-\delta }\hfill \\ & \le C\left(\sum _{\overline{k}>\gamma n-1}|1-\overline{k}{|}^{-1-\delta }+\sum _{\overline{k}<1-\gamma n}{\left|\overline{k}\right|}^{-1-\delta }\right)\to 0,n\to +\infty .\hfill \end{array}$$

□

Lemma 2. 

Let ${\varphi }_{\sigma ,A}\left(x\right)$ be defined as in 4. For any $x\in [-1,1]$, $n\in \mathbb{N}$, the following inequality holds:

$$\sum _{k=-2n}^{2n-1}{\varphi }_{\sigma ,A}(nx-k)\ge {\varphi }_{\sigma ,A}\left(3\right).$$

Proof. 

For all $k\in [-2n,2n-1]\bigcap \mathbb{Z}$, it is easy to see that

$$\begin{array}{c}\hfill \sum _{k=-2n}^{2n-1}{\varphi }_{\sigma ,A}(nx-k)=\sum _{k=-2n}^{2n-1}{\varphi }_{\sigma ,A}\left(\right|nx-k\left|\right)\ge {\varphi }_{\sigma ,A}\left(\right|nx-k_0\left|\right).\end{array}$$

Further, we can fix a specific $k_0\in \mathbb{Z}$ such that $|nx-k_0|\le 1$; therefore,

$${\varphi }_{\sigma ,A}\left(\right|nx-k_0\left|\right)\ge {\varphi }_{\sigma ,A}\left(3\right)>0,$$

which completes the proof of Lemma 2. □

3. Degree of the Approximation by $K_{n,A}(f,x)$ and $K_{n,A}^{\ast }(f,x)$ in $C[-1,1]$

The main aim of this section is to prove the convergence theorem as well as the quantitative approximation theorem of operators $K_{n,A}^{\ast }\left(f\right)$ and $K_{n,A}\left(f\right)$ to functions $f\in C[-1,1]$. For the EKNNO $K_{n,A}^{\ast }\left(f\right)$, Theorem 3 can be established.

Theorem 3. 

Assume that the function $\sigma \left(x\right)$ satisfies the condition in Theorem 1, and then for $f\in C[-1,1]$, there exists a constant $C>0$ such that

$$\begin{array}{cc}\hfill \parallel f-K_{n,A}^{\ast }{\left(f\right)\parallel }_{\infty }& \le C\left[{\parallel f\parallel }_{\infty }\left[\frac{1}{\delta A^{1+\delta }}+\frac{1}{\delta }{\left(\frac{A}{n}\right)}^{\delta }+\frac{A^{\delta }}{n^{\delta (1-\alpha )-\alpha }}\right]\right.\hfill \\ \hfill & \left.+\left(2+\frac{1}{n^{1-\alpha }}\right)\left(1+\frac{1}{\delta A^{1+\delta }}\right)\omega \left(f,\frac{1}{n^{\alpha }}\right)\right],\hfill \end{array}$$

where ${\displaystyle \alpha =\frac{1}{2}-\frac{1}{|\delta -2|+2},\delta >1}$.

The symbol $\omega (f,t)$ in Theorem 3 denotes the modulus of continuity of f [23], defined by

$${\displaystyle \omega (f,t):=\underset{x,y\in [-1,1],|x-y|\le t}{sup}|f\left(x\right)-f\left(y\right)|,t>0.}$$

(9)

Proof. 

In view of Theorem 1,

$$\sum _{k=-\infty }^{+\infty }{\varphi }_{\sigma ,A}(nx-k)=1+2\sum _{k=1}^{+\infty }{\widehat{\varphi }}_{\sigma ,A}\left(Ak\right)cos2kn\pi x,$$

then

$$\begin{array}{cc}& f\left(x\right)-K_{n,A}^{\ast }(f,x)\hfill \\ & =\sum _{k\in \mathbb{Z}}f\left(x\right){\varphi }_{\sigma ,A}(nx-k)-K_{n,A}^{\ast }(f,x)-2f\left(x\right)\sum _{k=1}^{+\infty }{\widehat{\varphi }}_{\sigma ,A}\left(Ak\right)cos2kn\pi x\hfill \\ & =\sum _{k=-\infty }^{-2n-1}f\left(x\right){\varphi }_{\sigma ,A}(nx-k)+\sum _{k=2n}^{+\infty }f\left(x\right){\varphi }_{\sigma ,A}(nx-k)\hfill \\ & +\sum _{k=-2n}^{2n-1}\left[f\left(x\right)-n{\int }_{\frac{k}{n}}^{\frac{k+1}{n}}{f}_{1,e}\left(u\right)\mathrm{d}u\right]{\varphi }_{\sigma ,A}(nx-k)-2f\left(x\right)\sum _{k=1}^{+\infty }{\widehat{\varphi }}_{\sigma ,A}\left(Ak\right)cos2kn\pi x\hfill \\ & :=I_1+I_2+I_3+I_4.\hfill \end{array}$$

Now, we estimate $I_1,I_2,I_3,I_4$ separately.

For $I_1,x\in [-1,1],k\le -2n-1$, we have $nx-k\ge 2n+1+nx\ge n+1$. By the definition of ${\varphi }_{\sigma ,A}\left(x\right)$ and the property that $|{\varphi }_{\sigma }{\left(x\right)|\le C(1+\left|x\right|)}^{-1-\delta }$, we have:

$$\begin{array}{cc}\hfill |I_1|& \le {\parallel f\parallel }_{\infty }\sum _{k=-\infty }^{-2n-1}\left|{\varphi }_{\sigma ,A}(nx-k)\right|\le C{\parallel f\parallel }_{\infty }\frac{1}{A}\sum _{k=-\infty }^{-2n-1}\frac{A^{1+\delta }}{{(nx-k)}^{1+\delta }}\hfill \\ & ={C\parallel f\parallel }_{\infty }{A}^{\delta }\sum _{k=n+1}^{\infty }\frac{1}{k^{1+\delta }}\le C{\parallel f\parallel }_{\infty }\frac{1}{\delta }{\left(\frac{A}{n}\right)}^{\delta }.\hfill \end{array}$$

Similarly,

$$\begin{array}{c}\hfill |I_2|=\left|\sum _{k=2n}^{+\infty }f\left(x\right){\varphi }_{\sigma ,A}(nx-k)\right|\le C{\parallel f\parallel }_{\infty }\frac{1}{\delta }{\left(\frac{A}{n}\right)}^{\delta },\end{array}$$

(10)

and

$$\begin{array}{c}\hfill |I_4|=\left|-2f\left(x\right)\sum _{k=1}^{+\infty }{\widehat{\varphi }}_{\sigma ,A}\left(Ak\right)cos2kn\pi x\right|\le {C\parallel f\parallel }_{\infty }\sum _{k=1}^{+\infty }\frac{1}{{\left(Ak\right)}^{1+\delta }}\le \frac{C}{\delta A^{1+\delta }}{\parallel f\parallel }_{\infty }.\end{array}$$

(11)

While for $I_3$,

$$\begin{array}{cc}\hfill |I_3|& \le \sum _{k=-2n}^{2n}\left|f\left(x\right)-n{\int }_{\frac{k}{n}}^{\frac{k+1}{n}}{f}_{1,e}\left(u\right)\mathrm{d}u\right|·\left|{\varphi }_{\sigma ,A}(nx-k)\right|\hfill \\ & \le \left(\sum _{\left|\frac{k}{n}-x\right|\le \frac{1}{n^{\alpha }}}+\sum _{\left|\frac{k}{n}-x\right|>\frac{1}{n^{\alpha }}}\right)\left|f\left(x\right)-n{\int }_{\frac{k}{n}}^{\frac{k+1}{n}}{f}_{1,e}\left(u\right)\mathrm{d}u\right|·\left|{\varphi }_{\sigma ,A}(nx-k)\right|\hfill \\ & :=\left|I_3^1\right|+\left|I_3^2\right|,\hfill \end{array}$$

then

$$\begin{array}{cc}\hfill \left|I_3^1\right|& =\sum _{\left|\frac{k}{n}-x\right|\le \frac{1}{n^{\alpha }}}n{\int }_{\frac{k}{n}}^{\frac{k+1}{n}}\left|f\left(x\right)-f_{1,e}\left(u\right)\right|\mathrm{d}u\left|{\varphi }_{\sigma ,A}(nx-k)\right|\hfill \\ & \le \sum _{\left|\frac{k}{n}-x\right|\le \frac{1}{n^{\alpha }}}n{\int }_{\frac{k}{n}}^{\frac{k+1}{n}}\omega (f,|u-x\left|\right)\mathrm{d}u\left|{\varphi }_{\sigma ,A}(nx-k)\right|\hfill \\ & \le \sum _{\left|\frac{k}{n}-x\right|\le \frac{1}{n^{\alpha }}}n{\int }_{\frac{k}{n}}^{\frac{k+1}{n}}\left[1+n^{\alpha }|u-x|\right]\omega \left(f,\frac{1}{n^{\alpha }}\right)\mathrm{d}u\left|{\varphi }_{\sigma ,A}(nx-k)\right|\hfill \\ & \le \sum _{\left|\frac{k}{n}-x\right|\le \frac{1}{n^{\alpha }}}n\omega \left(f,\frac{1}{n^{\alpha }}\right){\int }_{\frac{k}{n}}^{\frac{k+1}{n}}\left[1+n^{\alpha }\left|u-\frac{k}{n}\right|+n^{\alpha }\left|\frac{k}{n}-x\right|\right]\mathrm{d}u\left|{\varphi }_{\sigma ,A}(nx-k)\right|\hfill \\ & \le \sum _{\left|\frac{k}{n}-x\right|\le \frac{1}{n^{\alpha }}}n\omega \left(f,\frac{1}{n^{\alpha }}\right){\int }_{\frac{k}{n}}^{\frac{k+1}{n}}\left[1+n^{\alpha }·\frac{1}{n}+n^{\alpha }·\frac{1}{n^{\alpha }}\right]\mathrm{d}u\left|{\varphi }_{\sigma ,A}(nx-k)\right|\hfill \\ & \le n\omega \left(f,\frac{1}{\sqrt{n}}\right)·\left(2+\frac{1}{n^{1-\alpha }}\right)·\frac{1}{n}\sum _{k=-\infty }^{+\infty }\left|{\varphi }_{\sigma ,A}(nx-k)\right|.\hfill \end{array}$$

Furthermore, because

$$\begin{array}{c}\hfill \sum _{k=-\infty }^{+\infty }\left|{\varphi }_{\sigma ,A}(nx-k)\right|\le 1+\frac{C}{\delta A^{1+\delta }}\le C\left(1+\frac{1}{\delta A^{1+\delta }}\right),\end{array}$$

(12)

thus

$$\begin{array}{c}\hfill \left|I_3^1\right|\le C\left(2+\frac{1}{n^{1-\alpha }}\right)\left(1+\frac{1}{\delta A^{1+\delta }}\right)\omega \left(f,\frac{1}{n^{\alpha }}\right),\end{array}$$

while

$$\begin{array}{cc}\hfill \left|I_3^2\right|& =\sum _{\left|\frac{k}{n}-x\right|>\frac{1}{n^{\alpha }}}\left|f\left(x\right)-n{\int }_{\frac{k}{n}}^{\frac{k+1}{n}}f\left(u\right)\mathrm{d}u\right|·\left|{\varphi }_{\sigma ,A}(nx-k)\right|\hfill \\ & \le {2\parallel f\parallel }_{\infty }\sum _{\left|\frac{k}{n}-x\right|>\frac{1}{n^{\alpha }}}\frac{1}{A}·\frac{A^{1+\delta }}{{|nx-k|}^{1+\delta }}\hfill \\ & \le {C\parallel f\parallel }_{\infty }{A}^{\delta }\sum _{\left|\frac{k}{n}-x\right|>\frac{1}{n^{\alpha }}}\frac{1}{n^{(1+\delta )(1-\alpha )}}\le C{\parallel f\parallel }_{\infty }{A}^{\delta }{\int }_{n^{1-\alpha }}^{\infty }\frac{1}{t^{1+\delta }}\mathrm{d}t\hfill \\ & \le {C\parallel f\parallel }_{\infty }{A}^{\delta }\frac{1}{\delta }\frac{1}{n^{\delta (1-\alpha )}}.\hfill \end{array}$$

Therefore,

$$\begin{array}{c}\hfill \left|I_3\right|\le C\left[\left(2+\frac{1}{n^{1-\alpha }}\right)\left(1+\frac{1}{\delta A^{1+\delta }}\right)\omega \left(f,\frac{1}{n^{\alpha }}\right)+{\parallel f\parallel }_{\infty }{A}^{\delta }\frac{1}{\delta n^{\delta (1-\alpha )}}\right].\end{array}$$

Combining the above estimates together, we have

$$\begin{array}{cc}\hfill \parallel f-K_{n,A}^{\ast }{\left(f\right)\parallel }_{\infty }& \le C\left[{\parallel f\parallel }_{\infty }\left(\frac{1}{\delta A^{1+\delta }}+\frac{1}{\delta }{\left(\frac{A}{n}\right)}^{\delta }+\frac{A^{\delta }}{\delta n^{\delta (1-\alpha )}}\right)+\right.\hfill \\ & \left.\left(2+\frac{1}{n^{1-\alpha }}\right)\left(1+\frac{1}{\delta A^{1+\delta }}\right)\omega \left(f,\frac{1}{n^{\alpha }}\right)\right].\hfill \end{array}$$

□

Remark 2. 

For a fixed A, the terms ${\displaystyle \frac{A}{n},\frac{A^{\delta }}{n^{\delta (1-\alpha )}}}$, and $\omega \left({\displaystyle f,\frac{1}{n^{\alpha }}}\right)$ are approaching 0 when $n\to \infty$; consequently,

$${\displaystyle \underset{n\to +\infty }{lim}\parallel f-K_{n,A}^{\ast }{\left(f\right)\parallel }_{\infty }\le \frac{C}{A^{1+\delta }}{\parallel f\parallel }_{\infty }}$$

for some constant $C>0$. Especially, if we choose $A=1$, we have

$${\displaystyle \underset{n\to +\infty }{lim}\parallel f-K_{n,A}^{\ast }{\left(f\right)\parallel }_{\infty }\le C{\parallel f\parallel }_{\infty }.}$$

Remark 3. 

Taking the activation function ${\displaystyle y=\sigma \left(x\right)=\frac{1}{1+e^{-x}}}$, then ${\displaystyle \sum _{k=-\infty }^{+\infty }{\varphi }_{\sigma }(x-k)=1}$ in this case. Following the similar (but more simple) procedure, we have

$${∥f-K_{n,A}^{\ast }\left(f\right)∥}_{\infty }\le C\left[{\parallel f\parallel }_{\infty }\left(\frac{1}{\delta }{\left(\frac{A}{n}\right)}^{\delta }+\frac{A^{\delta }}{n^{\delta (1-\alpha )-\alpha }}\right)+\left(2+\frac{1}{n^{1-\alpha }}\right)\omega \left(f,\frac{1}{n^{\alpha }}\right)\right]\to 0,$$

when $n\to +\infty .$

Remark 4. 

Notice that in Theorem 3, we set

$${\displaystyle \alpha =\frac{1}{2}-\frac{1}{|\delta -2|+2},\delta >0.}$$

Then, it is easy to verify that

$${\displaystyle \delta (1-\alpha )=\frac{\delta |\delta -2|+4\delta }{2|\delta -2|+4}>\frac{\delta }{2},}$$

as shown in Figure 1. Moreover, for $\delta \to +\infty$, α tends to ${\displaystyle \frac{1}{2}}$.

In approximation theory, Theorem 3 is called a direct theorem of approximation by the operators, which gives the upper bound of the approximation. The direct theorems of the Kantorovich-types operators were investigated in much of the literature, see, for example, [24,25], etc.

The results on the upper bound imply the convergence of the NN operators to the target function and also provide a quantitative measurement of how accurately the target function can be approximated.

As for the NEKNNO $K_{n,A}\left(f\right)$, we have the following theorem.

Theorem 4. 

Assume that $f:[-1,1]\to \mathbb{R}$ is bounded on $[-1,1]$ and continuous at $x_0\in \mathbb{R}$, then

$$\begin{array}{c}\hfill \underset{n\to +\infty }{lim}{K}_{n,A}(f,x_0)=f\left(x_0\right).\end{array}$$

(13)

Furthermore, if $f\in C[-1,1]$, we have

$$\begin{array}{c}\hfill \underset{n\to +\infty }{lim}{∥K_{n,A}\left(f\right)-f∥}_{\infty }=0.\end{array}$$

(14)

Proof. 

f is continuous at $x=x_0$⇔ For any $\epsilon >0$, there exists a $\gamma >0$, such that for arbitrary $y\in [-1,1]\bigcap [x-\gamma ,x+\gamma ]$, it holds $|f\left(x_0\right)-f\left(y\right)|<\epsilon$. While for $u\in [k/n,(k+1)/n]$ and $|n{x}_0-k|\le n\gamma /2$, we have

$$\begin{array}{c}\hfill |u-x_0|\le \left|u-\frac{k}{n}\right|+\left|\frac{k}{n}-x_0\right|\le \frac{1}{n}+\frac{\gamma }{2},\end{array}$$

then

$$\begin{array}{cc}\hfill \left|K_{n,A}(f,x_0)-f\left(x_0\right)\right|& \le \left|K_{n,A}\left(\right|f-f_x|,x_0)\right|\hfill \\ & \le \frac{1}{{\varphi }_{\sigma ,A}\left(3\right)}\sum _{k=-2n}^{2n}n{\int }_{\frac{k}{n}}^{\frac{k+1}{n}}|f\left(u\right)-f\left(x_0\right)|\mathrm{d}u{\varphi }_{\sigma ,A}(n{x}_0-k)\hfill \\ & \le \frac{1}{{\varphi }_{\sigma ,A}\left(3\right)}max\left\{{\displaystyle \sum _{|nx-k|\le n\gamma /2},\sum _{|nx-k|>n\gamma /2}}\right\}n\times \hfill \\ & {\int }_{\frac{k}{n}}^{\frac{k+1}{n}}|f\left(u\right)-f\left(x_0\right)|\mathrm{d}u{\varphi }_{\sigma ,A}(n{x}_0-k)\hfill \\ & :=\frac{1}{{\varphi }_{\sigma ,A}\left(3\right)}max(J_1,J_2).\hfill \end{array}$$

Notice that $|u-x_0|<\gamma$ for sufficiently large $n\in \mathbb{N}$.

Estimating $J_1,J_2$, respectively, we have:

$$\begin{array}{cc}\hfill J_1& \le \sum _{|nx-k|\le n\gamma /2}n{\int }_{\frac{k}{n}}^{\frac{k+1}{n}}|f\left(u\right)-f\left(x_0\right)|\mathrm{d}u{\varphi }_{\sigma ,A}(n{x}_0-k)\hfill \\ & \le n{\int }_{\frac{k}{n}}^{\frac{k+1}{n}}\epsilon \mathrm{d}u\sum _{k=-\infty }^{+\infty }\left|{\varphi }_{\sigma ,A}(n{x}_0-k)\right|\le C\left(1+\frac{1}{A^{1+\delta }}\right)\epsilon ;\hfill \\ \hfill J_2& \le \sum _{|nx-k|>n\gamma /2}n{\int }_{\frac{k}{n}}^{\frac{k+1}{n}}2{\parallel f\parallel }_{\infty }\mathrm{d}u·{\varphi }_{\sigma ,A}(n{x}_0-k)\to 0,n\to +\infty .\hfill \end{array}$$

$$\Rightarrow \underset{n\to +\infty }{lim}{K}_{n,A}(f,x_0)=f\left(x_0\right).$$

Relation (13) is proved, and (14) can be proved similarly. □

Remark 5. 

In Figure 2, we compare the approximation efficiency of the NEKNNO $K_{n,A}(f,x)$ and $N_n(f,x)$ when both approximate to the quadratic function $f\left(x\right)=x^2$ on $[-1,1]$, while parameters n and A vary. It is very clear to see from Figure 2a–c that the NEKNNO $K_{n,A}(f,x)$ has a better approximation performance, especially at the endpoints. At the same time, the change in the parameter A in Figure 2d obviously affects the approximation efficiency of the NEKNNO $K_{n,A}(f,x)$ at the endpoints. Then, whether the optimal solution of A exists or not is an open question worth discussing.

4. Degree of the Approximation by NEKNNO $K_{n,A}\left(f\right)$ and EKNNO $K_{n,A}^{\ast }\left(f\right)$ in $L_p[-1,1]$

Now, consider the efficiency of the approximation of f by proposed operators EKNNO $K_{n,A}^{\ast }\left(f\right)$ and NEKNNO $K_{n,A}\left(f\right)$ in the Lebesgue space $L_p[-1,1](1\le p<+\infty )$, where

$$L_p[-1,1]=\left\{f:{\int }_{-1}^1{\left|f\left(x\right)\right|}^p\mathrm{d}x<+\infty \right\},$$

equipped with the norm

$${\parallel f\parallel }_p={\left({\int }_{-1}^1{\left|f\left(x\right)\right|}^p\mathrm{d}x\right)}^{1/p},1\le p<+\infty .$$

We first give Lemma 3.

Lemma 3. 

For any functions $f,g\in L_p[-1,1],1\le p<+\infty$, the following inequality holds:

$${∥K_{n,A}\left(f\right)-K_{n,A}\left(g\right)∥}_p\le \frac{C}{{\varphi }_{\sigma ,A}^{1/p}\left(3\right)}{∥f-g∥}_p.$$

Proof. 

By the definition of ${\parallel ·\parallel }_p$, Theorem 2, Lemma 2, and applying Jenson’s inequality due to the convexity of ${|·|}^p(1\le p<+\infty )$, we have

$$\begin{array}{cc}\hfill {∥K_{n,A}\left(f\right)-K_{n,A}\left(g\right)∥}_p^p& ={\int }_{-1}^1{\left|K_{n,A}(f,x)-K_{n,A}(g,x)\right|}^p\mathrm{d}x\le {\int }_{-1}^1{\left|K_{n,A}\left(\right|f-g|,x)\right|}^p\mathrm{d}x\hfill \\ & {\displaystyle \le {\int }_{-1}^1\frac{{\displaystyle \sum _{k=-2n}^{2n-1}{\varphi }_{\sigma ,A}(nx-k){\left(n{\int }_{\frac{k}{n}}^{\frac{k+1}{n}}\left|f_{1,e}\left(u\right)-g_{1,e}\left(u\right)\right|\mathrm{d}u\right)}^p}}{{\displaystyle \sum _{k=-2n}^{2n-1}{\varphi }_{\sigma ,A}(nx-k)}}\mathrm{d}x}\hfill \\ & \le \frac{1}{{\varphi }_{\sigma ,A}\left(3\right)}{\int }_{-1}^1\sum _{k=-2n}^{2n-1}{\varphi }_{\sigma ,A}(nx-k){\left(n{\int }_{\frac{k}{n}}^{\frac{k+1}{n}}\left|f_{1,e}\left(u\right)-g_{1,e}\left(u\right)\right|\mathrm{d}u\right)}^p\mathrm{d}x\hfill \\ & \le \frac{1}{{\varphi }_{\sigma ,A}\left(3\right)}{\int }_{\mathbb{R}}\sum _{k=-2n}^{2n-1}n{\varphi }_{\sigma ,A}(nx-k)\mathrm{d}x{\int }_{\frac{k}{n}}^{\frac{k+1}{n}}{\left|f_{1,e}\left(u\right)-g_{1,e}\left(u\right)\right|}^p\mathrm{d}u.\hfill \end{array}$$

In view of Theorem 1, we obtain

$$\begin{array}{c}\hfill {\int }_{\mathbb{R}}n{\varphi }_{\sigma ,A}(nx-k)\mathrm{d}x={\int }_{\mathbb{R}}{\varphi }_{\sigma }\left(\frac{nx-k}{A}\right)\mathrm{d}\frac{nx}{A}=1.\end{array}$$

Therefore,

$$\begin{array}{cc}\hfill \parallel K_{n,A}\left(f\right)-K_{n,A}{\left(g\right)\parallel }_p^p& \le \frac{1}{{\varphi }_{\sigma ,A}\left(3\right)}\sum _{k=-2n}^{2n-1}{\int }_{\frac{k}{n}}^{\frac{k+1}{n}}{\left|f_{1,e}\left(u\right)-g_{1,e}\left(u\right)\right|}^p\mathrm{d}u\hfill \\ & =\frac{1}{{\varphi }_{\sigma ,A}\left(3\right)}{\int }_{-2}^2{\left|f_{1,e}\left(u\right)-g_{1,e}\left(u\right)\right|}^p\mathrm{d}u\le \frac{C}{{\varphi }_{\sigma ,A}\left(3\right)}{\parallel f-g\parallel }_p^p.\hfill \end{array}$$

This completes the proof of Lemma 3. □

Utilizing Lemma 3, we can prove the convergence of $K_{n,A}\left(f\right)$ in $L_p[-1,1]$.

Theorem 5. 

Assume that $f\in L_p[-1,1]$, then

$$\underset{n\to +\infty }{lim}{∥K_{n,A}\left(f\right)-f∥}_p=0.$$

Proof. 

It is well-known that $C[-1,1]$ is dense in $L_p[-1,1]$. Let $f\in L_p[-1,1]$. Then, for any $\epsilon >0$, there exists a function $g\in C[-1,1]$ such that ${\parallel f-g\parallel }_p\le \epsilon$. By Lemma 3 and (14),

$$\begin{array}{cc}\hfill {∥K_{n,A}\left(g\right)-g∥}_p& ={\left[{\int }_{-1}^1{\left|K_{n,A}(g,x)-g\left(x\right)\right|}^p\mathrm{d}x\right]}^{1/p}\hfill \\ & \le {\left[{\int }_{-1}^1{∥K_{n,A}(g,x)-g\left(x\right)∥}_{\infty }^p\mathrm{d}x\right]}^{1/p}\hfill \\ & \le 2^{1/p}\epsilon .\hfill \end{array}$$

Therefore, for sufficiently large $n\in \mathbb{N}$, we have

$$\begin{array}{cc}\hfill {∥K_{n,A}\left(f\right)-f∥}_p& \le {∥K_{n,A}\left(f\right)-K_{n,A}\left(g\right)∥}_p+{∥K_{n,A}\left(g\right)-g∥}_p+{\parallel f-g\parallel }_p\hfill \\ & \le \left(C+\frac{1}{{\varphi }_{\sigma ,A}^{1/p}\left(3\right)}\right){\parallel f-g\parallel }_p+{∥K_{n,A}\left(g\right)-g∥}_p\hfill \\ & \le \left(C+\frac{1}{{\varphi }_{\sigma ,A}^{1/p}\left(3\right)}+2^{1/p}\right)\epsilon .\hfill \end{array}$$

Theorem 5 is proved. □

Next, we consider $K_{n,A}^{\ast }\left(f\right)$. We need Lemma 4, whose proof is similar to that of Lemma 3.

Lemma 4. 

For any functions $f,g\in L_p[-1,1],1\le p<+\infty$, the following inequality holds:

$${∥K_{n,A}^{\ast }\left(f\right)-K_{n,A}^{\ast }\left(g\right)∥}_p\le {\parallel f-g\parallel }_p.$$

Now, we can establish the approximation theorem of $K_{n,A}^{\ast }(f,x)$ in $L_p[-1,1]$ space.

Theorem 6. 

Let $C^1[-1,1]$ denote the set of all functions that are differentiable and have continuous derivatives on $[-1,1]$. Then, for $f\in L_p[-1,1],1\le p<+\infty$, $g\in C^1[-1,1]$, the following inequality holds:

$${∥K_{n,A}^{\ast }\left(f\right)-f∥}_p\le \underset{g\in C^1[-1,1]}{inf}\left\{2{∥f-g∥}_p+\frac{M_{A,n,\delta }^1}{{\varphi }_{\sigma ,A}^{1/p}\left(3\right)}{∥g_{1,e}∥}_{L_p[-2,2]}+\frac{M_{A,n,\delta }^2}{n{\varphi }_{\sigma ,A}\left(3\right)}{\parallel g^{\prime }\parallel }_{\infty }\right\},$$

where ${\displaystyle {∥g_{1,e}∥}_{L_p[-2,2]}={\left({\int }_{-2}^2{\left|g_{1,e}\left(u\right)\right|}^p\mathrm{d}u\right)}^{1/p}}$, ${\displaystyle M_{A,n,\delta }^1=\frac{C}{\delta }\left[\frac{1}{A^{1+\delta }}+{\left(\frac{A}{n}\right)}^{\delta }\right],}$and $M_{A,n,\delta }^2=C\left[\left(\frac{1}{n^{\frac{1}{2(\delta -1)}}}+1\right)\left({\displaystyle 1+\frac{1}{A^{1+\delta }}}\right)+\frac{1}{\delta -1}\sqrt{n}\right].$

Proof. 

Let $f\in L_p[-1,1],1\le p<+\infty$, and $g\left(x\right)\in C^1[-1,1]$. By Minkowski’s inequality,

$$\begin{array}{cc}\hfill {∥K_{n,A}^{\ast }\left(f\right)-f∥}_p& \le {∥K_{n,A}^{\ast }\left(f\right)-K_{n,A}^{\ast }\left(g\right)∥}_p+{∥K_{n,A}^{\ast }\left(g\right)-K_{n,A}\left(g\right)∥}_p\hfill \\ & +{∥K_{n,A}\left(g\right)-g∥}_p+{∥f-g∥}_p.\hfill \end{array}$$

In view of Lemma 4,

$$\begin{array}{c}\hfill {∥K_{n,A}^{\ast }\left(f\right)-K_{n,A}^{\ast }\left(g\right)∥}_p\le {\parallel f-g\parallel }_p.\end{array}$$

(15)

Therefore,

$$\begin{array}{cc}\hfill {∥K_{n,A}^{\ast }\left(f\right)-f∥}_p& \le {∥K_{n,A}^{\ast }\left(g\right)-K_{n,A}\left(g\right)∥}_p+{∥K_{n,A}\left(g\right)-g∥}_p+2{∥f-g∥}_p\hfill \\ & :=S_1+S_2+2{∥f-g∥}_p.\hfill \end{array}$$

According to the definition of $K_{n,A}^{\ast }$ and $K_{n,A}$, we have

$$S_1={∥K_{n,A}^{\ast }\left(g\right)-K_{n,A}\left(g\right)∥}_p\le \left|1-\sum _{k=-2n}^{2n-1}{\varphi }_{\sigma ,A}(nx-k)\right|{∥K_{n,A}\left(g\right)∥}_p.$$

Then, in view of (10) and (11),

$$\begin{array}{cc}\hfill \left|1-\sum _{k=-2n}^{2n-1}{\varphi }_{\sigma ,A}(nx-k)\right|& \le \sum _{k=-\infty }^{-2n-1}{\varphi }_{\sigma ,A}(nx-k)+\sum _{k=2n}^{+\infty }{\varphi }_{\sigma ,A}(nx-k)\hfill \\ & +2\sum _{k=1}^{+\infty }\left|{\widehat{\varphi }}_{\sigma ,A}\left(Ak\right)cos2kn\pi x\right|\hfill \\ & \le \frac{C}{\delta }\left[\frac{1}{A^{1+\delta }}+{\left(\frac{A}{n}\right)}^{\delta }\right].\hfill \end{array}$$

By Jensen’s inequality, Hölder’s inequality, and Lemma 2,

$$\begin{array}{cc}\hfill {∥K_{n,A}\left(g\right)∥}_p^p& {\displaystyle ={\int }_{-1}^1{\left|\frac{{\displaystyle \sum _{k=-2n}^{2n-1}n{\int }_{\frac{k}{n}}^{\frac{k+1}{n}}{g}_{1,e}\left(u\right)\mathrm{d}u{\varphi }_{\sigma ,A}(nx-k)}}{{\displaystyle \sum _{k=-2n}^{2n-1}{\varphi }_{\sigma ,A}(nx-k)}}\right|}^p\mathrm{d}x}\hfill \\ & {\displaystyle \le {\int }_{-1}^1\frac{{\displaystyle \sum _{k=-2n}^{2n-1}{\left(n{\int }_{\frac{k}{n}}^{\frac{k+1}{n}}{g}_{1,e}\left(u\right)\mathrm{d}u\right)}^p{\varphi }_{\sigma ,A}(nx-k)}}{{\displaystyle \sum _{k=-2n}^{2n-1}{\varphi }_{\sigma ,A}(nx-k)}}\mathrm{d}x}\hfill \\ & \le \frac{1}{{\varphi }_{\sigma ,A}\left(3\right)}{\int }_{-1}^1\sum _{k=-2n}^{2n-1}{\left(n{\int }_{\frac{k}{n}}^{\frac{k+1}{n}}{g}_{1,e}\left(u\right)\mathrm{d}u\right)}^p{\varphi }_{\sigma ,A}(nx-k)\mathrm{d}x\hfill \\ & \le \frac{1}{{\varphi }_{\sigma ,A}\left(3\right)}{\int }_{\mathbb{R}}\sum _{k=-2n}^{2n-1}n{\varphi }_{\sigma ,A}(nx-k)\mathrm{d}x{\int }_{\frac{k}{n}}^{\frac{k+1}{n}}{\left|g_{1,e}\left(u\right)\right|}^p\mathrm{d}u\hfill \\ & \le \frac{1}{{\varphi }_{\sigma ,A}\left(3\right)}\sum _{k=-2n}^{2n-1}{\int }_{\frac{k}{n}}^{\frac{k+1}{n}}{\left|g_{1,e}\left(u\right)\right|}^p\mathrm{d}u=\frac{1}{{\varphi }_{\sigma ,A}\left(3\right)}{∥g_{1,e}∥}_{L_p[-2,2]}^p\hfill \\ & \Rightarrow S_1={∥K_{n,A}^{\ast }\left(g\right)-K_{n,A}\left(g\right)∥}_p\hfill \end{array}$$

(16)

$$\begin{array}{cc}& \le \frac{C}{\delta {\varphi }_{\sigma ,A}^{1/p}\left(3\right)}\left[\frac{1}{A^{1+\delta }}+{\left(\frac{A}{n}\right)}^{\delta }\right]{∥g_{1,e}∥}_{L_p[-2,2]}.\hfill \end{array}$$

(17)

For $S_2$, $\forall x,t\in [-1,1]$, according to the Lagrange mean value theorem, we have

$$|g\left(t\right)-g\left(x\right)|\le \parallel g^{\prime }{\parallel }_{\infty }|t-x|.$$

Therefore,

$$\begin{array}{cc}\hfill \left|K_{n,A}(g,x)-g\left(x\right)\right|& \le \parallel g^{\prime }{\parallel }_{\infty }{K}_{n,A}\left(\right|t-x|,x)\hfill \\ & \le \frac{\parallel g^{\prime }{\parallel }_{\infty }}{{\sigma }_{\sigma ,A}\left(3\right)}\left[\sum _{k=-2n}^{2n-1}n{\int }_{\frac{k}{n}}^{\frac{k+1}{n}}\left(\left|x-\frac{k}{n}\right|+\left|\frac{k}{n}-u\right|\right)\mathrm{d}u{\varphi }_{\sigma ,A}(nx-k)\right]\hfill \\ & \le \frac{\parallel g^{\prime }\parallel \infty }{n{\sigma }_{\sigma ,A}\left(3\right)}\left[\sum _{k=-2n}^{2n-1}|nx-k|{\varphi }_{\sigma ,A}(nx-k)+\sum _{k=-2n}^{2n-1}{\varphi }_{\sigma ,A}(nx-k)\right].\hfill \end{array}$$

Let $\beta \in (0,1)$. We have

$$\begin{array}{cc}\hfill \sum _{k=-2n}^{2n-1}|nx-k|{\varphi }_{\sigma ,A}(nx-k)& \le \sum _{\left|\frac{k}{n}-x\right|\le \frac{1}{n^{\beta }}}|nx-k|{\varphi }_{\sigma ,A}(nx-k)\hfill \\ & +\sum _{\left|\frac{k}{n}-x\right|>\frac{1}{n^{\beta }}}|nx-k|{\varphi }_{\sigma ,A}(nx-k)\hfill \\ & \le \frac{1}{n^{\beta }}\sum _{k=-\infty }^{+\infty }{\varphi }_{\sigma ,A}(nx-k)+C\sum _{\left|\frac{k}{n}-x\right|>\frac{1}{n^{\beta }}}\frac{|nx-k|}{{|nx-k|}^{1+\delta }}\hfill \\ & \le \frac{C}{n^{\beta }}\left(1+\frac{1}{\delta A^{1+\delta }}\right)+C{\int }_{\frac{1}{n^{\beta }}}^{+\infty }\frac{1}{t^{\delta }}\mathrm{d}t\hfill \\ & =C\left[\frac{1}{n^{\beta }}\left(1+\frac{1}{\delta A^{1+\delta }}\right)+\frac{1}{\delta -1}{n}^{\beta (\delta -1)}\right].\hfill \end{array}$$

Meanwhile, by (12),

$$\sum _{k=-2n}^{2n-1}{\varphi }_{\sigma ,A}(nx-k)\le \sum _{k=-\infty }^{+\infty }{\varphi }_{\sigma ,A}(nx-k)\le C\left(1+\frac{1}{\delta A^{1+\delta }}\right),$$

setting ${\displaystyle \beta =\frac{1}{2(\delta -1)}}$, we have

$$\begin{array}{c}\hfill \left|K_{n,A}(g,x)-g\left(x\right)\right|\le \frac{C\parallel g^{\prime }{\parallel }_{\infty }}{n{\varphi }_{\sigma ,A}\left(3\right)}\left[\left(\frac{1}{n^{\frac{1}{2(\delta -1)}}}+1\right)\left(1+\frac{1}{\delta A^{1+\delta }}\right)+\frac{1}{\delta -1}\sqrt{n}\right].\end{array}$$

(18)

In summary, combined with Equations (15)–(17), Theorem 6 is proved. □

Remark 6. 

If we take ${\displaystyle y=\sigma \left(x\right)=\frac{1}{1+e^{-x}}}$ as the activation function, then we have

$${\displaystyle \sum _{k=-\infty }^{+\infty }{\varphi }_{\sigma }(x-k)=1.}$$

Consequently,

$$\begin{array}{cc}\hfill {∥K_{n,A}^{\ast }\left(f\right)-f∥}_p& \le \underset{g\in C^1[-1,1]}{inf}\left\{2{∥f-g∥}_p+\frac{C}{\delta {\varphi }_{\sigma ,A}\left(3\right)}{\left(\frac{A}{n}\right)}^{\delta }{∥g_{1,e}∥}_{L_p[-2,2]}\right.\hfill \\ & \left.+\frac{C\parallel g^{\prime }{\parallel }_{\infty }}{n{\varphi }_{\sigma ,A}^{1/p}\left(3\right)}\left[\left(\frac{1}{n^{\frac{1}{2(\delta -1)}}}+1\right)+\frac{1}{\delta -1}\sqrt{n}\right]\right\}.\hfill \end{array}$$

Choosing appropriate $g\in C^1[-1,1]$ with $\delta >1$ and $n\in N$ leads to

$${\displaystyle \underset{n\to +\infty }{lim}{\parallel f-K_{n,A}^{\ast }\left(f\right)\parallel }_p=0.}$$

If we define g as the Steklov mean function of f, that is,

$$g\left(x\right)=\left\{\begin{array}{c}{\displaystyle \frac{1}{h}{\int }_x^{x+h}f\left(t\right)dt,x\in [-1,1-h),}\hfill \\ {\displaystyle \frac{1}{h}{\int }_{1-h}^{1}f\left(t\right)dt,x\in [1-h,1],}\hfill \end{array}\right.$$

where $0<h<2$, then g is absolutely continuous and f related. The upper bound of ${∥K_{n,A}^{\ast }\left(f\right)-f∥}_p$ can be estimated by using the modulus of smoothness in $L_p$ space. We will derive this type of estimate in future work.

5. Conclusions

In this paper, we propose two types of NN operators, the EKNNO and NEKNNO, which can be regarded as feedforward neural networks with multiple layers. We construct the EKNNO and NEKNNO using the following ideas: (1) integral averaging leads to a Kantorovich-type NN operator for removing time-jitters; (2) a function extension improves the EKNNO’s and NEKNNO’s approximation abilities, especially at the endpoints of a compact interval; (3) the introduction of a flexible parameter A can fine tune the approximation ability of the operators; and (4) normalization allows us to further discuss the approximation performance of the NEKNNO in an integrable function space. All these features combined provide a better approximation performance. We further prove the convergence of these operators as well as attain the quantitative estimates, while in the latter some important approximation tools, such as the modulus of continuity and the idea of K-functional, are utilized. Numerical examples are used to verify the validity of the theoretical results and some potential superiorities of our NN operators.

However, in this paper, we only considered the direct theorems of the NN operators, and the target function is univariate. The converse results and higher dimensional case will be investigated in our future work. Moreover, we use the sigmoid-type function as the activation function in our paper, while many other activation functions, such as ReLU and some other variations of ReLU (such as LeakReLU, PReLu, ELU, SELU, etc.) are widely used in the machine learning and deep learning fields. It is worth exploring if the similar construction of an NN can work well with different activation functions.

In conclusion, we utilize methods and tools in approximation theory to obtain some interesting results in the field of neural networks, which may lead to more applications in neural networks.