Learning Rate of Regularized Regression Associated with Zonal Translation Networks

Abstract

We give a systematic investigation on the reproducing property of the zonal translation network and apply this property to kernel regularized regression. We propose the concept of the Marcinkiewicz–Zygmund setting (MZS) for the scattered nodes collected from the unit sphere. We show that under the MZ condition, the corresponding convolutional zonal translation network is a reproducing kernel Hilbert space. Based on these facts, we propose a kind of kernel regularized regression learning framework and provide the upper bound estimate for the learning rate. We also give proof for the density of the zonal translation network with spherical Fourier-Laplace series.

1. Introduction

It is known that convolutional neural networks provide various models and algorithms to process data models in many fields, such as computer vision (see [1]), natural language processing (see [2]), and sequence analysis in bioinformatics (see [3]). Regularized neural network learning has thus become an attractive research topic (see [4,5,6,7,8,9]). In this paper, we shall give theory analysis for the learning rate of regularized regression associated with the zonal translation network on the unit sphere.

1.1. Kernel Regularized Learning

Let $\Omega$ be a compact subset in the d-dimensional Euclidean space $R^d$ with the usual norm $\parallel x\parallel =\sqrt{{\displaystyle \sum _{k=1}^d}{x}_k^2}$ for $x=(x_1,x_2,\cdots ,x_d)\in R^d$ and Y be a nonempty closed subset contained in $[-M,M]$ for a given $M>0$. The aim of regression learning is to learn the target function, which describes the relationship between the input $x\in \Omega$ and the output $y\in Y$ from a hypothesis function space. In most cases, the target function is offered as a set of observations $z={\left\{z_i\right\}}_{i=1}^m={\left\{(x_i,y_i)\right\}}_{i=1}^m\in Z^m$ drawn independently and identically distributed (i.i.d.) according to a probability joint distribution (measure) $\rho (x,y)={\rho }_{\Omega }\left(x\right)\rho \left(y\right|x)$ on $Z=\Omega \times Y$, where $\rho \left(y\right|x\left)\right(x\in \Omega )$ is the conditional probability of y for a given x and ${\rho }_{\Omega }\left(x\right)$ is the marginal probability about x, i.e., for every integrable functions $\phi :\Omega \times Y\to R$, there hold

$$\begin{array}{c}\hfill {\int }_{\Omega \times Y}\phi (x,y)d\rho ={\int }_{\Omega }\left({\int }_Y\phi (x,y)d\rho (y\mid x)\right)d{\rho }_{\Omega }.\end{array}$$

For a given normed space $(B,\parallel ·{\parallel }_B)$ consisting of real functions on $\Omega$, we define the regularized learning framework with B as

$$\begin{array}{c}\hfill f_{z,\lambda }^{\left(B\right)}:=arg\underset{f\in B}{min}\left({\mathcal{E}}_z\left(f\right)+{\displaystyle \frac{\lambda }{2}}{\parallel f\parallel }_B^2\right),\end{array}$$

(1)

where $\lambda >0$ is the regularization parameter, ${\mathcal{E}}_z\left(f\right)$ is the empirical mean

$${\mathcal{E}}_z\left(f\right)={\displaystyle \frac{1}{m}}\sum _{i=1}^m{(y_i-f\left(x_i\right))}^2.$$

To give theory analysis for the convergence of algorithm (1) quantitatively, we often use the integral framework (see [10,11])

$$\begin{array}{c}\hfill f_{\rho ,\lambda }^{\left(B\right)}:=arg\underset{f\in B}{min}\left({\mathcal{E}}_{\rho }\left(f\right)+{\displaystyle \frac{\lambda }{2}}{\parallel f\parallel }_B^2\right),\end{array}$$

where ${\mathcal{E}}_{\rho }\left(f\right)={\int }_Z{(y-f\left(x\right))}^{2}d\rho$.

The optimal target function is the regression function

$$f_{\rho }\left(x\right)={\int }_{Y}yd\rho \left(y\right|x)$$

satisfying

$$\begin{array}{c}\hfill f_{\rho }=\underset{f}{inf}{\mathcal{E}}_{\rho }\left(f\right),\end{array}$$

where the inf is taken over all the ${\rho }_{\Omega }$-measurable functions f. Moreover, there holds the famous equality (see [12])

$$\begin{array}{c}\hfill \parallel f-f_{\rho }{\parallel }_{L^2\left({\rho }_{\Omega }\right)}^2={\mathcal{E}}_{\rho }\left(f\right)-{\mathcal{E}}_{\rho }\left(f_{\rho }\right),f\in L^2\left({\rho }_{\Omega }\right).\end{array}$$

(2)

The choices for the hypothesis space B in (1) are rich. For example, C.P. An et al. choose the algebraic polynomial class as B (see [13,14,15]). In [16], C. De Mol et al chose the dictionary as B. Recently, some papers chose the Sobolev space as the hypothesis space B (see [17,18]). By the kernel method, we mean traditionally replacing B with a reproducing kernel Hilbert space (RKHS) $(H_K,{\langle ·,·\rangle }_K)$, which is a Hilbert space consisting of real functions defined on $\Omega$, and there is a Mercer kernel $K_x\left(y\right)=K(x,y)$ on $\Omega \times \Omega$ (i.e., $K_x\left(y\right)$ is a continuous and symmetric function on $\Omega \times \Omega$ and for any $n\ge 1$ and any $\{x_1,x_2,\cdots ,x_n\}\subset X$ the Mercer matrices ${\left(K(x_i,x_j)\right)}_{i,j=1,2,\cdots ,n}$ are positive semi-definite) such that

$$\begin{array}{c}\hfill f\left(x\right)={\langle f,K_x\rangle }_K,\forall x\in \Omega ,\forall f\in H_K.\end{array}$$

(3)

and there holds the embedded inequality

$$\begin{array}{c}\hfill \left|f\left(x\right)\right|\le {c\parallel f\parallel }_K,\forall x\in \Omega ,\forall f\in H_K,\end{array}$$

(4)

where c is a constant independent of f and x. There are two results for the optimal solution $f_{z,\lambda }^{\left(B\right)}$. The reproducing property (3) yields the representation

$$\begin{array}{c}\hfill f_{z,\lambda }^{\left(B\right)}\left(x\right)=\sum _{k=1}^m{c}_k{K}_{x_k}\left(x\right),\forall x\in \Omega .\end{array}$$

(5)

The embedded inequality (4) yields the inequality

$$\begin{array}{c}\hfill |f_{z,\lambda }^{\left(B\right)}\left(x\right)|\le \sqrt{{\displaystyle \frac{2M}{\lambda }}}\forall x\in \Omega .\end{array}$$

(6)

Representation (5) is the theory basis for kernel regularized regression (see [19,20]). Inequality (6) is the key inequality for bounding the learning rate with covering number method (see [21,22,23]). For other skills of the kernel method, one can cite [10,11,12,24] et al.

1.2. Marcinkiewicz-Zygmund Setting (MZS)

It is particularly important to mention here that the translation network has recently been used as the hypothesis space of regularized learning (see [25,26]). From the view of approximation theory, a simple single-layer translation network with m neurons is a function class produced by translating a given function $\varphi$, and can be written as

$$\begin{array}{c}\hfill {\Delta }_{\varphi ,\overline{X}}^{\Omega }=\{\sum _{i=1}^n{c}_i{T}_{x_i}(\varphi ,·):c_i\in R,x_i\in \Omega ,i=1,2,\cdots ,n\},\end{array}$$

where $\overline{X}={\left\{x_i\right\}}_{i=1}^n\subset \Omega$ is a given node set, and a given $x\in \Omega T_x(\varphi ,y)$ is a translation operator corresponding to $\Omega$. For example, when $\Omega =R^d$ and $\Omega ={[-\pi ,\pi ]}^d$, we choose $T_x(\varphi ,y)$ as the usual convolution translation operator $\varphi (x-y)$ for a $\varphi$ defined on $R^d$ or an $2\pi$-periodic function $\varphi$ (see [27,28]. When $\Omega =S^{d-1}=\{x\in R^d:\parallel x\parallel =1\}$ is the unit sphere in $R^d$, one can choose $T_x(\varphi ,y)$ as the zonal translation operator $\varphi \left(xy\right)$ for a given $\varphi$ defined on the interval $[-1,1]$ (see [29]). In [30], we defined a kind of translation operator $T_x(\varphi ,y)$ for $\Omega =[-1,1]$. To ensure that the single-layer translation network ${\Delta }_{\varphi ,\overline{X}}^{\Omega }$ can approximate the constant function, ${\Delta }_{\varphi ,\overline{X}}^{\Omega }$ is modified as

$$\begin{array}{c}\hfill N_{\varphi ,\overline{X}}^{\Omega }=\{\sum _{i=1}^n{c}_i{T}_{x_i}(\varphi ,·)+c_0:c_0,c_i\in R,x_i\in \Omega ,i=1,2,\cdots ,n\}.\end{array}$$

(7)

In the case of $T_y(\varphi ,y)=\sigma (w^{\top }x+b)$ and $w,x\in R^d,b\in R$, R.D.Nowak et al. use (7) to design regularized learning frameworks (see [31]). An algorithm is provided by S.B. Lin et al. [26] for designing such a kind of network, and is applied to construct regularized learning algorithms. In [5], $N_{\varphi ,\overline{X}}^{\Omega }$ was used to construct deep neural network learning frameworks. The same type of investigations are given in [32,33,34].

It is easy to see that the approximation ability and the construction of a translation network depend upon the node set $\overline{X}={\left\{x_i\right\}}_{i=1}^n$ (see [35,36,37]). On the other hand, according to the view of [38], the quadrature rule and the Marcinkiewicz–Zygmund (MZ) inequality associated with $\overline{X}$ also influence the construction of the translation network ${\Delta }_{\varphi ,\overline{X}}$. For a bounded closed set $\Omega \subset R^d$, with measure $d\omega$ satisfying ${\int }_{\Omega }d\omega =V<+\infty$. We denote by $P_n\subset L^2(\Omega )$ the linear space of polynomials on $\Omega$ of a degree of, at most, n, equipped with the $L^2$-inner product $\langle v,z\rangle ={\int }_{\Omega }vzd\omega .$ The n-point quadrature rule (QR) is

$$\begin{array}{c}\hfill {\int }_{\Omega }gd\omega \approx \sum _{j=1}^n{w}_{j}g\left(x_j\right),\end{array}$$

(8)

where $\overline{X}={\left\{x_j\right\}}_{j=1}^n\subset \Omega$ and weights $w_j$ are all positive for $j=1,2,\cdots ,n.$ We say the QR (8) has polynomial exactness n if there is a positive integer $\gamma$ such that

$$\begin{array}{c}\hfill {\int }_{\Omega }gd\omega =\sum _{j=1}^n{w}_{j}g\left(x_j\right),\forall g\in P_{\gamma n}.\end{array}$$

(9)

The Marcinkiewicz–Zygmund (MZ) inequality based on the set $\overline{X}={\left\{x_j\right\}}_{j=1}^n\subset \Omega$ is

$$\begin{array}{c}\hfill {\left(\sum _{j=1}^n{A}_j{\left|g\left(x_j\right)\right|}^p\right)}^{{\displaystyle \frac{1}{p}}}\sim {\left({\int }_{\Omega }{\left|g\left(\omega \right)\right|}^{p}d\omega \right)}^{{\displaystyle \frac{1}{p}}},1<p<+\infty ,\forall g\in P_n,\end{array}$$

(10)

where the weights $A_j$ in (10) may be not the same as the $w_j$ in (8) and (9). Another important inequality, which is called the MZ condition in the case of unit sphere in [39], associated with polynomial approximation, is

$$\begin{array}{c}\hfill {\left(\sum _{j=1}^n{A}_j{\left|g\left(x_j\right)\right|}^p\right)}^{{\displaystyle \frac{1}{p}}}\le C{\left({\int }_{\Omega }{\left|g\left(\omega \right)\right|}^{p}d\omega \right)}^{{\displaystyle \frac{1}{p}}},1<p<+\infty ,\forall g\in P_{rn},\end{array}$$

(11)

where C is a constant independent of p, and n and r are any positive integers.

We give examples to show explanations for (9)–(11), which will form the main idea of this paper:

(i) In many cases of the domain $\Omega$, relations (9)–(11) are coexistent. For example, when $\Omega =[-1,1]$, (9)–(11) hold in the case that $x_j=x_{j,n}(j=1,2,\cdots ,n)$ are the zeros of the n-th Jacobi polynomial orthogonal with respect to $d\omega$, and $w_j=A_j(j=1,2,\cdots ,n)$ are the Cotes–Christoffel numbers associated with $d\omega$ (see Theorem A and Theorem B in [40] and Theorem 3.4.1 in [41]). H.N. Mhaskar et al first showed in [42] that (9) and (10) are coexistent on the unit sphere $S^{d-1}$, and the corresponding relation (11) was shown in [43].

(ii) The relations (9)–(11) are equivalent.

Accord to the view of [38], the quadrature rule (QR) follows automatically from the Marcinkiewicz–Zygmund (MZ) inequality in many cases of $\Omega$. H.N. Mhaskar et al. gave a general method of transition from MZ inequality to the polynomial exact QR in [44]; see also Theorem 4.1 in [42]. In particular, in the case of $\Omega =[-1,1]$, (10) may be obtained from (9) and (11) directly (see [45]).

These show that besides the polynomial exactness formula (QR) (9), the MZ inequality (10) is also an important feature for describing the node set $\overline{X}$. For this reason, the node set $\overline{X}={\left\{x_j\right\}}_{j=1}^n\subset \Omega$, which yields an MZ inequality, is given a special terminology, called the Marcinkiewicz–Zygmynd Family (MZF) (see [38,46,47,48]). However, from this literature, we know that the MZF does not totally coincide with the Lagrange interpolation nodes in the case of $d>1$. The hyperinterpolations are then developed with the help of exact QR (see [49,50,51,52,53]) and are applied to approximation theory and regularized learning (see [13,14,15,54]). On the other hand, we find that the problem of the polynomial exact QR is investigated individually (see [55,56]). The concept of spherical t-design was first defined in [57], and has been given investigations by many papers subsequently; one can see the classical references [58,59]. We say $T_t={\left\{x_i\right\}}_{i=1}^{|T_t|}\subset S^{d-1}$ is a spherical t-design if

$$\begin{array}{c}\hfill {\displaystyle \frac{1}{|T_t|}}\sum _{i=1}^{|T_t|}\pi \left(x_i\right)={\displaystyle \frac{1}{{\omega }_{d-1}}}{\int }_{S^{d-1}}\pi \left(x\right)d\omega \left(x\right),\end{array}$$

(12)

where ${\omega }_{d-1}$ is the volume of $S^{d-1}$ and $\pi \left(x\right)$ is a spherical polynomial with the degree t. Moreover, in many applications, the polynomial exact QR and the MZF have been used as assumptions. For example, C.P. An et al. gave an approximation order for the hyperinterpolation approximation under the assumptions that (9), (12), and the MZ inequality (10) hold (see [60,61]). Also, in [25], Lin et al. gave investigations on regularized regression associated with a zonal translation network by assuming that the node set $\overline{X}={\left\{x_j\right\}}_{j=1}^n\subset S^{d-1}$ is a type of spherical t-design.

The polynomial exact QR is also a good tool in approximation theory. For example, H.N. Mhaskar et al used a polynomial exact QR to construct the first periodic translation operators (see [27]) and the zonal translation network operators (see [29]). Along this line, the translation operators defined on the unit ball, on the Euclidean space $R^d$, and on the interval $[-1,1]$ are constructed (see [28,30,62]).

The above investigations encourage us to define a terminology that contains both the Marcinkiewicz–Zygmynd family (MZF) and the polynomial exact QR: we call it the Marcinkiewicz–Zygmund setting (MZS).

Definition 1

(Marcinkiewicz–Zygmund setting (MZS) on Ω). We say a given finite node set ${\Omega }^{\left(n\right)}\subset \Omega$ forms a Marcinkiewicz–Zygmund setting on Ω if (9)–(11) simultaneously hold.

In this paper, we design the translation network $N_{\varphi ,\overline{X}}^{\Omega }$ by taking $\Omega =S^{d-1}$, assuming that $\overline{X}={\Omega }^{\left(n\right)}={\left\{x_j\right\}}_{j=1}^n\subset S^{d-1}$ satisfies MZS and choosing the zonal translation $T_x(\varphi ,y)=\varphi \left(xy\right)$ with $\varphi$ being a given integrable function $\varphi$ on $[-1,1]$. Under these assumptions, we provide a learning framework with $N_{\varphi ,{\Omega }^{\left(n\right)}}^{S^{d-1}}$ being the hypothesis space, and show the learning rate.

The contributions of this paper are twofold. First, after absorbing the ideas of [38,46,47,48] and the successful experience of [13,25,27,29,42,60,61,63], we propose the concept of the Marcinkiewicz–Zygmund setting (MZS) for scattered nodes on the sphere unit; based on this assumption, we show the convergence rate for the approximation error of kernel regularized learning associated with spherical Fourier analysis. Second, we give a new application of translation network and, at the same way, expand the application scopes of kernel regularized learning.

The paper is organized as follows. In Section 2, we first show the density for the zonal translation class and then show the reproducing property for the translation network ${\Delta }_{\varphi ,{\Omega }^{\left(n\right)}}^{S^{d-1}}$. In Section 3, we provide the results in the present paper, for example, a new regression learning framework and a learning setting, the error decomposition for the error analysis, and an estimate for the convergence rate. In Section 4, we give several lemmas, which are used to prove the main results. The proofs for all the theorems and propositions are given in Section 5.

Throughout the paper, we write $A=O\left(B\right)$ if there is a positive constant C independent of A and B such that $A\le CB$. In particular, by $A=O\left(1\right)$ we show that A is a bounded quantity. We write A∼B if both $A=O\left(B\right)$ and $B=O\left(A\right)$.

2. The Properties of the Translation Networks on the Unit Sphere

Let $\varphi \in L_{W_{\eta }}^1={\{\varphi :\parallel \varphi \parallel }_{1,W_{\eta }}={\int }_{-1}^1\left|\varphi \left(x\right)\right|W_{\eta }\left(x\right)dx<+\infty \},W_{\eta }\left(x\right)={(1-x^2)}^{\eta -{\displaystyle \frac{1}{2}}},\eta >-{\displaystyle \frac{1}{2}}$. Then, H.N. Mhaskar et al constructed in [29] a sequence of approximation operators to show that the zonal translation class

$$\begin{array}{ccc}\hfill {\Delta }_{\varphi }^{S^{d-1}}& =& cl\{\varphi \left(xy\right):y\in S^{d-1}\}\cup \left\{1\right\}\hfill \\ & =& cl\{\sum _{i=1}^n{c}_i\varphi (x_i·)+c_0:c_0,c_i\in R,x_i\in S^{d-1},i=1,2,\cdots ,n;n=1,2,\cdots \}\hfill \end{array}$$

is dense in $L^p\left(S^{d-1}\right)(1\le p\le +\infty )$ if $\widehat{a_l^{\eta }\left(\varphi \right)}>0$ for all $l=0,1,2,\cdots ,$ where

$$\widehat{a_l^{\eta }\left(\varphi \right)}=c_{\eta }{\int }_{-1}^1\varphi \left(x\right){\displaystyle \frac{C_l^{\eta }\left(x\right)}{C_l^{\eta }\left(1\right)}}{W}_{\eta }\left(x\right)dx,\eta ={\displaystyle \frac{d-2}{2}}$$

and $C_n^{\eta }\left(x\right)=p_n^{(\eta -{\displaystyle \frac{1}{2}},\eta -{\displaystyle \frac{1}{2}})}\left(x\right)$ is the n-th Legendre polynomial satisfies the orthogonal relation

$$\begin{array}{c}\hfill c_{\eta }{\int }_{-1}^1{C}_n^{\eta }\left(x\right)C_m^{\eta }\left(x\right)W_{\eta }\left(x\right)dx=h_n^{\eta }{\delta }_{n,m}.\end{array}$$

with $c_{\eta }={\displaystyle \frac{\Gamma (\eta +1)}{\Gamma \left(\frac{1}{2}\right)\Gamma (\eta +\frac{1}{2})}}$, $h_n^{\eta }={\displaystyle \frac{\eta }{n+\eta }}{C}_n^{\eta }\left(1\right)$, and it is known that (see from (B.2.1), (B.2.2) and (B.5.1) of [64]) $C_n^{\eta }\left(x\right)\le C_n^{\eta }\left(1\right)=n^{2\eta -1},x\in [-1,1].$ It follows that

$$\begin{array}{c}\hfill \varphi \left(t\right)=\sum _{l=0}^{+\infty }\widehat{a_l^{\eta }\left(\varphi \right)}{\displaystyle \frac{l+\eta }{\eta }}{C}_l^{\eta }\left(t\right)=\sum _{l=0}^{+\infty }\widehat{a_l^{\eta }\left(\varphi \right)}{Z}_l^{\eta }\left(t\right),\end{array}$$

(13)

where $Z_l^{\eta }\left(t\right)={\displaystyle \frac{l+\eta }{\eta }}{C}_l^{\eta }\left(t\right),\eta ={\displaystyle \frac{d-2}{2}}$.

Let $P_n^d$ denote the space of all homogeneous polynomials of degree n in d variables. We denote by $L^p\left(S^{d-1}\right)$ the class of all measurable functions defined on $S^{d-1}$ with the finite norm

$$\begin{array}{ccc}& & {\parallel f\parallel }_{p,S^{d-1}}=\left\{\begin{array}{cc}{\left({\int }_{S^{d-1}}{\left|f\left(x\right)\right|}^{p}d\sigma \left(x\right)\right)}^{{\displaystyle \frac{1}{p}}},\hfill & 1\le p<+\infty ,\hfill \\ \\ \underset{x\in S^{d-1}}{max}\left|f\left(x\right)\right|,\hfill & p=+\infty ,\hfill \end{array}\right.\hfill \end{array}$$

and for $p=+\infty$, we assume that $L^{+\infty }\left(S^{d-1}\right)$ is the space $C\left(S^{d-1}\right)$ of continuous functions on $S^{d-1}$ with the uniform norm.

For a given integer $n\ge 0$, the restriction to $S^{d-1}$ of a homogeneous harmonic polynomial of degree n is called the spherical harmonics of degree n. If $Y\in P_n^d$, then $Y\left(x\right)={\parallel x\parallel }^{n}Y\left(x^{\prime }\right),x^{\prime }={\displaystyle \frac{x}{\parallel x\parallel }}$, so that Y is determined by its restriction on the unit sphere. Let $H_n\left(S^{d-1}\right)$ denote the space of the spherical harmonics of degree n. Then,

$$dim{H}_n\left(S^{d-1}\right)=(\begin{array}{c}n+d-2\\ n\end{array})+(\begin{array}{c}n+d-3\\ n-1\end{array}),n=1,2,3,\cdots ,$$

Spherical harmonics of different degrees are orthogonal on the unit sphere. For further properties about harmonics, one can refer to [65].

For $n=0,1,2,\cdots ,$ let $\{Y_l^n:1\le l\le dim{H}_n\left(S^{d-1}\right)\}$ be an orthonormal basis of $H_n\left(S^{d-1}\right)$. Then,

$$\begin{array}{c}\hfill {\displaystyle \frac{1}{{\omega }_{d-1}}}{\int }_{S^{d-1}}{Y}_l^n\left(\xi \right)Y_{l^{\prime }}^m\left(\xi \right)d\sigma \left(\xi \right)={\delta }_{l,l^{\prime }}{\delta }_{m,n},\end{array}$$

where ${\omega }_{d-1}$ denotes the surface area of $S^{d-1}$ and ${\omega }_{d-1}={\displaystyle \frac{2{\pi }^{\frac{d}{2}}}{\Gamma \left(\frac{d}{2}\right)}}$. Furthermore, by (1.2.8) in [64], we have,

$$\begin{array}{c}\hfill \sum _{j=1}^{dim{H}_n\left(S^{d-1}\right)}{Y}_l^n\left(x\right)Y_l^n\left(y\right)={\displaystyle \frac{n+\eta }{\eta }}{C}_n^{\eta }\left(xy\right),x,y\in S^{d-1},\end{array}$$

(14)

where $C_n^{\eta }\left(t\right)$ is the n-th generalized Legendre polynomial, the same as in (13). Combining (13) and (14), we have

$$\begin{array}{c}\hfill \varphi \left(xy\right)=\sum _{l=0}^{+\infty }\widehat{a_l^{\eta }\left(\varphi \right)}{\displaystyle \frac{l+\eta }{\eta }}{C}_l^{\eta }(x·y)=\sum _{l=0}^{+\infty }\widehat{a_l^{\eta }\left(\varphi \right)}\sum _{k=1}^{dim{H}_l\left(S^{d-1}\right)}{Y}_k^l\left(x\right)Y_k^l\left(y\right),x,y\in S^{d-1}.\end{array}$$

(15)

Also, there holds the Funk–Hecke formula (see (1.2.11) in [64] or (1.2.6) in [66]):

$$\begin{array}{c}\hfill {\int }_{S^{d-1}}f(x·y)Y_n\left(y\right)d\sigma \left(y\right)=\widehat{a_l^{\eta }\left(f\right)}{Y}_n\left(x\right),Y_n\in H_n\left(S^{d-1}\right).\end{array}$$

(16)

In particular, there holds

$$\begin{array}{c}\hfill {\int }_{S^{d-1}}\varphi (x·y)d\sigma \left(y\right)={\omega }_{d-1}{\parallel \varphi \parallel }_{1,W_{\eta }},\forall x\in S^{d-1},\varphi \in L_{W_{\eta }}^1.\end{array}$$

(17)

For a $f\in L^1\left(S^{d-1}\right)$ we define $a_l^{\left(n\right)}\left(f\right)={\int }_{S^{d-1}}f\left(x\right)Y_l^n\left(x\right)d\sigma \left(x\right)$. Then,

$$\begin{array}{ccc}\hfill f\left(x\right)& \sim & \sum _{l=0}^{\infty }\sum _{j=1}^{dim{H}_l\left(S^{d-1}\right)}{a}_j^{\left(l\right)}\left(f\right)Y_j^l\left(x\right)\hfill \\ & =& \sum _{l=0}^{\infty }{Y}_l(f,x),\hfill \end{array}$$

where $Y_l(f,x)={\displaystyle \sum _{j=1}^{dim{H}_l\left(S^{d-1}\right)}}{a}_j^{\left(l\right)}\left(f\right)Y_j^l\left(x\right).$ It is known that (see (6.1.4) in [66])

$$\begin{array}{ccc}\hfill {\parallel f\parallel }_{2,S^{d-1}}& =& {\left(\sum _{l=0}^{\infty }\sum _{j=1}^{dim{H}_l\left(S^{d-1}\right)}{\left|a_j^{\left(l\right)}\left(f\right)\right|}^2\right)}^{{\displaystyle \frac{1}{2}}}\hfill \\ & =& {\left(\sum _{l=0}^{\infty }{\parallel Y_l\left(f\right)\parallel }_{2,S^{d-1}}^2\right)}^{{\displaystyle \frac{1}{2}}},f\in L^2\left(S^{d-1}\right).\hfill \end{array}$$

(18)

2.1. Density

We first give a general discrimination method for density.

Proposition 1

(see Lemma 1 in Chapter 18 of [67]). For a subset V in a normed linear space E, the following two properties are equivalent:

(a) V is fundamental in E (that is, its linear span is dense in E).

(b) $V^{\perp }=\left\{0\right\}$ (that is, 0 is the only element of $E^{*}$ that annihilates V).

Based on this proposition, we can show the density of ${\Delta }_{\varphi }^{S^{d-1}}$ in a qualitative way.

Theorem 1.

Let $\varphi \in L_{W_{\eta }}^2$ satisfy $\widehat{a_l^{\eta }\left(\varphi \right)}>0$ for all $l=0,1,2,\cdots .$ Then, ${\Delta }_{\varphi }^{S^{d-1}}$ is dense in $L^2\left(S^{d-1}\right)$.

Proof. 

See the proof in Section 5. □

We can quantitatively show the density of ${\Delta }_{\varphi }^{S^{d-1}}$ in $L^{+\infty }\left(S^{d-1}\right)$.

Let $C\left([-1,1]\right)=L_{W_{\eta }}^{\infty }$ denote the set of all continuous functions defined on $[-1,1]$ and

$${\parallel \varphi \parallel }_{\infty ,W_{\eta }}={\parallel \varphi \parallel }_{C\left(\right[-1,1\left]\right)}=\underset{x\in [-1,1]}{sup}\left|\varphi \left(x\right)\right|.$$

Define a differential operator $P_{\eta }\left(D\right)$ as

$$P_{\eta }\left(D\right)=W_{\eta }{\left(x\right)}^{-1}{\displaystyle \frac{d}{dx}}{W}_{\eta }\left(x\right)(1-x^2){\displaystyle \frac{d}{dx}}$$

and $P_{\eta }{\left(D\right)}^l=P_{\eta }\left(D\right)\left(P_{\eta }{\left(D\right)}^{l-1}\right),l=1,2,\cdots .$

Theorem 2.

Let $\varphi \in L_{W_{\eta }}^2$ be sufficient smoothness (for example, $P_{\eta }{\left(D\right)}^l\varphi \in C\left([-1,1]\right)$ for all $l\ge 1$) and satisfy $\widehat{a_l^{\eta }\left(\varphi \right)}>0$ for all $l=0,1,2,\cdots .$ Then, for a given $f\in L^{+\infty }\left(S^{d-1}\right)$ and $\forall \epsilon >0$, there is a $g\in {\Delta }_{\varphi }^{S^{d-1}}$ such that

$$\begin{array}{c}\hfill {\parallel f-g\parallel }_{\infty ,S^{d-1}}<\epsilon .\end{array}$$

(19)

Proof. 

See the proof in Section 5. □

2.2. MZS on the Unit Sphere

We first restate a proposition.

Proposition 2.

There is a finite subset ${\Omega }^{\left(n\right)}\subset S^{d-1}$ and positive constants $N_d,c$ such that for any given $n\ge N_d$, we have two nonnegative number sets $\{{\mu }_k:x_k\in {\Omega }^{\left(n\right)}\}$ and $\{A_k:x_k\in {\Omega }^{\left(n\right)}\}$ satisfying

$$\begin{array}{c}\hfill \sum _{x_k\in {\Omega }^{\left(n\right)}}{\displaystyle \frac{{\mu }_k^{\left(n\right)}}{A_k^{\left(n\right)}}}\le c,\end{array}$$

(20)

such that

$$\begin{array}{c}\hfill {\int }_{S^{d-1}}f\left(x\right)d\sigma \left(x\right)=\sum _{x_k\in {\Omega }^{\left(n\right)}}{\mu }_k^{\left(n\right)}f\left(x_k\right),f\in H_n\left(S^{d-1}\right).\end{array}$$

(21)

and

$$\begin{array}{c}\hfill {\int }_{S^{d-1}}{\left|f\left(x\right)\right|}^{p}d\sigma \left(x\right)\sim \sum _{x_k\in {\Omega }^{\left(n\right)}}{A}_k^{\left(n\right)}{\left|f\left(x_k\right)\right|}^p,f\in H_n\left(S^{d-1}\right),1\le p<+\infty .\end{array}$$

(22)

Moreover, for any $m\ge n$ and $p\ge 1$, there exists a constant $c_{p,d}>0$ such that

$$\begin{array}{c}\hfill \sum _{x_k\in {\Omega }^{\left(n\right)}}{A}_k^{\left(n\right)}|f\left(x_k\right){|}^p\le c_{p,d}{\left({\displaystyle \frac{m}{n}}\right)}^{d-1}{\int }_{S^{d-1}}{\left|f\left(x\right)\right|}^{p}d\sigma \left(x\right),f\in H_m\left(S^{d-1}\right)\end{array}$$

(23)

and

$$\begin{array}{c}\hfill |\{x_k\in {\Omega }^{\left(n\right)}:{\mu }_k^{\left(n\right)}\ne 0\}|\sim dim\left(H_n\left(S^{d-1}\right)\right).\end{array}$$

(24)

Proof. 

Inequalities (21)–(22) were proved by H.N. Mhaskar et al in [42] or see [29], which have now been extended to other domains (see [68]). Inequality (24) may be found from [29]. Inequality (23) is followed by (22) and the following facts (see Theorem 2.1 in [43]):

Suppose that Ω is a finite subset of $S^{d-1}$, $\{{\mu }_{\omega }:\omega \in \Omega \}$ is a set of positive numbers, and n is a positive integer. If for a $0<p_o<+\infty$ holds the inequality

$$\begin{array}{c}\hfill \sum _{\omega \in \Omega }{\mu }_{\omega }{\left|f\left(\omega \right)\right|}^{p_0}\le C_1{\int }_{S^{d-1}}{\left|f\left(x\right)\right|}^{p_0}d\sigma \left(x\right),f\in H_n\left(S^{d-1}\right)\end{array}$$

with $C_1$ independent of f, then for any $0<p<+\infty$ and any $f\in H_m\left(S^{d-1}\right)$ with $m\ge n$

$$\begin{array}{c}\hfill \sum _{\omega \in \Omega }{\mu }_{\omega }{\left|f\left(\omega \right)\right|}^p\le C_{d,p}{\left({\displaystyle \frac{m}{n}}\right)}^{d-1}{\int }_{S^{d-1}}{\left|f\left(x\right)\right|}^{p}d\sigma \left(x\right),f\in H_m\left(S^{d-1}\right),\end{array}$$

where $C_{d,p}>0$ depends only on d and p.

Inequalities (21)–(22) show the existence of ${\Omega }^{\left(n\right)}\subset S^{d-1}$, which has the polynomial exact QR (9) and satisfies the MZ inequality (10). Inequality (23) often goes with (21) and (22), and is needed in discussing the approximation order (see [63]). □

Proposition 3.

For any given $n\ge 1$, there exists a finite subset ${\Omega }^{\left(n\right)}\subset S^{d-1}$, which forms an MZS on $S^{d-1}$.

Proof. 

The results follow from (21)–(23). □

2.3. The Reproducing Property

Let $\varphi \in C\left(\right[-1,1\left]\right)$ be a given even function. For a given finite set ${\Omega }^{\left(n\right)}\subset S^{d-1}$, i.e., $|{\Omega }^{\left(n\right)}|<+\infty$, and the corresponding finite number set $\{{\mu }_k^{\left(n\right)}:x_k\in {\Omega }^{\left(n\right)}\}$, we define a zonal translation network as

$$\begin{array}{c}\hfill H_{{\Omega }^{\left(n\right)}}^{\varphi }:=cl\{f\left(x\right)=\sum _{x_k\in {\Omega }^{\left(n\right)}}{c}_k{\mu }_k^{\left(n\right)}{T}_x\left(\varphi \right)\left(x_k\right)+c_0:c_k\in R,k=0,1,2,\cdots ,|{\Omega }^{\left(n\right)}\left|\right\},\end{array}$$

where $T_x\left(\varphi \right)\left(y\right)=\varphi \left(xy\right)$. Then, it is easy to see that

$$\begin{array}{c}\hfill H_{{\Omega }^{\left(n\right)}}^{\varphi }=H_{\varphi }^{\left(n\right)}⨁R,\end{array}$$

where for $A,B\subset R$, and $A\bigcap B=\left\{0\right\}$ we define $A⨁B=\{a+b:a\in A,b\in B\}$ and

$$\begin{array}{c}\hfill H_{\varphi }^{\left(n\right)}:=cl\{f\left(x\right)=\sum _{x_k\in {\Omega }^{\left(n\right)}}{c}_k{\mu }_k^{\left(n\right)}{T}_x\left(\varphi \right)\left(x_k\right):c_k\in R,k=0,1,2,\cdots ,|{\Omega }^{\left(n\right)}\left|\right\}.\end{array}$$

For $f\left(x\right)={\displaystyle \sum _{x_k\in {\Omega }^{\left(n\right)}}}{c}_k{\mu }_k^{\left(n\right)}{T}_x\left(\varphi \right)\left(x_k\right)\in H_{\varphi }^{\left(n\right)},g\left(x\right)={\displaystyle \sum _{x_k\in {\Omega }^{\left(n\right)}}}{d}_k{\mu }_k^{\left(n\right)}{T}_x\left(\varphi \right)\left(x_k\right)\in H_{\varphi }^{\left(n\right)}$, we define a bivariate operation as

$$\begin{array}{c}\hfill {\langle f,g\rangle }_{\varphi }=\sum _{x_k\in {\Omega }^{\left(n\right)}}{c}_k{d}_k{\mu }_k^{\left(n\right)}\end{array}$$

and

$$\begin{array}{c}\hfill {\parallel f\parallel }_{\varphi }={\left(\sum _{x_k\in {\Omega }^{\left(n\right)}}{\mu }_k^{\left(n\right)}{\left|c_k\right|}^2\right)}^{{\displaystyle \frac{1}{2}}}.\end{array}$$

Because of (15), we see by Theorem 4 in Chapter 17 of [67] that the matrix ${\left(T_{x_i}(\varphi ,x_j)\right)}_{i,j=1,2,\cdots ,|{\Omega }^{\left(n\right)}|}$ is positive and definite for a given n.

It follows that the vector $c={\left\{{\mu }_k^{\left(n\right)}{c}_k\right\}}_{x_k\in {\Omega }^{\left(n\right)}}$ is unique. Then, for a given n$(H_{\varphi }^{\left(n\right)},\parallel ·{\parallel }_{\varphi })$ is a finite-dimensional Hilbert space, whose dimensional $dim{H}_{\varphi }^{\left(n\right)}=\left|{\Omega }^{\left(n\right)}\right|\sim n^{d-1}$, and is isometrically isomorphic with $l^2\left({\Omega }^{\left(n\right)}\right)$, where

$$\begin{array}{c}\hfill l^2\left({\Omega }^{\left(n\right)}\right)=\{c={\left\{{\mu }_k^{\left(n\right)}{c}_k\right\}}_{x_k\in {\Omega }^{\left(n\right)}}{:\parallel c\parallel }_{l^2\left({\Omega }^{\left(n\right)}\right)}={\left(\sum _{x_k\in {\Omega }^{\left(n\right)}}{\mu }_k^{\left(n\right)}{\left|c_k\right|}^2\right)}^{{\displaystyle \frac{1}{2}}}<+\infty \}.\end{array}$$

Since $H_{\varphi }^{\left(n\right)}$ is a finite-dimensional Hilbert space, we know by Theorem A in section 3 of Part I in [69] that $H_{\varphi }^{\left(n\right)}$ must be a reproducing kernel Hilbert space; what we need to do is to find the reproducing kernel.

We have a proposition.

Proposition 4.

Let $\varphi \in C[-1,1]$ satisfy $P_{\eta }{\left(D\right)}^l\varphi \in C\left([-1,1]\right)$ and

$$\begin{array}{c}\hfill {\left(\sum _{x_k\in {\Omega }^{\left(n\right)}}\left|{\mu }_k^{\left(n\right)}\right|\right)}^{{\displaystyle \frac{1}{2}}}=O\left(n^{\alpha }\right),2l>\alpha \ge 0.\end{array}$$

(25)

If ${\Omega }^{\left(n\right)}\subset S^{d-1}$ satisfies the MZ condition (23) by $A_k^{\left(n\right)}=\left|{\mu }_k^{\left(n\right)}\right|$, then $(H_{\varphi }^{\left(n\right)},\parallel ·{\parallel }_{\varphi })$ is a finite-dimensional reproducing kernel Hilbert space associated with the kernel

$$\begin{array}{c}\hfill K_x^{*}\left(\varphi \right)\left(y\right)=K^{*}(\varphi ,x,y)=\sum _{x_k\in {\Omega }^{\left(n\right)}}{\mu }_k^{\left(n\right)}{T}_{x_k}(\varphi ,x)T_{x_k}(\varphi ,y),\end{array}$$

(26)

i.e.,

$$\begin{array}{c}\hfill f\left(x\right)={〈f,K_x^{*}\left(\varphi \right)(·)〉}_{\varphi }\forall x\in S^{d-1},\forall f\in H_{\varphi }^{\left(n\right)}\end{array}$$

(27)

and there is a constant $k^{*}>0$ such that

$$\begin{array}{c}\hfill \left|f\left(x\right)\right|\le k^{*}{\parallel f\parallel }_{\varphi }\forall f\in H_{\varphi }^{\left(n\right)},\forall x\in S^{d-1}.\end{array}$$

(28)

Proof. 

See the proof in Section 5. □

Corollary 1.

Under the assumptions of Proposition 4, $H_{{\Omega }^{\left(n\right)}}^{\varphi }$ is a finite-dimensional reproducing kernel Hilbert space associated with the inner product defined by

$$\begin{array}{c}{\langle f,g\rangle }_{H_{{\Omega }^{\left(n\right)}}^{\varphi }}={\langle f_1,g_1\rangle }_{\varphi }+c_0{d}_0,\hfill \end{array}$$

where

$$f\left(x\right)=f_1\left(x\right)+c_0,g\left(x\right)=g_1\left(x\right)+d_0,$$

$$f_1\left(x\right)=\sum _{x_k\in {\Omega }^{\left(n\right)}}{c}_k{\mu }_k^{\left(n\right)}{T}_x\left(\varphi \right)\left(x_k\right),g_1\left(x\right)=\sum _{x_k\in {\Omega }^{\left(n\right)}}{d}_k{\mu }_k^{\left(n\right)}{T}_x\left(\varphi \right)\left(x_k\right).$$

and the corresponding reproducing kernel $K_x\left(\varphi \right)\left(y\right)$ is

$$\begin{array}{c}\hfill K_x\left(\varphi \right)\left(y\right)=K_x^{*}\left(\varphi \right)\left(y\right)+1,x,y\in S^{d-1}.\end{array}$$

(29)

Furthermore, there is a constant $\kappa >0$ such that

$$\begin{array}{c}\hfill \left|f\left(x\right)\right|\le {\kappa \parallel f\parallel }_{H_{{\Omega }^{\left(n\right)}}^{\varphi }},\forall f{\in }_{H_{{\Omega }^{\left(n\right)}}^{\varphi }},\forall x\in S^{d-1}.\end{array}$$

(30)

Proof. 

The results can be obtained from Proposition 4, and the fact that the real set R is a reproducing kernel Hilbert space whose reproducing kernel is 1, and the inner product is the usual product of two real numbers. □

Corollary 2.

Let ${\Omega }^{\left(n\right)}\subset S^{d-1}$ be defined as in Proposition 2 and $\varphi \in L_{W_{\eta }}^{\infty }$. Then, $H_{{\Omega }^{\left(n\right)}}^{\varphi }$ is a finite-dimensional reproducing kernel Hilbert space associated with kernel (29), and there holds inequality (30).

Proof. 

See the proof in Section 5. □

3. Apply to Kernel Regularized Regression

We now apply the above reproducing kernel Hilbert spaces to kernel regularized regression.

3.1. Learning Framework

For a set of observations $z={\left\{(x_i,y_i)\right\}}_{i=1}^m$ drawn i.i.d. according to a joint distribution $\rho (x,y)$ on $Z=S^{d-1}\times Y,Y=[-M,M],M>0$ is a given real number, satisfying $\rho (x,y)=\rho \left(y\right|x){\rho }_{S^{d-1}}\left(x\right)$; we define a regularized framework as

$$\begin{array}{c}\hfill f_{z,\lambda }^{{\Omega }^{\left(n\right)}}:=arg\underset{f\in H_{{\Omega }^{\left(n\right)}}^{\varphi }}{min}\left({\mathcal{E}}_z\left(f\right)+{\displaystyle \frac{\lambda }{2}}{\parallel f\parallel }_{H_{{\Omega }^{\left(n\right)}}^{\varphi }}^2\right),\end{array}$$

(31)

where $\lambda =\lambda \left(m\right)\to 0(m\to +\infty )$ are the regularization parameters, and

$${\mathcal{E}}_z\left(f\right)={\displaystyle \frac{1}{m}}\sum _{i=1}^m{(y_i-f\left(x_i\right))}^2.$$

It can be seen that the n in (31) may be different from the sample number m. But it can be chosen according to our needs for the purpose of increasing the learning rate.

The general integral model of (31) is

$$\begin{array}{c}\hfill f_{\rho ,\lambda }^{{\Omega }^{\left(n\right)}}:=arg\underset{f\in H_{{\Omega }^{\left(n\right)}}^{\varphi }}{min}\left({\mathcal{E}}_{\rho }\left(f\right)+{\displaystyle \frac{\lambda }{2}}{\parallel f\parallel }_{H_{{\Omega }^{\left(n\right)}}^{\varphi }}^2\right),\end{array}$$

(32)

where

$${\mathcal{E}}_{\rho }\left(f\right)={\int }_Z{(y-f\left(x\right))}^{2}d\rho .$$

To show the convergence analysis for (31), we need to bound the error

$$\begin{array}{c}\hfill {\parallel f_{z,\lambda }^{{\Omega }^{\left(n\right)}}-f_{\rho }\parallel }_{L^2\left({\rho }_{S^{d-1}}\right)},\end{array}$$

which is an approximation problem whose convergence rate depends upon the approximation ability of ${\mathcal{H}}_{{\Omega }^{\left(n\right)}}$. An error decomposition will be given in Section 3.2.

3.2. Error Decompositions

By (2) and the definition of $f_{\rho ,\lambda }^{{\Omega }^{\left(n\right)}}$, we have

$$\begin{array}{cc}& {\parallel f_{z,\lambda }^{{\Omega }^{\left(n\right)}}-f_{\rho }\parallel }_{L^2\left({\rho }_{S^{d-1}}\right)}\hfill \\ \le & {\parallel f_{z,\lambda }^{{\Omega }^{\left(n\right)}}-f_{\rho ,\lambda }^{{\Omega }^{\left(n\right)}}\parallel }_{L^2\left({\rho }_{S^{d-1}}\right)}+{\parallel f_{\rho ,\lambda }^{{\Omega }^{\left(n\right)}}-f_{\rho }\parallel }_{L^2\left({\rho }_{S^{d-1}}\right)}\hfill \\ =& {\parallel f_{z,\lambda }^{{\Omega }^{\left(n\right)}}-f_{\rho ,\lambda }^{{\Omega }^{\left(n\right)}}\parallel }_{L^2\left({\rho }_{S^{d-1}}\right)}+\sqrt{{\mathcal{E}}_{\rho }\left(f_{\rho ,\lambda }^{{\Omega }^{\left(n\right)}}\right)-{\mathcal{E}}_{\rho }\left(f_{\rho }\right)+{\displaystyle \frac{\lambda }{2}}{\parallel f_{\rho ,\lambda }^{{\Omega }^{\left(n\right)}}\parallel }_{H_{{\Omega }^{\left(n\right)}}^{\varphi }}^2}\hfill \\ \le & {\parallel f_{z,\lambda }^{{\Omega }^{\left(n\right)}}-f_{\rho ,\lambda }^{{\Omega }^{\left(n\right)}}\parallel }_{L^2\left({\rho }_{S^{d-1}}\right)}+D^{{\Omega }^{\left(n\right)}}{(f_{\rho },\lambda )}_{L^2\left({\rho }_{S^{d-1}}\right)},\hfill \end{array}$$

(33)

where we have used the fact that for $a>0,b>0,c>0$ and $0<p\le 1$, there holds

$$\begin{array}{c}\hfill {(a+b+c)}^p\le a^p+b^p+c^p\end{array}$$

and

$$\begin{array}{c}\hfill D^{{\Omega }^{\left(n\right)}}(f_{\rho },\lambda )=\underset{g\in H_{{\Omega }^{\left(n\right)}}^{\varphi }}{inf}\left(\parallel g-f_{\rho }{\parallel }_{L^2\left({\rho }_{S^{d-1}}\right)}+\sqrt{{\displaystyle \frac{\lambda }{2}}}{\parallel g\parallel }_{H_{{\Omega }^{\left(n\right)}}^{\varphi }}\right)\end{array}$$

is a K-functional that denotes the approximation error, whose decay will be described later. So, the main estimate that we need to deal with is the sample error

$$\begin{array}{c}\hfill {\parallel f_{z,\lambda }^{{\Omega }^{\left(n\right)}}-f_{\rho ,\lambda }^{{\Omega }^{\left(n\right)}}\parallel }_{L^2\left({\rho }_{S^{d-1}}\right)}.\end{array}$$

3.3. Convergence Rate for the K-Functional

We first provide a convergence rate for the K-functional $D^{{\Omega }^{\left(n\right)}}(f,\lambda )$.

Proposition 5.

Let $\varphi \in L_{W_{\eta }}^{\infty }$ satisfy $\widehat{a_l^{\eta }\left(\varphi \right)}>0$ for all $l\ge 0$, and let $N\gg n$ be a positive integer. Then, there is a ${\Omega }^{\left(2\right(N+n\left)\right)}\subset S^{d-1}$, which forms an MZS on $S^{d-1}$ such that

$$\begin{array}{c}\hfill D^{{\Omega }^{\left(2\right(N+n\left)\right)}}{(f,\lambda )}_{L^2\left({\rho }_{S^{d-1}}\right)}=O\left({\displaystyle \frac{n^{\frac{1}{2}}{E}_N{\left(\varphi \right)}_{2,W_{\eta }}{\parallel f\parallel }_{2,S^{d-1}}}{{\varphi }_n}}+E_n{\left(f\right)}_{2,S^{d-1}}+{\displaystyle \frac{n^{\frac{1}{2}}\sqrt{\lambda }{\parallel f\parallel }_{2,S^{d-1}}}{{\varphi }_n}}\right),\end{array}$$

(34)

where ${\varphi }_n=\underset{0\le l\le 2n}{min}\widehat{a_l^{\eta }\left(\varphi \right)}.$

Proof. 

See the proof in Section 5. □

Corollary 3.

Let $\varphi \in L_{W_{\eta }}^{\infty }$ satisfy $\widehat{a_l^{\eta }\left(\varphi \right)}>0$ for all $l\ge 0$, and for a given l, there holds $P_{\eta }{\left(D\right)}^l\in C\left([-1,1]\right)$. If $m=m\left(n\right)\gg N\gg n$ are chosen such that ${\displaystyle \frac{n^{\frac{1}{2}}}{N^{2l}{\varphi }_n}}\to 0(n\to +\infty )$ and ${\displaystyle \frac{n^{\frac{1}{2}}\sqrt{\lambda }}{{\varphi }_n}}\to 0(n\to +\infty )$, then

$$\begin{array}{c}\hfill D^{{\Omega }^{\left(2\right(N+n\left)\right)}}{(f,\lambda )}_{L^2\left({\rho }_{S^{d-1}}\right)}\to 0^{+},n\to +\infty .\end{array}$$

(35)

3.4. The Learning Rate

Theorem 3.

Let ${\Omega }^{\left(n\right)}\subset S^{d-1}$ satisfy (25) and $\gamma ={\left({\int }_Z{y}^{2}d\rho (x,y)\right)}^{{\displaystyle \frac{1}{2}}}<+\infty .$

If $\kappa D^{{\Omega }^{\left(n\right)}}{(f_{\rho },\lambda )}_{L^2\left({\rho }_{S^{d-1}}\right)}>M\sqrt{\lambda }$, then we have a constant $C>0$ such that for any $\delta \in (0,1)$, with confidence $1-\delta$, holds

$$\begin{array}{c}\hfill {\parallel f_{z,\lambda }^{{\Omega }^{\left(n\right)}}-f_{\rho ,\lambda }^{{\Omega }^{\left(n\right)}}\parallel }_{H_{{\Omega }^{\left(n\right)}}^{\varphi }}\le \left({\displaystyle \frac{\kappa \gamma }{\lambda \sqrt{m}}}+{\displaystyle \frac{2{\kappa }^2{D}^{{\Omega }^{\left(n\right)}}{(f_{\rho },\lambda )}_{L^2\left({\rho }_{S^{d-1}}\right)}}{{\lambda }^{\frac{3}{2}}m}}\right)log{\displaystyle \frac{2}{\delta }}.\end{array}$$

(36)

If $\kappa D^{{\Omega }^{\left(n\right)}}{(f_{\rho },\lambda )}_{L^2\left({\rho }_{S^{d-1}}\right)}\le M\sqrt{\lambda }$, then we have a constant $C>0$ such that for any $\delta \in (0,1)$, with confidence $1-\delta$, holds

$$\begin{array}{c}\hfill {\parallel f_{z,\lambda }^{{\Omega }^{\left(n\right)}}-f_{\rho ,\lambda }^{{\Omega }^{\left(n\right)}}\parallel }_{H_{{\Omega }^{\left(n\right)}}^{\varphi }}\le {\displaystyle \frac{2(M+\gamma )}{\lambda \sqrt{m}}}log{\displaystyle \frac{2}{\delta }}.\end{array}$$

(37)

Proof. 

See the proof in Section 5. □

Corollary 4.

Let ${\Omega }^{\left(n\right)}\subset S^{d-1}$ satisfy (25) and $\gamma ={\left({\int }_Z{y}^{2}d\rho (x,y)\right)}^{{\displaystyle \frac{1}{2}}}<+\infty .$

If $\kappa D^{{\Omega }^{\left(n\right)}}{(f_{\rho },\lambda )}_{L^2\left({\rho }_{S^{d-1}}\right)}>M\sqrt{\lambda }$, then we have a constant $C>0$ such that for any $\delta \in (0,1)$, with confidence $1-\delta$, holds

$$\begin{array}{cc}& {\parallel f_{z,\lambda }^{{\Omega }^{\left(n\right)}}-f_{\rho ,\lambda }^{{\Omega }^{\left(n\right)}}\parallel }_{H_{{\Omega }^{\left(n\right)}}^{\varphi }}\hfill \\ \le & \left({\displaystyle \frac{\kappa \gamma }{\lambda \sqrt{m}}}+{\displaystyle \frac{2{\kappa }^2{D}^{{\Omega }^{\left(n\right)}}{(f_{\rho },\lambda )}_{L^2\left({\rho }_{S^{d-1}}\right)}}{{\lambda }^{\frac{3}{2}}m}}\right)log{\displaystyle \frac{2}{\delta }}+D^{{\Omega }^{\left(n\right)}}{(f_{\rho },\lambda )}_{L^2\left({\rho }_{S^{d-1}}\right)}.\hfill \end{array}$$

(38)

If $\kappa D^{{\Omega }^{\left(n\right)}}{(f_{\rho },\lambda )}_{L^2\left({\rho }_{S^{d-1}}\right)}\le M\sqrt{\lambda }$, then we have a constant $C>0$ such that for any $\delta \in (0,1)$, with confidence $1-\delta$ holds

$$\begin{array}{c}\hfill {\parallel f_{z,\lambda }^{{\Omega }^{\left(n\right)}}-f_{\rho ,\lambda }^{{\Omega }^{\left(n\right)}}\parallel }_{H_{{\Omega }^{\left(n\right)}}^{\varphi }}\le {\displaystyle \frac{2(M+\gamma )}{\lambda \sqrt{m}}}log{\displaystyle \frac{2}{\delta }}+D^{{\Omega }^{\left(n\right)}}{(f_{\rho },\lambda )}_{L^2\left({\rho }_{S^{d-1}}\right)}.\end{array}$$

(39)

Proof. 

See the proof in Section 5. □

Corollary 5.

Let $\varphi \in L_{W_{\eta }}^{\infty }$ satisfy $\widehat{a_l^{\eta }\left(\varphi \right)}>0$ for all $l\ge 0$ and ${\Omega }^{\left(2\right(N+n\left)\right)}\in S^{d-1}$ be the MZS defined as in Proposition 5. If $f_{\rho }\in H_{{\Omega }^{\left(n\right)}}^{\varphi }$, then for any $\delta \in (0,1)$, with confidence $1-\delta$, holds

$$\begin{array}{cc}& {\parallel f_{z,\lambda }^{{\Omega }^{\left(n\right)}}-f_{\rho }\parallel }_{L^2\left({\rho }_{S^{d-1}}\right)}\hfill \\ =& O\left({\displaystyle \frac{log\frac{2}{\delta }}{\lambda \sqrt{m}}}\right)+O\left({\displaystyle \frac{n^{\frac{1}{2}}{E}_N{\left(\varphi \right)}_{2,W_{\eta }}{\parallel f_{\rho }\parallel }_{2,S^{d-1}}}{{\varphi }_n}}+E_n{\left(f_{\rho }\right)}_{2,S^{d-1}}+{\displaystyle \frac{n^{\frac{1}{2}}\sqrt{\lambda }{\parallel f\parallel }_{2,S^{d-1}}}{{\varphi }_n}}\right).\hfill \end{array}$$

(40)

Proof. 

In this case, $D^{{\Omega }^{\left(n\right)}}{(f_{\rho },\lambda )}_{L^2\left({\rho }_{S^{d-1}}\right)}=O\left(\sqrt{\lambda }\right)$. We have (40) by (39) and (34). □

3.5. Comments

We propose the concept of MZS for a scattered node set on the unit sphere, with which we show that the related convolutional zonal translation network is a reproducing kernel Hilbert space, and we show the learning rate for the kernel regularized least square regression model. We give further comments for the results.

(1) The zonal translation network that we have chosen is a finite-dimensional reproducing kernel Hilbert space; our discussions belong to the scope of the kernel method, which is a combination and application of (neural) translation networks with learning theory.

(2) Compared with the existing convergence rate estimate of neural network learning, our upper estimates are dimensional-independent (see Theorem in [25], Theorem 3.1 in [70], Theorem 7 in [71], Theorem 1 in [26]).

(3) The density derivation in Theorem 1 for the zonal translation network is qualitative; the density deduction in Theorem 2 is quantitative with the help of spherical Fourier analysis. We think that this method can be extended to other domains such as the unit ball, the Euclidean space $R^d$ and $R_{+}=[0,+\infty )$, et al.

(4) It is hopeful that with the help of the MZ-condition, one may show the reproducing property for a deep translation network, and thus give investigations for the performance of deep convolutional translation learning with the kernel method (see [6,7,33]).

(5) We provide a method of constructing a finite-dimensional reproducing kernel Hilbert space with a convolutional kernel defined on the domains having a near-best-approximation operator, for example, the interval $[-1,1]$, the unit sphere $S^{d-1}$, and the unit ball $B^d$, etc. (see [64]). The only assumption that we need is (25). The set ${\Omega }^{\left(n\right)}\subset \Omega$ may be any finite scattered sets satisfying the MZ condition (23), whose parameters $A_k^{\left(n\right)}$ can be obtained according to (5.3.5) in Theorem 5.3.6 of [64], as we know that this is the first time that the reproducing property of the zonal neural network is shown.

(6) In many research references, to obtain an explicit learning rate, one often assumes that the approximation error (i.e., the K-functional) has a decay of power convergence, i.e., $D^{{\Omega }^{\left(n\right)}}{(f_{\rho },\lambda )}_{L^2\left({\rho }_{S^{d-1}}\right)}=O\left({\lambda }^{\beta }\right),\lambda \to 0^{+},0<\beta \le 1.$ It was proved in [72] that the K-functional is equivalent to a modulus of smoothness. In this paper, it is the first time that an upper estimate for the convergence rate of the K-functional has been provided (see (34)).

(7) One advantage of framework (31) is that it is a finite-dimensional quadratic nonlinear strict convex optimization problem; since the structure of $H_{{\Omega }^{\left(n\right)}}^{\varphi }$, so the optimal solution for (31) is unique and can be obtained by the gradient descent algorithm.

(8) It is easy to see that the optimal solution $f_{\rho ,\lambda }^{{\Omega }^{\left(n\right)}}$ depends upon both the distribution $\rho$ and the function $\varphi$. How to quantitatively describe the influence level, i.e., the robustness of $f_{\rho ,\lambda }^{{\Omega }^{\left(n\right)}}$ with respect to $\rho$ and $\varphi$, is a significant research direction; for such kinds of research, one can refer to references [73,74,75].

(9) Combining the upper estimate (40) and convergence (35), we know that if $m=m\left(n\right)\gg N\gg n$ are chosen such that ${\displaystyle \frac{n^{\frac{1}{2}}}{N^{2l}{\varphi }_n}}\to 0(n\to +\infty )$ and ${\displaystyle \frac{n^{\frac{1}{2}}\sqrt{\lambda }}{{\varphi }_n}}\to 0(n\to +\infty )$, then for any $\delta \in (0,1)$, with confidence $1-\delta$, holds the convergence

$$\begin{array}{c}\hfill \underset{n\to +\infty }{lim}\parallel f_{z,\lambda }^{{\Omega }^{\left(n\right)}}-f_{\rho }{\parallel }_{L^2\left({\rho }_{S^{d-1}}\right)}=0,f_{\rho }\in H_{{\Omega }^{\left(n\right)}}^{\varphi }.\end{array}$$

(41)

Convergence (41) shows that under these assumptions, algorithm (31) is convergent.

(10) We now make a comparison of the learning framework in present paper with the general learning framework (1) associated with a reproducing kernel Hilbert space $H_K$. The Theorem 1 in [76] shows the sample error

$$\begin{array}{c}\hfill \parallel f_{z,\lambda }^{\left(H_K\right)}-f_{\rho ,\lambda }^{\left(H_K\right)}{\parallel }_K\le {\displaystyle \frac{6cMlog(2/\delta )}{\lambda \sqrt{m}}}.\end{array}$$

(42)

Inequality (42) is a fundamental inequality for obtaining the optimal learning rate with the integral operator approach (see Theorem 2 in [76]). Inequality (37) in Theorem 3 shows that the sample error estimate (42) also holds for (31). But $H_{{\Omega }^{\left(n\right)}}^{\varphi }$ is a finite-dimensional proper subset of ${\Delta }_{\varphi }^{S^{d-1}}$, which is a reproducing kernel Hilbert space. These show that the learning framework (1) may obtain algorithm convergence as well as the optimal learning rate if $n,N$ and $\varphi$ are chosen properly.

(11) In this paper, we have shown our idea of the kernel regularized translation network learning with the zonal translation network. The essence is an application of the MZ inequality and the exact QR, or the MZ-condition and the MZS. We conjecture that the results in the present may be extended to many other translation networks whose domains satisfy the MZ-condition and the MZS, for example, the periodic translation network (see [27]), the translation on the interval $[-1,1]$ (see [30]), the unit sphere $S^{d-1}$ (see [29]), and the unit ball $B^d$ (see [62]).

(12) Recently, the exact spherical QR is used to investigate the convergence for the spherical scattered data-fitting problem (see [25,77,78]). The Tikhonov regularization model used is the following type:

$$\begin{array}{c}\hfill f_{{\Lambda }^{\left(m\right)},\lambda }^{\left({\mathcal{N}}_{\varphi }\right)}:=arg\underset{f\in {\mathcal{N}}_{\varphi }}{min}\left(\sum _{z_k\in {\Lambda }^{\left(m\right)}}{\mu }_k^{\left(m\right)}{(f\left(z_k\right)-y_k)}^2+\lambda {\parallel f\parallel }_{{\mathcal{N}}_{\varphi }}^2\right),\end{array}$$

where ${\mathcal{N}}_{\varphi }$ is a native space of the type ${\Delta }_{\varphi }^{S^{d-1}}$, ${\Lambda }^{\left(m\right)}={\left\{z_k\right\}}_{z_k\in {\Lambda }^{\left(m\right)}}\subset S^{d-1}$ is a scattered set, and $y_k=f^{*}\left(z_k\right)+{\epsilon }_k(z_k\in {\Lambda }^{\left(m\right)})$ and ${\mu }_k^{\left(m\right)}$ are the positive numbers defined in the polynomial exact QR as in (21); $f^{*}\left(x\right)$ is the target function to be fitted. It is hopeful that the method used in the present may be used to investigate the convergence of the algorithm

$$\begin{array}{c}\hfill f_{{\Lambda }^{\left(m\right)},\lambda }^{{\Omega }^{\left(n\right)}}:=arg\underset{f\in H_{{\Omega }^{\left(n\right)}}^{\varphi }}{min}\left(\sum _{z_k\in {\Lambda }^{\left(m\right)}}{\mu }_k^{\left(m\right)}{(f\left(z_k\right)-y_k)}^2+\lambda {\parallel f\parallel }_{H_{{\Omega }^{\left(n\right)}}^{\varphi }}^2\right),\end{array}$$

where ${\Omega }^{\left(n\right)}$ and $H_{{\Omega }^{\left(n\right)}}^{\varphi }$ are defined as in Section 2.

4. Lemmas

To give a capacity-independent generalization error for algorithm (31), we need some concepts of convex analysis.

$G\widehat{a}teaux$ differentiable. Let $(H,\parallel ·{\parallel }_H)$ be a Hilbert space and $F\left(f\right):H\to R\cup \{\mp \infty \}$ be a real function. We say that F is $G\widehat{a}teaux$ differentiable at $f_0\in H$ if there is an $\xi \in H$ such that for any $g\in H$, there holds

$$\begin{array}{c}\hfill \underset{t\to 0}{lim}{\displaystyle \frac{F(f_0+tg)-F\left(f_0\right)}{t}}={\langle g,\xi \rangle }_H\end{array}$$

and we write $F_G^{\prime }\left(f_0\right)=\xi$ or ${\nabla }_{f}F\left(f\right)=\xi$. It is known that for a differentiable convex function, $F\left(f\right)$ on H $f_0=arg\underset{f\in H}{min}F\left(f\right)$ if and only if $\left({\nabla }_{f}F\left(f\right)\right){|}_{f=f_0}=0$. (see Proposition 17.4 in [79]).

To prove the main results, we need some lemmas.

Lemma 1.

Let $(H,\parallel ·\parallel )$ be a Hilbert space, ξ be a random variable on $(Z,\rho )$ with values in H, and ${\left\{z_i\right\}}_{i=1}^m$ be independent sample drawers of ρ. Assume that ${\parallel \xi \parallel }_H\le \tilde{M}<+\infty$ almost surely. Denote ${\sigma }^2\left(\xi \right)={E(\parallel \xi \parallel }_H^2)$. Then, for any $0<\delta <1$, with confidence $1-\delta$, holds

$$\begin{array}{c}\hfill {\parallel {\displaystyle \frac{1}{m}}\sum _{i=1}^m\xi \left(z_i\right)-E\left(\xi \right)\parallel }_H\le {\displaystyle \frac{2\tilde{M}log\left(\frac{2}{\delta }\right)}{\sqrt{m}}}+\sqrt{{\displaystyle \frac{2{\sigma }^2\left(\xi \right)log\left(\frac{2}{\delta }\right)}{m}}}.\end{array}$$

(43)

Proof. 

Find it from [76]. □

Lemma 2.

Let $(H,\parallel ·{\parallel }_H)$ be a Hilbert space over X with respect to kernel K. If $(E,\parallel ·{\parallel }_E)$ and $(F,\parallel ·{\parallel }_F)$ be closed subspaces of H such that $E\perp F$ and $E⨁F=H$; then, $K=L+M$, where L and M are the reproducing kernels of E and F, respectively. Moreover, for $h=e+f,e\in E,f\in F$, we have

$$\begin{array}{c}\hfill {\parallel h\parallel }_H={\left({\parallel e\parallel }_E^2+{\parallel f\parallel }_F^2\right)}^{{\displaystyle \frac{1}{2}}}.\end{array}$$

Proof. 

See Corollary 1 in Chapter 31 of [67] or the Theorem in Section 6 in part I of [69]. □

Lemma 3.

There hold the following equalities:

$$\begin{array}{c}\hfill {\nabla }_f{\mathcal{E}}_z\left(f\right)(·)=-{\displaystyle \frac{2}{m}}\sum _{i=1}^m(y_i-f\left(x_i\right))K_{x_i}\left(\varphi \right)(·),f\in H_{{\Omega }^{\left(n\right)}}^{\varphi }.\end{array}$$

(44)

and

$$\begin{array}{c}\hfill {\nabla }_f{\mathcal{E}}_{\rho }\left(f\right)(·)=-2{\int }_Z(y-f\left(x\right))K_x\left(\varphi \right)(·)d\rho ,f\in H_{{\Omega }^{\left(n\right)}}^{\varphi }.\end{array}$$

(45)

Proof of (44). By the equality

$$\begin{array}{c}\hfill a^2-b^2=2(a-b)b+{(a-b)}^2,a\in R,b\in R\end{array}$$

(46)

we have

$$\begin{array}{ccc}& & \underset{t\to 0}{lim}{\displaystyle \frac{{\mathcal{E}}_z(f+tg)-{\mathcal{E}}_z\left(f\right)}{t}}\hfill \\ & =& \underset{t\to 0}{lim}{\displaystyle \frac{\frac{1}{m}{\sum }_{i=1}^m(-2)t(y_i-f\left(x_i\right))g\left(x_i\right)+t^{2}g{\left(x_i\right)}^2}{t}}\hfill \\ & =& -{\displaystyle \frac{2}{m}}\sum _{i=1}^m(y_i-f\left(x_i\right))g\left(x_i\right)\hfill \end{array}$$

Since $g\left(x\right)={\langle g,K_x(\varphi ;·)\rangle }_{H_{{\Omega }^{\left(n\right)}}^{\varphi }}$ and the definition of $G\widehat{a}teaux$ is derivative, we have by the above equality that

$$\begin{array}{c}\hfill \underset{t\to 0}{lim}{\displaystyle \frac{{\mathcal{E}}_z(f+tg)-{\mathcal{E}}_z\left(f\right)}{t}}={〈g,-{\displaystyle \frac{2}{m}}\sum _{i=1}^m(y_i-f\left(x_i\right))g\left(x_i\right)K_{x_i}\left(\varphi \right)(·)〉}_{H_{{\Omega }^{\left(n\right)}}^{\varphi }}.\end{array}$$

We then have (44). By the same way, we can have (45).

Lemma 4.

Let $(H,{\langle ·\rangle }_H)$ be a Hilbert space consisting of real functions on X. Then,

$$\begin{array}{c}\hfill {\parallel f\parallel }_H^2-{\parallel g\parallel }_H^2=2{\langle f-g,g\rangle }_H+{\parallel f-g\parallel }_H^2,\forall f,g\in H\end{array}$$

(47)

and

$$\begin{array}{c}\hfill {\nabla }_f{(\parallel f\parallel }_H^2)(·)=2f(·),\forall f\in H.\end{array}$$

(48)

Proof. 

Equality (47) is the deformation of the parallelogram formula. Equality (48) can be shown with (47).

□

Lemma 5.

Framework (31) has a unique solution $f_{z,\lambda }^{{\Omega }^{\left(n\right)}}$ and (32) has a unique solution $f_{\rho ,\lambda }^{{\Omega }^{\left(n\right)}}$. Moreover, There holds the bound

$$\begin{array}{c}\hfill {\parallel f_{\rho ,\lambda }^{{\Omega }^{\left(n\right)}}\parallel }_{C\left(S^{d-1}\right)}\le {\displaystyle \frac{2\kappa D^{{\Omega }^{\left(n\right)}}(f_{\rho },\lambda )}{\sqrt{\lambda }}},\end{array}$$

(49)

where κ is defined as in (30).

There hold the equality

$$\begin{array}{c}\hfill \lambda f_{z,\lambda }^{{\Omega }^{\left(n\right)}}(·)={\displaystyle \frac{2}{m}}\sum _{i=1}^m\left(y_i-f_{z,\lambda }^{{\Omega }^{\left(n\right)}}\left(x_i\right)\right)K_{x_i}\left(\varphi \right)(·)\end{array}$$

(50)

and the equality

$$\begin{array}{c}\hfill \lambda f_{\rho ,\lambda }^{{\Omega }^{\left(n\right)}}(·)=2{\int }_Z\left(y-f_{\rho ,\lambda }^{{\Omega }^{\left(n\right)}}\left(x\right)\right)K_x\left(\varphi \right)(·)d\rho \end{array}$$

(51)

Proof. 

Proof of (49). Since ${\mathcal{E}}_{\rho }\left(f_{\rho ,\lambda }^{{\Omega }^{\left(n\right)}}\right)\ge {\mathcal{E}}_{\rho }\left(f_{\rho }\right)$, we have by (2) that

$$\begin{array}{ccc}\hfill {\displaystyle \frac{\lambda }{2}}{\parallel f_{\rho ,\lambda }^{{\Omega }^{\left(n\right)}}\parallel }_{H_{{\Omega }^{\left(n\right)}}^{\varphi }}^2& \le & {\mathcal{E}}_{\rho }\left(f_{\rho ,\lambda }^{{\Omega }^{\left(n\right)}}\right)-{\mathcal{E}}_{\rho }\left(f_{\rho }\right)+{\displaystyle \frac{\lambda }{2}}{\parallel f_{\rho ,\lambda }^{{\Omega }^{\left(n\right)}}\parallel }_{H_{{\Omega }^{\left(n\right)}}^{\varphi }}^2\hfill \\ & =& \underset{g\in H_{{\Omega }^{\left(n\right)}}^{\varphi }}{inf}\left({\mathcal{E}}_{\rho }\left(g\right)-{\mathcal{E}}_{\rho }\left(f_{\rho }\right)+{\displaystyle \frac{\lambda }{2}}{\parallel g\parallel }_{H_{{\Omega }^{\left(n\right)}}^{\varphi }}^2\right)\hfill \\ & =& \underset{g\in H_{{\Omega }^{\left(n\right)}}^{\varphi }}{inf}\left(\parallel g-f_{\rho }{\parallel }_{L^2\left({\rho }_{S^{d-1}}\right)}^2+{\displaystyle \frac{\lambda }{2}}{\parallel g\parallel }_{H_{{\Omega }^{\left(n\right)}}^{\varphi }}^2\right).\hfill \end{array}$$

(52)

By (52), we have (49).

Proof of (50). By the definition of $f_{z,\lambda }^{{\Omega }^{\left(n\right)}}$ and (48), we have

$$\begin{array}{c}\hfill 0={\nabla }_f\left({\mathcal{E}}_z\left(f\right)+{\displaystyle \frac{\lambda }{2}}{\parallel f\parallel }_{H_{{\Omega }^{\left(n\right)}}^{\varphi }}^2\right){|}_{f=f_{z,\lambda }^{{\Omega }^{\left(n\right)}}},\end{array}$$

i.e.,

$$\begin{array}{ccc}\hfill 0& =& {\nabla }_f{\mathcal{E}}_z{\left(f\right)|}_{f=f_{z,\lambda }^{{\Omega }^{\left(n\right)}}}+\lambda {\nabla }_f({\displaystyle \frac{1}{2}}{\parallel f\parallel }_{H_{{\Omega }^{\left(n\right)}}^{\varphi }}^2{\left)\right|}_{f=f_{z,\lambda }^{{\Omega }^{\left(n\right)}}}\hfill \\ & =& -{\displaystyle \frac{2}{m}}\sum _{i=1}^m(y_i-f_{z,\lambda }^{{\Omega }^{\left(n\right)}}\left(x_i\right))K_{x_i}\left(\varphi \right)(·)+\lambda f_{z,\lambda }^{{\Omega }^{\left(n\right)}}(·).\hfill \end{array}$$

Hence, (50) holds. We can prove (51) in the same way. □

Lemma 6.

The solutions $f_{z,\lambda }^{{\Omega }^{\left(n\right)}}$ and $f_{\rho ,\lambda }^{{\Omega }^{\left(n\right)}}$ satisfy the inequality

$$\begin{array}{c}\hfill {\parallel f_{z,\lambda }^{{\Omega }^{\left(n\right)}}-f_{\rho ,\lambda }^{{\Omega }^{\left(n\right)}}\parallel }_{H_{{\Omega }^{\left(n\right)}}^{\varphi }}\le {\displaystyle \frac{2A\left(z\right)}{\lambda }},\end{array}$$

(53)

where

$$A\left(z\right)=\parallel {\int }_Z\left(y-f_{\rho ,\lambda }^{{\Omega }^{\left(n\right)}}\left(x\right)\right)K_x\left(\varphi \right)(·)d\rho -{\displaystyle \frac{1}{m}}\sum _{i=1}^m\left(y_i-f_{\rho ,\lambda }^{{\Omega }^{\left(n\right)}}\left(x_i\right)\right)K_{x_i}\left(\varphi \right)(·){\parallel }_{H_{{\Omega }^{\left(n\right)}}^{\varphi }}.$$

Proof. 

By (46), we have

$$\begin{array}{c}\hfill a^2-b^2\ge 2(a-b)b,a\in R,b\in R\end{array}$$

It follows that

$$\begin{array}{cc}& {\mathcal{E}}_z\left(f_{z,\lambda }^{{\Omega }^{\left(n\right)}}\right)-{\mathcal{E}}_z\left(f_{\rho ,\lambda }^{{\Omega }^{\left(n\right)}}\right)\hfill \\ =& {\displaystyle \frac{1}{m}}\sum _{i=1}^m\left[{(y_i-f_{z,\lambda }^{{\Omega }^{\left(n\right)}}\left(x_i\right))}^2-{(y_i-f_{\rho ,\lambda }^{{\Omega }^{\left(n\right)}}\left(x_i\right))}^2\right]\hfill \\ \ge & -{\displaystyle \frac{2}{m}}\sum _{i=1}^m(y_i-f_{\rho ,\lambda }^{{\Omega }^{\left(n\right)}}\left(x_i\right))\times \left(f_{z,\lambda }^{{\Omega }^{\left(n\right)}}\left(x_i\right)-f_{\rho ,\lambda }^{{\Omega }^{\left(n\right)}}\left(x_i\right)\right)\hfill \\ =& {〈f_{z,\lambda }^{{\Omega }^{\left(n\right)}}-f_{\rho ,\lambda }^{{\Omega }^{\left(n\right)}},-{\displaystyle \frac{2}{m}}\sum _{i=1}^m\left(y_i-f_{\rho ,\lambda }^{{\Omega }^{\left(n\right)}}\left(x_i\right)\right)K_{x_i}\left(\varphi \right)(·)〉}_{H_{{\Omega }^{\left(n\right)}}^{\varphi }},\hfill \end{array}$$

(54)

where we have used the reproducing property

$$f_{z,\lambda }^{{\Omega }^{\left(n\right)}}\left(x_i\right)-f_{\rho ,\lambda }^{{\Omega }^{\left(n\right)}}\left(x_i\right)={〈f_{z,\lambda }^{{\Omega }^{\left(n\right)}}-f_{\rho ,\lambda }^{{\Omega }^{\left(n\right)}},K_{x_i}\left(\varphi \right)(·)〉}_{H_{{\Omega }^{\left(n\right)}}^{\varphi }}.$$

By the definition of $f_{z,\lambda }^{{\Omega }^{\left(n\right)}}$, we have

$$\begin{array}{ccc}\hfill 0& \ge & \left({\mathcal{E}}_z\left(f_{z,\lambda }^{{\Omega }^{\left(n\right)}}\right)+{\displaystyle \frac{\lambda }{2}}{\parallel f_{z,\lambda }^{{\Omega }^{\left(n\right)}}\parallel }_{H_{{\Omega }^{\left(n\right)}}^{\varphi }}^2\right)-\left({\mathcal{E}}_z\left(f_{\rho ,\lambda }^{{\Omega }^{\left(n\right)}}\right)+{\displaystyle \frac{\lambda }{2}}{\parallel f_{\rho ,\lambda }^{{\Omega }^{\left(n\right)}}\parallel }_{H_{{\Omega }^{\left(n\right)}}^{\varphi }}^2\right).\hfill \end{array}$$

On the other hand, by the above inequality (54) and (47), we have

$$\begin{array}{ccc}\hfill 0& \ge & \left({\mathcal{E}}_z\left(f_{z,\lambda }^{{\Omega }^{\left(n\right)}}\right)-{\mathcal{E}}_z\left(f_{\rho ,\lambda }^{{\Omega }^{\left(n\right)}}\right)\right)+{\displaystyle \frac{\lambda }{2}}\left(\parallel f_{z,\lambda }^{{\Omega }^{\left(n\right)}}{\parallel }_{H_{{\Omega }^{\left(n\right)}}^{\varphi }}^2-{\parallel f_{\rho ,\lambda }^{{\Omega }^{\left(n\right)}}\parallel }_{H_{{\Omega }^{\left(n\right)}}^{\varphi }}^2\right)\hfill \\ & \ge & {〈f_{z,\lambda }^{{\Omega }^{\left(n\right)}}-f_{\rho ,\lambda }^{{\Omega }^{\left(n\right)}},-{\displaystyle \frac{2}{m}}\sum _{i=1}^m\left(y_i-f_{\rho ,\lambda }^{{\Omega }^{\left(n\right)}}\left(x_i\right)\right)K_{x_i}\left(\varphi \right)(·)〉}_{H_{{\Omega }^{\left(n\right)}}^{\varphi }}\hfill \\ & +& {〈f_{z,\lambda }^{{\Omega }^{\left(n\right)}}-f_{\rho ,\lambda }^{{\Omega }^{\left(n\right)}},\lambda j_q\left(f_{\rho ,\lambda }^{{\Omega }^{\left(n\right)}}\right)〉}_{H_{{\Omega }^{\left(n\right)}}^{\varphi }}+\lambda \parallel f_{z,\lambda }^{{\Omega }^{\left(n\right)}}-f_{\rho ,\lambda }^{{\Omega }^{\left(n\right)}}{\parallel }_{H_{{\Omega }^{\left(n\right)}}^{\varphi }}^2.\hfill \end{array}$$

Because of (51), we have

$$\begin{array}{ccc}\hfill 0& \ge & 2\langle f_{z,\lambda }^{{\Omega }^{\left(n\right)}}-f_{\rho ,\lambda }^{{\Omega }^{\left(n\right)}},{\int }_Z\left(y-f_{\rho ,\lambda }^{{\Omega }^{\left(n\right)}}\left(x\right)\right)K_x\left(\varphi \right)(·)d\rho \hfill \\ & & -{\displaystyle \frac{1}{m}}\sum _{i=1}^m\left(y_i-f_{\rho ,\lambda }^{{\Omega }^{\left(n\right)}}\left(x_i\right)\right)K_{x_i}\left(\varphi \right)(·){\rangle }_{H_{{\Omega }^{\left(n\right)}}^{\varphi }}+\lambda \parallel f_{z,\lambda }^{{\Omega }^{\left(n\right)}}-f_{\rho ,\lambda }^{{\Omega }^{\left(n\right)}}{\parallel }_{H_{{\Omega }^{\left(n\right)}}^{\varphi }}^2.\hfill \end{array}$$

By the Cauchy inequality, we have

$$\begin{array}{ccc}& & \lambda \parallel f_{z,\lambda }^{{\Omega }^{\left(n\right)}}-f_{\rho ,\lambda }^{{\Omega }^{\left(n\right)}}{\parallel }_{H_{{\Omega }^{\left(n\right)}}^{\varphi }}^2\hfill \\ & \le & 2\langle f_{\rho ,\lambda }^{{\Omega }^{\left(n\right)}}-f_{z,\lambda }^{{\Omega }^{\left(n\right)}},{\int }_Z\left(y-f_{\rho ,\lambda }^{{\Omega }^{\left(n\right)}}\left(x\right)\right)K_x\left(\varphi \right)(·)d\rho \hfill \\ & & -{\displaystyle \frac{1}{m}}\sum _{i=1}^m\left(y_i-f_{\rho ,\lambda }^{{\Omega }^{\left(n\right)}}\left(x_i\right)\right)K_{x_i}\left(\varphi \right)(·){\rangle }_{H_{{\Omega }^{\left(n\right)}}^{\varphi }}\hfill \\ & \le & \parallel {\int }_Z\left(y-f_{\rho ,\lambda }^{{\Omega }^{\left(n\right)}}\left(x\right)\right)K_x\left(\varphi \right)(·)d\rho \hfill \\ & & -{\displaystyle \frac{1}{m}}\sum _{i=1}^m\left(y_i-f_{\rho ,\lambda }^{{\Omega }^{\left(n\right)}}\left(x_i\right)\right)K_{x_i}\left(\varphi \right)(·){\parallel }_{H_{{\Omega }^{\left(n\right)}}^{\varphi }}\times 2\parallel f_{z,\lambda }^{{\Omega }^{\left(n\right)}}-f_{\rho ,\lambda }^{{\Omega }^{\left(n\right)}}{\parallel }_{H_{{\Omega }^{\left(n\right)}}^{\varphi }}.\hfill \end{array}$$

We then have (53). □

5. Proof for Theorems and Propositions

Proof of Theorem 1.

If ${\Delta }_{\varphi }^{S^{d-1}}$ is not dense in $L^2\left(S^{d-1}\right)$, i.e.,

$$clspan\left({\Delta }_{\varphi }^{S^{d-1}}\right)\ne L^2\left(S^{d-1}\right).$$

then by (b) in Proposition 1, we know ${\left(clspan\left({\Delta }_{\varphi }^{S^{d-1}}\right)\right)}^{\perp }\ne \left\{0\right\}$, and there is a nonzero functional $F\in L^2\left(S^{d-1}\right)$ such that

$$F\left(f\right)=0,f\in clspan\left({\Delta }_{\varphi }^{S^{d-1}}\right).$$

It follows that $F\left(\varphi \right(·y\left)\right)=0$ for all $y\in S^{d-1}$. By the Riesz representation Theorem, F corresponds to a nonzero $h\in L^2\left(S^{d-1}\right)$ in such a way that

$$\begin{array}{c}\hfill F\left(f\right)={\int }_{S^{d-1}}f\left(x\right)h\left(x\right)d\sigma \left(x\right),\forall f\in L^2\left(S^{d-1}\right).\end{array}$$

Consequently, ${\int }_{S^{d-1}}\varphi \left(xy\right)h\left(y\right)d\sigma \left(y\right)=0,\forall x\in S^{d-1},$ which gives

$$\begin{array}{c}\hfill {\int }_{S^{d-1}}\left({\int }_{S^{d-1}}\varphi \left(xy\right)h\left(y\right)d\sigma \left(y\right)\right)h\left(x\right)d\sigma \left(x\right)=0.\end{array}$$

(55)

Combining (55) with (15), we have

$$\begin{array}{c}\hfill \sum _{l=0}^{+\infty }\widehat{a_l^{\eta }\left(\varphi \right)}\sum _{k=1}^{dim{H}_l\left(S^{d-1}\right)}{\left({\int }_{S^{d-1}}h\left(y\right)Y_k^l\left(y\right)d\sigma \left(y\right)\right)}^2=0.\end{array}$$

It follows that ${\int }_{S^{d-1}}h\left(y\right)Y_k^l\left(y\right)d\sigma \left(y\right)=0$ for all $l\ge 0.$ Therefore, $h=0.$ We have obtained a contradiction. □

Proof of Theorem 2.

For a nonnegative function $\widehat{a}\in C^{\infty }\left(R\right)$ satisfying (a) $supp\widehat{a}\subset [0,2]$ and $\widehat{a}\left(t\right)=1,t\in [0,1]$, or (b) $supp\widehat{a}\subset [{\displaystyle \frac{1}{2}},2]$, we define a near-best-approximation operator $V_n^{\left(\eta \right)}(f,x)$ as

$$\begin{array}{c}\hfill V_n^{\left(\eta \right)}(f,x)=\sum _{l=0}^{+\infty }\widehat{a}\left({\displaystyle \frac{l}{n}}\right)\widehat{a_l^{\eta }\left(f\right)}{\displaystyle \frac{l+\eta }{\eta }}{C}_l^{\eta }\left(x\right),x\in [-1,1].\end{array}$$

Then, by [80], we know

$$\begin{array}{c}\hfill V_n^{\left(\eta \right)}(p,x)=p\left(x\right),p\in P_n,\end{array}$$

$V_n^{\left(\eta \right)}\left(f\right)\in P_{2n}$, and for any $f\in L_{W_{\eta }}^p$, there hold $\parallel V_n^{\left(\eta \right)}{\left(f\right)\parallel }_{p,W_{\eta }}\le c{\parallel f\parallel }_{p,W_{\eta }}$,

$$\begin{array}{c}\hfill \widehat{a_l^{\eta }\left(V_n^{\left(\eta \right)}\left(f\right)\right)}=\widehat{a_l^{\eta }\left(f\right)},0\le l\le n\end{array}$$

and

$$\begin{array}{c}\hfill \parallel V_n^{\left(\eta \right)}{\left(f\right)-f\parallel }_{p,W_{\eta }}\le c{E}_n{\left(f\right)}_{p,W_{\eta }},\end{array}$$

where

$$E_n{\left(f\right)}_{p,W_{\eta }}=\underset{p\in P_n}{inf}{\parallel f-p\parallel }_{p,W_{\eta }},\forall f\in L_{W_{\eta }}^p.$$

By Theorem 5.4 in [81], we have

$$\begin{array}{c}\hfill E_n{\left(f\right)}_{p,W_{\eta }}\le C{K}_l{\left(f,P_{\eta }\left(D\right),n^{-2l}\right)}_{p,W_{\eta }},l=1,2,\cdots ,1\le p\le +\infty ,\end{array}$$

(56)

where

$$K_l{\left(f,P_{\eta }\left(D\right),{\lambda }^{-l}\right)}_{p,W_{\eta }}=\underset{g\in C^{2l}[-1,1]}{inf}\left({\parallel f-g\parallel }_{p,W_{\eta }}+{\lambda }^{-l}{\parallel P_{\eta }{\left(D\right)}^l\left(f\right)\parallel }_{p,W_{\eta }}\right).$$

By (56), we know that if $P_{\eta }{\left(D\right)}^l\left(f\right)\in C^{2l}[-1,1]$, then

$$\begin{array}{c}\hfill E_n{\left(f\right)}_{p,W_{\eta }}=O\left({\displaystyle \frac{1}{n^{2l}}}\right),l=1,2,\cdots ,1\le p\le +\infty .\end{array}$$

(57)

By the same way, we define a near-best-approximation operator $V_n(f,x)$ as

$$\begin{array}{c}\hfill V_n(f,x)=\sum _{l=0}^{+\infty }\widehat{a}\left({\displaystyle \frac{l}{n}}\right)Y_l(f,x),x\in S^{d-1},f\in L^1\left(S^{d-1}\right).\end{array}$$

Then, it is known that (see Lemma 4.1.1 in [66] or see Theorem 2.6.3 in [64]) $V_n\left(f\right)\in H_{2n}\left(S^{d-1}\right)$ and $V_n\left(f\right)=f$ for $f\in H_n\left(S^{d-1}\right)$, and there is a constant $c>0$ such that for any $f\in L^p\left(S^{d-1}\right)$

$$\begin{array}{c}\hfill \parallel V_n{\left(f\right)-f\parallel }_{p,S^{d-1}}\le c{E}_n{\left(f\right)}_{p,S^{d-1}},\parallel V_n{\left(f\right)\parallel }_{p,S^{d-1}}\le c{\parallel f\parallel }_{p,S^{d-1}}.\end{array}$$

Since $V_n\left(f\right)\in H_{2n}\left(S^{d-1}\right)$, we have

$$\begin{array}{c}\hfill V_n(f,x)=a_0^{\left(0\right)}\left(V_n\left(f\right)\right)+\sum _{l=1}^{2n}\sum _{j=1}^{dim{H}_l\left(S^{d-1}\right)}{a}_j^{\left(l\right)}\left(V_n\left(f\right)\right)y_j^l\left(x\right),x\in S^{d-1}.\end{array}$$

(58)

On the other hand, by (16), we have for $N\gg 2n$ and $1\le l\le 2n$ that

$$\begin{array}{ccc}\hfill Y_j^l\left(x\right)& =& {\displaystyle \frac{1}{\widehat{a_l^{\eta }\left(V_N^{\left(\eta \right)}\left(\varphi \right)\right)}}}{\int }_{S^{d-1}}{V}_N^{\left(\eta \right)}(\varphi ,x·y)Y_j^l\left(y\right)d\sigma \left(y\right)\hfill \\ & =& {\displaystyle \frac{1}{\widehat{a_l^{\eta }\left(\varphi \right)}}}{\int }_{S^{d-1}}{V}_N^{\left(\eta \right)}(\varphi ,x·y)Y_j^l\left(y\right)d\sigma \left(y\right),Y_j^l\in H_l\left(S^{d-1}\right).\hfill \end{array}$$

(59)

Since $V_N^{\left(\eta \right)}(\varphi ,x·)Y_j^l(·)\in H_{2(N+n)}\left(S^{d-1}\right)$, we have by (21) that

$$\begin{array}{c}\hfill {\int }_{S^{d-1}}{V}_N^{\left(\eta \right)}(\varphi ,x·y)Y_j^l\left(y\right)d\sigma \left(y\right)=\sum _{x_k\in {\Omega }^{\left(2\right(N+n\left)\right)}}{\mu }_k^{\left(2\right(N+n\left)\right)}{V}_N^{\left(\eta \right)}(\varphi ,x·x_k)Y_j^l\left(x_k\right).\end{array}$$

It follows by (59) and (58) that

$$\begin{array}{ccc}\hfill V_n(f,x)& =& a_0^{\left(0\right)}\left(V_n\left(f\right)\right)+\sum _{l=1}^{2n}\sum _{j=1}^{dim{H}_l\left(S^{d-1}\right)}{a}_j^{\left(l\right)}\left(V_n\left(f\right)\right)y_j^l\left(x\right)\hfill \\ & =& a_0^{\left(0\right)}\left(V_n\left(f\right)\right)+\sum _{x_k\in {\Omega }^{\left(2\right(N+n\left)\right)}}{\mu }_k^{\left(2\right(N+n\left)\right)}{V}_N^{\left(\eta \right)}(\varphi ,x·x_k)\hfill \\ & & \times \sum _{l=1}^{2n}\sum _{j=1}^{dim{H}_l\left(S^{d-1}\right)}{\displaystyle \frac{a_j^{\left(l\right)}\left(V_n\left(f\right)\right)y_j^l\left(x_k\right)}{\widehat{a_l^{\eta }\left(\varphi \right)}}}\hfill \\ & =& a_0^{\left(0\right)}\left(V_n\left(f\right)\right)+\sum _{x_k\in {\Omega }^{\left(2\right(N+n\left)\right)}}{\mu }_k^{\left(2\right(N+n\left)\right)}{V}_N^{\left(\eta \right)}(\varphi ,x·x_k)\left(\sum _{l=1}^{2n}{\displaystyle \frac{Y_l(V_n\left(f\right),x_k)}{\widehat{a_l^{\eta }\left(\varphi \right)}}}\right).\hfill \end{array}$$

Define an operator as

$$\begin{array}{c}\hfill V_n^{\varphi }(f,x)=a_0^{\left(0\right)}\left(V_n\left(f\right)\right)+\sum _{x_k\in {\Omega }^{\left(2\right(N+n\left)\right)}}{\mu }_k^{\left(2\right(N+n\left)\right)}\varphi (x·x_k)\left(\sum _{l=1}^{2n}{\displaystyle \frac{Y_l(V_n\left(f\right),x_k)}{\widehat{a_l^{\eta }\left(\varphi \right)}}}\right).\end{array}$$

Then, $V_n^{\varphi }\left(f\right)\in H_{{\Omega }^{\left(2\right(N+n\left)\right)}}^{\varphi }\subset {\Delta }_{\varphi }^{S^{d-1}}$ and $V_n(f,x)=V_n^{V_N^{\left(\eta \right)}\left(\varphi \right)}(f,x).$

It follows that

$$\begin{array}{ccc}\hfill |V_n^{\varphi }(f,x)-f\left(x\right)|& \le & |V_n^{\varphi }(f,x)-V_n(f,x)|+|V_n(f,x)-f\left(x\right)|\hfill \\ & \le & c{E}_n{\left(f\right)}_{\infty ,S^{d-1}}+|V_n^{\varphi }(f,x)-V_n^{V_N^{\left(\eta \right)}\left(\varphi \right)}(f,x)|,\hfill \end{array}$$

(60)

where

$$\begin{array}{cc}& |V_n^{\varphi }(f,x)-V_n^{V_N^{\left(\eta \right)}\left(\varphi \right)}(f,x)|\hfill \\ \le & \sum _{x_k\in {\Omega }^{\left(2\right(N+n\left)\right)}}{\mu }_k^{\left(2\right(N+n\left)\right)}|\varphi (x·x_k)-V_N^{\left(\eta \right)}(\varphi ,x·x_k)|\left(\sum _{l=1}^{2n}{\displaystyle \frac{|Y_l(V_n\left(f\right),x_k)|}{\widehat{a_l^{\eta }\left(\varphi \right)}}}\right)\hfill \\ \le & c{E}_N{\left(\varphi \right)}_{\infty ,[-1,1]}\sum _{x_k\in {\Omega }^{\left(2\right(N+n\left)\right)}}{\mu }_k^{\left(2\right(N+n\left)\right)}\left(\sum _{l=1}^{2n}{\displaystyle \frac{|Y_l(V_n\left(f\right),x_k)|}{\widehat{a_l^{\eta }\left(\varphi \right)}}}\right).\hfill \end{array}$$

(61)

Because of (22) and (20), we have

$$\begin{array}{cc}& \sum _{x_k\in {\Omega }^{\left(2\right(N+n\left)\right)}}{\mu }_k^{\left(2\right(N+n\left)\right)}\left(\sum _{l=1}^{2n}{\displaystyle \frac{|Y_l(V_n\left(f\right),x_k)|}{\widehat{a_l^{\eta }\left(\varphi \right)}}}\right)\hfill \\ \le & \sum _{x_k\in {\Omega }^{\left(2\right(N+n\left)\right)}}{\displaystyle \frac{{\mu }_k^{\left(2\right(N+n\left)\right)}}{A_k^{\left(2\right(N+n\left)\right)}}}\times A_k^{\left(2\right(N+n\left)\right)}\left(\sum _{l=1}^{2n}{\displaystyle \frac{|Y_l(V_n\left(f\right),x_k)|}{\widehat{a_l^{\eta }\left(\varphi \right)}}}\right)\hfill \\ \le & c\sum _{x_k\in {\Omega }^{\left(2\right(N+n\left)\right)}}{A}_k^{\left(2\right(N+n\left)\right)}\left(\sum _{l=1}^{2n}{\displaystyle \frac{|Y_l(V_n\left(f\right),x_k)|}{\widehat{a_l^{\eta }\left(\varphi \right)}}}\right),\hfill \end{array}$$

where we have used the fact that ${\sum }_{x_k\in {\Omega }^{\left(2\right(N+n\left)\right)}}{\displaystyle \frac{{\mu }_k^{\left(2\right(N+n\left)\right)}}{A_k^{\left(2\right(N+n\left)\right)}}}\le c$. It follows that

$$\begin{array}{cc}& \sum _{x_k\in {\Omega }^{\left(2\right(N+n\left)\right)}}{\mu }_k^{\left(2\right(N+n\left)\right)}\left(\sum _{l=1}^{2n}{\displaystyle \frac{|Y_l(V_n\left(f\right),x_k)|}{\widehat{a_l^{\eta }\left(\varphi \right)}}}\right)\hfill \\ \le & {\displaystyle \frac{c}{{\varphi }_n}}\sum _{x_k\in {\Omega }^{\left(2\right(N+n\left)\right)}}{A}_k^{\left(2\right(N+n\left)\right)}\left(\sum _{l=1}^{2n}\left|Y_l(V_n\left(f\right),x_k)\right|\right)\hfill \\ \le & {\displaystyle \frac{c}{{\varphi }_n}}{\int }_{S^{d-1}}\sum _{l=1}^{2n}{\int }_{S^{d-1}}\left|Y_l(V_n\left(f\right),x)\right|d\sigma \left(x\right)\hfill \\ \le & {\displaystyle \frac{2c{\omega }_{d-1}{n}^{\frac{1}{2}}}{{\varphi }_n}}{\left(\sum _{l=1}^{2n}{\parallel Y_l\left(V_n\left(f\right)\right)\parallel }_{2,S^{d-1}}^2\right)}^{{\displaystyle \frac{1}{2}}}\hfill \\ \le & {\displaystyle \frac{2c{\omega }_{d-1}{n}^{\frac{1}{2}}{\parallel V_n\left(f\right)\parallel }_{2,S^{d-1}}}{{\varphi }_n}}\hfill \\ =& O\left({\displaystyle \frac{n^{\frac{1}{2}}{\parallel f\parallel }_{2,S^{d-1}}}{{\varphi }_n}}\right),\hfill \end{array}$$

(62)

where we have used equality (18) and ${\varphi }_n=\underset{1\le l\le 2n}{min}\widehat{a_l^{\eta }\left(\varphi \right)}$. Take (62) into (61). Then,

$$\begin{array}{c}\hfill |V_n^{\varphi }(f,x)-V_n^{V_N^{\left(\eta \right)}\left(\varphi \right)}(f,x)|=O\left({\displaystyle \frac{n^{\frac{1}{2}}{\parallel f\parallel }_{2,S^{d-1}}{E}_N{\left(\varphi \right)}_{\infty ,[-1,1]}}{{\varphi }_n}}\right).\end{array}$$

(63)

Collecting (63) and (60), we have a constant $C>0$ such that

$$\begin{array}{c}\hfill |V_n^{\varphi }(f,x)-f\left(x\right)|\le C\left({\displaystyle \frac{n^{\frac{1}{2}}{\parallel f\parallel }_{2,S^{d-1}}{E}_N{\left(\varphi \right)}_{\infty ,[-1,1]}}{{\varphi }_n}}+E_n{\left(f\right)}_{\infty ,S^{d-1}}\right).\end{array}$$

(64)

Since ${\varphi }_n$ depends upon n and $N\gg n$, we can choose sufficient l and N such that

$$\begin{array}{c}\hfill {\displaystyle \frac{n^{\frac{1}{2}}{\parallel f\parallel }_{2,S^{d-1}}{E}_N{\left(\varphi \right)}_{\infty ,[-1,1]}}{{\varphi }_n}}\le {\displaystyle \frac{n^{\frac{1}{2}}O(\parallel P_{\eta }{\left(D\right)}^l{\varphi \parallel }_{\infty ,[-1,1]}{)\parallel f\parallel }_{2,S^{d-1}}}{N^{2l}{\varphi }_n}}<{\displaystyle \frac{\epsilon }{2C}}.\end{array}$$

(65)

Also, since $E_n\left(f\right)\to 0$ for $n\to +\infty$, we have for sufficient large n that

$$\begin{array}{c}\hfill E_n{\left(f\right)}_{\infty ,S^{d-1}}\le {\displaystyle \frac{\epsilon }{2C}}.\end{array}$$

(66)

Taking (65) and (66) into (64), we finally arrive at (19). □

Proof of Proposition 4.

By the definition of ${\langle ·,·\rangle }_{\varphi }$ and the definition kernel $K_x^{*}(\varphi ,y)$ in (26), we have for any $f\left(x\right)={\displaystyle \sum _{x_k\in {\Omega }^{\left(n\right)}}}{c}_k{\mu }_k^{\left(n\right)}{T}_x\left(\varphi \right)\left(x_k\right)$ that

$$\begin{array}{c}\hfill {\langle f,K_x^{*}(\varphi ,·)\rangle }_{\varphi }=\sum _{x_k\in {\Omega }^{\left(n\right)}}{c}_k{\mu }_k^{\left(n\right)}{T}_x\left(\varphi \right)\left(x_k\right)=f\left(x\right).\end{array}$$

Kernel (27) then holds. We now show (28). In fact, by Cauchy’s inequality, we have

$$\begin{array}{ccc}\hfill \left|f\right(x\left)\right|& =& |\sum _{x_k\in {\Omega }^{\left(n\right)}}{c}_k{\mu }_k^{\left(n\right)}{T}_x\left(\varphi \right)\left(x_k\right)|\hfill \\ & \le & {\parallel c\parallel }_{L^2\left({\Omega }^{\left(n\right)}\right)}{\left(\sum _{x_k\in {\Omega }^{\left(n\right)}}|{\mu }_k^{\left(n\right)}\left|\right|T_x\left(\varphi \right)\left(x_k\right){|}^2\right)}^{{\displaystyle \frac{1}{2}}}\hfill \\ & =& {\parallel f\parallel }_{\varphi }{\left(\sum _{x_k\in {\Omega }^{\left(n\right)}}|{\mu }_k^{\left(n\right)}\left|\right|\varphi (x_k·x){|}^2\right)}^{{\displaystyle \frac{1}{2}}}.\hfill \end{array}$$

On the other hand, by the Minkowski inequality and inequality (23), we have

$$\begin{array}{ccc}& & {\left(\sum _{x_k\in {\Omega }^{\left(n\right)}}|{\mu }_k^{\left(n\right)}\left|\right|\varphi (x_k·x){|}^2\right)}^{{\displaystyle \frac{1}{2}}}\hfill \\ & \le & {\left(\sum _{x_k\in {\Omega }^{\left(n\right)}}|{\mu }_k^{\left(n\right)}\left|\right|(\varphi -V_n\left(\varphi \right))(x_k·x){|}^2\right)}^{{\displaystyle \frac{1}{2}}}+{\left(\sum _{x_k\in {\Omega }^{\left(n\right)}}|{\mu }_k^{\left(n\right)}\left|\right|V_n\left(\varphi \right)(x_k·x){|}^2\right)}^{{\displaystyle \frac{1}{2}}}\hfill \\ & \le & c{E}_n{\left(\varphi \right)}_{C\left(\right[-1,1\left]\right)}(\sum _{x_k\in {\Omega }^{\left(n\right)}}|{\mu }_k^{\left(n\right)}{\left|\right)}^{{\displaystyle \frac{1}{2}}}+O\left(1\right){\left({\int }_{S^{d-1}}{\left|V_n\left(\varphi \right)(x·y)\right|}^{2}d\sigma \left(x\right)\right)}^{{\displaystyle \frac{1}{2}}}\hfill \\ & =& O\left(n^{\alpha }\right)E_n{\left(\varphi \right)}_{C\left(\right[-1,1\left]\right)}+O\left(1\right){\left({\int }_{-1}^1{\left|V_n\left(\varphi \right)\left(u\right)\right|}^2{W}_{\eta }\left(u\right)du\right)}^{{\displaystyle \frac{1}{2}}}\hfill \\ & =& O\left(1\right),\hfill \end{array}$$

where we have used (17), (57), and (25). Kernel (28) thus holds, where we have used (16). □

Proof of Corollary 2.

Since ${\Omega }^{\left(n\right)}$ is defined as in Proposition 2, we know ${\mu }_k^{\left(n\right)}\ge 0$ and ${\displaystyle \sum _{x_k\in {\Omega }^{\left(n\right)}}}\left|{\mu }_k^{\left(n\right)}\right|={\omega }_{d-1}<+\infty$, and condition (25) is satisfied with $\alpha =0$; we then have the results of Corollary 2 by Proposition 4.

□

Proof of Proposition 5.

Since $V_n^{\varphi }\left(f\right)\in H_{{\Omega }^{\left(2\right(N+n\left)\right)}}^{\varphi }$, we have

$$\begin{array}{ccc}\hfill D^{{\Omega }^{\left(2\right(N+n\left)\right)}}(f,\lambda )& \le & \parallel f-V_n^{\varphi }{\left(f\right)\parallel }_{L^2\left({\rho }_{S^{d-1}}\right)}+\sqrt{{\displaystyle \frac{\lambda }{2}}}{\parallel V_n^{\varphi }\left(f\right)\parallel }_{H_{{\Omega }^{\left(2\right(N+n\left)\right)}}^{\varphi }}\hfill \\ & =& A+\sqrt{{\displaystyle \frac{\lambda }{2}}}B,\hfill \end{array}$$

(67)

where

$$A=\parallel f-V_n^{\varphi }{\left(f\right)\parallel }_{L^2\left({\rho }_{S^{d-1}}\right)}$$

and

$$B={\left(|a_0^{\left(0\right)}\left(V_n\left(f\right)\right){|}^2+\sum _{x_k\in {\Omega }^{\left(2\right(N+n\left)\right)}}{\mu }_k^{\left(2\right(N+n\left)\right)}{\left|\sum _{l=1}^{2n}{\displaystyle \frac{Y_l(V_n\left(f\right),x_k)}{\widehat{a_l^{\eta }\left(\varphi \right)}}}\right|}^2\right)}^{{\displaystyle \frac{1}{2}}}.$$

We first bound A. By (60), we have

$$\begin{array}{ccc}\hfill A& \le & \parallel V_n^{\varphi }\left(f\right)-V_n{\left(f\right)\parallel }_{2,S^{d-1}}+{\parallel V_n\left(f\right)-f\parallel }_{2,S^{d-1}}\hfill \\ & =& \parallel V_n^{\varphi }\left(f\right)-V_n{\left(f\right)\parallel }_{2,S^{d-1}}+O\left(E_n{\left(f\right)}_{2,S^{d-1}}\right),\hfill \end{array}$$

(68)

where

$$\begin{array}{cc}& \parallel V_n^{\varphi }\left(f\right)-V_n{\left(f\right)\parallel }_{2,S^{d-1}}\hfill \\ \le & \sum _{x_k\in {\Omega }^{\left(2\right(N+n\left)\right)}}{\mu }_k^{\left(2\right(N+n\left)\right)}{\left({\int }_{S^{d-1}}{|\varphi (x·x_k)-V_N^{\left(\eta \right)}(\varphi ,x·x_k)|}^{2}d\sigma \left(x\right)\right)}^{{\displaystyle \frac{1}{2}}}\hfill \\ & \times \left(\sum _{l=1}^{2n}{\displaystyle \frac{|Y_l(V_n\left(f\right),x_k)|}{\widehat{a_l^{\eta }\left(\varphi \right)}}}\right)\hfill \\ \le & c{E}_N{\left(\varphi \right)}_{2,W_{\eta }}\sum _{x_k\in {\Omega }^{\left(2\right(N+n\left)\right)}}{\mu }_k^{\left(2\right(N+n\left)\right)}\left(\sum _{l=1}^{2n}{\displaystyle \frac{|Y_l(V_n\left(f\right),x_k)|}{\widehat{a_l^{\eta }\left(\varphi \right)}}}\right)\hfill \\ =& O\left({\displaystyle \frac{n^{\frac{1}{2}}{E}_N{\left(\varphi \right)}_{2,W_{\eta }}{\parallel f\parallel }_{2,S^{d-1}}}{{\varphi }_n}}\right),\hfill \end{array}$$

(69)

where we have used (62) and (17). Collecting (69) and (68), we have

$$\begin{array}{c}\hfill A=O\left({\displaystyle \frac{n^{\frac{1}{2}}{E}_N{\left(\varphi \right)}_{2,W_{\eta }}{\parallel f\parallel }_{2,S^{d-1}}}{{\varphi }_n}}\right)+O\left(E_n{\left(f\right)}_{2,S^{d-1}}\right).\end{array}$$

(70)

We now bound B. By (20), (22), and (18), we have

$$\begin{array}{ccc}\hfill B& \le & {\left(|a_0^{\left(0\right)}\left(V_n\left(f\right)\right){|}^2+\sum _{x_k\in {\Omega }^{\left(2\right(N+n\left)\right)}}{\displaystyle \frac{{\mu }_k^{\left(2\right(N+n\left)\right)}}{A_k^{\left(2\right(N+n\left)\right)}}}\times A_k^{\left(2\right(N+n\left)\right)}{\left|\sum _{l=1}^{2n}{\displaystyle \frac{Y_l(V_n\left(f\right),x_k)}{\widehat{a_l^{\eta }\left(\varphi \right)}}}\right|}^2\right)}^{{\displaystyle \frac{1}{2}}}\hfill \\ & \le & {\left(|a_0^{\left(0\right)}\left(V_n\left(f\right)\right){|}^2+c\sum _{x_k\in {\Omega }^{\left(2\right(N+n\left)\right)}}{A}_k^{\left(2\right(N+n\left)\right)}{\left|\sum _{l=1}^{2n}{\displaystyle \frac{Y_l(V_n\left(f\right),x_k)}{\widehat{a_l^{\eta }\left(\varphi \right)}}}\right|}^2\right)}^{{\displaystyle \frac{1}{2}}}\hfill \\ & =& O{\left(|a_0^{\left(0\right)}\left(V_n\left(f\right)\right){|}^2+{\int }_{S^{d-1}}{\left|\sum _{l=1}^{2n}{\displaystyle \frac{Y_l(V_n\left(f\right),x)}{\widehat{a_l^{\eta }\left(\varphi \right)}}}\right|}^{2}d\sigma \left(x\right)\right)}^{{\displaystyle \frac{1}{2}}}\hfill \\ & =& O{\left({\displaystyle \frac{n^{\frac{1}{2}}}{{\varphi }_n}}\sum _{l=1}^{2n}{\parallel Y_l\left(V_n\left(f\right)\right)\parallel }_{2,S^{d-1}}^2\right)}^{{\displaystyle \frac{1}{2}}}\hfill \\ & =& O\left({\displaystyle \frac{n^{\frac{1}{2}}{\parallel V_n\left(f\right)\parallel }_{2,S^{d-1}}}{{\varphi }_n}}\right)\hfill \\ & =& O\left({\displaystyle \frac{n^{\frac{1}{2}}{\parallel f\parallel }_{2,S^{d-1}}}{{\varphi }_n}}\right).\hfill \end{array}$$

(71)

Taking (71) and (70) into (67), we have (34). □

Proof of Theorem 3.

Taking $\xi (x,y)(·)=\left(y-f_{\rho ,\lambda }^{{\Omega }^{\left(n\right)}}\left(x\right)\right)K_x\left(\varphi \right)(·)$, by (49), we have

$$\begin{array}{ccc}\hfill {\parallel \xi (x,y)(·)\parallel }_{H_{{\Omega }^{\left(n\right)}}^{\varphi }}& =& |K_x\left(\varphi \right)\left(x\right)\left|\right|y-f_{\rho ,\lambda }^{{\Omega }^{\left(n\right)}}\left(x\right)|\hfill \\ & \le & \kappa (M+|f_{\rho ,\lambda }^{{\Omega }^{\left(n\right)}}\left(x\right)\left|\right)\hfill \\ & \le & \kappa (M+{\displaystyle \frac{\kappa D^{{\Omega }^{\left(n\right)}}{(f_{\rho },\lambda )}_{L^2\left({\rho }_{S^{d-1}}\right)}}{\sqrt{\lambda }}}).\hfill \end{array}$$

(72)

If $\kappa D^{{\Omega }^{\left(n\right)}}{(f_{\rho },\lambda )}_{L^2\left({\rho }_{S^{d-1}}\right)}>M\sqrt{\lambda }$, then we have by (72) that

$$\begin{array}{c}\hfill {\parallel \xi (x,y)(·)\parallel }_{H_{{\Omega }^{\left(n\right)}}^{\varphi }}\le {\displaystyle \frac{2{\kappa }^2{D}^{{\Omega }^{\left(n\right)}}(f_{\rho },\lambda )}{\sqrt{\lambda }}}.\end{array}$$

(73)

Also,

$$\begin{array}{ccc}\hfill {\sigma }^2\left(\xi \right)& =& E(\parallel \xi {\parallel }_{H_{{\Omega }^{\left(n\right)}}^{\varphi }}^2)\hfill \\ & \le & {\kappa }^2{\int }_{S^{d-1}}{\left(y-f_{\rho ,\lambda }^{{\Omega }^{\left(n\right)}}\left(x\right)\right)}^{2}d\rho (x,y)\hfill \\ & \le & {\kappa }^2({\int }_{S^{d-1}}{\left(y-f_{\rho ,\lambda }^{{\Omega }^{\left(n\right)}}\left(x\right)\right)}^{2}d\rho (x,y)+{\displaystyle \frac{\lambda }{2}}\parallel f_{\rho ,\lambda }^{{\Omega }^{\left(n\right)}}{\parallel }_{H_{{\Omega }^{\left(n\right)}}^{\varphi }}^2)\hfill \\ & \le & {\kappa }^2\underset{f\in H_{{\Omega }^{\left(n\right)}}^{\varphi }}{min}\left({\mathcal{E}}_{\rho }\left(f\right)+{\displaystyle \frac{\lambda }{2}}{\parallel f\parallel }_{H_{{\Omega }^{\left(n\right)}}^{\varphi }}^2\right)\hfill \\ & \le & {\kappa }^2{\int }_Z{y}^{2}d\rho (x,y)<+\infty .\hfill \end{array}$$

(74)

Taking (73) and (74) into (43), we have (36).

Also, if $\kappa D^{{\Omega }^{\left(n\right)}}{(f_{\rho },\lambda )}_{L^2\left({\rho }_{S^{d-1}}\right)}>M\sqrt{\lambda }$, then we have by (72) that

$$\begin{array}{c}\hfill {\parallel \xi (x,y)(·)\parallel }_{H_{{\Omega }^{\left(n\right)}}^{\varphi }}\le 2M.\end{array}$$

(75)

Taking (75) and (74) into (43), we have (37). □

Proof of Corollary 4.

Because of (36), we have by (30) that

$$\begin{array}{ccc}\hfill \parallel f_{z,\lambda }^{{\Omega }^{\left(n\right)}}-f_{\rho ,\lambda }^{{\Omega }^{\left(n\right)}}{\parallel }_{L^2\left({\rho }_{S^{d-1}}\right)}& \le & \parallel f_{z,\lambda }^{{\Omega }^{\left(n\right)}}-f_{\rho ,\lambda }^{{\Omega }^{\left(n\right)}}{\parallel }_{C\left(S^{S^{d-1}}\right)}\hfill \\ & \le & \kappa \parallel f_{z,\lambda }^{{\Omega }^{\left(n\right)}}-f_{\rho ,\lambda }^{{\Omega }^{\left(n\right)}}{\parallel }_{H_{{\Omega }^{\left(n\right)}}^{\varphi }}\hfill \\ & \le & 4\kappa \left({\displaystyle \frac{M}{\lambda \sqrt{m}}}+{\displaystyle \frac{D^{{\Omega }^{\left(n\right)}}{(f_{\rho },\lambda )}_{L^2\left({\rho }_{S^{d-1}}\right)}}{{\lambda }^{\frac{3}{2}}m}}\right)log{\displaystyle \frac{4}{\delta }}.\hfill \end{array}$$

(76)

Taking (76) into (33), we have (38). By the same way, we can have (39). □