Wavelet Density and Regression Estimators for Continuous Time Functional Stationary and Ergodic Processes

Abstract

In this study, we look at the wavelet basis for the nonparametric estimation of density and regression functions for continuous functional stationary processes in Hilbert space. The mean integrated squared error for a small subset is established. We employ a martingale approach to obtain the asymptotic properties of these wavelet estimators. These findings are established under rather broad assumptions. All we assume about the data is that they are ergodic, but beyond that, we make no assumptions. In this paper, the mean integrated squared error findings in the independence or mixing setting were generalized to the ergodic setting. The theoretical results presented in this study are (or will be) valuable resources for various cutting-edge functional data analysis applications. Applications include conditional distribution, conditional quantile, entropy, and curve discrimination.

1. Introduction and Motivations

In recent years, the statistical literature has become increasingly interested in statistical issues pertaining to the study of functional random variables or variables with values in an infinite-dimensional space. The availability of data measured on ever-finer temporal/spatial grids, such as in meteorology, medicine, and satellite images, is driving the expansion of this research topic; statistical modeling of these data as random functions uncovered numerous complex theoretical and numerical research challenges. The reader may consult the monographs to summarize functional data analysis’s theoretical and practical aspects. [1] for linear models for random variables with values in a Hilbert space, [2] for scalar-on-function and function-on-function linear models, functional principal component analysis, and parametric discriminant analysis. [3], on the other hand, concentrated on nonparametric methods, particularly kernel-type estimation for scalar-on-function nonlinear regression models. Such tools were extended to classification and discrimination analysis. [4] discussed the application of several interesting statistical concepts to the functional data framework, including goodness-of-fit tests, portmanteau tests, and change point problems. [5] was interested in analyzing variance for functional data, whereas [6] was more concerned with regression analysis for gaussian processes. Recent studies and surveys on functional data modeling and analysis can be found in [7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22].

The subject of estimating conditional models has been extensively studied in the statistical literature, utilizing several estimation techniques, the most prevalent of which is the conventional kernel method. This is because these methods have broad applications and play an important role in statistical inference. However, such methods may have certain drawbacks when predicting compactly supported or discontinuous curves at boundary locations. The prevalence of alternative wavelet approaches can be attributed to the adaptability of these methods to discontinuities in the curve that is to be approximated. In application, the wavelet method offers a straightforward estimating algorithm to both implement and compute. Refs. [23,24,25,26] provided more information on wavelet theory. We quote the work of [27], which discusses the properties of wavelet approximation and analyzes the application of wavelets in various curve estimation issues in depth. In [28], numerous applications of wavelet theory are described for the independent unidimensional case, with an emphasis on calculating the integrated squared derivative density function. The findings of [28] were later extended by [29] to estimate the density derivatives for negatively and positively related sequences, respectively. In [30], wavelet estimators for partial derivatives of a multivariate probability density function were constructed, and convergence rates for an independence scenario could be determined. In detail, [31] discusses the estimation of partial derivatives of a multivariate probability density function in the presence of additive noise. We could consult [32,33] for the most recent information on this subject. Using the independent and identically distributed paradigm, [34] examined density and regression estimation issues unique to functional data. Ref. [34] proposed a novel adaptive method based on the term-by-term selection of wavelet coefficient estimators and wavelet bases for Hilbert spaces of functions.

The primary purpose of this paper is to provide further context for the earlier discussion of stationary ergodic processes. The lack of research on the general dependence framework for wavelet analysis prompted us to conduct the present study. Several arguments for contemplating an ergodic dependency structure in that data as opposed to a mixing structure are presented in [35,36,37,38,39,40,41,42], where further information on the notion of the ergodic property of processes and examples of such processes are provided. In [43], one of the arguments used to justify the ergodic setting is that establishing ergodic characteristics rather than the mixing condition can be significantly simpler for some classes of processes. Therefore, the ergodicity hypothesis provides the ideal framework for examining data series created by chaotic noise.

In the body of statistical research that has been published, estimation issues based on discretely sampled observations as well as continuous time have been investigated. In finance, even if the underlying process is continuous across time, only its values at a finite number of points are known. The first one is more important from the standpoint of financial econometrics (see [44,45]). On the other hand, the technical development of statistical inference using a continuous time record of observations is far easier to accomplish than an inference based on discretely observed data. It makes it possible to advance further in the model’s statistical analysis and find solutions to various concerns concerning discretely observed diffusions that had not been resolved until this point. Further, remember that the discrete-time model hits its limits when the discretization step equals zero, which is when the continuous-time model becomes the limit (see [46]). As a result, the asymptotic behavior of estimate processes in actuality may be close to the asymptotic behavior established theoretically for continuous-time observations if the available data are “dense enough” relative to the observation period. The knowledge of the most accurate estimate derived from continuous-time data is also of practical significance. Ref. [47] investigated the challenge of estimating an invariant density function for a continuous-time Markov process and established the mean-square consistency of kernel-type estimators. Ref. [48] demonstrated the homogeneity and consistency of these estimators. Ref. [48] showed that a family of smoothed estimators, including kernel-type estimators, are asymptotically normal with a rate of convergence $T^{1/2}$ for continuous-time processes. This finding is unexpected because the invariant density estimators operate differently in this instance than in discrete-time processes, where the convergence rate is often smaller than $1/2$. Numerous physical phenomena, such as seismic waves, the Earth’s magnetic field, and isotherms and isobars in meteorology, are functional variables observable in continuous time, as is evident.

In [49], we have considered the wavelet basis for the nonparametric estimation of density and regression functions for continuous functional stationary processes in Hilbert space. We have characterized the mean integrated square errors. This research aims to provide the first complete theoretical rationale for wavelet-based functional density and regression function estimation for continuous stationary processes by extending our earlier work [49] to the continuous setting. The employment of wavelet estimators in functional ergodic data frames and the resulting difficulty of establishing the mean integrated square error over suitable decomposition spaces is, to the best of our knowledge, still an unresolved topic in the literature. By merging different martingale theory techniques utilized in the mathematical construction of the proofs, we aim to contribute to the literature by addressing this gap. In the independent or mixing setting, the tools employed for regression estimation change significantly in the ergodic setting. Nevertheless, we shall see that more than merely mixing preexisting ideas and outcomes is needed to address the issue. In order to deal with wavelet estimators in an ergodic setting, one must resort to complex mathematical derivations.

The paper’s structure is as follows: The multiresolution analysis is described in Section 2. The principal density estimate results are presented in Section 3. Section 4 summarizes the principal outcomes for regression estimation. Section 5 provides some examples of potential applications. There are some concluding observations in Section 6. All proof is compiled in Section 7.

2. Multiresolution Analysis

Following [34,49,50], we will now introduce some basic notations for defining wavelet bases for Hilbert spaces of functions with a few modifications to accommodate our context. In this study, nonlinear, thresholded, wavelet-based estimators are examined. Beginning with a description of the fundamental theory of wavelet approaches, we then introduce nonlinear wavelet-based estimators. The interested reader should consult [23,24], see also [51,52] and the references therein, despite the fact that the wavelet is based on a separable Hilbert space $\mathbf{H}$ of real or complex-valued functions on a complete separable metric space. Let $\mathbf{H}$ represent the separable Hilbert space of real-valued functions defined on a separable complete metric space $\mathbf{S}$. Given that $\mathbf{H}$ is separable; it possesses an orthonormal basis

$$\mathcal{E}=\left\{e_j:j\in \Delta \right\},$$

where $\Delta$ is a countable index set. The space $\mathbf{H}$ is equipped with an inner product $\langle ·,·\rangle$ and a norm $\Vert ·\Vert$. Consider the sequence of subsets $\left\{{\mathcal{I}}_k;k\ge 0\right\}$ an increasing sequence of finite subsets of $\Delta$ such that

$$\bigcup _{k\ge 0}{\mathcal{I}}_k=\Delta .$$

The subset ${\mathcal{J}}_k$ denotes the orthogonal complement of ${\mathcal{I}}_k$ in ${\mathcal{I}}_{k+1}$, i.e.,

$${\mathcal{J}}_k={\mathcal{I}}_{k+1}{\mathcal{I}}_k.$$

Choose, for any $k\ge 0$, ${\zeta }_{k,\ell }\in \mathbf{S}$, $\ell \in {\mathcal{I}}_k$ and ${\eta }_{k,\ell }\in \mathbf{S}$, $\ell \in {\mathcal{J}}_k$, such that the following matrices

$$A_k={\left(e_j\left({\zeta }_{k,\ell }\right)\right)}_{(j,\ell )\in {\mathcal{I}}_k\times {\mathcal{I}}_k},B_k={\left(e_j\left({\eta }_{k,\ell }\right)\right)}_{(j,\ell )\in {\mathcal{J}}_k\times {\mathcal{J}}_k},$$

(1)

fulfill one of the two following conditions, for instance, see [34,50] and the references therein.

(A.1)

$A_k^{*}{A}_k=\mathrm{diag}{\left(a_{k,\ell }\right)}_{\ell \in {\mathcal{I}}_k}$ and $B_k^{*}{B}_k=\mathrm{diag}{\left(b_{k,\ell }\right)}_{{\ell }^{\prime }\in {\mathcal{J}}_k}$ where $a_{k,\ell }$ and $b_{k,\ell }$ for $\ell \in {\mathcal{I}}_k$ and ${\ell }^{\prime }\in {\mathcal{J}}_k$ are positive constants.

(A.2)

$A_k{A}_k^{*}=\mathrm{diag}{\left(c_{k,\ell }\right)}_{\ell \in {\mathcal{I}}_k}$ and $B_k{B}_k^{*}=\mathrm{diag}{\left(d_{k,\ell }\right)}_{{\ell }^{\prime }\in {\mathcal{J}}_k}$ where $c_{k,\ell }$ and $d_{k,\ell }$ for $\ell \in {\mathcal{I}}_k$ and ${\ell }^{\prime }\in {\mathcal{J}}_k$ are positive constants.

Condition (A.1) implies that

$$a_{k,\ell }=\sum _{j\in {\mathcal{I}}_k}|e_j\left({\zeta }_{k,\ell }\right){|}^2,\ell \in {\mathcal{I}}_k,\mathrm{a}\mathrm{n}\mathrm{d}{b}_{k,\ell }=\sum _{j\in {\mathcal{J}}_k}{\left|e_j\left({\eta }_{k,\ell }\right)\right|}^2,\ell \in {\mathcal{I}}_k,$$

(2)

which means that all the columns of $A_k$ and $B_k$ are not zero vectors. As for (A.2), it gives

$$c_{k,\ell }=\sum _{\ell \in {\mathcal{I}}_k}|e_j\left({\zeta }_{k,\ell }\right){|}^2,j\in {\mathcal{I}}_k,\mathrm{a}\mathrm{n}\mathrm{d}{d}_{k,\ell }=\sum _{\ell \in {\mathcal{J}}_k}{|e_j\left({\eta }_{k,\ell }\right)|}^2,j\in {\mathcal{I}}_k,$$

(3)

saying that all the rows of $A_k$ and $B_k$ are not zero vectors. For any $\mathbf{x}\in \mathbf{S}$, we set

$$\left\{\begin{array}{c}{\displaystyle {\varphi }_k(·;{\zeta }_{k,\ell })=\sum _{j\in {\mathcal{I}}_k}\frac{1}{\sqrt{g_{j,k,\ell }}}\overline{e_j\left({\zeta }_{k,\ell }\right)}{e}_j(·),}\\ \\ {\displaystyle {\psi }_k(·;{\eta }_{k,\ell })=\sum _{j\in {\mathcal{J}}_k}\frac{1}{\sqrt{h_{j,k,\ell }}}\overline{e_j\left({\eta }_{k,\ell }\right)}{e}_j(·),}\end{array}\right.$$

(4)

where

$$g_{j,k,\ell }=\left\{\begin{array}{c}{a}_{k,\ell }\mathrm{i}\mathrm{f}\left(\mathbf{A}.\mathbf{1}\right),\\ \\ c_{k,\ell }\mathrm{i}\mathrm{f}\left(\mathbf{A}.\mathbf{2}\right),\end{array}\right.h_{j,k,\ell }=\left\{\begin{array}{c}{b}_{k,\ell }\mathrm{i}\mathrm{f}\left(\mathbf{A}.\mathbf{1}\right),\\ \\ d_{k,\ell }\mathrm{i}\mathrm{f}\left(\mathbf{A}.\mathbf{2}\right),\end{array}\right.$$

(5)

The following collection serves as the orthonormal basis for $\mathbf{H}$ (see Theorem 2 of [50]):

$$\mathcal{B}=\left\{{\varphi }_0\left(x,{\zeta }_{0,\ell }\right),\ell \in {\mathcal{I}}_0;{\psi }_k\left(x,{\eta }_{k,\ell }\right),k\ge 0,\ell \in {\mathcal{J}}_k\right\}.$$

(6)

For further details, see [34,50,53]. Hence, we infer that for any $f\in \mathbf{H}$, we have

$$f\left(\mathbf{x}\right)=\sum _{\ell \in {\mathcal{I}}_0}{\alpha }_{0,\ell }{\varphi }_0\left(\mathbf{x};{\zeta }_{0,\ell }\right)+\sum _{k\ge 0}\sum _{\ell \in {\mathcal{J}}_k}{\beta }_{k,\ell }{\psi }_k\left(\mathbf{x};{\eta }_{k;\ell }\right),$$

(7)

where

$${\alpha }_{0,\ell }=\langle f,{\varphi }_0\left(·;{\zeta }_{0,\ell }\right)\rangle ,{\beta }_{k,\ell }=\langle f,{\psi }_k\left(·;{\eta }_{k;\ell }\right)\rangle .$$

(8)

Adding two additional assumptions to the orthonormal basis $\mathcal{E}$:

(E.1)

There exists a constant $C_1>0$ such that, for any integer $k\ge 0$, we have

(i)

$$\sum _{j\in {\mathcal{I}}_k}\frac{1}{g_{j,k,\ell }}{\left|e_j\left({\zeta }_{k,\ell }\right)\right|}^2\le C_1,$$

(ii)

$$\sum _{j\in {\mathcal{I}}_k}\frac{1}{h_{j,k,\ell }}{\left|e_j\left({\eta }_{k,\ell }\right)\right|}^2\le C_1.$$

(E.2)

There exists a constant $C_2>0$ such that, for any integer $k\ge 0$, we have

$$\underset{\mathbf{x}\in \mathcal{S}}{sup}\sum _{j\in {\mathcal{J}}_k}{\left|e_j\left(\mathbf{x}\right)\right|}^2\le C_2\left|{\mathcal{J}}_k\right|.$$

 Remark 1.

Clearly, the assumption (E.1) is satisfied under assumption (A.1) whenever $C_1=1$; we may also consult [50], Example 2 and its applications for further details. [50,53] have provided three examples satisfying condition (E.2) taking

$$\underset{\mathbf{x}\in \mathcal{S}}{sup}\sum _{j\in {\mathcal{J}}_k}{\left|e_j\left(\mathbf{x}\right)\right|}^2\le 1,$$

see also [53] Theorem 3.2. Moreover, [34] has used both assumptions in the independent and identically distributed functional data.

Besov Space

Over the years, numerous statisticians have pondered the following question: given an estimating method and a required estimation rate for a certain loss function, what is the largest space over which this rate can be accomplished, for example, see [27,54] and the references therein. We are interested in estimating methods based on thresholding methods given by wavelet bases in a natural situation. It is common knowledge that wavelet bases serve to characterize the smoothness of spaces such as the Hölder spaces $C^s$, Sobolev spaces $W^s\left(L_p\right)$, and Besov spaces $B_q^s\left(L_p\right)$ for a range of indices s that depend on both the smoothness properties that are dependent on both the smoothness of $\psi$ and its dual function $\tilde{\psi }$, for instance, see [54] for more detail and examples, at this point, we may consult [55]. The following statistical definition is used in approximation theory for the study of nonlinear procedures, such as thresholding and greedy algorithms; see [27,52,54,56].

 Definition 1

(Besov space). Let $s>0$. We say that the function $f\in \mathbf{H}$, defined by statement (7), belongs to the Besov space ${\mathcal{B}}_{\infty }^s\left(\mathbf{H}\right)$ if and only if:

$$\underset{m\ge 0}{sup}|{\mathcal{J}}_m{|}^{2s}\sum _{k\ge m}\sum _{\ell \in {\mathcal{J}}_k}{\left|{\beta }_{k,\ell }\right|}^2<\infty .$$

(9)

 Definition 2

(Weak Besov space). Let $r>0$. We say that the function $f\in \mathbf{H}$, defined by statement (7), belongs to the weak Besov space ${\mathcal{W}}^r\left(\mathbf{H}\right)$ if and only if:

$$\underset{\lambda \ge 0}{sup}{\lambda }^r\sum _{k\ge 0}\sum _{\ell \in {\mathcal{J}}_k}{\displaystyle {\mathrm{𝟙}}_{\left\{|{\beta }_{k,\ell }|\ge \lambda \right\}}}<\infty .$$

(10)

3. The Density Estimation

Let ${\{{\mathbf{X}}_t,Y_t\}}_{t\ge 0}$ denote a sequence of strictly stationary ergodic pairs of random elements, where ${\mathbf{X}}_t$ takes values in a complete separable metric space of Hilbert space $\mathbf{S}$ associated with the corresponding Borel $\sigma$-algebra $\mathcal{B}$ and $Y_i$ is a real or complex-valued variable. Let ${\mathbb{P}}_{\mathbf{X}}$ denote the probability measure induced by ${\mathbf{X}}_0$ on $(\mathbf{S},\mathcal{B})$. Assume that there exists $\sigma$-finite measure $\nu$ on the measurable space $(\mathbf{S},\mathcal{B})$ in such a way that ${\mathbb{P}}_{\mathbf{X}}$ is dominated by $\nu$. The Radon–Nikodym theorem guarantees the existence of a measurable function that is nonnegative $f(·)$ in such a way that

$${\mathbb{P}}_{\mathbf{X}}\left(B\right)={\int }_{B}f\left(\mathbf{x}\right)\nu \left(d\mathbf{x}\right),B\in \mathcal{B}.$$

(11)

In this framework, we intend to estimate $f(·)$ on the basis of n observed functional data ${\left({\mathbf{X}}_t\right)}_{\{0\le t\le T\}}$. We assume that $f\in \mathbf{H}$, where $\mathbf{H}$ is a separable Hilbert space of real or complex-valued functions defined on $\mathbf{S}$ and square integrable with respect to the $\sigma$-finite measure $\nu$. In this research, we are especially interested in the wavelet estimation processes created in the 1990s, see Meyer’s work for the functional data of a Hilbert space and, more specifically, the nonlinear estimators. The majority of this model’s approaches involve introducing kernel estimator techniques to estimate the functional component of the model, see [57]. Let $f(·)$ be the sample’s common density function ${\left({\mathbf{X}}_t\right)}_{\{0\le t\le T\}}$, which is assumed to be

(F.1)

$\exists C_f>0$ is a known constant such that

$$\underset{\mathbf{x}\in \mathbf{S}}{sup}f\left(\mathbf{x}\right)\le C_f.$$

(12)

Density Function Estimator

From now, suppose that the density function $f(·)\in \mathbf{H}$ is a separable Hilbert space. Then $f(·)$ satisfies the wavelet representation (7). Assume that we observe a sequence ${\left\{({\mathbf{X}}_t,Y_t)\right\}}_{0\le t\le T}$ of copies of $(\mathbf{X},Y)$ that is supposed to be functional stationary and ergodic with $\mathbf{X}$ with the density function $f(·)$. We examine density estimation utilizing wavelet bases for Hilbert spaces of functions of [50]. We consider the estimated coefficients $\left\{{\alpha }_{k,\ell }\right\}$ and $\left\{{\beta }_{k,\ell }\right\}$ given, respectively, by (14) and (15), for any $j_0\le m$. Here the resolution level $m=m\left(T\right)\to \infty$ at the rate specified below. Since we assume that $\varphi (·)$ and ${\psi }_i(·)$ have a compact support so that the summations in (7) are finite for each fixed $\mathbf{x}$ (note that in this case, the support of $\varphi (·)$ and ${\psi }_i(·)$ is a monotonically increasing function of their degree of differentiability [24]). We focus our attention on the nonlinear estimators (13), which will be studied in the mean integrated squared error over adapted decomposition spaces, in a similar way as in [34] in the setting of the independent and identically distributed functional processes, in particular, we refer to [49]. The density wavelet hard thresholding estimator $\widehat{f}(·)$ is defined for all $\mathbf{x}\in \mathbf{S}$, by

$${\widehat{f}}_T\left(\mathbf{x}\right)=\sum _{\ell \in {\mathcal{I}}_0}{\widehat{\alpha }}_{0,\ell }{\varphi }_0\left(\mathbf{x};{\zeta }_{0,\ell }\right)+\sum _{k=0}^{m_T}\sum _{\ell \in {\mathcal{J}}_k}{\widehat{\beta }}_{k,\ell }{\mathrm{𝟙}}_{\left\{|{\widehat{\beta }}_{k,\ell }|\ge \kappa \sqrt{\frac{lnT}{T}}\right\}}{\psi }_k\left(\mathbf{x};{\eta }_{k;\ell }\right),$$

(13)

where

$$\begin{array}{ccc}\hfill {\widehat{\alpha }}_{k,\ell }& =& \frac{1}{T}{\int }_0^T{\varphi }_k\left({\mathbf{X}}_t;{\zeta }_{k,\ell }\right)dt,\hfill \end{array}$$

(14)

$$\begin{array}{ccc}\hfill {\widehat{\beta }}_{k,\ell }& =& \frac{1}{T}{\int }_0^T{\psi }_k\left({\mathbf{X}}_t;{\eta }_{k,\ell }\right)dt.\hfill \end{array}$$

(15)

Here $\kappa$ denotes a large-enough constant, and $m_T$ is the integer satisfying

$$\frac{1}{2}\frac{T}{lnT}\le \left|{\mathcal{J}}_{m_T}\right|\le \frac{T}{lnT}.$$

(16)

See [36,38] for further details on the multivariate case.

Comments on the Method of Estimation

Our estimation method is divided into three steps:

1.

Estimation of the wavelet coefficients ${\alpha }_{k,\ell }$ and ${\beta }_{k,\ell }$, see statement (8), ${\widehat{\alpha }}_{k,\ell }$ and ${\widehat{\beta }}_{k,\ell }$ given by Equations (14) and (15);

2.

The greatest ${\widehat{\beta }}_{k,\ell }$ is selected by applying hard thresholding;

3.

Reconstruction of the selected elements of the initial wavelet basis.

It is essential to highlight that our choice is the universal threshold $\kappa {\left(\frac{lnT}{T}\right)}^{1/2}$ and the definition of $m_T$ is based on theoretical considerations. The estimator under consideration is not dependent on the smoothness of $f(·)$; see [53] for more details in the case of the linear wavelet estimator of $f(·)$. Moreover, for additional information on the case of $\mathbf{H}=\mathbb{L}\left(\right[a,b\left]\right)$ and more standard nonparametric models, refer to [27,58]. Notation is necessary for stating the results. Throughout the paper, we shall denote by $B\in \mathcal{B},$ the open set of the Borel $\sigma -$ algebra $\mathcal{B}$. For any $0\le t\le T$ and $\delta >0$ small real, we define

$$F_{{\mathbf{X}}_t}=\mathbb{P}\left({\mathbf{X}}_t\in B\right)={\mathbb{P}}_X\left(B\right),\mathrm{s}\mathrm{e}\mathrm{e} \mathrm{s}\mathrm{t}\mathrm{a}\mathrm{t}\mathrm{e}\mathrm{m}\mathrm{e}\mathrm{n}\mathrm{t} \left(11\right),$$

and

$$F_{{\mathbf{X}}_t}^{{\mathcal{F}}_{t-\delta }}=\mathbb{P}\left({\mathbf{X}}_t\in B|{\mathcal{F}}_{t-\delta }\right)$$

as the distribution function and the conditional distribution function, given the $\sigma -$field ${\mathcal{F}}_{t-\delta }$, respectively. Before presenting our results, we present supplementary notation and our hypotheses. For the remainder of the paper, for a positive real $\delta$, we will denote by

$$n=\frac{T}{\delta }\in \mathcal{N},\mathrm{a}\mathrm{n}\mathrm{d}{T}_j=j\delta ,\mathrm{f}\mathrm{o}\mathrm{r}j=1,\dots ,n.$$

Let ${\mathcal{F}}_{t-\delta }$ be the $\sigma -$field generated by

$$\left\{(X_s,Y_s):0\le s<t-\delta \right\}$$

Let ${\mathcal{F}}_j$ be the $\sigma -$field generated by

$$\left\{(X_s,Y_s):0\le s\le T_j\right\}$$

and

$${\mathcal{S}}_t=\sigma \left\{(X_s,Y_s);\left(X_r\right):0\le s\le t,t\le r\le t+\delta \right\}.$$

The following assumptions are required for the entire paper.

(C.0)

There is a nonnegative measurable function $f_t^{{\mathcal{F}}_{t-\delta }}$ such that

$${\mathbb{P}}_{\mathbf{X}}^{{\mathcal{F}}_{t-\delta }}\left(B\right)={\int }_B{f}_t^{{\mathcal{F}}_{t-\delta }}\left(\mathbf{x}\right)\nu \left(d\mathbf{x}\right),B\in \mathcal{B}.$$

(17)

We may refer to [3,35,59] for further details.

(C.1)

For any $\mathbf{x}\in \mathbf{S}$

$$\underset{n\to \infty }{lim}\frac{1}{T}{\int }_0^T{f}_t^{{\mathcal{F}}_{t-\delta }}\left(\mathbf{x}\right)=f\left(\mathbf{x}\right),\mathrm{i}\mathrm{n} \mathrm{t}\mathrm{h}\mathrm{e}a.s.\mathrm{a}\mathrm{n}\mathrm{d}{L}^2\mathrm{s}\mathrm{e}\mathrm{n}\mathrm{s}\mathrm{e}.$$

At this point, we may refer to [60] for further details.

Comments on hypotheses. Approximating the integral ${\displaystyle {\int }_0^T{f}_t^{{\mathcal{F}}_{t-\delta }}\left(\mathbf{x}\right)dt}$ by its Riemann’s sum, we have

$${\displaystyle \frac{1}{T}{\int }_0^T{f}_t^{{\mathcal{F}}_{t-\delta }}\left(\mathbf{x}\right)dt=\frac{1}{T}\sum _{i=1}^n{\int }_{T_{i-1}}^{T_i}{f}_t^{{\mathcal{F}}_{t-\delta }}\left(\mathbf{x}\right)dt\backsimeq \frac{1}{n}\sum _{i=1}^n{f}_{T_i}^{{\mathcal{F}}_{T_{i-1}}}\left(\mathbf{x}\right).}$$

By the fact that process ${\left(X_{T_j}\right)}_{j\ge 1}$ is stationary and ergodic, in a similar way as in [61] (see, Lemma 4 and Corollary 1 together with their proofs), one may establish that the sequence

$${\left(f_{T_i}^{{\mathcal{F}}_{T_{i-1}}}\left(\mathbf{x}\right)\right)}_{i\ge 1}={\left(f_{i\delta ,(i-1)\delta }\left(\mathbf{x}\right)\right)}_{i\ge 1}$$

of random functions is stationary and ergodic. Indeed, it suffices to substitute the conditional densities in the work of Delecroix with $f_{i\delta ,(i-1)\delta }$ and the density by function $f(·)$. Refer to [35,62] and the references therein.

 Theorem 1. 

Under the assumptions (C.0), (C.1), (F.1), (E.1), and (E.2), and Equation (16) for any $\theta \in (0,1)$,

$$f\in {\mathcal{B}}_{\infty }^{\theta /2}\left(H\right)\cap {\mathcal{W}}^{2(1-\theta )}\left(H\right),$$

there is a constant $C_1>0$ such that

$$\mathbb{E}\left({∥{\widehat{f}}_T\left(\mathbf{x}\right)-f\left(\mathbf{x}\right)∥}^2\right)\le C_1{\left(\frac{lnT}{T}\right)}^{\theta },$$

(18)

for a large-enough T.

The following upper bound finding is an immediate consequence: if $f\in$${\mathcal{B}}_{\infty }^{s/(2s+1)}\left(\mathbf{H}\right)\cap {\mathcal{W}}^{2/(2s+1)}\left(\mathbf{H}\right)$ for $s>0$, then there is a constant $C_2>0$ in such a way that

$$\mathbb{E}\left(\parallel {\widehat{f}}_T{-f\parallel }^2\right)\le C_2{\left(\frac{lnT}{T}\right)}^{2s/(2s+1)}.$$

This convergence rate is close to optimal in the “standard” minimax configuration (see, [27]). Moreover, on the application of [52] (Theorem 3.2), one obtains that ${\mathcal{B}}_{\infty }^{\theta /2}\left(\mathbf{H}\right)\cap$${\mathcal{W}}^{2(1-\theta )}\left(\mathbf{H}\right)$ is the “maxiset” associated with $\widehat{f}(·)$ at the convergence rate of ${(lnT/T)}^{\theta }$, i.e.,

$$\underset{T\to \infty }{lim}{\left(\frac{T}{lnT}\right)}^{\theta }\mathbb{E}\left(\parallel {\widehat{f}}_T{-f\parallel }^2\right)<\infty \iff f\in {\mathcal{B}}_{\infty }^{\theta /2}\left(\mathbf{H}\right)\cap {\mathcal{W}}^{2(1-\theta )}\left(\mathbf{H}\right).$$

4. The Regression Estimation

Let $\rho :{\mathbb{R}}^q\to \mathbb{R}$ be a measurable function. The regression function $\mathbf{m}(·,\rho )$ is defined by

$$\rho \left(Y\right)=\mathbf{m}(\mathbf{X},\rho )+ϵ,$$

(19)

where $ϵ$ is a random variable independent of $\mathbf{X}$ with $\mathcal{N}(0,1)$. We assume that $\mathbf{m}(·,\rho )\in \mathbf{H}$, where $\mathbf{H}$ is a separable Hilbert space of real or complex-valued functions defined on $\mathbf{S}$ and a square-integrable with respect to the $\sigma$-finite measure $\nu$. We shall assume that there is a known constant and $C_{\mathbf{m}}>0$ in such a way that

$$\underset{\mathbf{x}\in \mathbf{S}}{sup}\mathbf{m}(\mathbf{x},\rho )\le C_{\mathbf{m}}.$$

(20)

In this framework, we redefine the probability measure ${\mathbb{P}}_{\mathbf{X}}$ in (11) and assume that $f(·)$ is a nonnegative measurable known function.

(M.1)

We shall assume that there is a known constant $C_{\mathbf{m}}>0$ in such a way that

$$\underset{\mathbf{x}\in \mathbf{S}}{sup}\mathbf{m}(\mathbf{x};\rho )\le C_{\mathbf{m}}.$$

(M.2)

We shall suppose that there is a known constant $c_f>0$ in such a way that

$$\underset{\mathbf{x}\in \mathbf{S}}{inf}f\left(\mathbf{x}\right)\ge c_f.$$

Regression Function Estimator

In this framework, our goal is to estimate $\mathbf{m}(·,\rho )$ based on observed functional data ${({\mathbf{X}}_t,Y_t)}_{\{0\le t\le T\}}$. The kernel estimator for the regression function of functional data has been suggested by [59]

$${\widehat{\mathbf{m}}}_{n;h_T}(\mathbf{x},\rho ):=\frac{{\displaystyle {\int }_0^T\rho \left(Y_t\right)K\left(\frac{d(\mathbf{x},{\mathbf{X}}_t)}{h_T^d}\right)dt}}{{\displaystyle {\int }_0^{T}K\left(\frac{d(\mathbf{x},{\mathbf{X}}_t)}{h_T^d}\right)dt}}.$$

This estimator is similar to the one proposed by [59] in the discrete framework. By using the idea of [58], we propose the hard wavelet thresholding estimator $\widehat{\mathbf{m}}(·,\rho )$, for all $\mathbf{x}\in \mathbf{S}$, by

$$\widehat{\mathbf{m}}(\mathbf{x},\rho )=\sum _{\ell \in {\mathcal{I}}_0}{\widehat{\eta }}_{0,\ell }{\varphi }_0\left(\mathbf{x};{\zeta }_{0,\ell }\right)+\sum _{k=0}^{m_T}\sum _{\ell \in {\mathcal{J}}_k}{\widehat{\theta }}_{k,\ell }{\displaystyle {\mathrm{𝟙}}_{\left\{{\displaystyle |{\widehat{\theta }}_{k,\ell }|\ge \kappa \sqrt{\frac{lnT}{T}}}\right\}}}{\psi }_k\left(\mathbf{x};{\eta }_{k;\ell }\right),$$

(21)

where

$$\begin{array}{ccc}\hfill {\widehat{\eta }}_{k,\ell }& =& \frac{1}{T}{\int }_0^T\frac{\rho \left(Y_t\right)}{f\left({\mathbf{X}}_t\right)}{\varphi }_k\left({\mathbf{X}}_t;{\zeta }_{k,\ell }\right)dt,\hfill \end{array}$$

(22)

$$\begin{array}{ccc}\hfill {\widehat{\theta }}_{k,\ell }& =& \frac{1}{T}{\int }_0^T\frac{\rho \left(Y_t\right)}{f\left({\mathbf{X}}_t\right)}{\psi }_k\left({\mathbf{X}}_t;{\eta }_{k,\ell }\right)dt.\hfill \end{array}$$

(23)

where $m_T$ is the integer satisfying

$$\frac{1}{2}\frac{T}{{(lnT)}^2}\le \left|{\mathcal{J}}_{m_T}\right|\le \frac{T}{{(lnT)}^2},$$

(24)

and $\kappa$ is a large-enough constant. The multivariate case for discrete- and continuous-time linear wavelet estimators was examined by [36,38].

 Theorem 2.

Under the assumptions (E.1), (E.2) (M.1), (M.2), (C.0), and (C.1), combined with condition (24), for any $\theta \in (0,1)$, $\mathbf{m}(·,\rho )\in {\mathcal{B}}_{\infty }^{\theta /2}\left(H\right)\cap {\mathcal{W}}^{2(1-\theta )}\left(H\right)$, there is a constant $C_3>0$ such that

$$\mathbb{E}\left({∥\widehat{\mathbf{m}}(·,\rho )-\mathbf{m}(·;\rho )∥}^2\right)\le C{\left(\frac{lnT}{T}\right)}^{\theta },$$

(25)

for T large enough.

Assume that $\mathbf{m}(·,\rho )$ and $f(·)$ fulfill (M.1) and, for any $\theta \in (0,1),$

$$\mathbf{m}(·,\rho )\in {\mathcal{B}}_{\infty }^{\theta /2}\left(\mathbf{H}\right)\cap {\mathcal{W}}^{2(1-\theta )}\left(\mathbf{H}\right),$$

where ${\mathcal{B}}_{\infty }^{\theta /2}\left(\mathbf{H}\right)$ is in (Definition 1) with $s=\theta /2$ and ${\mathcal{W}}^{2(1-\theta )}\left(\mathbf{H}\right)$ is in (Definition 2) with $r=2(1-\theta )$, then there is a constant $C_3>0$ in such a way that

$$\mathbb{E}\left({∥\widehat{\mathbf{m}}(·,\rho )-\mathbf{m}(·;\rho )∥}^2\right)\le C_3{\left(\frac{{(lnT)}^2}{T}\right)}^{\theta }$$

for T large enough. Again, note that for $s>0$, if

$$\mathbf{m}(·,\rho )\in {\mathcal{B}}_{\infty }^{s/(2s+1)}\left(\mathbf{H}\right)\cap {\mathcal{W}}^{2/(2s+1)}\left(\mathbf{H}\right),$$

then there is a constant $C_4>0$ in such a way that

$$\mathbb{E}\left({∥\widehat{\mathbf{m}}(·,\rho )-\mathbf{m}(·;\rho )∥}^2\right)\le C_4{\left(\frac{{(lnT)}^2}{T}\right)}^{2s/(2s+1)}.$$

This convergence rate is close to optimal in the “standard” minimax framework (see, [27]) up to an extra logarithmic term. According to our knowledge, Theorem 2 is the first one to examine an adaptive wavelet-based estimator for functional data in the context of nonparametric regression for an ergodic process. Ref. [49] studied the same estimator in a discrete time setting.

From the fact that the coefficients defined in Equations (22) and (23) depend on the unknown function $f(·)$, it is possible to use

$$\begin{array}{ccc}\hfill {\tilde{\eta }}_{k,\ell }& =& \frac{1}{T}{\int }_0^T\frac{\rho \left(Y_t\right)}{\widehat{f}\left({\mathbf{X}}_t\right)}{\varphi }_k\left({\mathbf{X}}_t;{\zeta }_{k,\ell }\right)dt,\hfill \end{array}$$

(26)

$$\begin{array}{ccc}\hfill {\tilde{\theta }}_{k,\ell }& =& \frac{1}{T}{\int }_0^T\frac{\rho \left(Y_t\right)}{\widehat{f}\left({\mathbf{X}}_t\right)}{\psi }_k\left({\mathbf{X}}_t;{\eta }_{k,\ell }\right)dt.\hfill \end{array}$$

(27)

The wavelet hard thresholding estimator is defined by $\tilde{\mathbf{m}}(·,\rho )$, for all $\mathbf{x}\in \mathbf{S}$, by

$$\tilde{\mathbf{m}}(\mathbf{x},\rho )=\sum _{\ell \in {\mathcal{I}}_0}{\tilde{\eta }}_{0,\ell }{\varphi }_0\left(\mathbf{x};{\zeta }_{0,\ell }\right)+\sum _{k=0}^{m_T}\sum _{\ell \in {\mathcal{J}}_k}{\tilde{\theta }}_{k,\ell }{\mathrm{𝟙}}_{\left\{\right|{\tilde{\theta }}_{k,\ell }|\ge \kappa \sqrt{\frac{lnT}{T}}\}}{\psi }_k\left(\mathbf{x};{\eta }_{k;\ell }\right).$$

(28)

Keep in mind the following elementary observation

$$\frac{1}{\widehat{f}(·)}=\frac{1}{f(·)}+\frac{(f(·)-\widehat{f}(·))}{f(·)\widehat{f}(·)}.$$

We can infer, from the last equation, the following

$$\begin{array}{ccc}\hfill {\tilde{\eta }}_{k,\ell }& =& {\widehat{\eta }}_{k,\ell }+\frac{1}{T}{\int }_0^T\frac{(f\left({\mathbf{X}}_t\right)-\widehat{f}\left({\mathbf{X}}_t\right))}{f\left({\mathbf{X}}_t\right)\widehat{f}\left({\mathbf{X}}_t\right)}\rho \left(Y_t\right){\varphi }_k\left({\mathbf{X}}_t;{\zeta }_{k,\ell }\right)dt,\hfill \\ \hfill {\tilde{\theta }}_{k,\ell }& =& {\widehat{\theta }}_{k,\ell }+\frac{1}{T}{\int }_0^T\frac{(f\left({\mathbf{X}}_t\right)-\widehat{f}\left({\mathbf{X}}_t\right))}{f\left({\mathbf{X}}_t\right)\widehat{f}\left({\mathbf{X}}_t\right)}\rho \left(Y_t\right){\psi }_k\left({\mathbf{X}}_t;{\eta }_{k,\ell }\right)dt.\hfill \end{array}$$

Notice that

$$\begin{array}{ccc}\hfill {({\tilde{\eta }}_{k,\ell }-{\widehat{\eta }}_{k,\ell })}^2& =& {\left\{\frac{1}{T}{\int }_0^T\frac{(f\left({\mathbf{X}}_t\right)-\widehat{f}\left({\mathbf{X}}_t\right))}{f\left({\mathbf{X}}_t\right)\widehat{f}\left({\mathbf{X}}_t\right)}\rho \left(Y_t\right){\varphi }_k\left({\mathbf{X}}_t;{\zeta }_{k,\ell }\right)dt\right\}}^2\hfill \\ & \le & \frac{1}{T}{\int }_0^T{\left\{\frac{(f\left({\mathbf{X}}_t\right)-\widehat{f}\left({\mathbf{X}}_t\right))}{f\left({\mathbf{X}}_t\right)\widehat{f}\left({\mathbf{X}}_t\right)}\right\}}^{2}dt\hfill \\ & & \times \frac{1}{T}{\int }_0^T{\left\{\rho \left(Y_t\right){\varphi }_k\left({\mathbf{X}}_t;{\zeta }_{k,\ell }\right)\right\}}^{2}dt.\hfill \end{array}$$

This gives

$$\begin{array}{ccc}\hfill \mathbb{E}{({\tilde{\eta }}_{k,\ell }-{\widehat{\eta }}_{k,\ell })}^2& \le & \mathbb{E}\left\{\frac{1}{T}{\int }_0^T{\left\{\frac{(f\left({\mathbf{X}}_t\right)-\widehat{f}\left({\mathbf{X}}_t\right))}{f\left({\mathbf{X}}_t\right)\widehat{f}\left({\mathbf{X}}_t\right)}\right\}}^{2}dt\right.\hfill \\ & & \left.\times \frac{1}{T}{\int }_0^T\mathbb{E}(\rho {\left(Y_t\right)}^2\mid {\mathbf{X}}_t){\varphi }_k{\left({\mathbf{X}}_t;{\zeta }_{k,\ell }\right)}^{2}dt\right\}.\hfill \end{array}$$

Under the conditions of Theorems 1 and 2, one can find a positive $\mathfrak{C}$ constant such that

$$\begin{array}{ccc}\hfill \mathbb{E}{({\tilde{\eta }}_{k,\ell }-{\widehat{\eta }}_{k,\ell })}^2& \le & \mathfrak{C}{\left(\frac{lnT}{T}\right)}^{\theta }.\hfill \end{array}$$

A combination of Theorem 1 with Theorem 2 gives the following corollary.

 Corollary 1.

Under the hypotheses of Theorems 1 and 2, there exists a constant $C_5>0$ in such a way that

$$\mathbb{E}\left({∥\tilde{\mathbf{m}}(\mathbf{x};\rho )-\mathbf{m}(\mathbf{x};\rho )∥}^2\right)\le C_5{\left(\frac{lnT}{T}\right)}^{\theta },$$

(29)

for T large enough.

 Remark 2. 

Let $\left\{{\mathbf{X}}_n,n\in \mathbb{Z}\right\}$ be a stationary sequence. Consider the backward field ${\mathcal{A}}_n=\sigma \left({\mathbf{X}}_k:\right.$$\left.k\le n\right)$ and the forward field ${\mathcal{B}}_n=\sigma \left({\mathbf{X}}_k:k\ge n\right)$. The sequence is strongly mixing if

$$\underset{A\in {\mathcal{A}}_0,B\in {\mathcal{B}}_n}{sup}|\mathbb{P}(A\cap B)-\mathbb{P}\left(A\right)\mathbb{P}\left(B\right)|=\alpha \left(n\right)\to 0\mathrm{as}n\to \infty .$$

The sequence is ergodic if

$$\underset{n\to \infty }{lim}\frac{1}{n}\sum _{k=0}^{n-1}\left|\mathbb{P}\left(A\cap {\tau }^{-k}B\right)-\mathbb{P}\left(A\right)\mathbb{P}\left(B\right)\right|=0,$$

where τ is the shift transformation or time-evolution. The labeling of strong mixing in the preceding definition is more rigorous than what is commonly referred to as strong mixing (when using the terminology of measure-preserving dynamical systems), meaning that it satisfies the following conditions

$$\underset{n\to \infty }{lim}\mathbb{P}\left(A\cap {\tau }^{-n}B\right)=\mathbb{P}\left(A\right)\mathbb{P}\left(B\right)$$

for any two measurable sets $A,B$, see, for instance, [63]. Therefore, substantial mixing requires ergodicity, although the opposite is not necessarily true (see, for instance, Remark 2.6 on page 50 about Proposition 2.8 on page 51 in [64]).

 Remark 3.

Ref. [43] provided an example of an ergodic but non-mixing process in their discussion, which can be summarized as follows: Let $(T_i,{\lambda }_i):i\in \mathbb{Z}$ be a strictly stationary process such that $T_i\mid {\mathcal{T}}_{i-1}$ is a Poisson process with parameter $\lambda i$, where ${\mathcal{T}}_i$ is the $sigma$-field generated by $(T_i,{\lambda }_i,T_{i-1},\dots )$. Assume

$${\lambda }_i=f({\lambda }_{i-1},T_{i-1}),$$

and $f:[0,\infty ]\times \mathbb{N}\to (0,\infty )$ is a given function. This process is not mixing in general (see Remark 3 of [65]). It is known that any sequence ${\left({\epsilon }_i\right)}_{i\in \mathbb{Z}}$ of independent and identically distributed random variables is ergodic. Hence, according to Proposition 2.10 in [64], it is easy to see that ${\left(Y_i\right)}_{i\in \mathbb{Z}}$ with

$$Y_i=\vartheta ((\dots ,{\epsilon }_{i-1},{\epsilon }_i),({\epsilon }_{i+1},{\epsilon }_{i+2},\dots )),$$

for some Borel-measurable function $\vartheta (·)$. Ref. [66] has constructed an example of a non-mixing ergodic continuous-time process. It is well known that the fractional Brownian motion $\{W_{\mathrm{t}}^H:t\ge 0\}$ with parameter $H\in (0,1)$ has strictly stationary increments. Otherwise, the fractional Gaussian noise, defined for every $s>0$ by

$$\{G_{\mathrm{t}}^H:t\ge 0\}:=\{W_{\mathrm{t}+s}^H-W_{\mathrm{t}}^H:t\ge 0\},$$

is a strictly stationary-centered, long memory process when $H\in (\frac{1}{2},1)$ (for instance, see [67] p. 55, [68] p. 17), hence the condition of strong mixing is not satisfied. Let $\{G_{\mathrm{t}}:t\ge 0\}$ be a strictly stationary-centered Gaussian process with correlation function

$$R\left(t\right)=\mathbb{E}\left[G_0{G}_{\mathrm{t}}\right].$$

Relying on the work of [69], Lemma 4.2, it follows that the process $\{G_{\mathrm{t}}:t\ge 0\}$ is ergodic whenever

$$\underset{\mathrm{t}\to \infty }{lim}R\left(t\right)=0,$$

which is the case for the process $\{G_{\mathrm{t}}^H:t\ge 0\}$.

 Remark 4.

In continuous time, sampling is frequently used to obtain data. Several discretization strategies, including deterministic and random sampling, have been proposed in the literature. The interested reader is referred to [70,71,72,73,74]. To simplify the concept, we consider the density estimator of $f(·)$ based on $\{{\mathbf{X}}_t:t\in [0,T]\}$ and its sampled discrete sequence $\{\mathbf{X}\left(t_k\right):k=1,\dots ,n\}$. The estimator of the sampled density $f(·)$ is

$$f_n\left(\mathbf{x}\right)=\frac{1}{n{h}_n^d}\sum _{i=1}^{n}K\left(\frac{\mathbf{x}-{\mathbf{X}}_{t_j}}{h_n}\right).$$

As stated in [70], we can only recollect two designs: irregular sampling and random sampling.

• Deterministic sampling.Consider the situation where ${\left(t_k\right)}_{1\le k\le n}$ is deterministically irregularly spaced with

  $$\underset{1\le k\le n}{inf}|t_{j+1}-t_j|=\frac{1}{\tau },$$

  for some $\tau >0$. ${\mathcal{G}}_k:=\sigma \left(X\left(t_k\right)\right)$ the σ-field generated by $\{{\mathbf{X}}_s:0\le s\le t_k\}$. Clearly, ${\left({\mathcal{G}}_k\right)}_{1\le k\le n}$ in a family of increasing σ-fields.
• Random sampling. Assume that the instants ${\left(t_k\right)}_{1\le k\le n}$ in the interval $[0,T]$ form a sequence of uniform random variables independent of the process $\{{\mathbf{X}}_t:t\in [0,T]\}$. Define

  $$0\le {\tau }_1<\cdots <{\tau }_n\le T,$$

  as the corresponding order statistics. Observe that ${\left({\tau }_k\right)}_{1\le k\le n}$ are the observation points for the procedure. Clearly, all of the distances between these sites are positive. As a consequence, taking ${\mathcal{G}}_k:=\sigma \left(X\left(t_k\right)\right)$ the σ-field generated by $\{{\mathbf{X}}_s:0\le s\le {\tau }_k\}$, it follows that ${\left({\mathcal{G}}_k\right)}_{1\le k\le n}$ is a sequence of increasing σ-fields.

As established in [75], the penalization approach for the choice of the mesh δ of the observations provides an optimal rate of convergence; we leave this subject open for future investigation within the context of ergodic processes.

 Remark 5.

In a previous publication [36], we tackled the nonparametric estimation of the density and regression function in a finite-dimensional setting using an orthonormal wavelet basis. Our findings differ significantly from those presented in this publication. In [36], we demonstrated the strong uniform consistency characteristics of these estimators over compact subsets of ${\mathbb{R}}^d$ under a general ergodic condition on the underlying processes. In addition, we demonstrate the asymptotic normality of wavelet-based estimators. The Burkholder–Rosenthal inequality, a more complex technique than the exponential inequality used in the previous publication, was the central concept of this study. Significantly, the current study studies the mean integrated square error over compact subsets, which is fundamentally different from the conclusions of the prior publication.

5. Applications

5.1. The Conditional Distribution

Our findings can be utilized to examine the conditional distribution $F(\mathbf{y}\mid \mathbf{x})$ for $\mathbf{y}\in {\mathbb{R}}^d$. To be more precise, let $\rho \left(\mathbf{y}\right)=\mathrm{𝟙}\{\mathbf{y}\le \mathbf{z}\}$. The wavelet’s hard thresholding estimator of $F(\mathbf{y}\mid \mathbf{x})$, is defined, for all $\mathbf{x}\in \mathbf{S}$, by

$$\widehat{F}(\mathbf{y}\mid \mathbf{x})=\sum _{\ell \in {\mathcal{I}}_0}{\breve{\eta }}_{0,\ell }{\varphi }_0\left(\mathbf{x};{\zeta }_{0,\ell }\right)+\sum _{k=0}^{m_T}\sum _{\ell \in {\mathcal{J}}_k}{\breve{\theta }}_{k,\ell }{\mathrm{𝟙}}_{{\displaystyle \left\{|{\breve{\theta }}_{k,\ell }|\ge \kappa \sqrt{\frac{lnT}{T}}\right\}}}{\psi }_k\left(\mathbf{x};{\eta }_{k;\ell }\right),$$

(30)

where

$$\begin{array}{ccc}\hfill {\breve{\eta }}_{k,\ell }& =& \frac{1}{T}{\int }_0^T\frac{\mathrm{𝟙}\{{\mathbf{Y}}_t\le \mathbf{y}\}}{f\left({\mathbf{X}}_t\right)}{\varphi }_k\left({\mathbf{X}}_t;{\zeta }_{k,\ell }\right)dt,\hfill \end{array}$$

(31)

$$\begin{array}{ccc}\hfill {\breve{\theta }}_{k,\ell }& =& \frac{1}{T}{\int }_0^T\frac{\mathrm{𝟙}\{{\mathbf{Y}}_t\le \mathbf{y}\}}{f\left({\mathbf{X}}_t\right)}{\psi }_k\left({\mathbf{X}}_t;{\eta }_{k,\ell }\right)dt.\hfill \end{array}$$

(32)

A direct consequence of Theorem 2 is

$$\mathbb{E}\left({∥\widehat{F}(\mathbf{y}\mid \mathbf{x})-F(\mathbf{y}\mid \mathbf{x})∥}^2\right)\le C{\left(\frac{lnT}{T}\right)}^{\theta }.$$

(33)

5.2. The Conditional Quantile

Remark that whenever $F(·\mid \mathbf{x})$ is strictly increasing and continuous in a neighborhood of $q_{\alpha }\left(\mathbf{x}\right)$, the function $F(·\mid \mathbf{x})$ has a unique quantile of order $\alpha$ at a point $q_{\alpha }\left(\mathbf{x}\right)$, which is $F\left(q_{\alpha }\left(\mathbf{x}\right)\mid \mathbf{x}\right)=\alpha$. In this situation

$$q_{\alpha }\left(\mathbf{x}\right)=F^{-1}(\alpha \mid \mathbf{x})=inf\{y\in \mathbb{R}:F(y\mid \mathbf{x})\ge \alpha \},$$

which can be estimated by ${\widehat{q}}_{T,\alpha }\left(x\right)={\widehat{F}}^{-1}(\alpha \mid \mathbf{x})$. Consequently, the fact

$$F\left(q_{\alpha }\left(\mathbf{x}\right)\mid x\right)=\alpha =\widehat{F}\left({\widehat{q}}_{T,\alpha }\left(\mathbf{x}\right)\mid \mathbf{x}\right)$$

and $\widehat{F}(·\mid \mathbf{x})$ is continuous and strictly increasing, we then have

$$\forall ϵ>0,\exists \eta \left(ϵ\right)>0,\forall y,\left|\widehat{F}(y\mid \mathbf{x})-\widehat{F}\left(q_{\alpha }\left(\mathbf{x}\right)\mid \mathbf{x}\right)\right|\le \eta \left(ϵ\right)\Rightarrow \left|y-q_{\alpha }\left(\mathbf{x}\right)\right|\le ϵ,$$

implying that, $\forall ϵ>0,\exists \eta \left(ϵ\right)>0$

$$\begin{array}{c}\mathbb{P}\left(\left|{\widehat{q}}_{T,\alpha }\left(x\right)-q_{\alpha }\left(\mathbf{x}\right)\right|\ge \eta \left(ϵ\right)\right)\hfill \\ \le \mathbb{P}\left(\left|\widehat{F}\left({\widehat{q}}_{T,\alpha }\left(x\right)\mid x\right)-\widehat{F}\left(q_{\alpha }\left(\mathbf{x}\right)\mid x\right)\right|\ge \eta \left(ϵ\right)\right)\hfill \\ =\mathbb{P}\left(\mid F\left(q_{\alpha }\left(\mathbf{x}\right)\mid x\right)-\widehat{F}\left(q_{\alpha }\left(\mathbf{x}\right)\mid \mathbf{x}\right)\ge \eta \left(ϵ\right)\right).\hfill \end{array}$$

In addition, suppose that, for fixed ${\mathbf{x}}_0\in \mathcal{C}\subset \mathbf{S},F\left(y\mid {\mathbf{x}}_0\right)$ is differentiable at $q_{\alpha }\left({\mathbf{x}}_0\right)$ with

$${\left.\frac{\partial }{\partial y}F\left(y\mid x_0\right)\right|}_{y=q_{\alpha }\left({\mathbf{x}}_0\right)}:=g\left(q_{\alpha }\left(x_0\right)\mid x_0\right)>\nu >0,$$

where $\nu$ is a real number, and $g(·\mid \mathbf{x})$ is uniformly continuous for all $\mathbf{x}\in \mathcal{C}$. From Taylor’s expansion of the function $F\left({\widehat{q}}_{T,\alpha }\left(x\right)\mid x\right)$ in the neighborhood of $q_{\alpha }\left(\mathbf{x}\right)$, we have

$$F\left({\widehat{q}}_{T,\alpha }\left(\mathbf{x}\right)\mid x\right)-F\left(q_{\alpha }\left(\mathbf{x}\right)\mid \mathbf{x}\right)=\left({\widehat{q}}_{T,\alpha }\left(x\right)-q_{\alpha }\left(\mathbf{x}\right)\right)g\left(q_{T,\alpha }^{*}\left(\mathbf{x}\right)\mid \mathbf{x}\right)$$

where $q_{T,\alpha }^{*}\left(\mathbf{x}\right)$ is a point between $q_{\alpha }\left(\mathbf{x}\right)$ and ${\widehat{q}}_{T,\alpha }\left(\mathbf{x}\right)$. Using the fact that ${\widehat{q}}_{T,\alpha }\left(\mathbf{x}\right)$ converges a.s. toward $q_{\alpha }\left(\mathbf{x}\right)$ as T goes to infinity, combined with the uniform continuity of $g(·\mid \mathbf{x})$, allows us to write that

$$\underset{\mathbf{x}\in \mathcal{C}}{sup}\left|{\widehat{q}}_{T,\alpha }\left(\mathbf{x}\right)-q_{\alpha }\left(\mathbf{x}\right)\right|\underset{\mathbf{x}\in \mathcal{C}}{sup}\left|g\left(q_{\alpha }\left(\mathbf{x}\right)\mid \mathbf{x}\right)\right|=O_{a.s.}\left(\underset{y\in S}{sup}\underset{\mathbf{x}\in \mathcal{C}}{sup}\left|{\widehat{F}}_T(y\mid \mathbf{x})-F(y\mid \mathbf{x})\right|\right).$$

By the fact that $g\left(q_{\alpha }\left(\mathbf{x}\right)\mid \mathbf{x}\right)$ is uniformly bounded from below, we can then claim that

$$\mathbb{E}\left({∥{\widehat{q}}_{T,\alpha }\left(\mathbf{x}\right)-q_{\alpha }\left(\mathbf{x}\right)∥}^2\right)\le C{\left(\frac{lnT}{T}\right)}^{\theta }.$$

(34)

 Remark 6.

(Expectile regression). For $p\in (0,1)$, the choice given by $\psi (T-\theta )=(p-\mathrm{𝟙}\{T-\theta \le 0\left\}\right)|T-\theta |$ leads to quantities called expectiles by [76]. Expectiles, as defined by [76], may be introduced either as a generalization of the mean or as an alternative to quantiles. Indeed, classical regression provides us with a high sensitivity to extreme values, allowing for more reactive risk management. Quantile regression, on the other hand, provides the ability to acquire exhaustive information on the effect of the explanatory variable on the response variable by examining its conditional distribution; refer to [77,78,79] for further details on expectiles in functional data settings.

 Remark 7.

(Conditional winsorized mean). As in [80], if we consider $\psi (T-\theta )=-k,T-\theta ,k$ if $T-\theta <-k$, $|T-\theta |\le k$, or $T-\theta >k$, then $\mathbf{m}(\mathbf{x};\psi )$ will be the conditional winsorized mean. Notably, this parameter was not considered in the literature on nonparametric functional data analysis involving wavelet estimators. Our paper offers asymptotic results for the conditional winsorized mean when the covariates are functions.

5.3. Shannon’s Entropy

The differential (or Shannon) entropy of $f(·)$ is defined to be

$$\begin{array}{ccc}\hfill H\left(f\right)& =& -{\int }_{\mathbf{S}}{f}_{\mathbf{X}}\left(\mathbf{x}\right)log\left(f_{\mathbf{X}}\left(\mathbf{x}\right)\right)\nu \left(d\mathbf{x}\right),\hfill \end{array}$$

(35)

whenever this integral is meaningful. We will apply the convention $0log\left(0\right)=0$ since $ulog\left(u\right)\to 0$ as $u\to 0$. The notion of differential entropy was originally introduced in [81]. Since this epoch, the concept of entropy has been the topic of extensive theoretical and applied research. We are referencing [82] (Chapter 8) for a thorough overview of differential entropy and its applications and mathematical characteristics. Entropy concepts and principles play a fundamental role in many applications, such as quantization theory [83], statistical decision theory [84], and contingency table analysis [85]. Ref. [86] introduced the concept of convergence in entropy and showed that the latter convergence concept implies convergence in ${\mathcal{L}}_1$. This property indicates that entropy is a useful concept for measuring “closeness in distribution”, and also heuristically justifies the usage of sample entropy as a test statistic when designing entropy-based tests of goodness-of-fit. This line of research has been pursued by [87,88,89] (including the references therein). The idea here is that many families of distributions are characterized by maximization of entropy subject to constraints (see [90,91]). Given ${\widehat{f}}_T(·)$ in Equation (13), we estimate $H\left(f\right)$ using representation (35), by setting

$$\begin{array}{c}\hfill H_T\left(f\right)=-{\int }_{{\mathbf{A}}_n}{\widehat{f}}_T\left(\mathbf{x}\right)log\left({\widehat{f}}_T\left(\mathbf{x}\right)\right)\nu \left(d\mathbf{x}\right),\end{array}$$

(36)

where

$${\mathbf{A}}_n=\{\mathbf{x}\in \mathbf{S}:{\widehat{f}}_T\left(\mathbf{x}\right)\ge {\gamma }_n\},$$

and ${\gamma }_n\downarrow 0$ is a sequence of positive constants. To prove the strong consistency of $H_T\left(f\right)$, we shall consider another, but more appropriate and more computationally convenient, centering factor than the expectation $\mathbb{E}{H}_T\left(f\right)$, which is delicate to handle. This is given by

$$\begin{array}{c}\hfill \widehat{\mathbb{E}}{H}_T\left(f\right)=-{\int }_{{\mathbf{A}}_n}\mathbb{E}{\widehat{f}}_T\left(\mathbf{x}\right)log\left(\mathbb{E}{\widehat{f}}_T\left(\mathbf{x}\right)\right)\nu \left(d\mathbf{x}\right).\end{array}$$

We first decompose $H_T\left(f\right)-\widehat{\mathbb{E}}{H}_T\left(f\right)$, as in [92,93,94,95], into the sum of two components, by writing

$$\begin{array}{c}{\displaystyle H_T\left(f\right)-\widehat{\mathbb{E}}{H}_T\left(f\right)}\hfill \\ =-{\int }_{{\mathbf{A}}_n}{\widehat{f}}_T\left(\mathbf{x}\right)log\left({\widehat{f}}_T\left(\mathbf{x}\right)\right)\nu \left(d\mathbf{x}\right)\hfill \\ +{\int }_{{\mathbf{A}}_n}\mathbb{E}{\widehat{f}}_T\left(\mathbf{x}\right)log\left(\mathbb{E}{\widehat{f}}_T\left(\mathbf{x}\right)\right)\nu \left(d\mathbf{x}\right)\hfill \\ =-{\int }_{{\mathbf{A}}_n}\left\{log{\widehat{f}}_T\left(\mathbf{x}\right)-log\mathbb{E}{\widehat{f}}_T\left(\mathbf{x}\right)\right\}\mathbb{E}{\widehat{f}}_T\left(\mathbf{x}\right)\nu \left(d\mathbf{x}\right)\hfill \\ -{\int }_{{\mathbf{A}}_n}\left\{{\widehat{f}}_T\left(\mathbf{x}\right)-\mathbb{E}{\widehat{f}}_T\left(\mathbf{x}\right)\right\}log{\widehat{f}}_T\left(\mathbf{x}\right)\nu \left(d\mathbf{x}\right).\hfill \end{array}$$

(37)

We remark that for all $z>0$,

$$\left|logz\right|\le \left|\frac{1}{z}-1\right|+\left|z-1\right|.$$

Thus, for any $\mathbf{x}\in {\mathbf{A}}_n$, we obtain

$$\begin{array}{c}{\displaystyle |log{\widehat{f}}_T\left(\mathbf{x}\right)-log\mathbb{E}{\widehat{f}}_T\left(\mathbf{x}\right)|=\left|log\frac{{\widehat{f}}_T\left(\mathbf{x}\right)}{\mathbb{E}{\widehat{f}}_T\left(\mathbf{x}\right)}\right|}\hfill \\ \le \left|\frac{\mathbb{E}{\widehat{f}}_T\left(\mathbf{x}\right)}{{\widehat{f}}_T\left(\mathbf{x}\right)}-1\right|+\left|\frac{{\widehat{f}}_T\left(\mathbf{x}\right)}{\mathbb{E}{\widehat{f}}_T\left(\mathbf{x}\right)}-1\right|\hfill \\ =\frac{\left|\mathbb{E}{\widehat{f}}_T\left(\mathbf{x}\right)-{\widehat{f}}_T\left(\mathbf{x}\right)\right|}{{\widehat{f}}_T\left(\mathbf{x}\right)}+\frac{\left|{\widehat{f}}_T\left(\mathbf{x}\right)-\mathbb{E}{\widehat{f}}_T\left(\mathbf{x}\right)\right|}{\mathbb{E}{\widehat{f}}_T\left(\mathbf{x}\right)}.\hfill \end{array}$$

Making use of Theorem 1, gives

$$\mathbb{E}\left({(H_T\left(f\right)-\widehat{\mathbb{E}}{H}_T\left(f\right))}^2\right)\le C{\left(\frac{lnT}{T}\right)}^{\theta },$$

(38)

for positive constant C. Note that this is the first estimation result for the entropy of functional continuous time series processes. A similar idea can be used to estimate other density functions, such as the extropy; refer to [96] for a definition.

5.4. The Curve Discrimination

The curve discrimination problem can be stated as follows. Let ${\left\{{\mathbf{X}}_t\right\}}_{0\le t\le T}$ denote a sample of curves, and each of them is known to pertain to one among $\mathbf{G}$ groups $\iota =1,\dots ,\mathbf{G}$. We denote the group of the curve ${\mathbf{X}}_t$ by ${\mathcal{T}}_t$. Suppose that each pair of variables $\left({\mathbf{X}}_t,{\mathcal{T}}_t\right)$ has the same distribution as the pair $(\mathbf{X},\mathcal{T})$. Given a new curve $\mathbf{x}$, the question is to determine its class membership. To do so, we will estimate the conditional probability for every $\iota =1,\dots ,\mathbf{G}$, as follows:

$$p_{\iota }\left(\mathbf{x}\right)=\mathbb{P}(\mathcal{T}=\iota \mid \mathbf{X}=\mathbf{x}).$$

Following the proposal made in [97,98] permitting the estimation of these probabilities by

$${\widehat{p}}_{\iota }\left(\mathbf{x}\right)=\sum _{\ell \in {\mathcal{I}}_0}{\breve{\stackrel{˘}{\eta }}}_{0,\ell }{\varphi }_0\left(\mathbf{x};{\zeta }_{0,\ell }\right)+\sum _{k=0}^{m_T}\sum _{\ell \in {\mathcal{J}}_k}{\breve{\stackrel{˘}{\theta }}}_{k,\ell }{\mathrm{𝟙}}_{{\displaystyle \left\{|{\breve{\stackrel{˘}{\theta }}}_{k,\ell }|\ge \kappa \sqrt{\frac{lnT}{T}}\right\}}}{\psi }_k\left(\mathbf{x};{\eta }_{k;\ell }\right),$$

(39)

where

$$\begin{array}{ccc}\hfill {\breve{\stackrel{˘}{\eta }}}_{k,\ell }& =& \frac{1}{T}{\int }_0^T\frac{\mathrm{𝟙}\left\{T_t{=}_{\iota }\right\}}{f\left({\mathbf{X}}_t\right)}{\varphi }_k\left({\mathbf{X}}_t;{\zeta }_{k,\ell }\right)dt,\hfill \end{array}$$

(40)

$$\begin{array}{ccc}\hfill {\breve{\stackrel{˘}{\theta }}}_{k,\ell }& =& \frac{1}{T}{\int }_0^T\frac{\mathrm{𝟙}\left\{T_t{=}_{\iota }\right\}}{f\left({\mathbf{X}}_t\right)}{\psi }_k\left({\mathbf{X}}_t;{\eta }_{k,\ell }\right)dt.\hfill \end{array}$$

(41)

As remarked by [97,98], for each $\iota$ we make use of the notation

$$Y=\left\{\begin{array}{cc}1\hfill & \mathrm{if}\mathcal{T}=\iota \hfill \\ 0\hfill & \mathrm{otherwise},\hfill \end{array}\right.$$

then we can write

$$p_{\iota }\left(\mathbf{x}\right)=\mathbb{E}(Y\mid \mathbf{X}=\mathbf{x}).$$

An application of Theorem 2 is

$$\mathbb{E}\left({∥{\widehat{p}}_{\iota }\left(\mathbf{x}\right)-p_{\iota }(\mathbf{x}∥}^2\right)\le C{\left(\frac{lnT}{T}\right)}^{\theta }.$$

6. Concluding Remarks

In this study, we investigated the nonparametric estimation of density and regression function using continuous functional stationary processes and wavelet bases for Hilbert spaces of functions. We described the mean square error integrated over compact subsets. The martingale method was used to determine the asymptotic properties of these estimators, which differs significantly from the mixing and independent settings. Ergodicity is the assumption of the process’s dependence. Extending nonparametric functional concepts to local stationary processes is a relatively new area of research. It would be interesting to extend our work to the case of the functional local stationary process, but this would require nontrivial mathematics and is well outside the scope of this paper. The exact logarithm rates of convergence depend on the smoothness parameters $\theta$ of functions $f(·)$ and $\mathbf{m}(·,\rho )$ defined in the space $\mathbf{H}$. These results are typical of all nonparametric estimations and are consistent with the vast majority of academic references. Although $\theta$ is unknown, the smoothness metrics assumed here are more nuanced than the differentiability criterion of integer orders required for convolution kernel methods. To make the estimator more practical, techniques have been developed to select the optimal adaptive value of $\tau$, such that the most commonly used methods of “Stein,” the “rule of thumb,” and “cross-validation”, we refer to [99] and [27] for details on an asymptotically optimal empirical bandwidth selection rule. It would be worthwhile to investigate this subject in the future.

7. Proofs

This section is dedicated to proving our findings. The previously presented notations are used again in the sequel. In this research, we need an upper bound inequality for partial sums of unbounded martingale differences to derive the asymptotic results for the estimations of the density and regression functions based on strictly stationary and ergodic functional data. Here and in the following, “C” denotes a positive constant that may vary from line to line. The following lemmas express this inequality.

 Lemma 1.

(Burkholder–Rosenthal inequality) Following Notation1in [100].

Let ${\left(X_i\right)}_{i\ge 1}$ be a stationary martingale adapted to the filtration ${\left({\mathcal{F}}_i\right)}_{i\ge 1}$, define ${\left(d_i\right)}_{i\ge 1}$ as the sequence of martingale differences adapted to ${\left({\mathcal{F}}_i\right)}_{i\ge 1}$ and

$$S_n=\sum _{i=1}^n{d}_i,$$

then for any positive integer n,

$$\parallel \underset{1\le j\le n}{max}|S_j{\left|\right)\parallel }_p\ll n^{1/p}{\parallel d_1\parallel }_p+{∥\sum _{k=1}^n\mathbb{E}(d_k^2/{\mathcal{F}}_{k-1})∥}_{p/2}^{1/2},for anyp\ge 2;$$

(42)

where, as usual, the norm ${\parallel ·\parallel }_p={\left(\mathbb{E}\left[\right|·{|}^p]\right)}^{1/p}$.

 Lemma 2

([101]). Let $\{Z_i,i\ge 1\}$ denote a sequence of martingale differences in such a way that

$$|Z_i|\le B,a.s.,$$

then, for all $ϵ>0$ and all n large enough, we obtain

$$\mathbb{P}\left\{\left|\sum _{i=1}^n{Z}_i\right|>ϵ\right\}\le 2exp\left\{-\frac{{ϵ}^2}{2n{B}^2}\right\}.$$

The subsequent lemmas characterize the asymptotic behavior of estimators ${\widehat{\alpha }}_{k,\ell }$ and ${\widehat{\beta }}_{k,\ell }$.

 Lemma 3.

For each $k\in \{0,\dots ,m_T\}$ and each $\ell \in {\mathcal{I}}_k$, under conditions (C.0), (C.1), (F.1), and (E.1)(i), there is a constant $C>0$ in such a way that

$$\mathbb{E}\left({\left|{\widehat{\alpha }}_{k,\ell }-{\alpha }_{k,\ell }\right|}^2\right)\le C\left(\frac{lnT}{T}\right).$$

(43)

 Lemma 4.

For each $k\in \{0,\dots ,m_T\}$ and each $\ell \in {\mathcal{J}}_k$, and under conditions (C.0), (C.1), (F.1), (E.1), and (E.2), and condition (16), there is a constant $C>0$ in such a way that

$$\mathbb{E}\left({\left|{\widehat{\beta }}_{k,\ell }-{\beta }_{k,\ell }\right|}^4\right)=C{\left(\frac{lnT}{T}\right)}^2,a.s.$$

(44)

 Lemma 5.

For each $k\in \{0,\dots ,m_T\}$ and each $\ell \in {\mathcal{J}}_k$, for $\kappa >0$ sufficiently large and under conditions (C.0), (C.1), (F.1), (E.1), and (E.2), and assumption (16) there is a constant $C>0$ in such a way that

$$\mathbb{P}\left(\left|{\widehat{\beta }}_{k,\ell }-{\beta }_{k,\ell }\right|\ge \frac{\kappa }{2}\sqrt{\frac{lnT}{T}}\right)\le C{\left(\frac{lnT}{T}\right)}^2.$$

(45)

 Proof of Theorem 1. 

Keep in mind that the proof of Theorem 1 is a straightforward application is a direct application of [52] (Theorem 3.1), using Lemmas 3–5 alongside $c\left(n\right)={(lnT/T)}^{1/2}$, ${\sigma }_i=1$, $r=2$. We modified and expanded the approach used to prove Theorem 3.1 of [34], and to include the stationary ergodic process. □

 Proof of Theorem 3. 

Consider the subsequent decomposition

$$\begin{array}{ccc}\hfill {\widehat{\alpha }}_{k,\ell }-{\alpha }_{k,\ell }& =& {\widehat{\alpha }}_{k,\ell }-{\tilde{\alpha }}_{k,\ell }+{\tilde{\alpha }}_{k,\ell }-{\alpha }_{k,\ell }\hfill \\ & =& A_{k,\ell ,1}+A_{k,\ell ,2},\hfill \end{array}$$

(46)

where

$${\tilde{\alpha }}_{k,\ell }=\frac{1}{T}{\int }_0^T\mathbb{E}\left[{\varphi }_k\left({\mathbf{x}}_t;{\zeta }_{k,\ell }\right)|{\mathcal{F}}_{t-\delta }\right]dt.$$

Under the assumptions (C.0) and (C.2), we have

$$\begin{array}{ccc}\hfill {\tilde{\alpha }}_{k,\ell }& =& \frac{1}{T}{\int }_0^T{\int }_{\mathcal{S}}{\varphi }_k\left(\mathbf{x};{\zeta }_{k,\ell }\right)f_t^{{\mathcal{F}}_{t-\delta }}\left(\mathbf{x}\right)\nu \left(d\mathbf{x}\right)dt\hfill \\ & =& {\int }_{\mathbf{S}}{\varphi }_k\left(\mathbf{x};{\zeta }_{k,\ell }\right)\left(\frac{1}{T}{\int }_0^T{f}_t^{{\mathcal{F}}_{t-\delta }}\left(\mathbf{x}\right)dt\right)\nu \left(d\mathbf{x}\right)\hfill \\ & =& {\int }_{\mathbf{S}}{\varphi }_k\left(\mathbf{x};{\zeta }_{k,\ell }\right)\left(f\left(\mathbf{x}\right)+o\left(1\right)\right)\nu \left(d\mathbf{x}\right)\hfill \\ & =& {\int }_{\mathbf{S}}{\varphi }_k\left(\mathbf{x};{\zeta }_{k,\ell }\right)f\left(\mathbf{x}\right)\nu \left(d\mathbf{x}\right)+o\left(1\right)\hfill \\ & =& {\alpha }_{k,\ell }+o\left(1\right).\hfill \end{array}$$

We readily obtain

$${\tilde{\alpha }}_{k,\ell }={\alpha }_{k,\ell },\mathrm{a}\mathrm{s},T\to \infty ,$$

(47)

implying that

$$A_{k,\ell ,2}=o\left(1\right),\mathrm{a}.\mathrm{s}.$$

(48)

Therefore, we infer that

$$\begin{array}{ccc}\hfill {\widehat{\alpha }}_{k,\ell }-{\alpha }_{k,\ell }& =& A_{k,\ell ,1}+o\left(1\right),\mathrm{a}.\mathrm{s}.\hfill \end{array}$$

We now consider the term $A_{k,\ell ,1}$. We infer

$$\begin{array}{ccc}\hfill A_{k,\ell ,1}& =& {\widehat{\alpha }}_{k,\ell }-{\tilde{\alpha }}_{k,\ell }\hfill \\ & =& \frac{1}{T}{\int }_0^T\left({\varphi }_k\left({\mathbf{x}}_t;{\zeta }_{k,\ell }\right)-\mathbb{E}\left[{\varphi }_k\left({\mathbf{x}}_t;{\zeta }_{k,\ell }\right)|{\mathcal{F}}_{t-\delta }\right]\right)dt\hfill \\ & =& \frac{1}{n}\sum _{i=1}^n{\mathrm{\Phi }}_k\left({\zeta }_{k,\ell }\right),\hfill \end{array}$$

where

$${\mathrm{\Phi }}_{i,k}\left({\zeta }_{k,\ell }\right)=\frac{1}{\delta }{\int }_{T_{i-1}}^{T_i}\left({\varphi }_k\left({\mathbf{x}}_t;{\zeta }_{k,\ell }\right)-\mathbb{E}\left[{\varphi }_k\left({\mathbf{x}}_t;{\zeta }_{k,\ell }\right)|{\mathcal{F}}_{t-\delta }\right]\right)dt.$$

Notice that, with respect to the sequence of $\sigma -$fields ${\left({\mathcal{F}}_i\right)}_{0\le k\le n}$, ${\left({\mathrm{\Phi }}_{i,k}\left({\zeta }_{k,\ell }\right)\right)}_{0\le k\le n}$ is a sequence of martingale differences. It is evident, by Lemma 1, to infer that

$$\begin{array}{ccc}\hfill \mathbb{E}\left[|A_{k,\ell ,1}{|}^2\right]& =& \frac{1}{n^2}\mathbb{E}\left[{\left|\sum _{i=1}^n{\mathrm{\Phi }}_{i,k}\left({\zeta }_{k,\ell }\right)\right|}^2\right],\hfill \end{array}$$

where

$$\begin{array}{ccc}\hfill {\left(\mathbb{E}\left[{\left|\sum _{i=1}^n{\mathrm{\Phi }}_{i,k}\left({\zeta }_{k,\ell }\right)\right|}^2\right]\right)}^{\frac{1}{2}}& \le & n^{1/2}{∥{\mathrm{\Phi }}_{1,k}\left({\zeta }_{k,\ell }\right)∥}_2+{∥\sum _{i=1}^n\mathbb{E}\left[{\mathrm{\Phi }}_{i,k}^2\left({\zeta }_{k,\ell }\right)|{\mathcal{F}}_{i-1}\right]∥}_1^{1/2}\hfill \\ \hfill & =& {\mathrm{\Phi }}_{\left(1\right)}+{\mathrm{\Phi }}_{\left(2\right)}.\hfill \end{array}$$

(49)

Observe that for all $0\le t\le \delta$, the $\sigma$-field ${\mathcal{F}}_{t-\delta }={\mathcal{F}}_0$ represents the trivial one. On the one hand, by using the classical decomposition in connection with the fact that ${\mathcal{F}}_0$ is the trivial $\sigma -$field, we have

$$\begin{array}{ccc}\hfill {\displaystyle \frac{1}{n}}{\mathrm{\Phi }}_{\left(1\right)}^2& =& \Vert {\mathrm{\Phi }}_{1,k}\left({\zeta }_{k,\ell }\right){\Vert }_2^2\hfill \\ \hfill & =& \mathbb{E}\left[{\left|\frac{1}{\delta }{\int }_0^{\delta }\left({\varphi }_k\left({\mathbf{x}}_t;{\zeta }_{k,\ell }\right)-\mathbb{E}\left[{\varphi }_k\left({\mathbf{x}}_t;{\zeta }_{k,\ell }\right)|{\mathcal{F}}_0\right]\right)dt\right|}^2\right]\hfill \\ \hfill & \le & \frac{1}{{\delta }^2}{\int }_0^{\delta }\mathbb{E}\left[\sum _{j=0}^2\mid {\varphi }_k\left({\mathbf{x}}_t;{\zeta }_{k,\ell }\right){\mid }^j{\left(\mathbb{E}\left[\mid {\varphi }_k\left({\mathbf{x}}_t;{\zeta }_{k,\ell }\right)\mid \right]\right)}^{2-j}\right]dt\hfill \\ \hfill & =& \frac{1}{{\delta }^2}{\int }_0^{\delta }\left(\sum _{j=0}^2{C}_2^j\mathbb{E}\left[\mid {\varphi }_k\left({\mathbf{x}}_t;{\zeta }_{k,\ell }\right){\mid }^j\right].{\left(\mathbb{E}\left[\mid {\varphi }_k\left({\mathbf{x}}_t;{\zeta }_{k,\ell }\right)\mid \right]\right)}^{2-j}\right)dt\hfill \\ \hfill & =& \frac{1}{{\delta }^2}{\int }_0^{\delta }\left(C_2^2\mathbb{E}\left[\mid {\varphi }_k\left({\mathbf{x}}_t;{\zeta }_{k,\ell }\right){\mid }^2\right]+C_2^1{\left(\mathbb{E}\left[\mid {\varphi }_k\left({\mathbf{x}}_t;{\zeta }_{k,\ell }\right)\mid \right]\right)}^2+C_2^0\mathbb{E}\left[\mid {\varphi }_k\left({\mathbf{x}}_t;{\zeta }_{k,\ell }\right){\mid }^2\right]\right)dt.\hfill \end{array}$$

Remark that, under the conditions (F.1) and (E.1)(i) and the fact that $\mathcal{E}$ is an orthonormal basis of H, we obtain

$$\begin{array}{ccc}\hfill \mathbb{E}\left[{\left|{\varphi }_k\left({\mathbf{x}}_t;{\zeta }_{k,\ell }\right)\right|}^2\right]& =& {\int }_{\mathbf{S}}{\left|{\varphi }_k\left(\mathbf{x};{\zeta }_{k,\ell }\right)\right|}^{2}f\left(\mathbf{x}\right)\nu \left(d\mathbf{x}\right)\hfill \\ \hfill & \le & C_f{\int }_{\mathbf{S}}{\left|{\varphi }_k\left(\mathbf{x};{\zeta }_{k,\ell }\right)\right|}^2\nu \left(d\mathbf{x}\right)\hfill \\ \hfill & =& C_f{\int }_{\mathbf{S}}{\left|\sum _{j\in {\mathcal{I}}_k}\frac{1}{\sqrt{g_{j,k,\ell }}}{e}_j\left({\zeta }_{k,\ell }\right)e_j\left(\mathbf{x}\right)\right|}^2\nu \left(d\mathbf{x}\right)\hfill \\ \hfill & =& C_f{\int }_{\mathbf{S}}\sum _{j\in {\mathcal{I}}_k}\frac{1}{g_{j,k,\ell }}{\left|e_j\left({\zeta }_{k,\ell }\right)\right|}^2\nu \left(d\mathbf{x}\right)\hfill \\ \hfill & \le & C_f{C}_1,\hfill \end{array}$$

(50)

where $C_1$ is a positive constant,

$$\begin{array}{ccc}\hfill \mathbb{E}\left[\mid {\varphi }_k\left({\mathbf{x}}_t;{\zeta }_{k,\ell }\right){\mid }^2\right]& =& O\left(1\right),\hfill \end{array}$$

(51)

therefore,

$$\begin{array}{c}\hfill {\mathrm{\Phi }}_{\left(1\right)}=O\left(T^{1/2}\right).\end{array}$$

(52)

Furthermore, we examine the second term of decomposition (49), and remark that

$$\begin{array}{ccc}\hfill {\mathrm{\Phi }}_2& =& {\left(\mathbb{E}\left(\sum _{i=1}^n\mathbb{E}\left[{\mathrm{\Phi }}_{i,k}^2\left({\zeta }_{k,\ell }\right)|{\mathcal{F}}_{i-1}\right]\right)\right)}^{1/2}\hfill \\ & =& {\left(\sum _{i=1}^n\mathbb{E}\left(\mathbb{E}\left[{\mathrm{\Phi }}_k^2\left({\mathbf{x}}_i;{\zeta }_{k,\ell }\right)|{\mathcal{F}}_{i-1}\right]\right)\right)}^{1/2}\hfill \\ & =& {\left(\sum _{i=1}^n\mathbb{E}\left[{\mathrm{\Phi }}_{i,k}^2\left({\zeta }_{k,\ell }\right)\right]\right)}^{1/2},\hfill \end{array}$$

using the notable identity combined with Jensen’s inequality, we obtain

$$\begin{array}{ccc}\hfill \mathbb{E}\left[{\mathrm{\Phi }}_{i,k}^2\left({\zeta }_{k,\ell }\right)\right]& =& \mathbb{E}\left[{\left(\left|\frac{1}{\delta }{\int }_{T_{i-1}}^{T_i}\left({\varphi }_k\left({\mathbf{x}}_t;{\zeta }_{k,\ell }\right)-\mathbb{E}\left[{\varphi }_k\left({\mathbf{x}}_t;{\zeta }_{k,\ell }\right)|{\mathcal{F}}_{t-\delta }\right]\right)dt\right|\right)}^2\right]\hfill \\ & \le & \frac{1}{{\delta }^2}{\int }_{T_{i-1}}^{T_i}\mathbb{E}\left[{\left(\left|\left({\varphi }_k\left({\mathbf{x}}_t;{\zeta }_{k,\ell }\right)-\mathbb{E}\left[{\varphi }_k\left({\mathbf{x}}_t;{\zeta }_{k,\ell }\right)|{\mathcal{F}}_{t-\delta }\right]\right)\right|\right)}^2\right]dt\hfill \\ & \le & \frac{1}{{\delta }^2}{\int }_{T_{i-1}}^{T_i}\mathbb{E}\left[2\mid {\varphi }_k\left({\mathbf{x}}_t;{\zeta }_{k,\ell }\right){\mid }^2+2\mathbb{E}\left[\mid {\varphi }_k\left({\mathbf{x}}_t;{\zeta }_{k,\ell }\right){\mid }^2|{\mathcal{F}}_{t-\delta }\right]\right]dt\hfill \\ & \le & \frac{1}{{\delta }^2}{\int }_{T_{i-1}}^{T_i}\left(2\mathbb{E}\left[\mid {\varphi }_k\left({\mathbf{x}}_t;{\zeta }_{k,\ell }\right){\mid }^2\right]+2\mathbb{E}\left[\mathbb{E}\left[\mid {\varphi }_k\left({\mathbf{x}}_t;{\zeta }_{k,\ell }\right){\mid }^2|{\mathcal{F}}_{t-\delta }\right]\right]\right)dt\hfill \\ & \le & \frac{4}{{\delta }^2}{\int }_{T_{i-1}}^{T_i}\mathbb{E}\left[\mid {\varphi }_k\left({\mathbf{x}}_t;{\zeta }_{k,\ell }\right){\mid }^2\right]dt,\hfill \end{array}$$

observe that, using (50), we have

$$\begin{array}{c}\hfill {\mathrm{\Phi }}_2=O\left(T^{1/2}\right).\end{array}$$

(53)

Therefore, combining Equations (52) and (53), we obtain

$$\begin{array}{c}\hfill {\left(\mathbb{E}\left[{\left|\sum _{i=1}^n{\mathrm{\Phi }}_{i,k}\left({\zeta }_{k,\ell }\right)\right|}^2\right]\right)}^{\frac{1}{2}}=O\left(T^{1/2}\right).\end{array}$$

Hence,

$$\begin{array}{ccc}\hfill \mathbb{E}\left[|A_{k,\ell ,1}{|}^2\right]& =& \frac{1}{n^2}\mathbb{E}\left[{\left|\sum _{i=1}^n{\mathrm{\Phi }}_{i,k}\left({\zeta }_{k,\ell }\right)\right|}^2\right]\hfill \\ & =& \frac{{\delta }^2}{T^2}O\left(T\right)\hfill \\ & \le & C\left({\displaystyle \frac{lnT}{T}}\right).\hfill \end{array}$$

Therefore, there exists a constant $C=C_f{C}_1>0$, such that

$$\mathbb{E}\left({\left|{\widehat{\alpha }}_{k,\ell }-{\alpha }_{k,\ell }\right|}^2\right)\le \frac{4C\delta }{T}\le 4C\left(\frac{lnT}{T}\right).$$

(54)

Hence the proof is complete. □

 Proof of Lemma 4. 

Consider the following decomposition

$$\begin{array}{ccc}\hfill {\widehat{\beta }}_{k,\ell }-{\beta }_{k,\ell }& =& {\widehat{\beta }}_{k,\ell }-{\tilde{\beta }}_{k,\ell }+{\tilde{\beta }}_{k,\ell }-{\beta }_{k,\ell }\hfill \\ \hfill & =& B_{k,\ell ,1}+B_{k,\ell ,2},\hfill \end{array}$$

(55)

where

$${\tilde{\beta }}_{k,\ell }=\frac{1}{T}{\int }_0^T\mathbb{E}\left[{\psi }_k\left({\mathbf{x}}_i;{\eta }_{k,\ell }\right)|{\mathcal{F}}_{t-\delta }\right]dt.$$

Remark that, under conditions (F.1) and (E.1)(i) and making use of the fact that $\mathcal{E}$ is an orthonormal basis of H, and reasoning in a similar way as in Equation (47), we infer that

$${\tilde{\beta }}_{k,\ell }={\beta }_{k,\ell },\mathrm{a}\mathrm{s},n\to \infty .$$

(56)

This, in turn, implies that

$$B_{k,\ell ,2}=o\left(1\right),\mathrm{a}.\mathrm{s}.$$

(57)

Therefore, we obtain

$$\begin{array}{ccc}\hfill {\widehat{\beta }}_{k,\ell }-{\beta }_{k,\ell }& =& B_{k,\ell ,1}+o\left(1\right),\mathrm{a}.\mathrm{s}.\hfill \end{array}$$

Hence, we readily infer

$$\begin{array}{ccc}\hfill \mathbb{E}\left({\left|{\widehat{\beta }}_{k,\ell }-{\beta }_{k,\ell }\right|}^4\right)& =& \frac{1}{n^4}\mathbb{E}\left({\left|\sum _{i=1}^n{\mathrm{\Psi }}_{i,k,\ell }\right|}^4\right),\hfill \end{array}$$

(58)

where

$${\mathrm{\Psi }}_{i,k,\ell }=\frac{1}{\delta }{\int }_{T_{i-1}}^{T_i}\left({\psi }_k\left({\mathbf{x}}_t;{\eta }_{k,\ell }\right)-\mathbb{E}\left[{\psi }_k\left({\mathbf{x}}_t;{\eta }_{k,\ell }\right)|{\mathcal{F}}_{t-\delta }\right]\right)dt.$$

Remark that, with respect to the sequence of $\sigma -$fields ${\left({\mathcal{F}}_{i-1}\right)}_{0\le k\le n}$, ${\left({\mathrm{\Psi }}_{i,k,\ell }\right)}_{0\le k\le n}$ is a sequence of martingale differences, an application of the Burkholder–Rosenthal inequality (see Lemma 1), gives

$$\begin{array}{ccc}\hfill {\left(\mathbb{E}\left({\left|\sum _{i=1}^n{\mathrm{\Psi }}_{i,k,\ell }\right|}^4\right)\right)}^{1/4}& \le & {∥\underset{1\le j\le n}{max}\left|\sum _{i=1}^j{\mathrm{\Psi }}_{i,k,\ell }\right|∥}_4\hfill \\ & \ll & n^{1/4}{\parallel {\mathrm{\Psi }}_{1,k,\ell }\parallel }_4+{∥\sum _{i=1}^n\mathbb{E}\left({\mathrm{\Psi }}_{i,k,\ell }^2\right|{\mathcal{F}}_{i-2})∥}_{4/2}^{1/2}\hfill \\ & =& {\mathrm{\Psi }}_{k,\ell }^{\left(1\right)}+{\mathrm{\Psi }}_{k,\ell }^{\left(2\right)}.\hfill \end{array}$$

(59)

Consider the first term of Equation (59). We recall that for all $0\le t\le \delta$, the $\sigma$-field ${\mathcal{F}}_{t-\delta }={\mathcal{F}}_0$ represents the trivial one. Applying Jensen’s inequality, we have

$$\begin{array}{ccc}\hfill \frac{1}{n}{\left({\mathrm{\Psi }}_{k,\ell }^{\left(1\right)}\right)}^4& =& \parallel {\mathrm{\Psi }}_{1,k,\ell }{\parallel }_4^4\hfill \\ & =& \mathbb{E}\left({\left|\frac{1}{\delta }{\int }_0^{\delta }\left({\psi }_k\left({\mathbf{x}}_t;{\eta }_{k,\ell }\right)-\mathbb{E}\left[{\psi }_k\left({\mathbf{x}}_t;{\eta }_{k,\ell }\right)|{\mathcal{F}}_0\right]\right)dt\right|}^4\right)\hfill \\ & \le & \frac{1}{{\delta }^4}{\int }_0^{\delta }\mathbb{E}\left[{\left(\left|{\psi }_k\left({\mathbf{x}}_t;{\eta }_{k,\ell }\right)\right|+\mathbb{E}\left[\left|{\psi }_k\left({\mathbf{x}}_1;{\eta }_{k,\ell }\right)\right|\right]\right)}^4\right]dt.\hfill \end{array}$$

Using the Minkowski inequality, we obtain

$$\begin{array}{c}{\displaystyle {\int }_0^{\delta }\mathbb{E}\left[{\left(\left|{\psi }_k\left({\mathbf{x}}_t;{\eta }_{k,\ell }\right)\right|+\mathbb{E}\left[\left|{\psi }_k\left({\mathbf{x}}_1;{\eta }_{k,\ell }\right)\right|\right]\right)}^4\right]dt}\hfill \\ ={\int }_0^{\delta }{\left({\mathbb{E}}^{1/4}\left[{\left|{\psi }_k\left({\mathbf{x}}_t;{\eta }_{k,\ell }\right)\right|}^4\right]+{\mathbb{E}}^{1/4}\left[{\left(\mathbb{E}\left[\left|{\psi }_k\left({\mathbf{x}}_1;{\eta }_{k,\ell }\right)\right|\right]\right)}^4\right]\right)}^{4}dt\hfill \\ =16{\int }_0^{\delta }\mathbb{E}\left[{\left|{\psi }_k\left({\mathbf{x}}_t;{\eta }_{k,\ell }\right)\right|}^4\right]dt.\hfill \end{array}$$

(60)

This gives that

$$\begin{array}{ccc}\hfill \frac{1}{n}{\left({\mathrm{\Psi }}_{k,\ell }^{\left(1\right)}\right)}^4& \le & 16{\int }_0^{\delta }{\left({\mathbb{E}}^{1/4}\left[{\left|{\psi }_k\left({\mathbf{x}}_t;{\eta }_{k,\ell }\right)\right|}^4\right]\right)}^{4}dt.\hfill \end{array}$$

By reasoning in a similar way as in Equation (50) and making use of conditions (F.1) and (E.1)(i), for all $0\le t\le T$, we infer that

$$\mathbb{E}\left[{\left|{\psi }_k\left({\mathbf{x}}_t;{\eta }_{k,\ell }\right)\right|}^2\right]\le C,$$

(61)

where C denotes a positive constant. In addition, by the Cauchy–Schwarz inequality in combination with conditions (E.1)(ii), (E.2) and condition (16), we infer

$$\begin{array}{ccc}\hfill \underset{\mathbf{x}\in \mathbf{S}}{sup}\left|{\psi }_k\left(\mathbf{x};{\eta }_{k,\ell }\right)\right|& \le & \underset{\mathbf{x}\in \mathbf{S}}{sup}\sum _{j\in {\mathcal{J}}_k}\frac{1}{\sqrt{h_{j,k,\ell }}}|e_j\left({\eta }_{k,\ell }\right)\left|\right|e_j\left(\mathbf{x}\right)|\hfill \\ \hfill & \le & {\left(\sum _{j\in {\mathcal{J}}_k}\frac{1}{h_{j,k,\ell }}{\left|e_j\left({\eta }_{k,\ell }\right)\right|}^2\right)}^{1/2}{\left(\underset{\mathbf{x}\in \mathbf{S}}{sup}\sum _{j\in {\mathcal{J}}_k}{\left|e_j\left(\mathbf{x}\right)\right|}^2\right)}^{1/2}\hfill \\ & \le & C_1^{1/2}{C}_2^{1/2}\sqrt{|{\mathcal{J}}_k|}\hfill \\ \hfill & \le & C_4\sqrt{|{\mathcal{J}}_{m_n}|}\hfill \\ & \le & C_4\sqrt{\frac{T}{lnT}}.\hfill \end{array}$$

(62)

We then obtain

$$\begin{array}{ccc}\hfill \mathbb{E}\left[{\left|{\psi }_k\left({\mathbf{x}}_1;{\eta }_{k,\ell }\right)\right|}^4\right]& =& O\left(\frac{T}{lnT}\right).\hfill \end{array}$$

(63)

Recall that $T=\delta n$, we deduce that

$${\mathrm{\Psi }}_{k,\ell }^{\left(1\right)}=O\left(\frac{T^{1/2}}{{\left(lnT\right)}^{1/4}}\right).$$

(64)

We now consider the upper bound of ${\mathrm{\Psi }}_{k,\ell }^{\left(2\right)}$ in Equation (59). Remark that

$$\begin{array}{ccc}\hfill {\mathrm{\Psi }}_{k,\ell }^{\left(2\right)}& =& {∥\sum _{i=1}^n\mathbb{E}({\mathrm{\Psi }}_{i,k,\ell }^2\mid {\mathcal{F}}_{i-2})∥}_2^{1/2}\hfill \\ & =& {\left(\mathbb{E}\left[{\left(\sum _{i=1}^n\mathbb{E}\left[{\mathrm{\Psi }}_{i,k,\ell }^2\mid {\mathcal{F}}_{i-2}\right]\right)}^2\right]\right)}^{1/4}.\hfill \end{array}$$

Observe that for all $T_{i-1}\le t\le T_i$ for all $i=1,\dots ,n$; we have ${\mathcal{F}}_{i-2}\subset {\mathcal{F}}_{t-\delta }$. Making use of Jensen and Minkowski’s inequality, it follows that

$$\begin{array}{c}{\displaystyle \sum _{i=1}^n\mathbb{E}\left[{\mathrm{\Psi }}_{i,k,\ell }^2\mid {\mathcal{F}}_{i-2}\right]}\hfill \\ =\sum _{i=1}^n\left(\mathbb{E}\left[{\left(\frac{1}{\delta }{\int }_{T_{i-1}}^{T_i}\left({\psi }_k\left({\mathbf{x}}_t;{\eta }_{k,\ell }\right)-\mathbb{E}\left[{\psi }_k\left({\mathbf{x}}_t;{\eta }_{k,\ell }\right)\mid {\mathcal{F}}_{t-\delta }\right]\right)dt\right)}^2\mid {\mathcal{F}}_{i-2}\right]\right)\hfill \\ \le \frac{1}{{\delta }^2}\sum _{i=1}^n{\int }_{T_{i-1}}^{T_i}{\left({\mathbb{E}}^{1/2}\left[{\psi }_k^2\left({\mathbf{x}}_t;{\eta }_{k,\ell }\right)\mid {\mathcal{F}}_{i-2}\right]+{\mathbb{E}}^{1/2}\left[\mathbb{E}\left[{\psi }_k^2\left({\mathbf{x}}_t;{\eta }_{k,\ell }\right)\mid {\mathcal{F}}_{t-\delta }\right]\mid {\mathcal{F}}_{i-2}\right]\right)}^{2}dt\hfill \\ =\frac{4}{{\delta }^2}\sum _{i=1}^n{\int }_{T_{i-1}}^{T_i}\mathbb{E}\left[{\psi }_k^2\left({\mathbf{x}}_t;{\eta }_{k,\ell }\right)\mid {\mathcal{F}}_{i-2}\right]dt.\hfill \end{array}$$

For all $T_{i-1}\le t\le T_i$, using the stationarity of the process ${\left({\mathbf{X}}_t\right)}_{t\ge 0}$, we have

$$f_t^{{\mathcal{F}}_{i-2}}\left(\mathbf{x}\right)=f_{T_{i-1}}^{{\mathcal{F}}_{i-2}}\left(\mathbf{x}\right).$$

Remark, under conditions (F.1), (E.1)(i), (C.0), and (C.1) and statement (61), that

$$\begin{array}{ccc}\hfill \sum _{i=1}^n{\int }_{T_{i-1}}^{T_i}\mathbb{E}\left[{\left({\psi }_k\left({\mathbf{x}}_t;{\eta }_{k,\ell }\right)\right)}^2\mid {\mathcal{F}}_{i-2}\right]dt& =& n{\int }_{\mathbf{S}}{\left|{\psi }_k\left(\mathbf{x};{\eta }_{k,\ell }\right)\right|}^2\left(\frac{1}{n}\sum _{i=1}^n{\int }_{T_{i-1}}^{T_i}{f}_t^{{\mathcal{F}}_{i-2}}\left(\mathbf{x}\right)dt\right)\nu \left(d\mathbf{x}\right)\hfill \\ \hfill & \le & n{\int }_{\mathbf{S}}{\left|{\psi }_k\left(\mathbf{x};{\eta }_{k,\ell }\right)\right|}^2\left(\delta \frac{1}{n}\sum _{i=1}^n{f}_{T_{i-1}}^{{\mathcal{F}}_{i-2}}\left(\mathbf{x}\right)\right)\nu \left(d\mathbf{x}\right)\hfill \\ & =& n{\int }_{\mathbf{S}}{\left|{\psi }_k\left(\mathbf{x};{\eta }_{k,\ell }\right)\right|}^2\left(f\left(\mathbf{x}\right)+o\left(1\right)\right)\nu \left(d\mathbf{x}\right)\hfill \\ \hfill & \le & n\left(C_f+o\left(1\right)\right){\int }_{\mathbf{S}}{\left|{\psi }_k\left(\mathbf{x};{\zeta }_{k,\ell }\right)\right|}^2\nu \left(d\mathbf{x}\right)\hfill \\ & \le & \frac{T{C}_f{C}_1}{\delta }.\hfill \end{array}$$

(65)

It follows that

$$\begin{array}{ccc}\hfill {\mathrm{\Psi }}_{k,\ell }^{\left(2\right)}& =& O\left(T^{1/2}\right).\hfill \end{array}$$

(66)

Combining statements (59), (64), and (66), we obtain

$$\begin{array}{ccc}\hfill \mathbb{E}\left({\left|\sum _{i=1}^n{\mathrm{\Psi }}_{i,k,\ell }\right|}^4\right)& =& O\left(\frac{T^2}{lnT}\right)+O\left(T^2\right).\hfill \end{array}$$

We conclude that

$$\begin{array}{ccc}\hfill \mathbb{E}\left({\left|{\widehat{\beta }}_{k,\ell }-{\beta }_{k,\ell }\right|}^4\right)& =& O\left(\frac{1}{T^{2}lnT}\right)+O\left(\frac{1}{T^2}\right).\hfill \end{array}$$

(67)

This implies that there exists a constant $C>0$, such that

$$\begin{array}{ccc}\hfill \mathbb{E}\left({\left|{\widehat{\beta }}_{k,\ell }-{\beta }_{k,\ell }\right|}^4\right)& \le & C{\left(\frac{lnT}{T}\right)}^2.\hfill \end{array}$$

(68)

The proof is achieved. □

 Proof of Lemma 5. 

Consider the previous decomposition in Lemma 4, to write that

$$\begin{array}{ccc}\hfill {\widehat{\beta }}_{k,\ell }-{\beta }_{k,\ell }& =& \left({\widehat{\beta }}_{k,\ell }-{\tilde{\beta }}_{k,\ell }\right)+\left({\tilde{\beta }}_{k,\ell }-{\beta }_{k,\ell }\right)\hfill \\ & =& B_{k,\ell ,1}+B_{k,\ell ,2},\hfill \end{array}$$

where

$$\begin{array}{ccc}\hfill B_{k,\ell ,1}& =& \frac{1}{n}\sum _{i=1}^n{\mathrm{\Psi }}_{i,k,\ell }=\frac{1}{n}\sum _{i=1}^n\left(\frac{1}{\delta }{\int }_{T_{i-1}}^{T_i}\left({\psi }_k\left({\mathbf{x}}_t;{\eta }_{k,\ell }\right)-\mathbb{E}\left[{\psi }_k\left({\mathbf{x}}_t;{\eta }_{k,\ell }\right)|{\mathcal{F}}_{t-\delta }\right]\right)dt\right),\hfill \\ \hfill B_{k,\ell ,2}& =& \frac{1}{n}\sum _{i=1}^n\left(\frac{1}{\delta }{\int }_{T_{i-1}}^{T_i}\left(\mathbb{E}\left[{\psi }_k\left({\mathbf{x}}_t;{\eta }_{k,\ell }\right)|{\mathcal{F}}_{t-\delta }\right]-{\beta }_{k,\ell }\right)dt\right).\hfill \end{array}$$

Making use of Equation (57), we achieve the desired result for the term $B_{k,\ell ,2}$

$$\begin{array}{ccc}\hfill {\widehat{\beta }}_{k,\ell }-{\beta }_{k,\ell }& =& B_{k,\ell ,1}+o\left(1\right).\hfill \end{array}$$

We now remark

$$\begin{array}{ccc}\hfill \mathbb{P}\left(\left|\frac{1}{n}\sum _{i=1}^n{\mathrm{\Psi }}_{i,k,\ell }\right|\ge \frac{\kappa }{2}\sqrt{\frac{lnT}{T}}\right)& \le & \mathbb{P}\left(\left|\sum _{i=1}^n{\mathrm{\Psi }}_{i,k,\ell }\right|\ge \frac{\delta \kappa }{2}\sqrt{TlnT}\right).\hfill \end{array}$$

The application of Lemma 2 implies that

$$\begin{array}{ccc}\hfill \left|{\mathrm{\Psi }}_{i,k,\ell }\right|& =& \left|{\psi }_k\left({\mathbf{x}}_i;{\eta }_{k,\ell }\right)-\mathbb{E}\left[{\psi }_k\left({\mathbf{x}}_i;{\eta }_{k,\ell }\right)\mid {\mathcal{F}}_{i-1}\right]\right|\hfill \\ \hfill & \le & 2\underset{\mathbf{x}\in \mathbf{S}}{sup}\left|{\psi }_k\left(\mathbf{x};{\eta }_{k,\ell }\right)\right|\hfill \\ \hfill & \le & C_3\sqrt{\frac{T}{lnT}}\hfill \\ \hfill & \le & C_3\sqrt{T}.\hfill \end{array}$$

(69)

Let $B=C_3\sqrt{T}$. Then, for all ${ϵ}_T=\frac{\delta \kappa }{2}\sqrt{TlnT}$ where n is sufficiently large, we have

$$\begin{array}{ccc}\hfill \mathbb{P}\left(\left|\sum _{i=1}^n{\mathrm{\Psi }}_{i,k,\ell }\right|\ge \frac{\kappa }{2}\sqrt{TlnT}\right)& \le & 2exp\left\{-\frac{{ϵ}_T^2}{2n{B}^2}\right\}\hfill \\ \hfill & =& 2exp\left\{-\frac{{\left(\frac{\delta \kappa }{2}\sqrt{TlnT}\right)}^2}{2(T/\delta ){\left(C_3\sqrt{T}\right)}^2}\right\}\hfill \\ & =& 2exp\left\{-\frac{{\delta }^3{\kappa }^{2}lnT}{8C_3^{2}T}\right\}\hfill \\ \hfill & =& 2exp\left\{ln{T}^{-\frac{{\delta }^3{\kappa }^2}{8C_3^{2}T}}\right\}\hfill \\ & =& 2T^{-w(\kappa ,T)},\hfill \end{array}$$

(70)

where

$$w(\kappa ,T)=\frac{{\delta }^3{\kappa }^2}{8C_3^{2}T}.$$

By choosing $\kappa$ such that $w(T,\kappa )=2,$ we have

$$\begin{array}{ccc}\hfill \mathbb{P}\left(\left|{\widehat{\beta }}_{k,\ell }-{\beta }_{k,\ell }\right|\ge \frac{\kappa }{2}\sqrt{\frac{lnT}{T}}\right)& \le & C\frac{1}{T^2}+o\left(1\right)\hfill \\ & \le & C{\left(\frac{lnT}{T}\right)}^2.\hfill \end{array}$$

(71)

The proof of (45) is achieved. □

Recall that

$$\widehat{\mathbf{m}}(\mathbf{x},\rho )=\sum _{\ell \in {\mathcal{I}}_0}{\widehat{\eta }}_{0,\ell }{\varphi }_0\left(\mathbf{x};{\zeta }_{0,\ell }\right)+\sum _{k=0}^{m_T}\sum _{\ell \in {\mathcal{J}}_k}{\widehat{\theta }}_{k,\ell }{\mathrm{𝟙}}_{\left\{\right|{\widehat{\theta }}_{k,\ell }|\ge \kappa \sqrt{\frac{lnT}{T}}\}}{\psi }_k\left(\mathbf{x};{\eta }_{k;\ell }\right),$$

where

$$\begin{array}{ccc}\hfill {\widehat{\eta }}_{k,\ell }& =& \frac{1}{T}{\int }_0^T\frac{\rho \left(Y_t\right)}{f\left({\mathbf{X}}_t\right)}{\varphi }_k\left({\mathbf{X}}_t;{\zeta }_{k,\ell }\right)dt,{\widehat{\theta }}_{k,\ell }=\frac{1}{T}{\int }_0^T\frac{\rho \left(Y_t\right)}{f\left({\mathbf{X}}_t\right)}{\psi }_k\left({\mathbf{X}}_t;{\eta }_{k,\ell }\right)dt.\hfill \end{array}$$

 Lemma 6.

For each $k\in \{0,\dots ,m_T\}$ and each $\ell \in {\mathcal{I}}_k$ and under conditions (E.1)(i), (M.1), (M.2), (C.0), and (C.1), there is a constant $C>0$ in such a way that

$$\mathbb{E}\left({\left|{\widehat{\eta }}_{k,\ell }-{\eta }_{k,\ell }\right|}^2\right)\le C\left(\frac{lnT}{T}\right).$$

(72)

 Lemma 7. 

For each $k\in \{0,\dots ,m_T\}$ and each $\ell \in {\mathcal{J}}_k$, and under conditions (E.1), (E.2) (M.1), (M.2), (C.0), and (C.1), in combination with condition (24), there is a constant $C>0$ in such a way that

$$\mathbb{E}\left({\left|{\widehat{\theta }}_{k,\ell }-{\theta }_{k,\ell }\right|}^4\right)=C{\left(\frac{lnT}{T}\right)}^2,a.s.$$

(73)

 Lemma 8. 

For each $k\in \{0,\dots ,m_T\}$ and each $\ell \in {\mathcal{J}}_k$, for $\kappa >0$ sufficiently large, (E.1), (E.2) (M.1), (M.2), (C.0), and (C.1), in combination with condition (24), there is a constant $C>0$ in such a way that

$$\mathbb{P}\left(\left|{\widehat{\theta }}_{k,\ell }-{\theta }_{k,\ell }\right|\ge \frac{\kappa }{2}\sqrt{\frac{lnT}{T}}\right)\le C{\left(\frac{lnT}{T}\right)}^2.$$

(74)

 Proof of Lemma 2. 

Remark that the proof of Theorem 2 is a consequence of [52] (Theorem 3.1) with $c\left(n\right)={(lnT/T)}^{1/2}$, ${\sigma }_i=1$, $r=2$ in connection with Lemmas 6–8. We generalized the method of the proof in [34] (Theorem 4.1). □

 Proof of Lemma 6. 

Consider the subsequent decomposition

$$\begin{array}{ccc}\hfill {\widehat{\eta }}_{k,\ell }-{\eta }_{k,\ell }& =& {\widehat{\eta }}_{k,\ell }-{\tilde{\eta }}_{k,\ell }+{\tilde{\eta }}_{k,\ell }-{\eta }_{k,\ell }\hfill \\ \hfill & =& A_{k,\ell ,1}+A_{k,\ell ,2},\hfill \end{array}$$

(75)

where

$$\begin{array}{ccc}\hfill {\tilde{\eta }}_{k,\ell }& =& \frac{1}{T}{\int }_0^T\mathbb{E}\left[\frac{\rho \left(Y_t\right)}{f\left({\mathbf{X}}_t\right)}{\varphi }_k\left({\mathbf{X}}_t;{\zeta }_{k,\ell }\right)|{\mathcal{F}}_{t-\delta }\right]dt\hfill \\ & =& \frac{1}{T}{\int }_0^T\mathbb{E}\left[\frac{\left(\mathbf{m}({\mathbf{X}}_t,\rho )+{ϵ}_t\right)}{f\left({\mathbf{X}}_t\right)}{\varphi }_k\left({\mathbf{X}}_t;{\zeta }_{k,\ell }\right)|{\mathcal{F}}_{t-\delta }\right]dt\hfill \\ & =& \frac{1}{T}{\int }_0^T\mathbb{E}\left[\frac{\mathbf{m}({\mathbf{X}}_t,\rho )}{f\left({\mathbf{X}}_t\right)}{\varphi }_k\left({\mathbf{X}}_t;{\zeta }_{k,\ell }\right)|{\mathcal{F}}_{t-\delta }\right]dt+\frac{1}{T}{\int }_0^T\mathbb{E}\left[\frac{{ϵ}_t}{f\left({\mathbf{X}}_t\right)}{\varphi }_k\left({\mathbf{X}}_t;{\zeta }_{k,\ell }\right)|{\mathcal{F}}_{t-\delta }\right]dt.\hfill \end{array}$$

For all $0\le t\le T$; we have ${\mathcal{F}}_{t-\delta }\subset {\mathcal{S}}_{t-\delta }$ and from the independence between ${ϵ}_t$ and ${\mathbf{X}}_t$, we have

$$\begin{array}{ccc}\hfill \mathbb{E}\left[{ϵ}_t|{\mathcal{S}}_{t-\delta }\right]& =& \mathbb{E}\left[{ϵ}_t|{\mathbf{X}}_t\right]\hfill \\ & =& \mathbb{E}\left[{ϵ}_t\right]\hfill \\ & =& 0.\hfill \end{array}$$

(76)

Observe that

$$\begin{array}{ccc}\hfill \mathbb{E}\left[\frac{{ϵ}_t}{f\left({\mathbf{X}}_t\right)}{\varphi }_k\left({\mathbf{X}}_t;{\zeta }_{k,\ell }\right)|{\mathcal{F}}_{t-\delta }\right]& =& \mathbb{E}\left[\frac{\mathbb{E}\left[{ϵ}_t|{\mathcal{S}}_{t-\delta }\right]}{f\left({\mathbf{X}}_t\right)}{\varphi }_k\left({\mathbf{X}}_t;{\zeta }_{k,\ell }\right)|{\mathcal{F}}_{t-\delta }\right]\hfill \\ & =& \mathbb{E}\left[\frac{\mathbb{E}\left[{ϵ}_t|{\mathbf{X}}_t\right]}{f\left({\mathbf{X}}_t\right)}{\varphi }_k\left({\mathbf{X}}_t;{\zeta }_{k,\ell }\right)|{\mathcal{F}}_{t-\delta }\right]\hfill \\ & =& \mathbb{E}\left[\frac{\mathbb{E}\left[{ϵ}_t\right]}{f\left({\mathbf{X}}_t\right)}{\varphi }_k\left({\mathbf{X}}_t;{\zeta }_{k,\ell }\right)|{\mathcal{F}}_{t-\delta }\right]\hfill \\ & =& 0.\hfill \end{array}$$

This implies that

$$\begin{array}{ccc}\hfill {\tilde{\eta }}_{k,\ell }& =& \frac{1}{T}{\int }_0^T\mathbb{E}\left[\frac{\mathbf{m}({\mathbf{X}}_t,\rho )}{f\left({\mathbf{X}}_t\right)}{\varphi }_k\left({\mathbf{X}}_t;{\zeta }_{k,\ell }\right)|{\mathcal{F}}_{t-\delta }\right]dt.\hfill \end{array}$$

Using conditions (M.1), (M.2), (C.0), and (C.1), we obtain

$$\begin{array}{ccc}\hfill {\tilde{\eta }}_{k,\ell }& =& \frac{1}{T}{\int }_0^T{\int }_{\mathcal{S}}\frac{\mathbf{m}(\mathbf{x},\rho )}{f\left(\mathbf{x}\right)}{\varphi }_k\left(\mathbf{x};{\zeta }_{k,\ell }\right)f_t^{{\mathcal{F}}_{t-\delta }}\left(\mathbf{x}\right)\nu \left(d\mathbf{x}\right)dt\hfill \\ & =& {\int }_{\mathbf{S}}\frac{\mathbf{m}(\mathbf{x},\rho )}{f\left(\mathbf{x}\right)}{\varphi }_k\left(\mathbf{x};{\zeta }_{k,\ell }\right)\left(\frac{1}{T}{\int }_0^T{f}_t^{{\mathcal{F}}_{t-\delta }}\left(\mathbf{x}\right)dt\right)\nu \left(d\mathbf{x}\right)\hfill \\ & =& {\int }_{\mathbf{S}}\frac{\mathbf{m}(\mathbf{x},\rho )}{f\left(\mathbf{x}\right)}{\varphi }_k\left(\mathbf{x};{\zeta }_{k,\ell }\right)\left(f\left(\mathbf{x}\right)+o\left(1\right)\right)\nu \left(d\mathbf{x}\right)\hfill \\ & =& {\int }_{\mathbf{S}}\frac{\mathbf{m}(\mathbf{x},\rho )}{f\left(\mathbf{x}\right)}{\varphi }_k\left(\mathbf{x};{\zeta }_{k,\ell }\right)f\left(\mathbf{x}\right)\nu \left(d\mathbf{x}\right)+o\left(1\right)\hfill \\ & =& {\eta }_{k,\ell }+o\left(1\right).\hfill \end{array}$$

We readily obtain that

$${\tilde{\eta }}_{k,\ell }={\eta }_{k,\ell },\mathrm{a}\mathrm{s},T\to \infty ,$$

(77)

implying that

$$A_{k,\ell ,2}=o\left(1\right),\mathrm{a}.\mathrm{s}.$$

(78)

Therefore, we infer that

$$\begin{array}{ccc}\hfill {\widehat{\eta }}_{k,\ell }-{\eta }_{k,\ell }& =& A_{k,\ell ,1}+o\left(1\right),\mathrm{a}.\mathrm{s}.\hfill \end{array}$$

Let us now concentrate on the term $A_{k,\ell ,1}$ in Equation (75), we infer

$$\begin{array}{ccc}\hfill A_{k,\ell ,1}& =& {\widehat{\eta }}_{k,\ell }-{\tilde{\eta }}_{k,\ell }\hfill \\ & =& \frac{1}{T}{\int }_0^T\left(\frac{\rho \left(Y_t\right)}{f\left({\mathbf{X}}_t\right)}{\varphi }_k\left({\mathbf{X}}_t;{\zeta }_{k,\ell }\right)-\mathbb{E}\left[\frac{\rho \left(Y_t\right)}{f\left({\mathbf{X}}_t\right)}{\varphi }_k\left({\mathbf{X}}_t;{\zeta }_{k,\ell }\right)|{\mathcal{F}}_{t-\delta }\right]\right)dt\hfill \\ & =& \frac{1}{n}\sum _{i=1}^n\left(\frac{1}{\delta }{\int }_{T_{i-1}}^{T_i}\left(\frac{\rho \left(Y_t\right)}{f\left({\mathbf{X}}_t\right)}{\varphi }_k\left({\mathbf{X}}_t;{\zeta }_{k,\ell }\right)-\mathbb{E}\left[\frac{\rho \left(Y_t\right)}{f\left({\mathbf{X}}_t\right)}{\varphi }_k\left({\mathbf{X}}_t;{\zeta }_{k,\ell }\right)|{\mathcal{F}}_{t-\delta }\right]\right)dt\right)\hfill \\ & =& \frac{1}{n}\sum _{i=1}^n{\mathrm{\Phi }}_{k,i}\left({\zeta }_{k,\ell }\right),\hfill \end{array}$$

where

$${\mathrm{\Phi }}_{k,i}\left({\zeta }_{k,\ell }\right)=\frac{1}{\delta }{\int }_{T_{i-1}}^{T_i}\left(\frac{\rho \left(Y_t\right)}{f\left({\mathbf{X}}_t\right)}{\varphi }_k\left({\mathbf{X}}_t;{\zeta }_{k,\ell }\right)-\mathbb{E}\left[\frac{\rho \left(Y_t\right)}{f\left({\mathbf{X}}_t\right)}{\varphi }_k\left({\mathbf{X}}_t;{\zeta }_{k,\ell }\right)|{\mathcal{F}}_{t-\delta }\right]\right)dt.$$

Remark that, with respect to the sequence of $\sigma -$fields ${\left({\mathcal{F}}_{i-1}\right)}_{0\le k\le m_T}$, ${\left({\mathrm{\Phi }}_{k,i}\left({\zeta }_{k,\ell }\right)\right)}_{0\le k\le m_T}$ is a sequence of martingale differences. It is obvious, going with the proof of Equation (54), to observe that

$$\begin{array}{ccc}\hfill \mathbb{E}\left[|A_{k,\ell ,1}{|}^2\right]& =& \frac{1}{n^2}\mathbb{E}\left[{\left|\sum _{i=1}^n{\mathrm{\Phi }}_{k,i}\left({\zeta }_{k,\ell }\right)\right|}^2\right],\hfill \end{array}$$

where

$$\begin{array}{ccc}\hfill {\left(\mathbb{E}\left[{\left|\sum _{i=1}^n{\mathrm{\Phi }}_{k,i}\left({\zeta }_{k,\ell }\right)\right|}^2\right]\right)}^{\frac{1}{2}}& \le & n^{1/2}\Vert {\mathrm{\Phi }}_{k,1}\left({\zeta }_{k,\ell }\right){\Vert }_2+{\Vert \sum _{i=1}^n\mathbb{E}\left[{\mathrm{\Phi }}_{k,i}^2\left({\zeta }_{k,\ell }\right)|{\mathcal{F}}_{i-2}\right]\Vert }_1^{1/2}\hfill \\ & =& {\mathrm{\Phi }}_{\left(1\right)}+{\mathrm{\Phi }}_{\left(2\right)}.\hfill \end{array}$$

(79)

On the one hand, using Jensen and Minkowski’s inequalities combined, we obtain

$$\begin{array}{ccc}\hfill {\displaystyle \frac{1}{n}}{\mathrm{\Phi }}_{\left(1\right)}^2& =& \Vert {\mathrm{\Phi }}_{k,1}\left({\zeta }_{k,\ell }\right){\Vert }_2^2\hfill \\ \hfill & =& \mathbb{E}\left[{\left|\frac{1}{\delta }{\int }_0^{\delta }\left(\frac{\rho \left(Y_t\right)}{f\left({\mathbf{X}}_t\right)}{\varphi }_k\left({\mathbf{X}}_t;{\zeta }_{k,\ell }\right)-\mathbb{E}\left[\frac{\rho \left(Y_t\right)}{f\left({\mathbf{X}}_t\right)}{\varphi }_k\left({\mathbf{X}}_t;{\zeta }_{k,\ell }\right)|{\mathcal{F}}_{t-\delta }\right]\right)dt\right|}^2\right]\hfill \\ \hfill & \le & \frac{1}{{\delta }^2}{\int }_0^{\delta }\mathbb{E}\left[{\left|\frac{\rho \left(Y_t\right)}{f\left({\mathbf{X}}_t\right)}{\varphi }_k\left({\mathbf{X}}_t;{\zeta }_{k,\ell }\right)-\mathbb{E}\left[\frac{\rho \left(Y_t\right)}{f\left({\mathbf{X}}_t\right)}{\varphi }_k\left({\mathbf{X}}_t;{\zeta }_{k,\ell }\right)|{\mathcal{F}}_{t-\delta }\right]\right|}^2\right]dt\hfill \\ \hfill & \le & \frac{1}{{\delta }^2}{\int }_0^{\delta }{\left({\mathbb{E}}^{1/2}\left[{\left|\frac{\rho \left(Y_t\right)}{f\left({\mathbf{X}}_t\right)}{\varphi }_k\left({\mathbf{X}}_t;{\zeta }_{k,\ell }\right)\right|}^2\right]+{\mathbb{E}}^{1/2}\left[\mathbb{E}\left[{\left|\frac{\rho \left(Y_t\right)}{f\left({\mathbf{X}}_t\right)}{\varphi }_k\left({\mathbf{X}}_t;{\zeta }_{k,\ell }\right)\right|}^2|{\mathcal{F}}_{t-\delta }\right]\right]\right)}^{2}dt\hfill \\ \hfill & \le & \frac{4}{{\delta }^2}{\int }_0^{\delta }\mathbb{E}\left[{\left|\frac{\rho \left(Y_t\right)}{f\left({\mathbf{X}}_t\right)}{\varphi }_k\left({\mathbf{X}}_t;{\zeta }_{k,\ell }\right)\right|}^2\right]dt.\hfill \end{array}$$

It follows, from assumptions (M.1) and (M.2), that

$$|\rho \left(Y_t\right)|\le C_{\mathbf{m}}+\left|{ϵ}_t\right|,$$

(80)

combined with the independence between ${\mathbf{X}}_t$ and ${ϵ}_t$, $\mathbb{E}\left[{ϵ}_t^2\right]=1$. Remark that, under condition (E.1)(i) and using the fact that $\mathcal{E}$ is an orthonormal basis of H, we infer

$$\begin{array}{ccc}\hfill \mathbb{E}\left[{\left|\frac{\rho \left(Y_t\right)}{f\left({\mathbf{X}}_t\right)}{\varphi }_k\left({\mathbf{X}}_t;{\zeta }_{k,\ell }\right)\right|}^2\right]& \le & \left(C_{\mathbf{m}}^2+1\right)\frac{1}{C_f}\mathbb{E}\left[{\left|\frac{1}{f\left({\mathbf{X}}_t\right)}{\varphi }_k\left({\mathbf{X}}_t;{\zeta }_{k,\ell }\right)\right|}^2\right]\hfill \\ \hfill & \le & \left(C_{\mathbf{m}}^2+1\right){\int }_{\mathbf{S}}\frac{1}{f\left(\mathbf{x}\right)}{\left|{\varphi }_k\left(\mathbf{x};{\zeta }_{k,\ell }\right)\right|}^{2}f\left(\mathbf{x}\right)\nu \left(d\mathbf{x}\right)\hfill \\ & =& \left(C_{\mathbf{m}}^2+1\right){\int }_{\mathbf{S}}{\left|\sum _{j\in {\mathcal{I}}_k}\frac{1}{\sqrt{g_{j,k,\ell }}}{e}_j\left({\zeta }_{k,\ell }\right)e_j\left(\mathbf{x}\right)\right|}^2\nu \left(d\mathbf{x}\right)\hfill \\ \hfill & =& \left(C_{\mathbf{m}}^2+1\right){\int }_{\mathbf{S}}\sum _{j\in {\mathcal{I}}_k}\frac{1}{g_{j,k,\ell }}{\left|e_j\left({\zeta }_{k,\ell }\right)\right|}^2\nu \left(d\mathbf{x}\right)\hfill \\ & \le & \left(C_{\mathbf{m}}^2+1\right)C_1=O\left(1\right).\hfill \end{array}$$

(81)

Therefore, we have

$$\begin{array}{c}\hfill {\mathrm{\Phi }}_{\left(1\right)}={\left(\frac{\delta n\left(C_{\mathbf{m}}^2+1\right)C_1}{{\delta }^2})\right)}^{1/2}=O\left(T^{1/2}\right).\end{array}$$

(82)

Furthermore, we examine the second term of decomposition (79) and continue as per the proof of (53) and considering (81), one obtains

$$\begin{array}{c}\hfill {\mathrm{\Phi }}_2=O\left(T^{1/2}\right).\end{array}$$

(83)

therefore, combining Equations (82) and (83), we obtain

$$\begin{array}{c}\hfill {\left(E\left[{\left|\sum _{i=1}^n{\mathrm{\Phi }}_k\left({\mathbf{X}}_i;{\zeta }_{k,\ell }\right)\right|}^2\right]\right)}^{1/2}=O\left(T^{1/2}\right).\end{array}$$

Hence, there exists a positive constant C such that

$$\begin{array}{ccc}\hfill \mathbb{E}\left[|A_{k,\ell ,1}{|}^2\right]& =& \frac{1}{n^2}\mathbb{E}\left[{\left|\sum _{i=1}^n{\mathrm{\Phi }}_k\left({\mathbf{x}}_i;{\zeta }_{k,\ell }\right)\right|}^2\right]\hfill \\ & =& \frac{{\delta }^2}{T^2}O\left(T\right)\hfill \\ & \le & C\left({\displaystyle \frac{lnT}{T}}\right).\hfill \end{array}$$

(84)

Therefore, combining Equations (78) and (84) there exists a constant C such as

$$\mathbb{E}\left({\left|{\widehat{\alpha }}_{k,\ell }-{\alpha }_{k,\ell }\right|}^2\right)\le C\left(\frac{lnT}{T}\right).$$

(85)

Thus, the proof of Lemma 6 is completed. □

 Proof of Lemma 7. 

Consider the subsequent decomposition

$$\begin{array}{ccc}\hfill {\widehat{\theta }}_{k,\ell }-{\theta }_{k,\ell }& =& {\widehat{\theta }}_{k,\ell }-{\breve{\theta }}_{k,\ell }+{\breve{\theta }}_{k,\ell }-{\theta }_{k,\ell }\hfill \\ & =& B_{k,\ell ,1}+B_{k,\ell ,2},\hfill \end{array}$$

(86)

where

$${\breve{\theta }}_{k,\ell }=\frac{1}{T}{\int }_0^T\mathbb{E}\left[\frac{\rho \left(Y_t\right)}{f\left({\mathbf{X}}_t\right)}{\psi }_k\left({\mathbf{X}}_t;{\eta }_{k,\ell }\right)|{\mathcal{F}}_{t-\delta }\right]dt.$$

Remark that, under conditions (M.1), (M.2), and (C.1), Equation (76), and using the fact that $\mathcal{E}$ is an orthonormal basis of H, by reasoning as in Equation (77), we obtain

$${\breve{\theta }}_{k,\ell }={\theta }_{k,\ell },\mathrm{a}\mathrm{s},n\to \infty ,$$

(87)

giving that

$$B_{k,\ell ,2}=o\left(1\right),\mathrm{a}.\mathrm{s}.$$

(88)

Therefore, we have

$$\begin{array}{ccc}\hfill {\widehat{\theta }}_{k,\ell }-{\theta }_{k,\ell }& =& B_{k,\ell ,1}+o\left(1\right),\mathrm{a}.\mathrm{s}.\hfill \end{array}$$

Hence we obtain

$$\begin{array}{ccc}\hfill \mathbb{E}\left({\left|{\widehat{\theta }}_{k,\ell }-{\theta }_{k,\ell }\right|}^4\right)& =& \frac{1}{n^4}\mathbb{E}\left({\left|\sum _{i=1}^n{\mathrm{\Psi }}_{i,k,\ell }\right|}^4\right),\hfill \end{array}$$

(89)

where

$${\mathrm{\Psi }}_{i,k,\ell }=\frac{1}{\delta }{\int }_{T_{i-1}}^{T_i}\left(\frac{Y_t}{f\left({\mathbf{X}}_t\right)}{\psi }_k\left({\mathbf{X}}_t;{\eta }_{k,\ell }\right)-\mathbb{E}\left[\frac{Y_i}{f\left({\mathbf{x}}_t\right)}{\psi }_k\left({\mathbf{x}}_t;{\eta }_{k,\ell }\right)|{\mathcal{F}}_{t-\delta }\right]\right)dt.$$

Notice that ${\left({\mathrm{\Psi }}_{i,k,\ell }\right)}_{0\le k\le n}$ is a sequence of martingale differences with respect to the sequence of $\sigma -$fields ${\left({\mathcal{F}}_{i-1}\right)}_{0\le k\le n}$, applying the Burkholder–Rosenthal inequality (see Lemma 1), we obtain

$$\begin{array}{ccc}\hfill {\left(\mathbb{E}\left({\left|\sum _{i=1}^n{\mathrm{\Psi }}_{i,k,\ell }\right|}^4\right)\right)}^{1/4}& \le & {∥\underset{1\le j\le n}{max}\left|\sum _{i=1}^j{\mathrm{\Psi }}_{i,k,\ell }\right|∥}_4\hfill \\ & \ll & n^{1/4}{\parallel {\mathrm{\Psi }}_{1,k,\ell }\parallel }_4+{∥\sum _{i=1}^n\mathbb{E}\left({\mathrm{\Psi }}_{i,k,\ell }^2\right|{\mathcal{F}}_{i-2})∥}_{4/2}^{1/2}\hfill \\ & =& {\mathrm{\Psi }}_{k,\ell }^{\left(1\right)}+{\mathrm{\Psi }}_{k,\ell }^{\left(2\right)}.\hfill \end{array}$$

(90)

Consider the first term of the Equation (90). Applying Jensen and Minkowski’s inequalities, we have

$$\begin{array}{c}{\displaystyle \frac{1}{n}{\left({\mathrm{\Psi }}_{k,\ell }^{\left(1\right)}\right)}^4={\parallel {\mathrm{\Psi }}_{1,k,\ell }\parallel }_4^4}\hfill \\ =\mathbb{E}\left({\left|\frac{1}{\delta }{\int }_0^{\delta }\left(\frac{\rho \left(Y_t\right)}{f\left({\mathbf{X}}_t\right)}{\psi }_k\left({\mathbf{X}}_t;{\eta }_{k,\ell }\right)-\mathbb{E}\left[\frac{\rho \left(Y_t\right)}{f\left({\mathbf{X}}_t\right)}{\psi }_k\left({\mathbf{X}}_t;{\eta }_{k,\ell }\right)|{\mathcal{F}}_{t-\delta }\right]\right)dt\right|}^4\right)\hfill \\ \le \frac{1}{{\delta }^4}{\int }_0^{\delta }\mathbb{E}\left[{\left(\left|\frac{\rho \left(Y_t\right)}{f\left({\mathbf{X}}_t\right)}{\psi }_k\left({\mathbf{X}}_t;{\eta }_{k,\ell }\right)\right|+\mathbb{E}\left[\left|\frac{\rho \left(Y_t\right)}{f\left({\mathbf{X}}_t\right)}{\psi }_k\left({\mathbf{X}}_t;{\eta }_{k,\ell }\right)\right||{\mathcal{F}}_{t-\delta }\right]\right)}^4\right]\hfill \\ \le \frac{1}{{\delta }^4}{\int }_0^{\delta }{\left({\mathbb{E}}^{1/4}\left[{\left|\frac{\rho \left(Y_t\right)}{f\left({\mathbf{X}}_t\right)}{\varphi }_k\left({\mathbf{X}}_t;{\zeta }_{k,\ell }\right)\right|}^4\right]+{\mathbb{E}}^{1/4}\left[\mathbb{E}\left[{\left|\frac{\rho \left(Y_t\right)}{f\left({\mathbf{X}}_t\right)}{\varphi }_k\left({\mathbf{X}}_t;{\zeta }_{k,\ell }\right)\right|}^4|{\mathcal{F}}_{t-\delta }\right]\right]\right)}^{4}dt\hfill \\ \le \frac{16}{{\delta }^4}{\int }_0^{\delta }\mathbb{E}\left[{\left|\frac{\rho \left(Y_t\right)}{f\left({\mathbf{X}}_t\right)}{\varphi }_k\left({\mathbf{X}}_t;{\zeta }_{k,\ell }\right)\right|}^4\right]dt.\hfill \end{array}$$

(91)

By proceeding as in Equation (81) and under the same conditions (M.1), (M.2), and (E.1)(i), for all $0\le t\le T$, we obtain

$$\mathbb{E}\left[{\left|\frac{\rho \left(Y_t\right)}{f\left({\mathbf{X}}_t\right)}{\psi }_k\left({\mathbf{X}}_t;{\eta }_{k,\ell }\right)\right|}^2\right]\le C,$$

(92)

where C is a positive constant. Moreover, by the Cauchy–Schwarz inequality in connection with conditions (E.1)(ii) and (E.2) and assumption (24), we have

$$\begin{array}{ccc}\hfill \underset{\mathbf{x}\in \mathbf{S}}{sup}\left|{\psi }_k\left(\mathbf{x};{\eta }_{k,\ell }\right)\right|& \le & \underset{\mathbf{x}\in \mathbf{S}}{sup}\sum _{j\in {\mathcal{J}}_k}\frac{1}{\sqrt{h_{j,k,\ell }}}|e_j\left({\eta }_{k,\ell }\right)\left|\right|e_j\left(\mathbf{x}\right)|\hfill \\ \hfill & \le & {\left(\sum _{j\in {\mathcal{J}}_k}\frac{1}{h_{j,k,\ell }}{\left|e_j\left({\eta }_{k,\ell }\right)\right|}^2\right)}^{1/2}{\left(\underset{\mathbf{x}\in \mathbf{S}}{sup}\sum _{j\in {\mathcal{J}}_k}{\left|e_j\left(\mathbf{x}\right)\right|}^2\right)}^{1/2}\hfill \\ & \le & C_1^{1/2}{C}_2^{1/2}\sqrt{|{\mathcal{J}}_k|}\hfill \\ \hfill & \le & C_3\sqrt{|{\mathcal{J}}_{m_T}|}\hfill \\ & \le & C_3\sqrt{\frac{T}{{\left(lnT\right)}^2}}.\hfill \end{array}$$

(93)

From statements (92) and (93), we deduce that

$$\begin{array}{ccc}\hfill \frac{1}{n}{\left({\mathrm{\Psi }}_{k,\ell }^{\left(1\right)}\right)}^4& \le & C_4\left(\frac{T}{{\left(lnT\right)}^2}\right).\hfill \end{array}$$

It follows that

$${\mathrm{\Psi }}_{k,\ell }^{\left(1\right)}\le \frac{C_4}{\delta }{\left(\frac{T}{lnT}\right)}^{1/2}.$$

(94)

Let us now examine the upper bound of ${\mathrm{\Psi }}_{k,\ell }^{\left(2\right)}$ of Equation (59). Remark that

$$\begin{array}{ccc}\hfill {\mathrm{\Psi }}_{k,\ell }^{\left(2\right)}& =& {∥\sum _{i=1}^n\mathbb{E}({\mathrm{\Psi }}_{i,k,\ell }^2/{\mathcal{F}}_{i-2})∥}_2^{1/2}\hfill \\ & =& {\left(\mathbb{E}\left[{\left(\sum _{i=1}^n\mathbb{E}\left[{\mathrm{\Psi }}_{i,k,\ell }^2|{\mathcal{F}}_{i-2}\right]\right)}^2\right]\right)}^{1/4},\hfill \end{array}$$

Observe that for all $T_{i-1}\le t\le T_i$ for all $i=1,\dots ,n$; we have

$${\mathcal{F}}_{i-2}\subset {\mathcal{F}}_{t-\delta }\subset {\mathcal{S}}_{t-\delta }.$$

Making use of Jensen and Minkowski’s inequality in the same manner as in Equation (91), it follows

$$\begin{array}{c}{\displaystyle \sum _{i=1}^n\mathbb{E}\left[{\mathrm{\Psi }}_{i,k,\ell }^2|{\mathcal{F}}_{i-2}\right]}\hfill \\ =\sum _{i=1}^n\left(\mathbb{E}\left[{\left(\frac{1}{\delta }{\int }_{T_{i-1}}^{T_i}\left(\frac{\rho \left(Y_t\right)}{f\left({\mathbf{X}}_t\right)}{\psi }_k\left({\mathbf{x}}_t;{\eta }_{k,\ell }\right)-\mathbb{E}\left[\frac{\rho \left(Y_t\right)}{f\left({\mathbf{x}}_t\right)}{\psi }_k\left({\mathbf{X}}_t;{\eta }_{k,\ell }\right)|{\mathcal{F}}_{t-\delta }\right]\right)dt\right)}^2|{\mathcal{F}}_{i-2}\right]\right)\hfill \\ \le \frac{4}{{\delta }^2}\sum _{i=1}^n{\int }_{T_{i-1}}^{T_i}\mathbb{E}\left[{\left(\frac{\rho \left(Y_t\right)}{f\left({\mathbf{X}}_t\right)}{\psi }_k\left({\mathbf{X}}_t;{\eta }_{k,\ell }\right)\right)}^2|{\mathcal{F}}_{i-2}\right]dt.\hfill \end{array}$$

From the independence between ${ϵ}_t$ and ${\mathbf{X}}_t$, for all $T_{i-1}\le t\le T_i$, we have

$$\begin{array}{ccc}\hfill \mathbb{E}\left[{ϵ}_t^2|{\mathcal{S}}_{t-\delta }\right]& =& \mathbb{E}\left[{ϵ}_t^2|{\mathbf{X}}_t\right]\hfill \\ \hfill & =& \mathbb{E}\left[{ϵ}_t^2\right]=1.\hfill \end{array}$$

(95)

Under conditions (M.1), (M.2), (E.1)(i), and (C.1) and Equations (61) and (95), we obtain

$$\begin{array}{c}{\displaystyle \frac{4}{{\delta }^2}\sum _{i=1}^n{\int }_{T_{i-1}}^{T_i}\mathbb{E}\left[{\left(\frac{\rho \left(Y_t\right)}{f\left({\mathbf{X}}_t\right)}{\psi }_k\left({\mathbf{X}}_t;{\eta }_{k,\ell }\right)\right)}^2|{\mathcal{F}}_{i-2}\right]dt}\hfill \\ =\frac{4}{{\delta }^2}\sum _{i=1}^n\mathbb{E}\left[\mathbb{E}\left[{\left(\frac{\rho \left(Y_i\right)}{f\left({\mathbf{X}}_i\right)}{\psi }_k\left({\mathbf{X}}_i;{\eta }_{k,\ell }\right)\right)}^2|{\mathcal{S}}_{t-\delta }\right]|{\mathcal{F}}_{i-2}\right]\hfill \\ \le \frac{4n\left(C_m^2+1\right)}{{\delta }^2{C}_f}{\int }_{\mathbf{S}}{\left|{\psi }_k\left(\mathbf{x};{\eta }_{k,\ell }\right)\right|}^2\left(\frac{\frac{1}{n}{\sum }_{i=1}^n{\int }_{T_{i-1}}^{T_i}{f}_t^{{\mathcal{F}}_{i-2}}\left(\mathbf{x}\right)dt}{f\left(\mathbf{x}\right)}\right)\nu \left(d\mathbf{x}\right).\hfill \end{array}$$

For all $T_{i-1}\le t\le T_i$, using the stationarity of the process ${\left({\mathbf{X}}_t\right)}_{t\ge 0}$, we have

$$f_t^{{\mathcal{F}}_{i-2}}\left(\mathbf{x}\right)=f_{T_{i-1}}^{{\mathcal{F}}_{i-2}}\left(\mathbf{x}\right).$$

We deduce

$$\begin{array}{c}{\displaystyle \frac{4}{{\delta }^2}\sum _{i=1}^n{\int }_{T_{i-1}}^{T_i}\mathbb{E}\left[{\left(\frac{\rho \left(Y_t\right)}{f\left({\mathbf{X}}_t\right)}{\psi }_k\left({\mathbf{X}}_t;{\eta }_{k,\ell }\right)\right)}^2|{\mathcal{F}}_{i-2}\right]dt}\hfill \\ \le \frac{4\delta n\left(C_m^2+1\right)}{{\delta }^2{C}_f}{\int }_{\mathbf{S}}{\left|{\psi }_k\left(\mathbf{x};{\eta }_{k,\ell }\right)\right|}^2\left(\frac{\frac{1}{n}{\sum }_{i=1}^n{f}_{T_{i-1}}^{{\mathcal{F}}_{i-2}}\left(\mathbf{x}\right)}{f\left(\mathbf{x}\right)}\right)\nu \left(d\mathbf{x}\right)\hfill \\ =\frac{T\left(C_m^2+1\right)}{{\delta }^2{C}_f}(1+o\left(1\right)){\int }_{\mathbf{S}}{\left|{\psi }_k\left(\mathbf{x};{\eta }_{k,\ell }\right)\right|}^2\nu \left(d\mathbf{x}\right)\hfill \\ \le C_{5}T,\hfill \end{array}$$

(96)

where

$$C_5=\frac{\left(C_m^2+1\right)}{{\delta }^2{C}_f}(1+o\left(1\right)).$$

It follows

$$\begin{array}{ccc}\hfill {\mathrm{\Psi }}_{k,\ell }^{\left(2\right)}& =& C{T}^{1/2}.\hfill \end{array}$$

(97)

Combining Equations (59), (64), and (66), we obtain

$$\begin{array}{ccc}\hfill \mathbb{E}\left({\left|\sum _{i=1}^n{\mathrm{\Psi }}_{i,k,\ell }\right|}^4\right)& =& O\left({\left(\frac{T}{lnT}\right)}^2\right)+O\left(T^2\right),\hfill \end{array}$$

combining this with Equation (89), we conclude

$$\begin{array}{ccc}\hfill \mathbb{E}\left({\left|{\widehat{\theta }}_{k,\ell }-{\theta }_{k,\ell }\right|}^4\right)& =& O\left(\left(\frac{{\delta }^4}{T^4}\right){\left(\frac{T}{lnT}\right)}^2\right)+O\left(\left(\frac{{\delta }^4}{T^4}\right)T^2\right).\hfill \end{array}$$

(98)

Hence there exists a constant $C>0$, such that

$$\begin{array}{ccc}\hfill \mathbb{E}\left({\left|{\widehat{\theta }}_{k,\ell }-{\theta }_{k,\ell }\right|}^4\right)& \le & C{\left(\frac{1}{T}\right)}^2\le C{\left(\frac{lnT}{T}\right)}^2.\hfill \end{array}$$

(99)

The proof is achieved. □

 Proof of Lemma 8. 

Consider decomposition (86) in Lemma 7, we have

$$\begin{array}{ccc}\hfill {\widehat{\theta }}_{k,\ell }-{\theta }_{k,\ell }& =& \left({\widehat{\theta }}_{k,\ell }-{\breve{\theta }}_{k,\ell }\right)+\left({\breve{\theta }}_{k,\ell }-{\theta }_{k,\ell }\right)\hfill \\ & =& B_{k,\ell ,1}+B_{k,\ell ,2},\hfill \end{array}$$

where

$$\begin{array}{ccc}\hfill B_{k,\ell ,1}& =& \frac{1}{n}\sum _{i=1}^n{\mathrm{\Psi }}_{i,k,\ell }\hfill \\ \hfill & =& \frac{1}{n}\sum _{i=1}^n\left(\frac{1}{\delta }{\int }_{T_{i-1}}^{T_i}\left(\frac{\rho \left(Y_t\right)}{f\left({\mathbf{X}}_t\right)}{\psi }_k\left({\mathbf{X}}_t;{\eta }_{k,\ell }\right)-\mathbb{E}\left[\frac{\rho \left(Y_t\right)}{f\left({\mathbf{X}}_i\right)}{\psi }_k\left({\mathbf{X}}_t;{\eta }_{k,\ell }\right)|{\mathcal{F}}_{t-\delta }\right]\right)dt\right),\hfill \end{array}$$

(100)

$$\begin{array}{ccc}\hfill B_{k,\ell ,2}& =& \frac{1}{n}\sum _{i=1}^n\frac{1}{\delta }{\int }_{T_{i-1}}^{T_i}\left(\mathbb{E}\left[\frac{\rho \left(Y_t\right)}{f\left({\mathbf{X}}_t\right)}{\psi }_k\left({\mathbf{X}}_t;{\eta }_{k,\ell }\right)|{\mathcal{F}}_{t-\delta }\right]-{\theta }_{k,\ell }\right)dt.\hfill \end{array}$$

(101)

Statement (88) achieves the desired result for the term $B_{k,\ell ,2}$

$$\begin{array}{ccc}\hfill {\widehat{\theta }}_{k,\ell }-{\theta }_{k,\ell }& =& B_{k,\ell ,1}+o\left(1\right).\hfill \end{array}$$

We consider the next decomposition

$$\begin{array}{ccc}\hfill {\mathrm{\Psi }}_{i,k,\ell }& =& V_{i,k,\ell }+W_{i,k,\ell },\hfill \end{array}$$

(102)

where

$$V_{i,k,\ell }=\frac{1}{\delta }{\int }_{T_{i-1}}^{T_i}\left(\frac{\rho \left(Y_t\right)}{f\left({\mathbf{X}}_t\right)}{\psi }_k\left({\mathbf{x}}_t;{\eta }_{k,\ell }\right){\mathrm{𝟙}}_{{\mathcal{A}}_t}-\mathbb{E}\left[\frac{\rho \left(Y_t\right)}{f\left({\mathbf{X}}_t\right)}{\psi }_k\left({\mathbf{X}}_t;{\eta }_{k,\ell }\right){\mathrm{𝟙}}_{{\mathcal{A}}_t}|{\mathcal{F}}_{t-\delta }\right]\right)dt,$$

$$W_{i,k,\ell }=\frac{1}{\delta }{\int }_{T_{i-1}}^{T_i}\left(\frac{\rho \left(Y_t\right)}{f\left({\mathbf{X}}_t\right)}{\psi }_k\left({\mathbf{X}}_t;{\eta }_{k,\ell }\right){\mathrm{𝟙}}_{{\mathcal{A}}_t}^c-\mathbb{E}\left[\frac{\rho \left(Y_t\right)}{f\left({\mathbf{X}}_t\right)}{\psi }_k\left({\mathbf{X}}_t;{\eta }_{k,\ell }\right){\mathrm{𝟙}}_{{\mathcal{A}}_t}^c|{\mathcal{F}}_{t-\delta }\right]\right)dt,$$

and

$${\mathrm{𝟙}}_{{\mathcal{A}}_t}=\left\{|{ϵ}_t|\ge c_{*}\sqrt{lnT}\right\},$$

and $c_{*}$ represents a constant that will be selected later. Remark that

$$\begin{array}{ccc}\hfill \mathbb{P}\left(\left|{\widehat{\theta }}_{k,\ell }-{\breve{\theta }}_{k,\ell }\right|\ge \frac{\kappa }{2}\sqrt{\frac{lnT}{T}}\right)& \le & \mathbb{P}\left(\left|B_{k,\ell ,1}\right|\ge \frac{\kappa }{2}\sqrt{\frac{lnT}{T}}\right)+o\left(1\right)\hfill \\ & =& \mathbb{P}\left(\left|\sum _{i=1}^n{\mathrm{\Psi }}_{i,k,\ell }\right|\ge \frac{\kappa }{2}\sqrt{TlnT}\right)+o\left(1\right)\hfill \\ & =& I_1+I_2+o\left(1\right),\hfill \end{array}$$

(103)

where

$$\begin{array}{ccc}\hfill I_1& =& \mathbb{P}\left(\left|\sum _{i=1}^n{V}_{i,k,\ell }\right|\ge \frac{\kappa }{2}\sqrt{TlnT}\right),\hfill \\ \hfill I_2& =& \mathbb{P}\left(\left|\sum _{i=1}^n{W}_{i,k,\ell }\right|\ge \frac{\kappa }{2}\sqrt{TlnT}\right).\hfill \end{array}$$

First, we aim to bound the term $I_1$ of Equation (103). The Markov inequality and the Cauchy–Schwarz inequality yield

$$\begin{array}{ccc}\hfill I_1& \le & \frac{2}{\kappa \sqrt{TlnT}}\mathbb{E}\left(\left|\sum _{i=1}^n{V}_{i,k,\ell }\right|\right)\hfill \\ & \le & \frac{2}{\kappa \sqrt{TlnT}}\sum _{i=1}^n\mathbb{E}\left(\left|V_{i,k,\ell }\right|\right).\hfill \end{array}$$

(104)

Observe that

$$\begin{array}{c}{\displaystyle \mathbb{E}\left(\left|V_{i,k,\ell }\right|\right)}\hfill \\ \le \frac{1}{\delta }{\int }_{T_{i-1}}^{T_i}\mathbb{E}\left(\left|\frac{\rho \left(Y_t\right)}{f\left({\mathbf{X}}_t\right)}{\psi }_k\left({\mathbf{X}}_t;{\eta }_{k,\ell }\right){\mathrm{𝟙}}_{{\mathcal{A}}_t}-\mathbb{E}\left[\frac{\rho \left(Y_t\right)}{f\left({\mathbf{X}}_t\right)}{\psi }_k\left({\mathbf{X}}_t;{\eta }_{k,\ell }\right){\mathrm{𝟙}}_{{\mathcal{A}}_t}|{\mathcal{F}}_{t-\delta }\right]\right|\right)dt\hfill \\ \le \frac{1}{\delta }{\int }_{T_{i-1}}^{T_i}\left(\mathbb{E}\left(\left|\frac{\rho \left(Y_t\right)}{f\left({\mathbf{X}}_t\right)}{\psi }_k\left({\mathbf{X}}_t;{\eta }_{k,\ell }\right){\mathrm{𝟙}}_{{\mathcal{A}}_t}\right|\right)+\mathbb{E}\left(\mathbb{E}\left[\left|\frac{\rho \left(Y_t\right)}{f\left({\mathbf{X}}_t\right)}{\psi }_k\left({\mathbf{X}}_t;{\eta }_{k,\ell }\right){\mathrm{𝟙}}_{{\mathcal{A}}_t}\right||{\mathcal{F}}_{t-\delta }\right]\right)\right)dt\hfill \\ \le \frac{2}{\delta }{\int }_{T_{i-1}}^{T_i}\mathbb{E}\left(\left|\frac{\rho \left(Y_t\right)}{f\left({\mathbf{X}}_t\right)}{\psi }_k\left({\mathbf{X}}_t;{\eta }_{k,\ell }\right){\mathrm{𝟙}}_{{\mathcal{A}}_t}\right|\right)dt\hfill \\ \le \frac{2}{\delta }{\left(\mathbb{E}\left({\left|\frac{\rho \left(Y_t\right)}{f\left({\mathbf{X}}_t\right)}{\psi }_k\left({\mathbf{X}}_t;{\eta }_{k,\ell }\right)\right|}^2\right)\right)}^{1/2}{\left(\mathbb{P}\left({\mathcal{A}}_t\right)\right)}^{1/2}.\hfill \end{array}$$

(105)

Using Equation (92) combined with an elementary Gaussian inequality and taking $c_{*}$ to have

$$\frac{c_{*}^2}{4}-1/2=2.$$

We obtain

$$\begin{array}{ccc}\hfill I_1& \le & \frac{2C}{\delta \kappa }\sqrt{\frac{T}{lnT}}exp\left\{-\frac{c_{*}^{2}lnT}{4}\right\}\hfill \\ & \le & \frac{2C}{\delta \kappa }\frac{T^{-\left(\frac{c_{*}^2}{4}-1/2\right)}}{\sqrt{lnT}}\hfill \\ & \le & C_{\kappa }\left(\frac{1}{T^2}\right),\hfill \end{array}$$

(106)

where $C_{\kappa }=\frac{2C}{\delta \kappa }.$ Now, we are going to look into determining an upper bound for $I_2$ from decomposition (103). We begin by confirming Lemma’s 2 conditions. Assume that conditions (M.1) and (M.2) are fulfilled in connection with Equation (93), we infer

$$\begin{array}{ccc}\hfill |Y_t{\mathrm{𝟙}}_{{\mathcal{A}}_t^c}|& \le & C_m+c_{*}\sqrt{lnT}\hfill \\ \hfill & \le & C\sqrt{lnT},\hfill \end{array}$$

(107)

which implies

$$\begin{array}{ccc}\hfill \left|W_{i,k,\ell }\right|& \le & \frac{1}{\delta }{\int }_{T_{i-1}}^{T_i}\left|\frac{\rho \left(Y_t\right)}{f\left({\mathbf{X}}_t\right)}{\psi }_k\left({\mathbf{X}}_t;{\eta }_{k,\ell }\right){\mathrm{𝟙}}_{{\mathcal{A}}_t}^c-\mathbb{E}\left[\frac{\rho \left(Y_t\right)}{f\left({\mathbf{X}}_t\right)}{\psi }_k\left({\mathbf{X}}_t;{\eta }_{k,\ell }\right){\mathrm{𝟙}}_{{\mathcal{A}}_t}^c|{\mathcal{F}}_{t-\delta }\right]\right|dt\hfill \\ \hfill & \le & \frac{2C\sqrt{lnT}}{\delta C}\underset{\mathbf{x}\in \mathbf{S}}{sup}\left|{\psi }_k\left(\mathbf{x};{\eta }_{k,\ell }\right)\right|\hfill \\ & \le & \frac{2C}{\delta C_f}\sqrt{lnT}\sqrt{\frac{T}{{\left(lnT\right)}^2}}\hfill \\ \hfill & \le & C_3\sqrt{\frac{T}{lnT}}\hfill \\ & \le & C_3\sqrt{T},\hfill \end{array}$$

(108)

where $C_3=\frac{2C}{\delta C_f}$, let $B=C_3\sqrt{T}$, then, for all ${ϵ}_T=\frac{\kappa }{2}\sqrt{TlnT}$ with a sufficiently large T, we have

$$\begin{array}{ccc}\hfill I_2=\mathbb{P}\left(\left|\sum _{i=1}^n{W}_{i,k,\ell }\right|\ge \frac{\kappa }{2}\sqrt{TlnT}\right)& \le & 2exp\left\{-\frac{{ϵ}_T^2}{2T{B}^2}\right\}\hfill \\ \hfill & =& 2exp\left\{-\frac{{\left(\frac{\kappa }{2}\sqrt{TlnT}\right)}^2}{2T{\left(C_3\sqrt{T}\right)}^2}\right\}\hfill \\ & =& 2exp\left\{-\frac{{\kappa }^{2}lnT}{4C_3^{2}T}\right\}\hfill \\ \hfill & =& 2exp\left\{ln{T}^{-\frac{{\kappa }^2}{4C_3^{2}T}}\right\}\hfill \\ & =& 2T^{-w(\kappa ,T)},\hfill \end{array}$$

(109)

where

$$w(\kappa ,T)=\frac{{\kappa }^2}{4C_3^{2}T}.$$

Taking $\kappa$ such that $w(n,\kappa )=2,$ we have

$$\begin{array}{ccc}\hfill I_2& \le & C\left(\frac{1}{T^2}\right).\hfill \end{array}$$

(110)

It follows from Equations (103), (106), and (110) that

$$\begin{array}{ccc}\hfill \mathbb{P}\left(\left|{\widehat{\theta }}_{k,\ell }-{\tilde{\theta }}_{k,\ell }\right|\ge \frac{\kappa }{2}\sqrt{\frac{lnT}{T}}\right)& \le & C\frac{1}{T^2}+o\left(1\right)\hfill \end{array}$$

(111)

$$\begin{array}{c}\hfill \le C{\left(\frac{lnT}{T}\right)}^2.\end{array}$$

(112)

The proof of Equation (74) is achieved. □

8. Besov Spaces

In terms of wavelet coefficients, in [23], the Besov space was described as follows: $0<s<r$ is a real-valued smoothness parameter of $\varrho \in L_p\left({\mathbb{R}}^d\right)$, then $\varrho \in {\mathcal{B}}_{p,q}^s\left({\mathbb{R}}^d\right)$ equivalent to

 (B.1) 

$J_{s,p,q}\left(\varrho \right)={\parallel P_{V_0}\varrho \parallel }_{L_p}+{\left({\sum }_{j>0}{\left(2^{js}{\parallel P_{W_j}\varrho \parallel }_{L_p}\right)}^q\right)}^{1/q}<\infty ,$

 (B.2) 

$J_{s,p,q}^{\prime }\left(\varrho \right)={\parallel a_0·\parallel }_{l_p}+{\left({\sum }_{j>0}{\left(2^{j(s+d(1/2-1/p\left)\right)}{\parallel b_j·\parallel }_{l_p}\right)}^q\right)}^{1/q}<\infty$

with

$$\parallel a_0{·\parallel }_{l_p}={\left(\sum _{\mathbf{k}\in {\mathbb{Z}}^d}{\left|a_{0,\mathbf{k}}\right|}^p\right)}^{1/p}$$

and

$$\parallel b_j{·\parallel }_{l_p}={\left(\sum _{i=1}^N\sum _{\mathbf{k}\in {\mathbb{Z}}^d}{\left|b_{i,j,\mathbf{k}}\right|}^p\right)}^{1/p}$$

and classical sup-norm modification for $q=\infty$. The Besov spaces are useful for describing the smoothness properties of functional estimation and approximation theory. They contain well-known statistical research spaces, such as the Hilbert–Sobolev space $\left(H_2^s={\mathcal{B}}_{2,2}^s\left({\mathbb{R}}^d\right)\right)$, Holder space $C^s={\mathcal{B}}_{\infty ,\infty }^s\left({\mathbb{R}}^d\right)$ for $0<s\notin \mathbb{N}$, and others. We refer readers to [58] for additional descriptions of Besov space and its merits in approximation theory and statistics. Reasoning as in [102], suppose that $1\le p\le \infty ,1\le q\le \infty .$ Let

$$\left(S_{\tau }f\right)\left(\mathbf{x}\right)=f(\mathbf{x}-\tau ).$$

For $0<s<1$, set

$$\begin{array}{cc}\hfill {\gamma }_{s,p,q}\left(f\right)& ={\left({\int }_{{\mathbb{R}}^d}{\left(\frac{{∥S_{\tau }f-f∥}_{L_p}}{{\parallel \tau \parallel }^s}\right)}^q\frac{d\tau }{{\parallel \tau \parallel }^d}\right)}^{1/q}\hfill \\ \hfill {\gamma }_{s,p,\infty }& =\underset{\tau \in {\mathbb{R}}^d}{sup}\frac{{∥S_{\tau }f-f∥}_{L_p}}{{\parallel \tau \parallel }^S}\hfill \end{array}$$

For $s=1,$ set

$$\begin{array}{cc}\hfill {\gamma }_{1,p,q}\left(f\right)& ={\left({\int }_{{\mathbb{R}}^d}{\left(\frac{{∥S_{\tau }f+S_{-\tau }f-2f∥}_{L_p}}{\parallel \tau \parallel }\right)}^q\frac{d\tau }{{\parallel \tau \parallel }^d}\right)}^{1/q},\hfill \\ \hfill {\gamma }_{1,p,\infty }& =\underset{\tau \in {\mathbb{R}}^d}{sup}\frac{{∥S_{\tau }f+S_{-\tau }f-2f∥}_{L_p}}{\parallel \tau \parallel }.\hfill \end{array}$$

For $0<s<1$ and $1\le p,q\le \infty ,$ define

$${\mathcal{B}}_{p,q}^s\left({\mathbb{R}}^d\right)=\left\{f\in L_p:{\gamma }_{s,p,q}<\infty \right\}.$$

For $s>1,$ put $s={\left[s\right]}^{-}+{\left\{s\right\}}^{+}$ with ${\left[s\right]}^{-}$ an integer and $0<{\left\{s\right\}}^{+}\le 1.$ Define ${\mathcal{B}}_{p,q}^s\left({\mathbb{R}}^d\right)$ to be the space of functions in $L_p\left({\mathbb{R}}^d\right)$ such that $D^{j}f\in {\mathcal{B}}_{{\left\{s\right\}}^{+},p,q}$ for all $\left|j\right|\le {\left[s\right]}^{-}.$ The norm is defined by

$${\parallel f\parallel }_{{\mathcal{B}}_{p,q}^s\left({\mathbb{R}}^d\right)}={\parallel f\parallel }_{L_p}+\sum _{{\left|j\right|\le s\}}^{-}}{\gamma }_{{\left\{s\right\}}^{+},p,q}\left(D^{j}f\right).$$