Limit Theorems in the Nonparametric Conditional Single-Index U-Processes for Locally Stationary Functional Random Fields under Stochastic Sampling Design

Abstract

In his work published in (Ann. Probab. 19, No. 2 (1991), 812–825), W. Stute introduced the notion of conditional U-statistics, expanding upon the Nadaraya–Watson estimates used for regression functions. Stute illustrated the pointwise consistency and asymptotic normality of these statistics. Our research extends these concepts to a broader scope, establishing, for the first time, an asymptotic framework for single-index conditional U-statistics applicable to locally stationary random fields $\{{\mathbf{X}}_{s,A_n}:sin{R}_n\}$ observed at irregularly spaced locations in $R_n$, a subset of ${\mathbb{R}}^d$. We introduce an estimator for the single-index conditional U-statistics operator that accommodates the nonstationary nature of the data-generating process. Our method employs a stochastic sampling approach that allows for the flexible creation of irregularly spaced sampling sites, covering both pure and mixed increasing domain frameworks. We establish the uniform convergence rate and weak convergence of the single conditional U-processes. Specifically, we examine weak convergence under bounded or unbounded function classes that satisfy specific moment conditions. These findings are established under general structural conditions on the function classes and underlying models. The theoretical advancements outlined in this paper form essential foundations for potential breakthroughs in functional data analysis, laying the groundwork for future research in this field. Moreover, in the same context, we show the uniform consistency for the nonparametric inverse probability of censoring weighted (I.P.C.W.) estimators of the regression function under random censorship, which is of its own interest. Potential applications of our findings encompass, among many others, the set-indexed conditional U-statistics, the Kendall rank correlation coefficient, and the discrimination problems.

1. Introduction

Statisticians and probability theorists have thoroughly examined the regression problem, resulting in a diverse array of techniques. Extensive research has been conducted on various aspects, including modeling, estimation methods, applications, and tests. Nonparametric methods are particularly notable for their flexibility, as they do not rely on predefined structural information, unlike parametric methods, which estimate a specific number of parameters based on a predetermined model structure. However, nonparametric methods can suffer from estimation biases and slower convergence rates. Among these, nonparametric kernel function estimation techniques have garnered significant interest. For detailed references and insights into the research literature and statistical applications in this field, see [1,2,3,4] and other pertinent works. In addition to nonparametric methods, approaches such as neural networks, wavelet analysis, splines, and nearest-neighbor methods are widely employed for creating reliable function estimators. These methodologies are applied across various data types and domains. In spatial data analysis, for instance, this article discusses the construction of kernel-type estimators for conditional U-statistics estimation. Numerous scientific disciplines, including econometrics, epidemiology, environmental science, image analysis, oceanography, meteorology, and geostatistics, routinely generate spatial data that are statistically analyzed at specific measurement locations. For comprehensive information and insights into statistical applications in spatial data analysis, refer to the research literature such as [5,6,7,8], and the references within these works. Recent advancements in nonparametric estimation for spatial data primarily focus on regression function and probability density estimation. Key references in this domain include works by [9,10,11,12,13,14], and other relevant sources cited within these works.

This research introduces a more general and abstract framework by exploring conditional U-processes for locally stationary random fields. The concept of U-statistics and U-processes has garnered significant interest due to their wide-ranging applications, including density estimation, nonparametric regression tests, and goodness-of-fit tests. U-processes effectively address complex statistical challenges across various contexts, such as higher-order terms in von Mises expansions. Notably, U-statistics play a crucial role in analyzing estimators and function estimators with varying degrees of smoothness. For instance, [15] analyzed the product limit estimator for truncated data, employing almost sure uniform bounds for $\mathbb{P}$-canonical U-processes. Additionally, [16] introduced two novel normality tests based on U-processes, while [17] proposed new normality tests using weighted $L_1$-distances between the standard normal density and local U-statistics from standardized data. Further applications include the estimation of the mean of multivariate functions under possibly heavy-tailed distributions by [18]. For a comprehensive understanding of U-statistics and U-processes, see references such as [19,20,21]. The versatility of U-statistics extends to estimation and machine learning applications, with notable works on U-statistics with random kernels of divergent orders by [22,23,24]. Infinite-order U-statistics are particularly valuable for constructing simultaneous prediction intervals, crucial for quantifying uncertainty in ensemble methods like sub-bagging and random forests. Additionally, the MeanNN method estimation for differential entropy, introduced by [25], showcases the versatility of the U-statistic. Novel test statistics for goodness-of-fit tests using U-statistics were proposed by [26].

This study primarily focuses on situations involving locally stationary spatial–functional data. For a comprehensive understanding of functional data analysis, readers are encouraged to explore the foundational works [27,28], which offer diverse case studies from disciplines such as criminology, economics, archaeology, and neurophysiology. It is worth noting that the extension of probability theory to random variables taking values in normed vector spaces predates the recent literature on functional data. Works by [29,30] address challenges posed by functional data with infinite dimensions, while nonparametric models for regression estimation are discussed in works by [31,32,33]. In recent years, modern empirical process theory has found new applications in processing functional data. Ref. [34] established consistency rates for various conditional models, including regression functions and conditional cumulative distributions, uniformly over subsets of the explanatory variables. Additionally, ref. [35] demonstrated strong convergence rates for local linear estimation of the regression function with a functional regressor, uniformly across bandwidth parameters. In the domain of strong mixing functional time series data, ref. [36] explored the k-nearest neighbors (kNN) estimator of the nonparametric regression model, determining its uniform, almost complete convergence rate under mild conditions. Recent references on U-statistics include works by [37,38,39], among others.

Stute [40] introduced a set of estimators for $r^{\left(m\right)}(\phi ,\mathbf{t})$, known as conditional U-statistics, to extend Nadaraya–Watson regression function estimations. First, we present Stute’s estimators. Consider a regular sequence of random elements $\{({\mathbf{X}}_i,Y_i),i\in {\mathbb{N}}^{*}\}$, where ${\mathbf{X}}_i\in {\mathbb{R}}^d$, and $Y_i\in \mathcal{Y}$, a Polish space (a topological space $\mathcal{Y}$ is Polish if its topology can be defined by a metric $\delta$, for which it is complete and separable; this means there exists a countable dense subset of $\mathcal{Y}$ or a countable base of open sets (e.g., p. 209, [41])), and ${\mathbb{N}}^{*}=\mathbb{N}\setminus \left\{0\right\}$. Let $\phi :{\mathcal{Y}}^m\to \mathbb{R}$ be a measurable function. In this investigation, our primary focus is on estimating the conditional expectation or regression function

$$\begin{array}{c}\hfill r^{\left(m\right)}(\phi ,\mathbf{t})=\mathbb{E}\left(\phi (Y_1,\dots ,Y_m)\mid ({\mathbf{X}}_1,\dots ,{\mathbf{X}}_m)=\mathbf{t}\right),for\mathbf{t}\in {\mathbb{R}}^{dm},\end{array}$$

(1)

whenever it exists, i.e.,

$$\mathbb{E}\left(\left|\phi (Y_1,\dots ,Y_m)\right|\right)<\infty .$$

We now introduce a kernel function $K:{\mathbb{R}}^d\to \mathbb{R}$ with support contained in ${[-B,B]}^d$, $B>0,$ satisfying

$$\underset{\mathbf{x}\in {\mathbb{R}}^d}{sup}\left|K\left(\mathbf{x}\right)\right|=:\kappa <\infty and\int K\left(\mathbf{x}\right)d\mathbf{x}=1.$$

(2)

Hence, the class of estimators for $r^{\left(m\right)}(\phi ,\mathbf{t})$, given by [40], is defined, for each $\mathbf{t}\in {\mathbb{R}}^{dm}$, as follows:

$$\begin{array}{c}\hfill {\widehat{r}}_n^{\left(m\right)}(\phi ,\mathbf{t};h_n)=\frac{{\displaystyle \sum _{\mathbf{i}\in I_n^m}\phi (Y_{i_1},\dots ,Y_{i_m})K\left(\frac{{\mathbf{t}}_1-{\mathbf{X}}_{i_1}}{h_n}\right)\dots K\left(\frac{{\mathbf{t}}_m-{\mathbf{X}}_{i_m}}{h_n}\right)}}{{\displaystyle \sum _{\mathbf{i}\in I_n^m}K\left(\frac{{\mathbf{t}}_1-{\mathbf{X}}_{i_1}}{h_n}\right)\dots K\left(\frac{{\mathbf{t}}_m-{\mathbf{X}}_{i_m}}{h_n}\right)}},\end{array}$$

(3)

where

$$I_n^m=\left\{\mathbf{i}=(i_1,\dots ,i_m):1\le i_j\le nand{i}_j\ne i_{r}ifj\ne r\right\},$$

denotes the set of all m-tuples of different integers $i_j$ between 1 and n and ${\{h_n:=h_n\}}_{n\ge 1}$ is a sequence of positive constants that converge to zero with rate $n{h}_n^m\to \infty$. For $m=1$, the $r^{\left(m\right)}(\phi ,t)$ becomes $r^{\left(1\right)}(\phi ,t)=\mathbb{E}\left(\phi \left(\mathbf{Y}\right)\right|\mathbf{X}=t)$, and Stute’s estimate transforms into the Nadaraya–Watson estimator of $r^{\left(1\right)}(\phi ,t)$. In this framework, Sen [42] aimed to estimate the rate of uniform convergence in $\mathbf{t}$ of ${\widehat{r}}_n^{\left(m\right)}(\phi ,\mathbf{t};h_n)$ to $r^{\left(m\right)}(\phi ,\mathbf{t})$. Meanwhile, the study by Prakasa Rao and Sen [43] explored the limit distributions of ${\widehat{r}}_n^{\left(m\right)}(\phi ,\mathbf{t};h_n)$, discussing and contrasting Stute’s findings. Similarly, under appropriate mixing conditions, Harel and Puri [44] extended the results of [40] to weakly dependent data and applied their findings to validate the Bayes risk consistency of relevant discrimination rules. In another line of research, Stute [45] proposed symmetrized nearest neighbor conditional U-statistics as alternatives to conventional kernel estimators. Fu [46] considered the functional conditional U-statistic and established its finite-dimensional asymptotic normality. Despite the relevance of the subject, the nonparametric estimation of conditional U-statistics in the functional data framework has not received significant attention. Recent developments are discussed in reference [47], where the authors examine challenges related to uniform-in-bandwidth consistency in a general framework.

The necessity for regression models facilitating dimension reduction has been strongly advocated by [28]. Recent developments in Functional Data Analysis underscore the significance of devising models to mitigate dimensionality effects (see [33,48,49] for recent surveys). Among these models, single-index models are prevalent, operating on the premise that the impact of predictors on the response can be simplified to a single index. This index represents a projection onto a specified direction and is coupled with a nonparametric link function, condensing predictors into a single-variable index while preserving essential characteristics. Importantly, utilizing a nonparametric link function solely on a one-dimensional index helps alleviate challenges associated with high-dimensional data, commonly referred to as the curse of dimensionality. The single-index model extends beyond traditional linear regression by integrating a link function equivalent to the identity function. For further insights, interested readers may refer to [50,51,52].

Semiparametric concepts naturally emerge as promising candidates for this purpose. In this context, the Functional Single-Index Model (FSIM) has been examined by [53,54]. Functional single-index quantile regression, introduced by [55], estimates the unknown slope function and link function using B-spline basis functions. A novel compact functional single-index model, with the coefficient function only nonzero in a subregion, was proposed by [56]. The estimation of a general functional single-index model, where the conditional distribution of the response depends on the functional predictor via a functional single-index structure, was investigated by [57]. Tang et al. [58] devised a unique estimation procedure combining functional principal component analysis of functional predictors, B-spline models for parameters, and profile estimation of unknown parameters and functions in the model. For strong mixing time series data, Ling et al. [59,60] investigated the estimation of the functional single-index regression model with missing responses at random. Feng et al. [61] introduced a functional single-index varying coefficient model with the functional predictor as the single-index part, utilizing functional principal components analysis and basis function approximation for estimating slope and coefficient functions through an iterative procedure. Novo et al. [62] developed an automatic and location-adaptive procedure for estimating regression in an FSIM based on k-Nearest Neighbours (kNN) ideas. In the realm of imaging data analysis, Li et al. [63] proposed a novel functional varying-coefficient single-index model for regression analysis of functional response data on a set of covariates. Attaoui and Ling [64] investigated a functional Hilbertian regressor for nonparametric estimation of the conditional cumulative distribution with a scalar response variable in the single-index structure. An alternative approach was presented by [65], encompassing the multi-index case and not anchoring the true parameter on a pre-specified sieve. Their methodology includes a detailed theoretical analysis of a direct kernel-based estimation scheme, establishing a polynomial convergence rate. In comparison to the aforementioned studies on time series analysis, research concerning nonparametric methods for locally stationary random fields is scarce, even for nonparametric density estimation and regression, despite empirical interest in modeling spatially varying dependence structures. Moreover, for the analysis of spatial data on ${\mathbb{R}}^d$, it is generally assumed that the underlying model is a parametric Gaussian process. For empirical motivation and discussion regarding the modeling of nonstationary random fields, we refer to [66], and for discussions regarding the importance of nonparametric, nonstationary, and non-Gaussian spatial models, see [67]. For additional references, see [13,68,69,70,71,72,73].

1.1. Paper Contribution

This study aims to establish a comprehensive framework and elucidate the weak convergence and consistency of locally stationary random function fields utilizing conditional U-processes within a single-index structure. Locally stationary processes, as introduced by [74], denote time series exhibiting nonstationary behavior, allowing parameters to vary over time. These processes can be approximated locally by stationary time series, facilitating the establishment of asymptotic theories for estimating time-varying characteristics. In time series analysis, locally stationary models are primarily examined within a parametric framework featuring coefficients that vary with time. Refer to [75] for general theory in the literature of locally stationary processes, and also see [76]. The analysis is intricate due to the inherent challenge of ensuring asymptotic equicontinuity with minimal conditions in this broad context, which remains unresolved in the existing literature. To address this gap, we propose integrating insights from [49,77,78], while incorporating techniques relevant to functional data from [73,79,80]. However, as detailed in the subsequent sections, addressing this challenge goes beyond a mere amalgamation of concepts from existing studies. Complex mathematical derivations are indispensable to tackle the specific characteristics of functional data in our context. Successfully addressing this necessitates applying large-sample theoretical tools previously established for empirical processes, with reference to the works of [13,49,77,78]. This paper rigorously addresses several technical challenges. The first issue pertains to the nonlinear extension of the single-index idea in the framework of the conditional U-processes. The second challenge involves extending the Hoeffding decomposition for non-stationary time series. Finally, the third problem arises from the unbounded nature of the function class, resulting in lengthy and technical proofs. Our main results concerning the uniform convergence with rate are presented in Proposition 1 and Theorem 1. In Theorem 1, we present our main result concerning the weak convergence. In the realm of spatial functional data, there exists a paucity of literature regarding single-index models. In a previous study [81], we examined the expectile single index within spatial functional data. However, the current paper addresses a broader and more abstract problem, amalgamating various techniques from functional data analysis, spatial data observed at irregular intervals, and the theory of empirical processes and U-processes within a dependent framework. The proofs of our results diverge significantly from those applicable to equally spaced time series (mixing sequence) or spatial data observed on lattice points. Our emphasis lies on spatial dependence and irregularly spaced sampling points. In numerous scientific domains like ecology, meteorology, seismology, and spatial econometrics, irregular sampling points are intrinsic due to physical constraints. Measurement stations cannot be uniformly distributed on a regular grid. The stochastic sampling design employed in this study accommodates non-uniform density among sampling sites across the region. This flexibility allows the number of sampling sites to vary at different rates relative to the region’s volume, approximately $O\left(A^{d}n\right)$. For more comprehensive details and rationale, please refer to [73]. This paper extends our previous research [13] to encompass the single-index context. Numerous significant questions within this framework remain unanswered, as outlined in Section 8.

1.2. Paper Organization

The structure of this article is as follows. In Section 2, we present the functional framework along with the necessary definitions for our study. Briefly outlined are the assumptions that form the basis of our asymptotic analysis. Section 3 discusses the uniform rates of strong convergence, while Section 4 reveals the main results concerning the uniform weak convergence for the conditional U-processes. Exploring potential applications is the focus of Section 5, including: metric learning in Section 5.1, the multipartite ranking in Section 5.3, the discrimination in Section 5.4 and the Kendall rank correlation coefficient in Section 5.5. Section 6 delves into the consideration of conditional U-statistics within the right-censored data framework including the conditional U-statistics for left truncated and right censored data in Section 6.1. The selection of bandwidth through cross-validation procedures is detailed in Section 7. Concluding remarks and possible future developments are covered in Section 8. To ensure a smooth flow of presentation, all proofs are consolidated in Section 9. Finally, Section 10 provides relevant technical results, and includes the examples of classes of functions in Section 10.1 and examples of U-kernels in Section 10.2.

2. The Functional Framework

2.1. Notation

For each set $A\subset {\mathbb{R}}^d$, $\left|A\right|$ denotes the Lebesgue measure of A, and $[![A]!]$ represents the number of elements in A. Regarding positive sequences $a_n$ and $b_n$, we use the notation $a_n\lesssim b_n$ if there exists a constant $C>0$, independent of n, such that $a_n\le C{b}_n$ for all n. Similarly, $a_n\sim b_n$ if $a_n\lesssim b_n$ and $b_n\lesssim a_n$, and $a_n\ll b_n$ if $a_n/b_n\to 0$ as $n\to \infty$. The symbol $\stackrel{d}{\to }$ indicates convergence in the distribution. We use $X\stackrel{d}{=}Y$ to denote that the random variables X and Y have the same distribution. $P_{\mathit{S}}$ denotes the joint probability distribution of the sequence of independent and identically distributed (i.i.d.) random vectors ${\left\{{\mathit{S}}_{0,j}\right\}}_{j\ge 1}$, and $P_{·|\mathit{S}}$ is the conditional probability distribution for ${\left\{{\mathit{S}}_{0,j}\right\}}_{j\ge 1}$. Additionally, $E_{·|\mathit{S}}$ represents the conditional expectation, and ${Var}_{·|\mathit{S}}$ represents the variance for ${\left\{{\mathit{S}}_{0,j}\right\}}_{j\ge 1}$.

2.2. Generality on the Model

Let $\langle ·,·\rangle$ and $\parallel ·\parallel$ represent the inner product and the associated norm on the Hilbert space $\mathcal{H}$, respectively, with ${\left(e_p\right)}_{p\ge 1}$ as a complete orthonormal system of $\mathcal{H}$. Let $R_n={[0,A_n]}^d\subset {\mathbb{R}}^d$ be a sampling region with $A_n\to \infty$ as $n\to \infty$. We consider the semi-metric $d_{\theta }(·,·)$ associated with the single index $\theta \in \mathcal{H}$, defined by

$$d_{\theta }(u,v):=\left|\langle \theta ,u-v\rangle \right|,\mathrm{for}u,v\in \mathcal{H}.$$

In this investigation, we assume the existence of a fixed $\mathit{\theta }=({\theta }_1,\dots ,{\theta }_m)\in {\Theta }^m\subset {\mathcal{H}}^m$, $(x_1,\dots ,x_m)\in {\mathcal{H}}^m$, and the observations are related by the following relation for ${\mathit{s}}_i\in R_n,i=1,\dots ,m$, $\mathbf{i}=(1,\dots ,m)$ and ${\mathit{s}}_{\mathbf{i}}=({\mathit{s}}_1,\dots ,{\mathit{s}}_m)$,

$$\begin{array}{c}\hfill {\displaystyle r^{\left(m\right)}\left(\phi ,\mathbf{x},\mathit{\theta },\frac{{\mathit{s}}_{\mathbf{i}}}{A_n}\right)}\\ & :=& \mathbb{E}\left(\phi ({\mathbf{Y}}_{{\mathit{s}}_1,A_n},\dots ,{\mathbf{Y}}_{{\mathit{s}}_m,A_n})\mid \langle {\mathbf{X}}_{{\mathit{s}}_1,A_n},{\theta }_1\rangle =\langle x_1,{\theta }_1\rangle ,\dots ,\langle {\mathbf{X}}_{{\mathit{s}}_m,A_n},{\theta }_m\rangle =\langle x_m,{\theta }_m\rangle \right),\hfill \end{array}$$

and

$$\begin{array}{ccc}\hfill \phi ({\mathbf{Y}}_{{\mathit{s}}_1,A_n},\dots ,{\mathbf{Y}}_{{\mathit{s}}_m,A_n})& =& r^{\left(m\right)}\left(\phi ,\mathbf{x},\theta ,\frac{{\mathit{s}}_{\mathbf{i}}}{A_n}\right)+\prod _{j=1}^m\sigma \left(\frac{{\mathit{s}}_j}{A_n},{\mathbf{X}}_{{\mathit{s}}_j,A_n}\right){ϵ}_{i_j}\hfill \\ & =:& r^{\left(m\right)}\left(\phi ,\mathbf{x},\theta ,\frac{{\mathit{s}}_{\mathbf{i}}}{A_n}\right)+\prod _{j=1}^m{ϵ}_{{\mathit{s}}_j,A_n},\hfill \end{array}$$

(4)

where $\mathbb{E}\left[{ϵ}_{\mathit{s},A_n}\right|{\mathbf{X}}_{\mathit{s},A_n}]=0$ and $\frac{{\mathit{s}}_{\mathbf{i}}}{A_n}=(\frac{s_1}{A_n},\dots ,\frac{s_m}{A_n})$. Here, ${\mathbf{Y}}_{\mathit{s},A_n}$ and ${\mathbf{X}}_{\mathit{s},A_n}$ represent random functions in $\mathcal{H}$ and $\mathcal{Y}$, respectively. To address the identifiability issue, we assume that the regression function is differentiable, and for $i=1,\dots ,m$, $\langle {\theta }_i,e_1\rangle =1$, where $e_1$ denotes the first element of the orthonormal basis of $\mathcal{H}$. For a more in-depth exploration of the identifiability problem in the single functional index model, one can refer to [53]. We view $\{{\mathbf{X}}_{\mathit{s},A_n}:\mathit{s}\in R_n\}$ as a locally stationary random function field on $R_n\subset {\mathbb{R}}^d$ ($d\ge 2$). As suggested by [74], locally stationary processes are nonstationary time series in which the parameters of the time series can change over time. Locally in time, they can be modeled by a stationary time series, allowing the use of asymptotic theories to estimate the parameters of models that depend on time. Time series analysis predominantly centers on locally stationary models within a parametric framework, characterized by coefficients that vary over time. Without assuming the single-index structure, the model (4) is given by

$$\begin{array}{ccc}\hfill r^{\left(m\right)}\left(\phi ,\mathbf{x},\frac{{\mathit{s}}_{\mathbf{i}}}{A_n}\right)& :=& \mathbb{E}\left(\phi ({\mathbf{Y}}_{{\mathit{s}}_1,A_n},\dots ,{\mathbf{Y}}_{{\mathit{s}}_m,A_n})\mid {\mathbf{X}}_{{\mathit{s}}_1,A_n},=x_1,\dots ,{\mathbf{X}}_{{\mathit{s}}_m,A_n}=x_m\right),\hfill \end{array}$$

and

$$\begin{array}{ccc}\hfill \phi ({\mathbf{Y}}_{{\mathit{s}}_1,A_n},\dots ,{\mathbf{Y}}_{{\mathit{s}}_m,A_n})& =& r^{\left(m\right)}\left(\phi ,\mathbf{x},\frac{{\mathit{s}}_{\mathbf{i}}}{A_n}\right)+\prod _{j=1}^m\sigma \left(\frac{{\mathit{s}}_j}{A_n},{\mathbf{X}}_{{\mathit{s}}_j,A_n}\right){ϵ}_{i_j}\hfill \\ & =:& r^{\left(m\right)}\left(\phi ,\mathbf{x},\frac{{\mathit{s}}_{\mathbf{i}}}{A_n}\right)+\prod _{j=1}^m{ϵ}_{{\mathit{s}}_j,A_n},\hfill \end{array}$$

(5)

where $\mathbb{E}\left[{ϵ}_{\mathit{s},A_n}\right|{\mathbf{X}}_{\mathit{s},A_n}]=0$. The following examples are given for the particular case $m=1$, in the finite dimensional setting and are presented in [82]. The following examples may help to illustrate the model (5), without considering the dependence of the processes that we will consider.

Example 1

(Multivariate nonlinear time series). Consider the scenario where we observe multivariate time series ${\{{\mathbf{Y}}_t={(Y_{1,t},\cdots ,Y_{p,t})}^{\prime }\}}_{t=1}^T$ and ${\{{\mathbf{X}}_t={(X_{1,t},\cdots ,X_{d,t})}^{\prime }\}}_{t=1}^T$ such that

$$\begin{array}{cc}\hfill Y_{j,t}& =r_j\left({\mathbf{X}}_t\right)+{\sigma }_j\left({\mathbf{X}}_t\right){\epsilon }_{j,t},j=1,\cdots ,p.\hfill \end{array}$$

(6)

The model (6) corresponds to (i) a multivariate nonlinear AR model when ${\mathbf{X}}_t={({\mathbf{Y}}_{t-1}^{\prime },\cdots ,{\mathbf{Y}}_{t-q}^{\prime })}^{\prime }$ for some $q\ge 1$ and (ii) a multivariate nonlinear time series regression with exogenous variables when ${\sigma }_j(·)=1$ and ${\left\{{\mathbf{X}}_t\right\}}_{t=1}^T$ is uncorrelated with ${\{{\epsilon }_t={({\epsilon }_{1,t},\cdots ,{\epsilon }_{p,t})}^{\prime }\}}_{t=1}^T$. If one seeks to estimate the mean function $\mathbf{r}={(r_1,\cdots ,r_p)}^{\prime }:{\mathbb{R}}^d\to {\mathbb{R}}^p$, then it suffices to estimate each component $r_j(·)$.

Example 2

(Time-varying nonlinear models). Consider a nonlinear time-varying model:

$$\begin{array}{cc}\hfill Y_t& =r^{\left(1\right)}\left(\frac{t}{T},Y_{t-1},\cdots ,Y_{t-p}\right)+\sigma \left(\frac{t}{T},Y_{t-1},\cdots ,Y_{t-q}\right)v_t,\hfill \end{array}$$

(7)

where $1\le p,q\le d-1$. This example corresponds to the model (5), $m=1$, with $X_t={(t/T,Y_{t-1},\cdots ,Y_{t-d+1})}^{\prime }$ as well as $r^{\left(1\right)}(·)$ and $\sigma (·)$ considered as functions on ${\mathbb{R}}^d$ in the canonical way. If the random variables $v_t$ are i.i.d., then the model (7) corresponds to that considered in [83]. Moreover, the model (7) covers, for instance, time-varying AR(p)-ARCH(q) models when $r^{\left(1\right)}(u,x_1,\cdots ,x_p)=r_0\left(u\right)+{\sum }_{j=1}^p{r}_j\left(u\right)x_j,$ and $\sigma (u,x_1,\cdots ,x_q)={({\sigma }_0\left(u\right)+{\sum }_{j=1}^q{\sigma }_j\left(u\right)x_j^2)}^{1/2},$ with some functions $r_j:[0,1]\to \mathbb{R}$, ${\sigma }_j:[0,1]\to [0,\infty )$, as referenced in [84].

2.3. Local Stationarity

Consider a random function field $\{{\mathit{X}}_{\mathit{s},A_n}:\mathit{s}\in R_n\}$, where $A_n\to \infty$ as $n\to \infty$. This field is regarded as locally stationary if it demonstrates behavior that is approximately stationary in the local space. (For example, consider a continuous function $f:[0,1]\to$$\mathbb{R}$ and a sequence of i.i.d. random variables ${\left({\epsilon }_t\right)}_{t\in \mathbb{N}}$. The stochastic process $X_{t,T}=$$f(t/T)+{\epsilon }_t$, $t\in \{1,\dots ,T\},T\in \mathbb{N}$ can be expected to behave “almost” stationary for $t\in$$\{1,\dots ,T\}$ close to $t^{*}$, for some $t^{*}\in \{1,\dots ,T\}$, as in this case $f\left(t^{*}/T\right)\approx f(t/T)$, but this process is not weakly stationary.) To ensure local stationarity around each rescaled space point $\mathit{u}$, a process ${\mathit{X}}_{\mathit{s},A_n}$ can be stochastically approximated by a stationary random function field $\{{\mathit{X}}_{\mathit{u}}\left(\mathit{s}\right):\mathit{s}\in {\mathbb{R}}^d\}$, as discussed, for example, in [85]. The following definition represents one possible approach to define this concept.

Definition 1.

The $\mathcal{H}$-valued stochastic process $\{{\mathit{X}}_{\mathit{s},A_n}:\mathit{s}\in R_n\}$ is considered locally stationary if, for each rescaled time point $\mathit{u}\in {[0,1]}^d$, there exists an associated $\mathcal{H}$-valued process $\{{\mathit{X}}_{\mathit{u}}\left(\mathit{s}\right):\mathit{s}\in {\mathbb{R}}^d\}$ with the following properties:

(i) 

$\{{\mathit{X}}_{\mathit{u}}\left(\mathit{s}\right):\mathit{s}\in {\mathbb{R}}^d\}$ is strictly stationary.

(ii) 

It holds that

$$\begin{array}{c}\hfill d_{\theta }\left({\mathit{X}}_{\mathit{s},A_n},{\mathit{X}}_{\mathit{u}}\left(\mathit{s}\right)\right)\le \left({∥\frac{\mathit{s}}{A_n}-\mathit{u}∥}_2+\frac{1}{A_n^d}\right)U_{\mathit{s},A_n}\left(\mathit{u}\right),\mathrm{a}.\mathrm{s}.,\end{array}$$

(8)

where $\left\{U_{\mathit{s},A_n}\left(\mathit{u}\right)\right\}$ is a process of positive variables satisfying $\mathbb{E}\left[{\left(U_{\mathit{s},A_n}\left(\mathit{u}\right)\right)}^{\rho }\right]<C$ for some $\rho >0$, $C<\infty$; C is independent of $\mathit{u},\mathit{s}$, and $A_n$. ${\parallel ·\parallel }_2$ denotes an arbitrary norm in ${\mathbb{R}}^d$.

The concept of local stationarity for real-valued time series was initially introduced by [74]. Definition 1 is a natural extension of that idea. Additionally, the definition we provide aligns with (Definition 2.1, [86]) when $\mathcal{H}$ is the Hilbert space $L_{\mathbb{R}}^2\left([0,1]\right)$ of all real-valued functions that are square integrable with respect to the Lebesgue measure on the unit interval $[0,1]$ with the $L_2$-norm given by

$${\parallel f\parallel }_2=\sqrt{\langle f,f\rangle },\langle f,g\rangle ={\int }_0^{1}f\left(t\right)g\left(t\right)dt,$$

where $f,g\in L_{\mathbb{R}}^2\left([0,1]\right)$. Additionally, the authors establish necessary conditions for an $L_{\mathbb{R}}^2\left([0,1]\right)$-valued stochastic process ${\mathbf{X}}_{t,A_n}$ to satisfy (8) with $d(f,g)={|f-g|}_2$ and $\rho =2$.

2.4. Sampling Design

We will explore the stochastic sampling strategy tailored to accommodate irregularly spaced data. Let us denote $R_n$ as the sampling region. Consider a sequence of positive numbers ${\left\{A_n\right\}}_{n\ge 1}$ such that $A_n\to \infty$ as $n\to \infty$. The sampling region is defined as follows:

$$\begin{array}{c}\hfill R_n={[0,A_n]}^d.\end{array}$$

(9)

Next, we will discuss the (random) sample designs to be employed. Let $f_{\mathit{S}}\left({\mathit{s}}_0\right)$ be a continuous, everywhere positive probability density function on $R_0={[0,1]}^d$, and let ${\left\{{\mathit{S}}_{0,j}\right\}}_{j\ge 1}$ be a sequence of i.i.d. random vectors with the probability density $f_{\mathit{S}}\left({\mathit{s}}_0\right)$. These random vectors are such that ${\left\{{\mathit{S}}_{0,j}\right\}}_{j\ge 1}$ and $\{{\mathbf{X}}_{\mathit{s},A}:\mathit{s}\in R_n\}$ share a common probability space $(\Omega ,\mathcal{F},P)$ and are independent. The realizations ${\mathit{s}}_{0,1},\cdots ,{\mathit{s}}_{0,n}$ of random vectors ${\mathit{S}}_{0,1},\cdots ,{\mathit{S}}_{0,n}$ are determined by the following relation:

$${\mathit{s}}_j=A_n{\mathit{s}}_{0,j},j=1,\cdots ,n.$$

This relation provides the sampling sites ${\mathit{s}}_1,\cdots ,{\mathit{s}}_n$. Herein, we assume that $n{A}_n^{-d}\to \infty$ as $n\to \infty$.

Remark 1.

In practical applications, $A_n$ can be determined by considering the diameter of the sampling region. We can extend the applicability of the assumption (9) to $R_n$ for a broader range of situations, namely,

$$R_n=\prod _{j=1}^d[0,A_{j,n}],$$

where $A_{j,n}$ represents sequences of positive constants with $A_{j,n}\to \infty$ as $n\to \infty$. To avoid more challenging outcomes, we have retained the assumption (9). For further discussion, please refer to [73,80,87,88,89,90] and (Chapter 12, [91]).

2.5. Mixing Condition

The sequence ${\mathbf{Z}}_1,{\mathbf{Z}}_2,$ is considered $\beta$-mixing or absolutely regular, as described in [92,93], if

$$\beta \left(k\right):=\mathbb{E}\underset{l⩾1}{sup}\left\{\left|\mathbb{P}\left(A|{\sigma }_1^l\right)-\mathbb{P}\left(A\right)\right|:A\in {\sigma }_{l+k}^{\infty }\right\}⟶0\mathrm{as}k\to \infty .$$

Notably, Ibragimov and Solev [94] provided a comprehensive description of stationary Gaussian processes matching the last condition. Now, we define $\beta$-mixing coefficients for a random function field $\tilde{\mathbf{X}}$. Let ${\sigma }_{\tilde{\mathbf{X}}}\left(T\right)=\sigma \left(\{\tilde{\mathbf{X}}\left(\mathit{s}\right):\mathit{s}\in T\}\right)$ be the $\sigma$-field generated by variables $\left\{\widetilde{\mathbf{X}(}\mathit{s}\right):\mathit{s}\in T\}$, $T\subset {\mathbb{R}}^d$. For subsets $T_1$ and $T_2$ of ${\mathbb{R}}^d$, let

$$\overline{\beta }(T_1,T_2)=sup\frac{1}{2}\sum _{j=1}^J\sum _{k=1}^K|P(A_j\cap B_k)-P\left(A_j\right)P\left(B_k\right)|,$$

where the supremum is taken over all pairs of (finite) partitions $\{A_1,\cdots ,A_J\}$ and $\{B_1,\cdots ,B_K\}$ of ${\mathbb{R}}^d$ such that $A_j\in {\sigma }_{\tilde{\mathbf{X}}}\left(T_1\right)$ and $B_k\in {\sigma }_{\tilde{\mathbf{X}}}\left(T_2\right)$. Furthermore, let

$$d(T_1,T_2)=inf\left\{\right|\mathbf{x}-\mathbf{y}|:\mathbf{x}\in T_1,\mathbf{y}\in T_2\},$$

where $\left|\mathbf{x}\right|={\sum }_{j=1}^d\left|x_j\right|$ for $\mathbf{x}\in {\mathbb{R}}^d$, and let $\mathcal{R}\left(b\right)$ be the collection of all finite disjoint unions of cubes in ${\mathbb{R}}^d$ with a total volume not exceeding b. Subsequently, the $\beta$-mixing coefficients for the random field $\tilde{\mathbf{X}}$ can be defined as

$$\begin{array}{c}\hfill \beta (a;b)=sup\{\overline{\beta }(T_1,T_2):d(T_1,T_2)\ge a,T_1,T_2\in \mathcal{R}\left(b\right)\}.\end{array}$$

(10)

We assume that a non-increasing function ${\beta }_1(·)$ with ${lim}_{a\to \infty }{\beta }_1\left(a\right)=0$ and a non-decreasing function $g_1(·)$ exist such that the $\beta$-mixing coefficient $\beta (a;b)$ satisfies the following inequality:

$$\beta (a;b)\le {\beta }_1\left(a\right)g_1\left(b\right),a>0,b>0,$$

(11)

where $g_1(·)$ may be unbounded for $d\ge 2$.

Remark 2

(Some remarks about mixing conditions). The size of index sets $T_1$ and $T_2$ in the definition of $\beta (a;b)$ must be constrained. To elaborate on this point, if the β-mixing coefficients of a random field $\mathit{X}$ are defined similarly to the β-mixing coefficients for time series as follows: Let ${\mathcal{O}}_1$ and ${\mathcal{O}}_2$ be half-planes with boundaries $L_1$ and $L_2$, respectively. For each real number $a>0$, define

$$\beta \left(a\right)=sup\left\{\overline{\beta }({\mathcal{O}}_1,{\mathcal{O}}_2):d({\mathcal{O}}_1,{\mathcal{O}}_2)\ge a\right\},$$

where the supremum is taken over all pairs of parallel lines $L_1$ and $L_2$ such that $d(L_1,L_2)\ge a$. Subsequently, (Theorem 1, [95]) shows that if $\{\mathit{X}\left(s\right):s\in {\mathbb{R}}^2\}$ is a strictly stationary mixing random field, and $a>0$ is a real number, then $\beta \left(a\right)=1$ or 0. This means that if a random field $\mathbf{X}$ is β-mixing $\left({lim}_{a\to \infty }\beta \left(a\right)=0\right)$, then, for η a positive constant and some $a>\eta$, the random field $\mathit{X}$ is "ℓ-dependent", i.e., $\beta \left(a\right)=0$. However, this is highly restricted in practice. To relax these results and make them more flexible for practical purposes, it will be necessary to limit the size of $T_1$ and $T_2$ and adopt Definition 10 for β-mixing. For additional information on mixing coefficients for random fields, we refer to [73,80,96,97,98].

Lahiri [89] discussed the mixing condition given in Equation (11) for the $\alpha$-mixing condition, as also explored in the studies of [99,100]. While we specifically focus on the $\beta$-mixing case, it is well established that $\beta$-mixing implies $\alpha$-mixing. In general, within the expression (11), ${\beta }_1(·)$ is a function that could potentially depend on n, considering that the random field ${\mathit{X}}_{\mathit{s},A_n}$ is contingent on n. However, for simplicity, we assume that $g(·)$ does not depend on n, although addressing cases where $g(·)$ changes with n is not inherently complex. It is crucial to note that the random field ${\mathit{Y}}_{\mathit{s},A_n}$ (or $\phi \left({\mathit{Y}}_{\mathit{s},A_n}\right)$) may not necessarily adhere to the mixing condition (11), as the mixing condition is postulated for ${\mathit{X}}_{\mathit{s},A_n}$. Nevertheless, with the regression form represented by the model in (4), ${\mathit{Y}}_{\mathit{s},A_n}$ (or $\phi \left({\mathit{Y}}_{\mathit{s},A_n}\right)$) may exhibit a flexible dependence structure.

Remark 3.

Kurisu [73] explored examples of locally stationary random fields on ${\mathbb{R}}^d$ that satisfy our mixing conditions and the other regularity conditions outlined in Section 5 of the same reference. To achieve this, the author introduced the notion of approximately ${\mathit{\ell }}_n$-dependent locally stationary random fields (where ${\ell }_n$ tends to infinity as n approaches infinity), and extended the framework of continuous autoregressive and moving average (CARMA) random fields as developed in [101] to encompass locally stationary CARMA-type random fields. CARMA random fields are identified by solutions to (fractional) stochastic partial differential equations, as delineated in [102], and are acknowledged as a versatile class of models for spatial data, as indicated in [69,101]. It has been demonstrated that a broad class of Lévy-driven moving average random fields, inclusive of locally stationary CARMA-type random fields, constitute (approximately ${\ell }_n$-dependent) locally stationary random fields. One of the distinguishing characteristics of CARMA random fields is their capability to represent non-Gaussian as well as Gaussian random fields, provided the driving Lévy random measures are purely non-Gaussian. Conversely, the statistical models prevalent in most of the existing spatial data literature pertaining to ${\mathbb{R}}^2$ rely heavily on Gaussian processes. For non-Gaussian processes, verifying mixing conditions, as exemplified in works such as [99,100], can often pose significant challenges. In [73], locally stationary Lévy-driven MA random fields were introduced by considering the expressions:

$$\begin{array}{cc}\hfill X_{\mathit{s},A_n}& ={\int }_{{\mathbb{R}}^d}g\left(\frac{\mathit{s}}{A_n},\parallel \mathit{s}-\mathit{v}\parallel \right)L\left(d\mathit{v}\right),\hfill \\ \hfill X_{\mathit{u}}\left(\mathit{s}\right)& ={\int }_{{\mathbb{R}}^d}g(\mathit{u},\parallel \mathit{s}-\mathit{v}\parallel )L\left(d\mathit{v}\right),\hfill \end{array}$$

where $g:{[0,1]}^d\times [0,\infty )\to \mathbb{R}$ is a bounded function satisfying specific conditions. Notably, $X_{\mathit{u}}\left(\mathit{s}\right)$ denotes a strictly stationary random field for each $\mathit{u}$. Assuming $E\left[{\left|L\left(A\right)\right|}^{q^{\prime }}\right]<\infty$ for any $A\in \mathcal{B}\left({\mathbb{R}}^d\right)$ with bounded Lebesgue measure $\left|A\right|$ and for $q^{\prime }=1,2$, it follows that:

$$\begin{array}{cc}\hfill E\left[X_{\mathit{u}}\left(\mathit{s}\right)\right]& ={\mu }_0{\int }_{{\mathbb{R}}^d}g(\mathit{u},\parallel \mathit{s}\parallel )d\mathit{s},\hfill \\ \hfill E\left[{\left(X_{\mathit{u}}\left(\mathit{s}\right)\right)}^2\right]& ={\overline{\sigma }}_0^2{\int }_{{\mathbb{R}}^d}{g}^2(\mathit{u},\parallel \mathit{s}\parallel )d\mathit{s},\hfill \end{array}$$

where ${\mu }_0=-i{\psi }^{\prime }\left(0\right)$ and ${\overline{\sigma }}_0^2=-{\psi }^{\prime \prime }\left(0\right)$. Let $\ell >0$. Define the function $\iota (·:\ell ):[0,\infty )\to [0,1]$ as

$$\iota (x:\ell )=\left\{\begin{array}{cc}1,\hfill & \mathit{if}0\le x\le \frac{\ell }{2},\hfill \\ -\frac{2}{\ell }x+2,\hfill & \mathit{if}\frac{\ell }{2}<x\le \ell ,\hfill \\ 0,\hfill & \mathit{if}\ell <x.\hfill \end{array}\right.$$

Now consider the process

$$X_{\mathit{u}}\left(\mathit{s}:A_{2,n}\right)={\int }_{{\mathbb{R}}^d}g(\mathit{u},\parallel \mathit{s}-\mathit{v}\parallel )\iota \left(\parallel \mathit{s}-\mathit{v}\parallel :A_{2,n}\right)L\left(d\mathit{v}\right).$$

Remarkably, $X_{\mathit{u}}(\mathit{s}$ : $\left.A_{2,n}\right)$ also constitutes a $2A_{2,n}$-dependent strictly stationary random field, whereby the β-mixing coefficients $\beta (a;b)={\beta }_1\left(a\right)g_1\left(b\right)$ of $X_{\mathit{u}}\left(\mathit{s}:A_{2,n}\right)$ satisfy ${\beta }_1\left(a\right)=0$ for $a\ge 2A_{2,n}$.

2.6. Estimation Procedures

Let $\{{\mathit{X}}_{\mathit{s},A_n},{\mathit{Y}}_{\mathit{s},A_n}:\mathit{s}\in R_n\}$ represent random variables, where ${\mathit{Y}}_{\mathit{s},A_n}$ belongs to $\mathcal{Y}$, and ${\mathit{X}}_{\mathit{s},A_n}$ assumes values in a semi-metric space $\mathcal{H}$ with a semi-metric $d_{\theta }(·,·)$. This metric defines a topology to measure the proximity between two elements of $\mathcal{H}$ and is independent of the definition of X to avoid concerns related to measurability. The aim of this investigation is to establish the weak convergence of the single-index conditional U-process employing the following single-index U-statistic. Given $\mathbf{x}=(x_1,\dots ,x_m)\in {\mathcal{H}}^m$, $\mathbf{u}=({\mathbf{u}}_1,\dots ,{\mathbf{u}}_m)\in {[0,1]}^{dm}$, and $\mathit{\theta }=({\theta }_1,\dots ,{\theta }_m)\in {\Theta }^m$,

$$\begin{array}{c}{\displaystyle {\widehat{r}}_n^{\left(m\right)}(\mathbf{x},\mathit{\theta },\mathbf{u};h_n):={\widehat{r}}_n^{\left(m\right)}(\phi ,\mathbf{x},\mathit{\theta },\mathbf{u};h_n)}\hfill \\ =\frac{{\displaystyle \sum _{\mathbf{i}\in I_n^m}\phi ({\mathbf{Y}}_{{\mathit{s}}_{i_1},A_n},\dots ,{\mathbf{Y}}_{{\mathit{s}}_{i_m,A_n}})\prod _{j=1}^m\left\{\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)K_2\left(\frac{d_{{\theta }_j}(x_j,{\mathbf{X}}_{{\mathit{s}}_{i_j},A_n})}{h_n}\right)\right\}}}{{\displaystyle \sum _{\mathbf{i}\in I_n^m}\prod _{j=1}^m\left\{\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)K_2\left(\frac{d_{{\theta }_j}(x_j,{\mathbf{X}}_{{\mathit{s}}_{i_j},A_n})}{h_n}\right)\right\}}}\hfill \\ =\frac{{\displaystyle \sum _{\mathbf{i}\in I_n^m}\phi ({\mathbf{Y}}_{{\mathit{s}}_{i_1},A_n},\dots ,{\mathbf{Y}}_{{\mathit{s}}_{i_m,A_n}})\prod _{j=1}^m\left\{\prod _{\ell =1}^d{K}_1\left(\frac{u_{j,\ell }-\frac{{\mathit{s}}_{i_{j,\ell }}}{A_n}}{h_n}\right)K_2\left(\frac{d_{{\theta }_j}(x_j,{\mathbf{X}}_{{\mathit{s}}_{i_j},A_n})}{h_n}\right)\right\}}}{{\displaystyle \sum _{\mathbf{i}\in I_n^m}\prod _{j=1}^m\left\{\prod _{\ell =1}^d{K}_1\left(\frac{u_{j,\ell }-\frac{{\mathit{s}}_{i_{j,\ell }}}{A_n}}{h_n}\right)K_2\left(\frac{d_{{\theta }_j}(x_j,{\mathbf{X}}_{{\mathit{s}}_{i_j},A_n})}{h_n}\right)\right\}}},\hfill \end{array}$$

(12)

where

$$\begin{array}{ccc}\hfill \overline{K}\left(\mathit{u}\right)& =& \prod _{\ell =1}^d{K}_1\left(u_{\ell }\right).\hfill \end{array}$$

(13)

To explore the weak convergence of the conditional empirical process and the conditional U-process within the context of functional data, it becomes essential to introduce new notations. Consider a symmetric, measurable function $\phi :{\mathcal{Y}}^m⟶\mathbb{R}$ that belongs to a class of functions denoted as ${\mathcal{F}}_m$. Additionally, let ${\left\{h_n\right\}}_{n\in {\mathbb{N}}^{*}}$ be a sequence of positive real numbers such that $h_n\to 0$ as $n\to \infty$. To formally express this, define

$${\mathcal{F}}_m=\{\phi :{\mathcal{Y}}^m\to \mathbb{R}\},$$

which constitutes a point-wise measurable class of real-valued symmetric measurable functions on ${\mathcal{Y}}^m$ with a measurable envelope function

$$F\left(\mathbf{y}\right)\ge \underset{\phi \in {\mathcal{F}}_m}{sup}\left|\phi \left(\mathbf{y}\right)\right|,for\mathbf{y}\in {\mathcal{Y}}^m.$$

(14)

For a kernel function $K(·)$, we define the point-wise measurable class of functions, for $1\le m\le n$

$${\mathcal{K}}_{\theta }^m:=\left\{(x_1,\dots ,x_m)\mapsto \prod _{i=1}^{m}K\left(\frac{d_{{\theta }_j}(x_i,·)}{h_n}\right),(x_1,\dots ,x_m)\in {\mathcal{H}}^m\right\},$$

and

$${\mathfrak{K}}_{\Theta }^m:=\bigcup _{\theta \in {\Theta }^m}{\mathcal{K}}_{\theta }^m=\bigcup _{\theta \in {\Theta }^m}\left\{(x_1,\dots ,x_m)\mapsto \prod _{i=1}^{m}K\left(\frac{d_{{\theta }_j}(x_i,·)}{h_n}\right),(x_1,\dots ,x_m)\in {\mathcal{H}}^m\right\}.$$

We use the notation

$$\psi (·,·)\in {\mathcal{F}}_m{\mathfrak{K}}_{\Theta }^m:=\left\{{\phi }_1(·){\phi }_2(·):{\phi }_1\in {\mathcal{F}}_m,{\phi }_2(·)\in {\mathfrak{K}}_{\Theta }^m\right\},$$

and

$$\psi (·,·)\in {\mathcal{F}}_1{\mathfrak{K}}_{\Theta }^1:=\mathcal{F}{\mathfrak{K}}_{\Theta }=\left\{{\phi }_1(·){\phi }_2(·):{\phi }_1\in {\mathcal{F}}_1,{\phi }_2(·)\in {\mathfrak{K}}_{\Theta }^1\right\}.$$

2.6.1. Small Ball Probability

In the absence of a universal reference measure, such as the Lebesgue measure, the density function of the functional variable is not defined, presenting a technical challenge in infinite-dimensional spaces. To tackle this issue, we utilize the concept of “small-ball probability”. Consider a ball $B_{\theta }(t,r)=\left\{z\in \mathcal{H}:d_{\theta }(z,t)⩽r\right\}$ in $\mathcal{H}$ with center $t\in \mathcal{H}$ and radius r. To characterize the topological structure of functional spaces, we introduce the small-ball probability for a fixed $t\in \mathcal{H},\theta \in \Theta$ and for all $r>0$ as follows:

$$\mathbb{P}(X\in B_{\theta }(t,r))=:{\varphi }_{\theta ,t}\left(r\right).$$

(15)

This concept is widely utilized in nonparametric functional data analysis to circumvent the need for density assumptions on the functional variable X and address the challenges associated with the infinite-dimensional nature of functional spaces, as discussed in [28,30,79,103].

2.6.2. VC-Type Classes of Functions

The asymptotic analysis of functional data involves concentration properties articulated through the concept of small-ball probability. In examining a process indexed by a class of functions, it becomes essential to incorporate additional topological concepts, such as metric entropy and VC-subgraph classes (with “VC” standing for Vapnik and Červonenkis).

Definition 2.

Let ${\mathcal{S}}_{\mathcal{E}}$ be a subset of a semi-metric space $\mathcal{E}$, and let $N_{\epsilon }$ be a positive integer. A finite set of points $\{e_1,\dots ,e_{N_{\epsilon }}\}\subset \mathcal{E}$ is called a ε-net of ${\mathcal{S}}_{\mathcal{E}}$ if:

$${\mathcal{S}}_{\mathcal{E}}\subseteq \bigcup _{j=1}^{N_{\epsilon }}{B}_{\theta }(e_j,\epsilon ).$$

If $N_{\epsilon }\left({\mathcal{S}}_{\mathcal{E}}\right)$ denotes the cardinality of the smallest ε-net (the minimal number of open balls of radius ε) needed to cover ${\mathcal{S}}_{\mathcal{H}}$ in $\mathcal{E}$, then the Kolmogorov’s entropy (metric entropy) of the set ${\mathcal{S}}_{\mathcal{E}}$ is given by the quantity:

$${\psi }_{{\mathcal{S}}_{\mathcal{E}}}\left(\epsilon \right):=log{N}_{\epsilon }\left({\mathcal{S}}_{\mathcal{E}}\right).$$

Named after Kolmogorov, the concept of metric entropy (cf. [104]) was introduced by him and later extended to various metric spaces. Dudley [105] applied this idea to establish necessary conditions for the continuity of Gaussian processes and laid the groundwork for significant expansions of Donsker’s theorem concerning the weak convergence of empirical processes. Represented by ${\mathcal{B}}_{\mathcal{H}}$ and ${\mathcal{S}}_{\mathcal{H}}$, these are two subsets of the space $\mathcal{H}$ with Kolmogorov’s entropy (for the radius $\epsilon$) denoted by ${\psi }_{{\mathcal{B}}_{\mathcal{H}}}\left(\epsilon \right)$ and ${\psi }_{{\mathcal{S}}_{\mathcal{H}}}\left(\epsilon \right)$, respectively. The Kolmogorov entropy for the subset ${\mathcal{B}}_{\mathcal{H}}\times {\mathcal{S}}_{\mathcal{H}}$ of the semi-metric space ${\mathcal{H}}^2$ is then given by:

$${\psi }_{{\mathcal{B}}_{\mathcal{H}}\times {\mathcal{S}}_{\mathcal{H}}}\left(\epsilon \right)={\psi }_{{\mathcal{B}}_{\mathcal{H}}}\left(\epsilon \right)+{\psi }_{{\mathcal{S}}_{\mathcal{H}}}\left(\epsilon \right).$$

Hence, $m{\psi }_{{\mathcal{S}}_{\mathcal{H}}}\left(\epsilon \right)$ is the Kolmogorov entropy of the subset ${\mathcal{S}}_{\mathcal{H}}^m$ of the semi-metric space ${\mathcal{H}}^m$. We specify by d the semi-metric on $\mathcal{H}$, then this semi-metric is defined on ${\mathcal{H}}^m$ by

$$d_{{\mathcal{H}}^m}\left(\mathbf{x},\mathbf{y}\right):=\frac{1}{m}d\left(x_1,y_1\right)+\dots +\frac{1}{m}d\left(x_m,y_m\right),$$

for

$$\mathbf{x}=(x_1,\dots ,x_m),\mathbf{y}=(y_1,\dots ,y_m)\in {\mathcal{H}}^m.$$

Let

$$d_{{\mathcal{H}}^m,\mathit{\theta }}\left(\mathbf{x},\mathbf{y}\right):=\frac{1}{m}{d}_{{\theta }_1}\left(x_1,y_1\right)+\cdots +\frac{1}{m}{d}_{{\theta }_m}\left(x_m,y_m\right),for\mathit{\theta }=({\theta }_1,\dots ,{\theta }_m),\in {\mathbf{\Theta }}^m.$$

In investigations of this nature, the choice of semi-metric plays a crucial role. For in-depth discussions on selecting the semi-metric, readers are encouraged to refer to [28] (specifically Chapters 3 and 13). Another topological term worth considering is VC-subgraph classes.

Definition 3.

A class of subsets $\mathcal{C}$ on a set C is called a VC-class if there exists a polynomial $\mathfrak{P}(·)$ such that, for every set of $N_{\epsilon }$ points in C, the class $\mathcal{C}$ selects at most $\mathfrak{P}\left(N_{\epsilon }\right)$ distinct subsets.

Definition 4.

A class of functions $\mathcal{F}$ is termed a VC-subgraph class if the graphs of the functions in $\mathcal{F}$ form a VC-class of sets. In other words, if we define the subgraph of a real-valued function f on S as the subset ${\mathcal{G}}_f$ on $S\times \mathbb{R}$ given by:

$${\mathcal{G}}_f=\{(s,t):0\le t\le f\left(s\right)\mathrm{or}f\left(s\right)\le t\le 0\},$$

then the class $\{{\mathcal{G}}_f:f\in \mathcal{F}\}$ is a VC-class of sets on $S\times \mathbb{R}$. Informally, VC-class functions are identified by their polynomial covering number, which is the minimum number of functions required to form a covering over the entire class of functions.

A VC-class of functions $\mathcal{F}$ with an envelope function F exhibits the following entropy property: for a given $1⩽q<\infty$, there exist constants a and b such that

$${N(ϵ,\mathcal{F},\parallel ·\parallel }_{L_q\left(Q\right)})\le a{\left(\frac{{\left(Q{F}^q\right)}^{1/q}}{ϵ}\right)}^b$$

(16)

for any $ϵ>0$ and each probability measure such that $Q{F}^q<\infty$. For further insights, see (Lemma 22, [106]), (Section 4.7, [107]), (Theorem 2.6.7, [108]), and (Section 9.1, [109]). Additionally, Deheuvels (Section 3.2, [110]) provides further discussions on sufficient conditions under which (16) holds. We have provided some classical examples of classes of functions in Section 10.1.

2.7. Conditions and Comments

Assumption 1.

[Model and distribution assumptions]

(M1) 

The stochastic process $\{{\mathit{X}}_{\mathit{s},A_n}:\mathit{s}\in R_n\}$, taking values in the Hilbert space $\mathcal{H}$, exhibits local stationarity. Therefore, for each time point $\mathbf{u}\in {[0,1]}^d$, there exists a strictly stationary process $\{{\mathit{X}}_{\mathit{u}}\left(\mathit{s}\right):\mathit{s}\in {\mathbb{R}}^d\}$. This process satisfies, for an arbitrary norm $\parallel ·\parallel$ on ${\mathbb{R}}^d$,

$$\begin{array}{c}\hfill d_{\theta }\left({\mathit{X}}_{\mathit{s},A_n},{\mathit{X}}_{\mathit{u}}\left(\mathit{s}\right)\right)\le \left({∥\frac{\mathit{s}}{A_n}-\mathit{u}∥}_2+\frac{1}{A_n^d}\right)U_{\mathit{s},A_n}\left(\mathit{u}\right)a.s.,\end{array}$$

(17)

where $\mathbb{E}\left[{\left(U_{\mathit{s},A_n}\left(\mathit{u}\right)\right)}^{\rho }\right]<C$ holds for some $\rho \ge 1$ and $C<\infty$, and these constants are independent of $\mathit{u}$, $\mathit{s}$, and $A_n$.

(M2) 

For$i=1,\dots ,m$, consider $B(x_i,h)=\{y\in \mathcal{H}:d_{{\theta }_i}(x_i,y)\le h\}$ as a ball centered at $x_i\in \mathcal{H}$ with radius h. Let $c_d<C_d$ be positive constants, and for all $\mathit{u}\in {[0,1]}^d$, define

$${\varphi }_{\mathbf{x},\mathit{\theta }}\left(h_n\right):=\mathbb{P}\left({\mathit{X}}_{\mathit{u}}\left({\mathit{s}}_1\right)\in B_{{\theta }_1}(x_1,h_n),\dots ,{\mathit{X}}_{\mathit{u}}\left({\mathit{s}}_m\right)\in B_{{\theta }_m}(x_m,h_n)\right)=F_{\mathit{u},\mathit{\theta }}(h_n,x_1,\dots ,x_m),$$

which satisfies the inequalities

$$\begin{array}{c}\hfill 0<c_d\varphi \left(h\right)f_1\left(\mathbf{x}\right)\le {\varphi }_{\mathbf{x},\theta }\left(h\right)\le C_d\varphi \left(h\right)f_1\left(\mathbf{x}\right),\end{array}$$

(18)

where $\varphi \left(h\right)\to 0$ as $h\to \infty$, and $f_1\left(\mathbf{x}\right)$ is a non-negative functional in $\mathbf{x}\in {\mathcal{H}}^m$. Additionally, there exist constants $C_{\varphi }>0$ and ${\epsilon }_0>0$ such that for any $0<\epsilon <{\epsilon }_0$,

$$\begin{array}{c}\hfill {\int }_0^{\epsilon }\varphi \left(u\right)du>C_{\varphi }\epsilon \varphi \left(\epsilon \right).\end{array}$$

(19)

(M3) 

Let ${\mathbb{X}}_{\mathit{s},A_n}=({\mathit{X}}_{{\mathit{s}}_m,A_n},\dots ,{\mathit{X}}_{{\mathit{s}}_1,A_n})$ and ${\mathbb{X}}_{\mathit{v},A_n}=({\mathit{X}}_{{\mathit{v}}_1,A_n},\dots ,{\mathit{X}}_{{\mathit{v}}_m,A_n})$ and $B_{\mathit{\theta }}(\mathbf{x},h)={\prod }_{i=1}^m{B}_{{\theta }_i}(x_i,h)$. Assume

$$\underset{\mathit{s},\mathbf{x},A_n}{sup}\underset{\mathit{s}\ne \mathit{v}}{sup}\mathbb{P}(({\mathbb{X}}_{\mathit{s},A_n},{\mathbb{X}}_{\mathit{v},A_n})\in B_{\mathit{\theta }}(\mathit{x},h)\times B_{\mathit{\theta }}(\mathit{x},h))\le \psi \left(h\right)f_2\left(\mathit{x}\right),$$

where $\psi \left(h\right)\to 0$ as $h\to 0$, and $f_2\left(\mathit{x}\right)$ is a non-negative functional in $\mathit{x}\in {\mathcal{H}}^m$. We assume that the ratio $\psi \left(h\right)/{\varphi }^2\left(h\right)$ is bounded.

(M4) 

${\mathcal{H}}^m\times {\mathbf{\Theta }}^m\times \sigma :[0,1]\to \mathbb{R}$ is bounded by some constant $C_{\sigma }<\infty$ from above and by some constant $c_{\sigma }>0$ from below, that is, $0<c_{\sigma }\le \sigma (\mathbf{x},\mathit{\theta },\mathbf{u})\le C_{\sigma }<\infty$ for all $\mathbf{u}$, $\mathbf{x}$ and θ.

(M5) 

$\sigma (·,·,·)$ is Lipschitz continuous with respect to $\mathbf{u}$.

(M6) 

${sup}_{\mathbf{u}\in {[0,1]}^m}{sup}_{\mathit{\theta }\in {\Theta }^m}{sup}_{z:d_{\theta }(\mathbf{x},\mathbf{z})\le \mathbf{h}}|\sigma (\mathbf{x},\mathit{\theta },\mathbf{u})-\sigma (\mathbf{u},\mathit{\theta },\mathit{z})|=o\left(1\right)$ as $h\to 0$.

(M7) 

$r^{\left(m\right)}(\mathbf{x},\mathit{\theta },\mathbf{u})$ is Lipschitz, and it satisfy

$$\underset{\mathit{\theta }\in {\mathbf{\Theta }}^m}{sup}|r^{\left(m\right)}({\mathbf{x}}_1,\mathit{\theta },{\mathbf{u}}_1)-r^{\left(m\right)}({\mathbf{x}}_2,\mathit{\theta },{\mathbf{u}}_2)|\le c_m\left(d_{{\mathcal{H}}^m,\mathit{\theta }}{\left({\mathbf{x}}_1,{\mathbf{x}}_2\right)}^{\alpha }+{\parallel {\mathbf{u}}_1-{\mathbf{u}}_2\parallel }^{\alpha }\right),$$

(20)

for some $c_m>0$, $\alpha >0$ and ${\mathbf{x}}_1=(x_{1,1},\dots ,x_{1,m}),{\mathbf{x}}_2=(x_{2,1},\dots ,x_{2,m})\in {\mathcal{H}}^m,$ and it is twice continuously partially differentiable with first derivatives

$${\partial }_{u_i}{r}^{\left(m\right)}(\mathbf{x},\mathit{\theta },\mathbf{u})=\frac{\partial }{\partial u_i}{r}^{\left(m\right)}(\mathbf{x},\mathit{\theta },\mathbf{u}),$$

and second derivatives

$${\partial }_{u_i{u}_j}^2{r}^{\left(m\right)}(\mathbf{x},\mathit{\theta },\mathbf{u})=\frac{{\partial }^2}{\partial u_i\partial u_j}{r}^{\left(m\right)}(\mathbf{x},\mathit{\theta },\mathbf{u}).$$

Assumption 2.

[Kernel assumptions]

(KB1) 

The kernel $K_2(·)$ is non-negative, bounded by $\tilde{\kappa }$, and has support in $[0,1]$ such that $0<K_2\left(0\right)$ and $K_2\left(1\right)=0$. Moreover, the derivative $K_2^{\prime }\left(v\right)=d{K}_2\left(v\right)/dv$ exists on $[0,1]$ and satisfies $C_1^{\prime }\le K_2^{\prime }\left(v\right)\le C_2^{\prime }$ for two real constants $-\infty <C_1^{\prime }<$$C_2^{\prime }<0$.

(KB2) 

The kernel $\overline{K}:{\mathbb{R}}^d\to [0,\infty )$ is bounded and has compact support ${[-C,C]}^d$. Moreover,

$${\int }_{{[-C,C]}^d}\overline{K}\left(\mathit{x}\right)dx=1,{\int }_{{[-C,C]}^d}{\mathit{x}}^{\alpha }\overline{K}\left(\mathit{x}\right)dx=0,forany\mathit{\alpha }\in {\mathbb{Z}}^{d}with\left|\mathit{\alpha }\right|=1,$$

and

$$|\overline{K}\left(\mathit{u}\right)-\overline{K}\left(\mathit{v}\right)|\le C\parallel \mathit{u}-\mathit{v}\parallel .$$

(KB3) 

The bandwidth h converges to zero at least at a polynomial rate; that is, there exists a small ${\xi }_1>0$ such that $h\le C{n}^{-{\xi }_1}$ for some constant $0<C<\infty$.

Assumption 3.

[Sampling design assumptions]

(S1) 

For any $\mathit{\alpha }\in {\mathbb{N}}^d$ with $\left|\mathit{\alpha }\right|=1,2$, ${\partial }^{\mathit{\alpha }}{f}_{\mathit{S}}\left(\mathit{s}\right)$ exists and is continuous on ${(0,1)}^d$.

(S2) 

$C_0\le n{A}_n^{-d}\le C_1{n}^{{\eta }_1}$ for some $C_0,C_1>0$ and small ${\eta }_1\in (0,1)$.

Assumption 4.

[Block decomposition assumptions]

(B1) 

Let ${\left\{A_{1,n}\right\}}_{n\ge 1}$ and ${\left\{A_{2,n}\right\}}_{n\ge 1}$ be two sequences of positive numbers such that $A_{1,n},A_{2,n}\to \infty ,A_{2,n}=o\left(A_{1,n}\right)$, and $A_{1,n}=o\left(A_n\right)$, or $\frac{A_{1,n}}{A_n}+\frac{A_{2,n}}{A_{1,n}}\le C_0^{-1}{n}^{-\eta }\to 0\mathit{for}\mathit{some}{C}_0>0\mathit{and}\eta >0.$

(B2) 

We have ${lim}_{n\to \infty }n{A}_n^{-d}=\kappa \in (0,\infty ]$ with $A_n\ge n^{\overline{\kappa }}$ for some $\overline{\kappa }>0$.

(B3) 

We have

$${\left(\frac{1}{n{h}^{md}\varphi \left(h\right)}\right)}^{1/3}{\left(\frac{A_{1,n}}{A_n}\right)}^{2d/3}{\left(\frac{A_{2,n}}{A_{1,n}}\right)}^{2/3}{g}_1^{1/3}\left(A_{1,n}^d\right)\sum _{k=1}^{A_n/A_{1,n}}{k}^{d-1}{\beta }_1^{1/3}\left(k{A}_{1,n}+A_{2,n}\right)\to 0.$$

(B4) 

We have ${lim}_{n\to \infty }{A}_n^d{A}_{1,n}^{-d}\beta \left(A_{2,n};A_n^d\right)=0$.

Assumption 5.

[Regularity conditions] Let ${\alpha }_n=\sqrt{logn/n{h}^{md}\varphi \left(h\right)}$. As $n\to \infty$,

(R1) 

$h^{-\left(md\right)}\varphi {\left(h\right)}^{-1}{\alpha }_n^{md}{A}_n^d{A}_{1,n}^{-d}\beta (A_{2,n};A_n^d)\to 0$ and $A_{1,n}^d{A}_n^{-d}n{h}^{md}\varphi \left(h\right)(logn)\to 0$,

(R2) 

$n^{1/2}{h}^{\left(md\right)/2}\varphi {\left(h\right)}^{1/2}/A_{1,n}^d{n}^{1/\zeta }\ge C_0{n}^{\eta }$ for some $0<C_0<\infty$ and $\eta >0$ and $\zeta >2$,

(R3) 

$A_n^{dp}\varphi \left(h\right)\to \infty$, where p is defined in the sequel.

Assumption 6.

(E1) 

For $W_{{\mathit{s}}_{\mathbf{i}},A_n}={\prod }_{j=1}^m{ϵ}_{{\mathit{s}}_{i_j},A_n}$, it holds that ${sup}_{\mathbf{x}\in {\mathcal{H}}^m}\mathbb{E}{\left|W_{\mathit{s},A_n}\right|}^{\zeta }\le C$ and

$$\underset{\mathbf{x}\in {\mathcal{H}}^m}{sup}\mathbb{E}\left[{\left|W_{\mathit{s},A_n}\right|}^{\zeta }\mid X_{\mathbf{i},n}=\mathbf{x}\right]\le C,$$

for $\zeta >2$ and $C<\infty$.

(E2) 

The β-mixing coefficients of the array $\left\{{\mathit{X}}_{\mathit{s},A_n},W_{\mathit{s},A_n}\right\}$ satisfy $\beta (a;b)\le {\beta }_1\left(a\right)g_1\left(b\right)$ with ${\beta }_1\left(a\right)\to 0$ as $a\to \infty .$

Assumption 7.

[Class of functions assumptions]

The classes of functions ${\mathcal{K}}^m$ and ${\mathcal{F}}_m$ satisfy the following conditions:

(C1) 

The class of functions ${\mathcal{F}}_m$ is bounded, and its envelope function satisfies, for some $0<M<\infty$

$$F\left(\mathbf{y}\right)\le M,\mathbf{y}\in {\mathcal{Y}}^m.$$

(C2) 

The class of functions ${\mathcal{F}}_m$ is unbounded, and its envelope function satisfies, for some $\zeta >2$

$${\theta }_{\zeta }:=\underset{\mathbf{t}\in {\mathcal{S}}_{\mathcal{H}}^m}{sup}\mathbb{E}\left(F^{\zeta }\left(\mathbf{Y}\right)\mid \mathbf{x}=\mathbf{x}\right)<\infty .$$

(C3) 

The metric entropy of the class ${\mathcal{F}}_m{\mathcal{K}}^m$ satisfies, for some $2<\zeta <\infty$

$$\begin{array}{cc}\hfill & {\int }_0^{\infty }{(logN(u,{\mathcal{F}}_m{\mathfrak{K}}_{\Theta }^m,L_{\zeta }\left({\mathbb{P}}^m\right)))}^{\frac{1}{2}}du<\infty .\hfill \end{array}$$

Comments on the Assumptions

Traditional statistical methods face challenges when applied to functional data. In our non-parametric functional regression model, we addressed the complex theoretical task of establishing functional central limit theorems for the conditional U-process under functional absolute regular data in a dual framework. This involved aligning assumptions with certain properties of the infinite-dimensional space, covering the topological structure on ${\mathcal{H}}^m$, the probability distribution of $\mathbf{X}$, and the measurability of the classes ${\mathcal{F}}_m$ and ${\mathfrak{K}}_{\Theta }^m$. A discussion about these assumptions is crucial, drawing inspiration from works such as [28,30,49,71,73,79,83]. Assumption 1 initiates the formalization of the property of $X_i$ being locally stationary, with subsequent conditions providing additional precision. (M1) reflects the notion of a locally stationary time series, and various random fields satisfy this requirement, as demonstrated by [73]. Condition (M2), employed by [79], aligns with the fundamental axioms of probability calculus when ${\mathcal{H}}^m={\mathbb{R}}^m$ and exhibits exponential decay for an infinitely dimensional Hilbert space. Equation (18) controls the behavior of the small ball probability around zero, indicating that it can be approximated by the product of two independent functions ${\varphi }^m(·)$ and $f_1(·)$. This condition is typical for various processes, including Ornstein–Uhlenbeck, general diffusion, and fractal processes. Conditions (M4), (M5), (M6) and Assumption 2 represent the regularity conditions, forming the foundation for the limiting theorems. Moreover, Assumption 3 addresses the flexibility of the non-uniform sampling density and the infill sampling criteria. The proposed sampling design accommodates both pure and mixed increasing domain cases. Condition (B1) in Assumption 4 relates to the Blocking technique used for decomposing the sampling region $R_n$ into large and small blocks. The sequences $\left\{A_{1,n}\right\}$ and $\left\{A_{2,n}\right\}$ are associated with the large-block–small-block argument, facilitating the proof of central limit theorems for sums of mixing random variables. Assumptions 6 contribute to deriving the weak convergence of the conditional U-statistic $\widehat{\psi }$, while condition (C1) lays the groundwork for bounded functions. However, (C2) replaces it to establish the functional central limit theorem for conditional U-processes indexed by an unbounded class of functions.

For Assumption (C3), see (Examples 26 and 38, [111]), (Lemma 22, [106]), (§4.7., [107]), (Theorem 2.6.7, [108]), (§9.1, [109]) provide a number of sufficient conditions under which (C3) holds, we may also refer to (§3.2, [37,111,112,113]) for further discussions. For instance, it is satisfied, for general $d\ge 1$, whenever $l\left(\mathbf{x}\right)=\Psi \left(p\right(\mathbf{x}\left)\right)$, with $p\left(\mathbf{x}\right)$ is either a polynomial in d variables or the $\alpha$th power of the absolute value of a real polynomial for some $\alpha >0$ and $\Psi (·)$ is a real-valued function of bounded variation, which covers commonly used kernels, such as Gaussian, Epanečnikov, Uniform, etc., we refer the reader to (p. 1381, [114]). We also mention that condition (C3) is satisfied whenever that class of functions contains functions of bounded variation on ${\mathbb{R}}^q$ (in the sense of Hardy and Kauser ([115,116,117]), see, e.g., [118,119,120,121]). Assumption (C3) ensures that $\mathcal{L}$ is VC type with characteristics $C^{\prime }$ and $\nu$. The condition (C3) as in [122] can formulated as follows for the kernel part:

(C4)

$K\left(x\right)>0$, is a bounded and compactly supported measurable function that belongs to the linear span (the set of finite linear combinations) of functions $k\left(x\right)\ge 0$ satisfying the following property: the subgraph of $k(·),$$\left\{\right(s,u):k(s)\ge u\}$, can be represented as a finite number of Boolean operations among sets of the form

$$\left\{(s,u):p(s,u)\ge \phi \left(u\right)\right\},$$

where p is a polynomial on $\mathbb{R}\times \mathbb{R}$ and $\phi$ is an arbitrary real function.

Indeed, for a fixed polynomial p, the family of sets

$$\left\{\left\{\right(s,u):p((s-t)/h,u)\ge \phi (u\left)\right\}:t\in \mathbb{R},h>0\right\}$$

is contained in the family of positivity sets of a finite-dimensional space of functions, and then the entropy bound follows by Theorems 4.2.1 and 4.2.4 in [107]. These assumptions collectively provide a robust foundation for the investigation, encompassing the topological structure, probability measure, measurability, and uniformity in entropy characteristics of functional variables.

Remark 4.

Note that Assumption (C2) in (7) may be replaced by more general hypotheses on the moments of $\mathbf{Y}$, as in [111,123]. Specifically, we can consider the following alternative assumption:

(C4)’ 

We denote by $\left\{\mathcal{M}\right(x):x\ge 0\}$ a non-negative continuous function, increasing on $[0,\infty )$, and such that, for some $s>2$, ultimately as $x\uparrow \infty$:

$$x^{-s}\mathcal{M}\left(x\right)\downarrow ;x^{-1}\mathcal{M}\left(x\right)\uparrow .$$

(21)

For each $t\ge \mathcal{M}\left(0\right)$, we define ${\mathcal{M}}^{inv}\left(t\right)\ge 0$ by $\mathcal{M}\left({\mathcal{M}}^{inv}\left(t\right)\right)=t$. Assuming further that:

$$\mathbb{E}\left(\mathcal{M}\right(\mid F\left(\mathbf{Y}\right)\mid \left)\right)<\infty .$$

The following choices of $\mathcal{M}(·)$ are of particular interest:

(i) 

$\mathcal{M}\left(x\right)=x^{\zeta }$ for some $\zeta >2$;

(ii) 

$\mathcal{M}\left(x\right)=exp\left(sx\right)$ for some $s>0$.

3. Uniform Convergence Rates for Kernel Estimators

Before detailing the asymptotic behavior of our estimator as presented in (12), we will extend our analysis to a U-statistic estimator defined by:

$$\widehat{\psi }(\mathbf{x},\mathit{\theta },\mathbf{u})=\frac{(n-m)!}{n!h^{md}{\varphi }^m\left(h\right)}\sum _{\mathbf{i}\in I_n^m}\prod _{j=1}^m\left\{\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)K_2\left(\frac{d_{{\theta }_j}(x_j,{\mathbf{X}}_{{\mathit{s}}_{i_j},A_n})}{h_n}\right)\right\}{W}_{{\mathit{s}}_{\mathbf{i}},A_n},$$

(22)

where $W_{{\mathit{s}}_{\mathbf{i}},A_n}$ is an array of one-dimensional random variables. In this study, we consider results with $W_{{\mathit{s}}_{\mathbf{i}},A_n}=1$ and $W_{{\mathit{s}}_{\mathbf{i}},A_n}={\prod }_{j=1}^m{ϵ}_{{\mathit{s}}_{i_j},A_n}$.

3.1. Hoeffding’s Decomposition

Note $\widehat{\psi }(\mathbf{x},\mathit{\theta },\mathbf{u})$ is a standard U-statistic with a kernel depending on n. We define

$$\begin{array}{ccc}\hfill {\xi }_j& :=& \frac{1}{h^d}\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right),\hfill \\ \hfill H(Z_1,\dots ,Z_m)& :=& \prod _{j=1}^m\frac{1}{\varphi \left(h\right)}{K}_2\left(\frac{d_{{\theta }_j}(x_j,{\mathbf{X}}_{{\mathit{s}}_{i_j},A_n})}{h_n}\right)W_{{\mathit{s}}_{\mathbf{i}},A_n},\hfill \end{array}$$

thus, the U-statistic in (22) can be viewed as a weighted U-statistic of degree m:

$$\widehat{\psi }(\mathbf{x},\mathit{\theta },\mathbf{u})=\frac{(n-m)!}{n!}\sum _{\mathbf{i}\in I_n^m}{\xi }_{i_1}\dots {\xi }_{i_m}H(Z_{i_1},\dots ,Z_{i_m}).$$

(23)

We can express Hoeffding’s decomposition in this scenario as in [124]. In the absence of assuming symmetry for $W_{{\mathit{s}}_{\mathbf{i}},A_n}$ or H, it is necessary to define

• The expectation of $H(Z_{i_1},\dots ,Z_{i_m})$

  $$\theta \left(\mathbf{i}\right):=\mathbb{E}H(Z_{i_1},\dots ,Z_{i_m})=\int W_{{\mathit{s}}_{\mathbf{i}},A_n}\prod _{j=1}^m\frac{1}{\varphi \left(h\right)}{K}_2\left(\frac{d_{{\theta }_j}(x_j,{\nu }_{{\mathit{s}}_{i_j},A_n})}{h_n}\right)d{\mathbb{P}}_{\mathbf{i}}\left(z_{\mathbf{i}}\right).$$

  (24)
• For all $\ell \in \{1,\dots ,m\}$ the position of the argument, construct the function ${\pi }_{\ell }(·)$ such that

  $${\pi }_{\ell }(z;z_1,\dots ,z_{m-1}):=(z_1,\dots ,z_{\ell -1},z,z_{\ell },\dots ,z_{m-1}).$$
• Define

  $$\begin{array}{cc}\hfill H^{(\ell )}\left(z;z_1,\dots ,z_{m-1}\right)& :=H\left\{{\pi }_{\ell }\left(z;z_1,\dots ,z_{m-1}\right)\right\},\hfill \\ \hfill {\vartheta }^{(\ell )}\left(i;i_1,i_2,\dots ,i_{m-1}\right)& :=\vartheta \left\{{\pi }_{\ell }\left(i;i_1,i_2,\dots ,i_{m-1}\right)\right\}.\hfill \end{array}$$

Hence, the first-order expansion of $H(·)$ will be seen as

$$\begin{array}{c}{\displaystyle {\tilde{H}}^{(\ell )}\left(z\right):=\mathbb{E}\left(H^{(\ell )}\left(z,Z_1,\dots ,Z_{m-1}\right)\right)}\hfill \\ =\int W_{{\mathit{s}}_{(1,\dots ,\ell -1,i,\ell ,\dots ,m-1)},A_n}\prod _{\underset{j\ne i}{j=1}}^{m-1}\frac{1}{\varphi \left(h\right)}{K}_2\left(\frac{d_{{\theta }_j}(x_j,{\nu }_{{\mathit{s}}_j,A_n})}{h}\right)\times \frac{1}{\varphi \left(h\right)}{K}_2\left(\frac{d_{{\theta }_i}(x_i,{\nu }_{{\mathit{s}}_i,A_n})}{h}\right)\hfill \\ \times \mathbb{P}(d{\nu }_1,\dots ,d{\nu }_{\ell -1},d{\nu }_{\ell },\dots ,d{\nu }_{m-1})\hfill \\ :=\frac{1}{\varphi \left(h\right)}{K}_2\left(\frac{d_{{\theta }_i}(x_i,X_{{\mathit{s}}_i,A_n})}{h}\right)W_{{\mathit{s}}_i,A_n}\times \int W_{{\mathit{s}}_{(1,\dots ,\ell -1,\ell ,\dots ,m-1)},A_n}\prod _{\underset{j\ne i}{j=1}}^{m-1}\frac{1}{\varphi \left(h\right)}{K}_2\left(\frac{d_{{\theta }_j}(x_j,{\nu }_{{\mathit{s}}_j,A_n})}{h}\right)\hfill \\ \times \mathbb{P}(d{\nu }_1,\dots ,d{\nu }_{\ell -1},d{\nu }_{\ell },\dots ,d{\nu }_{m-1}),\hfill \end{array}$$

(25)

with $\mathbb{P}$ is the underlying probability measure, and define

$$f_{i,i_1,\dots ,i_{m-1}}^{(\ell )}:=\sum _{\ell =1}^m{\xi }_{i_1}\dots {\xi }_{i_{\ell -1}}{\xi }_i{\xi }_{i_{\ell }}\dots {\xi }_{i_{m-1}}\left({\tilde{H}}^{(\ell )}\left(z\right)-{\theta }^{(\ell )}\left(i;i_1,\dots ,i_{m-1}\right)\right).$$

(26)

Then, the first order projection can be defined as

$${\widehat{H}}_{1,i}(\mathbf{x},\mathit{\theta },\mathbf{u}):=\frac{(n-m)!}{(n-1)!}\sum _{I_{n-1}^{m-1}(-i)}{f}_{i,i_1,\dots ,i_{m-1}}^{(\ell )},$$

(27)

where

$$I_{n-1}^{m-1}(-i):=\left\{1\le i_1<\dots <i_{m-1}\le n\mathrm{and}{i}_j\ne i\mathrm{for}\mathrm{all}j\in \{1,\dots ,m-1\}\right\}.$$

For the remainder terms, we denote by $\mathbf{i}\setminus i_{\ell }:=(i_1,\dots ,i_{l-1},i_{l+1},\dots ,i_m)$ and for $\ell \in \{1,\dots ,m\}$, let

$$H_{2,i}\left(\mathbf{z}\right):=H\left(\mathbf{z}\right)-\sum _{l=1}^m{\tilde{H}}_{\mathbf{i}\setminus i_{\ell }}^{(\ell )}\left(z_{\ell }\right)+(m-1)\vartheta \left(\mathbf{i}\right),$$

(28)

where

$${\tilde{H}}_{\mathbf{i}\setminus i_{\ell }}^{(\ell )}\left(z_{\ell }\right)=\mathbb{E}\left(H\left(Z_1,\dots ,Z_{\ell -1},z,Z_{\ell +1}{Z}_{m-1}\right)\right),$$

defined in (25), this projection gives the following remainder term:

$${\widehat{\psi }}_2(\mathbf{x},\mathit{\theta },\mathbf{u}):=\frac{(n-m)!}{\left(n\right)!}\sum _{\mathbf{i}\in I_n^m}{\xi }_{i_1}\dots {\xi }_{i_m}{H}_{2,\mathbf{i}}\left(\mathit{z}\right).$$

(29)

Finally, using Equations (27) and (29), and under conditions that

$$\begin{array}{ccc}\hfill \mathbb{E}\left({\widehat{H}}_{1,i}(\mathbf{X},\mathit{\theta },\mathbf{u})\right)& =& 0,\hfill \end{array}$$

(30)

$$\begin{array}{ccc}\hfill \mathbb{E}\left(H_{2,\mathbf{i}}(\mathbf{Z}\mid Z_k)\right)& =& 0a.s.,\hfill \end{array}$$

(31)

we obtain the Hoeffding [125] decomposition:

$$\begin{array}{ccc}\hfill \widehat{\psi }(\mathbf{x},\mathit{\theta },\mathbf{u})-\mathbb{E}\left(\widehat{\psi }(\mathbf{x},\mathit{\theta },\mathbf{u})\right)& =& \frac{1}{n}\sum _{i=1}^n{\widehat{H}}_{1,i}(\mathbf{x},\mathit{\theta },\mathbf{u})+{\widehat{\psi }}_{2,\mathbf{i}}(\mathbf{x},\mathit{\theta },\mathbf{u})\hfill \\ & :=& {\widehat{\psi }}_1(\mathbf{x},\mathit{\theta },\mathbf{u})+{\widehat{\psi }}_2(\mathbf{x},\mathit{\theta },\mathbf{u}).\hfill \end{array}$$

3.2. Strong Uniform Convergence Rate

We begin by presenting the following general result regarding the convergence rate of the U-process defined in (22).

Proposition 1.

Let ${\mathcal{F}}_m{\mathcal{K}}^m$ be a measurable VC-subgraph class of functions, satisfying Assumption 7. Additionally, assume that Assumptions 2, 3, 4(B1), 5, 6 are satisfied. Then, the following result holds:

$$\begin{array}{cc}\hfill \underset{{\mathcal{F}}_m{\mathfrak{K}}_{\Theta }^m}{sup}\underset{\theta \in {\Theta }^m}{sup}\underset{\mathbf{x}\mathit{\in }{\mathcal{H}}^m}{sup}\underset{\mathbf{u}\in {[0,1]}^m}{sup}\left|\widehat{\psi }(\mathbf{x},\mathit{\theta },\mathbf{u})-\mathbb{E}\left[\widehat{\psi }(\mathbf{x},\mathit{\theta },\mathbf{u})\right]\right|& =O_{P_{.\mid \mathit{S}}}\left(\sqrt{\frac{logn}{n{h}^{md}\varphi \left(h\right)}}\right),P_{\mathit{S}}-a.s.\hfill \end{array}$$

Next, the uniform rate of convergence of the estimator (12) of the mean function $r^{\left(m\right)}$ in the model (4) will be given, using the results of the last proposition.

Theorem 1.

Let ${\mathcal{F}}_m{\mathcal{K}}^m$ be a measurable VC-subgraph class of functions satisfying Assumption 7. Let $I_h={[C_{1}h,1-C_{1}h]}^{dm}$ and let $S_c$ be a compact subset of ${\mathcal{H}}^m$. Suppose that

$$\underset{\mathit{u}\in {[0,1]}^d}{inf}f\left(\mathit{u}\right)>0.$$

Then, under Assumptions 1–3, Condition (B1) in Assumptions 4–6 (with $W_{{\mathit{s}}_{\mathbf{i}},A_n}=1$ and ${ϵ}_{{\mathit{s}}_{\mathbf{i}},A_n}$), the following result holds for $P_{\mathit{S}}$ almost surely

$$\begin{array}{c}{\displaystyle \underset{{\mathcal{F}}_m{\mathfrak{K}}_{\Theta }^m}{sup}\underset{\mathit{\theta }\in {\Theta }^m}{sup}\underset{\mathbf{x}\in S_c}{sup}\underset{\mathit{u}\in I_h}{sup}\left|{\widehat{r}}_n^{\left(m\right)}(\mathbf{x},\mathbf{u};h_n)-r^{\left(m\right)}(\mathbf{x},\mathbf{u};h_n)\right|}\hfill \\ =O_{P_{.|\mathit{S}}}\left(\sqrt{logn/n{h}^{md}\varphi \left(h\right)}+h^{2\wedge \alpha }+\frac{1}{A_n^{dp}{\varphi }^{1/m}\left(h\right)}\right),\hfill \end{array}$$

where $p=min\{1,\rho \}$, and $\rho >0$ given in Definition 1.

Remark 5.

We can consider the scenario where $\Theta ={\Theta }_n$, where ${\Theta }_n$ satisfies the conditions $card\left({\Theta }_n\right)=n^{\alpha }$ with $\alpha >0$, and for every $\theta \in {\Theta }_n$, we have ${〈\theta -{\theta }_0,\theta -{\theta }_0〉}^{1/2}\le C_7{b}_n$, where $b_n$ converges to zero, as discussed in [62].

The functional directions set, ${\Theta }_n$, is constructed following a similar approach to [54], as outlined below:

(i) Each direction $\theta \in {\Theta }_n$ is derived from a $d_n$-dimensional space formed by $\mathrm{B}$-spline basis functions, denoted by $\left\{e_1(·),\dots ,e_{d_n}(·)\right\}$. Thus, we express directions as

$$\begin{array}{c}\hfill \theta (·)=\sum _{j=1}^{d_n}{\alpha }_j{e}_j(·)\mathit{where}\left({\alpha }_1,\dots ,{\alpha }_{d_n}\right)\in \mathcal{V}\end{array}$$

(32)

(ii) The set of coefficient vectors in (32), denoted by $\mathcal{V}$, is generated through the following steps:

Step 1

For each $\left({\beta }_1,\dots ,{\beta }_{d_n}\right)\in {\mathcal{C}}^{d_n}$, where $\mathcal{C}=\left\{c_1,\dots ,c_J\right\}\subset {\mathbb{R}}^J$ represents a set of J ’seed-coefficients’, construct the initial functional direction as

$${\theta }_{\mathit{init}}(·)=\sum _{j=1}^{d_n}{\beta }_j{e}_j(·).$$

Step 2 

For each ${\theta }_{\mathit{init}}$ from  Step 1   satisfying ${\theta }_{\mathit{init}}\left(t_0\right)>0$, where $t_0$ denotes a fixed value in the domain of ${\theta }_{\mathit{init}}(·)$, compute $〈{\theta }_{\mathit{init}},{\theta }_{\mathit{init}}〉$ and form $\left({\alpha }_1,\dots ,{\alpha }_{d_n}\right)=\left({\beta }_1,\dots ,{\beta }_{d_n}\right)/{〈{\theta }_{\mathit{init}},{\theta }_{\mathrm{init}}〉}^{1/2}$.

Step 3 

Define $\mathcal{V}$ as the collection of vectors $({\alpha }_1,\dots ,{\alpha }_{d_n})$ obtained in  Step 2. Consequently, the final set of permissible functional directions is represented as

$${\Theta }_n=\left\{\theta (·)=\sum _{j=1}^{d_n}{\alpha }_j{e}_j(·);\left({\alpha }_1,\dots ,{\alpha }_{d_n}\right)\in \mathcal{V}\right\}.$$

Remark 6.

It is important to highlight that the error term $A_n^{-dp}{\varphi }_{\mathbf{x},\mathit{\theta }}^{-1/m}\left(h_n\right)$ arises from approximating the functional time series ${\mathit{X}}_{\mathit{s},A_n}$ with a functional stationary random field ${\mathit{X}}_{\mathit{u}}\left(\mathit{s}\right)$.

4. Weak Convergence for Kernel Estimators

In this section, our focus lies in examining the weak convergence of the conditional U-process, as expressed in Equation (12), when dealing with absolutely regular spatial observations. The following theorem stands as the primary result in this study, outlining the weak convergence of the functional locally stationary random field estimator. Let us denote, for ${\phi }_1,{\phi }_2\in {\mathcal{F}}_m$,

$$\begin{array}{ccc}\hfill \sigma ({\phi }_1,{\phi }_2)& =& \underset{n\to \infty }{lim}{E}_{.\mid \mathit{S}}\left(\sqrt{n{h}^{md}{\varphi }_{\mathbf{x},\mathit{\theta }}^{1/m}\left(h_n\right)}\right({\widehat{r}}_n^{\left(m\right)}({\phi }_1,\mathbf{x},\mathit{\theta },\mathbf{u};h_n)-r^{\left(m\right)}({\phi }_1,\mathbf{x},\mathit{\theta },\mathbf{u})\hfill \\ & & \times \sqrt{n{h}^{md}{\varphi }_{\mathbf{x},\mathit{\theta }}^{1/m}\left(h_n\right)}({\widehat{r}}_n^{\left(m\right)}({\phi }_2,\mathbf{x},\mathit{\theta },\mathbf{u};h_n)-r^{\left(m\right)}({\phi }_2,\mathbf{x},\mathit{\theta },\mathbf{u})).\hfill \end{array}$$

(33)

Theorem 2.

Let ${\mathcal{F}}_m{\mathfrak{K}}_{\Theta }^m$ be a measurable VC-subgraph class of functions satisfying Assumption 7. Suppose that $f_{\mathit{S}}\left(\mathit{u}\right)>0$ and ${ϵ}_{{\mathit{s}}_{i_j},A_n}=\sigma \left({\mathit{s}}_{i_j}/A_n,\mathit{x}\right){ϵ}_{i_j}$, where $\sigma (·,·)$ is continuous, and ${\left\{{ϵ}_i\right\}}_{i=1}^n$ is a sequence of i.i.d. random variables with a mean of 0 and variance 1. Moreover, suppose $n{h}^{m(d+1)+4}\to c_0$ for a constant $c_0$. If all assumptions assumed in Theorem 1 hold in addition to the Conditions (B2), (B3), and (B4), then the following result holds for $P_{\mathit{S}}$ almost surely:

$$\begin{array}{c}\hfill \sqrt{n{h}^{md}{\varphi }_{\mathbf{x},\mathit{\theta }}^{1/m}\left(h\right)}\left[{\widehat{r}}_n^{\left(m\right)}(\phi ,\mathbf{x},\mathit{\theta },\mathbf{u};h)-r^{\left(m\right)}(\phi ,\mathbf{x},\mathit{\theta },\mathbf{u})-B_{\mathit{u},\mathit{x},\mathit{\theta }}\right]\end{array}$$

converges to a Gaussian process ${\mathbb{G}}_n$ over ${\mathcal{F}}_m{\mathfrak{K}}_{\Theta }^m$, whose simple paths are bounded and informally continuous with respect to ${\parallel .\parallel }_2-$norm, with a covariance function given by (33), and where the bias term $B_{\mathit{u},\mathit{x},\mathit{\theta }}=O_{P_{.\mid \mathit{S}}}\left(h^{2\wedge \alpha }\right)$.

Remark 7.

Set $A_n^d=O\left(n^{1-{\overline{\eta }}_1}\right)$ for some ${\overline{\eta }}_1\in [0,1),A_{1,n}=O\left(A_n^{{\gamma }_{A_1}}\right),A_{2,n}=O\left(A_n^{{\gamma }_{A_2}}\right)$ with $0<{\gamma }_{A_2}<{\gamma }_{A_1}<1/3$ and $p=min\{1,\rho \}=1$. Assume that we can take a sufficiently large $\zeta >2$ such that $\frac{2}{\zeta }<\left(1-{\overline{\eta }}_1\right)\left(1-3{\gamma }_{A_1}\right)$. Then, Assumption 4 is satisfied for $d\ge 1$.

Remark 8.

It is straightforward to modify the proofs of our results to show that it remains true when the entropy condition is substituted by the bracketing condition. (Given two functions l and u, the interval $[l,u]$ represents the set of all functions f such that $l\le f\le u$. An ε-bracket is defined as $[l,u]$ with $\parallel u-l\parallel <\epsilon$. The bracketing number $N_{\left[\right]}(\mathcal{F},\parallel ·\parallel ,\epsilon )$ corresponds to the minimum number of ε-brackets required to encompass the class $\mathcal{F}$. The entropy with bracketing is expressed as the logarithm of the bracketing number. It is important to note that, in the definition of the bracketing number, the upper and lower bounds u and l of the brackets need not be part of $\mathcal{F}$ itself, but they are assumed to have finite norms, refer to Definition 2.1.6 in [108].) For some $C_0>0$ and $v_0>0,$

$${\mathcal{N}}_{\left[\right]}\left(\mathcal{F},L_2\left(\mathbb{P}\right),ϵ\right)\le C_0{ϵ}^{-v_0},0<ϵ<1.$$

Remark 9.

There are essentially no constraints on the choice of the kernel function in our framework, as long as it satisfies some mild conditions. However, selecting the bandwidth is a more intricate task. It is crucial for achieving a good consistency rate, impacting the bias size of the estimate significantly. Generally, we aim for a bandwidth selection that strikes a balance between bias and variance, adapting to the applied criteria and the available data. This flexibility cannot be achieved through classical methods. For more details and discussions on this subject, interested readers can refer to [126,127]. It would be interesting to establish uniform-in-bandwidth central limit theorems in our setting. Specifically, we want to let $h>0$ vary such that $h_n^{\prime }\le h\le h_n^{″}$, where ${\left\{h_n^{\prime }\right\}}_{n\ge 1}$ and ${\left\{h_n^{″}\right\}}_{n\ge 1}$ are two sequences of positive constants, ensuring $0<h_n^{\prime }\le h_n^{″}<\infty$. For either choice of $h_n=h_n^{\prime }$ or $h_n=h_n^{″}$ fulfilling our conditions, it will be of interest to demonstrate that

$$\underset{h_n^{\prime }\le h\le h_n^{″}}{sup}\sqrt{n{h}^{md}{\varphi }_{\mathbf{x},\mathit{\theta }}^{1/m}\left(h_n\right)}\left[{\widehat{r}}_n^{\left(m\right)}(\phi ,\mathbf{x},\mathit{\theta },\mathbf{u};h)-r^{\left(m\right)}(\phi ,\mathbf{x},\mathit{\theta },\mathbf{u})-B_{\mathit{u},\mathit{x},\mathit{\theta }}\right]$$

converges to a Gaussian process ${\mathbb{G}}_n$ over ${\mathcal{F}}_m{\mathfrak{K}}_{\mathbf{\Theta }}^m$.

5. Applications

5.1. Metric Learning

Metric learning has attracted considerable attention in recent years due to its ability to tailor the metric to the underlying data. For an extensive overview of metric learning and its diverse applications, please refer to [128,129]. This concept finds utility in various fields, ranging from computer vision to bioinformatics-driven information retrieval. To illustrate the practical significance of metric learning, consider the supervised classification scenario described in [129]. We examine dependent copies

$$({\mathit{X}}_{{\mathit{s}}_{\mathbf{1}},A_n},Y_1),\dots ,({\mathit{X}}_{{\mathit{s}}_{\mathit{n}},A_n},Y_n)$$

of an $\mathcal{H}\times \mathcal{Y}$ valued random couple $(X,Y)$, where $\mathcal{H}$ represents a feature space and $\mathcal{Y}=\{1,\dots ,C\}$ is a finite set of labels with $C\ge 2$. The objective of metric learning in this context is to identify a measure under which points with the same label are close together, while those with different labels are far apart. The risk of a metric D is conventionally defined as follows:

$$R\left(D\right)=\mathbb{E}\left[\varphi \left(\left(1-D\left(X,X^{\prime }\right)·\left(21\mathrm{I}\left\{Y=Y^{\prime }\right\}-1\right)\right)\right],\right.$$

(34)

where $\varphi \left(u\right)$ is a convex loss function upper bounding the indicator function $1\mathrm{I}\{u\ge 0\}$—for instance, the hinge loss $\varphi \left(u\right)=max(0,1-u)$. To estimate $R\left(D\right)$, the natural empirical estimator is given by

$$\begin{array}{ccc}\hfill R_n\left(D\right)& =& \frac{2}{n(n-1)}\sum _{1\le i<j\le n}\overline{K}\left(\frac{{\mathbf{u}}_i-{\mathbf{s}}_i/A_n}{h_n}\right)\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_j/A_n}{h_n}\right)\hfill \\ & & \times \varphi \left(\left(D\left({\mathit{X}}_{{\mathit{s}}_{\mathit{i}},A_n},{\mathit{X}}_{{\mathit{s}}_{\mathit{j}},A_n}\right)-1\right)·\left(21\mathrm{I}\left\{Y_i=Y_j\right\}-1\right)\right),\hfill \end{array}$$

(35)

which is a one-sample U-statistic of degree two with the kernel given by

$${\phi }_{\mathrm{D}}\left((x,y),\left(x^{\prime },y^{\prime }\right)\right)=\varphi \left(\left(D\left(x,x^{\prime }\right)-1\right)·\left(21\mathrm{I}\left\{y=y^{\prime }\right\}-1\right)\right).$$

The convergence of a minimizer of (35) to (34) in the non-spatial setting has been studied within the frameworks of algorithmic stability [130], algorithmic robustness [131], and based on the theory of U-processes under appropriate regularization [132].

5.2. Ranking Problems

For its great importance, the problem of ranking instances has received particular attention in machine learning. The ordering problems have many applications in different areas of banking (data mining process for direct marketing data extraction), document type classification, and so on. In some specific ranking problems, it is necessary to compare two different observations based on their observed characteristics and decide which one is better instead of simply classifying them. The problems of ordering/ranking are frequent problems in which U-statistics come into play. In this kind of challenge, the aim is to establish a universal and consistent ordering method. Suppose that we want to establish an order between the first components of the two pairs $\left(\mathbf{X},\mathbf{Y}\right),\left({\mathbf{X}}^{\prime },{\mathbf{Y}}^{\prime }\right)$ of independent and identically distributed observations in $\mathcal{X}\times \mathbb{R}$. The variables $\mathbf{Y}$ and ${\mathbf{Y}}^{\prime }$ are respective labels of the variables $\mathbf{X}$ and ${\mathbf{X}}^{\prime }$ that we want to order by observing them (and not their labels). Usually, we decide that $\mathbf{X}$ is better than ${\mathbf{X}}^{\prime }$ if $\mathbf{Y}>{\mathbf{Y}}^{\prime }$. To see things more clearly, we introduce the new variable

$$\mathbf{Z}=\frac{\mathbf{Y}-{\mathbf{Y}}^{\prime }}{2},$$

then $\mathbf{Y}>{\mathbf{Y}}^{\prime }$ is equivalent to $\mathbf{Z}>0.$ As mentioned, the goal is to establish a classification rule between $\mathbf{X}$ and ${\mathbf{X}}^{\prime }$ with minimal risk, i.e., the probability that the label of the highest-ranked variable is the smallest is small. Mathematically speaking, the decision rule is given by the function

$$r(\mathbf{x},{\mathbf{x}}^{\prime })=\left\{\begin{array}{ccc}1\hfill & \mathrm{if}& \hfill \mathbf{x}>{\mathbf{x}}^{\prime },\\ -1\hfill & & \hfill \mathrm{else}.\end{array}\right.$$

The following ranking risk gives the performance measure of r:

$$L\left(r\right)=\mathbb{P}\left(\mathbf{Z}.r\left(\mathbf{X},{\mathbf{X}}^{\prime }\right)\right).$$

A natural estimate for $L(·)$ according to [133] is

$$L_n\left(r\right):=\frac{1}{n(n-1)}\sum _{i\ne j}{1\mathrm{I}}_{\left\{{\mathbf{Z}}_{i,j}.r({\mathbf{X}}_i,{\mathbf{X}}_j)<0\right\}},$$

where $\left({\mathbf{X}}_1,{\mathbf{Y}}_1\right),\dots ,\left({\mathbf{X}}_n,{\mathbf{Y}}_n\right)$ are n independent, identically distributed copies of $(\mathbf{X},\mathbf{Y})$, and ${\mathbf{Z}}_{i,j}=\frac{{\mathbf{Y}}_i-{\mathbf{Y}}_j}{2}.$ One can easily see that $L_n$ is a U-statistic with $k=2.$ For more details the reader is invited to consult [133,134].

5.3. Multipartite Ranking

Let us present the problem from [129]. Consider a random vector $X\in \mathcal{H}$ representing attributes or features and associated ordinal labels $Y\in \{1,\dots ,K\}$. In multipartite ranking, the objective, given a training set of labeled examples, is to order the features according to the assigned labels. This statistical learning problem finds applications in various fields such as medicine, finance, search engines, and e-commerce. Rankings are typically established using a scoring function s: $\mathcal{H}\to \mathbb{R}$, which maps the feature space to the natural order on the real line. The ROC manifold, or its summary measure VUS (Volume Under the ROC Surface), serves as the benchmark for evaluating the ranking performance of $s\left(x\right)$. According to [135], the most effective scoring functions are those optimal for all bipartite subproblems. Specifically, they are increasing transformations of the likelihood ratio ${\mathrm{dF}}_{k+1}/{\mathrm{dF}}_k$, where ${\mathrm{F}}_k$ denotes the class-conditional distribution for the kth class. In cases where there is a set of optimal scoring functions, ref. [135] demonstrated that it coincides with the set of functions maximizing the space under the ROC surface

$$VUS\left(s\right)=\mathbb{P}\left\{s\left({\mathit{X}}_{{\mathit{s}}_{\mathbf{1}}}\right)<\cdots <s\left({\mathit{X}}_{{\mathit{s}}_{\mathit{K}}}\right)\mid Y_1=1,\dots ,Y_K=K\right\}.$$

Given K independent samples $\left({\mathit{X}}_{{\mathit{s}}_{\mathbf{1}},A_{n_k}}^{\left(k\right)},\dots ,{\mathit{X}}_{{\mathit{s}}_{{\mathit{n}}_{\mathit{k}}},A_{n_k}}^{\left(k\right)}\right)$ with distribution $F_k\left(dx\right)$ for $k=1,\dots ,K$, the empirical counterpart of the VUS can be written in the following way:

$$\begin{array}{ccc}\hfill \widehat{VUS}\left(s\right)& =& \frac{1}{{\displaystyle \prod _{k=1}^K{n}_k}}\sum _{i_1=1}^{n_1}\cdots \sum _{i_k=1}^{n_K}\prod _{j=1}^K\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)\hfill \\ & & 1\mathrm{I}\left\{s\left({\mathit{X}}_{{\mathit{s}}_{i_1},A_{n_1}}^{\left(1\right)}\right)<\cdots <s\left({\mathit{s}}_{i_K},A_{n_K}^{\left(K\right)}\right)\right\}.\hfill \end{array}$$

(36)

The empirical VUS (36) is a K-sample U-statistic of degree $(1,\dots ,K)$ with a kernel given by

$${\phi }_s\left(x_1,\dots ,x_K\right)=1\mathrm{I}\left\{s\left(x_1\right)<\cdots <s\left(x_K\right)\right\}.$$

5.4. Discrimination

Now, we implement the findings in addressing the discrimination problem detailed in Section 3 of [136], with additional reference to [137]. We adopt similar notations and configurations. Consider any function $\phi (·)$ with a finite range, say $1,\dots ,M$. Define sets

$$A_j=\left\{({\mathbf{y}}_1,\dots ,{\mathbf{y}}_k):\phi ({\mathbf{y}}_1,\dots ,{\mathbf{y}}_k)=j\right\},1\le j\le M.$$

These sets form a partition of the feature space. Predicting the value of $\phi ({\mathbf{y}}_1,\dots ,{\mathbf{y}}_k)$ is equivalent to predicting the set in the partition to which $({\mathbf{Y}}_1,\dots ,{\mathbf{Y}}_k)$ belongs. For any discrimination rule $g(·)$, the following inequality holds:

$$\mathbb{P}(g({\mathbf{X}}_1,\dots ,{\mathbf{X}}_m)=\phi ({\mathbf{Y}}_1,\dots ,{\mathbf{Y}}_m))\le \sum _{j=1}^M{\int }_{\{\mathbf{x}:g(\mathbf{t})=j\}}max{\mathfrak{m}}^j(\mathbf{t},\mathit{\theta })d\mathbb{P}\left(\mathbf{t}\right),$$

where, for $\mathit{u}=\frac{\mathbf{i}}{A_n}$,

$${\mathfrak{m}}^j(\mathbf{t},\mathit{\theta })={\mathfrak{m}}^j(\mathbf{t},\mathit{\theta },\mathbf{u})=\mathbb{P}(\phi ({\mathbf{Y}}_1,\dots ,{\mathbf{Y}}_m)=j\mid \langle {\mathbf{X}}_1,\dots ,{\mathbf{X}}_m,\mathit{\theta }\rangle =\langle \mathbf{t},\mathit{\theta }\rangle ),\mathbf{t}\in {\mathcal{H}}^m.$$

Equality in the above holds if

$$g_0(\mathbf{t},\mathit{\theta })=arg\underset{1\le j\le M}{max}{\mathfrak{m}}^j(\mathbf{t},\mathit{\theta }).$$

The function $g_0(·)$ is the Bayes rule and the associated probability of error

$${\mathbf{L}}^{*}=1-\mathbb{P}(g_0({\mathbf{X}}_1,\dots ,{\mathbf{X}}_m)=\phi ({\mathbf{Y}}_1,\dots ,{\mathbf{Y}}_m))=1-\mathbb{E}\left\{\underset{1\le j\le M}{max}{\mathfrak{m}}^j(\mathbf{t},\mathit{\theta })\right\}$$

is the Bayes risk. Each of the unknown functions ${\mathfrak{m}}^j(·)$ can be consistently estimated by methods discussed in the preceding sections. For $1\le j\le M$, let

$$\begin{array}{c}\hfill {\displaystyle {\mathfrak{m}}_n^j(\mathbf{x},\mathit{\theta },\mathbf{u})=\sum _{\mathbf{i}\in I_n^m}1\mathrm{I}\{\phi ({\mathbf{Y}}_{{\mathit{s}}_{i_1},A_n},\dots ,{\mathbf{Y}}_{{\mathit{s}}_{i_m,A_n}})=j\}{W}_{\mathbf{i}}(\mathbf{x},\mathit{\theta },\mathbf{u}),}\end{array}$$

where

$$\begin{array}{c}\hfill W_{\mathbf{i}}^{\left(m\right)}(\mathbf{x},\mathit{\theta },\mathbf{u})=\frac{{\displaystyle \prod _{j=1}^m\left\{{\displaystyle \prod _{\ell =1}^d{K}_1\left(\frac{u_{j,\ell }-\frac{{\mathit{s}}_{i_{j,\ell }}}{A_n}}{h_n}\right)K_2\left(\frac{d_{{\theta }_j}(x_j,{\mathbf{X}}_{{\mathit{s}}_{i_j},A_n})}{h_n}\right)}\right\}}}{{\displaystyle \sum _{\mathbf{i}\in I_n^m}\prod _{j=1}^m\left\{\prod _{\ell =1}^d{K}_1\left(\frac{u_{j,\ell }-\frac{{\mathbf{s}}_{i_{j,\ell }}}{A_n}}{h_n}\right)K_2\left(\frac{d_{{\theta }_j}(x_j,{\mathbf{X}}_{{\mathit{s}}_{i_j},A_n})}{h_n}\right)\right\}}}.\end{array}$$

(37)

Define

$$g_{0,n}(\mathbf{t},\mathit{\theta })=arg\underset{1\le j\le M}{max}{\mathfrak{m}}_n^j(\mathbf{t},\mathit{\theta },\mathbf{u}).$$

Introduce

$${\mathbf{L}}_n^{*}=\mathbb{P}(g_{0,n}({\mathbf{X}}_1,\dots ,{\mathbf{X}}_m)\ne \phi ({\mathbf{Y}}_1,\dots ,{\mathbf{Y}}_m)).$$

It can be demonstrated that the discrimination rule $g_{0,n}(·)$ is asymptotically Bayes’ risk consistent:

$${\mathbf{L}}_n^{*}\to {\mathbf{L}}^{*}.$$

This follows from the evident relation

$$\left|{\mathbf{L}}^{*}-{\mathbf{L}}_n^{*}\right|\le 2\mathbb{E}\left[\underset{1\le j\le M}{max}\left|{\mathfrak{m}}_n^j(\mathbf{X},\mathit{\theta },\mathbf{u})-{\mathfrak{m}}^j(\mathit{X},\mathit{\theta })\right|\right].$$

5.5. Kendall Rank Correlation Coefficient

To test the independence of one-dimensional random variables $Y_1$ and $Y_2$, Ref. [138] proposed a method based on the U-statistic $K_n$ with the kernel function:

$$\phi \left(\left(s_1,t_1\right),\left(s_2,t_2\right)\right)={1\mathrm{I}}_{\left\{\left(s_2-s_1\right)\left(t_2-t_1\right)>0\right\}}-{1\mathrm{I}}_{\left\{\left(s_2-s_1\right)\left(t_2-t_1\right)⩽0\right\}·}$$

(38)

Its rejection on the region is of the form $\left\{\sqrt{n}{K}_n>\gamma \right\}$. In this example, we consider a multivariate case. To test the conditional independence of $\mathit{\xi },\mathit{\eta }:Y=(\mathit{\xi },\mathit{\eta })$ given X, we propose a method based on the conditional U-statistic:

$${\displaystyle {\widehat{r}}_n^{\left(2\right)}(\phi ,\mathbf{x},\mathit{\theta },\mathbf{u})=\sum _{\mathbf{i}\in I_n^2}\phi \left(Y_{i_1},Y_{i_2}\right)W_{\mathbf{i}}^{\left(2\right)}(\mathbf{x},\mathit{\theta },\mathbf{u}),}$$

where $\mathbf{x}=\left(x_1,x_2\right)\in \mathbb{I}\subset {\mathbb{R}}^2$ and $\phi (·)$ is Kendall’s kernel (38). Suppose that $\mathit{\xi }$ and $\mathit{\eta }$ are $d_1$ and $d_2$-dimensional random vectors, respectively, and $d_1+d_2=d$. Furthermore, suppose that ${\mathrm{Y}}_1,\dots ,{\mathrm{Y}}_n$ are observations of $(\mathit{\xi },\mathit{\eta })$ that we are interested in testing:

$${\mathrm{H}}_0:\mathit{\xi }and\mathit{\eta }areconditionallyindependentgivenX.vs{\mathrm{H}}_a:{\mathrm{H}}_{0}isnottrue.$$

(39)

Let $\mathbf{a}=\left({\mathbf{a}}_1,{\mathbf{a}}_2\right)\in {\mathbb{R}}^d$ such as $\parallel \mathbf{a}\parallel =1$ and ${\mathbf{a}}_1\in {\mathbb{R}}^{d_1},{\mathbf{a}}_2\in {\mathbb{R}}^{d_2}$, and $F(·),G(·)$ be the distribution functions of $\mathit{\xi }$ and $\mathit{\eta }$ respectively. Suppose $F^{{\mathbf{a}}_1}(·)$ and $G^{{\mathbf{a}}_2}(·)$ to be continuous for any unit vector $\mathbf{a}=\left({\mathbf{a}}_1,{\mathbf{a}}_2\right)$ where ${\mathrm{F}}^{{\mathbf{a}}_1}\left(t\right)=\mathbb{P}\left({\mathbf{a}}_1^{\top }\mathit{\xi }<t\right)$ and ${\mathrm{G}}^{{\mathbf{a}}_2}\left(t\right)=\mathbb{P}\left({\mathbf{a}}_2^{\top }\mathit{\eta }<t\right)$ and ${\mathbf{a}}_1^{\mathrm{T}}$ means the transpose of the vector ${\mathbf{a}}_i,1⩽i⩽2$. For $n=2$, let $Y^{\left(1\right)}=\left({\mathit{\xi }}^{\left(1\right)},{\mathit{\eta }}^{\left(1\right)}\right)$ and $Y^{\left(2\right)}=\left({\mathit{\xi }}^{\left(2\right)},{\mathit{\eta }}^{\left(2\right)}\right)$ such as ${\mathit{\xi }}^{\left(i\right)}\in {\mathbb{R}}^{d_1}$ and ${\mathit{\eta }}^{\left(i\right)}\in {\mathbb{R}}^{d_2}$ for $i=1,2$, and:

$${\phi }^{\mathbf{a}}\left({\mathrm{Y}}^{\left(1\right)},{\mathrm{Y}}^{\left(2\right)}\right)=\phi \left(\left({\mathbf{a}}_1^{\top }{\mathit{\xi }}^{\left(1\right)},{\mathbf{a}}_2^{\top }{\mathit{\eta }}^{\left(1\right)}\right),\left({\mathbf{a}}_1^{\top }{\mathit{\xi }}^{\left(2\right)},{\mathbf{a}}_2^{\top }{\mathit{\eta }}^{\left(2\right)}\right)\right).$$

An application of Theorem 1 gives

$$\left|{\widehat{r}}_n^{\left(2\right)}({\phi }^{\mathbf{a}},\mathbf{x},\mathit{\theta },\mathbf{u})-r^{\left(2\right)}({\phi }^{\mathbf{a}},\mathbf{x},\mathit{\theta },\mathbf{u})\right|⟶0,\mathrm{in} \mathrm{probability}.$$

(40)

5.6. Set Indexed Conditional U-Statistics

Our goal is to examine the connections between X and Y by estimating functional operators related to the conditional distribution of Y given X, such as the regression operator, for $C_1\times \cdots \times C_m:=\tilde{\mathbb{C}}$ within a class of sets ${\mathcal{C}}^m$,

$$\begin{array}{c}{\displaystyle {\mathbb{G}}^{\left(m\right)}\left(C_1\times \cdots \times C_m,\mathbf{x},\mathit{\theta },\frac{{\mathit{s}}_{\mathbf{i}}}{A_n}\right)}\hfill \\ =\mathbb{E}\left(\prod _{i=1}^m{\mathbb{1}}_{\{Y_i\in C_i\}}\mid \langle {\mathit{X}}_{{\mathit{s}}_1,A_n},{\theta }_1\rangle =\langle x_1,{\theta }_1\rangle ,\dots ,\langle {\mathit{X}}_{{\mathit{s}}_m,A_n},{\theta }_m\rangle =\langle x_m,{\theta }_m\rangle \right),\mathrm{for}\mathbf{x}\in {\mathcal{H}}^m.\hfill \end{array}$$

We define the metric entropy with the inclusion of the class of sets $\mathcal{C}$ of ${\mathbb{R}}^d$. For each $\epsilon >0$, the covering number is defined as:

$$\begin{array}{cc}\hfill \mathcal{N}& (\epsilon ,\mathcal{C},{\mathbb{G}}^{\left(1\right)}(·\mid \langle \theta ,x\rangle ))\hfill \\ & =inf\{n\in \mathbb{N}:\exists C_1,\dots ,C_n\in \mathcal{C}\mathrm{such}\mathrm{that}\forall C\in \mathcal{C}\exists 1\le i,j\le n\hfill \\ & \mathrm{with}{C}_i\subset C\subset C_j\mathrm{and}{\mathbb{G}}^{\left(1\right)}(C_j\setminus C_i\mid x)<\epsilon \},\hfill \end{array}$$

where $log\left(\mathcal{N}(\epsilon ,\mathcal{C},{\mathbb{G}}^{\left(1\right)}(·\mid \langle \theta ,x\rangle ))\right)$ is called metric entropy with the inclusion of $\mathcal{C}$ with respect to ${\mathbb{G}}^{\left(1\right)}(·\mid \langle \theta ,x\rangle )$. The quantity $log\mathcal{N}(\epsilon ,\mathcal{C},{\mathbb{G}}^{\left(1\right)}(·\mid \langle \theta ,x\rangle ))$ is called metric entropy with inclusion of $\mathcal{C}$ with respect to $\mathbb{G}(·\mid \langle \theta ,x\rangle )$. Estimates for such covering numbers are known for many classes (e.g., [139]). We will often assume below that either $log\mathcal{N}(\epsilon ,\mathcal{C},{\mathbb{G}}^{\left(1\right)}(·\mid \langle \theta ,x\rangle ))$ or $\mathcal{N}(\epsilon ,\mathcal{C},{\mathbb{G}}^{\left(1\right)}(·\mid \langle \theta ,x\rangle ))$ behaves like powers of ${\epsilon }^{-1}$: we say that the condition ($R_{\gamma }$) holds if

$$log\mathcal{N}(\epsilon ,\mathcal{C},{\mathbb{G}}^{\left(1\right)}(·\mid \langle \theta ,x\rangle ))\le H_{\gamma }\left(\epsilon \right),\mathrm{for}\mathrm{all}\epsilon >0,$$

(41)

where

$$H_{\gamma }\left(\epsilon \right)=\left\{\begin{array}{cc}log\left(A\epsilon \right)\hfill & \mathrm{if}\gamma =0,\hfill \\ A{\epsilon }^{-\gamma }\hfill & \mathrm{if}\gamma >0,\hfill \end{array}\right.$$

for some constants $A,r>0$. As in [140], it is worth noticing that the condition (41), $\gamma =0$, holds for intervals, rectangles, balls, ellipsoids, and for classes which are constructed from the above by performing set operations union, intersection and complement finitely many times. The classes of convex sets in ${\mathbb{R}}^d$ ($d\ge 2$) fulfill the condition (41), $\gamma =(d-1)/2$. This and other classes of sets satisfying (41) with $\gamma >0$ can be found in [139]. As a particular case of (12), we estimate ${\mathbb{G}}^{\left(m\right)}(C_1\times \cdots \times C_m\mid \mathbf{x},\mathit{\theta })$

$$\begin{array}{cc}\hfill {\widehat{\mathbb{G}}}_n^{\left(m\right)}(\tilde{\mathbb{C}},\mathbf{x},\mathit{\theta },\mathbf{u},h_n)=& {\displaystyle \sum _{(i_1,\dots ,i_m)\in I(m,n)}\prod _{j=1}^m{\mathbb{1}}_{\{Y_{i_j}\in C_j\}}{W}_{\mathbf{i}}^{\left(m\right)}(\mathbf{x},\mathit{\theta },\mathbf{u})},\hfill \end{array}$$

(42)

where $W_{\mathbf{i}}^{\left(m\right)}$ is defined in (37). One can apply Theorem 1 to infer that

$$\underset{\tilde{\mathbb{C}}\times \tilde{K}\in {\mathcal{C}}^m{\mathcal{K}}^m}{sup}\underset{\mathit{\theta }\in {\Theta }^m}{sup}\underset{\mathbf{x}\in S_c}{sup}\underset{\mathit{u}\in I_h}{sup}\left|{\widehat{\mathbb{G}}}_n^{\left(m\right)}(\tilde{\mathbb{C}},\mathbf{x},\mathit{\theta },\mathbf{u};h_n)-{\mathbb{G}}^{\left(m\right)}(\tilde{\mathbb{C}},\mathbf{x},\mathit{\theta },\mathbf{u})\right|⟶0,\mathrm{in} \mathrm{probability}.$$

(43)

5.7. Generalized U-Statistics

The extension to the case of several samples is straightforward. Consider k independent collections, of independent observations

$$\left\{\left(X_1^{\left(1\right)},Y_1^{\left(1\right)}\right),\left(X_2^{\left(1\right)},Y_2^{\left(1\right)}\right),\dots \right\},\dots ,\left\{\left(X_1^{\left(k\right)},Y_1^{\left(k\right)}\right),\left(X_2^{\left(k\right)},Y_1^{\left(k\right)}\right),\dots \right\}.$$

Let, for $\mathbf{t}\in {\mathcal{H}}^{(m_1+\cdots +m_k)}$,

$$\begin{array}{c}{\displaystyle r^{(m,k)}(\phi ,\mathbf{t},\mathit{\theta },\mathbf{u})=r^{(m,k)}(\phi ,{\mathbf{t}}_1,\dots ,{\mathbf{t}}_k,\dots ,{\mathit{\theta }}_1,\dots ,{\mathit{\theta }}_k,\frac{{\mathit{s}}_{\mathbf{1}}}{A_n},\cdots ,\frac{{\mathit{s}}_{\mathbf{k}}}{A_n})}\hfill \\ = \mathbb{E}\left(\phi \left(Y_1^{\left(1\right)},\dots ,Y_{m1}^{\left(1\right)};\dots ;Y_1^{\left(k\right)},\dots ,Y_{m_k}^{\left(k\right)}\right)\mid \langle \left(X_{1,A_n}^{\left(j\right)},\dots ,X_{m_j,A_n}^{\left(j\right)}\right),{\mathit{\theta }}_j\rangle =\langle {\mathbf{t}}_j,{\mathit{\theta }}_j\rangle ,j=1,\dots ,k\right),\hfill \end{array}$$

where $\phi$ is assumed, without the loss of generality, to be symmetric within each of its k blocks of arguments. Corresponding to the “kernel” $\phi$ and assuming $n_1\ge m_1,\dots ,n_k\ge m_k$, the the conditional U -statistic for estimation of $r^{(m,k)}(\phi ,\mathbf{t})$ is defined as

$$\begin{array}{c}{\displaystyle {\widehat{r}}_n^{(m,k)}(\phi ,\mathbf{t},\mathbf{u},\mathit{\theta },{\mathbf{h}}_n)}\hfill \\ = \frac{{\displaystyle \sum _c\phi \left(Y_{i_{11}}^{\left(1\right)}\dots ,Y_{i_{1m_1}}^{\left(1\right)};\dots ;Y_{i_{k1}}^{\left(k\right)},\dots ,Y_{i_{k{m}_k}}^{\left(k\right)}\right)\mathbf{K}\left(X_{i_{11}}^{\left(1\right)}\dots ,X_{i_{1m_1}}^{\left(1\right)};\dots ;X_{i_{k1}}^{\left(k\right)},\dots ,X_{i_{k{m}_k}}^{\left(k\right)}\right)}}{{\displaystyle \sum _c\mathbf{K}\left(X_{i_{11}}^{\left(1\right)}\dots ,X_{i_{1m_1}}^{\left(1\right)};\dots ;X_{i_{k1}}^{\left(k\right)},\dots ,X_{i_{k{m}_k}}^{\left(k\right)}\right)}},\hfill \end{array}$$

where

$$\begin{array}{c}{\displaystyle \mathbf{K}\left(X_{i_{11}}^{\left(1\right)}\dots ,X_{i_{1m_1}}^{\left(1\right)};\dots ;X_{i_{k1}}^{\left(k\right)},\dots ,X_{i_{k{m}_k}}^{\left(k\right)}\right)}\hfill \\ = \prod _{\nu =1}^k\prod _{j_{\nu }=1}^m\left\{\prod _{\ell =1}^d{K}_1\left(\frac{u_{j_{\nu },\ell }-\frac{{\mathbf{s}}_{i_{j_{\nu },\ell }}}{A_n}}{h_n}\right)K_2\left(\frac{d_{{\theta }_{j_{\nu }}}(x_{j_{\nu }},{\mathbf{X}}_{{\mathit{s}}_{i_{j_{\nu }}},A_n})}{h_n}\right)\right\}.\hfill \end{array}$$

Here $\left\{i_{j1},\dots ,i_{j{m}_j}\right\}$ denotes a set of $m_j$ distinct elements of the set $\left.\{1,2,\dots ,n_j\right\}$$1\le j\le k$, and ${\sum }_c$ denotes summation over all such combinations. The extension of [125] treatment of one-sample U -statistics to the k sample case is due to [141,142]. One can use Theorem 1 gives to infer that

$$\underset{1\le j\le k}{max}\underset{t\in {\mathcal{S}}_{\mathcal{H}}^{m_1+\cdots +m_k}}{sup}\left|{\widehat{r}}_n^{(m,k)}(\phi ,\mathbf{t},\mathbf{u},\mathit{\theta },{\mathbf{h}}_n)-r^{(m,k)}(\phi ,\mathbf{t},\mathbf{u},\mathit{\theta })\right|⟶0\mathrm{in} \mathrm{probability}.$$

(44)

6. Extension to the Censored Case

Consider the triple $(Y,C,\mathbf{X})$ of random variables defined in $\mathbb{R}\times \mathbb{R}\times \mathcal{H}$, where Y is the variable of interest, C is a censoring variable, and $\mathbf{X}$ is a concomitant variable. We work with a sample $\left\{(Y_i,C_i,{\mathbf{X}}_{{\mathit{s}}_i,A_n})\right\}$, replicating identically distributed instances of $(Y,C,\mathbf{X})$ for $n\ge 1$. In the right-censorship model, the pairs $(Y_i,C_i)$, $1\le i\le n$, are not directly observed. Instead, the corresponding information is given by $Z_i:=min\{Y_i,C_i\}$ and ${\delta }_i:=1\mathrm{I}\{Y_i\le C_i\}$, $1\le i\le n$. The observed sample is denoted as ${\mathcal{D}}_n=\{(Z_i,{\delta }_i,{\mathbf{X}}_{{\mathit{s}}_i,A_n}),i=1,\dots ,n\}$. Such censoring is common in survival data in clinical trials or failure time data in reliability studies. We impose assumptions on the distribution of $(\mathbf{X},Y)$. For $-\infty <t<\infty$, let $F_Y\left(t\right)$, $G\left(t\right)$, and $H\left(t\right)$ be the right-continuous distribution functions of Y, C, and Z, respectively. Define $T_L=sup\{t\in \mathbb{R}:L\left(t\right)<1\}$ for any right-continuous distribution function L. Consider a pointwise measurable class $\mathcal{F}$ of real measurable functions defined on $\mathbb{R}$, assuming that $\mathcal{F}$ is of VC-type. We recall the regression function of $\psi \left(Y\right)$ evaluated at $\mathbf{X}=\mathbf{x}$, denoted as $r^{\left(1\right)}(\psi ,\mathbf{x})=\mathbb{E}(\psi \left(Y\right)\mid \mathbf{X}=\mathbf{x})$, when Y is right-censored. To estimate $r^{\left(1\right)}(\psi ,·)$, we utilize Inverse Probability of Censoring Weighted (I.P.C.W.) estimators, which have gained popularity in the censored data literature (see [143,144]). The fundamental concept behind I.P.C.W. estimators can be outlined as follows. Introduce the real-valued function ${\Phi }_{\psi }(·,·)$ defined on ${\mathbb{R}}^2$ by

$${\Phi }_{\psi }(y,c)=\frac{1\mathrm{I}\{y\le c\}\psi (y\wedge c)}{1-G(y\wedge c)}.$$

(45)

Assuming the known function $G(·)$, observe that ${\Phi }_{\psi }(Y_i,C_i)={\delta }_i\psi \left(Z_i\right)/(1-G\left(Z_i\right))$ is observed for every $1\le i\le n$. Additionally, under Assumption ($\mathcal{I}$) stating that C and $(Y,\mathbf{X})$ are independent, we have

$$\begin{array}{ccc}\hfill r^{\left(1\right)}({\Phi }_{\psi },\mathbf{x})& :=& \mathbb{E}({\Phi }_{\psi }(Y,C)\mid \mathbf{X}=\mathbf{x})\hfill \\ \hfill & =& \mathbb{E}\left\{\frac{1\mathrm{I}\{Y\le C\}\psi \left(Z\right)}{1-G\left(Z\right)}\mid \mathbb{X}=\mathbf{x}\right\}\hfill \\ \hfill & =& \mathbb{E}\left\{\frac{\psi \left(Y\right)}{1-G\left(Y\right)}\mathbb{E}(1\mathrm{I}\{Y\le C\}\mid \mathbf{X},Y)\mid \mathbf{X}=\mathbf{x}\right\}\hfill \\ & =& r^{\left(1\right)}(\psi ,\mathbf{x}).\hfill \end{array}$$

(46)

Consequently, any estimate of $r^{\left(1\right)}({\Phi }_{\psi },·)$, derived from fully observed data, serves as an estimate for $r^{\left(1\right)}(\psi ,·)$ as well. This property facilitates the natural extension of most statistical procedures designed for estimating the regression function in the uncensored case to the censored case. For example, constructing kernel-type estimates is particularly straightforward. For $\mathbf{x}\in \mathcal{H}$, $h\ge l_n$, and $1\le i\le n$, define

$$\begin{array}{c}\hfill {\overline{\omega }}_{n,K_{1,2},h_n,i}^{\left(1\right)}(x,\theta ,u):=\frac{{\displaystyle \prod _{\ell =1}^d{K}_1\left(\frac{u_{\ell }-\frac{{\mathbf{s}}_{j,\ell }}{A_n}}{h_n}\right)K_2\left(\frac{d_{\theta }(x,{\mathbf{X}}_{{\mathit{s}}_j,A_n})}{h_n}\right)}}{{\displaystyle \sum _{j=1}^n\prod _{\ell =1}^d{K}_1\left(\frac{u_{\ell }-\frac{{\mathbf{s}}_{j,\ell }}{A_n}}{h_n}\right)K_2\left(\frac{d_{\theta }(x,{\mathbf{X}}_{{\mathit{s}}_j,A_n})}{h_n}\right)}}.\end{array}$$

(47)

Given (45)–(47), and assuming knowledge of $G(·)$, a kernel estimator of $r^{\left(1\right)}(\psi ,·)$ is expressed as

$$\begin{array}{c}\hfill {\breve{r}}_n^{\left(1\right)}(\psi ,x,\theta ,u;h_n)=\sum _{i=1}^n{\overline{\omega }}_{n,K_{1,2},h_n,i}^{\left(1\right)}(x,\theta ,u)\frac{{\delta }_i\psi \left(Z_i\right)}{1-G\left(Z_i\right)}.\end{array}$$

(48)

However, since $G(·)$ is generally unknown, it needs to be estimated. Denote by $G_n^{*}(·)$ the Kaplan–Meier estimator of $G(·)$[145]. With conventions ${\prod }_{\varnothing }=1$ and $0^0=1$, and setting $N_n\left(u\right)={\sum }_{i=1}^{n}1\mathrm{I}\{Z_i\ge u\}$, we have

$$G_n^{*}\left(u\right)=1-\prod _{i:Z_i\le u}{\left\{\frac{N_n\left(Z_i\right)-1}{N_n\left(Z_i\right)}\right\}}^{(1-{\delta }_i)},\mathrm{for}u\in \mathbb{R}.$$

Given this notation, we consider the following estimator of $r^{\left(1\right)}(\psi ,·)$

$$\begin{array}{c}\hfill {\breve{r}}_n^{\left(1\right)*}(\psi ,x,\theta ,u;h_n)=\sum _{i=1}^n{\overline{\omega }}_{n,K_{1,2},h_n,i}^{\left(1\right)}(x,\theta ,u)\frac{{\delta }_i\psi \left(Z_i\right)}{1-G_n^{*}\left(Z_i\right)},\end{array}$$

(49)

as discussed in [143,146]. Adopting the convention $0/0=0$, this quantity is well defined. The expression $G_n^{*}\left(Z_i\right)=1$ holds if and only if $Z_i=Z_{\left(n\right)}$ and ${\delta }_{\left(n\right)}=0$. Here, $Z_{\left(k\right)}$ is the kth ordered statistic associated with the sample $(Z_1,\dots ,Z_n)$ for $k=1,\dots ,n$, and ${\delta }_{\left(k\right)}$ is the ${\delta }_j$ corresponding to $Z_k=Z_j$. Given that the variable of interest is right-censored, functional estimation of the (conditional) law cannot generally be achieved on the complete support. To establish our results, we operate under the following assumptions.

(A.1) 

$\mathcal{F}=\{\psi :={\psi }_{1}1\mathrm{I}\left\{{(-\infty ,\tau )}^m\right\},{\psi }_1\in {\mathcal{F}}_1\}$, where $\tau <T_H$, and ${\mathcal{F}}_1$ is a pointwise measurable class of real measurable functions defined on $\mathbb{R}$ and of VC type.

(A.2) 

The class of functions $\mathcal{F}$ has a measurable and uniformly bounded envelope function $\mathbf{{\rm Y}}$ such that

$$\mathbf{{\rm Y}}(y_1,\dots ,y_k)\ge \underset{\psi \in \mathcal{F}}{sup}\left|\psi (y_1,\dots ,y_k)\right|,y_i\le T_H.$$

Combining the results of Proposition 9.6 and Lemma 9.7 from [147] and Theorem 1, we have, in probability,

$$\begin{array}{c}\hfill \underset{x,\theta ,u}{sup}\left|{\breve{r}}_n^{\left(1\right)*}(\psi ,x,\theta ,u;h_n)-\widehat{\mathbb{E}}\left({\breve{r}}_n^{\left(1\right)*}(\psi ,x,\theta ,u;h_n)\right)\right|\to 0.\end{array}$$

(50)

A right-censored version of an unconditional U-statistic with a kernel of degree $m\ge 1$ is introduced by the principle of a mean-preserving reweighting scheme in [148]. Stute and Wang [149] have proved almost sure convergence of multi-sample U-statistics under random censorship and provided applications by considering the consistency of a new class of tests designed for testing equality in distribution. To overcome potential biases arising from right-censoring of the outcomes and the presence of confounding covariates, Chen and Datta [150] proposed adjustments to the classical U-statistics. Yuan et al. [151] proposed a different way in the estimation procedure of the U-statistic by using a substitution estimator of the conditional kernel given the observed data. To the best of our knowledge, the problem of the estimation of conditional U-statistics has been open until now and serves as the main motivation for this section. A natural extension of the function defined in (45) is given by

$$\begin{array}{c}{\displaystyle {\Phi }_{\psi }(y_1,\dots ,y_m,c_1,\dots ,c_m)}\hfill \\ = \frac{{\displaystyle \prod _{i=1}^m\{1\mathrm{I}\{y_i\le c_i\}\psi (y_1\wedge c_1,\dots ,y_m\wedge c_m)}}{{\displaystyle \prod _{i=1}^m\{1-G(y_i\wedge c_i)\}}}.\hfill \end{array}$$

(51)

From this, we have an analogous relation to (46) given by

$$\begin{array}{c}{\displaystyle \mathbb{E}({\Phi }_{\psi }(Y_1,\dots ,Y_m,C_1,\dots ,C_m)\mid \langle {\mathbf{X}}_1,\dots ,{\mathbf{X}}_m,\mathit{\theta }\rangle =\langle \mathbf{t},\mathit{\theta }\rangle )}\hfill \\ =\mathbb{E}\left(\frac{{\displaystyle \prod _{i=1}^m\{1\mathrm{I}\{Y_i\le C_i\}\psi (Y_1\wedge C_1,\dots ,Y_k\wedge C_m)}}{{\displaystyle \prod _{i=1}^m\{1-G(Y_i\wedge C_i)\}}}\mid \langle {\mathbf{X}}_1,\dots ,{\mathbf{X}}_m,\mathit{\theta }\rangle =\langle \mathbf{t},\mathit{\theta }\rangle \right)\hfill \\ =\mathbb{E}\left(\frac{{\displaystyle \psi (Y_1,\dots ,Y_m)}}{{\displaystyle \prod _{i=1}^m\{1-G\left(Y_i\right)\}}}\mathbb{E}\left(\prod _{i=1}^m\{1\mathrm{I}\{Y_i\le C_i\}\mid (Y_1,{\mathbf{X}}_1),\dots (Y_m,{\mathbf{X}}_m)\right)\mid \langle {\mathbf{X}}_1,\dots ,{\mathbf{X}}_m,\mathit{\theta }\rangle =\langle \mathbf{t},\mathit{\theta }\rangle \right)\hfill \\ =\mathbb{E}\left(\psi (Y_1,\dots ,Y_m)\mid \langle {\mathbf{X}}_1,\dots ,{\mathbf{X}}_m,\mathit{\theta }\rangle =\langle \mathbf{t},\mathit{\theta }\rangle \right)=m_{\psi }(\mathbf{t},\mathit{\theta }).\hfill \end{array}$$

(52)

An analogous estimator to (12) in the censored case is given by

$$\begin{array}{c}{\displaystyle {\breve{r}}_n^{\left(m\right)}(\psi ,\mathbf{t},\mathit{\theta },\mathbf{u};h_n)}\hfill \\ = \sum _{(i_1,\dots ,i_m)\in I(m,n)}\frac{{\displaystyle {\delta }_{i_1}\cdots {\delta }_{i_m}\psi (Z_{i_1},\dots ,Z_{i_m})}}{{\displaystyle (1-G\left(Z_{i_1}\right)\cdots (1-G\left(Z_{i_k}\right))}}{\overline{\omega }}_{n,K_{1,2},h_n,\mathbf{i}}^{\left(m\right)}(\mathbf{t},\mathit{\theta },\mathbf{u}),\hfill \end{array}$$

(53)

where, for $\mathbf{i}=(i_1,\dots ,i_m)\in I_n^m$,

$$\begin{array}{c}\hfill {\overline{\omega }}_{n,K_{1,2},h_n,\mathbf{i}}^{\left(k\right)}(\mathbf{x},\mathit{\theta },\mathbf{u})\frac{{\displaystyle \prod _{j=1}^m\left\{\prod _{\ell =1}^d{K}_1\left(\frac{u_{j,\ell }-\frac{{\mathit{s}}_{i_{j,\ell }}}{A_n}}{h_n}\right)K_2\left(\frac{d_{{\theta }_j}(x_j,{\mathbf{X}}_{{\mathit{s}}_{i_j},A_n})}{h_n}\right)\right\}}}{{\displaystyle \sum _{\mathbf{i}\in I_n^m}\prod _{j=1}^m\left\{\prod _{\ell =1}^d{K}_1\left(\frac{u_{j,\ell }-\frac{{\mathit{s}}_{i_{j,\ell }}}{A_n}}{h_n}\right)K_2\left(\frac{d_{{\theta }_j}(x_j,{\mathbf{X}}_{{\mathit{s}}_{i_j},A_n})}{h_n}\right)\right\}}}.\end{array}$$

(54)

The estimator that we will investigate is given by

$$\begin{array}{c}\hfill {\breve{r}}_n^{\left(m\right)*}(\psi ,\mathbf{t},\mathit{\theta },\mathbf{u};h_n)=\sum _{(i_1,\dots ,i_k)\in I(m,n)}\frac{{\displaystyle {\delta }_{i_1}\cdots {\delta }_{i_k}\psi (Z_{i_1},\dots ,Z_{i_m})}}{{\displaystyle (1-G_n^{*}\left(Z_{i_1}\right)\cdots (1-G_n^{*}\left(Z_{i_m}\right))}}{\overline{\omega }}_{n,K_{1,2},h_n,\mathbf{i}}^{\left(k\right)}(\mathbf{t},\mathit{\theta },\mathbf{u}).\end{array}$$

(55)

Corollary 1.

Under the Assumptions  (A.1)–(A.2)   and the conditions of Theorem 1, we have

$$\begin{array}{c}{\displaystyle \underset{\mathit{\theta }\in {\Theta }^m}{sup}\underset{\mathbf{x}\in {\mathcal{H}}^m}{sup}\underset{\mathit{u}\in I_h,\mathbf{x}\in S_c}{sup}\left|{\breve{r}}_n^{\left(m\right)*}(\psi ,\mathbf{t},\mathit{\theta },\mathbf{u};h_n)-\mathbb{E}{\breve{r}}_n^{\left(m\right)*}(\psi ,\mathbf{t},\mathit{\theta },\mathbf{u};h_n)\right|}\hfill \\ =O_{P_{.|\mathit{S}}}\left(\sqrt{logn/n{h}^{md}\varphi \left(h\right)}+\frac{1}{A_n^{dp}{\varphi }^{1/m}\left(h\right)}\right).\hfill \end{array}$$

In the last corollary, we use the law of iterated logarithm for $G_n^{*}(·)$ established in [152] ensuring that

$$\underset{t\le \tau }{sup}|G_n^{*}-G\left(t\right)|=O\left(\sqrt{\frac{loglogn}{n}}\right),\mathrm{almost}\mathrm{surely}\mathrm{as}n\to \infty .$$

(56)

6.1. Conditional U-Statistics for Left Truncated and Right Censored Data

Keeping in mind the notation of the preceding section, we now introduce a truncation variable denoted as L and assume that $(L,C)$ is independent of Y. Let us consider a situation where we have random vectors $(Z_i{ϵ}_i,{\Delta }_i)$, with ${ϵ}_i=1\mathrm{I}(L_i\le Z_i)$. In this section, we aim to define U-statistics conditional for data that are left truncated and right censored (LTRC), by following ideas from [153] in the unconditional setting. To achieve this, we propose an extension of the function (51) for LTRC data as follows:

$$\begin{array}{c}{\displaystyle {\tilde{\Phi }}_{\psi }(y_1,\dots ,y_k,l_1,\dots ,l_k,c_1,\dots ,c_k)}\hfill \\ = \frac{{\displaystyle \psi (y_1\wedge c_1,\dots ,y_k\wedge c_k)\prod _{i=1}^{k}1\mathrm{I}\{y_i\le c_i\}1\mathrm{I}\{l_i\le z_i\}}}{{\displaystyle \prod _{i=1}^k\mathbb{P}\left(l_i<z_i<c_i\right)}}.\hfill \end{array}$$

According to (52), we obtain that

$$\begin{array}{ccc}& & \mathbb{E}({\Phi }_{\psi }(Y_1,\dots ,Y_k,L_1,\dots ,L_k,C_1,\dots ,C_k)\mid ({\mathbf{X}}_1,\dots ,{\mathbf{X}}_k)=\mathbf{t})\hfill \\ & & =\mathbb{E}\left(\psi (Y_1,\dots ,Y_k)\mid ({\mathbf{X}}_1,\dots ,{\mathbf{X}}_k)=\mathbf{t}\right).\hfill \end{array}$$

An analog estimator to (12) for LTRC data can be expressed as follows:

$$\begin{array}{c}{\displaystyle {\breve{\stackrel{˘}{r}}}_n^{\left(k\right)}(\psi ,\mathit{\theta },\mathbf{u},\mathbf{x},h_n)}\hfill \\ = \sum _{(i_1,\dots ,i_k)\in I_n^k}\frac{{\displaystyle {\Delta }_{i_1}\cdots {\Delta }_{i_k}{ϵ}_{i_1}\cdots {ϵ}_{i_k}\psi (Z_{i_1},\dots ,Z_{i_k})}}{{\displaystyle \mathbb{P}\left(L_{i_1}<Z_{i_1}<C_{i_1}\right)\cdots \mathbb{P}\left(L_{i_k}<Z_{i_k}<C_{i_k}\right)}}{\overline{\omega }}_{n,\mathbf{i}}^{\left(k\right)}(\mathit{\theta },\mathbf{u},\mathbf{x}),\hfill \end{array}$$

(57)

where ${\overline{\omega }}_{n,\mathbf{i}}^{\left(k\right)}(\mathit{\theta },\mathbf{u},\mathbf{x})$ is defined as in (54). As $\mathbb{P}\left(L_i<Z_i<C_i\right)$ is not known, we need to estimate it. We introduce $N_i\left(t\right)=$$1\mathrm{I}(L_i<Z_i\le t,{\Delta }_i=1)$ and $N_i^c\left(t\right)=1\mathrm{I}\left(L_i<Z_i\le t,{\Delta }_i=0\right)$ as the counting process corresponding to the variable of interest and the censoring variable. Furthermore, let

$$N\left(t\right)=\sum _{i=1}^n{N}_i\left(t\right)$$

and

$$N^c\left(t\right)=\sum _{i=1}^n{N}_i^c\left(t\right).$$

We introduce the risk indicators as $R_i\left(t\right)=1\mathrm{I}\left(Z_i\ge t\ge L_i\right)$ and

$$R\left(t\right)=\sum _{i=1}^n{R}_i\left(t\right).$$

It is important to note that the risk set $R\left(t\right)$ at t contains the subjects who entered the study before t and are still under study at t. Indeed, $N_i^c\left(t\right)$ is a local sub-martingale with the appropriate filtration ${\mathbb{F}}_t$. The martingale associated with the censoring counting process with filtration ${\mathbb{F}}_t$ is given by

$$M_i^c\left(t\right)=N_i^c\left(t\right)-{\int }_0^t{R}_i\left(u\right){\lambda }_c\left(u\right)\mathrm{d}u,i=1,2\dots ,n.$$

Here, ${\lambda }_c(·)$ represents the hazard function associated with the censoring variable C under left truncation. The cumulative hazard function for the censoring variable C is defined as

$${\Lambda }_c\left(t\right)={\int }_0^t{\lambda }_c\left(u\right)\mathrm{d}u.$$

Denote

$$M^c\left(t\right)=\sum _{i=1}^n{M}_i^c\left(t\right).$$

Now, we define the sub-distribution function of $T_1$ corresponding to ${\Delta }_1=1$ and ${ϵ}_1=1$ as

$$S\left(x\right)=\mathbb{P}\left(T_1\le x,{\Delta }_1{ϵ}_1=1\right).$$

Let

$$w\left(t\right)={\int }_0^{\infty }\frac{h_1\left(x\right)}{\mathbb{P}\left(L_1\le x\le C_1\right)}1\mathrm{I}(x>t)\mathrm{d}S\left(x\right),$$

where

$$h_1\left(x\right)=\mathbb{E}\left(\psi \left(\left(T_1,{\Delta }_1\right),\dots ,\left(T_k,{\Delta }_k\right)\right)\mid \left(T_1,{\Delta }_1\right)=\left(x,{\Delta }_1\right)\right).$$

Also, denote

$$\tilde{z}\left(t\right)=\mathbb{P}\left(T_1\ge t\ge L_1\right).$$

Then, an estimate for the survival function of the censoring variable C under left truncation, denoted as ${\widehat{K}}_c(·)$, see [154], can be formulated as follows:

$$\begin{array}{c}\hfill {\widehat{K}}_c\left(\tau \right)=\prod _{t\le \tau }\left(1-\frac{d{N}^c\left(t\right)}{\tilde{Z}\left(t\right)}\right).\end{array}$$

(58)

Similar to the Nelson–Aalen estimator, for instance, see [155], the estimator for the cumulative hazard function of the censoring variable C under left truncation is represented as

$$\begin{array}{c}\hfill {\widehat{\Lambda }}_c\left(\tau \right)={\int }_0^{\tau }\frac{d{N}^c\left(t\right)}{\tilde{Z}\left(t\right)}.\end{array}$$

(59)

In both the definitions presented in (58) and (59), we make the assumption that $\tilde{Z}\left(t\right)$ is non-zero with a probability of one. The interrelation between ${\widehat{K}}_c\left(\tau \right)$ and ${\widehat{\Lambda }}_c\left(\tau \right)$ can be expressed as

$${\widehat{K}}_c\left(\tau \right)=exp\left[-{\widehat{\Lambda }}_c\left(\tau \right)\right].$$

Let $a_K=inf\{t:K\left(t\right)>0\}$ and $b_K=sup\{t:K\left(t\right)<1\}$ denote the left and right endpoints of the support. For LTRC data, as in [156], $F(·)$ is identifiable if $a_G⩽a_W$ and $b_G⩽b_W$. By Corollary 2.2. [156], for $b<b_W$ we readily infer that

$$\begin{array}{c}\hfill \underset{a_W<\tau <b}{sup}|{\widehat{K}}_c\left(\tau \right)-K_c\left(\tau \right)|=O\left(\sqrt{n^{-1}loglogn}\right).\end{array}$$

(60)

From the above, the estimator (57) can be rewritten directly as follows:

$$\begin{array}{c}{\displaystyle {\breve{\stackrel{˘}{r}}}_n^{\left(k\right)*}(\psi ,\mathit{\theta },\mathbf{u},\mathbf{x},h_n)}\hfill \\ = \sum _{(i_1,\dots ,i_k)\in I_n^k}\frac{{\displaystyle {\Delta }_{i_1}\cdots {\Delta }_{i_k}{ϵ}_{i_1}\cdots {ϵ}_{i_k}\psi (Z_{i_1},\dots ,Z_{i_k})}}{{\displaystyle {\widehat{K}}_c\left(Z_{i_1}\right)\cdots {\widehat{K}}_c\left(Z_{i_k}\right)}}{\overline{\omega }}_{n,\mathbf{i}}^{\left(k\right)}(\mathit{\theta },\mathbf{u},\mathbf{x}).\hfill \end{array}$$

(61)

The last estimator is the conditional version of that studied in [153]. Following the same reasoning as that of Corollary 1, one can infer that as $n\to \infty ,$

$$\begin{array}{c}{\displaystyle \underset{\mathit{\theta }\in {\Theta }^m}{sup}\underset{\mathbf{x}\in {\mathcal{H}}^m}{sup}\underset{\mathit{u}\in I_h,\mathbf{x}\in S_c}{sup}\left|{\breve{\stackrel{˘}{r}}}_n^{\left(k\right)*}(\psi ,\mathit{\theta },\mathbf{u},\mathbf{x},h_n)-\mathbb{E}{\breve{\stackrel{˘}{r}}}_n^{\left(k\right)*}(\psi ,\mathit{\theta },\mathbf{u},\mathbf{x},h_n)\right|}\hfill \\ =O_{P_{.|\mathit{S}}}\left(\sqrt{logn/n{h}^{md}\varphi \left(h\right)}+\frac{1}{A_n^{dp}{\varphi }^{1/m}\left(h\right)}\right).\hfill \end{array}$$

7. The Bandwidth Selection Criterion

Various methodologies have been developed to derive asymptotically optimal bandwidth selection rules for nonparametric kernel estimators, particularly for the Nadaraya–Watson regression estimator. Prominent works in this field include contributions from [157,158,159]. The appropriate choice of this parameter is crucial, whether in the standard finite-dimensional case or within the infinite-dimensional framework, to ensure optimal practical performance. However, to the best of our knowledge, investigations addressing the selection of such a general functional conditional U-statistic are currently lacking. Nevertheless, an extension of the leave-one-out cross-validation procedure enables the definition of, for any fixed $\mathbf{j}=(j_1,\dots ,j_m)\in I_n^m$,

$$\begin{array}{c}\hfill {\displaystyle {\widehat{r}}_{n,\mathbf{j}}^{\left(m\right)}(\mathbf{x},\mathit{\theta },\mathbf{u};h_n)=\sum _{\mathbf{i}\in I_n^m\left(\mathbf{j}\right)}\phi ({\mathbf{Y}}_{{\mathit{s}}_{i_1},A_n},\dots ,{\mathbf{Y}}_{{\mathit{s}}_{i_m,A_n}}){\widehat{\mathbb{W}}}_{n,\mathbf{i}}^{\left(m\right)}(\mathbf{x},\mathit{\theta },\mathbf{u};h_n),}\end{array}$$

(62)

where

$$\begin{array}{c}\hfill {\widehat{\mathbb{W}}}_{n,\mathbf{i}}^{\left(m\right)}(\mathbf{x},\mathit{\theta },\mathbf{u};h_n)=\frac{{\displaystyle \prod _{j=1}^m\left\{\prod _{\ell =1}^d{K}_1\left(\frac{u_{j,\ell }-\frac{{\mathbf{s}}_{i_{j,\ell }}}{A_n}}{h_n}\right)K_2\left(\frac{d_{{\theta }_j}(x_j,{\mathbf{X}}_{{\mathit{s}}_{i_j},A_n})}{h_n}\right)\right\}}}{{\displaystyle \sum _{\mathbf{i}\in I_n^m}\prod _{j=1}^m\left\{\prod _{\ell =1}^d{K}_1\left(\frac{u_{j,\ell }-\frac{{\mathbf{s}}_{i_{j,\ell }}}{A_n}}{h_n}\right)K_2\left(\frac{d_{{\theta }_j}(x_j,{\mathbf{X}}_{{\mathit{s}}_{i_j},A_n})}{h_n}\right)\right\}}},\end{array}$$

and

$$I_n^m\left(\mathbf{j}\right):=\left\{\mathbf{i}\in I_n^{m}and\mathbf{i}\ne \mathbf{j}\right\}=I_n^m\setminus \left\{\mathbf{j}\right\}.$$

The Equation (62) serves as the leave-out-$\left({\mathbf{X}}_{\mathbf{j}},{\mathbf{Y}}_{\mathbf{j}}\right)$ estimator for functional regression and can also be regarded as a predictor for $\phi ({\mathbf{Y}}_{{\mathit{s}}_{j_1},A_n},\dots ,{\mathbf{Y}}_{{\mathit{s}}_{j_m,A_n}}):=\phi \left({\mathbf{Y}}_{\mathbf{j}}\right)$. To minimize the quadratic loss function, we introduce the following criterion, where $\mathcal{W}(·)$ is a non-negative weight function known in advance:

$$CV\left(\phi ,h_n\right):=\frac{(n-m)!}{n!}\sum _{\mathbf{j}\in I_n^m}{\left(\phi \left({\mathbf{Y}}_{{\mathbf{s}}_{\mathbf{j}},A_n}\right)-{\widehat{r}}_{n,\mathbf{j}}^{\left(m\right)}({\mathbf{X}}_{{\mathbf{s}}_{\mathbf{j}},A_n},\mathit{\theta },\mathbf{u};h_n)\right)}^2\tilde{\mathcal{W}}\left({\mathbf{X}}_{{\mathbf{s}}_{\mathbf{j}},A_n}\right),$$

(63)

where ${\mathbf{X}}_{\mathbf{j}}:={\mathbf{X}}_{{\mathbf{s}}_{\mathbf{j}},A_n}=({\mathbf{X}}_{{\mathit{s}}_{j_1},A_n},\dots ,{\mathbf{X}}_{{\mathit{s}}_{j_m,A_n}})$. Following the ideas developed by [159], a natural way for choosing the bandwidth is to minimize the precedent criterion, so let us choose ${\widehat{h}}_n\in [a_n,b_n]$ minimizing among $h\in [a_n,b_n]$

$$CV\left(\phi ,h_n\right).$$

Consistent with [160], where the bandwidths are locally determined using a data-driven approach involving the minimization of a functional version of a cross-validated criterion, we can substitute (63) with

$$CV\left(\phi ,h_n\right):=\frac{(n-m)!}{n!}\sum _{\mathbf{j}\in I_n^m}{\left(\phi \left({\mathbf{Y}}_{\mathbf{j}}\right)-{\widehat{r}}_{n,\mathbf{j}}^{\left(m\right)}({\mathbf{X}}_{\mathbf{j}},\mathit{\theta },\mathbf{u};h_n)\right)}^2\widehat{\mathcal{W}}\left({\mathbf{X}}_{\mathbf{j}},\mathbf{x},\mathit{\theta }\right),$$

(64)

where

$$\widehat{\mathcal{W}}\left(\mathbf{s},\mathbf{t},\mathit{\theta },\right):=\prod _{i=1}^m\widehat{W}({\mathbf{s}}_i,{\mathbf{t}}_i,{\theta }_i).$$

In practice, one takes for $\mathbf{i}\in I_n^m$, the uniform global weights $\tilde{\mathcal{W}}\left({\mathbf{X}}_{\mathbf{i}}\right)=1$, and the local weights

$$\widehat{W}({\mathbf{X}}_{\mathbf{i}},\mathbf{t},\mathit{\theta })=\left\{\begin{array}{ccc}1& \mathrm{if}& d_{{\mathcal{H}}^m,\mathit{\theta }}({\mathbf{X}}_{\mathbf{i}},\mathbf{t})\le h,\hfill \\ 0& & \mathrm{otherwise}.\hfill \end{array}\right.$$

To be concise, we have focused on the widely adopted approach, namely, the cross-validated selected bandwidth. However, this can be extended to any alternative bandwidth selector, such as the one rooted in Bayesian principles [161].

Remark 10.

It is worth noting that an alternative bandwidth criterion, proposed by [1], is the rule of thumb. Strictly speaking, since the cross-validated bandwidth is stochastic, justifying its asymptotic theory requires a specific stochastic equicontinuity argument. Cross-validation has been applied by [162] to assess the equality of unconditional and conditional functions in scenarios involving mixed categorical and continuous data. However, while optimal for estimation, this approach loses optimality when extended to nonparametric kernel testing. For testing a parametric model for the conditional mean function against a nonparametric alternative, Horowitz and Spokoiny [163] introduced an adaptive-rate-optimal rule. Another bandwidth selection method is presented by [164]. They propose, based on the Edgeworth expansion of the asymptotic distribution of the test, selecting the bandwidth to maximize the power function of the test while controlling the size function. Future investigations will delve into these three approaches.

8. Concluding Remarks

In this paper, we explored kernel-type estimation for single-index conditional U-statistics, encompassing the single-index Nadaraya–Watson estimator, within a functional framework involving random fields. Our results hinge on assumptions necessitating some regularity in conditional U-statistics and conditional moments, specific decay rates for the probability of variables within shrinking open balls, and appropriate decreasing rates for mixing coefficients. Notably, the conditional moment assumption facilitates the consideration of unbounded function classes. The proof of weak convergence adheres to a typical approach: finite-dimensional convergence and equicontinuity of conditional U-processes. We underscore the significance of adopting absolutely regular conditions or $\beta$-mixing conditions, which are independent of the entropy dimension of the class, a departure from other mixing conditions. In contrast to strong mixing, $\beta$-mixing offers greater flexibility, allowing decoupling and accommodation of diverse examples. Ref. [94] provided a comprehensive characterization of stationary Gaussian processes satisfying the $\beta$-mixing condition. Additionally, $\beta$-mixing aligns with the $L_2\left(\mathbb{P}\right)$-norm, playing a crucial role. Unlike strong mixing, which demands a polynomial rate of decay for strong mixing coefficients contingent on the entropy dimension of the function class, $\beta$-mixing involves the $L_1$-norm and the metric entropy function.

Both the uniform rate of convergence and weak convergence results are underpinned by a general Blocking technique tailored for irregularly spaced sampling sites, where careful attention is paid to the impact of non-equidistant sampling sites. We intricately streamline the process to the independent setting to address this concern. In particular, for spatial points where introducing order is not as straightforward as in time series (Corollary 2.7, [165]), (Lemma 5) constructed exactly independent blocks of observations, allowing results from independent data to be directly applied to these blocks. Yu [165] noted that the uniform convergence result requires the $\beta$-mixing condition to link the original sequence with the sequence of independent blocks. This connection persists under $\varphi$-mixing conditions but is not necessary under $\alpha$-mixing conditions. Thus, we opt for the $\beta$-mixing sequence, aiming to derive weak convergence for processes indexed by classes of functions.

In his work [73], Kurisu presented a potential extension of the sampling region inspired by [89]. This extension can be elucidated as follows. We can generalize the definition of the sample region $R_n$ to include nonstandard forms using the sample region concept from [89]. First, let $R_n$ be the sampling region. Define $R_0^{*}$ as an open connected subset of ${(-2,2]}^d$ containing ${[-1,1]}^d$, and $R_0$ a Borel set such that $R_0^{*}\subset R_0\subset {\overline{R}}_0^{*}$, where $\overline{S}$ signifies the closure of set S. For any set $S\subset {\mathbb{R}}^d$, let ${\left\{A_n\right\}}_{n\ge 1}$ be a sequence of positive numbers with $A_n\to \infty$ as $n\to \infty$, and define $R_n=A_n{R}_0$ as a sampling region. Moreover, for any sequence of positive numbers ${\left\{a_n\right\}}_{n\ge 1}$ with $a_n\to 0$ as $n\to \infty$, let $O\left(a_n^{-d+1}\right)$, as $n\to \infty$, be the number of cubes of the form $a_n\left(\mathbf{i}+{[0,1)}^d\right)$, where $\mathbf{i}\in {\mathbb{Z}}^d$ with their lower left corner $a_n\mathbf{i}$ on the lattice $a_n{\mathbb{Z}}^d$ that intersects both $R_0$ and $R_0^c$ (see Condition B in [89] Chapter 12, Section 12.2). This condition prevents pathological situations and must be assumed on the region $R_n$. It is satisfied by the majority of areas of practical significance and, in the plane ($d=2$), for instance, if the boundary $\partial R_0$ of $R_0$ is defined by a simple rectifiable curve of limited length or when sample sites are defined on the integer grid ${\mathbb{Z}}^d$. Additionally, define $f(·)$ as a continuous, everywhere-present positive probability density function on $R_0$, and let ${\left\{S_{0,i}\right\}}_{i\ge 1}$ be a sequence of i.i.d. random vectors with density $f(·)$. Assume that ${\left\{{\mathit{S}}_{0,i}\right\}}_{i\ge 1}$ and ${\mathbf{X}}_{s,A_n}$ are independent. When replacing our setting in Section 2.4 with this new one, our results remain valid, and it will be possible to demonstrate uniform convergence and weak convergence under the same assumptions and identical proofs. For future investigations, it would be interesting to relax the mixing conditions to the weak dependence (or the ergodicity framework [38,166,167]). This generalization is nontrivial due to the need for maximal moment inequalities in our asymptotic results, which are not available in this setting. Another interesting direction is to consider the incomplete data setting (missing at random, censored in different schemes) for locally spatial–functional data. A natural question is how to adapt our results to wavelet-based estimators [38,166], delta sequence estimators [168], kNN estimators [37], and local linear estimators [169]. The bootstrap method is frequently employed in statistical problems to evaluate the limiting distribution. It has proven to be a valuable tool in addressing common practical issues, including the determination of confidence intervals and critical values for composite hypotheses, as mentioned in [170,171]. Investigating the bootstrap in the context of the present paper would be of interest. In forthcoming investigations, we believe there is a necessity to focus on constructing locally stationary functional random fields. A significant endeavor should be dedicated to generating tangible examples, as in Section 5.5, that can be demonstrated using finite samples derived from either simulated or real data. Infinite-degree U-statistics (or infinite-order U-statistics) are useful tool for constructing simultaneous prediction intervals that quantify the uncertainty of several methods such as sub-bagging and random forests. In a future investigation, we will consider the uniform limit theory for a U-statistic of increasing degree, also called an infinite-degree U-statistic based on the locally stationary functional random fields.

9. Mathematical Developments

The proofs for our results are detailed in this section. The following will continue to use the notations introduced earlier. The proof techniques extend those of [13] to the single-index setting. In addition, some intricate steps of [77] are used in our proofs as in [49,78]. To prevent the repetition of the Blocking technique and the associated notation, the subsequent subsection will introduce all the notations required for this decomposition.

A.1.

Preliminaries

This approach necessitates an expansion of Bernstein’s Blocking techniques to the spatial process, as discussed in [73]. To facilitate our discussion, we will introduce some notations pertinent to this technique. Remember that $\left\{A_{1,n}\right\}$ and $\left\{A_{2.n}\right\}$ denote sequences of positive numbers such that

$$A_{1,n}/A_n+A_{2,n}/A_{1,n}\to 0asn\to \infty .$$

Let

$$A_{3,n}=A_{1,n}+A_{2,n}.$$

We consider a partition of ${\mathbb{R}}^d$ by hypercubes of the form ${\Gamma }_n(\ell ;\mathbf{0})=\left(\ell +{(0,1]}^d\right)A_{3,n},\ell ={\left({\ell }_1,\dots ,{\ell }_d\right)}^{\prime }\in {\mathbb{Z}}^d$ and divide ${\Gamma }_n(\ell ;0)$ into $2^d$ hypercubes as follows:

$${\Gamma }_n(\ell ;\mathit{ϵ})=\prod _{j=1}^d{I}_j\left({ϵ}_j\right),\mathit{ϵ}={\left({ϵ}_1,\dots ,{ϵ}_d\right)}^{\prime }\in {\{1,2\}}^d,$$

(65)

where for $j=1,\dots ,d$,

$$I_j\left({ϵ}_j\right)=\left\{\begin{array}{cc}\left({\ell }_j{A}_{3,n},{\ell }_j{A}_{3,n}+A_{1,n}\right]\hfill & \mathrm{if}{ϵ}_j=1,\\ \left({\ell }_j{A}_{3,n}+A_{1,n},\left({\ell }_j+1\right)A_{3,n}\right]\hfill & \mathrm{if}{ϵ}_j=2.\end{array}\right.$$

(66)

We note that

$$\left|{\Gamma }_n(\ell ;\mathit{ϵ})\right|=A_{1,n}^{q\left(\mathit{ϵ}\right)}{A}_{2,n}^{d-q\left(\mathit{ϵ}\right)}$$

(67)

for any $\ell \in {\mathbb{Z}}^d$ and $ϵ\in {\{1,2\}}^d$, where

$$q\left(\mathit{ϵ}\right)=\left[\left\{1\le j\le d:{ϵ}_j=1\right\}\right].$$

Let ${ϵ}_0={(1,\dots ,1)}^{\prime }$. The partitions ${\Gamma }_n\left(\ell ;{ϵ}_0\right)$ correspond to “large blocks” and the partitions $\Gamma (\ell ;ϵ)$ for $ϵ\ne {ϵ}_0$ correspond to “small blocks”. Let

$$L_{1,n}=\left\{\ell \in {\mathbb{Z}}^d:{\Gamma }_n(\ell ,0)\subset R_n\right\}$$

be the index set of all hypercubes ${\Gamma }_n(\ell ,0)$ that are contained in $R_n$, and let

$$L_{2,n}=\left\{\ell \in {\mathbb{Z}}^d:{\Gamma }_n(\ell ,0)\cap R_n\ne 0,{\Gamma }_n(\ell ,0)\cap R_n^c\ne \varnothing \right\}$$

denote the boundary hypercubes index set. Define $L_n=L_{1,n}\cup L_{2,n}$.

Proof of Proposition 1.

As mentioned earlier, our statistic is a weighted U-statistic that can be expressed as a sum of U-statistics through the Hoeffding decomposition. To achieve the desired result, let us delve into the details of this decomposition in Section 3. In that particular section, it has been demonstrated that

$$\widehat{\psi }(\mathbf{x},\mathit{\theta },\mathbf{u})-\mathbb{E}\left(\widehat{\psi }(\mathbf{x},\mathit{\theta },\mathbf{u})\right)={\widehat{\psi }}_1(\mathbf{x},\mathit{\theta },\mathbf{u})+{\widehat{\psi }}_2(\mathbf{x},\mathit{\theta },\mathbf{u}),$$

where the linear term ${\widehat{\psi }}_1(\mathbf{x},\mathit{\theta },\mathbf{u})$ and the remainder term ${\widehat{\psi }}_2(\mathbf{x},\mathit{\theta },\mathbf{u})$ are well defined in (27) and (29), respectively. Our goal is to prove that the linear term leads to the rate of convergence of this statistic, while the remaining term converges to zero almost surely as $n\to \infty$. Let us focus on dealing with the first term in the decomposition. Consider $B=[0,1]$, ${\alpha }_n=\sqrt{logn/n{h}^{md}\varphi \left(h\right)}$, and ${\tau }_n={\rho }_n{n}^{1/\zeta }$, where $\zeta$ is a positive constant given in Assumption 6 part i), with ${\rho }_n={(logn)}^{{\zeta }_0}$ for some ${\zeta }_0>0$. Define

$$\begin{array}{cc}\hfill {\tilde{H}}_1^{(\ell )}\left(z\right)& :={\tilde{H}}^{(\ell )}\left(z\right){1\mathrm{I}}_{\left\{\left|W_{{\mathit{s}}_i,A_n}\right|\le {\tau }_n\right\}},\hfill \end{array}$$

(68)

$$\begin{array}{cc}\hfill {\tilde{H}}_2\left(z\right)& :={\tilde{H}}^{(\ell )}\left(z\right){1\mathrm{I}}_{\left\{\left|W_{{\mathit{s}}_i,A_n}\right|>{\tau }_n\right\}},\hfill \end{array}$$

(69)

and

$$\begin{array}{ccc}\hfill {\widehat{\psi }}_1^{\left(1\right)}(\mathbf{x},\mathit{\theta },\mathbf{u})-\vartheta \left(\mathbf{i}\right)& =& \frac{1}{n}\sum _{i=1}^n\frac{(n-m)!}{(n-1)!}\sum _{I_{n-1}^{m-1}(-i)}\sum _{\ell =1}^m{\xi }_{i_1}\cdots {\xi }_{i_{\ell -1}}{\xi }_i{\xi }_{i_{\ell }}\cdots {\xi }_{i_{m-1}}{\tilde{H}}_1^{(\ell )}\left(z\right),\hfill \\ \hfill {\widehat{\psi }}_1^{\left(2\right)}(\mathbf{x},\mathit{\theta },\mathbf{u})-\vartheta \left(\mathbf{i}\right)& =& \frac{1}{n}\sum _{i=1}^n\frac{(n-m)!}{(n-1)!}\sum _{I_{n-1}^{m-1}(-i)}\sum _{\ell =1}^m{\xi }_{i_1}\cdots {\xi }_{i_{\ell -1}}{\xi }_i{\xi }_{i_{\ell }}\cdots {\xi }_{i_{m-1}}{\tilde{H}}_2^{(\ell )}\left(z\right).\hfill \end{array}$$

Evidently, we observe that

$$\begin{array}{c}{\displaystyle {\widehat{\psi }}_1(\mathbf{x},\mathit{\theta },\mathbf{u})-\mathbb{E}{\widehat{\psi }}_1(\mathbf{x},\mathit{\theta },\mathbf{u})}\hfill \\ = \left[{\widehat{\psi }}_1^{\left(1\right)}(\mathbf{x},\mathit{\theta },\mathbf{u}))-\mathbb{E}{\widehat{\psi }}_1^{\left(1\right)}(\mathbf{x},\mathit{\theta },\mathbf{u})\right]+\left[{\widehat{\psi }}_1^{\left(2\right)}(\mathbf{x},\mathit{\theta },\mathbf{u})-\mathbb{E}{\widehat{\psi }}_1^{\left(2\right)}(\mathbf{x},\mathit{\theta },\mathbf{u})\right].\hfill \end{array}$$

(70)

To start, it is evident that

$$\begin{array}{c}{\displaystyle P_{·\mid \mathit{S}}\left(\underset{{\mathcal{F}}_m{\mathfrak{K}}_{\Theta }^m}{sup}\underset{\mathit{\theta }\in {\Theta }^m}{sup}\underset{\mathbf{x}\in {\mathcal{H}}^m}{sup}\underset{\mathbf{u}\in B^m}{sup}\left|{\widehat{\psi }}_1^{\left(2\right)}(\mathbf{x},\mathit{\theta },\mathbf{u})-\vartheta \left(\mathbf{i}\right)\right|>{\alpha }_n\right)}\hfill \\ = P_{·\mid \mathit{S}}\left(\underset{{\mathcal{F}}_m{\mathfrak{K}}_{\Theta }^m}{sup}\underset{\mathit{\theta }\in {\Theta }^m}{sup}\underset{\mathbf{x}\in {\mathcal{H}}^m}{sup}\underset{\mathbf{u}\in B^m}{sup}\left|{\widehat{\psi }}_1^{\left(2\right)}(\mathbf{x},\mathit{\theta },\mathbf{u})-\vartheta \left(\mathbf{i}\right)\right|>{\alpha }_n\right)\hfill \\ \bigcap \left\{\underset{{\mathcal{F}}_m{\mathfrak{K}}_{\Theta }^m}{sup}\underset{\mathit{\theta }\in {\Theta }^m}{sup}\underset{\mathbf{x}\in {\mathcal{H}}^m}{sup}\bigcup _{i=1}^n\left|W_{{\mathit{s}}_i,A_n}\right|>{\tau }_n\right\}\bigcup \left\{\underset{{\mathcal{F}}_m{\mathfrak{K}}_{\Theta }^m}{sup}\underset{\mathit{\theta }\in {\Theta }^m}{sup}\underset{\mathbf{x}\in {\mathcal{H}}^m}{sup}{\left\{\bigcup _{i=1}^n\left|W_{{\mathit{s}}_i,A_n}\right|>{\tau }_n\right\}}^{\mathbf{c}}\right\}\hfill \\ \le P_{·\mid \mathit{S}}\left\{\underset{{\mathcal{F}}_m{\mathfrak{K}}_{\Theta }^m}{sup}\underset{\mathit{\theta }\in {\Theta }^m}{sup}\underset{\mathbf{x}\in {\mathcal{H}}^m}{sup}\underset{\mathbf{u}\in B^m}{sup}\left|{\widehat{\psi }}_1^{\left(2\right)}(\mathbf{x},\mathit{\theta },\mathbf{u})-\vartheta \left(\mathbf{i}\right)\right|>{\alpha }_n\right.\hfill \\ \left.\bigcap \left\{\underset{{\mathcal{F}}_m{\mathfrak{K}}_{\Theta }^m}{sup}\underset{\mathit{\theta }\in {\Theta }^m}{sup}\underset{\mathbf{x}\in {\mathcal{H}}^m}{sup}\underset{\mathbf{u}\in B^m}{sup}\bigcup _{i=1}^n\left|W_{{\mathit{s}}_i,A_n}\right|>{\tau }_n\right\}\right\}\hfill \\ +P_{·\mid \mathit{S}}\left\{\underset{{\mathcal{F}}_m{\mathfrak{K}}_{\Theta }^m}{sup}\underset{\mathit{\theta }\in {\Theta }^m}{sup}\underset{\mathbf{x}\in {\mathcal{H}}^m}{sup}\underset{\mathbf{u}\in B^m}{sup}\left|{\widehat{\psi }}_2^{\left(2\right)}(\mathbf{x},\mathit{\theta },\mathbf{u})-\vartheta \left(\mathbf{i}\right)\right|>{\alpha }_n\right.\hfill \\ \left.\bigcap \left\{{\left\{\underset{{\mathcal{F}}_m{\mathfrak{K}}_{\Theta }^m}{sup}\underset{\mathit{\theta }\in {\Theta }^m}{sup}\underset{\mathbf{x}\in {\mathcal{H}}^m}{sup}\underset{\mathbf{u}\in B^m}{sup}\bigcup _{i=1}^n\left|W_{{\mathit{s}}_i,A_n}\right|>{\tau }_n\right\}}^{\mathbf{c}}\right\}\right\}\hfill \\ \le P_{·\mid \mathit{S}}\left(\underset{{\mathcal{F}}_m{\mathfrak{K}}_{\Theta }^m}{sup}\underset{\mathit{\theta }\in {\Theta }^m}{sup}\underset{\mathbf{x}\in {\mathcal{H}}^m}{sup}\underset{\mathbf{u}\in B^m}{sup}\left|W_{{\mathit{s}}_i,A_n}\right|>{\tau }_n\mathrm{for}\mathrm{some}i=1,\dots ,n\right)+P_{·\mid \mathit{S}}(⌀)\hfill \\ \le {\tau }_n^{-\zeta }\sum _{i=1}^n{E}_{·\mid \mathit{S}}\left[\underset{{\mathcal{F}}_m{\mathcal{K}}^m}{sup}\underset{\mathit{\theta }\in {\Theta }^m}{sup}\underset{\mathbf{x}\in {\mathcal{H}}^m}{sup}\underset{\mathbf{u}\in B^m}{sup}{\left|W_{{\mathit{s}}_i,A_n}\right|}^{\zeta }\right]\le n{\tau }_n^{-\zeta }\hfill \\ = {\rho }_n^{-\zeta }\to 0.\hfill \end{array}$$

(71)

We infer that

$$\begin{array}{c}{\displaystyle E_{·\mid \mathit{S}}\left[\left|{\widehat{\psi }}_1^{\left(2\right)}(\mathbf{x},\mathit{\theta },\mathbf{u})\right|\right]}\hfill \\ \le \frac{1}{n}\sum _{i=1}^n\frac{(n-m)!}{(n-1)!}\sum _{I_{n-1}^{m-1}(-i)}\sum _{\ell =1}^m{\xi }_{i_1}\cdots {\xi }_{i_{\ell -1}}{\xi }_i{\xi }_{i_{\ell }}\cdots {\xi }_{i_{m-1}}{E}_{·\mid \mathit{S}}\left(\left|{\tilde{H}}_2^{(\ell )}\left(z\right)\right|\right),\hfill \end{array}$$

where

$$\begin{array}{c}{\displaystyle E_{·\mid \mathit{S}}\left(\left|{\tilde{H}}_2^{(\ell )}\left(z\right)\right|\right)}\hfill \\ = E_{·\mid \mathit{S}}\left[\right|\frac{1}{\varphi \left(h\right)}{K}_2\left(\frac{d_{{\theta }_i}(x_i,X_{{\mathit{s}}_i,A_n})}{h}\right)W_{{\mathit{s}}_i,A_n}\times \int W_{{\mathit{s}}_{(1,\dots ,\ell -1,\ell ,\dots ,m-1)},A_n}\prod _{\underset{j\ne i}{j=1}}^{m-1}\frac{1}{\varphi \left(h\right)}\hfill \\ \left.\left.K_2\left(\frac{d_{{\theta }_j}(x_j,{\nu }_{{\mathit{s}}_j,A_n})}{h}\right)\mathbb{P}(d{\nu }_1,\dots ,d{\nu }_{\ell -1},d{\nu }_{\ell },\dots ,d{\nu }_{m-1})\right|\right]\hfill \\ \le E_{·\mid \mathit{S}}\left[\left|\frac{1}{\varphi \left(h\right)}\left(K_2\left(\frac{d_{{\theta }_i}\left(x_i,X_{{\mathit{s}}_i,A_n}\right)}{h}\right)+K_2\left(\frac{d_{{\theta }_i}\left(x_i,X_{u_i}\left(s_i\right)\right)}{h}\right)\right.\right.\right.\hfill \\ \left.\left.\left.-K_2\left(\frac{d_{{\theta }_i}\left(x_i,X_{u_i}\left(s_i\right)\right)}{h}\right)\right)W_{{\mathit{s}}_i,A_n}{1\mathrm{I}}_{\left\{\left|W_{{\mathit{s}}_i,A_n}\right|>{\tau }_n\right\}}\right|\right]\hfill \\ \le \frac{{\tau }_n^{-(\zeta -1)}}{\varphi \left(h\right)}{E}_{·\mid \mathit{S}}\left[\left|K_2\left(\frac{d_{{\theta }_i}\left(x_i,X_{{\mathit{s}}_i,A_n}\right)}{h}\right)-K_2\left(\frac{d_{{\theta }_i}\left(x_i,X_{u_i}\left(s_i\right)\right)}{h}\right)\right|{\left|W_{{\mathit{s}}_i,A_n}\right|}^{\zeta }\right.\hfill \\ \left.+\left|K_2\left(\frac{d_{{\theta }_i}\left(x_i,X_{u_i}\left(s_i\right)\right)}{h}\right)\right|{\left|W_{{\mathit{s}}_i,A_n}\right|}^{\zeta }\right]\hfill \\ \le \frac{{\tau }_n^{-(\zeta -1)}}{\varphi \left(h\right)}{E}_{·\mid \mathit{S}}\left[h^{-1}\left|d_{{\theta }_i}\left(x_i,X_{{\mathit{s}}_i,A_n}\right)-d_{{\theta }_i}\left(x_i,X_{u_i}\left(s_i\right)\right)\right|{\left|W_{{\mathit{s}}_i,A_n}\right|}^{\zeta }\right]\hfill \\ +E_{·\mid \mathit{S}}\left[\left|K_2\left(\frac{d_{{\theta }_i}\left(x_i,X_{u_i}\left(s_i\right)\right)}{h}\right)\right|{\left|W_{{\mathit{s}}_i,A_n}\right|}^{\zeta }\right]\hfill \\ \lesssim \frac{{\tau }_n^{-(\zeta -1)}}{\varphi \left(h\right)}\times \left[\frac{1}{nh}+\varphi \left(h\right)\right]\hfill \\ \lesssim \frac{{\tau }_n^{-(\zeta -1)}}{nh\varphi \left(h\right)}+{\tau }_n^{-(\zeta -1)}.\hfill \end{array}$$

(72)

Hence, we have

$$\begin{array}{c}{\displaystyle E_{·\mid \mathit{S}}\left[\left|{\widehat{\psi }}_1^{\left(2\right)}(\mathbf{x},\mathit{\theta },\mathbf{u})\right|\right]}\hfill \\ \lesssim \frac{1}{n}\sum _{i=1}^n\frac{(n-m)!}{(n-1)!}\sum _{I_{n-1}^{m-1}(-i)}\sum _{\ell =1}^m{\xi }_{i_1}\cdots {\xi }_{i_{\ell -1}}{\xi }_i{\xi }_{i_{\ell }}\cdots {\xi }_{i_{m-1}}\left[{\tau }_n^{-(\zeta -1)}\right]\hfill \\ \lesssim {\tau }_n^{-(\zeta -1)}\frac{1}{n^m}\sum _{\mathbf{i}\in I_n^m}\prod _{j=1}^m\frac{1}{h^d}\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)\hfill \\ = \frac{C}{{\tau }_n^{(\zeta -1)}}\left(f_{\mathbf{S}}\left(\mathbf{u}\right)+O\left(\sqrt{\frac{logn}{n{h}^{md}}}+h^2\right)\right)\left(\mathrm{Using} \mathrm{Lemma} 2\right)\hfill \\ \le \frac{C}{{\tau }_n^{(\zeta -1)}}=C{\rho }_n^{-(\zeta -1)}{n}^{-(\zeta -1)/\zeta }\le C{\alpha }_n{P}_S-a.s.\hfill \end{array}$$

Consequently, we derive that

$$\underset{{\mathcal{F}}_m{\mathfrak{K}}_{\Theta }^m}{sup}\underset{\mathit{\theta }\in {\Theta }^m}{sup}\underset{\mathbf{x}\in {\mathcal{H}}^m}{sup}\underset{\mathbf{u}\in B^m}{sup}\left|{\widehat{\psi }}_1^{\left(2\right)}(\mathbf{x},\mathit{\theta },\mathbf{u})-E_{·\mid \mathit{S}}{\widehat{\psi }}_1^{\left(2\right)}(\mathbf{x},\mathit{\theta },\mathbf{u})\right|=O_{P_{·\mid \mathit{S}}}\left({\alpha }_n\right).$$

(73)

Next, let us address

$$\underset{{\mathcal{F}}_m{\mathfrak{K}}_{\Theta }^m}{sup}\underset{\mathit{\theta }\in {\Theta }^m}{sup}\underset{\mathbf{x}\in {\mathcal{H}}^m}{sup}\underset{\mathbf{u}\in B^m}{sup}\left|{\widehat{\psi }}_1^{\left(1\right)}(\mathbf{x},\mathit{\theta },\mathbf{u})-\mathbb{E}{\widehat{\psi }}_1^{\left(1\right)}(\mathbf{x},\mathit{\theta },\mathbf{u})\right|.$$

Remember the large blocks and small blocks as well as the notation provided in Section 9, and define

$$\begin{array}{ccc}\hfill S_{\mathit{s},A_n}(\mathbf{x},\mathit{\theta },\mathbf{u})& :=& \frac{(n-m)!}{(n-1)!}\sum _{I_{n-1}^{m-1}(-i)}\sum _{\ell =1}^m{\xi }_{i_1}\cdots {\xi }_{i_{\ell -1}}{\xi }_i{\xi }_{i_{\ell }}\cdots {\xi }_{i_{m-1}}{\tilde{H}}_1^{(\ell )}\left(z\right),\hfill \\ \hfill S_n(\ell ;\mathit{ϵ})& =& \sum _{i:s_i\in {\Gamma }_n(\ell ;ϵ)\cap R_n}{S}_{\mathit{s},A_n}(\mathbf{x},\mathit{\theta },\mathbf{u})={\left(S_n^{\left(1\right)}(\ell ;\mathit{ϵ}),\dots ,S_n^{\left(p\right)}(\ell ;\mathit{ϵ})\right)}^{\prime }.\hfill \end{array}$$

Then we have, for $m=1$,

$$\begin{array}{ccc}\hfill S_n& =& {\left(S_n^{\left(1\right)},\dots ,S_n^{\left(m\right)}\right)}^{\prime }\hfill \\ & =& \sum _{i=1}^n{S}_{\mathit{s},A_n}(\mathbf{x},\mathit{\theta },\mathbf{u})\hfill \\ & =& \sum _{\ell \in L_n}{S}_n\left(\ell ;{\mathit{ϵ}}_0\right)+\sum _{\mathit{ϵ}\ne {ϵ}_0}\underset{=:S_{2,n}\left(\mathit{ϵ}\right)}{\underbrace{{\sum }_{\ell \in L_{1,n}}{S}_n(\ell ;\mathit{ϵ})}}+\sum _{ϵ\ne {ϵ}_0}\underset{=:S_{3,n}\left(\mathit{ϵ}\right)}{\underbrace{{\sum }_{\ell \in L_{2,n}}{S}_n(\ell ;\mathit{ϵ})}}\hfill \\ & =:& S_{1,n}+\sum _{ϵ\ne {ϵ}_0}{S}_{2,n}\left(\mathit{ϵ}\right)+\sum _{ϵ\ne {ϵ}_0}{S}_{3,n}\left(\mathit{ϵ}\right).\hfill \end{array}$$

(74)

Here, we give the notation only for $m=1$, that will be used in treating the linear term in the Hoeffding decomposition. The case of $m=2$ will be stated in the proof of Lemma 1.

• To attain our result, we will proceed through the following two steps.
• Step 1 (Reduction to independence). Recall

  $$S_n(\ell ;\mathit{ϵ})=\sum _{i:s_i\in {\Gamma }_n(\ell ;ϵ)\cap R_n}{S}_{\mathbf{s},A_n}(\mathbf{x},\mathit{\theta },\mathbf{u}).$$

  For each $\mathit{ϵ}\in {\{1,2\}}^d$, let $\left\{{\breve{S}}_n(\ell ;ϵ):\ell \in L_n\right\}$ be a sequence of independent random variables in $\mathbb{R}$ under $P_{·\mid \mathit{S}}$ such that

  $${\breve{S}}_n(\ell ;\mathit{ϵ})\stackrel{d}{=}{S}_n(\ell ;\mathit{ϵ}),\mathrm{under}{P}_{.\mid \mathit{S}},\ell \in L_n.$$

  Define

  $${\breve{S}}_{1,n}=\sum _{\ell \in L_n}{\breve{S}}_n\left(\ell ;{ϵ}_0\right)={\left({\breve{S}}_{1,n}^{\left(1\right)},\dots ,{\breve{S}}_{1,n}^{\left(m\right)}\right)}^{\prime }$$

  and for $ϵ\ne {ϵ}_0$, define

  $${\breve{S}}_{2,n}\left(ϵ\right)=\sum _{\ell \in L_{1,n}}{\breve{S}}_n(\ell ;ϵ)$$

  and

  $${\breve{S}}_{3,n}\left(\mathit{ϵ}\right)=\sum _{\ell \in L_{2,n}}{\breve{S}}_n(\ell ;\mathit{ϵ}).$$

  We begin by verifying the following results:

  $$\begin{array}{c}{\displaystyle \underset{t>0}{sup}\left|P_{·\mid \mathbf{S}}\left(S_{1,n}>t\right)-P_{·\mid \mathbf{S}}\left({\breve{S}}_{1,n}>t\right)\right|}\hfill \end{array}$$

  $$\begin{array}{c} \le C{\left(\frac{A_n}{A_{1,n}}\right)}^d\beta \left(A_{2,n};A_n^d\right),\hfill \\ {\displaystyle \underset{t>0}{sup}\left|P_{·\mid \mathbf{S}}\left({∥S_{2,n}\left(\mathit{ϵ}\right)∥}_{\infty }>t\right)-P_{·\mid \mathbf{S}}\left({∥{\breve{S}}_{2,n}\left(\mathit{ϵ}\right)∥}_{\infty }>t\right)\right|}\hfill \end{array}$$

  (75)

  $$\begin{array}{c} \le C{\left(\frac{A_n}{A_{1,n}}\right)}^d\beta \left(A_{2,n};A_n^d\right),\hfill \\ {\displaystyle \underset{t>0}{sup}\left|P_{·\mid \mathbf{S}}\left({∥S_{3,n}\left(\mathit{ϵ}\right)∥}_{\infty }>t\right)-P_{·\mid \mathbf{S}}\left({∥{\breve{S}}_{3,n}\left(\mathit{ϵ}\right)∥}_{\infty }>t\right)\right|}\hfill \end{array}$$

  (76)

  $$\begin{array}{c} \le C{\left(\frac{A_n}{A_{1,n}}\right)}^d\beta \left(A_{2,n};A_n^d\right).\hfill \end{array}$$

  (77)

  Keep in mind that

  $$〚L_n〛=O\left({\left(A_n/A_{3,n}\right)}^d\right)\lesssim {\left(A_n/A_{1,n}\right)}^d.$$

  For $\mathit{ϵ}\in {\{1,2\}}^d$ and ${\ell }_1,{\ell }_2\in L_n$ with ${\ell }_1\ne {\ell }_2$, let

  $$\begin{array}{ccc}\hfill {\mathcal{J}}_1\left(\mathit{ϵ}\right)& =& \left\{1\le i_1\le n:{\mathit{s}}_{i_1}\in {\Gamma }_n\left({\ell }_1,\mathit{ϵ}\right)\right\},\hfill \\ \hfill {\mathcal{J}}_2\left(\mathit{ϵ}\right)& =& \left\{1\le i_2\le n:{\mathit{s}}_{i_2}\in {\Gamma }_n\left({\ell }_2;\mathit{ϵ}\right)\right\}.\hfill \end{array}$$

  For any ${\mathit{s}}_{i_k}=\left(s_{1,i_k},\dots ,s_{d,i_k}\right),k=1,2$ in such a way that $i_1\in {\mathcal{J}}_1\left(\mathit{ϵ}\right)$ and $i_2\in {\mathcal{J}}_2\left(\mathit{ϵ}\right)$, we obtain ${max}_{1\le u\le d}\mid s_{u,i_1}-$$s_{u,i_2}\mid \ge A_{2,n}$ using the definition of $\Gamma (\ell ;\mathit{ϵ})$. This gives

  $$\left|{\mathit{s}}_{i_1}-{\mathit{s}}_{i_2}\right|\ge A_{2,n}.$$

  For any $\mathit{ϵ}\in {\{1,2\}}^d$, let $S_n\left({\ell }_1;\mathit{ϵ}\right),\dots ,S_n\left({\ell }_{\left[L_n\right]};\mathit{ϵ}\right)$ be an arrangement of $\left\{S_n(\ell ;ϵ):\ell \in L_n\right\}$. Let $P_{.\mid \mathbf{S}}^{\left(a\right)}$ be the marginal distribution of $S_n\left({\ell }_a;\mathit{ϵ}\right)$ and let $P_{·\mid \mathbf{S}}^{(a:b)}$ be the joint distribution of

  $$\{S_n\left({\ell }_k;\mathit{ϵ}\right):a\le k\le b\}.$$

  The $\beta$-mixing property of $\mathit{X}$ gives that for $1\le k\le 〚L_n〛-1$,

  $${∥P_{·\mid \mathbf{S}}-P_{·\mid \mathbf{S}}^{(1:k)}\times P_{·\mid \mathbf{S}}^{\left(k+1:\left[L_n\right]\right)}∥}_{\mathrm{TV}}\lesssim \beta \left(A_{2,n};A_n^d\right),$$

  where ${\parallel ·\parallel }_{\mathrm{TV}}$ is the total variation. The inequality is independent of the arrangement of $\left\{S_n(\ell ;\mathit{ϵ}):\ell \in L_n\right\}$. Thus, the assumption in (125) of Lemma 5 is satisfied for $\left\{S_n(\ell ;\mathit{ϵ}):\ell \in L_n\right\}$ with $\tau \sim \beta \left(A_{2,n};A_n^d\right)$ and $m\lesssim {\left(A_n/A_{1,n}\right)}^d$. By combining the boundary condition on $R_n$ and Lemma 5, we obtain (75)–(77).

Remark 11.

Since

$$〚\left\{\mathit{ϵ}\in {\{1,2\}}^d:\mathit{ϵ}\ne {\mathit{ϵ}}_0\right\}〛=2^d-1,〚L_{1,n}〛\sim {\left(A_n/A_{3,n}\right)}^d\sim {\left(A_n/A_{1,n}\right)}^d,$$

and

$$〚L_{2,n}〛\sim {\left(A_n/A_{3,n}\right)}^{d-1}\sim {\left(A_n/A_{1,n}\right)}^{d-1}\ll 〚L_{1,n}〛,$$

For sufficiently large n, Lemma 6 and Equation (67) imply that the individual terms in the sums of $S_{2,n}$ and $S_{3,n}$ are, at most

$$O\left(A_{1,n}^{d-1}{A}_{2,n}n{A}_n^{-d}{\left(A_n/A_{1,n}\right)}^d\right)=O\left(\frac{A_{2,n}}{A_{1,n}}n\right)$$

and

$$O\left(A_{1,n}^{d-1}{A}_{2,n}n{A}_n^{-d}{\left(A_n/A_{1,n}\right)}^{d-1}\right)=O\left(\frac{A_{2,n}}{A_n}n\right),$$

respectively.

• Step 2: Keep in mind our intention to address

  $$\underset{{\mathcal{F}}_m{\mathfrak{K}}_{\Theta }^m}{sup}\underset{\mathit{\theta }\in {\Theta }^m}{sup}\underset{\mathbf{x}\in {\mathcal{H}}^m}{sup}\underset{\mathbf{u}\in B^m}{sup}\left|{\widehat{\psi }}_1^{\left(1\right)}(\mathbf{x},\mathit{\theta },\mathbf{u})-E_{·\mid \mathbf{S}}{\widehat{\psi }}_1^{\left(1\right)}(\mathbf{x},\mathit{\theta },\mathbf{u})\right|.$$

  To achieve the aimed result, we will cover the region $B^m={[0,1]}^{dm}$ by

  $$\bigcup _{k_1,\dots ,k_m=1}^{N_{\left(\mathbf{u}\right)}}\prod _{j=1}^m\mathbf{B}({\mathbf{u}}_{k_j},r),$$

  for some radius r. Hence, for each $\mathbf{u}=({\mathbf{u}}_1,\dots ,{\mathbf{u}}_m)\in {[0,1]}^{dm}$, there exists $\mathbf{l}\left(\mathbf{u}\right)=(l\left({\mathbf{u}}_1\right),\dots ,l\left({\mathbf{u}}_m\right))$, where $\forall 1\le i\le m,1\le l\left({\mathbf{u}}_i\right)\le N_{\left(\mathbf{u}\right)}$ in such a way that

  $$\mathbf{u}\in \prod _{i=1}^m\mathbf{B}({\mathbf{u}}_{l\left({\mathbf{u}}_i\right)},r)and|{\mathbf{u}}_i-{\mathbf{u}}_{l\left({\mathbf{u}}_i\right)}|\le r,for1\le i\le m,$$

  then for each $\mathit{u}\in {[0,1]}^{dm}$, the closest center will be ${\mathbf{u}}_{\mathbf{l}}\left(\mathit{u}\right)$, and the ball with the closest center will be defined by

  $$\mathcal{B}(\mathbf{u},\mathbf{l}\left(\mathbf{u}\right),r):=\prod _{j=1}^m\mathbf{B}({\mathbf{u}}_{k_j},r).$$

  In the same way, ${\Theta }^m\times {\mathcal{H}}^m$ should be covered by

  $$\bigcup _{{\tilde{k}}_1,\dots ,{\tilde{k}}_m=1}^{N_{\left(\theta \right)}}\bigcup _{k_1,\dots ,k_m=1}^{N_{\left(x\right)}}\prod _{j=1}^m{\mathbf{B}}_{{\theta }_{{\tilde{k}}_j}}(x_{k_j},r),$$

  for some radius r. Hence, for each $\mathbf{x}=(x_1,\dots ,x_m)\in {\mathcal{H}}^m$, there exists $\mathbf{l}\left(\mathbf{x}\right)=(l\left(x_1\right),\dots ,l\left(x_m\right))$, where $\forall 1\le i\le m,1\le l\left(x_i\right)\le N_{\left(x\right)}$ in such a way that

  $$\mathbf{x}\in \prod _{i=1}^m{B}_{{\theta }_i}(u_{l\left(x_i\right)},r)\mathrm{and}{d}_{{\theta }_i}(x_i,x_{l\left(u_i\right)})\le r,for1\le i\le m,$$

  then for each $\mathbf{x}\in {\mathcal{H}}^m$, the closest center will be ${\mathbf{x}}_{\mathbf{l}}\left(\mathbf{x}\right)$, and the ball with the closest centre will be defined by

  $${\mathbf{B}}_{\theta }(\mathbf{x},\mathbf{l}\left(\mathbf{x}\right),r):=\prod _{i=1}^m{B}_{{\theta }_i}(x_{l\left(x_i\right)},r).$$

  We define

  $$K^{*}(\mathit{\omega },\mathit{v})\ge C_0\prod _{j=1}^m\prod _{\ell =1}^d{1\mathrm{I}}_{(\left|{\omega }_{j,\ell }\right|\le 2C_1)}\prod _{j=1}^m{K}_2\left(v_k\right)for(\omega ,v)\in {\mathbb{R}}^2.$$

  We can show that, for $(u,x)\in B_{j,n}$ and large-enough n,

  $$\begin{array}{cc}\hfill & \left|\overline{K}\left(\frac{\mathbf{u}-\mathbf{s}/A_n}{h_n}\right)K_2\left(\frac{d_{{\theta }_i}(x_i,X_{{\mathit{s}}_i,A_n})}{h}\right)-\overline{K}\left(\frac{{\mathbf{u}}_n-\mathbf{s}/A_n}{h_n}\right)K_2\left(\frac{d_{{\theta }_i}(x_{n,i},X_{{\mathit{s}}_i,A_n})}{h}\right)\right|\hfill \\ \hfill & \le {\alpha }_n{K}^{*}\left(\frac{{\mathbf{u}}_n-\mathbf{s}/A_n,d_{{\theta }_i}(x_{n,i},X_{{\mathit{s}}_i,A_n})}{h_n}\right).\hfill \end{array}$$

  Then, we have

  $$\begin{array}{c}{\displaystyle {\widehat{\psi }}_1^{\left(1\right)}(\mathbf{x},\mathit{\theta },\mathbf{u})=\frac{1}{n}\sum _{i=1}^n{\xi }_i\frac{1}{{\varphi }^{1/m}\left(h\right)}{K}_2\left(\frac{d_{{\theta }_i}(x_i,X_{{\mathit{s}}_i,A_n})}{h}\right)W_{{\mathit{s}}_i,A_n}{1\mathrm{I}}_{\left\{\left|W_{{\mathit{s}}_i,A_n}\right|\le {\tau }_n\right\}}}\hfill \\ \times \frac{(n-m)!}{(n-1)!}\sum _{I_{n-1}^{m-1}(-i)}\sum _{\ell =1}^m{\xi }_{i_1}\cdots {\xi }_{i_{\ell -1}}{\xi }_{i_{\ell }}\cdots {\xi }_{i_{m-1}}\int W_{{\mathit{s}}_{(1,\dots ,\ell -1,\ell ,\dots ,m-1)},A_n}\hfill \\ \times \prod _{\underset{j\ne i}{j=1}}^{m-1}\frac{1}{{\varphi }^{1/m}\left(h\right)}{K}_2\left(\frac{d_{{\theta }_j}(x_j,{\nu }_{{\mathit{s}}_j,A_n})}{h}\right)\mathbb{P}(d{\nu }_1,\dots ,d{\nu }_{\ell -1},d{\nu }_{\ell },\dots ,d{\nu }_{m-1}).\hfill \end{array}$$

  Let us define

  $$\begin{array}{c}{\displaystyle {\overline{\psi }}_1^{\left(1\right)}(\mathbf{x},\mathit{\theta },\mathbf{u})=\frac{1}{n{h}^d{\varphi }^{1/m}\left(h\right)}\sum _{i=1}^n{K}^{*}\left(\frac{{\mathbf{u}}_n-{\mathbf{s}}_{\mathbf{i}}/A_n,d_{{\theta }_i}(x_{n,i},X_{{\mathit{s}}_i,A_n})}{h_n}\right)W_{{\mathit{s}}_i,A_n}{1\mathrm{I}}_{\left\{\left|W_{{\mathit{s}}_i,A_n}\right|\le {\tau }_n\right\}}}\hfill \\ \times \frac{(n-m)!}{(n-1)!}\sum _{I_{n-1}^{m-1}(-i)}\sum _{\ell =1}^m{\xi }_{i_1}\cdots {\xi }_{i_{\ell -1}}{\xi }_{i_{\ell }}\cdots {\xi }_{i_{m-1}}\int W_{{\mathit{s}}_{(1,\dots ,\ell -1,\ell ,\dots ,m-1)},A_n}\hfill \\ \times \prod _{\underset{j\ne i}{j=1}}^{m-1}\frac{1}{{\varphi }^{1/m}\left(h\right)}{K}_2\left(\frac{d_{{\theta }_j}(x_j,{\nu }_{{\mathit{s}}_j,A_n})}{h}\right)\mathbb{P}(d{\nu }_1,\dots ,d{\nu }_{\ell -1},d{\nu }_{\ell },\dots ,d{\nu }_{m-1})\hfill \\ := \frac{1}{n{h}^d{\varphi }^{1/m}\left(h\right)}\sum _{i=1}^n{S}_{\mathit{s},A_n}^{\prime }(\mathbf{x},\mathit{\theta },\mathbf{u}).\hfill \end{array}$$

  (78)

  We note that

  $$E_{·\mid \mathbf{S}}\left[\left|{\overline{\psi }}_1^{\left(1\right)}(\mathbf{x},\mathit{\theta },\mathbf{u})\right|\right]\le M<\infty ,$$

  for some sufficiently large M. Let $\mathfrak{N}=N_{{\mathcal{F}}_m{\mathfrak{K}}_{\Theta }^m}{N}_{\left(\theta \right)}^m{N}_{\left(x\right)}^m{N}_{\left(u\right)}$ denote the covering number associated with the classes of functions ${\mathcal{F}}_m{\mathcal{K}}^m$, the balls covering ${[0,1]}^m$, the balls covering ${\mathcal{H}}^m$, and the balls covering ${\Theta }^m$. Consequently, we obtain

  $$\begin{array}{c}{\displaystyle \underset{{\mathcal{F}}_m{\mathfrak{K}}_{\Theta }^m}{sup}\underset{\mathit{\theta }\in {\Theta }^m}{sup}\underset{\mathbf{x}\in {\mathcal{H}}^m}{sup}\underset{\mathbf{u}\in \mathcal{B}}{sup}\left|{\widehat{\psi }}_1^{\left(1\right)}(\mathbf{x},\mathit{\theta },\mathbf{u})-E_{·\mid \mathbf{S}}\left[{\widehat{\psi }}_1^{\left(1\right)}(\mathbf{x},\mathit{\theta },\mathbf{u})\right]\right|}\hfill \\ \le \mathfrak{N}\underset{1\le i_1<\cdots <i_m\le m}{max}\underset{B_{{\theta }_{\mathbf{i}\left(x\right)}}(x_{\mathbf{i}\left(x\right)},r)}{sup}\underset{1\le i_1<\cdots <i_m\le m}{max}\underset{B({\mathbf{u}}_{\mathbf{i}\left(\mathbf{u}\right)},r)}{sup}\left|{\widehat{\psi }}_1^{\left(1\right)}(\mathbf{u},\mathbf{x},\phi )-E_{·\mid \mathbf{S}}\left[{\widehat{\psi }}_1^{\left(1\right)}(\mathbf{u},\mathbf{x},\phi )\right]\right|\hfill \\ \le \mathfrak{N}\underset{1\le i_1<\cdots <i_m\le m}{max}\underset{B_{{\theta }_{\mathbf{i}\left(x\right)}}(x_{\mathbf{i}\left(x\right)},r)}{sup}\underset{1\le i_1<\cdots <i_m\le m}{max}\underset{B({\mathbf{u}}_{\mathbf{i}\left(\mathbf{u}\right)},r)}{sup}\left|{\widehat{\psi }}_1^{\left(1\right)}(\mathbf{x},\mathit{\theta },{\mathbf{u}}_n)-E_{·\mid \mathbf{S}}\left[{\widehat{\psi }}_1^{\left(1\right)}(\mathbf{x},\mathit{\theta },{\mathbf{u}}_n)\right]\right|\hfill \\ +\mathfrak{N}\underset{1\le i_1<\cdots <i_m\le m}{max}\underset{B_{{\theta }_{\mathbf{i}\left(x\right)}}(x_{\mathbf{i}\left(x\right)},r)}{sup}\underset{1\le i_1<\cdots <i_m\le m}{max}\underset{B({\mathbf{u}}_{\mathbf{i}\left(\mathbf{u}\right)},r)}{sup}{\alpha }_n\left(\left|{\overline{\psi }}_1^{\left(1\right)}(\mathbf{x},\mathit{\theta },{\mathbf{u}}_n)\right|+E_{·\mid \mathbf{S}}\left[\left|{\overline{\psi }}_1^{\left(1\right)}(\mathbf{x},\mathit{\theta },{\mathbf{u}}_n)\right|\right]\right)\hfill \\ \le \mathfrak{N}\underset{1\le i_1<\cdots <i_m\le m}{max}\underset{B_{{\theta }_{\mathbf{i}\left(x\right)}}(x_{\mathbf{i}\left(x\right)},r)}{sup}\underset{1\le i_1<\cdots <i_m\le m}{max}\underset{B({\mathbf{u}}_{\mathbf{i}\left(\mathbf{u}\right)},r)}{sup}\left|{\widehat{\psi }}_1^{\left(1\right)}(\mathbf{x},\mathit{\theta },{\mathbf{u}}_n)-E_{·\mid \mathbf{S}}\left[{\widehat{\psi }}_1^{\left(1\right)}(\mathbf{x},\mathit{\theta },{\mathbf{u}}_n)\right]\right|\hfill \\ +\mathfrak{N}\underset{1\le i_1<\cdots <i_m\le m}{max}\underset{B_{{\theta }_{\mathbf{i}\left(x\right)}}(x_{\mathbf{i}\left(x\right)},r)}{sup}\underset{1\le i_1<\cdots <i_m\le m}{max}\underset{B({\mathbf{u}}_{\mathbf{i}\left(\mathbf{u}\right)},r)}{sup}\left|{\overline{\psi }}_1^{\left(1\right)}(\mathbf{x},\mathit{\theta },{\mathbf{u}}_n)-E_{·\mid \mathbf{S}}\left[{\overline{\psi }}_1^{\left(1\right)}(\mathbf{x},\mathit{\theta },{\mathbf{u}}_n)\right]\right|\hfill \\ +2MF\left(\mathbf{y}\right){\alpha }_n\hfill \\ \le \mathfrak{N}\underset{1\le i_1<\cdots <i_m\le m}{max}\underset{B_{{\theta }_{\mathbf{i}\left(x\right)}}(x_{\mathbf{i}\left(x\right)},r)}{sup}\underset{1\le i_1<\cdots <i_m\le m}{max}\underset{B({\mathbf{u}}_{\mathbf{i}\left(\mathbf{u}\right)},r)}{sup}\left|\sum _{\ell \in L_n}{S}_n\left(\ell ;{\mathit{ϵ}}_0\right)\right|\hfill \\ +\mathfrak{N}\underset{1\le i_1<\cdots <i_m\le m}{max}\underset{B_{{\theta }_{\mathbf{i}\left(x\right)}}(x_{\mathbf{i}\left(x\right)},r)}{sup}\underset{1\le i_1<\cdots <i_m\le m}{max}\underset{B({\mathbf{u}}_{\mathbf{i}\left(\mathbf{u}\right)},r)}{sup}\left|\sum _{ϵ\ne {\mathit{ϵ}}_0}\sum _{\ell \in L_{1,n}}{S}_n(\ell ;ϵ)\right|\hfill \\ +\mathfrak{N}\underset{1\le i_1<\cdots <i_m\le m}{max}\underset{B_{{\theta }_{\mathbf{i}\left(x\right)}}(x_{\mathbf{i}\left(x\right)},r)}{sup}\underset{1\le i_1<\cdots <i_m\le m}{max}\underset{B({\mathbf{u}}_{\mathbf{i}\left(\mathbf{u}\right)},r)}{sup}\left|\sum _{ϵ\ne {ϵ}_0}\sum _{\ell \in L_{2,n}}{S}_n(\ell ;\mathit{ϵ})\right|\hfill \\ +\mathfrak{N}\underset{1\le i_1<\cdots <i_m\le m}{max}\underset{B_{{\theta }_{\mathbf{i}\left(x\right)}}(x_{\mathbf{i}\left(x\right)},r)}{sup}\underset{1\le i_1<\cdots <i_m\le m}{max}\underset{B({\mathbf{u}}_{\mathbf{i}\left(\mathbf{u}\right)},r)}{sup}\left|\sum _{\ell \in L_n}{S}_n^{\prime }\left(\ell ;{\mathit{ϵ}}_0\right)\right|\hfill \\ +\mathfrak{N}\underset{1\le i_1<\cdots <i_m\le m}{max}\underset{B_{{\theta }_{\mathbf{i}\left(x\right)}}(x_{\mathbf{i}\left(x\right)},r)}{sup}\underset{1\le i_1<\cdots <i_m\le m}{max}\underset{B({\mathbf{u}}_{\mathbf{i}\left(\mathbf{u}\right)},r)}{sup}\left|\sum _{ϵ\ne {ϵ}_0}\sum _{\ell \in L_{1,n}}{S}_n^{\prime }(\ell ;\mathit{ϵ})\right|\hfill \\ +\mathfrak{N}\underset{1\le i_1<\cdots <i_m\le m}{max}\underset{B_{{\theta }_{\mathbf{i}\left(x\right)}}(x_{\mathbf{i}\left(x\right)},r)}{sup}\underset{1\le i_1<\cdots <i_m\le m}{max}\underset{B({\mathbf{u}}_{\mathbf{i}\left(\mathbf{u}\right)},r)}{sup}\left|\sum _{ϵ\ne {ϵ}_0}\sum _{\ell \in L_{2,n}}{S}_n^{\prime }(\ell ;\mathit{ϵ})\right|+2MF\left(\mathbf{y}\right){\alpha }_n.\hfill \end{array}$$

  (79)
• Furthermore, for each $\mathit{ϵ}\in {\{1,2\}}^d$, let $\left\{{\breve{S}}_n^{\prime }(\ell ;\mathit{ϵ}):\ell \in L_n\right\}$ represent a sequence of independent random vectors in ${\mathbb{R}}^m$ under $P_{·\mid \mathbf{S}}$, such that

  $${\breve{S}}_n^{\prime }(\ell ;\mathit{ϵ})\stackrel{d}{=}{S}_n^{\prime }(\ell ;\mathit{ϵ}),\mathrm{under}{P}_{.\mid \mathbf{S}},\ell \in L_n.$$

  Show that

  $$\begin{array}{c}{\displaystyle P_{·\mid \mathbf{S}}\left(\underset{1\le i_1<\cdots <i_m\le m}{max}\underset{B_{{\theta }_{\mathbf{i}\left(x\right)}}(x_{\mathbf{i}\left(x\right)},r)}{sup}\right.}\hfill \\ \left.\underset{1\le i_1<\cdots <i_m\le m}{max}\underset{B({\mathbf{u}}_{\mathbf{i}\left(\mathbf{u}\right)},r)}{sup}\left|{\widehat{\psi }}_1^{\left(1\right)}(\mathbf{x},\mathit{\theta },\mathbf{u})-E_{·\mid \mathit{S}}\left[{\widehat{\psi }}_1^{\left(1\right)}(\mathbf{x},\mathit{\theta },\mathbf{u})\right]\right|>2^{md+1}M{a}_n\right)\hfill \\ \le \mathfrak{N}\underset{1\le i_1<\cdots <i_m\le m}{max}\underset{B_{{\theta }_{\mathbf{i}\left(x\right)}}(x_{\mathbf{i}\left(x\right)},r)}{sup}\underset{1\le i_1<\cdots <i_m\le m}{max}\underset{B({\mathbf{u}}_{\mathbf{i}\left(\mathbf{u}\right)},r)}{sup}\hfill \\ P_{·\mid \mathit{S}}\left(\underset{(\mathit{u},\mathit{x})\in B_k}{sup}\left|{\widehat{\psi }}_1(\mathbf{x},\mathit{\theta },\mathbf{u})-E_{·\mid \mathit{S}}\left[{\widehat{\psi }}_1(\mathbf{x},\mathit{\theta },\mathbf{u})\right]\right|>2^{md+1}M{a}_n\right)\hfill \\ \le \sum _{\mathit{ϵ}\in {\{1,2\}}^d}{\widehat{Q}}_n\left(\mathit{ϵ}\right)+\sum _{\mathit{ϵ}\in {\{1,2\}}^d}{\overline{Q}}_n\left(\mathit{ϵ}\right)+2^{md+1}\mathfrak{N}{\left(\frac{A_n}{A_{1,n}}\right)}^d\beta \left(A_{2,n};A_n^d\right),\hfill \end{array}$$

  (80)

  where

  $$\begin{array}{cc}\hfill & {\widehat{Q}}_n\left({\mathit{ϵ}}_0\right)=\mathfrak{N}\underset{1\le i_1<\cdots <i_m\le m}{max}\underset{B_{{\theta }_{\mathbf{i}\left(x\right)}}(x_{\mathbf{i}\left(x\right)},r)}{sup}\hfill \\ \hfill & \underset{1\le i_1<\cdots <i_m\le m}{max}\underset{B({\mathbf{u}}_{\mathbf{i}\left(\mathbf{u}\right)},r)}{sup}{P}_{·\mid \mathit{S}}\left(\left|\sum _{\ell \in L_n}{\breve{S}}_n(\ell ;{\mathit{ϵ}}_0)\right|>M{a}_{n}n{h}^{md}\varphi \left(h\right)\right),\hfill \\ \hfill & {\overline{Q}}_n\left({\mathit{ϵ}}_0\right)=\mathfrak{N}\underset{1\le i_1<\cdots <i_m\le m}{max}\underset{B_{{\theta }_{\mathbf{i}\left(x\right)}}(x_{\mathbf{i}\left(x\right)},r)}{sup}\hfill \\ \hfill & \underset{1\le i_1<\cdots <i_m\le m}{max}\underset{B({\mathbf{u}}_{\mathbf{i}\left(\mathbf{u}\right)},r)}{sup}{P}_{·\mid \mathit{S}}\left(\left|\sum _{\ell \in L_n}{\breve{S}}_n^{\prime }(\ell ;{\mathit{ϵ}}_0)\right|>M{a}_{n}n{h}^{md}\varphi \left(h\right)\right),\hfill \end{array}$$

  and for $\mathit{ϵ}\ne {\mathit{ϵ}}_0$

  $$\begin{array}{cc}\hfill & {\widehat{Q}}_n\left(\mathit{ϵ}\right)=\mathfrak{N}\underset{1\le i_1<\cdots <i_m\le m}{max}\underset{B_{{\theta }_{\mathbf{i}\left(x\right)}}(x_{\mathbf{i}\left(x\right)},r)}{sup}\hfill \\ \hfill & \underset{1\le i_1<\cdots <i_m\le m}{max}\underset{B({\mathbf{u}}_{\mathbf{i}\left(\mathbf{u}\right)},r)}{sup}{P}_{·\mid \mathit{S}}\left(\left|\sum _{\ell \in L_n}{\breve{S}}_n(\ell ;\mathit{ϵ})\right|>M{a}_{n}n{h}^{md}\varphi \left(h\right)\right),\hfill \\ \hfill & {\overline{Q}}_n\left(\mathit{ϵ}\right)=\mathfrak{N}\underset{1\le i_1<\cdots <i_m\le m}{max}\underset{B_{{\theta }_{\mathbf{i}\left(x\right)}}(x_{\mathbf{i}\left(x\right)},r)}{sup}\hfill \\ \hfill & \underset{1\le i_1<\cdots <i_m\le m}{max}\underset{B({\mathbf{u}}_{\mathbf{i}\left(\mathbf{u}\right)},r)}{sup}{P}_{·\mid \mathit{S}}\left(\left|\sum _{\ell \in L_n}{\breve{S}}_n^{\prime }(\ell ;\mathit{ϵ})\right|>M{a}_{n}n{h}^{md}\varphi \left(h\right)\right).\hfill \end{array}$$

  Because of the similarity between the two cases: $\mathit{ϵ}\ne {\mathit{ϵ}}_0$ and $\mathit{ϵ}={\mathit{ϵ}}_0$, we will exclusively analyze ${\widehat{Q}}_n$ for $\mathit{ϵ}\ne {\mathit{ϵ}}_0$. By employing Lemma 6, considering that ${\breve{S}}_n(\ell ;\mathit{ϵ})$ are zero-mean random variables, we deduce that

  $$P_{·\mid \mathit{S}}\left(\left|\sum _{\ell \in L_n}{\breve{S}}_n(\ell ;\mathit{ϵ})\right|>M{a}_{n}n{h}^{md}\varphi \left(h\right)\right)\le 2P_{·\mid \mathit{S}}\left(\sum _{\ell \in L_n}{\breve{S}}_n(\ell ;\mathit{ϵ})>M{a}_{n}n{h}^{md}\varphi \left(h\right)\right)$$

  and

  $$\begin{array}{ccc}\hfill \left|{\breve{S}}_n(\ell ;\mathit{ϵ})\right|& \le & C{A}_{1,n}^{d-1}{A}_{2,n}(logn){\tau }_n,P_{\mathit{S}}-\mathrm{a}.\mathrm{s}.(\mathrm{from}\mathrm{Lemma}6)\hfill \\ \hfill E_{·\mid \mathit{S}}\left[{\left({\breve{S}}_n(\ell ;\mathit{ϵ})\right)}^2\right]& \le & C{h}^{md}\varphi \left(h\right)A_{1,n}^{d-1}{A}_{2,n}(logn),P_{\mathit{S}}-\mathrm{a}.\mathrm{s}.\left(\mathrm{By}\mathrm{Lemma}7\right)\hfill \end{array}$$

  (81)

  Applying Bernstein’s inequality as stated in Lemma 8, we obtain

  $$\begin{array}{c}{\displaystyle P_{·\mid \mathit{S}}\left(\sum _{\ell \in L_n}{\breve{S}}_n(\ell ;\mathit{ϵ})>M{a}_{n}n{h}^{md}\varphi \left(h\right)\right)}\hfill \\ \le exp\left(-\frac{{\displaystyle \frac{1}{2}Mn{h}^{md}\varphi \left(h\right)logn}}{{\displaystyle {\left(\frac{A_n}{A_{1,n}}\right)}^d{A}_{1,n}^{d-1}{A}_{2,n}{h}^{md}\varphi \left(h\right)(logn)+\frac{1}{3}{M}^{1/2}{n}^{1/2}{h}^{md/2}\varphi {\left(h\right)}^{1/2}{(logn)}^{1/2}{A}_{1,n}^{d-1}{A}_{2,n}{\tau }_n}}\right).\hfill \end{array}$$

  Observe that

  $$\begin{array}{ccc}\hfill \frac{{\displaystyle n{h}^{md}logn}}{{\displaystyle {\left(\frac{A_n}{A_{1,n}}\right)}^d{A}_{1,n}^{d-1}{A}_{2,n}{h}^{md}(logn)}}& =& n{A}_n^{-d}\left(\frac{A_{1,n}}{A_{2,n}}\right)\gtrsim \frac{A_{1,n}}{A_{2,n}}\gtrsim n^{\eta },\hfill \\ \hfill \frac{{\displaystyle n{h}^{md}\varphi \left(h\right)logn}}{{\displaystyle n^{1/2}{h}^{md/2}\varphi {\left(h\right)}^{1/2}{(logn)}^{1/2}{A}_{1,n}^{d-1}{A}_{2,n}{\tau }_n}}& =& \frac{{\displaystyle n^{1/2}{h}^{md/2}\varphi {\left(h\right)}^{1/2}{(logn)}^{1/2}}}{{\displaystyle A_{1,n}^{md}\left(\frac{A_{2,n}}{A_{1,n}}\right){\rho }_n{n}^{1/\zeta }}}\ge C_0{n}^{\eta /2}.\hfill \end{array}$$

  Selecting $M>0$ sufficiently large, and for $N\le C{h}^{-md}\varphi \left(h\right){\alpha }_n^{-m}$, this establishes the desired result. Now, we proceed to the nonlinear component of the Hoeffding decomposition. The objective is to demonstrate that

  $$P_{·\mid \mathit{S}}\left[\underset{{\mathcal{F}}_m{\mathfrak{K}}_{\Theta }^m}{sup}\underset{\mathit{\theta }\in {\Theta }^m}{sup}\underset{\mathbf{x}\in {\mathcal{H}}^m}{sup}\underset{\mathbf{u}\in B^m}{sup}\left|{\widehat{\psi }}_2(\mathbf{x},\mathit{\theta },\mathbf{u})\right|>\lambda \right]\to 0\mathrm{as}n\to \infty .$$

  (82)

Below, we present a lemma that serves as a technical element in proving our proposition. It aids us in attaining our objective in Expression (82). The proof of this lemma involves the use of the Blocking technique introduced earlier, specifically for the U-statistic, making the block treatment more intricate.

Lemma 1.

Let ${\mathcal{F}}_m{\mathcal{K}}^m$ be a uniformly bounded class of measurable canonical functions, $m\ge 2$. Assume there exist finite constants $\mathbf{a}$ and $\mathbf{b}$ such that the ${\mathcal{F}}_m{\mathcal{K}}^m$ covering number satisfies:

$$N(ϵ,{\mathcal{F}}_m{\mathcal{K}}^m{,\parallel ·\parallel }_{L_2\left(Q\right)})\le \mathbf{a}{\mathit{ϵ}}^{-\mathbf{b}},$$

(83)

for all $ϵ>0$ and all probability measure Q. If the mixing coefficients β of the local stationary sequence $\{Z_i={(X_{{\mathit{s}}_{\mathbf{i}},A_n},W_{{\mathit{s}}_{\mathbf{i}},A_n})}_{i\in {\mathbb{N}}^{★}}\}$ satisfy Condition (E2) in Assumption 6, then, for some $r>1$, we have

$$\underset{{\mathcal{F}}_m{\mathfrak{K}}_{\Theta }^m}{sup}\underset{\mathit{\theta }\in {\Theta }^m}{sup}\underset{\mathbf{x}\in {\mathcal{H}}^m}{sup}\underset{\mathbf{u}\in B^m}{sup}\mathbb{P}\left[h^{md/2}{\varphi }^{m/2}\left(h\right)n^{-m+1/2}\sum _{\mathbf{i}\in I_n^m}{\xi }_{i_1}\cdots {\xi }_{i_m}H(Z_{i_1},\dots ,Z_{i_m})\right]\to 0.$$

(84)

Remark 12.

As mentioned before, $W_{{\mathit{s}}_{\mathbf{i}},A_n}$ will be equal to 1 or ${ϵ}_{{\mathit{s}}_{\mathbf{i}},A_n}=\sigma \left(\frac{{\mathit{s}}_{\mathbf{i}}}{A_n},X_{{\mathit{s}}_{\mathbf{i}},A_n}\right){ϵ}_{{\mathit{s}}_{\mathbf{i}}}$. In the proof of the previous Lemma, $W_{{\mathit{s}}_{\mathbf{i}},A_n}$ will be equal to ${\epsilon }_{\mathbf{i},n}=\sigma \left(\frac{\mathbf{i}}{n},X_{\mathbf{i},n}\right){ϵ}_{\mathbf{i}}$, and we will use the notation $W_{{\mathit{s}}_{\mathbf{i}},A_n}^{\left(u\right)}$ to indicate $\sigma \left(\mathit{u},\mathbf{x}\right){ϵ}_{\mathbf{i}}$.

□

Proof of Lemma 1.

The proof of this lemma relies on the Blocking technique introduced by [77], known as Bernstein’s method [172]. This technique allows us to apply symmetrization and various other methods typically used for i.i.d. random variables. We extend this technique to spatial processes in the context of U-statistics, following the approach in [89]. In addition to the notation in Section 9, we define

$$L_n:=L_{1,n}\cup L_{2,n},$$

$${\Delta }_1=\{{\ell }_{\mathbf{2}}:{\displaystyle \underset{1\le i\le d}{min}}|{\ell }_{\mathbf{1}\mathit{i}}-{\ell }_{\mathbf{2}\mathit{i}}|\le \mathbf{1}\}$$

$${\Delta }_2=\{{\ell }_{\mathbf{2}}:{\displaystyle \underset{1\le i\le d}{min}}|{\ell }_{\mathbf{1}\mathit{i}}-{\ell }_{\mathbf{2}\mathit{i}}|\ge \mathbf{2}\}$$

With the introduced notation, it is straightforward to demonstrate that for $m=2$,

$$\begin{array}{c}{\displaystyle \frac{1}{h^{2d}{\varphi }^2\left(h\right)}\sum _{\mathbf{i}\in I_n^2}\prod _{j=1}^2\left\{\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)K_2\left(\frac{d_{{\theta }_j}(x_j,{\mathbf{X}}_{{\mathit{s}}_{i_j},A_n})}{h_n}\right)\right\}{W}_{{\mathit{s}}_{\mathbf{i}},A_n}}\hfill \\ = \frac{1}{h^{2d}{\varphi }^2\left(h\right)}\sum _{{\ell }_{\mathbf{1}}\ne {\ell }_{\mathbf{2}}\in L_n}\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};{ϵ}_0)\cap R_n}\hfill \\ \times \prod _{j=1}^2\left\{\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)K_2\left(\frac{d_{{\theta }_j}(x_j,{\mathbf{X}}_{{\mathit{s}}_{i_j},A_n})}{h_n}\right)\right\}{W}_{{\mathit{s}}_{\mathbf{i}},A_n}\hfill \\ +\frac{1}{h^{2d}{\varphi }^2\left(h\right)}\sum _{{\ell }_{\mathbf{1}}\in L_n}\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\underset{i_1\ne i_2}{\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}}\hfill \\ \times \prod _{j=1}^2\left\{\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)K_2\left(\frac{d_{{\theta }_j}(x_j,{\mathbf{X}}_{{\mathit{s}}_{i_j},A_n})}{h_n}\right)\right\}{W}_{{\mathit{s}}_{\mathbf{i}},A_n}\hfill \\ +2\frac{1}{h^{2d}{\varphi }^2\left(h\right)}\sum _{{\ell }_{\mathbf{1}}\in L_n}\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\sum _{{\Delta }_2}\underset{ϵ\ne {ϵ}_0}{\sum _{{\ell }_{\mathbf{2}}\in L_{1,n}\cup L_{2,n}}}\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};ϵ)\cap R_n}\hfill \\ \times \prod _{j=1}^2\left\{\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)K_2\left(\frac{d_{{\theta }_j}(x_j,{\mathbf{X}}_{{\mathit{s}}_{i_j},A_n})}{h_n}\right)\right\}{W}_{{\mathit{s}}_{\mathbf{i}},A_n}\hfill \\ +2\frac{1}{h^{2d}{\varphi }^2\left(h\right)}\sum _{{\ell }_{\mathbf{1}}\in L_n}\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\sum _{{\Delta }_1}\underset{ϵ\ne {ϵ}_0}{\sum _{{\ell }_{\mathbf{2}}\in L_{1,n}\cup L_{2,n}}}\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};ϵ)\cap R_n}\hfill \\ \times \prod _{j=1}^2\left\{\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)K_2\left(\frac{d_{{\theta }_j}(x_j,{\mathbf{X}}_{{\mathit{s}}_{i_j},A_n})}{h_n}\right)\right\}{W}_{{\mathit{s}}_{\mathbf{i}},A_n}\hfill \\ +\frac{1}{h^{2d}{\varphi }^2\left(h\right)}\underset{ϵ\ne {ϵ}_0}{\sum _{{\ell }_{\mathbf{1}}\ne {\ell }_{\mathbf{2}}\in L_{1,n}\cup L_{2,n}}}\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};ϵ)\cap R_n}\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};ϵ)\cap R_n}\hfill \\ \times \prod _{j=1}^2\left\{\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)K_2\left(\frac{d_{{\theta }_j}(x_j,{\mathbf{X}}_{{\mathit{s}}_{i_j},A_n})}{h_n}\right)\right\}{W}_{{\mathit{s}}_{\mathbf{i}},A_n}\hfill \\ +\frac{1}{h^{2d}{\varphi }^2\left(h\right)}\underset{ϵ\ne {ϵ}_0}{\sum _{{\ell }_{\mathbf{1}}\in L_{1,n}\cup L_{2,n}}}\sum _{i_1<i_2:s_{i_1},s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{1}};ϵ)\cap R_n}\hfill \\ \times \prod _{j=1}^2\left\{\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)K_2\left(\frac{d_{{\theta }_j}(x_j,{\mathbf{X}}_{{\mathit{s}}_{i_j},A_n})}{h_n}\right)\right\}{W}_{{\mathit{s}}_{\mathbf{i}},A_n}\hfill \\ :=\mathrm{I}+\mathrm{II}+\mathrm{III}+\mathrm{IV}+\mathrm{V}+\mathrm{VI}.\hfill \end{array}$$

(85)

(I):

The same type of blocks but not the same block:

Let ${\left\{{\eta }_i\right\}}_{i\in {\mathbb{N}}^{*}}$ be a sequence of independent blocks. An application of Lemma 5 shows that

$$\begin{array}{c}{\displaystyle \mathbb{P}\left(\underset{{\mathcal{F}}_m{\mathcal{K}}^m}{sup}\underset{\mathit{\theta }\in {\Theta }^m}{sup}\underset{\mathbf{x}\in {\mathcal{H}}^m}{sup}\underset{\mathbf{u}\in B^m}{sup}\left|n^{-3/2}\frac{1}{h^{2d}{\varphi }^2\left(h\right)}\sum _{{\ell }_{\mathbf{1}}\ne {\ell }_{\mathbf{2}}\in L_n}\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};{ϵ}_0)\cap R_n}\right.\right.}\hfill \\ \left.\left.\times \prod _{j=1}^2\left\{\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)K_2\left(\frac{d_{{\theta }_j}(x_j,{\mathbf{X}}_{{\mathit{s}}_{i_j},A_n})}{h_n}\right)\right\}{W}_{{\mathit{s}}_{\mathbf{i}},A_n}\right|>\delta \right)\hfill \\ \le \mathbb{P}(\underset{{\mathcal{F}}_m{\mathcal{K}}^m}{sup}\underset{\mathit{\theta }\in {\Theta }^m}{sup}\underset{\mathbf{x}\in {\mathcal{H}}^m}{sup}\underset{\mathbf{u}\in B^m}{sup}\left|\frac{1}{n^{3/2}{h}^{2d}{\varphi }^2\left(h\right)}\sum _{{\ell }_{\mathbf{1}}\ne {\ell }_{\mathbf{2}}\in L_n}\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};{ϵ}_0)\cap R_n}\right.\hfill \\ \prod _{j=1}^2\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)\left.\left[\prod _{j=1}^2{K}_2\left(\frac{d_{{\theta }_j}(x_j,{\mathbf{X}}_{{\mathit{s}}_{i_j},A_n})}{h_n}\right)-\prod _{j=1}^2{K}_2\left(\frac{d_{{\theta }_i}\left(x_i,X_{{\mathit{u}}_j}\left({\mathit{s}}_{i_j}\right)\right)}{h}\right)\right]W_{{\mathit{s}}_{\mathbf{i}},A_n}\right|>\delta )\hfill \\ +\mathbb{P}(\underset{{\mathcal{F}}_m{\mathcal{K}}^m}{sup}\underset{\mathit{\theta }\in {\Theta }^m}{sup}\underset{\mathbf{x}\in {\mathcal{H}}^m}{sup}\underset{\mathbf{u}\in B^m}{sup}\left|\frac{1}{n^{3/2}{h}^{2d}{\varphi }^2\left(h\right)}\sum _{{\ell }_{\mathbf{1}}\ne {\ell }_{\mathbf{2}}\in L_n}\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};{ϵ}_0)\cap R_n}\right.\hfill \\ \prod _{j=1}^2\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)\left.\prod _{j=1}^2{K}_2\left(\frac{d_{{\theta }_i}\left(x_i,X_{{\mathit{u}}_j}\left({\mathit{s}}_{i_j}\right)\right)}{h}\right)\left[W_{{\mathit{s}}_{\mathbf{i}},A_n}-W_{{\mathit{s}}_{\mathbf{i}},A_n}^{\left(u\right)}\right]\right|>\delta )\hfill \\ + \mathbb{P}(\underset{{\mathcal{F}}_m{\mathcal{K}}^m}{sup}\underset{\mathit{\theta }\in {\Theta }^m}{sup}\underset{\mathbf{x}\in {\mathcal{H}}^m}{sup}\underset{\mathbf{u}\in B^m}{sup}\left|\frac{1}{n^{3/2}{h}^{2d}{\varphi }^2\left(h\right)}\sum _{{\ell }_{\mathbf{1}}\ne {\ell }_{\mathbf{2}}\in L_n}\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};{ϵ}_0)\cap R_n}\right.\hfill \\ \prod _{j=1}^2\overline{K}\left(\frac{{\mathit{u}}_j-{\mathit{s}}_{i_j}/A_n}{h_n}\right)\left.\prod _{j=1}^2{K}_2\left(\frac{d_{{\theta }_i}\left(x_i,X_{{\mathit{u}}_j}\left({\mathit{s}}_{i_j}\right)\right)}{h}\right)W_{{\mathit{s}}_{\mathbf{i}},A_n}^{\left(u\right)}\right|>\delta )\hfill \\ \le \mathbb{P}(\underset{{\mathcal{F}}_m{\mathcal{K}}^m}{sup}\underset{\mathit{\theta }\in {\Theta }^m}{sup}\underset{\mathbf{x}\in {\mathcal{H}}^m}{sup}\underset{\mathbf{u}\in B^m}{sup}\left|\frac{1}{n^{3/2}{h}^{2d}{\varphi }^2\left(h\right)}\sum _{{\ell }_{\mathbf{1}}\ne {\ell }_{\mathbf{2}}\in L_n}\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};{ϵ}_0)\cap R_n}\right.\hfill \\ \prod _{j=1}^2\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)\left.\left.\prod _{j=1}^2{K}_2\left(\frac{d_{{\theta }_j}(x_j,{\eta }_{i_j}}{h}\right)W_{{\mathit{s}}_{\mathbf{i}},A_n}^{\left(u\right)}\right|>\delta \right)\hfill \\ +C{\left(\frac{A_n}{A_{1,n}}\right)}^d\beta \left(A_{2,n};A_n^d\right)+o\left(1\right),\hfill \end{array}$$

Because

$$\begin{array}{c}{\displaystyle E_{.\mid \mathbf{S}}\left|\frac{1}{n^{3/2}{h}^{2d}{\varphi }^2\left(h\right)}\sum _{{\ell }_{\mathbf{1}}\ne {\ell }_{\mathbf{2}}\in L_n}\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};{ϵ}_0)\cap R_n}\prod _{j=1}^2\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)\right.}\hfill \\ \left.\left[\prod _{j=1}^2{K}_2\left(\frac{d_{{\theta }_j}(x_j,{\mathbf{X}}_{{\mathit{s}}_{i_j},A_n})}{h_n}\right)-\prod _{j=1}^2{K}_2\left(\frac{d_{{\theta }_j}\left(x_j,X_{{\mathit{u}}_j}\left({\mathit{s}}_{i_j}\right)\right)}{h}\right)\right]W_{{\mathit{s}}_{\mathbf{i}},A_n}\right|\hfill \\ = \frac{1}{n^{3/2}{h}^{2d}{\varphi }^2\left(h\right)}\sum _{{\ell }_{\mathbf{1}}\ne {\ell }_{\mathbf{2}}\in L_n}\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};{ϵ}_0)\cap R_n}\prod _{j=1}^2\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)\hfill \\ E_{.\mid \mathbf{S}}\left|\left[\prod _{j=1}^2{K}_2\left(\frac{d_{{\theta }_j}(x_j,{\mathbf{X}}_{{\mathit{s}}_{i_j},A_n})}{h_n}\right)-\prod _{j=1}^2{K}_2\left(\frac{d_{{\theta }_j}\left(x_j,X_{{\mathit{u}}_j}\left({\mathit{s}}_{i_j}\right)\right)}{h}\right)\right]W_{{\mathit{s}}_{\mathbf{i}},A_n}\right|\hfill \\ = \frac{1}{n^{3/2}{h}^{2d}{\varphi }^2\left(h\right)}\sum _{{\ell }_{\mathbf{1}}\ne {\ell }_{\mathbf{2}}\in L_n}\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};{ϵ}_0)\cap R_n}\prod _{j=1}^2\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)\hfill \\ E_{.\mid \mathbf{S}}\left|\left[\prod _{j=1}^2{K}_2\left(\frac{d_{{\theta }_j}(x_j,{\mathbf{X}}_{{\mathit{s}}_{i_j},A_n})}{h_n}\right)-\prod _{j=1}^2{K}_2\left(\frac{d_{{\theta }_j}\left(x_j,X_{{\mathit{u}}_j}\left({\mathit{s}}_{i_j}\right)\right)}{h}\right)\right]\prod _{j=1}^m{ϵ}_{{\mathit{s}}_{i_j},A_n}\right|\hfill \\ = \frac{1}{n^{3/2}{h}^{2d}{\varphi }^2\left(h\right)}\sum _{{\ell }_{\mathbf{1}}\ne {\ell }_{\mathbf{2}}\in L_n}\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};{ϵ}_0)\cap R_n}\prod _{j=1}^2\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)\hfill \\ E_{.\mid \mathbf{S}}\left|\left[\prod _{j=1}^2{K}_2\left(\frac{d_{{\theta }_j}(x_j,{\mathbf{X}}_{{\mathit{s}}_{i_j},A_n})}{h_n}\right)-\prod _{j=1}^2{K}_2\left(\frac{d_{{\theta }_j}\left(x_j,X_{{\mathit{u}}_j}\left({\mathit{s}}_{i_j}\right)\right)}{h}\right)\right]\prod _{j=1}^m\sigma \left(\frac{{\mathit{s}}_{i_j}}{A_n},X_{{\mathit{s}}_{i_j},A_n}\right){ϵ}_{{\mathit{s}}_{i_j}}\right|\hfill \\ = \frac{1}{n^{3/2}{h}^{2d}{\varphi }^2\left(h\right)}\sum _{{\ell }_{\mathbf{1}}\ne {\ell }_{\mathbf{2}}\in L_n}\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};{ϵ}_0)\cap R_n}\prod _{j=1}^2\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)\prod _{j=1}^m{E}_{.\mid \mathbf{S}}\left({ϵ}_{{\mathit{s}}_{i_j}}\right)\hfill \\ E_{.\mid \mathbf{S}}\left|\left[\prod _{j=1}^2{K}_2\left(\frac{d_{{\theta }_j}(x_j,{\mathbf{X}}_{{\mathit{s}}_{i_j},A_n})}{h_n}\right)-\prod _{j=1}^2{K}_2\left(\frac{d_{{\theta }_j}\left(x_j,X_{{\mathit{u}}_j}\left({\mathit{s}}_{i_j}\right)\right)}{h}\right)\right]\right.\hfill \\ \left.\left[\prod _{j=1}^m\sigma \left(\frac{{\mathit{s}}_{i_j}}{A_n},X_{{\mathit{s}}_{i_j},A_n}\right)-\prod _{j=1}^m\sigma ({\mathbf{x}}_j,{\mathit{\theta }}_j,{\mathbf{u}}_j)+\prod _{j=1}^m\sigma ({\mathbf{x}}_j,{\theta }_j,{\mathit{u}}_j)\right]\right|\hfill \\ \le \frac{1}{n^{3/2}{h}^{2d}{\varphi }^2\left(h\right)}\sum _{{\ell }_{\mathbf{1}}\ne {\ell }_{\mathbf{2}}\in L_n}\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};{ϵ}_0)\cap R_n}\prod _{j=1}^2\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)\prod _{j=1}^m{E}_{.\mid \mathbf{S}}\left({ϵ}_{{\mathit{s}}_{i_j}}\right)\hfill \\ E_{.\mid \mathbf{S}}\left[C\sum _{j=1}^m{\left|K_2\left(\frac{d_{{\theta }_j}(x_j,X_{{\mathit{s}}_{i_j},A_n})}{h_n}\right)-K_2\left(\frac{d_{{\theta }_j}\left(x_j,X_{\frac{{\mathit{s}}_{i_j}}{A_n}}\left(s_{i_j}\right)\right)}{h}\right)\right|}^p\right]\prod _{j=1}^m\left[\sigma ({\mathbf{x}}_j,{\mathit{\theta }}_j,{\mathbf{u}}_j)+o_{\mathbb{P}}\left(1\right)\right]\hfill \\ (\mathrm{using}\mathrm{a}\mathrm{telescoping}\mathrm{argument},\mathrm{and}\mathrm{the}\mathrm{boundedness}\mathrm{of}{K}_2(·)\mathrm{for}p=min(\rho ,1)\mathrm{and}C<\infty )\hfill \\ \lesssim \frac{1}{n^{3/2}{h}^{2d}{\varphi }^2\left(h\right)}\sum _{{\ell }_{\mathbf{1}}\ne {\ell }_{\mathbf{2}}\in L_n}\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};{ϵ}_0)\cap R_n}\prod _{j=1}^2\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)\prod _{j=1}^m{E}_{.\mid \mathbf{S}}\left({ϵ}_{{\mathit{s}}_{i_j}}\right)\hfill \\ E_{.\mid \mathbf{S}}\left[{\varphi }^{m-1}\left(h\right){\left|\frac{C}{A_n^d}{U}_{{\mathit{s}}_{i_j},A_n}\left(\frac{{\mathit{s}}_{i_j}}{A_n}\right)\right|}^p\right]\prod _{j=1}^m\left[\sigma ({\mathbf{x}}_j,{\mathit{\theta }}_j,{\mathbf{u}}_j)+o_{\mathbb{P}}\left(1\right)\right]\hfill \\ \sim o_{\mathbb{P}}\left(1\right),\hfill \end{array}$$

(86)

and

$$\begin{array}{c}{\displaystyle E_{.\mid \mathbf{S}}\left|\frac{1}{n^{3/2}{h}^{2d}{\varphi }^2\left(h\right)}\sum _{{\ell }_{\mathbf{1}}\ne {\ell }_{\mathbf{2}}\in L_n}\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};{ϵ}_0)\cap R_n}\prod _{j=1}^2\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)\right.}\hfill \\ \left.\prod _{j=1}^2{K}_2\left(\frac{d_{{\theta }_i}\left(x_i,X_{{\mathit{u}}_j}\left({\mathit{s}}_{i_j}\right)\right)}{h}\right)\left[W_{{\mathit{s}}_{\mathbf{i}},A_n}-W_{{\mathit{s}}_{\mathbf{i}},A_n}^{\left(u\right)}\right]\right|\hfill \\ = \frac{1}{n^{3/2}{h}^{2d}{\varphi }^2\left(h\right)}\sum _{{\ell }_{\mathbf{1}}\ne {\ell }_{\mathbf{2}}\in L_n}\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};{ϵ}_0)\cap R_n}\prod _{j=1}^2\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)\prod _{j=1}^m{E}_{.\mid \mathbf{S}}\left({ϵ}_{{\mathit{s}}_{i_j}}\right)\hfill \\ E_{.\mid \mathbf{S}}\left|\prod _{j=1}^2{K}_2\left(\frac{d_{{\theta }_i}\left(x_i,X_{{\mathit{u}}_j}\left({\mathit{s}}_{i_j}\right)\right)}{h}\right)\left[\prod _{j=1}^m\sigma \left(\frac{{\mathit{s}}_{i_j}}{A_n},X_{{\mathit{s}}_{i_j},A_n}\right)-\prod _{j=1}^m\sigma ({\mathbf{x}}_j,{\mathit{\theta }}_j,{\mathbf{u}}_j)\right]\right|\hfill \\ \lesssim \frac{1}{n^{3/2}{h}^{2d}{\varphi }^2\left(h\right)}\sum _{{\ell }_{\mathbf{1}}\ne {\ell }_{\mathbf{2}}\in L_n}\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};{ϵ}_0)\cap R_n}\prod _{j=1}^2\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)\prod _{j=1}^m{E}_{.\mid \mathbf{S}}\left({ϵ}_{{\mathit{s}}_{i_j}}\right)\hfill \\ \times \left(o_{\mathbb{P}}\left(1\right)\right){\int }_0^h\prod _{k=1}^m{K}_2\left(\frac{y_k}{h}\right)d{F}_{i_k/n}(y_k,x_k)\hfill \\ \lesssim \frac{1}{n^{3/2}{h}^{2d}{\varphi }^2\left(h\right)}\sum _{{\ell }_{\mathbf{1}}\ne {\ell }_{\mathbf{2}}\in L_n}\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};{ϵ}_0)\cap R_n}\prod _{j=1}^2\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)\prod _{j=1}^m{E}_{.\mid \mathbf{S}}\left({ϵ}_{{\mathit{s}}_{i_j}}\right)\hfill \\ \times \left(o_{\mathbb{P}}\left(1\right)\right)\left({\varphi }^2\left(h\right)\right)\hfill \\ \sim o_{\mathbb{P}}\left(1\right).\hfill \end{array}$$

(87)

Under the lemma’s assumptions, we have

$$\beta (a;b)\le {\beta }_1\left(a\right)g_1\left(b\right),$$

where ${\beta }_1\left(a\right)\to 0$ as $a\to \infty$ and $n\to \infty$. Therefore, our focus will be on the first term of the sum. For the second part of the inequality, we will leverage the work of [173] in the non-fixed kernels settings. Specifically, we define

$$f_{i_1,\dots ,i_m}=\prod _{k=1}^m{\xi }_{i_k}\times H,$$

and ${\mathcal{F}}_{i_1,\dots ,i_m}$ as the collection of kernels and the class of functions related to this kernel. We will then use (Theorem 3.1.1, [21] and Remarks 3.5.4 part 2) for decoupling and randomization. As mentioned earlier, we assume that $m=2$. Consequently, we can observe that

$$\begin{array}{c}{\displaystyle E_{.\mid \mathbf{S}}∥\frac{1}{n^{3/2}{h}^{2d}{\varphi }^2\left(h\right)}\sum _{{\ell }_{\mathbf{1}}\ne {\ell }_{\mathbf{2}}\in L_n}\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};{ϵ}_0)\cap R_n}\prod _{j=1}^2\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)}\hfill \\ {\prod _{j=1}^2{K}_2\left(\frac{d_{{\theta }_j}(x_j,{\eta }_{i_j}}{h}\right)W_{\mathbf{i},\phi ,n}^{\left(u\right)}∥}_{{\mathcal{F}}_2{\mathcal{K}}^2}\hfill \\ = E_{.\mid \mathbf{S}}{∥\frac{1}{n^{3/2}{h}^{2d}{\varphi }^2\left(h\right)}\sum _{{\ell }_{\mathbf{1}}\ne {\ell }_{\mathbf{2}}\in L_n}\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};{ϵ}_0)\cap R_n}{f}_{i_1,i_2}(\mathit{u},\mathit{\eta })∥}_{{\mathcal{F}}_{i_1,i_2}}\hfill \\ \le c_2{E}_{.\mid \mathbf{S}}{∥\frac{1}{n^{3/2}{h}^{2d}{\varphi }^2\left(h\right)}\sum _{{\ell }_{\mathbf{1}}\ne {\ell }_{\mathbf{2}}\in L_n}{ϵ}_p{ϵ}_q\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};{ϵ}_0)\cap R_n}{f}_{i_1,i_2}(\mathit{u},\mathit{\eta })∥}_{{\mathcal{F}}_{i_1,i_2}}\hfill \\ \le c_2{E}_{.\mid \mathbf{S}}{\int }_0^{D_{nh}^{\left(U_1\right)}}N\left(t,{\mathcal{F}}_{i_1,i_2},{\tilde{d}}_{nh,2}^{\left(1\right)}\right)dt,(\mathrm{By}\mathrm{Lemma}10\mathrm{and}\mathrm{Proposition}2.)\hfill \end{array}$$

(88)

where $D_{nh}^{\left(U_1\right)}$ is the diameter of ${\mathcal{F}}_{i_1,i_2}$according to the distance ${\tilde{d}}_{nh,2}^{\left(1\right)},$ respectively defined as

$$\begin{array}{ccc}\hfill D_{nh}^{\left(U_1\right)}& :=& {∥{\mathbb{E}}_{ϵ}\left|\frac{1}{n^{3/2}{h}^{2d}{\varphi }^2\left(h\right)}\sum _{{\ell }_{\mathbf{1}}\ne {\ell }_{\mathbf{2}}\in L_n}{ϵ}_p{ϵ}_q\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};{ϵ}_0)\cap R_n}{f}_{i_1,i_2}(\mathit{u},\mathit{\eta })\right|∥}_{{\mathcal{F}}_{i_1,i_2}}\hfill \\ & =& ∥{\mathbb{E}}_{ϵ}\left|\frac{1}{n^{3/2}{h}^{2d}{\varphi }^2\left(h\right)}\sum _{{\ell }_{\mathbf{1}}\ne {\ell }_{\mathbf{2}}\in L_n}{ϵ}_p{ϵ}_q\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};{ϵ}_0)\cap R_n}\prod _{j=1}^2\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)\right.\hfill \\ & & {\left.\prod _{j=1}^2{K}_2\left(\frac{d_{{\theta }_j}(x_j,{\eta }_{i_j}}{h}\right)W_{{\mathit{s}}_{\mathbf{i}},A_n}^{\left(u\right)}\right|∥}_{{\mathcal{F}}_2{\mathcal{K}}^2},\hfill \end{array}$$

and

$$\begin{array}{c}{\displaystyle {\tilde{d}}_{nh,2}^{\left(1\right)}\left({\xi }_{1.}{K}_{2,1}{W}^{\prime \left(u\right)},{\xi }_{2.}{K}_{2,2}{W}^{″\left(u\right)}\right)}\hfill \\ := {\mathbb{E}}_{ϵ}\left|\frac{1}{n^{3/2}{h}^d{\varphi }^2\left(h\right)}\sum _{{\ell }_{\mathbf{1}}\ne {\ell }_{\mathbf{2}}\in L_n}{ϵ}_p{ϵ}_q\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};{ϵ}_0)\cap R_n}\left[{\xi }_{1i_1}{\xi }_{1i_2}\right.\right.\hfill \\ \prod _{k=1}^2{K}_{1,2}\left(\frac{d_{{\theta }_k}(x_k,{\eta }_{i_k})}{h}\right)W_{{\mathit{s}}_{\mathbf{i}},A_n}^{\prime \left(u\right)}-\left.\phantom{\sum _{p\ne q}^{〚L_n〛}{ϵ}_p{ϵ}_q\sum _{i\in H_p^{\left(U\right)}}\sum _{j\in H_q^{\left(U\right)}}}\left.{\xi }_{2i_1}{\xi }_{2i_2}\prod _{k=1}^2{K}_{2,2}\left(\frac{d_{{\theta }_k}(x_k,{\eta }_{i_k})}{h}\right)W_{{\mathit{s}}_{\mathbf{i}},A_n}^{″\left(u\right)}\right]\right|.\hfill \end{array}$$

Let consider another semi-norm ${\tilde{d}}_{nh,2}^{\left(2\right)}:$

$$\begin{array}{c}{\displaystyle {\tilde{d}}_{nh,2}^{\left(2\right)}\left({\xi }_{1.}{K}_{2,1}{W}^{\prime \left(u\right)},{\xi }_{2.}{K}_{2,2}{W}^{″\left(u\right)}\right)}\\ = \frac{1}{n{h}^d{\varphi }^2\left(h\right)}\left[\sum _{{\ell }_{\mathbf{1}}\ne {\ell }_{\mathbf{2}}\in L_n}\left({\xi }_{1i_1}{\xi }_{1i_2}\prod _{k=1}^2{K}_{1,2}\left(\frac{d_{{\theta }_k}(x_k,{\eta }_{i_k})}{h}\right)W_{{\mathit{s}}_{\mathbf{i}},A_n}^{\prime \left(u\right)}\right.\right.\hfill \\ -\phantom{\sum _{p\ne q}^{{\upsilon }_n}{ϵ}_p{ϵ}_q\sum _{i\in H_p^{\left(U\right)}}\sum _{j\in H_q^{\left(U\right)}}}{\left.{\xi }_{2i_1}{\xi }_{2i_2}\prod _{k=1}^2{K}_{2,2}\left(\frac{d_{{\theta }_k}(x_k,{\eta }_{i_k})}{h}\right)W_{{\mathit{s}}_{\mathbf{i}},A_n}^{″\left(u\right)}\right)}^2{]}^{1/2}.\hfill \end{array}$$

One can see that

$$\begin{array}{c}{\displaystyle {\tilde{d}}_{nh,2}^{\left(1\right)}\left({\xi }_{1.}{K}_{2,1}{W}^{\prime \left(u\right)},{\xi }_{2.}{K}_{2,2}{W}^{″\left(u\right)}\right)}\hfill \\ ⩽ A_{1,n}{n}^{-1/2}{h}^d\varphi \left(h\right){\tilde{d}}_{nh,2}^{\left(2\right)}\left({\xi }_{1.}{K}_{2,1}{W}^{\prime \left(u\right)},{\xi }_{2.}{K}_{2,2}{W}^{″\left(u\right)}\right).\hfill \end{array}$$

We readily infer that

$$\begin{array}{c}\hfill {\displaystyle E_{.\mid \mathbf{S}}∥\frac{1}{n^{3/2}{h}^{2d}{\varphi }^2\left(h\right)}\sum _{{\ell }_{\mathbf{1}}\ne {\ell }_{\mathbf{2}}\in L_n}\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};{ϵ}_0)\cap R_n}\prod _{j=1}^2\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)}\\ {\prod _{j=1}^2{K}_2\left(\frac{d_{{\theta }_j}(x_j,{\eta }_{i_j}}{h}\right)W_{\mathbf{i},\phi ,n}^{\left(u\right)}∥}_{{\mathcal{F}}_2{\mathcal{K}}^2}\hfill \\ ⩽ c_2{E}_{.\mid \mathbf{S}}{\int }_0^{D_{nh}^{\left(U_1\right)}}N\left(t{A}_{1,n}^{-d}{n}^{1/2},{\mathcal{F}}_{i,j},{\tilde{d}}_{nh,2}^{\left(2\right)}\right)dt\hfill \\ ⩽ c_2{A}_{1,n}^d{n}^{-1/2}\mathbb{P}\left\{D_{nh}^{\left(U_1\right)}{A}_{1,n}^{-d}{n}^{1/2}⩾{\lambda }_n\right\}+c_m{A}_{1,n}^d{n}^{-1/2}{\int }_0^{{\lambda }_n}log{t}^{-1}dt,\hfill \end{array}$$

(89)

where ${\lambda }_n\to 0.$ We have

$$\frac{{\displaystyle {\int }_0^{{\lambda }_n}log{t}^{-1}dt}}{{\displaystyle {\lambda }_{n}log{\lambda }_n^{-1}}}\to 0,$$

where ${\lambda }_n$ must be chosen in such a way that the following relation will be achieved

$$A_{1,n}^d{\lambda }_n{n}^{-1/2}log{\lambda }_n^{-1}\to 0.$$

(90)

By employing the triangle inequality along with Hoeffding’s trick, we can readily deduce that

$$\begin{array}{c}{\displaystyle A_{1,n}^d{n}^{-1/2}\mathbb{P}\left\{D_{nh}^{\left(U_1\right)}⩾{\lambda }_n{A}_{1,n}^d{n}^{-1/2}\right\}}\hfill \\ ⩽{\lambda }_n^{-2}{A}_{1,n}^{-d}{n}^{-5/2}h{\varphi }_{\mathbf{x},\theta }^{-1}\left(h_n\right)E_{.\mid \mathit{S}}∥\sum _{{\ell }_{\mathbf{1}}\ne {\ell }_{\mathbf{2}}\in L_n}\left[\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};{ϵ}_0)\cap R_n}{\xi }_{i_1}{\xi }_{i_2}{K}_2\left(\frac{d_{{\theta }_1}(x_1,{\eta }_{i_1})}{h}\right)\right.\hfill \\ {{\left.K_2\left(\frac{d_{{\theta }_2}(x_2,{\eta }_{i_2}^{\prime })}{h}\right)W_{{\mathit{s}}_{\mathbf{i}},A_n}^{\left(u\right)}\right]}^2∥}_{{\mathcal{F}}_2{\mathcal{K}}^2}\hfill \\ ⩽c_2\left[\left[L_n\right]\right]{\lambda }_n^{-2}{A}_{1,n}^{-d}{n}^{-5/2}h{\varphi }_{\mathbf{x},\theta }^{-1}\left(h_n\right)E_{.\mid \mathit{S}}∥\sum _{{\ell }_{\mathbf{1}}\in L_n}\left[\sum _{i_1,i_2:s_{i_1},s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}{\xi }_{i_1}{\xi }_{i_2}{K}_2\left(\frac{d_{{\theta }_1}(x_1,{\eta }_{i_1})}{h}\right)\right.\hfill \\ {{\left.K_2\left(\frac{d_{{\theta }_2}(x_2,{\eta }_{i_2}^{\prime })}{h}\right)W_{{\mathit{s}}_{\mathbf{i}},A_n}^{\left(u\right)}\right]}^2∥}_{{\mathcal{F}}_2{\mathcal{K}}^2},\hfill \end{array}$$

(91)

where ${\left\{{\eta }_i^{\prime }\right\}}_{i\in {\mathbb{N}}^{*}}$ are independent copies of ${\left({\eta }_i\right)}_{i\in {\mathbb{N}}^{*}}$. By imposing

$${\lambda }_n^{-2}{A}_{1,n}^{d-r}{n}^{-1/2}\to 0,$$

(92)

we readily infer that

$$\begin{array}{c}{\displaystyle {∥\left[\left[L_n\right]\right]{\lambda }_n^{-2}{A}_{1,n}^{-d}{n}^{-5/2}h{\varphi }_{\mathbf{x},\theta }^{-1}\left(h_n\right)E_{.\mid \mathit{S}}\sum _{{\ell }_{\mathbf{1}}\in L_n}{\left[\sum _{i_1,i_2:s_{i_1},s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}{\xi }_{i_1}{\xi }_{i_2}\prod _{k=1}^2{K}_2\left(\frac{d_{{\theta }_k}(x_k,{\eta }_{i_k})}{h}\right)W_{{\mathit{s}}_{\mathbf{i}},A_n}^{\left(u\right)}\right]}^2∥}_{{\mathcal{F}}_2{\mathcal{K}}^2}}\hfill \\ ⩽O\left({\lambda }_n^{-2}{A}_{1,n}^{d-r}{n}^{-1/2}\right).\hfill \end{array}$$

Symmetrizing the last inequality in (91) and then applying Proposition 2, yields

$$\begin{array}{c}\left[\left[L_n\right]\right]{\lambda }_n^{-2}{A}_{1,n}^{-d}{n}^{-5/2}h{\varphi }_{\mathbf{x},\theta }^{-1}\left(h_n\right)E_{.\mid \mathbf{S}}∥\sum _{{\ell }_{\mathbf{1}}\in L_n}\left[\sum _{i_1,i_2:s_{i_1},s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}{ϵ}_p{\xi }_{i_1}{\xi }_{i_2}{K}_2\left(\frac{d_{{\theta }_1}(x_1,{\eta }_{i_1})}{h}\right)\right.\\ {{\left.K_2\left(\frac{d_{{\theta }_2}(x_2,{\eta }_{i_2}^{\prime })}{h}\right)W_{{\mathit{s}}_{\mathbf{i}},A_n}^{\left(u\right)}\right]}^2∥}_{{\mathcal{F}}_2{\mathcal{K}}^2}\\ ⩽c_2{E}_{.\mid \mathit{S}}\left({\int }_0^{D_{nh}^{\left(U_2\right)}}{\left(logN(u,{\mathcal{F}}_{i,j},{\tilde{d}}_{nh,2}^{\prime })\right)}^{1/2}\right),\hfill \end{array}$$

(93)

where

$$\begin{array}{ccc}\hfill D_{nh}^{\left(U_2\right)}& =& \parallel {\mathbb{E}}_{ϵ}|\left[\left[L_n\right]\right]{\lambda }_n^{-2}{A}_{1,n}^{-d}{n}^{-5/2}{\varphi }_{\mathbf{x},\mathit{\theta }}^{-1}\left(h_n\right)\hfill \\ & & {\left.\sum _{{\ell }_{\mathbf{1}}\in L_n}{\left[\sum _{i_1,i_2:s_{i_1},s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}{\xi }_{i_1}{\xi }_{i_2}{K}_2\left(\frac{d_{{\theta }_1}(x_1,{\eta }_{i_1})}{h}\right)K_2\left(\frac{d_{{\theta }_2}(x_2,{\eta }_{i_2}^{\prime })}{h}\right)W_{{\mathit{s}}_{\mathbf{i}},A_n}^{\left(u\right)}\right]}^2\right|∥}_{{\mathcal{F}}_2{\mathcal{K}}^2}.\hfill \end{array}$$

and for ${\xi }_{1.}{K}_{2,1}{W}^{\prime },{\xi }_{2.}{K}_{2,2}{W}^{″}\in {\mathcal{F}}_{ij}$:

$$\begin{array}{c}{\displaystyle {\tilde{d}}_{nh,2}^{\prime }\left({\xi }_{1.}{K}_{2,1}{W}^{\prime \left(u\right)},{\xi }_{2.}{K}_{2,2}{W}^{″\left(u\right)}\right)}\hfill \\ := {\mathbb{E}}_{ϵ}\left|\left[\left[L_n\right]\right]{\lambda }_n^{-2}{A}_{1,n}^{-d}{n}^{-5/2}{\varphi }_{\mathbf{x},\mathit{\theta }}^{-1}\left(h_n\right)\sum _{{\ell }_{\mathbf{1}}\in L_n}{ϵ}_p\left[\left(\sum _{i_1,i_2:s_{i_1},s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}{\xi }_{1i_1}{\xi }_{1i_2}{K}_{2,1}\left(\frac{d_{{\theta }_1}(x_1,{\eta }_{i_1})}{h}\right)\right.\right.\right.\hfill \\ {\left.\times K_{2,1}\left(\frac{d_{{\theta }_2}(x_2,{\eta }_{i_2}^{\prime })}{h}\right)W_{{\mathit{s}}_{\mathbf{i}},A_n}^{\prime \left(u\right)}\right)}^2-\left.\left.{\left(\sum _{i_1,i_2\in H_p^{\left(U\right)}}{\xi }_{2i}{\xi }_{2j}{K}_{2,2}\left(\frac{d_{{\theta }_1}(x_1,{\eta }_{i_1})}{h}\right)K_{2,2}\left(\frac{d_{{\theta }_2}(x_2,{\eta }_{i_2}^{\prime })}{h}\right)W_{{\mathit{s}}_{\mathbf{i}},A_n}^{\prime \prime \left(u\right)}\right)}^2\right]\right|.\hfill \end{array}$$

By the fact that

$$\begin{array}{c}{\displaystyle {\mathbb{E}}_{ϵ}\left|\left[\left[L_n\right]\right]{\lambda }_n^{-2}{A}_{1,n}^{-d}{n}^{-5/2}{\varphi }_{\mathbf{x},\mathit{\theta }}^{-1}\left(h_n\right)\sum _{{\ell }_{\mathbf{1}}\in L_n}{ϵ}_p\left(\sum _{i_1,i_2:s_{i_1},s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}{\xi }_{i_1}{\xi }_{i_2}{K}_2\left(\frac{d_{{\theta }_1}(x_1,{\eta }_{i_1})}{h}\right)\right.\right.}\hfill \\ \left.{\left.K_2\left(\frac{d_{{\theta }_2}(x_2,{\eta }_{i_2}^{\prime })}{h}\right)W_{{\mathit{s}}_{\mathbf{i}},A_n}^{\left(u\right)}\right)}^2\right|\hfill \\ ⩽ A_{1,n}^{3d/2}{\lambda }_n^{-2}{n}^{-1}\left[{\left[\left[L_n\right]\right]}^{-1}{A}_{1,n}^{-2d}{\varphi }^{-2}\left(h_n\right)\sum _{{\ell }_{\mathbf{1}}\in L_n}\sum _{i_1,i_2:s_{i_1},s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\left({\xi }_{i_1}{\xi }_{i_2}{K}_2\left(\frac{d_{{\theta }_i}(x_i,{\eta }_{i_1})}{h}\right)\right.\right.\hfill \\ {\left.{\left.K_2\left(\frac{d(x_2,{\eta }_j^{\prime })}{h}\right)W_{{\mathit{s}}_{\mathbf{i}},A_n}^{\left(u\right)}\right)}^4\right]}^{1/2}.\hfill \end{array}$$

We readily infer that

$$A_{1,n}^{d3/2}{\lambda }_n^{-2}{n}^{-1}\to 0,$$

(94)

we have the convergence of (93) to zero. Recall that

$$〚L_n〛=O\left({\left(A_n/A_{3,n}\right)}^d\right)\lesssim {\left(A_n/A_{1,n}\right)}^d.$$

(II):

The same block:

$$\begin{array}{c}{\displaystyle \mathbb{P}\left(\underset{{\mathcal{F}}_m{\mathcal{K}}^m}{sup}\underset{\mathit{\theta }\in {\Theta }^m}{sup}\underset{\mathbf{x}\in {\mathcal{H}}^m}{sup}\underset{\mathbf{u}\in B^m}{sup}\right|\frac{1}{h^{2d}{\varphi }^2\left(h\right)}\sum _{{\ell }_{\mathbf{1}}\in L_n}\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\underset{i_1\ne i_2}{\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}}}\hfill \\ \left.\left.\times \prod _{j=1}^2\left\{\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)K_2\left(\frac{d_{{\theta }_j}(x_j,{\mathbf{X}}_{{\mathit{s}}_{i_j},A_n})}{h_n}\right)\right\}{W}_{{\mathit{s}}_{\mathbf{i}},A_n}\right|>\delta \right)\hfill \\ \le \mathbb{P}(\underset{{\mathcal{F}}_m{\mathcal{K}}^m}{sup}\underset{\mathit{\theta }\in {\Theta }^m}{sup}\underset{\mathbf{x}\in {\mathcal{H}}^m}{sup}\underset{\mathbf{u}\in B^m}{sup}\left|\frac{1}{n^{3/2}{h}^{2d}{\varphi }^2\left(h\right)}\sum _{{\ell }_{\mathbf{1}}\in L_n}\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\underset{i_1\ne i_2}{\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}}\right.\hfill \\ \prod _{j=1}^2\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)\left.\left[\prod _{j=1}^2{K}_2\left(\frac{d_{{\theta }_j}(x_j,{\mathbf{X}}_{{\mathit{s}}_{i_j},A_n})}{h_n}\right)-\prod _{j=1}^2{K}_2\left(\frac{d_{{\theta }_i}\left(x_i,X_{{\mathit{u}}_j}\left({\mathit{s}}_{i_j}\right)\right)}{h}\right)\right]W_{{\mathit{s}}_{\mathbf{i}},A_n}\right|>\delta )\hfill \\ +\mathbb{P}(\underset{{\mathcal{F}}_m{\mathcal{K}}^m}{sup}\underset{\mathit{\theta }\in {\Theta }^m}{sup}\underset{\mathbf{x}\in {\mathcal{H}}^m}{sup}\underset{\mathbf{u}\in B^m}{sup}\left|\frac{1}{n^{3/2}{h}^{2d}{\varphi }^2\left(h\right)}\sum _{{\ell }_{\mathbf{1}}\in L_n}\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\underset{i_1\ne i_2}{\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}}\right.\hfill \\ \prod _{j=1}^2\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)\left.\prod _{j=1}^2{K}_2\left(\frac{d_{{\theta }_i}\left(x_i,X_{{\mathit{u}}_j}\left({\mathit{s}}_{i_j}\right)\right)}{h}\right)\left[W_{{\mathit{s}}_{\mathbf{i}},A_n}-W_{{\mathit{s}}_{\mathbf{i}},A_n}^{\left(u\right)}\right]\right|>\delta )\hfill \\ +\mathbb{P}(\underset{{\mathcal{F}}_m{\mathcal{K}}^m}{sup}\underset{\mathit{\theta }\in {\Theta }^m}{sup}\underset{\mathbf{x}\in {\mathcal{H}}^m}{sup}\underset{\mathbf{u}\in B^m}{sup}\left|\frac{1}{n^{3/2}{h}^{2d}{\varphi }^2\left(h\right)}\sum _{{\ell }_{\mathbf{1}}\in L_n}\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\underset{i_1\ne i_2}{\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}}\right.\hfill \\ \prod _{j=1}^2\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)\left.\prod _{j=1}^2{K}_2\left(\frac{d_{{\theta }_i}\left(x_i,X_{{\mathit{u}}_j}\left({\mathit{s}}_{i_j}\right)\right)}{h}\right)W_{{\mathit{s}}_{\mathbf{i}},A_n}^{\left(u\right)}\right|>\delta )\hfill \\ \le \mathbb{P}(\underset{{\mathcal{F}}_m{\mathcal{K}}^m}{sup}\underset{\mathit{\theta }\in {\Theta }^m}{sup}\underset{\mathbf{x}\in {\mathcal{H}}^m}{sup}\underset{\mathbf{u}\in B^m}{sup}\left|\frac{1}{n^{3/2}{h}^{2d}{\varphi }^2\left(h\right)}\sum _{{\ell }_{\mathbf{1}}\in L_n}\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\underset{i_1\ne i_2}{\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}}\right.\hfill \\ \prod _{j=1}^2\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)\left.\left.\prod _{j=1}^2{K}_2\left(\frac{d_{{\theta }_j}(x_j,{\eta }_{i_j}}{h}\right)W_{{\mathit{s}}_{\mathbf{i}},A_n}^{\left(u\right)}\right|>\delta \right)\hfill \\ +C{\left(\frac{A_n}{A_{1,n}}\right)}^d\beta \left(A_{2,n};A_n^d\right)+o\left(1\right).\hfill \end{array}$$

Similarly to $\mathrm{I}$, we can demonstrate that both the first and the second terms in the previous inequality are of order $o\left(1\right)$. Therefore, as in the previous proof, it is sufficient to establish that

$$\begin{array}{c}\hfill {\displaystyle E_{.\mid \mathit{S}}(\parallel \frac{1}{n^{3/2}{h}^{2d}{\varphi }^2\left(h\right)}\sum _{{\ell }_{\mathbf{1}}\in L_n}\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}{\displaystyle \underset{i_1\ne i_2}{\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}}}\prod _{j=1}^2\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)}\\ {\prod _{j=1}^2{K}_2\left(\frac{d_{{\theta }_j}(x_j,{\eta }_{i_j}}{h}\right)W_{{\mathit{s}}_{\mathbf{i}},A_n}^{\left(u\right)}∥}_{{\mathcal{F}}_2{\mathcal{K}}^2})\to 0.\hfill \end{array}$$

Note that when dealing with uniformly bounded classes of functions, we obtain uniformity in $B^m\times {\mathcal{F}}_2{\mathcal{K}}^2$

$$E_{.\mid \mathbf{S}}\left(\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\underset{i_1\ne i_2}{\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}}\prod _{j=1}^2\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)\prod _{j=1}^2{K}_2\left(\frac{d_{{\theta }_j}(x_j,{\eta }_{i_j}}{h}\right)W_{{\mathit{s}}_{\mathbf{i}},A_n}^{\left(u\right)}\right)=O\left(a_n\right).$$

Hence, we need to establish that, for $\mathbf{u}\in B^m$,

$$\begin{array}{c}{\displaystyle \mathbb{E}{E}_{.\mid \mathbf{S}}(\parallel \frac{1}{n^{3/2}{h}^{2d}{\varphi }^2\left(h\right)}\sum _{{\ell }_{\mathbf{1}}\in L_n}\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}{\sum _{i_1\ne i_2}}_{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}}\hfill \\ \left[\prod _{j=1}^2\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)\prod _{j=1}^2{K}_2\left(\frac{d_{{\theta }_j}(x_j,{\eta }_{i_j}}{h}\right)W_{{\mathit{s}}_{\mathbf{i}},A_n}^{\left(u\right)}\right.\hfill \\ -\left.{\left.E_{.\mid \mathit{S}}\left(\prod _{j=1}^2\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)\prod _{j=1}^2{K}_2\left(\frac{d_{{\theta }_j}(x_j,{\eta }_{i_j}}{h}\right)W_{{\mathit{s}}_{\mathbf{i}},A_n}^{\left(u\right)}\right)\right]∥}_{{\mathcal{F}}_2{\mathcal{K}}^2}\right)\to 0\hfill \end{array}$$

(95)

As for empirical processes, to prove (95), it is enough to symmetrize and show that

$$\begin{array}{c}{\displaystyle E_{.\mid \mathbf{S}}\parallel \frac{1}{n^{3/2}{h}^{2d}{\varphi }^2\left(h\right)}\sum _{{\ell }_{\mathbf{1}}\in L_n}\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\underset{i_1\ne i_2}{\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}}{ϵ}_p\prod _{j=1}^2\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)}\hfill \\ {\prod _{j=1}^2{K}_2\left(\frac{d_{{\theta }_j}(x_j,{\eta }_{i_j}}{h}\right)W_{{\mathit{s}}_{\mathbf{i}},A_n}^{\left(u\right)}∥}_{{\mathcal{F}}_2{\mathcal{K}}^2}\to 0.\hfill \end{array}$$

Similarly to in (88), we have

$$\begin{array}{c}{E}_{.\mid \mathbf{S}}\parallel \frac{1}{n^{3/2}{h}^{d+1}\varphi \left(h\right)}\sum _{{\ell }_{\mathbf{1}}\in L_n}\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\underset{i_1\ne i_2}{\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}}{ϵ}_p\prod _{j=1}^2\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)\hfill \\ {\prod _{j=1}^2{K}_2\left(\frac{d_{{\theta }_j}(x_j,{\eta }_{i_j}}{h}\right)W_{{\mathit{s}}_{\mathbf{i}},A_n}^{\left(u\right)}∥}_{{\mathcal{F}}_2{\mathcal{K}}^2}\hfill \\ ⩽\mathbb{E}\left({\int }_0^{D_{nh}^{\left(U_3\right)}}{\left(logN\left(u,{\mathcal{F}}_{i_1,i_2},{\tilde{d}}_{nh,2}^{\left(3\right)}\right)\right)}^{1/2}du\right),\hfill \end{array}$$

where

$$\begin{array}{c}{D}_{nh}^{\left(U_3\right)}=\parallel {\mathbb{E}}_{ϵ}|\frac{1}{n^{3/2}{h}^d\varphi \left(h\right)}\sum _{{\ell }_{\mathbf{1}}\in L_n}{ϵ}_p\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\underset{i_1\ne i_2}{\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}}\prod _{j=1}^2\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)\\ {\left.\prod _{j=1}^2{K}_2\left(\frac{d_{{\theta }_j}(x_j,{\eta }_{i_j}}{h}\right)W_{{\mathit{s}}_{\mathbf{i}},A_n}^{\left(u\right)}\right|∥}_{{\mathcal{F}}_2{\mathcal{K}}^2},\end{array}$$

(96)

and the semi-metric ${\tilde{d}}_{nh,2}^{\left(3\right)}$ is defined by

$$\begin{array}{c}{\displaystyle {\tilde{d}}_{nh,2}^{\left(3\right)}\left({\xi }_{1.}{K}_{2,1}{W}^{\prime \left(u\right)},{\xi }_{2.}{K}_{2,2}{W}^{″\left(u\right)}\right)}\hfill \\ = {\mathbb{E}}_{ϵ}|\frac{1}{n^{3/2}{h}^d\varphi \left(h\right)}\sum _{{\ell }_{\mathbf{1}}\in L_n}{ϵ}_p\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\underset{i_1\ne i_2}{\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}}\left({\xi }_{1i}{\xi }_{1j}{K}_{2,1}\left(\frac{d_{{\theta }_1}(x_1,{\eta }_{i_1})}{h}\right)\right.\hfill \\ K_{2,1}\left(\frac{d_{{\theta }_2}(x_2,{\eta }_{i_2})}{h}\right)W_{{\mathit{s}}_{\mathbf{i}},A_n}^{\prime \left(u\right)}-\left.\left.{\xi }_{2i}{\xi }_{2j}{K}_{2,2}\left(\frac{d_{{\theta }_1}(x_1,{\eta }_{i_1})}{h}\right)K_{2,2}\left(\frac{d_{{\theta }_2}(x_2,{\eta }_{i_2})}{h}\right)W_{{\mathit{s}}_{\mathbf{i}},A_n}^{″\left(u\right)}\right)\right|.\hfill \end{array}$$

Given our consideration of uniformly bounded classes of functions, we obtain

$$\begin{array}{c}{\displaystyle {\mathbb{E}}_{ϵ}|n^{-3/2}h{\varphi }_{\mathbf{x},\mathit{\theta }}^{-1}\left(h_n\right)\sum _{{\ell }_{\mathbf{1}}\in L_n}{ϵ}_p\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\underset{i_1\ne i_2}{\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}}{\xi }_{i_1}{\xi }_{i_2}{K}_2\left(\frac{d_{{\theta }_1}(x_1,{\eta }_{i_1})}{h}\right)}\\ K_2\left(\frac{d_{{\theta }_2}(x_2,{\eta }_{i_2})}{h}\right)W_{{\mathit{s}}_{\mathbf{i}},A_n}^{\left(u\right)}|\\ ⩽A_{1,n}^{3d/2}{n}^{-1}h{\varphi }_{\mathbf{x},\mathit{\theta }}^{-1}\left(h_n\right)[\frac{1}{\left[\left[L_n\right]\right]A_{1,n}^2}\sum _{{\ell }_{\mathbf{1}}\in L_n}\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\underset{i_1\ne i_2}{\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}}\left({\xi }_{i_1}{\xi }_{i_2}{K}_2\left(\frac{d_{{\theta }_1}(x_1,{\eta }_{i_1})}{h}\right)\right.\\ {\left.K_2\left(\frac{d_{{\theta }_2}(x_2,{\eta }_{i_2})}{h}\right)W_{{\mathit{s}}_{\mathbf{i}},A_n}^{\left(u\right)}\right)}^2{]}^{1/2}⩽O\left(A_{1,n}^{3d/2}{n}^{-1}{\varphi }_{\mathbf{x},\mathit{\theta }}^{-1}\left(h_n\right)\right).\hfill \end{array}$$

Using the fact that $A_{1,n}^{3d/2}{n}^{-1}{\varphi }_{\mathbf{x},\mathit{\theta }}^{-1}\left(h_n\right)\to 0$, $D_{nh}^{\left(U_3\right)}\to 0$, we obtain $\mathrm{II}\to 0$ as $n\to \infty$.

(III):

Different types of blocks:

To avoid redundancy, we can directly observe that

$$\begin{array}{c}{\displaystyle \mathbb{P}\left(\underset{{\mathcal{F}}_m{\mathfrak{K}}_{\Theta }^m}{sup}\underset{\mathit{\theta }\in {\Theta }^m}{sup}\underset{\mathbf{x}\in {\mathcal{H}}^m}{sup}\underset{\mathbf{u}\in B^m}{sup}\right|\frac{1}{h^{2d}{\varphi }^2\left(h\right)}\sum _{{\ell }_{\mathbf{1}}\in L_n}\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\sum _{{\Delta }_2}\underset{ϵ\ne {ϵ}_0}{\sum _{{\ell }_{\mathbf{2}}\in L_{1,n}\cup L_{2,n}}}}\hfill \\ \left.\left.\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};ϵ)\cap R_n}\prod _{j=1}^2\left\{\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)K_2\left(\frac{d_{{\theta }_j}(x_j,{\mathbf{X}}_{{\mathit{s}}_{i_j},A_n})}{h_n}\right)\right\}{W}_{{\mathit{s}}_{\mathbf{i}},A_n}\right|>\delta \right)\hfill \\ \le \mathbb{P}\left(\underset{{\mathcal{F}}_m{\mathcal{K}}^m}{sup}\underset{\mathit{\theta }\in {\Theta }^m}{sup}\underset{\mathbf{x}\in {\mathcal{H}}^m}{sup}\underset{\mathbf{u}\in B^m}{sup}\right|\frac{1}{n^{3/2}{h}^{2d}{\varphi }^2\left(h\right)}\sum _{{\ell }_{\mathbf{1}}\in L_n}\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\sum _{{\Delta }_2}\underset{ϵ\ne {ϵ}_0}{\sum _{{\ell }_{\mathbf{2}}\in L_{1,n}\cup L_{2,n}}}\hfill \\ \sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};ϵ)\cap R_n}\left.\left.\prod _{j=1}^2\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)\prod _{j=1}^2{K}_2\left(\frac{d_{{\theta }_j}(x_j,{\eta }_{i_j}}{h}\right)W_{{\mathit{s}}_{\mathbf{i}},A_n}^{\left(u\right)}\right|>\delta \right)\hfill \\ +C{\left(\frac{A_n}{A_{1,n}}\right)}^d\beta \left(A_{2,n};A_n^d\right)+o\left(1\right).\hfill \end{array}$$

(97)

For $p=1$ and $p={\nu }_n$:

$$\begin{array}{c}{\displaystyle E_{.\mid \mathbf{S}}\parallel \frac{1}{n^{3/2}{h}^{2d}{\varphi }^2\left(h\right)}\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\sum _{{\Delta }_2}\underset{ϵ\ne {ϵ}_0}{\sum _{{\ell }_{\mathbf{2}}\in L_{1,n}\cup L_{2,n}}}\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};ϵ)\cap R_n}}\hfill \\ {\prod _{j=1}^2\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)\prod _{j=1}^2{K}_2\left(\frac{d_{{\theta }_j}(x_j,{\eta }_{i_j}}{h}\right)W_{{\mathit{s}}_{\mathbf{i}},A_n}^{\left(u\right)}∥}_{{\mathcal{F}}_2{\mathcal{K}}^2}\hfill \\ = E_{.\mid \mathit{S}}\parallel \frac{1}{n^{3/2}{h}^{2d}{\varphi }^2\left(h\right)}\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\sum _{{\ell }_2:{\displaystyle \underset{1\le i\le d}{min}}{\ell }_{2i}=3}\underset{ϵ\ne {ϵ}_0}{\sum _{{\ell }_{\mathbf{2}}\in L_{1,n}\cup L_{2,n}}}\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};ϵ)\cap R_n}\hfill \\ {\prod _{j=1}^2\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)\prod _{j=1}^2{K}_2\left(\frac{d_{{\theta }_j}(x_j,{\eta }_{i_j}}{h}\right)W_{{\mathit{s}}_{\mathbf{i}},A_n}^{\left(u\right)}∥}_{{\mathcal{F}}_2{\mathcal{K}}^2}.\hfill \end{array}$$

For $2⩽p⩽{\upsilon }_n-1$, we obtain

$$\begin{array}{c}{\displaystyle E_{.\mid \mathbf{S}}\parallel \frac{1}{n^{3/2}{h}^{2d}{\varphi }^2\left(h\right)}\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\sum _{{\Delta }_2}\underset{ϵ\ne {ϵ}_0}{\sum _{{\ell }_{\mathbf{2}}\in L_{1,n}\cup L_{2,n}}}\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};ϵ)\cap R_n}}\hfill \\ {\prod _{j=1}^2\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)\prod _{j=1}^2{K}_2\left(\frac{d_{{\theta }_j}(x_j,{\eta }_{i_j}}{h}\right)W_{{\mathit{s}}_{\mathbf{i}},A_n}^{\left(u\right)}∥}_{{\mathcal{F}}_2{\mathcal{K}}^2}\hfill \\ = E_{.\mid \mathbf{S}}\parallel \frac{1}{n^{3/2}{h}^{2d}{\varphi }^2\left(h\right)}\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\sum _{{\ell }_2:{\displaystyle \underset{1\le i\le d}{min}}{\ell }_{2i}=4}\underset{ϵ\ne {ϵ}_0}{\sum _{{\ell }_{\mathbf{2}}\in L_{1,n}\cup L_{2,n}}}\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};ϵ)\cap R_n}\hfill \\ {\prod _{j=1}^2\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)\prod _{j=1}^2{K}_2\left(\frac{d_{{\theta }_j}(x_j,{\eta }_{i_j}}{h}\right)W_{{\mathit{s}}_{\mathbf{i}},A_n}^{\left(u\right)}∥}_{{\mathcal{F}}_2{\mathcal{K}}^2}\hfill \\ ⩽ E_{.\mid \mathbf{S}}\parallel \frac{1}{n^{3/2}{h}^{2d}{\varphi }^2\left(h\right)}\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\sum _{{\ell }_2:{\displaystyle \underset{1\le i\le d}{min}}{\ell }_{2i}=3}\underset{ϵ\ne {ϵ}_0}{\sum _{{\ell }_{\mathbf{2}}\in L_{1,n}\cup L_{2,n}}}\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};ϵ)\cap R_n}\hfill \\ {\prod _{j=1}^2\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)\prod _{j=1}^2{K}_2\left(\frac{d_{{\theta }_j}(x_j,{\eta }_{i_j}}{h}\right)W_{{\mathit{s}}_{\mathbf{i}},A_n}^{\left(u\right)}∥}_{{\mathcal{F}}_2{\mathcal{K}}^2},\hfill \end{array}$$

therefore it suffices to show that

$$\begin{array}{ccc}& & E_{.\mid \mathbf{S}}\parallel \frac{1}{n^{3/2}{h}^{2d}{\varphi }^2\left(h\right)}\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\sum _{{\ell }_2:{\displaystyle \underset{1\le i\le d}{min}}{\ell }_{2i}=3}\underset{ϵ\ne {ϵ}_0}{\sum _{{\ell }_{\mathbf{2}}\in L_{1,n}\cup L_{2,n}}}\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};ϵ)\cap R_n}\hfill \\ & & {\prod _{j=1}^2\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)\prod _{j=1}^2{K}_2\left(\frac{d_{{\theta }_j}(x_j,{\eta }_{i_j}}{h}\right)W_{{\mathit{s}}_{\mathbf{i}},A_n}^{\left(u\right)}∥}_{{\mathcal{F}}_2{\mathcal{K}}^2}.\hfill \end{array}$$

(98)

Applying arguments similar to those in [77], we employ the standard symmetrization

$$\begin{array}{c}{\displaystyle E_{.\mid \mathbf{S}}\parallel \frac{〚L_n〛}{n^{3/2}{h}^{2d}{\varphi }^2\left(h\right)}\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\sum _{{\ell }_2:{\displaystyle \underset{1\le i\le d}{min}}{\ell }_{2i}=3}\underset{ϵ\ne {ϵ}_0}{\sum _{{\ell }_{\mathbf{2}}\in L_{1,n}\cup L_{2,n}}}\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};ϵ)\cap R_n}}\hfill \\ {\prod _{j=1}^2\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)\prod _{j=1}^2{K}_2\left(\frac{d_{{\theta }_j}(x_j,{\eta }_{i_j}}{h}\right)W_{{\mathit{s}}_{\mathbf{i}},A_n}^{\left(u\right)}∥}_{{\mathcal{F}}_2{\mathcal{K}}^2}\hfill \\ ⩽ 2E_{.\mid \mathbf{S}}\parallel \frac{〚L_n〛}{n^{3/2}{h}^{2d}{\varphi }^2\left(h\right)}\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\sum _{{\ell }_2:{\displaystyle \underset{1\le i\le d}{min}}{\ell }_{2i}=3}\underset{ϵ\ne {ϵ}_0}{\sum _{{\ell }_{\mathbf{2}}\in L_{1,n}\cup L_{2,n}}}\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};ϵ)\cap R_n}{ϵ}_q\hfill \\ {\prod _{j=1}^2\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)\prod _{j=1}^2{K}_2\left(\frac{d_{{\theta }_j}(x_j,{\eta }_{i_j}}{h}\right)W_{{\mathit{s}}_{\mathbf{i}},A_n}^{\left(u\right)}∥}_{{\mathcal{F}}_2{\mathcal{K}}^2}\hfill \\ = 2E_{.\mid \mathbf{S}}\{\parallel \frac{〚L_n〛}{n^{3/2}{h}^{2d}{\varphi }^2\left(h\right)}\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\sum _{{\ell }_2:{\displaystyle \underset{1\le i\le d}{min}}{\ell }_{2i}=3}\underset{ϵ\ne {ϵ}_0}{\sum _{{\ell }_{\mathbf{2}}\in L_{1,n}\cup L_{2,n}}}\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};ϵ)\cap R_n}{ϵ}_q\hfill \\ \left.{\prod _{j=1}^2\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)\prod _{j=1}^2{K}_2\left(\frac{d_{{\theta }_j}(x_j,{\eta }_{i_j}}{h}\right)W_{{\mathit{s}}_{\mathbf{i}},A_n}^{\left(u\right)}∥}_{{\mathcal{F}}_2{\mathcal{K}}^2}{1\mathrm{I}}_{\left\{D_{nh}^{\left(U_4\right)}⩽{\gamma }_n\right\}}\right\}\hfill \\ +2E_{.\mid \mathbf{S}}\{\parallel \frac{〚L_n〛}{n^{3/2}{h}^{2d}{\varphi }^2\left(h\right)}\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\sum _{{\ell }_2:{\displaystyle \underset{1\le i\le d}{min}}{\ell }_{2i}=3}\underset{ϵ\ne {ϵ}_0}{\sum _{{\ell }_{\mathbf{2}}\in L_{1,n}\cup L_{2,n}}}\underset{ϵ\ne {ϵ}_0}{\sum _{{\ell }_{\mathbf{2}}\in L_{1,n}\cup L_{2,n}}}\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};ϵ)\cap R_n}\hfill \\ \left.{{ϵ}_q\prod _{j=1}^2\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)\prod _{j=1}^2{K}_2\left(\frac{d_{{\theta }_j}(x_j,{\eta }_{i_j}}{h}\right)W_{{\mathit{s}}_{\mathbf{i}},A_n}^{\left(u\right)}∥}_{{\mathcal{F}}_2{\mathcal{K}}^2}{1\mathrm{I}}_{\left\{D_{nh}^{\left(U_4\right)}>{\gamma }_n\right\}}\right\}\hfill \\ = 2{\mathrm{III}}_1+2{\mathrm{III}}_2,\hfill \end{array}$$

(99)

where

$$\begin{array}{ccc}\hfill D_{nh}^{\left(U_4\right)}& =& \parallel \frac{〚L_n〛}{n^{3/2}{h}^{2d}{\varphi }^2\left(h\right)}[\underset{ϵ\ne {ϵ}_0}{\sum _{{\ell }_2:{\displaystyle \underset{1\le i\le d}{min}}{\ell }_{2i}=3}}\underset{ϵ\ne {ϵ}_0}{\sum _{{\ell }_{\mathbf{2}}\in L_{1,n}\cup L_{2,n}}}\left(\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};ϵ)\cap R_n}\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\right.\hfill \\ & & \prod _{j=1}^2\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right){{\left.{\left.\prod _{j=1}^2{K}_2\left(\frac{d_{{\theta }_j}(x_j,{\eta }_{i_j}}{h}\right)W_{{\mathit{s}}_{\mathbf{i}},A_n}^{\left(u\right)}\right)}^2\right]}^{1/2}∥}_{{\mathcal{F}}_2{\mathcal{K}}^2}.\hfill \end{array}$$

(100)

In a similar way as in (88), we infer that

$$\begin{array}{c}\hfill {\mathrm{III}}_1⩽c_2{\int }_0^{{\gamma }_n}{\left(logN\left(t,{\mathcal{F}}_{i_1,i_2},{\tilde{d}}_{nh,2}^{\left(4\right)}\right)\right)}^{1/2}dt,\end{array}$$

(101)

where

$$\begin{array}{c}{\displaystyle {\tilde{d}}_{nh,2}^{\left(4\right)}\left({\xi }_{1.}{K}_{2,1}{W}^{\prime \left(u\right)},{\xi }_{2.}{K}_{2,2}{W}^{″\left(u\right)}\right)}\hfill \\ := {\mathbb{E}}_{ϵ}|〚L_n〛n^{-3/2}h{\varphi }_{\mathbf{x},\mathit{\theta }}^{-1}\left(h_n\right)\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\sum _{{\ell }_2:{\displaystyle \underset{1\le i\le d}{min}}{\ell }_{2i}=3}\underset{ϵ\ne {ϵ}_0}{\sum _{{\ell }_{\mathbf{2}}\in L_{1,n}\cup L_{2,n}}}\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};ϵ)\cap R_n}\hfill \\ {ϵ}_q\left[{\xi }_{1i_1}{\xi }_{1i_2}\right.K_{2,1}\left(\frac{d_{{\theta }_1}(x_1,{\eta }_{i_1})}{h}\right)K_{2,1}\left(\frac{d_{{\theta }_2}(x_2,{\eta }_{i_2})}{h}\right)W_{{\mathit{s}}_{\mathbf{i}},A_n}^{\prime \left(u\right)}\hfill \\ -\left.{\xi }_{2i_1}{\xi }_{2i_2}{K}_{2,2}\left(\frac{d_{{\theta }_1}(x_1,{\eta }_{i_1})}{h}\right)K_{2,2}\left(\frac{d_{{\theta }_2}(x_2,{\eta }_{i_2})}{h}\right)W_{{\mathit{s}}_{\mathbf{i}},A_n}^{″\left(u\right)}\right]|.\hfill \end{array}$$

Since we have

$$\begin{array}{c}{\displaystyle {\mathbb{E}}_{ϵ}|〚L_n〛n^{3/2}{h}^{2d}{\varphi }^2\left(h\right)\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\sum _{{\ell }_2:{\displaystyle \underset{1\le i\le d}{min}}{\ell }_{2i}=3}\underset{ϵ\ne {ϵ}_0}{\sum _{{\ell }_{\mathbf{2}}\in L_{1,n}\cup L_{2,n}}}\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};ϵ)\cap R_n}{ϵ}_q{\xi }_{i_1}{\xi }_{i_2}}\hfill \\ K_2\left(\frac{d_{{\theta }_1}(x_1,{\eta }_{i_1})}{h}\right)\left.K_2\left(\frac{d_{{\theta }_2}(x_2,{\eta }_{i_2})}{h}\right)W_{{\mathit{s}}_{\mathbf{i}},A_n}^{\left(u\right)}\right|\hfill \\ ⩽ A_{1,n}^{-d/2}{A}_{2,n}^d{h}^{-d+1}\varphi \left(h\right)(\frac{1}{A_{1,n}^d{A}_{2,n}^d〚L_n〛h^{d-1}{\varphi }^4\left(h_n\right)}\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\sum _{{\ell }_2:{\displaystyle \underset{1\le i\le d}{min}}{\ell }_{2i}=3}\hfill \\ \underset{ϵ\ne {ϵ}_0}{\sum _{{\ell }_{\mathbf{2}}\in L_{1,n}\cup L_{2,n}}}\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};ϵ)\cap R_n}{\left.\left[{\xi }_{i_1}{\xi }_{i_2}{K}_2\left(\frac{d_{{\theta }_1}(x_1,{\eta }_{i_1})}{h}\right)\right.K_2\left(\frac{d_{{\theta }_2}(x_2,{\eta }_{i_2})}{h}\right)W_{{\mathit{s}}_{\mathbf{i}},A_n}^{\left(u\right)}\right]}^2{)}^{1/2},\hfill \end{array}$$

and considering the semi-metric

$$\begin{array}{c}{\displaystyle {\tilde{d}}_{nh,2}^{\left(5\right)}\left({\xi }_{1.}{K}_{2,1}{W}^{\prime \left(u\right)},{\xi }_{2.}{K}_{2,2}{W}^{″\left(u\right)}\right)}\hfill \\ := (\frac{1}{A_{1,n}^d{A}_{2,n}^d〚L_n〛h^{d-1}{\varphi }^4\left(h_n\right)}\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\sum _{{\ell }_2:{\displaystyle \underset{1\le i\le d}{min}}{\ell }_{2i}=3}\underset{ϵ\ne {ϵ}_0}{\sum _{{\ell }_{\mathbf{2}}\in L_{1,n}\cup L_{2,n}}}\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};ϵ)\cap R_n}\left[{\xi }_{1i_1}{\xi }_{1i_2}\right.\hfill \\ K_{2,1}\left(\frac{d(x_1,{\eta }_{i_1})}{h}\right)K_{2,1}\left(\frac{d(x_2,{\eta }_{i_2})}{h}\right)W_{{\mathit{s}}_{\mathbf{i}},A_n}^{\prime \left(u\right)}-{\left.{\left.{\xi }_{2i_1}{\xi }_{2i_2}{K}_{2,2}\left(\frac{d(x_1,{\eta }_{i_1})}{h}\right)K_{2,2}\left(\frac{d(x_2,{\eta }_{i_2})}{h}\right)W_{{\mathit{s}}_{\mathbf{i}},A_n}^{″\left(u\right)}\right]}^2\right)}^{1/2}.\hfill \end{array}$$

We demonstrate that the statement in (101) is bounded as follows

$$〚L_n{〛}^{1/2}{A}_{2,n}^d{n}^{-1/2}{h}^2\varphi \left(h\right){\int }_0^{〚L_n{〛}^{-1/2}{A}_{2,n}^{-d}{n}^{1/2}{h}^{2d}{\gamma }_n}{\left(logN\left(t,{\mathcal{F}}_{i_1,i_2},{\tilde{d}}_{nh,2}^{\left(5\right)}\right)\right)}^{1/2}dt,$$

by choosing ${\gamma }_n=n^{-\alpha }$ for some $\alpha >(17r-26)/\left(60r\right)$, we obtain the convergence of the preceding quantity to zero. To bind the second term on the right-hand side of (99), we can mention that

$$\begin{array}{ccc}\hfill {\mathrm{III}}_2& =& \mathbb{E}\{\parallel 〚L_n〛n^{-3/2}h{\varphi }^{-2}\left(h\right)\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\sum _{{\ell }_2:{\displaystyle \underset{1\le i\le d}{min}}{\ell }_{2i}=3}\underset{ϵ\ne {ϵ}_0}{\sum _{{\ell }_{\mathbf{2}}\in L_{1,n}\cup L_{2,n}}}\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};ϵ)\cap R_n}{ϵ}_q\hfill \\ & & \left.{\prod _{j=1}^2\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)\prod _{j=1}^2{K}_2\left(\frac{d_{{\theta }_j}(x_j,{\eta }_{i_j})}{h}\right)W_{{\mathit{s}}_{\mathbf{i}},A_n}^{\left(u\right)}∥}_{{\mathcal{F}}_2{\mathcal{K}}^2}{1\mathrm{I}}_{\left\{D_{nh}^{\left(U_4\right)}>{\gamma }_n\right\}}\right\}\hfill \\ & ⩽& A_{1,n}^{-1}{A}_{2,n}{n}^{1/2}{h}^d{\varphi }_{\mathbf{x},\mathit{\theta }}^{-1}\left(h_n\right)\mathbb{P}\{\parallel 〚L_n{〛}^2{n}^{-3}{h}^2{\varphi }^{-2}\left(h_n\right)\sum _{{\ell }_2:{\displaystyle \underset{1\le i\le d}{min}}{\ell }_{2i}=3}\underset{ϵ\ne {ϵ}_0}{\sum _{{\ell }_{\mathbf{2}}\in L_{1,n}\cup L_{2,n}}}\left(\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};ϵ)\cap R_n}\right.\hfill \\ & & {\left.\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\prod _{j=1}^2\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)\prod _{j=1}^2{K}_2\left(\frac{d_{{\theta }_j}(x_j,{\eta }_{i_j})}{h}\right)W_{{\mathit{s}}_{\mathbf{i}},A_n}^{\left(u\right)}\right)}^2{\parallel }_{{\mathcal{F}}_2{\mathcal{K}}^2}⩾{\gamma }_n^2\}.\hfill \end{array}$$

(102)

We will apply the square root method to the last expression conditionally on ${\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n$. Let $E_{ϵ\ne {ϵ}_0}$ denote the expectation with respect to $\sigma \left\{{\eta }_{i_2},ϵ\ne {ϵ}_0\right\}$. We assume that any class of functions ${\mathcal{F}}_m$ is unbounded, and its envelope function satisfies, for some $\zeta >2$,

$${\theta }_{\zeta }:=\underset{\mathbf{x}\in {\mathcal{S}}_{\mathcal{H}}^m}{sup}\mathbb{E}\left(F^{\zeta }\left(\mathbf{Y}\right)|\mathbf{X}=\mathbf{x}\right)<\infty ,$$

(103)

for $2r/(r-1)<s<\infty$, (in the notation in of Lemma 5.2 [174]).

$$\begin{array}{ccc}\hfill M_n& =& 〚L_n{〛}^{1/2}{E}_{ϵ\ne {ϵ}_0}\left(\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};ϵ)\cap R_n}\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\prod _{j=1}^2\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)\right.\hfill \\ & & {\left.\prod _{j=1}^2{K}_2\left(\frac{d_{{\theta }_j}(x_j,{\eta }_{i_j})}{h}\right)W_{{\mathit{s}}_{\mathbf{i}},A_n}^{\left(u\right)}\right)}^2,\hfill \end{array}$$

where

$$\begin{array}{ccc}\hfill x& =& {\gamma }_n^2{A}_{1,n}^{5d/2}{n}^{1/2}{h}^{md/2}{\varphi }^{-m/2}\left(h\right),\hfill \\ \hfill \rho & =& \lambda =2^{-4}{\gamma }_n{A}_{1,n}^{5d/4}{n}^{1/4}{h}^{md/4}{\varphi }^{-m/4}\left(h\right),\hfill \\ \hfill m& =& exp\left({\gamma }_n^{2}n{h}^{2d}{\varphi }^{-2}\left(h_n\right)A_{2,n}^{-2d}\right).\hfill \end{array}$$

Nevertheless, as we require $t>8M_n$ and $m\to \infty$, employing arguments akin to those in (page 69, [77]), we achieve the convergence of (101) and (102) to zero.

(IV):

Blocks of different types:

We have to prove that

$$\begin{array}{c}{\displaystyle \mathbb{P}\left(\underset{{\mathcal{F}}_m{\mathfrak{K}}_{\Theta }^m}{sup}\underset{\mathit{\theta }\in {\Theta }^m}{sup}\underset{\mathbf{x}\in {\mathcal{H}}^m}{sup}\underset{\mathbf{u}\in B^m}{sup}\right|\frac{1}{h^{2d}{\varphi }^2\left(h\right)}\sum _{{\ell }_{\mathbf{1}}\in L_n}\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\sum _{{\Delta }_1}\underset{ϵ\ne {ϵ}_0}{\sum _{{\ell }_{\mathbf{2}}\in L_{1,n}\cup L_{2,n}}}\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};ϵ)\cap R_n}}\hfill \\ \times \prod _{j=1}^2\left\{\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)K_2\left(\frac{d_{{\theta }_j}(x_j,{\mathbf{X}}_{{\mathit{s}}_{i_j},A_n})}{h_n}\right)\right\}{W}_{{\mathit{s}}_{\mathbf{i}},A_n}|>\delta )\to 0.\hfill \end{array}$$

• We have

  $$\begin{array}{ccc}& & \parallel n^{-3/2}\frac{1}{h^{2d}{\varphi }^2\left(h\right)}\sum _{{\ell }_{\mathbf{1}}\in L_n}\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\sum _{{\Delta }_1}\underset{ϵ\ne {ϵ}_0}{\sum _{{\ell }_{\mathbf{2}}\in L_{1,n}\cup L_{2,n}}}\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};ϵ)\cap R_n}.\hfill \\ & & \times \prod _{j=1}^2\left\{\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)K_2\left(\frac{d_{{\theta }_j}(x_j,{\mathbf{X}}_{{\mathit{s}}_{i_j},A_n})}{h_n}\right)\right\}{W}_{{\mathit{s}}_{\mathbf{i}},A_n}{\parallel }_{{\mathcal{F}}_2{\mathcal{K}}^2}\hfill \\ & & ⩽c_2〚L_n〛A_{1,n}^d{A}_{2,n}^d{n}^{-3/2}{h}^{-d}{\varphi }_{\mathbf{x},\mathit{\theta }}^{-1}\left(h_n\right)\to 0.\hfill \end{array}$$
  Therefore, the proof of the lemma is concluded. □
  The final stage in the proof of Proposition 1 involves employing Lemma 1 to establish the convergence of the nonlinear term to zero. □

Proof of Theorem 1

We have

$$\begin{array}{c}{\displaystyle {\widehat{r}}_n^{\left(m\right)}(\mathbf{x},\mathit{\theta },\mathbf{u};h_n)-r^{\left(m\right)}(\mathbf{x},\mathit{\theta },\mathbf{u})}\hfill \\ = \frac{1}{{\tilde{r}}_1(\mathbf{x},\mathit{\theta },\mathbf{u})}\left({\widehat{g}}_1(\mathbf{x},\mathit{\theta },\mathbf{u})+{\widehat{g}}_2(\mathbf{x},\mathit{\theta },\mathbf{u})-r^{\left(m\right)}(\phi ,\mathbf{x},\mathbf{u}){\tilde{r}}_1(\mathbf{x},\mathit{\theta },\mathbf{u})\right),\hfill \end{array}$$

where

$$\begin{array}{ccc}\hfill r^{\left(m\right)}(\mathbf{x},\mathit{\theta },\mathbf{u})& =& r^{\left(m\right)}\left(\phi ,\langle {\mathbf{x}}_1,{\theta }_1\rangle ,\dots ,\langle {\mathbf{x}}_1,{\theta }_m\rangle ,\frac{{\mathit{u}}_1}{A_n},\dots ,\frac{{\mathit{u}}_m}{A_n}\right),\hfill \\ \hfill {\tilde{r}}_1(\mathbf{x},\mathit{\theta },\mathit{u})& =& \frac{(n-m)!}{n!h^{md}{\varphi }^m\left(h\right)}\sum _{\mathbf{i}\in I_n^m}\prod _{j=1}^m\left\{\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)K_2\left(\frac{d_{{\theta }_j}(x_j,{\mathbf{X}}_{{\mathit{s}}_{i_j},A_n})}{h_n}\right)\right\},\hfill \\ \hfill {\widehat{g}}_1(\mathbf{x},\mathit{\theta },\mathit{u})& =& \frac{(n-m)!}{n!h^{md}{\varphi }^m\left(h\right)}\sum _{\mathbf{i}\in I_n^m}\prod _{j=1}^m\left\{\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)K_2\left(\frac{d_{{\theta }_j}(x_j,{\mathbf{X}}_{{\mathit{s}}_{i_j},A_n})}{h_n}\right)\right\}\prod _{j=1}^m{ϵ}_{{\mathit{s}}_{i_j},A_n},\hfill \\ \hfill {\widehat{g}}_2(\mathbf{x},\mathit{\theta },\mathbf{u})& =& \frac{(n-m)!}{n!h^{md}{\varphi }^m\left(h\right)}\sum _{\mathbf{i}\in I_n^m}\prod _{j=1}^m\left\{\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)K_2\left(\frac{d_{{\theta }_j}(x_j,{\mathbf{X}}_{{\mathit{s}}_{i_j},A_n})}{h_n}\right)\right\}\hfill \\ & & \times r^{\left(m\right)}\left(\phi ,\mathbf{x},\mathit{\theta },\frac{{\mathit{s}}_{\mathbf{i}}}{A_n}\right).\hfill \end{array}$$

The proof of this theorem is intricate and divided into the following four steps. In each step, our objective is to demonstrate that

Step 1.

$$\underset{{\mathcal{F}}_m{\mathfrak{K}}_{\Theta }^m}{sup}\underset{\mathit{\theta }\in {\Theta }^m}{sup}\underset{\mathbf{x}\in {\mathcal{H}}^m}{sup}\underset{\mathbf{u}\in B^m}{sup}\left|{\widehat{g}}_1(\mathbf{x},\mathit{\theta },\mathbf{u})\right|=O_{\mathbb{P}}\left(\sqrt{logn/n{h}^{md}{\varphi }^m\left(h\right)}\right).$$

Step 2.

$$\begin{array}{c}{\displaystyle \underset{{\mathcal{F}}_m{\mathfrak{K}}_{\Theta }^m}{sup}\underset{\mathit{\theta }\in {\Theta }^m}{sup}\underset{\mathbf{x}\in {\mathcal{H}}^m}{sup}\underset{\mathbf{u}\in B^m}{sup}|{\widehat{g}}_2(\mathbf{x},\mathit{\theta },\mathbf{u})-r^{\left(m\right)}(\mathbf{x},\mathit{\theta },\mathbf{u}){\tilde{r}}_1(\phi ,\mathbf{u},\mathbf{x};h_n)}\\ -E_{.\mid \mathit{S}}({\widehat{g}}_2(\mathbf{x},\mathit{\theta },\mathbf{u})-r^{\left(m\right)}(\mathbf{x},\mathit{\theta },\mathbf{u}){\tilde{r}}_1(\phi ,\mathbf{u},\mathbf{x};h_n))|=O_{\mathbb{P}}\left(\sqrt{logn/n{h}^{md}{\varphi }^m\left(h\right)}\right).\hfill \end{array}$$

Step 3.

Let ${\kappa }_2={\int }_{\mathbb{R}}{x}^{2}K\left(x\right)dx$.

$$\begin{array}{ccc}& & \underset{{\mathcal{F}}_m{\mathfrak{K}}_{\Theta }^m}{sup}\underset{\mathit{\theta }\in {\Theta }^m}{sup}\underset{\mathbf{x}\in {\mathcal{H}}^m}{sup}\underset{\mathbf{u}\in B^m}{sup}\left|E_{.\mid \mathbf{S}}\left({\widehat{g}}_2(\mathbf{x},\mathit{\theta },\mathbf{u})-r^{\left(m\right)}(\mathbf{x},\mathit{\theta },\mathbf{u}){\tilde{r}}_1(\phi ,\mathbf{u},\mathbf{x};h_n)\right)\right|\hfill \\ & & =O\left(\frac{1}{A_n^{dp}\varphi \left(h\right)}\right)+o\left(h^2\right),P_{P_{.\mid \mathit{S}}}-a.s.\hfill \end{array}$$

Step 4.

$$\underset{{\mathcal{F}}_m{\mathfrak{K}}_{\Theta }^m}{sup}\underset{\mathit{\theta }\in {\Theta }^m}{sup}\underset{\mathbf{x}\in {\mathcal{H}}^m}{sup}\underset{\mathbf{u}\in B^m}{sup}\left|{\tilde{r}}_1(\mathbf{x},\mathit{\theta },\mathit{u})-E_{.\mid \mathbf{S}}\left({\tilde{r}}_1(\mathbf{x},\mathit{\theta },\mathbf{u})\right)\right|=o_{P_{.\mid \mathit{S}}}\left(1\right).$$

• Step 1. is evidently a direct consequence of Proposition 1 when considering $W_{{\mathit{s}}_{\mathbf{i}},A_n}={\prod }_{j=1}^m{ϵ}_{{\mathit{s}}_{i_j},A_n}$. The same holds for Step 2., even if we replace $W_{{\mathit{s}}_{\mathbf{i}},A_n}$ with ${\widehat{g}}_2(\mathbf{x},\mathit{\theta },\mathbf{u})-r^{\left(m\right)}(\mathbf{x},\mathit{\theta },\mathbf{u}){\tilde{r}}_1(\phi ,\mathbf{u},\mathbf{x};h_n)$ and then apply Proposition 1. Now, we proceed to Step 4. Note that for $W_{{\mathit{s}}_{\mathbf{i}},A_n}\equiv 1$, the aforementioned proposition has already demonstrated that

  $$\underset{{\mathcal{F}}_m{\mathfrak{K}}_{\Theta }^m}{sup}\underset{\mathit{\theta }\in {\Theta }^m}{sup}\underset{\mathbf{x}\in {\mathcal{H}}^m}{sup}\underset{\mathbf{u}\in B^m}{sup}\left|{\tilde{r}}_1(\mathbf{x},\mathit{\theta },\mathbf{u})-E_{.\mid \mathbf{S}}\left({\tilde{r}}_1(\mathbf{x},\mathit{\theta },\mathbf{u})\right)\right|=o_{P_{.\mid \mathit{S}}}\left(1\right).$$

  Step 3. will be treated in what follows:

Let $K_0:[0,1]\to \mathbb{R}$ be a Lipschitz continuous function compactly support on $[-q{C}_1,q{C}_1]$ for some $q>1$ and such that $K_0\left(x\right)=1,\forall x\in [-C_1,C_1].$ Show that

$$E_{.\mid \mathbf{S}}\left[{\widehat{g}}_2(\mathbf{x},\mathit{\theta },\mathbf{u})-r^{\left(m\right)}(\mathbf{x},\mathit{\theta },\mathbf{u}){\tilde{r}}_1(\phi ,\mathbf{u},\mathbf{x};h_n))\right]=\sum _{i=1}^4{Q}_i(\mathbf{x},\mathit{\theta },\mathbf{u}),$$

(104)

where $Q_i$ can be defined as follows

$$Q_i(\mathbf{x},\mathit{\theta },\mathbf{u})=\frac{(n-m)!}{n!h^{md}{\varphi }^m\left(h\right)}\sum _{\mathbf{i}\in I_n^m}\left\{\prod _{j=1}^m\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)\right\}{q}_i(\mathbf{x},\mathit{\theta },\mathbf{u}),$$

(105)

such that

$$\begin{array}{ccc}\hfill q_1(\mathbf{x},\mathit{\theta },\mathbf{u})& =& E_{.\mid \mathbf{S}}\left[\prod _{j=1}^m{K}_0\left(\frac{d_{{\theta }_j}(x_j,X_{{\mathit{s}}_{i_j},A_n})}{h_n}\right)\left\{\prod _{j=1}^m{K}_2\left(\frac{d_{{\theta }_j}(x_j,X_{{\mathit{s}}_{i_j},A_n})}{h_n}\right)\right.\right.\hfill \\ & & \left.\left.-\prod _{j=1}^m{K}_2\left(\frac{d_{{\theta }_i}\left(x_i,X_{\frac{{\mathit{s}}_{i_j}}{A_n}}\left(s_{i_j}\right)\right)}{h}\right)\right\}\times \left\{r^{\left(m\right)}\left({\mathit{X}}_{{\mathit{s}}_{\mathbf{i}},A_n},\mathit{\theta },\frac{{\mathit{s}}_{\mathbf{i}}}{A_n}\right)-r^{\left(m\right)}(\mathbf{x},\mathit{\theta },\mathbf{u})\right\}\right],\hfill \\ \hfill q_2(\mathbf{x},\mathit{\theta },\mathbf{u})& =& E_{.\mid \mathbf{S}}\left[\prod _{j=1}^m\left\{K_0\left(\frac{d_{{\theta }_j}(x_j,X_{{\mathit{s}}_{i_j},A_n})}{h_n}\right)K_2\left(\frac{d_{{\theta }_i}\left(x_i,X_{\frac{{\mathit{s}}_{i_j}}{A_n}}\left(s_{i_j}\right)\right)}{h}\right)\right\}\right.\hfill \\ & & \left.\left\{r^{\left(m\right)}\left({\mathit{X}}_{{\mathit{s}}_{\mathbf{i}},A_n},\mathit{\theta },\frac{{\mathit{s}}_{\mathbf{i}}}{A_n}\right)-r^{\left(m\right)}\left({\mathit{X}}_{{\mathit{s}}_{\mathbf{i}},A_n},\mathit{\theta },{\mathit{s}}_{\mathbf{i}}/A_n\left(s_{\mathbf{i}}\right)\right)\right\}\right],\hfill \\ \hfill q_3(\mathbf{x},\mathit{\theta },\mathbf{u})& =& E_{.\mid \mathbf{S}}\left[\left\{\prod _{j=1}^m{K}_0\left(\frac{d_{{\theta }_j}(x_j,X_{{\mathit{s}}_{i_j},A_n})}{h_n}\right)-\prod _{j=1}^m{K}_0\left(\frac{d_{{\theta }_i}\left(x_i,X_{\frac{{\mathit{s}}_{i_j}}{A_n}}\left(s_{i_j}\right)\right)}{h}\right)\right\}\right.\hfill \\ & & \left.\prod _{j=1}^m{K}_2\left(\frac{d_{{\theta }_i}\left(x_i,X_{\frac{{\mathit{s}}_{i_j}}{A_n}}\left(s_{i_j}\right)\right)}{h}\right)\times \left\{r^{\left(m\right)}\left({\mathit{X}}_{{\mathit{s}}_{\mathbf{i}},A_n},\mathit{\theta },{\mathit{s}}_{\mathbf{i}}/A_n\left(s_{\mathbf{i}}\right)\right)-r^{\left(m\right)}(\mathbf{x},\mathit{\theta },\mathbf{u})\right\}\right],\hfill \\ \hfill q_4(\mathbf{x},\mathit{\theta },\mathbf{u})& =& E_{.\mid \mathbf{S}}\left[\prod _{j=1}^m{K}_2\left(\frac{d_{{\theta }_i}\left(x_i,X_{\frac{{\mathit{s}}_{i_j}}{A_n}}\left(s_{i_j}\right)\right)}{h}\right)\left\{r^{\left(m\right)}\left({\mathit{X}}_{{\mathit{s}}_{\mathbf{i}},A_n},\mathit{\theta },{\mathit{s}}_{\mathbf{i}}/A_n\left(s_{\mathbf{i}}\right)\right)-r^{\left(m\right)}(\mathbf{x},\mathit{\theta },\mathbf{u})\right\}\right].\hfill \end{array}$$

Observe that

$$\begin{array}{ccc}\hfill Q_1(\mathbf{x},\mathit{\theta },\mathbf{u})& \lesssim & \frac{(n-m)!}{n!h^{md}{\varphi }^m\left(h\right)}\sum _{\mathbf{i}\in I_n^m}\left\{\prod _{j=1}^m\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)E_{.\mid \mathbf{S}}\left[\prod _{j=1}^m{K}_0\left(\frac{d_{{\theta }_j}(x_j,X_{{\mathit{s}}_{i_j},A_n})}{h_n}\right)\right.\right.\hfill \\ & & \left|\prod _{j=1}^m{K}_2\left(\frac{d_{{\theta }_j}(x_j,X_{{\mathit{s}}_{i_j},A_n})}{h_n}\right)-\prod _{j=1}^m{K}_2\left(\frac{d_{{\theta }_i}\left(x_i,X_{\frac{{\mathit{s}}_{i_j}}{A_n}}\left(s_{i_j}\right)\right)}{h}\right)\right|\hfill \\ & & \left.\left.\times \left|r^{\left(m\right)}\left({\mathit{X}}_{{\mathit{s}}_{\mathbf{i}},A_n},\mathit{\theta },{\mathit{s}}_{\mathbf{i}}/A_n\left(s_{\mathbf{i}}\right)\right)-r^{\left(m\right)}(\mathbf{x},\mathit{\theta },\mathbf{u})\right|\right]\right\},\hfill \end{array}$$

using the properties of $r^{\left(m\right)}(\mathbf{x},\mathit{\theta },\mathbf{u})$ allow us to show that

$$\begin{array}{c}\hfill \prod _{j=1}^m{K}_0\left(\frac{d_{{\theta }_j}(x_j,{\mathbf{X}}_{{\mathit{s}}_{i_j},A_n})}{h_n}\right)\left|r^{\left(m\right)}\left({\mathit{X}}_{{\mathit{s}}_{\mathbf{i}},A_n},\mathit{\theta },{\mathit{s}}_{\mathbf{i}}/A_n\left(s_{\mathbf{i}}\right)\right)-r^{\left(m\right)}(\mathbf{x},\mathit{\theta },\mathbf{u})\right|\le C{h}^m,\end{array}$$

and

$$\begin{array}{c}{\displaystyle Q_1(\mathbf{x},\mathit{\theta },\mathbf{u})}\hfill \\ \le \frac{(n-m)!}{n!h^{md}\varphi \left(h\right)}\sum _{\mathbf{i}\in I_n^m}\left\{\prod _{j=1}^m\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)\right\}\hfill \\ \times E_{.\mid \mathit{S}}\left[C{h}^m\sum _{j=1}^m{\left|K_2\left(\frac{d_{{\theta }_j}(x_j,X_{{\mathit{s}}_{i_j},A_n})}{h_n}\right)-K_2\left(\frac{d_{{\theta }_i}\left(x_i,X_{\frac{{\mathit{s}}_{i_j}}{A_n}}\left(s_{i_j}\right)\right)}{h}\right)\right|}^p\right]\hfill \\ (\mathrm{using}\mathrm{the}\mathrm{telescoping}\mathrm{argument},\mathrm{and}\mathrm{the}\mathrm{boundedness}\mathrm{of}{K}_2(·)\hfill \\ \mathrm{for}p=min(\rho ,1)\mathrm{and}C<\infty )\hfill \\ \le \frac{(n-m)!}{n!h^{md}\varphi \left(h\right)}\sum _{\mathbf{i}\in I_n^m}\left\{\prod _{j=1}^m\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)\right\}{E}_{.\mid \mathbf{S}}\left[C{h}^m\sum _{j=1}^m{\left|\frac{C}{A_n^{d}h}{U}_{{\mathbf{s}}_{i_j},A_n}\left(\frac{{\mathit{s}}_{i_j}}{A_n}\right)\right|}^p\right]\hfill \\ \le \frac{C}{A_n^{pd}\varphi \left(h\right)h^{p-m}}\mathrm{uniformly}\mathrm{in}\mathbf{u}.\hfill \end{array}$$

In a similar way, and for

$$\mathbb{E}\left[\prod _{j=1}^m{K}_2\left(\frac{d_{{\theta }_i}\left(x_i,X_{\frac{{\mathit{s}}_{i_j}}{A_n}}\left(s_{i_j}\right)\right)}{h}\right)\right]\le C{\varphi }^{(m-1)/m}\left(h\right),$$

and since $r^{\left(m\right)}(·)$ is Lipschitz and

$$d_{\theta }\left({\mathbf{X}}_{{\mathit{s}}_{i_j},A_n},{\mathbf{X}}_{\frac{{\mathit{s}}_{i_j}}{A_n}}\left({\mathit{s}}_j\right)\right)\le \frac{C}{A_n^d}{U}_{{\mathit{s}}_{i_j},A_n}\left(\frac{{\mathit{s}}_{i_j}}{A_n}\right),$$

and the variable $U_{{\mathit{s}}_{i_j},A_n}\left(\frac{{\mathit{s}}_{i_j}}{A_n}\right)$ have finite p-th moment, we can see that

$$\begin{array}{c}{\displaystyle Q_2(\mathbf{x},\mathit{\theta },\mathbf{u})}\hfill \\ = \frac{(n-m)!}{n!h^{md}\varphi \left(h\right)}\sum _{\mathbf{i}\in I_n^m}\prod _{j=1}^m\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)\hfill \\ E_{.\mid \mathbf{S}}[\prod _{j=1}^m\left\{K_0\left(\frac{d_{{\theta }_j}(x_j,X_{{\mathit{s}}_{i_j},A_n})}{h_n}\right)K_2\left(\frac{d_{{\theta }_i}\left(x_i,X_{\frac{{\mathit{s}}_{i_j}}{A_n}}\left(s_{i_j}\right)\right)}{h}\right)\right\}\hfill \\ \left\{r^{\left(m\right)}\left(\phi ,\frac{{\mathit{s}}_{\mathbf{i}}}{A_n},{\mathit{X}}_{{\mathit{s}}_{\mathbf{i}},A_n}\right)-r^{\left(m\right)}\left(\phi ,\frac{{\mathit{s}}_{\mathbf{i}}}{A_n},X_{{\mathit{s}}_{\mathbf{i}}/A_n}\left(s_{\mathbf{i}}\right)\right)\right\}]\hfill \\ \le \frac{(n-m)!}{n!h^{md}\varphi \left(h\right)}\sum _{\mathbf{i}\in I_n^m}\left\{\prod _{j=1}^m\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)\right\}{E}_{.\mid \mathbf{S}}\left[{\varphi }^{(m-1)/m}\left(h\right){\left|\frac{C}{A_n^d}{U}_{{\mathbf{s}}_{i_j},A_n}\left(\frac{{\mathit{s}}_{i_j}}{A_n}\right)\right|}^p\right]\hfill \\ \le \frac{C}{A_n^{pd}{\varphi }^{1/m}\left(h\right)},\hfill \end{array}$$

(106)

and

$$\underset{{\mathcal{F}}_m{\mathfrak{K}}_{\Theta }^m}{sup}\underset{\mathit{\theta }\in {\Theta }^m}{sup}\underset{\mathbf{x}\in {\mathcal{H}}^m}{sup}\underset{\mathbf{u}\in I_h^m}{sup}{Q}_3(\mathbf{x},\mathit{\theta },\mathbf{u})\lesssim \frac{1}{A_n^{pd}\varphi \left(h\right)h^{p-m}}.$$

(107)

For the last term, we have

$$\begin{array}{c}{\displaystyle Q_4(\mathbf{x},\mathit{\theta },\mathbf{u})}\hfill \\ = \frac{(n-m)!}{n!h^{md}\varphi \left(h\right)}\sum _{\mathbf{i}\in I_n^m}\prod _{j=1}^m\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)\hfill \\ E_{.\mid \mathit{S}}\left[\prod _{j=1}^m{K}_2\left(\frac{d_{{\theta }_i}\left(x_i,X_{\frac{{\mathit{s}}_{i_j}}{A_n}}\left(s_{i_j}\right)\right)}{h}\right)\left\{r^{\left(m\right)}\left({\mathit{X}}_{{\mathit{s}}_{\mathbf{i}},A_n},\mathit{\theta },{\mathit{s}}_{\mathbf{i}}/A_n\left(s_{\mathbf{i}}\right)\right)-r^{\left(m\right)}(\mathbf{x},\mathit{\theta },\mathbf{u})\right\}\right].\hfill \end{array}$$

Using Lemma 2 and inequality (20) and under Assumption 1, it follows that

$$\begin{array}{c}{\displaystyle \left|Q_4(\mathbf{x},\mathit{\theta },\mathbf{u})\right|}\hfill \\ \le \frac{(n-m)!}{n!h^{md}\varphi \left(h\right)}\sum _{\mathbf{i}\in I_n^m}\prod _{j=1}^m\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)\hfill \\ E_{.\mid \mathbf{S}}\left[\prod _{j=1}^m{K}_2\left(\frac{d_{{\theta }_i}\left(x_i,X_{\frac{{\mathit{s}}_{i_j}}{A_n}}\left(s_{i_j}\right)\right)}{h}\right)\left|r^{\left(m\right)}\left({\mathit{X}}_{{\mathit{s}}_{\mathbf{i}},A_n},\mathit{\theta },{\mathit{s}}_{\mathbf{i}}/A_n\left(s_{\mathbf{i}}\right)\right)-r^{\left(m\right)}(\mathbf{x},\mathit{\theta },\mathbf{u})\right|\right]\hfill \\ \lesssim \frac{(n-m)!}{n!h^{md}\varphi \left(h\right)}\sum _{\mathbf{i}\in I_n^m}\prod _{j=1}^m\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)\hfill \\ E_{.\mid \mathbf{S}}\left[\prod _{j=1}^m{K}_2\left(\frac{d_{{\theta }_i}\left(x_i,X_{\frac{{\mathbf{s}}_{i_j}}{A_n}}\left(s_{i_j}\right)\right)}{h}\right){\left|d_{{\mathcal{H}}^m}\left({\mathbf{X}}_{{\mathit{s}}_{\mathbf{i}}/A_n}\left(s_{\mathbf{i}}\right),\mathbf{x}\right)+\parallel \mathbf{u}-\frac{{\mathit{s}}_{\mathbf{i}}}{A_n}\parallel \right|}^{\alpha }\right]\hfill \\ \lesssim \frac{(n-m)!}{n!h^{md}\varphi \left(h\right)}\sum _{\mathbf{i}\in I_n^m}\left|\prod _{j=1}^m\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)\right.\hfill \\ \left.-{\int }_0^1\cdots {\int }_0^1\frac{1}{h^m}\prod _{j=1}^m\overline{K}\left(\frac{(u_j-v_j)}{h}\right)d{v}_j\right|E_{.\mid \mathit{S}}\left[\left|\prod _{j=1}^m{K}_2\left(\frac{d_{{\theta }_i}\left(x_i,X_{\frac{{\mathit{s}}_{i_j}}{A_n}}\left(s_{i_j}\right)\right)}{h}\right)\right|\times h^{\alpha }\right]\hfill \\ +\frac{(n-m)!}{n!h^{md}\varphi \left(h\right)}\sum _{\mathbf{i}\in I_n^m}{\int }_0^1\cdots {\int }_0^1\frac{1}{h^{md}}\prod _{j=1}^m\overline{K}\left(\frac{u_j-v_j}{h}\right)d{v}_j\times E_{.\mid \mathbf{S}}\left[{\varphi }^{m-1}\left(h\right)h^{\alpha }\right]\hfill \\ \lesssim O_{P_{.\mid \mathbf{S}}}\left(h^{2\wedge \alpha }\right).\hfill \end{array}$$

(108)

Combining the results obtained for $Q_i$, $1\le i\le 4$, from Step 3, we can deduce the convergence rate for the estimator. □

Proof of Theorem 2.

Recall that

$$\begin{array}{c}{\displaystyle {\widehat{r}}_n^{\left(m\right)}(\mathbf{x},\mathit{\theta },\mathit{u};h_n)}\hfill \\ =\frac{{\displaystyle \sum _{\mathbf{i}\in I_n^m}\phi ({\mathbf{Y}}_{{\mathit{s}}_{i_1},A_n},\dots ,{\mathbf{Y}}_{{\mathit{s}}_{i_m,A_n}})\prod _{j=1}^m\left\{\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)K_2\left(\frac{d_{{\theta }_j}(x_j,{\mathbf{X}}_{{\mathit{s}}_{i_j},A_n})}{h_n}\right)\right\}}}{{\displaystyle \sum _{\mathbf{i}\in I_n^m}\prod _{j=1}^m\left\{\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{s}}_{i_j}/A_n}{h_n}\right)K_2\left(\frac{d_{{\theta }_j}(x_j,{\mathbf{X}}_{{\mathit{s}}_{i_j},A_n})}{h_n}\right)\right\}}}.\hfill \end{array}$$

For $\mathbf{x}\in {\mathcal{H}}^m,\mathbf{y}\in {\mathcal{Y}}^m,\mathit{\theta }\in \mathbf{\Theta }$, define

$$\begin{array}{ccc}\hfill G_{\phi ,\mathit{\theta },\mathbf{i}}(\mathbf{x},\mathbf{y})& :=& \frac{{\displaystyle \prod _{j=1}^m\left\{K_2\left(\frac{d_{{\theta }_j}(x_j,{\mathbf{X}}_{{\mathit{s}}_{i_j},A_n})}{h_n}\right)\right\}\phi ({\mathbf{Y}}_{{\mathit{s}}_{i_1},A_n},\dots ,{\mathbf{Y}}_{{\mathit{s}}_{i_m,A_n}})}}{{\displaystyle \mathbb{E}\prod _{j=1}^m\left\{K_2\left(\frac{d_{{\theta }_j}(x_j,{\mathbf{X}}_{{\mathit{s}}_{i_j},A_n})}{h_n}\right)\right\}}},\hfill \\ \\ \hfill \mathcal{G}& :=& \left\{G_{\phi ,\mathit{\theta },\mathbf{i}}(·,·):\phi \in {\mathcal{F}}_m,\mathbf{i}=(i_1,\dots ,i_m)\right\},\hfill \\ \hfill {\mathcal{G}}^{\left(k\right)}& :=& \left\{{\pi }_{k,m}{G}_{\phi ,\mathit{\theta },\mathbf{i}}(·,·):\phi \in {\mathcal{F}}_m\right\},\hfill \\ \hfill {\mathbb{U}}_n\left(\phi \right)& =& {\mathbb{U}}_n^{\left(m\right)}\left(G_{\phi ,\mathit{\theta },\mathbf{i}}\right):=\frac{(n-m)!}{n!}\sum _{i\in I_n^m}\prod _{j=1}^m{\xi }_{i_j}{G}_{\phi ,\mathit{\theta },\mathbf{i}}({\mathbf{X}}_i,{\mathbf{Y}}_i),\hfill \end{array}$$

and the U-empirical process is defined to be

$${\mu }_n\left(\phi \right):=\sqrt{n{h}^m{\varphi }_{\mathbf{x},\mathit{\theta }}^{1/m}\left(h\right)}\left\{{\mathbb{U}}_n\left(\phi \right)-\mathbb{E}\left({\mathbb{U}}_n\left(\phi \right)\right)\right\}.$$

Then, we have

$${\tilde{r}}_n^{\left(m\right)}(\mathbf{x},\mathit{\theta },\mathbf{u};h_n)=\frac{{\mathbb{U}}_n\left(\phi \right)}{{\mathbb{U}}_n\left(1\right)}.$$

To prove the weak convergence of our estimator, it is essential to first establish it for ${\mu }_n\left(\phi \right)$. Given that we are working with unbounded classes of functions, we need to truncate the function $G_{\phi ,\mathit{\theta },\mathbf{i}}(\mathbf{x},\mathbf{y})$. Let ${\lambda }_n=n^{1/\zeta }$ with $\zeta >2$ as in Assumption 7 (C.2), and define

$$\begin{array}{ccc}\hfill G_{\phi ,\mathit{\theta },\mathbf{i}}(\mathbf{x},\mathbf{y})& =& G_{\phi ,\mathit{\theta },\mathbf{i}}(\mathbf{x},\mathbf{y}){1\mathrm{I}}_{\left\{F\left(\mathbf{y}\right)\le {\lambda }_n\right\}}+G_{\phi ,\mathit{\theta },\mathbf{i}}(\mathbf{x},\mathbf{y}){1\mathrm{I}}_{\left\{F\left(\mathbf{y}\right)>{\lambda }_n\right\}}\hfill \\ & :=& G_{\phi ,\mathit{\theta },\mathbf{i}}^{\left(T\right)}(\mathbf{x},\mathbf{y})+G_{\phi ,\mathit{\theta },\mathbf{i}}^{\left(R\right)}(\mathbf{x},\mathbf{y}).\hfill \end{array}$$

We can write the U-statistic as follows:

$$\begin{array}{cc}\hfill {\mu }_n\left(\phi \right)=& \sqrt{n{h}^m{\varphi }_{\mathbf{x},\mathit{\theta }}^{1/m}\left(h\right)}\left\{{\mathbb{U}}_n^{\left(m\right)}\left(G_{\phi ,\mathit{\theta },\mathbf{i}}^{\left(T\right)}\right)-\mathbb{E}\left({\mathbb{U}}_n^{\left(m\right)}\left(G_{\phi ,\mathit{\theta },\mathbf{i}}^{\left(T\right)}\right)\right)\right\}\hfill \\ & +\sqrt{n{h}^m{\varphi }_{\mathbf{x},\mathit{\theta }}^{1/m}\left(h\right)}\left\{{\mathbb{U}}_n^{\left(m\right)}\left(G_{\phi ,\mathit{\theta },\mathbf{i}}^{\left(R\right)}\right)-\mathbb{E}\left({\mathbb{U}}_n^{\left(m\right)}\left(G_{\phi ,\mathit{\theta },\mathbf{i}}^{\left(R\right)}\right)\right)\right\}\hfill \\ \hfill :=& \sqrt{n{h}^m{\varphi }_{\mathbf{x},\mathit{\theta }}^{1/m}\left(h\right)}\left\{{\mathbb{U}}_n^{\left(T\right)}\left(\phi \right)-\mathbb{E}\left({\mathbb{U}}_n^{\left(T\right)}\left(\phi \right)\right)\right\}\hfill \\ \hfill & +\sqrt{n{h}^m{\varphi }_{\mathbf{x},\mathit{\theta }}^{1/m}\left(h\right)}\left\{{\mathbb{U}}_n^{\left(R\right)}\left(\phi \right)-\mathbb{E}\left({\mathbb{U}}_n^{\left(R\right)}\left(\phi \right)\right)\right\}\hfill \\ \hfill :=& {\mu }_n^{\left(T\right)}\left(\phi \right)+{\mu }_n^{\left(R\right)}\left(\phi \right).\hfill \end{array}$$

(109)

The first term represents the truncated part, while the second term corresponds to the remaining part. Our goal is to demonstrate that

1.

${\mu }_n^{\left(T\right)}\left(\phi \right)$ converges to a Gaussian process.

2.

The remainder part does not matter much, in the sense that

$${∥\sqrt{n{h}^m{\varphi }_{\mathbf{x},\mathit{\theta }}^{1/m}\left(h_n\right)}\left\{{\mathbb{U}}_n^{\left(R\right)}\left(\phi \right)-\mathbb{E}\left({\mathbb{U}}_n^{\left(R\right)}\left(\phi \right)\right)\right\}∥}_{{\mathcal{F}}_m{\mathcal{K}}^m}\stackrel{\mathbb{P}}{⟶}0.$$

For the initial step, we will employ the Hoeffding decomposition, which is akin to the previous decomposition in Section 3.1, with the distinction that we replace $W_{\mathbf{i},n}$ with $\phi \left(Y_{\mathbf{i},n}\right)$

$${\mathbb{U}}_n^{\left(T\right)}\left(\phi \right)-\mathbb{E}\left({\mathbb{U}}_n^{\left(T\right)}\left(\phi \right)\right):={\mathbb{U}}_{1,n}\left(\phi \right)+{\mathbb{U}}_{2,n}\left(\phi \right),$$

where

$$\begin{array}{ccc}\hfill {\mathbb{U}}_{1,n}\left(\phi \right)& :=& \frac{1}{n}\sum _{i=1}^n{\widehat{H}}_{1,i}(\mathbf{u},\mathbf{x},\phi ),\hfill \end{array}$$

(110)

$$\begin{array}{ccc}\hfill {\mathbb{U}}_{2,n}\left(\phi \right)& :=& \frac{(n-m)!}{\left(n\right)!}\sum _{\mathbf{i}\in I_n^m}{\xi }_{i_1}\cdots {\xi }_{i_m}{H}_{2,\mathbf{i}}\left(\mathit{z}\right).\hfill \end{array}$$

(111)

The convergence of ${\mathbb{U}}_{2,n}\left(\phi \right)$ to zero in probability has been established by Lemma 1. Therefore, our focus now is to demonstrate that ${\mathbb{U}}_{1,n}\left(\phi \right)$ converges weakly to a Gaussian process denoted as $\mathcal{G}\left(\phi \right)$. To achieve this, we will proceed with finite-dimensional convergence and equicontinuity. The finite-dimensional convergence requires that every finite set of functions $f_1,\dots ,f_q$ in $L_2$, with $\tilde{\mathbb{U}}$ being the centered form of $\mathbb{U}$, satisfies

$$\left(\sqrt{n{h}^m{\varphi }_{\mathbf{x},\mathit{\theta }}^{1/m}\left(h_n\right)}{\tilde{\mathbb{U}}}_{1,n}\left(f_1\right),\dots ,\sqrt{n{h}^m{\varphi }_{\mathbf{x},\mathit{\theta }}^{1/m}\left(h_n\right)}{\tilde{\mathbb{U}}}_{1,n}\left(f_q\right)\right)$$

(112)

converges to the corresponding finite-dimensional distributions of the process $\mathbb{G}\left(\phi \right)$. It is sufficient to show that for every fixed collection $(a_1,\dots ,a_q)\in {\mathbb{R}}^q$, we have

$$\sum _{j=1}^q{a}_j{\tilde{\mathbb{U}}}_{1,n}\left(f_j\right)\to N\left(0,v^2\right),$$

where

$$v^2=\sum _{j=1}^q{a}_j^2\mathrm{Var}\left({\tilde{\mathbb{U}}}_{1,n}\left(f_j\right)\right)+\sum _{s\ne r}{a}_s{a}_r\mathrm{Cov}\left({\tilde{\mathbb{U}}}_{1,n}\left(f_s\right),{\tilde{\mathbb{U}}}_{1,n}\left(f_r\right)\right).$$

(113)

Take

$$\Psi (·)=\sum _{j=1}^q{a}_j{f}_j(·).$$

By the linearity of $\Psi (·)$, we have to see that

$${\tilde{\mathbb{U}}}_{1,n}(\Psi ,\mathbf{i})\to \mathbb{G}(\Psi ).$$

Let

$$N=\mathbb{E}\prod _{j=1}^m\left\{K_2\left(\frac{d_{{\theta }_j}(x_j,{\mathbf{X}}_{{\mathit{s}}_{i_j},A_n})}{h_n}\right)\right\}.$$

We have

$$\begin{array}{c}\hfill {\displaystyle {\tilde{\mathbb{U}}}_{1,n}\left(h_n\right)}\\ & =& N^{-1}\times \frac{1}{n}\sum _{i=1}^n\frac{(n-m)!}{(n-1)!}\sum _{I_{n-1}^{m-1}(-i)}\sum _{\ell =1}^m{\xi }_{i_1}\cdots {\xi }_{i_{\ell -1}}{\xi }_i{\xi }_{i_{\ell }}\cdots {\xi }_{i_{m-1}}\frac{1}{{\varphi }_{\mathbf{x},\mathit{\theta }}\left(h_n\right)}{K}_2\left(\frac{d_{{\theta }_i}(x_i,X_{{\mathit{s}}_i,A_n})}{h_n}\right)\hfill \\ & & \times \int h(y_1,\dots ,y_{\ell -1},Y_i,y_{\ell },\dots ,y_{m-1})\prod _{\underset{j\ne i}{j=1}}^{m-1}\frac{1}{{\varphi }_{\mathbf{x},\mathit{\theta }}\left(h_n\right)}{K}_2\left(\frac{d_{{\theta }_j}(x_j,{\mathbf{X}}_{{\mathit{s}}_{i_j},A_n})}{h_n}\right)\hfill \\ & & \mathbb{P}(d({\nu }_1,y_1),\dots ,d({\nu }_{\ell -1},y_{\ell -1}),d({\nu }_{\ell },y_{\ell }),\dots ,d({\nu }_{m-1},y_{m-1})),\hfill \\ & :=& N^{-1}\frac{1}{n}\sum _{i=1}^n{\xi }_i\frac{1}{{\varphi }_{\mathbf{x},\mathit{\theta }}\left(h_n\right)}{K}_2\left(\frac{d_{{\theta }_i}(x_i,X_{{\mathit{s}}_i,A_n})}{h_n}\right)\tilde{h}\left(Y_{\mathbf{i}}\right).\hfill \end{array}$$

The subsequent step involves extending the Blocking techniques of Bernstein to the spatial process, with all notions defined in Section 9. Recall that $L_n=L_{1,n}\cup L_{2,n}$, and define

$${\mathbb{Z}}_{\mathit{s},A_n}(\mathbf{x},\mathit{\theta },\mathbf{u}):={\xi }_i\frac{1}{{\varphi }_{\mathbf{x},\mathit{\theta }}^{1/m}\left(h_n\right)}{K}_2\left(\frac{d_{{\theta }_i}(x_i,X_{{\mathit{s}}_i,A_n})}{h_n}\right)\tilde{h}\left(Y_{\mathbf{i}}\right),$$

(114)

and

$$\begin{array}{ccc}\hfill {\mathbb{Z}}_n(\ell ;\mathit{ϵ})& =& \sum _{i:s_i\in {\Gamma }_n(\ell ;ϵ)\cap R_n}{\mathbb{Z}}_{\mathit{s},A_n}(\mathbf{x},\mathit{\theta },\mathbf{u})\hfill \\ & =& {\left({\mathbb{Z}}_n^{\left(1\right)}(\ell ;\mathit{ϵ}),\dots ,{\mathbb{Z}}_n^{\left(p\right)}(\ell ;\mathit{ϵ})\right)}^{\prime }.\hfill \end{array}$$

(115)

Then, we have

$$\begin{array}{ccc}\hfill {\tilde{\mathbb{U}}}_{1,n}\left(h_n\right)& =& \sum _{i=1}^n{\mathbb{Z}}_{\mathit{s},A_n}(\mathbf{x},\mathit{\theta },\mathbf{u})\hfill \\ & =& \sum _{\ell \in L_n}{\mathbb{Z}}_n\left(\ell ;{\mathit{ϵ}}_0\right)+\sum _{\mathit{ϵ}\ne {\mathit{ϵ}}_0}\underset{=:{\mathbb{Z}}_{2,n}\left(\mathit{ϵ}\right)}{\underbrace{{\displaystyle \sum _{\ell \in L_{1,n}}}{\mathbb{Z}}_n(\ell ;\mathit{ϵ})}}+\sum _{ϵ\ne {ϵ}_0}\underset{=:{\mathbb{Z}}_{3,n}\left(\mathit{ϵ}\right)}{\underbrace{\sum _{\ell \in L_{2,n}}{\mathbb{Z}}_n(\ell ;\mathit{ϵ})}}\hfill \\ & =:& {\mathbb{Z}}_{1,n}+\sum _{ϵ\ne {ϵ}_0}{\mathbb{Z}}_{2,n}\left(\mathit{ϵ}\right)+\sum _{ϵ\ne {ϵ}_0}{\mathbb{Z}}_{3,n}\left(\mathit{ϵ}\right).\hfill \end{array}$$

(116)

Lemma 9 establishes that ${\mathbb{Z}}_{2,n}$ and ${\mathbb{Z}}_{3,n}$, for $ϵ\ne {ϵ}_0$, are asymptotically negligible. Addressing the variance of ${\mathbb{Z}}_{1,n}$ is straightforward. Initially, mixing conditions are employed to replace large blocks with independent random variables. Subsequently, Lyapunov’s condition for the central limit theorem is applied to the sum of independent random variables. Similar to the proof of Proposition 1 using Lemma 5, as in Equation (75), observe that

$$\underset{t>0}{sup}\left|P_{·\mid \mathbf{S}}\left({\mathbb{Z}}_{1,n}>t\right)-P_{·\mid \mathbf{S}}\left({\breve{\mathbb{Z}}}_{1,n}>t\right)\right|\le C{\left(\frac{A_n}{A_{1,n}}\right)}^d\beta \left(A_{2,n};A_n^d\right),$$

(117)

where $\left\{{\breve{\mathbb{Z}}}_n(\ell ;\mathit{ϵ}):\ell \in L_n\right\}$ denotes a sequence of independent random vectors in ${\mathbb{R}}^p$ under $P_{·\mid \mathbf{S}}$ such that

$${\breve{\mathbb{Z}}}_n(\ell ;\mathit{ϵ})\stackrel{d}{=}{\mathbb{Z}}_n(\ell ;\mathit{ϵ}),\mathrm{under}{P}_{\mid \mathbf{S}},\ell \in L_n.$$

By applying Lyapunov’s condition for the central limit theorem to the sum of independent random variables, the remaining condition of finite-dimensional convergence must be established. Next, we proceed to prove that

$$\begin{array}{c}\hfill \underset{\delta \to 0}{lim}\underset{n\to \infty }{lim}\mathbb{P}\left\{\sqrt{n{h}^m{\varphi }_{\mathbf{x},\mathit{\theta }}^{1/m}\left(h_n\right)}{∥{\tilde{\mathbb{U}}}_{1,n}(h_n,\mathbf{i})∥}_{\mathcal{FK}(\delta ,\parallel ·{\parallel }_p)}>ϵ\right\}=0,\end{array}$$

(118)

where

$$\begin{array}{c}\hfill {\displaystyle \mathcal{FK}(\delta ,\parallel ·{\parallel }_p)}\\ & & :=\left\{{\tilde{\mathbb{U}}}_{1,n}^{\prime }\left(h_n\right)-{\tilde{\mathbb{U}}}_{1,n}^{″}\left(h_n\right):∥{\tilde{\mathbb{U}}}_{1,n}^{\prime }\left(h_n\right)-{\tilde{\mathbb{U}}}_{1,n}^{″}\left(h_n\right)∥<\delta ,{\tilde{\mathbb{U}}}_{1,n}^{\prime }\left(h_n\right),{\tilde{\mathbb{U}}}_{1,n}^{″}\left(h_n\right)\in \mathcal{FK}\right\},\hfill \end{array}$$

(119)

for

$$\begin{array}{ccc}\hfill {\tilde{\mathbb{U}}}_{1,n}^{\prime }\left(h_n\right)& =& N^{-1}\frac{1}{n}\sum _{i=1}^n{\xi }_i\frac{1}{{\varphi }_{\mathbf{x},\mathit{\theta }}^{1/m}\left(h_n\right)}{K}_{2,1}\left(\frac{d_{{\theta }_i}(x_i,X_{{\mathit{s}}_i,A_n})}{h_n}\right){\tilde{h}}_1\left(Y_i\right)-\mathbb{E}\left({\mathbb{U}}_{1,n}^{\prime }\left(h_n\right)\right),\hfill \\ \hfill {\tilde{\mathbb{U}}}_{1,n}^{″}\left(h_n\right)& =& N^{-1}\frac{1}{n}\sum _{i=1}^n{\xi }_i\frac{1}{{\varphi }_{\mathbf{x},\mathit{\theta }}^{1/m}\left(h_n\right)}{K}_{2,2}\left(\frac{d_{{\theta }_i}(x_i,X_{{\mathit{s}}_i,A_n})}{h_n}\right){\tilde{h}}_2\left(Y_i\right)-\mathbb{E}\left({\mathbb{U}}_{1,n}^{″}\left(h_n\right)\right).\hfill \end{array}$$

(120)

Now, we will adapt the chaining technique found in [77], applying it to the conditional setting with the locally stationary process as in [175], but for random fields, similar to Lemma 1. We will employ the same strategy as in Lemma 1 to transition from the sequence of locally stationary random variables to the stationary one. Let ${\zeta }_i=({\eta }_i,{\varsigma }_i)$ denote the independent block sequences:

$$\begin{array}{c}\hfill {\displaystyle \mathbb{P}\left\{{∥{\left(n{\varphi }_{\mathbf{x},\mathit{\theta }}^{1/m}\left(h_n\right)\right)}^{-1/2}{h}^{m/2}{N}^{-1}\sum _{i=1}^n\left({\xi }_i{K}_2\left(\frac{d_{{\theta }_i}(x_i,X_i)}{h}\right)\tilde{h}\left(Y_i\right)-\mathbb{E}\left({\mathbb{U}}_{1,n}\left(h_n\right)\right)\right)∥}_{{\mathcal{FK}}_{(b,\parallel ·{\parallel }_p)}}>ϵ\right\}}\\ & \le & 2\mathbb{P}\left\{∥{\left(n{\varphi }_{\mathbf{x},\mathit{\theta }}^{1/m}\left(h_n\right)\right)}^{-1/2}{h}^{m/2}{N}^{-1}\sum _{\ell \in L_n}\sum _{i:s_i\in {\Gamma }_n(\ell ;{ϵ}_0)\cap R_n}\left({\xi }_i{K}_2\left(\frac{d_{{\theta }_i}(x_i,{\eta }_i)}{h}\right)\tilde{h}\left({\varsigma }_i\right)\right.\right.\hfill \\ & & -\mathbb{E}\left({\mathbb{U}}_{1,n}\left(h_n\right)\right)\phantom{\frac{1}{\sqrt{n{\varphi }_{\mathbf{x},\theta }\left(h_n\right)}}\sum _{p=1}^{{\upsilon }_n}\sum _{i\in H_p^{\left(U\right)}}\sum _{q:|q-p|⩾2}^{{\upsilon }_n}}){\parallel }_{{\mathcal{FK}}_{(b,\parallel ·{\parallel }_p)}}>{ϵ}^{\prime }\}+C{\left(\frac{A_n}{A_{1,n}}\right)}^d\beta \left(A_{2,n};A_n^d\right)+o\left(1\right).\hfill \end{array}$$

(121)

Exploiting Condition (E2) in Assumption 6, we deduce that $\beta \left(A_{2,n};A_n^d\right)⟶0$ as $n\to 0$. Consequently, it is straightforward to place the first phrase on the right-hand side of (121). Since the blocks are independent, we symmetrize using a sequence ${\left\{{ϵ}_j\right\}}_{j\in {\mathbb{N}}^{*}}$ of i.i.d. Rademacher variables, i.e., random variables with

$$\mathbb{P}({ϵ}_j=1)=\mathbb{P}({ϵ}_j=-1)=1/2.$$

It is crucial to note that the sequence ${\left\{{ϵ}_j\right\}}_{j\in {\mathbb{N}}^{*}}$ is independent of the sequence ${\left\{{\xi }_i=({\varsigma }_i,{\zeta }_i)\right\}}_{i\in {\mathbb{N}}^{*}}$. Therefore, it remains to be established that, for all $ϵ>0$ and $\delta \to 0$,

$$\begin{array}{ccc}\hfill & & \underset{\delta \to 0}{lim}\underset{n\to \infty }{lim}\mathbb{P}\left\{∥{\left(n{\varphi }_{\mathbf{x},\mathit{\theta }}^{1/m}\left(h_n\right)\right)}^{-1/2}{h}^{m/2}{N}^{-1}\sum _{\ell \in L_n}\sum _{i:s_i\in {\Gamma }_n(\ell ;{ϵ}_0)\cap R_n}\left({\xi }_i{K}_2\left(\frac{d_{{\theta }_i}(x_i,{\eta }_i)}{h}\right)\tilde{h}\left({\varsigma }_i\right)\right.\right.\hfill \\ & & -\mathbb{E}\left({\mathbb{U}}_{1,n}(h_n,\mathbf{i})\right)\phantom{\frac{1}{\sqrt{n{\varphi }_{\mathbf{x},\theta }^{1/m}\left(h_n\right)}}\sum _{p=1}^{{\upsilon }_n}\sum _{i\in H_p^{\left(U\right)}}\sum _{q:|q-p|⩾2}^{{\upsilon }_n}}){\parallel }_{{\mathcal{FK}}_{(b,\parallel ·{\parallel }_p)}}>ϵ\}<\delta .\hfill \end{array}$$

(122)

Define the semi-norm

$$\begin{array}{ccc}\hfill {\tilde{d}}_{n\varphi ,2}& :=& ({\left(n{\varphi }_{\mathbf{x},\mathit{\theta }}^{1/m}\left(h_n\right)\right)}^{-1/2}{h}^{m/2}{N}^{-1}\sum _{\ell \in L_n}\sum _{i:s_i\in {\Gamma }_n(\ell ;{ϵ}_0)\cap R_n}\left|\left({\xi }_i{K}_{2,1}\left(\frac{d_{{\theta }_i}(x_i,{\eta }_i)}{h}\right){\tilde{h}}_1\left({\varsigma }_i\right)\right.\right.\hfill \\ & & \left.-\mathbb{E}\left({{\mathbb{U}}^{\prime }}_{1,n}(h_n,\mathbf{i})\right)\right){\left.{\left.-\left({\xi }_i{K}_{2,2}\left(\frac{d_{{\theta }_i}(x_i,{\eta }_i)}{h}\right){\tilde{h}}_2\left({\varsigma }_i\right)-\mathbb{E}\left({{\mathbb{U}}^{″}}_{1,n}(h_n,\mathbf{i})\right)\right)\right|}^2\right)}^{1/2},\hfill \end{array}$$

and the covering number defined for any class of functions $\mathcal{E}$ by

$${\tilde{N}}_{n\varphi ,2}(u,\mathcal{E}):=N_{n\varphi ,2}(u,\mathcal{E},{\tilde{d}}_{n\varphi ,2}).$$

Given the aforementioned independence, we can bound (122), with more details provided in [78]. Following a similar approach as in [78] and previously in [77], due to the independence between the blocks and Assumption 7 (C3), and by applying ([174] Lemma 5.2), we achieve equicontinuity, leading to weak convergence. Now, our objective is to demonstrate that for all $ϵ>0$ and $\delta \to 0$,

$$\mathbb{P}\left\{{∥{\mu }_n^{\left(R\right)}(\phi ,\mathbf{t})∥}_{{\mathcal{F}}_m{\mathfrak{K}}_{\Theta }^m}>\lambda \right\}\to 0asn\to \infty .$$

For clarity, let us focus our discussion on the case where $m=2$. Using the same notation as in Lemma 1, we can decompose it as follows:

$$\begin{array}{ccc}\hfill {\mu }_n^{\left(R\right)}(\phi ,\mathbf{i})& =& \sqrt{n{h}^{m+d}{\varphi }_{\mathbf{x},\mathit{\theta }}^{1/m}\left(h_n\right)}\left\{{\mathbb{U}}_n^{\left(R\right)}(\phi ,\mathbf{i})-\mathbb{E}\left({\mathbb{U}}_n^{\left(R\right)}(\phi ,\mathbf{i})\right)\right\}\hfill \\ & =& \frac{\sqrt{n{h}^{m+d}{\varphi }_{\mathbf{x},\mathit{\theta }}^{1/m}\left(h_n\right)}}{n(n-1)}\sum _{i_1\ne i_2}^n{\xi }_{i_1}{\xi }_{i_2}\left\{G_{\phi ,\mathbf{t}}^{\left(R\right)}\left(\left((X_{i_1},X_{i_2}),(Y_{i_1},Y_{i_2})\right)\right)\right.\hfill \\ & & \left.-\mathbb{E}\left[G_{\phi ,\mathit{\theta },\mathbf{i}}^{\left(R\right)}\left((X_{i_1},X_{i_2}),(Y_{i_1},Y_{i_2})\right)\right]\right\}\hfill \\ & ⩽& \frac{1}{\sqrt{n{h}^{m+d}{\varphi }_{\mathbf{x},\mathit{\theta }}^{1/m}\left(h_n\right)}}\sum _{{\ell }_{\mathbf{1}}\ne {\ell }_{\mathbf{2}}\in L_n}\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};{ϵ}_0)\cap R_n}\hfill \\ & & \left.{\varphi }_{\mathbf{x},\mathit{\theta }}\left(h_n\right){\xi }_{i_1}{\xi }_{i_2}\left\{G_{\phi ,\mathit{\theta },\mathbf{i}}^{\left(R\right)}\left(\left(X_i,X_j),(Y_i,Y_j\right)\right)\right.-\mathbb{E}\left[G_{\phi ,\mathit{\theta },\mathbf{i}}^{\left(R\right)}\left((X_{i_1},X_{i_2}),(Y_{i_1},Y_{i_2})\right)\right]\right\}\hfill \\ & & +\frac{1}{\sqrt{n{h}^{m+d}{\varphi }_{\mathbf{x},\mathit{\theta }}^{1/m}\left(h_n\right)}}\sum _{{\ell }_{\mathbf{1}}\in L_n}\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\underset{i_1\ne i_2}{\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}}\hfill \\ & & \left.{\varphi }_{\mathbf{x},\mathit{\theta }}\left(h_n\right){\xi }_{i_1}{\xi }_{i_2}\left\{G_{\phi ,\mathit{\theta },\mathbf{i}}^{\left(R\right)}\left(\left(X_i,X_j),(Y_i,Y_j\right)\right)\right.-\mathbb{E}\left[G_{\phi ,\mathit{\theta },\mathbf{i}}^{\left(R\right)}\left((X_{i_1},X_{i_2}),(Y_{i_1},Y_{i_2})\right)\right]\right\}\hfill \\ & & +2\frac{1}{\sqrt{n{h}^{m+d}{\varphi }_{\mathbf{x},\mathit{\theta }}\left(h_n\right)}}\sum _{{\ell }_{\mathbf{1}}\in L_n}\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\sum _{{\Delta }_2}\underset{ϵ\ne {ϵ}_0}{\sum _{{\ell }_{\mathbf{2}}\in L_{1,n}\cup L_{2,n}}}\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};ϵ)\cap R_n}\hfill \\ & & \left.{\varphi }_{\mathbf{x},\mathit{\theta }}\left(h_n\right){\xi }_{i_1}{\xi }_{i_2}\left\{G_{\phi ,\theta ,\mathbf{i}}^{\left(R\right)}\left(\left(X_i,X_j),(Y_i,Y_j\right)\right)\right.-\mathbb{E}\left[G_{\phi ,\theta ,\mathbf{i}}^{\left(R\right)}\left((X_{i_1},X_{i_2}),(Y_{i_1},Y_{i_2})\right)\right]\right\}\hfill \\ \hfill \\ & & +2\frac{1}{\sqrt{n{h}^{m+d}{\varphi }_{\mathbf{x},\mathit{\theta }}^{1/m}\left(h_n\right)}}\sum _{{\ell }_{\mathbf{1}}\in L_n}\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\sum _{{\Delta }_1}\underset{ϵ\ne {ϵ}_0}{\sum _{{\ell }_{\mathbf{2}}\in L_{1,n}\cup L_{2,n}}}\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};ϵ)\cap R_n}\hfill \\ & & {\varphi }_{\mathbf{x},\mathit{\theta }}\left(h_n\right){\xi }_{i_1}{\xi }_{i_2}\left\{G_{\phi ,\mathbf{t}}^{\left(R\right)}\left(\left(X_i,X_j),(Y_i,Y_j\right)\right)\right.\left.-\mathbb{E}\left[G_{\phi ,\mathbf{t}}^{\left(R\right)}\left((X_{i_1},X_{i_2}),(Y_{i_1},Y_{i_2})\right)\right]\right\}\hfill \\ \hfill \\ & & +\frac{1}{\sqrt{n{h}^{m+d}{\varphi }_{\mathbf{x},\mathit{\theta }}^{1/m}\left(h_n\right)}}\underset{ϵ\ne {ϵ}_0}{\sum _{{\ell }_{\mathbf{1}}\ne {\ell }_{\mathbf{2}}\in L_{1,n}\cup L_{2,n}}}\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};ϵ)\cap R_n}\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};ϵ)\cap R_n}\hfill \\ & & \left.{\varphi }_{\mathbf{x},\mathit{\theta }}\left(h_n\right){\xi }_{i_1}{\xi }_{i_2}\left\{G_{\phi ,\mathbf{t}}^{\left(R\right)}\left(\left(X_i,X_j),(Y_i,Y_j\right)\right)\right.-\mathbb{E}\left[G_{\phi ,\mathbf{t}}^{\left(R\right)}\left((X_{i_1},X_{i_2}),(Y_{i_1},Y_{i_2})\right)\right]\right\}\hfill \\ & & +\frac{1}{\sqrt{n{h}^{m+d}{\varphi }_{\mathbf{x},\mathit{\theta }}^{1/m}\left(h_n\right)}}\underset{ϵ\ne {ϵ}_0}{\sum _{{\ell }_{\mathbf{1}}\in L_{1,n}\cup L_{2,n}}}\sum _{i_1<i_2:s_{i_1},s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{1}};ϵ)\cap R_n}\hfill \\ & & \left.{\varphi }_{\mathbf{x},\mathit{\theta }}\left(h_n\right){\xi }_{i_1}{\xi }_{i_2}\left\{G_{\phi ,\mathbf{t}}^{\left(R\right)}\left(\left(X_i,X_j),(Y_i,Y_j\right)\right)\right.-\mathbb{E}\left[G_{\phi ,\mathbf{t}}^{\left(R\right)}\left((X_{i_1},X_{i_2}),(Y_{i_1},Y_{i_2})\right)\right]\right\}\hfill \\ & =:& {\mathrm{I}}^{\prime }+{\mathrm{II}}^{\prime }+{\mathrm{III}}^{\prime }+{\mathrm{IV}}^{\prime }+{\mathrm{V}}^{\prime }+{\mathrm{VI}}^{\prime }.\hfill \end{array}$$

We will employ blocking arguments to analyze the resulting terms. Let us begin by examining the first term, ${\mathrm{I}}^{\prime }$. We obtain

$$\begin{array}{c}\hfill {\displaystyle \mathbb{P}\left\{∥\frac{1}{\sqrt{n{\varphi }_{\mathbf{x},\mathit{\theta }}^{1/m}\left(h_n\right)}}\sum _{{\ell }_{\mathbf{1}}\ne {\ell }_{\mathbf{2}}\in L_n}\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};{ϵ}_0)\cap R_n}{\varphi }_{\mathbf{x},\mathit{\theta }}\left(h_n\right){\xi }_{i_1}{\xi }_{i_2}\left\{G_{\phi ,\mathit{\theta },\mathbf{i}}^{\left(R\right)}\left(\left(X_{i_1},X_{i_2}),(Y_{i_1},Y_{i_2}\right)\right)\right.\right.}\\ & & {\left.-\mathbb{E}\left[G_{\phi ,\mathbf{t}}^{\left(R\right)}\left((X_{i_1},X_{i_2}),(Y_{i_1},Y_{i_2})\right)\right]\right\}∥}_{{\mathcal{F}}_2{\mathcal{K}}^2}>\delta \}\hfill \\ & ⩽& \mathbb{P}\left\{∥\frac{1}{\sqrt{n{\varphi }_{\mathbf{x},\mathit{\theta }}^{1/m}\left(h_n\right)}}\sum _{{\ell }_{\mathbf{1}}\ne {\ell }_{\mathbf{2}}\in L_n}\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};{ϵ}_0)\cap R_n}{\varphi }_{\mathbf{x},\mathit{\theta }}\left(h_n\right){\xi }_{i_1}{\xi }_{i_2}\right.\hfill \\ & & \left\{G_{\phi ,\theta ,\mathbf{i}}^{\left(R\right)}\left(\left({\varsigma }_{i_1},{\varsigma }_{i_2}),({\zeta }_{i_1},{\zeta }_{i_2}\right)\right)\right.{\left.-\mathbb{E}\left[G_{\phi ,\mathit{\theta },\mathbf{i}}^{\left(R\right)}\left(({\varsigma }_{i_1},{\varsigma }_{i_2}),({\zeta }_{i_1},{\zeta }_{i_2})\right)\right]\right\}∥}_{{\mathcal{F}}_2{\mathcal{K}}^2}>\delta \}\hfill \\ & & +C{\left(\frac{A_n}{A_{1,n}}\right)}^d\beta \left(A_{2,n};A_n^d\right).\hfill \end{array}$$

Recall that for all $\phi \in {\mathcal{F}}_m,$ and

$$\mathbf{x}\in {\mathcal{H}}^2,\mathbf{y}\in {\mathcal{Y}}^2:{1\mathrm{I}}_{\left\{d_{{\theta }_i}\left(\mathbf{x},X_{i,n}\right)⩽h\right\}}F\left(\mathbf{y}\right)⩾\phi \left(\mathbf{y}\right)K_2\left(\frac{d_{{\theta }_i}(x_i,X_{{\mathit{s}}_i,A_n})}{h_n}\right).$$

Hence, by the symmetry of $F(·)$,

$$\begin{array}{ccc}\hfill & & ∥\frac{1}{\sqrt{n{\varphi }_{\mathbf{x},\mathit{\theta }}^{1/m}\left(h_n\right)}}\sum _{{\ell }_{\mathbf{1}}\ne {\ell }_{\mathbf{2}}\in L_n}\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};{ϵ}_0)\cap R_n}{\varphi }_{\mathbf{x},\mathit{\theta }}\left(h_n\right){\xi }_{i_1}{\xi }_{i_2}\left\{G_{\phi ,\mathit{\theta },\mathbf{i}}^{\left(R\right)}\left(\left({\varsigma }_{i_1},{\varsigma }_{i_2}),({\zeta }_{i_1},{\zeta }_{i_2}\right)\right)\right.\hfill \\ & & \left.-\mathbb{E}\left[G_{\phi ,\mathbf{t}}^{\left(R\right)}\left(({\varsigma }_{i_1},{\varsigma }_{i_2}),({\zeta }_{i_1},{\zeta }_{i_2})\right)\right]\right\}{\parallel }_{{\mathcal{F}}_2{\mathcal{K}}^2}\hfill \\ & & \lesssim \left|\frac{1}{\sqrt{n{\varphi }_{\mathbf{x},\mathit{\theta }}^{1/m}\left(h_n\right)}}\sum _{{\ell }_{\mathbf{1}}\ne {\ell }_{\mathbf{2}}\in L_n}\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};{ϵ}_0)\cap R_n}{\varphi }_{\mathbf{x},\mathit{\theta }}\left(h_n\right){\xi }_{i_1}{\xi }_{i_2}\left\{F({\zeta }_i,{\zeta }_j){1\mathrm{I}}_{\left\{F>{\lambda }_n\right\}}\right.\right.\hfill \\ & & \left.\left.-\mathbb{E}\left[F({\zeta }_i,{\zeta }_j){1\mathrm{I}}_{\left\{F>{\lambda }_n\right\}}\right]\right\}\phantom{\frac{1}{\sqrt{n{\varphi }_{\mathbf{x},\theta }\left(h_n\right)}}\sum _{p\ne q}^{{\upsilon }_n}\sum _{i\in H_p^{\left(U\right)}}\sum _{j\in H_q^{\left(U\right)}}}\right|.\hfill \end{array}$$

(123)

We will employ Chebyshev’s inequality, Hoeffding’s trick, and inequalities, respectively, to obtain

$$\begin{array}{c}{\displaystyle \mathbb{P}\left\{\left|\frac{1}{\sqrt{n{\varphi }_{\mathbf{x},\mathit{\theta }}^{1/m}\left(h_n\right)}}\sum _{{\ell }_{\mathbf{1}}\ne {\ell }_{\mathbf{2}}\in L_n}\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};{ϵ}_0)\cap R_n}{\varphi }_{\mathbf{x},\mathit{\theta }}\left(h_n\right){\xi }_{i_1}{\xi }_{i_2}\left\{F({\zeta }_i,{\zeta }_j){1\mathrm{I}}_{\left\{F>{\lambda }_n\right\}}\right.\right.\right.}\hfill \\ \left.-\mathbb{E}\left[F({\zeta }_i,{\zeta }_j){1\mathrm{I}}_{\left\{F>{\lambda }_n\right\}}\right]\right\}\phantom{\frac{1}{\sqrt{n{\varphi }_{\mathbf{x},\theta }\left(h_n\right)}}\sum _{p\ne q}^{{\upsilon }_n}\sum _{i\in H_p^{\left(U\right)}}\sum _{j\in H_q^{\left(U\right)}}}|>\delta \}\hfill \\ \lesssim {\delta }^{-2}{n}^{-1}{\varphi }_{\mathbf{x},\mathit{\theta }}^{-1/m}\left(h_n\right)Var\left(\sum _{{\ell }_{\mathbf{1}}\ne {\ell }_{\mathbf{2}}\in L_n}\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};{ϵ}_0)\cap R_n}{\varphi }_{\mathbf{x},\mathit{\theta }}\left(h_n\right){\xi }_{i_1}{\xi }_{i_2}F({\zeta }_i,{\zeta }_j){1\mathrm{I}}_{\left\{F>{\lambda }_n\right\}}\right)\hfill \\ \lesssim c_2〚L_n〛{\delta }^{-2}{n}^{-1}{\varphi }_{\mathbf{x},\mathit{\theta }}^{-1/m}\left(h_n\right)Var\left(\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};{ϵ}_0)\cap R_n}{\varphi }_{\mathbf{x},\mathit{\theta }}\left(h_n\right){\xi }_{i_1}{\xi }_{i_2}F({\zeta }_i,{\zeta }_j^{\prime }){1\mathrm{I}}_{\left\{F>{\lambda }_n\right\}}\right)\hfill \\ \lesssim 2c_2〚L_n〛{\delta }^{-2}{n}^{-2}{\varphi }_{\mathbf{x},\mathit{\theta }}^{-1/m}\left(h_n\right)\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};{ϵ}_0)\cap R_n}{\varphi }_{\mathbf{x},\mathit{\theta }}\left(h_n\right){\xi }_{i_1}{\xi }_{i_2}\mathbb{E}\left[{\left(F({\zeta }_1,{\zeta }_2)\right)}^2{1\mathrm{I}}_{\left\{F>{\lambda }_n\right\}}\right].\hfill \end{array}$$

Under Assumption 7 (iii), we have for each $\lambda >0$

$$\begin{array}{c}{\displaystyle c_2〚L_n〛{\delta }^{-2}{n}^{-2}{\varphi }_{\mathbf{x},\mathcal{\theta }}^{-1/m}\left(h_n\right)\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};{ϵ}_0)\cap R_n}{\varphi }_{\mathbf{x},\mathcal{\theta }}\left(h_n\right){\xi }_{i_1}{\xi }_{i_2}\mathbb{E}\left[{\left(F({\zeta }_1,{\zeta }_2)\right)}^2{1\mathrm{I}}_{\left\{F>{\lambda }_n\right\}}\right]}\hfill \\ =c_2〚L_n〛{\delta }^{-2}{n}^{-2}{\varphi }_{\mathbf{x},\mathcal{\theta }}^{-1/m}\left(h_n\right)\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};{ϵ}_0)\cap R_n}{\varphi }_{\mathbf{x},\mathcal{\theta }}\left(h_n\right){\xi }_{i_1}{\xi }_{i_2}\hfill \\ \times {\int }_0^{\infty }\mathbb{P}\left\{{\left(F({\zeta }_1,{\zeta }_2)\right)}^2{1\mathrm{I}}_{\left\{F>{\lambda }_n\right\}}⩾t\right\}dt\hfill \\ =c_2〚L_n〛{\delta }^{-2}{n}^{-2}{\varphi }_{\mathbf{x},\mathcal{\theta }}^{-1/m}\left(h_n\right)\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};{ϵ}_0)\cap R_n}{\varphi }_{\mathbf{x},\mathcal{\theta }}\left(h_n\right){\xi }_{i_1}{\xi }_{i_2}{\int }_0^{{\lambda }_n}\mathbb{P}\left\{F>{\lambda }_n\right\}dt\hfill \\ +c_2〚L_n〛{\delta }^{-2}{n}^{-2}{\varphi }_{\mathbf{x},\mathcal{\theta }}^{-1/m}\left(h_n\right)\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};{ϵ}_0)\cap R_n}{\varphi }_{\mathbf{x},\mathcal{\theta }}\left(h_n\right){\xi }_{i_1}{\xi }_{i_2}{\int }_{{\lambda }_n}^{\infty }\mathbb{P}\left\{{\left(F\right)}^2>t\right\}dt,\hfill \end{array}$$

converging to 0 as $n\to \infty$. Terms ${\mathrm{II}}^{\prime },{\mathrm{V}}^{\prime }$, and ${\mathrm{VI}}^{\prime }$ will be handled similarly to the last term. However, ${\mathrm{II}}^{\prime }$ and ${\mathrm{VI}}^{\prime }$ diverge because the variables ${\{{\zeta }_i,{\zeta }_j\}}_{ϵ={ϵ}_0}$ (or ${\{{\zeta }_i,{\zeta }_j\}}_{ϵ\ne {ϵ}_0}$ for ${\mathrm{VI}}^{\prime }$) belong to the same blocks. Term ${\mathrm{IV}}^{\prime }$ can be deduced from the study of terms ${\mathrm{I}}^{\prime }$ and ${\mathrm{III}}^{\prime }$. Considering the term ${\mathrm{III}}^{\prime }$, we have

$$\begin{array}{c}{\displaystyle \mathbb{P}\left\{∥\frac{1}{\sqrt{n{\varphi }_{\mathbf{x},\mathcal{\theta }}^{1/m}\left(h_n\right)}}\sum _{{\ell }_{\mathbf{1}}\in L_n}\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\sum _{{\Delta }_2}\underset{ϵ\ne {ϵ}_0}{\sum _{{\ell }_{\mathbf{2}}\in L_{1,n}\cup L_{2,n}}}\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};ϵ)\cap R_n}{\varphi }_{\mathbf{x},\mathcal{\theta }}\left(h_n\right){\xi }_{i_1}{\xi }_{i_2}\right.}\hfill \\ \left\{G_{\phi ,\mathcal{\theta },\mathbf{i}}^{\left(R\right)}\left(\left(X_i,X_j),(Y_i,Y_j\right)\right)\right.\left.{\left.-\mathbb{E}\left[G_{\phi ,\mathcal{\theta },\mathbf{i}}^{\left(R\right)}\left((X_{i_1},X_{i_2}),(Y_{i_1},Y_{i_2})\right)\right]\right\}\phantom{\frac{1}{\sqrt{n{\varphi }_{\mathbf{x},\theta }\left(h_n\right)}}\sum _{p=1}^{{\upsilon }_n}\sum _{i\in H_p^{\left(U\right)}}\sum _{q:|q-p|⩾2}^{{\upsilon }_n}}∥}_{{\mathcal{F}}_2{\mathcal{K}}^2}>\delta \right\}\hfill \\ ⩽\mathbb{P}\left\{∥\frac{1}{\sqrt{n{\varphi }_{\mathbf{x},\mathcal{\theta }}^{1/m}\left(h_n\right)}}\sum _{{\ell }_{\mathbf{1}}\in L_n}\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\sum _{{\Delta }_2}\underset{ϵ\ne {ϵ}_0}{\sum _{{\ell }_{\mathbf{2}}\in L_{1,n}\cup L_{2,n}}}\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};ϵ)\cap R_n}{\varphi }_{\mathbf{x},\mathcal{\theta }}\left(h_n\right){\xi }_{i_1}{\xi }_{i_2}\right.\hfill \\ \left.{\left.\left\{G_{\phi ,\mathcal{\theta },\mathbf{i}}^{\left(R\right)}\left(\left({\varsigma }_{i_1},{\varsigma }_{i_2}),({\zeta }_{i_1},{\zeta }_{i_2}\right)\right)\right.-\mathbb{E}\left[G_{\phi ,\mathcal{\theta },\mathbf{i}}^{\left(R\right)}\left(({\varsigma }_{i_1},{\varsigma }_{i_2}),({\zeta }_{i_1},{\zeta }_{i_2})\right)\right]\right\}\phantom{\frac{1}{\sqrt{n{\varphi }_{\mathbf{x},\theta }\left(h_n\right)}}\sum _{p=1}^{{\upsilon }_n}\sum _{i\in H_p^{\left(U\right)}}\sum _{q:|q-p|⩾2}^{{\upsilon }_n}}∥}_{{\mathcal{F}}_2{\mathcal{K}}^2}>\delta \right\}\hfill \\ +\frac{〚L_n〛A_{1,n}^d{A}_{2,n}^d\beta \left(A_{2,n};A_n^d\right)}{\sqrt{n{\varphi }_{\mathbf{x},\mathcal{\theta }}^{1/m}\left(h_n\right)}}.\hfill \end{array}$$

We have also

$$\begin{array}{c}{\displaystyle \mathbb{P}\left\{∥\frac{1}{\sqrt{n{\varphi }_{\mathbf{x},\mathcal{\theta }}^{1/m}\left(h_n\right)}}\sum _{{\ell }_{\mathbf{1}}\in L_n}\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\sum _{{\Delta }_2}\underset{ϵ\ne {ϵ}_0}{\sum _{{\ell }_{\mathbf{2}}\in L_{1,n}\cup L_{2,n}}}\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};ϵ)\cap R_n}{\varphi }_{\mathbf{x},\mathcal{\theta }}\left(h_n\right){\xi }_{i_1}{\xi }_{i_2}\right.}\hfill \\ \left\{G_{\phi ,\mathcal{\theta },\mathbf{i}}^{\left(R\right)}\left(\left({\varsigma }_{i_1},{\varsigma }_{i_2}),({\zeta }_{i_1},{\zeta }_{i_2}\right)\right)\right.\left.{\left.-\mathbb{E}\left[G_{\phi ,\mathcal{\theta },\mathbf{i}}^{\left(R\right)}\left(({\varsigma }_{i_1},{\varsigma }_{i_2}),({\zeta }_{i_1},{\zeta }_{i_2})\right)\right]\right\}\phantom{\frac{1}{\sqrt{n{\varphi }_{\mathbf{x},\theta }\left(h_n\right)}}\sum _{p=1}^{{\upsilon }_n}\sum _{i\in H_p^{\left(U\right)}}\sum _{q:|q-p|⩾2}^{{\upsilon }_n}}∥}_{{\mathcal{F}}_2{\mathcal{K}}^2}>\delta \right\}\hfill \\ ⩽\mathbb{P}\left\{∥\frac{1}{\sqrt{n{\varphi }_{\mathbf{x},\mathcal{\theta }}^{1/m}\left(h_n\right)}}\sum _{{\ell }_{\mathbf{1}}\in L_n}\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\sum _{{\Delta }_2}\underset{ϵ\ne {ϵ}_0}{\sum _{{\ell }_{\mathbf{2}}\in L_{1,n}\cup L_{2,n}}}\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};ϵ)\cap R_n}{\varphi }_{\mathbf{x},\mathcal{\theta }}\left(h_n\right){\xi }_{i_1}{\xi }_{i_2}\right.\hfill \\ \left\{G_{\phi ,\mathcal{\theta },\mathbf{i}}^{\left(R\right)}\left(\left({\varsigma }_{i_1},{\varsigma }_{i_2}),({\zeta }_{i_1},{\zeta }_{i_2}\right)\right)\right.\left.{\left.-\mathbb{E}\left[G_{\phi ,\mathcal{\theta },\mathbf{i}}^{\left(R\right)}\left(({\varsigma }_{i_1},{\varsigma }_{i_2}),({\zeta }_{i_1},{\zeta }_{i_2})\right)\right]\right\}\phantom{\frac{1}{\sqrt{n{\varphi }_{\mathbf{x},\theta }\left(h_n\right)}}\sum _{p=1}^{{\upsilon }_n}\sum _{i\in H_p^{\left(U\right)}}\sum _{q:|q-p|⩾2}^{{\upsilon }_n}}∥}_{{\mathcal{F}}_2{\mathcal{K}}^2}>\delta \right\}.\hfill \end{array}$$

Keeping in mind (123), the problem can be reduced to

$$\begin{array}{c}{\displaystyle \mathbb{P}\left\{\left|\frac{1}{\sqrt{n{\varphi }_{\mathbf{x},\mathcal{\theta }}^{1/m}\left(h_n\right)}}\sum _{{\ell }_{\mathbf{1}}\in L_n}\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\sum _{{\Delta }_2}\underset{ϵ\ne {ϵ}_0}{\sum _{{\ell }_{\mathbf{2}}\in L_{1,n}\cup L_{2,n}}}\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};ϵ)\cap R_n}{\varphi }_{\mathbf{x},\mathcal{\theta }}\left(h_n\right){\xi }_{i_1}{\xi }_{i_2}\right.\right.}\hfill \\ \left.\left.\left.\left\{F({\zeta }_i,{\zeta }_j){1\mathrm{I}}_{\left\{F>{\lambda }_n\right\}}\right.-\mathbb{E}\left[F({\zeta }_i,{\zeta }_j){1\mathrm{I}}_{\left\{F>{\lambda }_n\right\}}\right]\right\}\phantom{\frac{1}{\sqrt{n{\varphi }_{\mathbf{x},\theta }\left(h_n\right)}}\sum _{p\ne q}^{{\upsilon }_n}\sum _{i\in H_p^{\left(U\right)}}\sum _{j\in H_q^{\left(U\right)}}}\right|>\delta \right\}\hfill \\ \lesssim {\delta }^{-2}{n}^{-1}\varphi \left(h_n\right)Var(\sum _{{\ell }_{\mathbf{1}}\in L_n}\sum _{i_1:s_{i_1}\in {\Gamma }_n({\ell }_{\mathbf{1}};{ϵ}_0)\cap R_n}\sum _{{\Delta }_2}\underset{ϵ\ne {ϵ}_0}{\sum _{{\ell }_{\mathbf{2}}\in L_{1,n}\cup L_{2,n}}}\sum _{i_2:s_{i_2}\in {\Gamma }_n({\ell }_{\mathbf{2}};ϵ)\cap R_n}{\varphi }_{\mathbf{x},\mathcal{\theta }}\left(h_n\right){\xi }_{i_1}{\xi }_{i_2}\hfill \\ \times F({\zeta }_i,{\zeta }_j){1\mathrm{I}}_{\left\{F>{\lambda }_n\right\}}\phantom{\frac{1}{\sqrt{n{\varphi }_{\mathbf{x},\theta }\left(h_n\right)}}\sum _{p=1}^{{\upsilon }_n}\sum _{i\in H_p^{\left(U\right)}}\sum _{q:|q-p|⩾2}^{{\upsilon }_n}}).\hfill \end{array}$$

We follow the identical technique as in (124). The remainder has just been demonstrated to be asymptotically negligible by considering ${\lambda }_n$ large enough. Finally, with

$${\widehat{r}}^{\left(m\right)}(\mathbf{x},\mathcal{\theta },\mathbf{u})\to \mathbb{E}\left({\mathbb{U}}_n(\phi ,\mathbf{i})\right),$$

and

$$\left({\mathbb{U}}_n(1,\mathbf{i})\right)\underset{\mathbb{P}}{\to }1,$$

the weak convergence of our estimator is accomplished. □

10. Technical Results

Assumption 8.

(KD1) 

(KB2) in Assumption 2 holds.

(KD2) 

For any $\mathcal{\alpha }\in {\mathbb{Z}}^d$ with $\left|\mathcal{\alpha }\right|=1,2$, ${\partial }^{\mathcal{\alpha }}{f}_{\mathit{S}}\left(\mathit{s}\right)$ exist and continuous on ${(0,1)}^d$.

Define

$${\widehat{f}}_{\mathit{S}}\left(\mathit{u}\right)=\frac{1}{n{h}^d}\sum _{j=1}^n{\overline{K}}_h\left(\mathit{u}-{\mathit{S}}_{0,j}\right).$$

Lemma 2

([176], Theorem 2). Under Assumption 8 and $h\to 0$ such that $n{h}^d/(logn)\to$∞ as $n\to \infty$, we have that

$$\underset{\mathit{u}\in {[0,1]}^d}{sup}\left|{\widehat{f}}_{\mathit{S}}\left(\mathit{u}\right)-f_{\mathit{S}}\left(\mathit{u}\right)\right|=O\left(\sqrt{\frac{logn}{n{h}^d}}+h^2\right)P_{\mathit{S}}-a.s.$$

Lemma 3.

Let $I_h=\left[C_{1}h,1-C_{1}h\right]$. Suppose that kernel $K_1(·)$ satisfies Assumption 8 part (i). Then, for $q=0,1,2$ and $m>1$,

$$\begin{array}{c}{\displaystyle \underset{u\in I_h}{sup}\left|\frac{1}{n^m{h}^{md}}\sum _{\mathbf{i}\in I_n^m}\right.\prod _{j=1}^m\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathbf{S}}_{0,i_j}}{h_n}\right){\left(\frac{{\mathbf{u}}_j-{\mathbf{S}}_{0,i_j}}{h}\right)}^q}\hfill \\ \left.-{\int }_{{\mathbb{R}}^{md}}\frac{1}{h^{md}}\prod _{j=1}^m\left\{\overline{K}\left(\frac{{\mathbf{u}}_j-{\mathit{\omega }}_j}{h_n}\right){\left(\frac{{\mathbf{u}}_j-{\mathit{\omega }}_j}{h}\right)}^q\right\}{f}_{\mathbf{S}}\left({\mathit{\omega }}_j\right)\prod _{j=1}^{m}d{\mathit{\omega }}_j\right|\hfill \\ =O\left(\sqrt{\frac{logn}{n{h}^{dm}}}\right)P_{\mathit{S}}-a.s.\hfill \end{array}$$

Lemma 4.

Suppose that kernel $\overline{K}$ satisfies Assumption 8. Let $g:{[0,1]}^{md}\times {\mathcal{H}}^m\to \mathbb{R}$, $(\mathbf{u},\mathbf{x})\mapsto g(\mathbf{u},\mathbf{x})$ be a function continuously partially differentiable with respect to ${\mathbf{u}}_j$. For $k=1,2$, we have

$$\begin{array}{c}\hfill {\displaystyle \underset{u\in I_h}{sup}\left|\frac{1}{n^m{h}^{md}}\sum _{\mathbf{i}\in I_n^m}\prod _{j=1}^m\overline{K}{\left(\frac{{\mathbf{u}}_j-{\mathbf{S}}_{0,j}}{h_n}\right)}^{k}g\left({\mathbf{S}}_{0,j},{\mathbf{x}}_j\right)-\prod _{j=1}^m{\kappa }_k{f}_{\mathbf{S}}\left({\mathbf{u}}_j\right)g({\mathbf{u}}_j,{\mathbf{x}}_j)\right|}\\ \hfill =O\left(\sqrt{\frac{logn}{n{h}^{md}}}\right)+o\left(h\right),P_{\mathit{S}}-a.s.\end{array}$$

(124)

where

$${\kappa }_k={\int }_{{\mathbb{R}}^d}{\overline{K}}^k\left(\mathbf{x}\right)d\mathbf{x}.$$

For any probability measure $\mathbb{Q}$ on a product measure space $\left({\Omega }_1\times {\Omega }_2,{\Sigma }_1\times {\Sigma }_2\right)$, we may define the $\beta$-mixing coefficients as follows:

Definition 5

(Definition 2.5, [165]). Let ${\mathbb{Q}}_1$ and ${\mathbb{Q}}_2$ be the marginal probability measures of $\mathbb{Q}$ on $\left({\Omega }_1,{\Sigma }_1\right)$ and $\left({\Omega }_2,{\Sigma }_2\right)$, respectively. We set

$$\beta \left({\Sigma }_1,{\Sigma }_2,\mathbb{Q}\right)=\mathbb{P}sup\left\{\left|\mathbb{Q}\left(B\mid {\Sigma }_1\right)-{\mathbb{Q}}_2\left(B\right)\right|:B\in {\Sigma }_2\right\}.$$

The following lemma holds for every finite n and is essential for the generation of independent blocks for $\beta$-mixing sequences.

Lemma 5

(Corollary 2.7, [165]). Let $m\in \mathbb{N}$ and let $\mathbb{Q}$ denote a probability measure on a product space $\left({\prod }_{i=1}^m{\Omega }_i,{\prod }_{i=1}^m{\Sigma }_i\right)$ with the associated marginal measures ${\mathbb{Q}}_i$ on $\left({\Omega }_i,{\Sigma }_i\right)$. Assume that h is a bounded measurable function on the product probability space in such a way that $\left|h\right|\le M_h<\infty$. For $1\le a\le b\le m$, let ${\mathbb{Q}}_a^b$ be the marginal measure on $\left({\prod }_{i=a}^b{\Omega }_i,{\prod }_{i=a}^b{\Sigma }_i\right)$. For a given $\tau >0$, suppose that, for all $1\le k\le m-1$,

$${∥\mathbb{Q}-{\mathbb{Q}}_1^k\times {\mathbb{Q}}_{k+1}^m∥}_{TV}\le 2\tau$$

(125)

where ${\mathbb{Q}}_1^k\times {\mathbb{Q}}_{k+1}^m$ is the product measure and ${\parallel ·\parallel }_{\mathrm{TV}}$ is the total variation. Then

$$|\mathbb{Q}h-\mathbb{P}h|\le 2M_h(m-1)\tau ,$$

where

$$\mathbb{P}=\prod _{i=1}^m{\mathbb{Q}}_i,\mathbb{Q}h=\int hd\mathbb{Q}and\mathbb{P}h=\int hd\mathbb{P}.$$

Lemma 6.

Let

$${\mathcal{I}}_n=\left\{\mathbf{i}\in {\mathbb{Z}}^d:\left(\mathbf{i}+{(0,1]}^d\right)\cap R_n\ne \varnothing \right\}.$$

Then we have

$$P_{\mathit{S}}\left(\sum _{j=1}^{n}1\mathrm{I}\left\{A_n{\mathit{S}}_{0,j}\in \left(\mathbf{i}+{(0,1]}^d\right)\cap R_n\right\}>2\left(logn+n{A}_n^{-d}\right)\mathrm{for}\mathrm{some}\mathbf{i}\in {\mathcal{I}}_n,\mathrm{i}.\mathrm{o}.\right)=0,$$

and

$$P_{\mathit{S}}\left(\sum _{j=1}^{n}1\mathrm{I}\left\{A_n{\mathit{S}}_{0,j}\in {\Gamma }_n(\ell ;\mathit{ϵ})\right\}>C{A}_{1,n}^{q\left(ϵ\right)}{A}_{2,n}^{d-q\left(ϵ\right)}n{A}_n^{-d}\mathrm{for}\mathrm{some}\ell \in L_{1,n},\mathrm{i}.\mathrm{o}.\right)=0,$$

for any $ϵ\in {\{1,2\}}^d$, where “i.o.” stands for infinitely often.

Proof. 

See the proof in (Lemma A.1, [89]) for each statement. □

Remark 13.

Lemma 6 implies that each ${\Gamma }_n(\ell ;\mathit{ϵ})$ contains at most $C{A}_{1,n}^{q\left(ϵ\right)}{A}_{2,n}^{d-q\left(ϵ\right)}n{A}_n^{-d}$ samples $P_S$-almost surely.

Lemma 7.

[13,175] Under Assumptions 2 and 3, Condition (B1) in Assumptions 4–6 and 8, we have

$$E_{.\mid \mathbf{S}}\left[{\left({\overline{S}}_n\left(\ell ;\mathit{ϵ}\right)\right)}^2\right]\le C{A}_{1,n}^{d-1}{A}_{2,n}(n{A}_n^{-d}+logn)h^{md}\varphi \left(h\right).$$

Lemma 8

(Bernstein’s inequality). Let $X_1,\dots ,X_n$ be zero-mean independent random variables. Assume that

$$\underset{1\le i\le n}{max}\left|X_i\right|\le M<\infty ,a.s.$$

For all $t>0$, we have

$$\mathbb{P}\left(\sum _{i=1}^n{X}_i\ge t\right)\le exp\left(-\frac{{\displaystyle \frac{t^2}{2}}}{{\displaystyle \sum _{i=1}^n\mathbb{E}\left[X_i^2\right]+\frac{Mt}{3}}}\right).$$

Lemma 9.

[13,175] Under Assumptions 2–4 and 6, we have

$$\frac{1}{n{h}^{md}\varphi \left(h\right)}{Var}_{·\mid \mathit{S}}\left(\sum _{\ell \in L_{1,n}}{\mathbb{Z}}_n(\ell ;\mathit{ϵ})\right)=o\left(1\right),P_{\mathit{S}}-a.s.,$$

(126)

$$\frac{1}{n{h}^{md}\varphi \left(h\right)}{Var}_{·\mid \mathit{S}}\left(\sum _{\ell \in L_{2,n}}{\mathbb{Z}}_n(\ell ;\mathit{ϵ})\right)=o\left(1\right),P_{\mathit{S}}-a.s.$$

(127)

Remark 14.

To establish the asymptotic negligibility of the summation over the small block, we can employ the method introduced in [80]. This method involves transitioning from the dependence structure of variables to independence as the first step. Subsequently, the convergence of the second-order expectation to zero is proven using a maximal inequality. This approach circumvents the need to explicitly handle covariance and relies on the use of a maximal inequality.

Proposition 2

(Proposition 2.6, [173]). If a process $\left\{X_t:t\in T\right\}$ satisfies

$${\left(\mathbb{E}{∥X_t-X_s∥}^p\right)}^{1/p}\le {\left(\frac{p-1}{q-1}\right)}^{m/2}{\left(\mathbb{E}{∥X_t-X_s∥}^q\right)}^{1/q},1<q<p<\infty ,$$

for some $m\ge 1$, and if

$$\rho (s,t)={\left(\mathbb{E}{∥X_t-X_s∥}^2\right)}^{1/2},$$

there is a constant $K=K\left(m\right)<\infty$ such that

$$\mathbb{E}\underset{s,t\in T}{sup}∥X_t-X_s∥\le K{\int }_0^D{[logN(T,\rho ,\epsilon )]}^{m/2}d\epsilon ,$$

D being the ρ-diameter of T. Moreover, if T is finite, that is, $T=\{1,\dots ,N\}$, and $N\ge 2$, then

$$\mathbb{E}\underset{i\le N}{max}∥X_i∥\le K{(logN)}^{m/2}\underset{i\le N}{max}{\left(\mathbb{E}{∥X_i∥}^2\right)}^{1/2}.$$

Lemma 10.

Ref. [177] Let $X_1,\dots ,X_n$ be a sequence of independent random elements taking values in a Banach space $(B,\parallel ·\parallel )$ with $\mathbb{E}{X}_i=0$ for all $i.$ Let $\left\{{\epsilon }_i\right\}$ be a sequence of independent Bernoulli random variables independent of $\left\{X_i\right\}.$ Then, for any convex increasing function Φ,

$$\mathbb{E}\Phi \left(\frac{1}{2}∥\sum _{i=1}^n{X}_i{\epsilon }_i∥\right)\le \mathbb{E}\Phi \left(∥\sum _{i=1}^n{X}_i∥\right)\le \mathbb{E}\Phi \left(2∥\sum _{i=1}^n{X}_i{\epsilon }_i∥\right).$$

10.1. Examples of Classes of Functions

Example 3.

The set $\mathcal{F}$ of all indicator functions ${1\mathrm{I}}_{\left\{\right(-\infty ,t\left]\right\}}$ of cells in $\mathbb{R}$ satisfies

$$N\left(ϵ,\mathcal{F},d_{\mathbb{P}}^{\left(2\right)}\right)\le \frac{2}{{ϵ}^2},$$

for any probability measure $\mathbb{P}$ and $ϵ\le 1$. Notice that

$${\int }_0^1\sqrt{log\left(\frac{1}{ϵ}\right)}dϵ\le {\int }_0^{\infty }{u}^{1/2}exp(-u)du\le 1.$$

For more details and discussion on this example refer to Example 2.5.4 of [108] and ([109], p. 157). The covering numbers of the class of cells $(-\infty ,t]$ in higher dimension satisfy a similar bound, but with a higher power of $(1/ϵ)$, see Theorem 9.19 of [109].

Example 4

(Classes of functions that are Lipschitz in a parameter , Section 2.7.4 in [108]).Let $\mathcal{F}$ be the class of functions $x\mapsto \phi (t,x)$ that are Lipschitz in the index parameter $t\in T$. Suppose that

$$|\phi (t_1,x)-\phi (t_2,x)|\le d(t_1,t_2)\kappa \left(x\right)$$

for some metric d on the index set T, the function $\kappa (·)$ is defined on the sample space $\mathcal{X}$. According to Theorem 2.7.11 of [108] and Lemma 9.18 of [109], it follows, for any norm ${\parallel ·\parallel }_{\mathcal{F}}$ on $\mathcal{F}$, that

$$N(ϵ\parallel F{\parallel }_{\mathcal{F}},\mathcal{F},\parallel ·{\parallel }_{\mathcal{F}})\le N(ϵ/2,T,d).$$

Hence if $(T,d)$ satisfy

$$J(\infty ,T,d)={\int }_0^{\infty }\sqrt{logN(ϵ,T,d)}dϵ<\infty ,$$

then the conclusions hold for $\mathcal{F}$.

Example 5.

Let us consider as an example the classes of functions that are smooth up to order α defined as follows, see Section 2.7.1 of [108] and Section 2 of [108]. For $0<\alpha <\infty$ let $\lfloor \alpha \rfloor$ be the greatest integer strictly smaller than α. For any vector $k=(k_1,\dots ,k_d)$ of d integers define the differential operator

$$D^{k_{.}}:=\frac{{\partial }^{k_{.}}}{{\partial }^{k_1}\cdots {\partial }^{k_d}},where{k}_{.}:=\sum _{i=1}^d{k}_i.$$

Then, for a function $f:\mathcal{X}\to \mathbb{R}$, let

$${\parallel f\parallel }_{\alpha }:=\underset{k_{.}\le \lfloor \alpha \rfloor }{max}\underset{x}{sup}\left|D^{k}f\left(x\right)\right|+\underset{k_{.}=\lfloor \alpha \rfloor }{max}\underset{x}{sup}\frac{D^{k}f\left(x\right)-D^{k}f\left(y\right)}{{\parallel x-y\parallel }^{\alpha -\lfloor \alpha \rfloor }},$$

where the suprema are taken over all $x,y$ in the interior of $\mathcal{X}$ with $x\ne y$. Let $C_M^{\alpha }\left(\mathcal{X}\right)$ be the set of all continuous functions $f:\mathcal{X}\to \mathbb{R}$ with

$${\parallel f\parallel }_{\alpha }\le M.$$

Note that for $\alpha \le 1$, this class consists of bounded functions f that satisfy a Lipschitz condition. Ref. [104] computed the entropy of the classes of $C_M^{\alpha }\left(\mathcal{X}\right)$ for the uniform norm. As a consequence of their results in [108], we know that there exists a constant K depending only on $\alpha ,d$ and the diameter of $\mathcal{X}$ such that for every measure γ and every $ϵ>0$,

$$log{\mathcal{N}}_{\left[\right]}(ϵM\gamma \left(\mathcal{X}\right),C_M^{\alpha }\left(\mathcal{X}\right),L_2\left(\gamma \right))\le K{\left(\frac{1}{ϵ}\right)}^{d/\alpha },$$

${\mathcal{N}}_{\left[\right]}$ is the bracketing number, refer to Definition 2.1.6 of [108] and we refer to Theorem 2.7.1 of [108] for a variant of the last inequality. By Lemma 9.18 of [109], we have

$$log\mathcal{N}(ϵM\gamma \left(\mathcal{X}\right),C_M^{\alpha }\left(\mathcal{X}\right),L_2\left(\gamma \right))\le K{\left(\frac{1}{2ϵ}\right)}^{d/\alpha }.$$

10.2. Examples of U-kernels

In this section, we present some classical U-kernels.

Example 6.

Ref. [125] introduced the parameter

$$▵={\int }_{-\infty }^{\infty }{\int }_{-\infty }^{\infty }{D}^2(y_1,y_2)dF(y_1,y_2),$$

where

$$D(y_1,y_2)=F(y_1,y_2)-F(y_1,\infty )F(\infty ,y_2),$$

and $F(·,·)$ is the distribution function of $Y_1$ and $Y_2$. The parameter ▵ has the property that $▵=0$ if and only if $Y_1$ and $Y_2$ are independent. From [178], an alternative expression for ▵ can be developed by introducing the functions

$$\psi \left(y_1,y_2,y_3\right)=\left\{\begin{array}{ccc}\hfill 1& \mathit{if}& y_2\le y_1<y_3,\hfill \\ \hfill 0& \mathit{if}& y_1<y_2,y_3\mathrm{or}{y}_1\ge y_2,y_3,\hfill \\ \hfill -1& \mathit{if}& y_3\le y_1<y_2,\hfill \end{array}\right.$$

and

$$\begin{array}{ccc}\hfill \phi \left(y_{1,1},y_{1,2},\dots ,y_{5,1},y_{5,2}\right)& =& \frac{1}{4}\psi \left(y_{1,1},y_{1,2},y_{1,3}\right)\psi \left(y_{1,1},y_{1,4},y_{1,5}\right)\hfill \\ & & \times \psi \left(y_{1,2},y_{2,2},y_{3,2}\right)\psi \left(y_{1,2},y_{4,2},y_{5,2}\right).\hfill \end{array}$$

We have

$$▵=\int \dots \int \phi \left(y_{1,1},y_{1,2},\dots ,y_{5,1},y_{5,2}\right)dF\left(y_{1,1},y_{1,2}\right)\dots dF\left(y_{1,5},y_{2,5}\right).$$

Example 7 (Hoeffding’s D).

From the symmetric kernel,

$$\begin{array}{c}{\displaystyle {\phi }_D\left(z_1,\dots ,z_5\right)}\hfill \\ := \frac{1}{16}\sum _{\left(i_1,\dots ,i_5\right)\in {\mathcal{P}}_5}\hfill \\ \times \left[\left\{1\mathrm{I}\left(z_{i_1,1}\le z_{i_5,1}\right)-1\mathrm{I}\left(z_{i_2,1}\le z_{i_5,1}\right)\right\}\left\{1\mathrm{I}\left(z_{i_3,1}\le z_{i_5,1}\right)-1\mathrm{I}\left(z_{i_4,1}\le z_{i_5,1}\right)\right\}\right]\hfill \\ \times \left[\left\{1\mathrm{I}\left(z_{i_1,2}\le z_{i_5,2}\right)-1\mathrm{I}\left(z_{i_2,2}\le z_{i_5,2}\right)\right\}\left\{1\mathrm{I}\left(z_{i_3,2}\le z_{i_5,2}\right)-1\mathrm{I}\left(z_{i_4,2}\le z_{i_5,2}\right)\right\}\right],\hfill \end{array}$$

we recover Hoeffding’s D statistic, a rank-based U-statistic of order 5, and gives rise to Hoeffding’s D correlation measure $\mathbb{E}{h}_D$.

Example 8

(Blum–Kiefer–Rosenblatt’s R). The symmetric kernel

$$\begin{array}{c}{\displaystyle {\phi }_R\left(z_1,\dots ,z_6\right)}\hfill \\ := \frac{1}{32}\sum _{\left(i_1,\dots ,i_6\right)\in {\mathcal{P}}_6}\hfill \\ \times \left[\left\{1\mathrm{I}\left(z_{i_1,1}\le z_{i_5,1}\right)-1\mathrm{I}\left(z_{i_2,1}\le z_{i_5,1}\right)\right\}\left\{1\mathrm{I}\left(z_{i_3,1}\le z_{i_5,1}\right)-1\mathrm{I}\left(z_{i_4,1}\le z_{i_5,1}\right)\right\}\right]\hfill \\ \times \left[\left\{1\mathrm{I}\left(z_{i_1,2}\le z_{i_6,2}\right)-1\mathrm{I}\left(z_{i_2,2}\le z_{i_6,2}\right)\right\}\left\{1\mathrm{I}\left(z_{i_3,2}\le z_{i_6,2}\right)-1\mathrm{I}\left(z_{i_4,2}\le z_{i_6,2}\right)\right\}\right],\hfill \end{array}$$

yields Blum–Kiefer–Rosenblatt’s R statistic [179].

Example 9

(Bergsma–Dassios–Yanagimoto’s ${\tau }^{*}$). Ref. [180] introduced a rank correlation statistic as a U-statistic of order 4 with the symmetric kernel

$$\begin{array}{c}\hfill {\displaystyle {\phi }_{{\tau }^{*}}\left(z_1\right.\left.,\dots ,z_4\right)}\\ & :=& \frac{1}{16}\sum _{\left(i_1,\dots ,i_4\right)\in {\mathcal{P}}_4}\left\{1\mathrm{I}\left(z_{i_1,1},z_{i_3,1}<z_{i_2,1},z_{i_4,1}\right)+1\mathrm{I}\left(z_{i_2,1},z_{i_4,1}<z_{i_1,1},z_{i_3,1}\right)\right.\hfill \\ & & \left.-1\mathrm{I}\left(z_{i_1,1},z_{i_4,1}<z_{i_2,1},z_{i_3,1}\right)-1\mathrm{I}\left(z_{i_2,1},z_{i_3,1}<z_{i_1,1},z_{i_4,1}\right)\right\}\hfill \\ & & \times \left\{1\mathrm{I}\left(z_{i_1,2},z_{i_3,2}<z_{i_2,2},z_{i_4,2}\right)+1\mathrm{I}\left(z_{i_2,2},z_{i_4,2}<z_{i_1,2},z_{i_3,2}\right)\right.\hfill \\ & & \left.-1\mathrm{I}\left(z_{i_1,2},z_{i_4,2}<z_{i_2,2},z_{i_3,2}\right)-1\mathrm{I}\left(z_{i_2,2},z_{i_3,2}<z_{i_1,2},z_{i_4,2}\right)\right\}.\hfill \end{array}$$

Here,

$$1\mathrm{I}\left(y_1,y_2<y_3,y_4\right):=1\mathrm{I}\left(y_1<y_3\right)1\mathrm{I}\left(y_1<y_4\right)1\mathrm{I}\left(y_2<y_3\right)1\mathrm{I}\left(y_2<y_4\right).$$

Example 10.

The Wilcoxon Statistic. Suppose that $E\subset \mathbb{R}$ is symmetric around zero. As an estimate of the quantity

$${\int }_{(x,y)\in E^2}\left\{2{1\mathrm{I}}_{\{x+y>0\}}-1\right\}dF\left(x\right)dF\left(y\right),$$

it is pertinent to consider the statistic

$$W_n=\frac{2}{n(n-1)}\sum _{1\le i<j\le n}\left\{2·{1\mathrm{I}}_{\left\{X_i+X_j>0\right\}}-1\right\},$$

which is relevant for testing whether or not μ is located at zero.

Example 11.

The Takens estimator. Denote by $\parallel ·\parallel$ the usual Euclidean norm on ${\mathbb{R}}^d$. In [181], the following estimate of the correlation integral,

$$C_F\left(r\right)=\int {1\mathrm{I}}_{\left\{∥x-x^{\prime }∥\le r\right\}}dF\left(x\right)dF\left(x^{\prime }\right),r>0,$$

is considered:

$$C_n\left(r\right)=\frac{1}{n(n-1)}\sum _{1\le i\ne j\le n}{1\mathrm{I}}_{\left\{∥X_i-X_j∥\le r\right\}}.$$

In the case where a scaling law holds for the correlation integral, i.e., when there exists $\left(\alpha ,r_0,c\right)\in$${\mathbb{R}}_{+}^{*3}$ such that $C_F\left(r\right)=c·r^{-\alpha }$ for $0<r\le r_0$, the U-statistic

$$T_n=\frac{1}{n(n-1)}\sum _{1\le i\ne j\le n}log\left(\frac{∥X_i-X_j∥}{r_0}\right),$$

is used in order to build the Takens estimator ${\widehat{\alpha }}_n=-T_n^{-1}$ of the correlation dimension α.

Example 12.

Let $\widehat{Y_1{Y}_2}$ denote the oriented angle between $Y_1,Y_2\in T$, T is the circle of radius 1 and center 0 in ${\mathbb{R}}^2$. Let

$${\phi }_t(Y_1,Y_2)=1\mathrm{I}\{\widehat{Y_1{Y}_2}\le t\}-t/\pi ,fort\in [0,\pi ).$$

Ref. [182] has used this kernel in order to propose a U-process to test uniformity on the circle.

Example 13.

For $m=3$, let

$$\phi (Y_1,Y_2,Y_3)=1\mathrm{I}\{Y_1-Y_2-Y_3>0\},$$

We have

$$r^{\left(3\right)}(\phi ,t_1,t_2,t_3)=\mathbb{P}(Y_1>Y_2+Y_3\mid X_1=X_2=X_3=t)$$

and the corresponding conditional U-Statistic can be considered a conditional analog of the Hollander–Proschan test-statistic [183]. It may be used to test the hypothesis that the conditional distribution of $Y_1$ given $X_1=t$, is exponential, against the alternative that it is of the New Better than Used type.

Example 14.

The Gini [184] mean difference. The Gini index provides another popular measure of dispersion. It corresponds to the case where $E\subset \mathbb{R}$ and $h(x,y)=|x-y|$

$$G_n=\frac{2}{n(n-1)}\sum _{1\le i<j\le n}\left|X_i-X_j\right|.$$

Example 15

([20]). Let the sample central moments of any order $m=2,3,\dots$ be given by

$$\begin{array}{ccc}\hfill {\theta }_m\left(F\right)& =& \mathbb{E}{\left(X_1-\mathbb{E}{X}_1\right)}^m\hfill \\ & =& \int {(x-\mathbb{E}{X}_1)}^{m}dF\left(x\right).\hfill \end{array}$$

In this case, the U-statistic has a symmetric kernel

$$\begin{array}{cc}& \phi \left(x_1,\dots ,x_m\right)=\frac{1}{m!}\sum \left[x_{i_1}^m-\left(\begin{array}{c}m\\ 1\end{array}\right)x_{i_1}^{m-1}{x}_{i_2}\right.\hfill \\ & \left.+\left(\begin{array}{c}m\\ 2\end{array}\right)x_{i_1}^{m-2}{x}_{i_2}{x}_{i_3}-\dots +{(-1)}^{m-1}\left(\left(\begin{array}{c}m\\ m-1\end{array}\right)-1\right)x_{i_1}{x}_{i_2}\dots x_{i_m}\right],\hfill \end{array}$$

where summation is carried out over all permutations $\left(i_1,\dots ,i_m\right)$ of the numbers $(1,\dots ,m)$. In particular, if $m=3$, then

$$\begin{array}{cc}\hfill \phi \left(x_1,x_2,x_3\right)=& \frac{1}{3}\left(x_1^3+x_2^3+x_3^3\right)\hfill \\ & -\frac{1}{2}\left(x_1^2{x}_2+x_2^2{x}_1+x_1^2{x}_3+x_3^2{x}_1+x_2^2{x}_3+x_2^2{x}_2\right)+2x_1{x}_2{x}_3.\hfill \end{array}$$

In the case of $m=2$,

$$\begin{array}{ccc}\hfill {\theta }_2\left(F\right)& =& \mathbb{E}{\left(X_1-\mathbb{E}{X}_1\right)}^2\hfill \\ & =& \int {(x-\mathbb{E}{X}_1)}^{2}dF\left(x\right).\hfill \end{array}$$

For the kernel

$$\begin{array}{ccc}\hfill \phi \left(x_1,x_2\right)& =& \frac{x_1^2+x_2^2-2x_1{x}_2}{2}\hfill \\ & =& \frac{1}{2}{\left(x_1-x_2\right)}^2.\hfill \end{array}$$

the corresponding U-statistic is the sample variance

$$\begin{array}{cc}\hfill U_n\left(h\right)& =\frac{2}{n(n-1)}\sum _{1\le i<j\le n}h\left(X_i,X_j\right)\hfill \\ & =\frac{1}{n-1}\left[\sum _{i=1}^n{X}_i^2-n{\left(\frac{1}{n}\sum _{i=1}^n{X}_i\right)}^2\right]\hfill \\ & =\frac{1}{n-1}\left(\sum _{i=1}^n{X}_i^2-n{\overline{X}}_n^2\right),\hfill \end{array}$$

we refer also to [19].

Example 16

([40]). Assume $Y_i={\left(Y_{i1},Y_{i2}\right)}^t$, and define h by

$$\phi \left[\left[\begin{array}{c}{y}_{11}\hfill \\ y_{12}\hfill \end{array}\right],\left[\begin{array}{c}{y}_{21}\hfill \\ y_{22}\hfill \end{array}\right]\right]=\frac{1}{2}\left(y_{11}{y}_{12}+y_{21}{y}_{22}-y_{11}{y}_{22}-y_{12}{y}_{21}\right),$$

that is, $m=2$, and

$$\begin{array}{ccc}\hfill r^{\left(2\right)}\left(x_1,x_2\right)& =& \frac{1}{2}\left[\mathbb{E}\left(Y_{11}{Y}_{12}\mid X_1=x_1\right)+\mathbb{E}\left(Y_{21}{Y}_{22}\mid X_2=x_2\right)\right.\hfill \\ & & \left.-\mathbb{E}\left(Y_{11}{Y}_{22}\mid X_1=x_1,X_2=x_2\right)-\mathbb{E}\left(Y_{12}{Y}_{21}\mid X_1=x_1,X_2=x_2\right)\right].\hfill \end{array}$$

In particular,

$$r^{\left(2\right)}\left(x_1,x_1\right)=\mathbb{E}\left(Y_{11}{Y}_{12}\mid X_1=x_1\right)-\mathbb{E}\left(Y_{11}\mid X_1=x_1\right)\mathbb{E}\left(Y_{12}\mid X_1=x_1\right),$$

the conditional covariance of $Y_1$ given $X_1=x_1$.

Example 17

([185]) This example is given in [185]. The accelerated failure time (AFT) model is a prevalent tool in survival analysis, assuming a linear relationship between the logarithm of failure time $T_i$ and associated covariates $X_i$. Mathematically, it is represented as

$$log{T}_i=X_i^{\top }\theta +{\zeta }_i.$$

Here, θ represents a vector of unknown regression parameters ($p\times 1$), and $\{{\zeta }_i,i=1,\dots ,N\}$ are independent and identically distributed (i.i.d.) random errors with an unspecified distribution function, independent of $\{X_i,i=1,\dots ,N\}$. Usually, $T_i$ is censored at $C_i$, leading to observed data denoted as $\{Z_i=({\tilde{T}}_i,{\delta }_i,X_i),i=1,\dots ,N\}$, where ${\tilde{T}}_i=min(T_i,C_i)$ is the observed failure time, and ${\delta }_i=I(T_i\le C_i)$ is the censoring indicator. Estimating AFT models often involves using the Gehan estimator [186]. This estimator is derived by minimizing an objective function, formulated as a U-statistic:

$$F_N\left(\theta \right)=\frac{1}{{\mathcal{C}}_N^2}\sum _{i\ne j}^N{\delta }_i(e_j\left(\theta \right)-e_i\left(\theta \right))I(e_i\left(\theta \right)\le e_j\left(\theta \right)).$$

Here, $e_i\left(\theta \right)=log{\tilde{T}}_i-X_i^{\top }\theta$. However, the non-smooth nature of this function poses computational challenges in estimating coefficients and their standard errors. To address this challenge, ref. [187] introduced an induced smoothing method. They replace the non-smooth objective function with a smooth approximation, minimizing the following empirical risk:

$$\frac{1}{{\mathcal{C}}_N^2}\sum _{i\ne j}^N{\delta }_i\left[(e_j\left(\theta \right)-e_i\left(\theta \right))\Phi \left(\frac{e_j\left(\theta \right)-e_i\left(\theta \right)}{r_{ij}}\right)+r_{ij}\varphi \left(\frac{e_j\left(\theta \right)-e_i\left(\theta \right)}{r_{ij}}\right)\right].$$

Here, $r_{ij}^2=\frac{1}{N}{(X_i-X_j)}^{\top }\Sigma (X_i-X_j)$, and Σ is a symmetric and positive definite matrix satisfying $\parallel {\Sigma }^{1/2}\parallel =O\left(1\right)$.