k-Nearest Neighbour Estimation of the Conditional Set-Indexed Empirical Process for Functional Data: Asymptotic Properties

Abstract

The main aim of this paper is to improve the existing limit theorems for set-indexed conditional empirical processes involving functional strong mixing random variables. To achieve this, we propose using the k-nearest neighbor approach to estimate the regression function, as opposed to the traditional kernel method. For the first time, we establish the weak consistency, asymptotic normality, and density of the proposed estimator. Our results are derived under certain assumptions about the richness of the index class $\mathcal{C}$, specifically in terms of metric entropy with bracketing. This work builds upon our previous papers, which focused on the technical performance of empirical process methodologies, and further refines the prior estimator. We highlight that the k-nearest neighbor method outperforms the classical approach due to several advantages.

1. Introduction

The framework of empirical processes has long been recognized as a fundamental component of statistical methodology, underpinning theoretical developments and applications in areas such as testing for independence, parameter estimation, and bootstrap procedures. Furthermore, this framework has offered insights into the asymptotic properties of semi-parametric mixture models and various other sophisticated methods. For detailed expositions of both theoretical and applied dimensions of empirical process theory, the reader is referred to [1,2]. Over approximately the last three decades, empirical process theory in finite contexts has undergone marked expansion, as reflected by numerous contributions focusing on the case of independent and identically distributed (i.i.d.) observations.

An early milestone in this line of inquiry was provided by [3], who examined the asymptotic distribution of the standard empirical process when sample sizes are random. The work of [3] established weak convergence for continuous empirical processes within suitably chosen metric spaces. Subsequently, ref. [4] investigated the classes C of sets for which the Glivenko–Cantelli theorem holds (up to measurability), thus solidifying the basis for uniform convergence of empirical distribution functions. Building on these results, ref. [5] introduced both strong and weak approximations of empirical processes by leveraging Gaussian processes, ultimately highlighting associated invariance principles. In a related direction, ref. [6] studied empirical processes pertaining to particular classes of sets, laying out the conditions under which weak convergence to Gaussian processes emerges in terms of metric entropy with inclusion. This analysis extends well beyond the classical framework of Donsker’s theorem and accommodates independent random variables. Additional seminal results are available in [7,8,9], among other contributions, underscoring the continued advancement of empirical process theory.

Over subsequent years, attention turned toward mixing processes and their conditional aspects in separable metric spaces. ref. [10] provided broad generalizations of the central limit theorem for three varieties of mixing variables defined on the interval $[0,1]$. ref. [11] later investigated weak convergence under $\varphi$-mixing conditions, while ref. [12] addressed specific regularity constraints leading to the asymptotic normality of parameter estimators. Extending these ideas further, ref. [13] examined set-indexed empirical processes in $\alpha$-mixing settings to establish results involving weak convergence in probability and asymptotic normality, as well as deriving variance expressions and continuity properties vital to asymptotic investigations. Subsequent research by [14] refined these theoretical insights. For comprehensive treatments and related studies on mixing empirical processes, see [15,16,17,18,19,20]. A parallel body of work addresses the need to analyze infinite-dimensional data—commonly referred to as functional data or functional variables—across fields such as environmental science, chemometrics, biometrics, medicine, and econometrics. These functional data structures often capture intricate phenomena more effectively than finite-dimensional representations, which is particularly evident in research areas like finance and biology. Comprehensive overviews of functional data analysis are provided by [21,22,23,24,25,26]. In recent years, this domain has experienced rapid expansion, as evidenced by a substantial increase in scholarly work and specialized monographs, highlighting the evolving and dynamic nature of functional data analysis.

Despite the significant advancements in functional data analysis, the integration of functional data within the framework of set-indexed empirical processes remains relatively underexplored. Notably, recent work by [27] made a substantial contribution by extending the results of [13], thereby proving the invariance principle for the set-indexed conditional empirical process. ref. [27] further applied these theoretical advancements to the problem of conditional independence testing, demonstrating the practical utility of the extended framework. This line of research was further developed in subsequent works by [28,29], which addressed the complexities associated with ergodic data. More recently, ref. [30] incorporated the presence of missing data into their analysis, leveraging the strengths of the k-nearest neighbors (k-NN) method for estimation purposes. The k-NN method has enjoyed widespread application within finite frameworks, with foundational contributions from [8], ref. [31], as well as adaptations for functional data models. Notably, ref. [32] investigated the asymptotic properties of the k-NN kernel estimator for regression tasks, providing comparative analyses with traditional kernel methods. ref. [33] offered asymptotic results for k-NN generalized regression estimators in abstract spaces. For further exploration of k-NN methods in the context of infinite-dimensional data, researchers can consult the works of [34,35,36,37].

1.1. Contribution

Despite the substantial rise in research on empirical processes and the application of k-NN methods to functional data, studies that examine the asymptotic properties of empirical processes unifying functional data with k-NN methods are still relatively scarce. Nonetheless, recent contributions have begun to address this gap. In particular, refs. [28,38,39] offered important theoretical and practical insights by establishing both the uniform consistency and weak convergence of k-NN empirical conditional processes and k-NN conditional U-processes adapted to functional mixing data. These developments hold considerable promise for set-indexed conditional U-statistics, Kendall’s rank correlation coefficient, classification tasks, and the prediction of time series. Building upon these findings, the present paper introduces an innovative methodological framework aimed at efficiently tackling the challenges inherent in set-indexed conditional empirical processes.

The main distinction between this work and [28] or [29] lies in our consideration of a more intricate data-driven bandwidth structure. In addition, we establish novel results concerning the uniform consistency with respect to the number of neighborhoods. Compared with [28,38,39], our results are derived under bracketing numbers that capture the complexity of the relevant sets in a manner that differs from the entropy-based techniques explored in our previous research. Furthermore, the strong mixing condition assumed here is more general than the regular processes context adopted in those earlier works to prove weak convergence.

1.2. Organization of the Paper

The structure of this article is meticulously organized to facilitate a coherent and comprehensive exposition of the subject matter. Section 2 delineates the notation and fundamental definitions essential for understanding the ensuing discussions. It also introduces the k-NN set-indexed conditional empirical process, laying the groundwork for the theoretical developments that follow. Section 3 is dedicated to presenting the main results of this study, highlighting the key theoretical contributions and their implications. In Section 4, an application of these theoretical results to the domain of statistical classification is discussed, demonstrating the practical relevance and utility of the proposed methods. Section 3.3 addresses the pragmatic aspects related to the selection of the parameter k, which is pivotal for the performance of the k-NN method. This section provides guidance on optimal bandwidth selection, balancing bias and variance considerations. The concluding remarks, along with prospective directions for future research, are provided in Section 5, with all proofs collected in Appendix A.

2. Presentation of the k-NN Set-Indexed Conditional Empirical Process

We consider a sequence of random pairs $(X_1,Y_1),\dots ,(X_n,Y_n)$, where each pair is a realization of the random variable $(X,Y)$, which takes values in the space $\mathcal{E}\times {\mathbb{R}}^d$. The functional space $\mathcal{E}$ is equipped with a semi-metric $d_{\mathcal{E}}(·,·)$ (A semi-metric, or pseudo-metric, allows $d(x_1,x_2)=0$ even if $x_1\ne x_2$). Our goal is to examine the relationship between X and Y by estimating functional operators related to the conditional distribution of Y given X, such as the regression operator. For any measurable set C in a class of sets $\mathcal{C}$, the conditional probability is defined as:

$$\mathrm{\Gamma }(C\mid x)=\mathbb{E}[{\mathbb{1}}_{\{Y\in C\}}\mid X=x].$$

To model this relationship, we use a Nadaraya–Watson estimator [40,41] and present a k-nearest neighbors (k-NN) version of the conditional empirical distribution. Specifically, we define:

$${\mathrm{\Gamma }}_n(C,x)={\displaystyle \frac{{\sum }_{i=1}^n{\mathbb{1}}_{\{Y_i\in C\}}K\left(\frac{d_{\mathcal{E}}(x,X_i)}{H_{n,k}\left(x\right)}\right)}{{\sum }_{i=1}^{n}K\left(\frac{d_{\mathcal{E}}(x,X_i)}{H_{n,k}\left(x\right)}\right)}},$$

(1)

where $K(·)$ is a real-valued kernel function defined on $[0,\infty )$, and $H_{n,k}\left(x\right)$ is a smoothing parameter that satisfies $H_{n,k}\left(x\right)⟶0$ as $n⟶\infty$. Here, C is a measurable set, and $x\in \mathcal{E}$. When $C=(-\infty ,z]$ for some $z\in {\mathbb{R}}^d$, the estimator reduces to the conditional empirical distribution function $F_n(z\mid x)={\mathrm{\Gamma }}_n((-\infty ,z],x)$. Hence, the corresponding class of sets is $\mathcal{C}=\{(-\infty ,z]:z\in {\mathbb{R}}^d\}$. We further define $H_{n,k}\left(x\right)$ as the smallest value $h\in {\mathbb{R}}^{+}$ such that:

$$\sum _{i=1}^n{\mathbb{1}}_{B(x,h)}\left(X_i\right)=k,$$

where $B(x,h)$ represents the ball of radius h around x. It is clear that $H_{n,k}\left(x\right)$ is a positive random variable with respect to the semi-metric topology on $\mathcal{E}$. To relate this work to the existing literature and to highlight the differences between the k-NN and classical kernel approaches, we recall the functional Nadaraya–Watson estimator from [23] and refer to [27,29,30] for more information on the asymptotic properties of nonparametric functional distribution estimators. The classical kernel-based estimator is defined as:

$${\widehat{\mathrm{\Gamma }}}_n(C,x)={\displaystyle \frac{{\sum }_{i=1}^n{\mathbb{1}}_{\{Y_i\in C\}}K\left(\frac{d_{\mathcal{E}}(x,X_i)}{h_n}\right)}{{\sum }_{i=1}^{n}K\left(\frac{d_{\mathcal{E}}(x,X_i)}{h_n}\right)}},$$

(2)

where $h_n$ is a smoothing parameter. To further clarify the notation, we use:

$$B(x,t)=\{x_1\in \mathcal{E}:d_{\mathcal{E}}(x_1,x)\le t\}$$

to represent the ball in $\mathcal{E}$ with center x and radius t and define the small ball probability function as:

$$F(t;x)=\mathbb{P}(d_{\mathcal{E}}(x,X_i)\le t)=\mathbb{P}(X_i\in B(x,t))=\mathbb{P}(D_i\le t),$$

where $D_i=d_{\mathcal{E}}(x,X_i)$. The behavior of $F(t;x)$ as $t⟶0$ is of particular interest, and [42] assumed that $F(h;x)=\varphi \left(h_n\right)f_1\left(x\right)$ as $h⟶0$, where $f_1\left(x\right)$ represents the probability density (functional). When $\mathcal{H}={\mathbb{R}}^m$, the small ball probability function takes the form $F(h;x)=\mathbb{P}(\parallel x-X_i\parallel \le h)$, where $\varphi \left(h_n\right)=C\left(m\right)h^m$ is the volume of a unit ball in ${\mathbb{R}}^m$. This result motivates the assumption $\left(\mathbf{H}\mathbf{4}\right)\left(i\right)-\left(ii\right)$, as discussed in [43].

In statistics, observations are often weakly dependent rather than independent. Ignoring this dependence can have serious consequences for statistical inference. The concept of dependence quantifies the extent to which a sequence of random variables deviates from independence and is useful for extending classical results to weakly dependent or mixing sequences. We refer to [44] for further details on dependence structures. For our analysis, we assume that the sequence ${\left\{(X_i,Y_i)\right\}}_{i=1}^n$ is strongly mixing.

Definition 1.

A sequence $\{{\zeta }_k,k\ge 1\}$ is said to be α-mixing if the α-mixing coefficient:

$$\alpha \left(n\right)=\underset{k\ge 1}{sup}sup\left\{|\mathbb{P}(A\cap B)-\mathbb{P}\left(A\right)\mathbb{P}\left(B\right)|:A\in {\mathcal{F}}_{n+k}^{\infty },B\in {\mathcal{F}}_1^k\right\}$$

converges to zero as $n⟶\infty$, where ${\mathcal{F}}_l^m=\sigma ({\zeta }_l,{\zeta }_{l+1},\dots ,{\zeta }_m)$ denotes the σ-algebra generated by ${\zeta }_l,{\zeta }_{l+1},\dots ,{\zeta }_m$ with $l\le m$. A sequence is geometrically strong mixing if, for some $a>0$ and $\beta >1$,

$$\alpha \left(j\right)\le a{j}^{-\beta },$$

and exponentially strong mixing if, for some $b>0$ and $0<\gamma <1$,

$$\alpha \left(k\right)\le b{\gamma }^k.$$

In the exploration of mixing conditions, $\alpha$-mixing stands out as a somewhat lenient criterion that is nonetheless broadly applicable across various stochastic processes, including many time series models. Works by [45,46] pinpointed the necessary conditions for $\alpha$-mixing in linear processes. These studies showed that both linear autoregressive models and the more complex bilinear time series models exhibit robust mixing properties characterized by mixing coefficients that decrease exponentially. Additionally, the study [47] shed light on the importance of $\alpha$-mixing, including aspects like geometric ergodicity, in understanding nonlinear time series models, as further discussed in [48,49,50]. Similarly, ref. [51] illustrated that functional autoregressive processes can achieve geometric ergodicity when certain conditions are met. Moreover, the studies [52,53] showed that with only minimal assumptions, both autoregressive conditional heteroscedastic processes and nonlinear additive autoregressive models that include exogenous variables maintain stationarity and $\alpha$-mixing.

Definition 2.

A class $\mathcal{C}$ of subsets of a set C is called a VC-class if there exists a polynomial $\mathfrak{B}(·)$ such that, for every set of N points in C, the class $\mathcal{C}$ can select at most $\mathfrak{B}\left(N\right)$ distinct subsets.

Definition 3.

A class of functions $\mathcal{F}$ is called a VC-subgraph class if the graphs of the functions in $\mathcal{F}$ form a VC-class of sets. Specifically, the subgraph of a real-valued function f on a set S is defined as

$${\mathcal{G}}_f=\{(s,t):0\le t\le f\left(s\right)\mathrm{or}f\left(s\right)\le t\le 0\},$$

and the collection $\{{\mathcal{G}}_f:f\in \mathcal{F}\}$ is a VC-class of sets on $S\times \mathbb{R}$.

Definition 4.

Let ${\mathcal{S}}_{\mathcal{E}}$ be a subset of a semi-metric space $\mathcal{E}$, and let $N_{\mathcal{E}}$ be a positive integer. A finite set $\{e_1,\dots ,e_{N_{\mathcal{E}}}\}\subset \mathcal{E}$ is called an ε-net of ${\mathcal{S}}_{\mathcal{E}}$ if:

$${\mathcal{S}}_{\mathcal{E}}\subseteq \bigcup _{j=1}^{N_{\mathcal{E}}}B(e_j;\mathcal{E}),$$

where $B(e_j;\mathcal{E})$ is the ball of radius ε around $e_j$. The Kolmogorov entropy (or metric entropy) of ${\mathcal{S}}_{\mathcal{E}}$ is defined as

$${\psi }_{{\mathcal{S}}_{\mathcal{E}}}\left(\mathcal{E}\right)=log{\mathcal{N}}_{\mathcal{E}}\left({\mathcal{S}}_{\mathcal{E}}\right),$$

where ${\mathcal{N}}_{\mathcal{E}}\left({\mathcal{S}}_{\mathcal{E}}\right)$ is the cardinality of the smallest $\mathcal{E}$-net required to cover ${\mathcal{S}}_{\mathcal{E}}$.

The concept of semi-metrics plays a central role in this study. For further discussions on how to choose the appropriate semi-metric, we refer the reader to [23] (see Chapters 3 and 13).

2.1. Notations and Hypothesis

Throughout this paper, we fix x as an element of the functional space $\mathcal{E}$. We introduce the notion of metric entropy with inclusion, which quantifies the complexity or richness of the class of sets $\mathcal{C}$. For each $\epsilon >0$, the covering number is defined as follows:

$$\begin{array}{c}\hfill {\displaystyle \mathcal{N}\left(\epsilon ,\mathcal{C},\mathrm{\Gamma }\left(·\mid x\right)\right)}\\ & =& inf\left\{n\in \mathbb{N}:\exists C_1,\cdots ,C_n\in \mathcal{C}\mathrm{such}\mathrm{that}\mathrm{for}\mathrm{all}C\in \mathcal{C},\exists 1\le i,j\le n\mathrm{with}{C}_i\subset C\subset C_j\right.\hfill \\ & & \left.\mathrm{and}\mathrm{\Gamma }(C_j\setminus C_i\mid x)<\epsilon \right\},\hfill \end{array}$$

The term $log\left(\mathcal{N}\left(\epsilon ,\mathcal{C},\mathrm{\Gamma }\left(·\mid x\right)\right)\right)$ is called the metric entropy with the inclusion of $\mathcal{C}$ with respect to $\mathrm{\Gamma }\left(·\mid x\right)$. Estimates for such covering numbers are available for many classes; see, for example, [54]. We will often assume below that either $log\mathcal{N}\left(\epsilon ,\mathcal{C},\mathrm{\Gamma }\left(·\mid x\right)\right)$ or $\mathcal{N}\left(\epsilon ,\mathcal{C},\mathrm{\Gamma }\left(·\mid x\right)\right)$ behaves like powers of ${\epsilon }^{-1}$. We say that the condition ($R_{\gamma }$) holds if

$$log\mathcal{N}\left(\epsilon ,\mathcal{C},\mathrm{\Gamma }\left(·\mid x\right)\right)\le H_{\gamma }\left(\epsilon \right),\mathrm{for}\mathrm{all}\epsilon >0,$$

(3)

where

$$H_{\gamma }\left(\epsilon \right)=\left\{\begin{array}{cc}log\left(A\epsilon \right)\hfill & \mathrm{if}\gamma =0,\hfill \\ A{\epsilon }^{-\gamma }\hfill & \mathrm{if}\gamma >0,\hfill \end{array}\right.$$

for some constants $A,r>0$. As in [13], it is worth noting that the condition (3), for $\gamma =0$, holds for intervals, rectangles, balls, ellipsoids, and for classes constructed from these by performing finitely many set operations such as union, intersection, and complement. The classes of convex sets in ${\mathbb{R}}^d$ ($d\ge 2$) satisfy the condition (3), where $\gamma =(d-1)/2$. Other classes of sets satisfying (3) with $\gamma >0$ can be found in [54].

In this section, we derive the almost complete convergence and central limit theorems for the conditional empirical process $\{{\mathrm{\Lambda }}_n(C,x):C\in \mathcal{C}\}$, defined by

$${\mathrm{\Lambda }}_n(C,x):=\sqrt{n\varphi \left(h_n\right)}\left({\mathrm{\Gamma }}_n(C,x)-\mathrm{\Gamma }(C,x)\right).$$

(4)

Next, we list the conditions required for our analysis:

(H1) 

On the distributions/small-ball probabilities

(H1)(i)

For $h\in {\mathbb{R}}^{+}$ and $x\in \mathcal{E}$, and positive constants $C_1,C_2$, we assume that

$$0<C_1\varphi \left(h_n\right)\le {\varphi }_x\left(h_n\right)\le C_2\varphi \left(h_n\right)<\infty .$$

(5)

(H1)(ii)

For all $i\ge 1$,

$$0<c_5\varphi \left(h_n\right)f_1\left(x\right)\le \mathbb{P}(X_i\in B(x,h))=F(h;x)\le c_6\varphi \left(h_n\right)f_1\left(x\right),$$

where $\varphi \left(h_n\right)⟶0$ as $h⟶0$ and $f_1\left(x\right)$ is a nonnegative functional in $x\in \mathcal{E}$.

(H1)(iii)

We have

$$\underset{i\ne j}{sup}\mathbb{P}((X_i,X_j)\in B(x,h)\times B(x,h))=\underset{i\ne j}{sup}\mathbb{P}(D_i\le h,D_j\le h)\le \psi \left(h\right)f_2\left(x\right),$$

where $\psi \left(h\right)⟶0$ as $h⟶0$ and $f_2\left(x\right)$ is a nonnegative functional in $x\in \mathcal{E}$. We assume that the ratio $\psi \left(h\right)/{\varphi }^2\left(h\right)$ is bounded.

(H2) 

On the smoothness of the model

(H2)(i)

There exist constants $\beta >0$ and ${\eta }_1>0$ such that for all $x_1,x_2\in N_x$, a neighborhood of x, we have

$$|\mathrm{\Gamma }(C\mid x_1)-\mathrm{\Gamma }(C\mid x_2)|\le {\eta }_1{d}_{\mathcal{E}}^{\beta }(x_1,x_2).$$

(H2)(ii)

Let $g_2\left(u\right)=\mathrm{Var}({\mathbb{1}}_{\{Y_j\in C\}}\mid X_j=u)$ for $u\in \mathcal{E}$. Assume that $g_2\left(u\right)$ is independent of j and continuous in some neighborhood of x, as $h⟶0$,

$$\underset{\{u:d(x,u)\le h\}}{sup}|g_2\left(u\right)-g_2\left(x\right)|=o\left(1\right),$$

and assume that

$$g_{\nu }\left(u\right)=\mathbb{E}\left({\left|{\mathbb{1}}_{\{Y_i\in C\}}-\mathrm{\Gamma }(C\mid x)\right|}^{\nu }\mid X_i=u\right)$$

is continuous in some neighborhood of x.

(H2)(iii)

Define, for $i\ne j,u,v\in \mathcal{E}$,

$$g(u,v;x)=\mathbb{E}\left(\left({\mathbb{1}}_{\{Y_i\in C\}}-\mathrm{\Gamma }(C\mid x)\right)\left({\mathbb{1}}_{\{Y_j\in C\}}-\mathrm{\Gamma }(C\mid x)\right)\mid X_i=u,X_j=v\right).$$

Assume that $g(u,v;x)$ does not depend on $i,j$ and is continuous in some neighborhood of $(x,x)$.

(H3) 

On the kernel function

(H3)(i)

The kernel function $K(·)$ is supported within $[0,1]$, and there exist constants $0<{\eta }_1\le {\eta }_2<\infty$ such that:

$${\int }_0^{1}K\left(x\right)dx=1,$$

and

$$0<{\eta }_1{\mathbb{1}}_{(0,1)}(·)\le K(·)\le {\eta }_2{\mathbb{1}}_{(0,1)}(·).$$

(H3)(ii)

The kernel $K(·)$ is a positive, differentiable function on $[0,1]$ with derivative $K^{\prime }(·)$ such that

$$-\infty <{\eta }_3<K^{\prime }(·)<{\eta }_4<0.$$

(H4) 

On the classes of sets

(H4)(i)

The metric entropy of the class $\mathcal{C}$ satisfies, for some $1\le p<\infty$,

$${\int }_0^{\infty }\left(log\mathcal{N}(u,{\mathcal{C}}_{\sigma }{,\parallel ·\parallel }_p{)}^{{\displaystyle \frac{1}{2}}}\right)du<\infty ,$$

where

$${\mathcal{C}}_{\sigma }=\{C_1,C_2\in \mathcal{C}:\mathrm{\Gamma }(C_1▵C_2\mid X=x)\le \sigma \}.$$

(H4)(ii)

The class of sets $\mathcal{C}$ is assumed to be of VC-type with the envelope function previously defined. Hence, there are two finite positive constants a and p such that:

$$\mathcal{N}\left({\epsilon ,\mathcal{C},\parallel ·\parallel }_{L_2\left(Q\right)}\right)\le {\left({\displaystyle \frac{{b\parallel F\kappa \parallel }_{L_2\left(Q\right)}}{ϵ}}\right)}^a,$$

for any $ϵ>0$ and each probability measure such that $Q{\left(F\right)}^2<\infty$.

(H5) 

On the dependence of the random variables

(H5)(i)

For some $v>2$ and $\varsigma >1-{\displaystyle \frac{2}{v}}$, we have

$$\sum _{\ell =1}^{\infty }{\ell }^{\varsigma }{\left[\alpha (\ell )\right]}^{1-{\displaystyle \frac{2}{v}}}<\infty .$$

(H5)(ii)

There exists a sequence of positive integers ${\left\{s_n\right\}}_{n\in {\mathbb{N}}^{*}}$ such that, as $n⟶\infty$,

$$s_n⟶\infty ,s_n=\left(\sqrt{n\varphi \left(h\right(x\left)\right)}\right),$$

and

$${\left({\displaystyle \frac{n}{\varphi \left(h\right(x\left)\right)}}\right)}^{{\displaystyle \frac{1}{2}}}\alpha \left(s_n\right)⟶0.$$

(H6) 

On the entropy For large enough n and some $\omega >1$, Kolmogorov’s entropy satisfies:

$${\displaystyle \frac{{(logn)}^2}{n\varphi \left(h_K\right)}}\le m{\psi }_{S_{\mathcal{E}}}{\displaystyle \frac{logn}{n}}\le {\displaystyle \frac{n}{logn}}\varphi \left(h_K\right),$$

(6)

$$\sum _{i=1}^{\infty }exp\left(m(1-\omega ){\psi }_{S_{\mathcal{E}}}{\displaystyle \frac{logn}{n}}\right)$$

(7)

where ${\psi }_{S_{\mathcal{E}}}\left(\epsilon \right):=log{\mathcal{N}}_{\epsilon }\left(S_{\mathcal{E}}\right)$, and ${\mathcal{N}}_{\epsilon }\left(S_{\mathcal{E}}\right)$ is the minimal number of open balls of radius $\epsilon$ of $\mathcal{E}$ needed to cover $S_{\mathcal{E}}$.

(H7) 

Condition on the smoothing parameter The smoothing parameter $h_n$ satisfies:

$$h_n⟶0\mathrm{and}{\displaystyle \frac{logn}{nmin(h_n^2,{\varphi }^2\left(h_n\right))}}⟶0\mathrm{as}n⟶\infty .$$

(H8) 

There exist sequences ${\rho }_n\subset (0,1),\left\{k_{1,n}\right\}\subset {\mathbb{Z}}^{+}$ such that

$$0<\mu \le \nu <\infty ,$$

and the conditions

$$\mu {\varphi }^{-1}\left({\displaystyle \frac{{\rho }_n{k}_{1,n}}{n}}\right)\le {\varphi }_x^{-1}\left({\displaystyle \frac{{\rho }_n{k}_{1,n}}{n}}\right),$$

(8)

and

$${\varphi }_x^{-1}\left({\displaystyle \frac{k_{2,n}}{{\rho }_{n}n}}\right)⟶0,$$

(9)

and

$$min\left\{{\displaystyle \frac{1-{\rho }_n}{4}}{\displaystyle \frac{k_{1,n}}{logn}},{\displaystyle \frac{{(1-{\rho }_n)}^2}{4{\rho }_n}}{\displaystyle \frac{k_{1,n}}{logn}}\right\}>2,$$

(10)

and

$${\displaystyle \frac{logn/n}{min\left\{\mu {\varphi }^{-1}\left(\frac{{\rho }_n{k}_{1,n}}{n}\right),\varphi \left(\mu {\varphi }^{-1}\left(\frac{{\rho }_n{k}_{1,n}}{n}\right)\right)\right\}}}.$$

(11)

Additional/Alternative conditions

(C1) 

For all $t>0$, we have $\varphi \left(t\right)>0$. For all $t\in (0,1)$, the function ${\tau }_0\left(t\right)$ exists, where

$${\tau }_0\left(t\right)=\underset{r⟶0}{lim}{\displaystyle \frac{\varphi \left(rt\right)}{\varphi \left(r\right)}}=\underset{r⟶0}{lim}\mathbb{P}(d_{\mathcal{E}}(x,X)\le rt\mid \mathbb{P}(d_{\mathcal{E}}(x,X)\le t))<\infty .$$

(C2) 

The kernel function $K(·)$ is supported within $[0,1]$, and there exist constants $0<{\vartheta }_2$ and $0<{\vartheta }_1^{\prime }\le {\vartheta }_2^{\prime }<\infty$ such that for $j=1,2$:

$${\int }_0^{1}K\left(x\right)dx=1,K(·)\le {\vartheta }_2{\mathbb{1}}_{[0,1]}(·),{\displaystyle \frac{h_n}{\varphi \left(h_K\left(x\right)\right)}}{\int }_0^1{K}^j\left(x\right){\varphi }^{\prime }\left(\nu h_K\right)d\nu ⟶{\vartheta }_2^{\prime },\mathrm{as}n⟶\infty .$$

2.2. Comments on the Hypothesis

We address a complex theoretical problem within the framework of a nonparametric functional regression model, focusing on functional central limit theorems for set-indexed conditional empirical processes under functionally strong mixing data. Specifically, we utilize k-nearest neighbors (k-NN)-based random (or data-dependent) bandwidths. The investigation begins with Assumption $\left(\mathbf{H}\mathbf{1}\right)$, which serves as a standard condition on the small-ball probability. This condition, adapted from [43], pertains to the case where $\mathcal{E}$ is an infinite-dimensional space. Under this assumption, $\varphi \left(h_k\right)$ converges to 0 exponentially as $n⟶\infty$, and the function ${\varphi }_x(·)$ is employed to control behavior around zero. Notably, the small-ball probability can be approximated as the product of two independent functions, $\tilde{\varphi }(·)$ and $f_1(·)$. For related examples, see [55] for diffusion processes, ref. [56] for Gaussian measures, and ref. [57] for general Gaussian processes. To adjust for the bias in nonparametric estimators, some understanding of the variability of the small-ball probability is typically required when dealing with functional data. This is achieved by assuming:

$${\tau }_0\left(t\right)=\underset{r⟶0}{lim}{\displaystyle \frac{\varphi \left(rt\right)}{\varphi \left(r\right)}}=\underset{r⟶0}{lim}\mathbb{P}(d_{\mathcal{E}}(x,X)\le rt\mid \mathbb{P}(d_{\mathcal{E}}(x,X)\le t))<\infty .$$

Assumption $\left(\mathbf{H}\mathbf{2}\right)$ pertains to the model’s regularity, encompassing mild requirements such as the continuity of certain conditional moments and the usual Lipschitz condition for regression ($\left(\mathbf{H}\mathbf{2}\right)\left(\mathbf{i}\right)$). $\left(\mathbf{H}\mathbf{3}\right)$ is a standard condition in functional nonparametric estimation, primarily concerning the kernel function $K(·)$. Notably, $\left(\mathbf{H}\mathbf{3}\right)\left(\mathbf{i}\right)$ can be substituted with hypothesis $\left(\mathbf{C}\mathbf{2}\right)$ to derive the asymptotic variance. Assumption $\left(\mathbf{H}\mathbf{4}\right)$ is invoked for bounded sets. For $\left(\mathbf{H}\mathbf{4}\right)\left(\mathbf{iii}\right)$, see examples in [58,59] (§4.7), [60] (Theorem 2.6.7), and [61] (§9.1), which provide necessary conditions. For further discussions, see [62] (§3.2). The class of functions is generally assumed to be pointwise measurable, a condition satisfied when $K(·)$ is defined on ${\mathbb{R}}^d$. See, for instance, [63,64,65]. Assumption $\left(\mathbf{H}\mathbf{5}\right)$ concerns the decay rate of the mixing coefficient $\alpha \left(n\right)$, a critical requirement for studying the weak convergence of empirical processes and establishing asymptotic equicontinuity. $\left(\mathbf{H}\mathbf{6}\right)$ addresses topological considerations by controlling the Kolmogorov entropy of the sets $S_{\mathcal{E}}$. This is standard in nonparametric models, aiding in the uniform convergence and consistency of estimators. It quantifies the complexity of function classes using tools such as covering numbers or $\epsilon$-nets. Examples include closed balls in Sobolev spaces, unit balls in Cameron-Martin spaces, and compact subsets in Hilbert spaces with projection semi-metrics. See [33,66,67] for further details. For specific functional spaces $\mathcal{E}$ and subsets $S_{\mathcal{E}}$, it is noted in [33] that ${\psi }_{S_{\mathcal{E}}}(log\left(n\right)/n)\gg log\left(n\right)$. Assumption $\left(\mathbf{H}\mathbf{7}\right)$ is fundamental, as convergence would not hold without it. This condition adapts $\left(\mathbf{H}\mathbf{8}\right)$ to the context of functional conditional processes in the k-NN framework.

3. Main Results

We adopt the notation $Z\stackrel{\mathcal{D}}{=}\mathcal{N}(\mu ,{\sigma }^2)$ to denote that the random variable Z follows a normal distribution with mean $\mu$ and variance ${\sigma }^2$. Convergence in distribution is denoted by $\stackrel{\mathcal{D}}{\to }$, while convergence in probability is indicated by $\stackrel{\mathbb{P}}{\to }$. Let ${\left(z_n\right)}_{n\in \mathbb{N}}$ be a sequence of real random variables. We say that $z_n$ converges almost completely (a.co.) to zero if and only if:

$$\forall ϵ>0,\sum _{n=1}^{\infty }\mathbb{P}\left(\right|z_n|>ϵ)<\infty .$$

Moreover, for a sequence ${\left(u_n\right)}_{n\in {\mathbb{N}}^{*}}$ of positive real numbers, we say $z_n=O_{a.co}\left(u_n\right)$ if and only if:

$$\exists ϵ>0,\sum _{n=1}^{\infty }\mathbb{P}\left(\right|z_n|>ϵu_n)<\infty .$$

This type of convergence implies both almost-sure convergence and convergence in probability.

3.1. Uniform in the Number of Neighbors Consistency

Now, we can state the main results of this section concerning the k-NN functional estimator. Recall the bandwidths $k_{1,n}$ and $k_{2,n}$ given in the condition (H8).

Theorem 1.

Consider the hypotheses (H1)(i), (H2)(i), (H3)(i), (H4)(i), (H4)(ii), (H6), (H7), and(H8). Let $(X_t,Y_t)$ be a geometrically strong mixing sequence with $\beta >2$. Furthermore, assume that for every $C\in \mathcal{C}$, as $n⟶\infty$, we have:

$$\begin{array}{cc}\hfill \underset{C\in \mathcal{C}}{sup}\underset{k_{1,n}\le k\le k_{2,n}}{sup}\underset{x\in S_{\mathcal{E}}}{sup}\left|{\mathrm{\Gamma }}_n(C,x)-\mathrm{\Gamma }(C,x)\right|=& O\left({\varphi }^{-1}\left({\displaystyle \frac{k_{2,n}}{{\rho }_{n}n}}\right)\right)\hfill \\ \hfill & +O_{a.co}\left(\sqrt{{\displaystyle \frac{{\psi }_{S_x}\left(\frac{logn}{n}\right)}{n\varphi \left(\mu {\varphi }^{-1}\left(\frac{{\rho }_n{k}_{1,n}}{n}\right)\right)}}}\right).\hfill \end{array}$$

3.2. Uniform Central Limit Theorems

The main objective of this section is to investigate the central limit theorems for the functional conditional empirical process defined by (4).

Theorem 2.

Assume that hypotheses (H1)(ii), (H1)(iii), (H2)(i), (H2)(ii), (C2),  (H5), and(H8) hold, and that $(X_i,Y_j)$ is geometrically strong mixing with $\beta >2$. Furthermore, suppose that $n\varphi \left(h_n\right)⟶\infty$ as $n⟶\infty$. For $m\ge 1$ and $C_1,\dots ,C_m\in \mathcal{C}$, if the smoothing parameter k satisfies the condition:

$$k{\left({\varphi }^{-1}\left({\displaystyle \frac{k}{2}}\right)\right)}^{2\gamma }⟶0\mathrm{as}n⟶\infty ,$$

then the sequence

$${\left\{\mathrm{\Lambda }(C_i,x)\right\}}_{i=1,\dots ,m}\stackrel{\mathcal{D}}{⟶}\mathcal{N}(0,\Sigma ),$$

where $\Sigma ={\sigma }_{ij}\left(x\right)$ for $i,j=1,\dots ,m$ is the covariance matrix given by:

$${\sigma }_{ij}\left(x\right)={\displaystyle \frac{{\vartheta }_2^{\prime }\mathrm{\Gamma }(C_i{C}_j,x)-\mathrm{\Gamma }(C_i,x)\mathrm{\Gamma }(C_j,x)}{{\vartheta }_1^{\prime }{f}_1\left(x\right)}},$$

whenever $f_1\left(x\right)>0$.

We limit our discussion to the beta mixing setting in the theorems presented below.

Theorem 3.

Assume that conditions(H3)(i),(H4)(i)–(H5)(i), and(H8)are satisfied. Then the family $\{{\mathrm{\Lambda }}_n\left(C\right):C\in \mathcal{C}\}$ is dense in the space $\left({\mathcal{L}}^{\infty }\left(\mathcal{C}\right),{\parallel ·\parallel }_{\mathcal{C}}\right)$, which can be expressed as

$$\underset{\sigma ⟶0}{lim}\underset{n⟶\infty }{lim}sup\mathbb{P}\left(\underset{C\in {\mathcal{C}}_{\sigma }}{sup}\left|{\mathrm{\Lambda }}_n\left(C\right)\right|>ϵ\right)=0,$$

where

$${\mathcal{C}}_{\sigma }=\{C_1,C_2\in \mathcal{C}:\mathrm{\Gamma }(C_1▵C_2\mid X=x)\le \sigma \}.$$

Theorem 4.

Under the conditions of Theorems 2 and 3, and with the additional conditions(H2)and(H4)(iii), the process:

$$\left\{{\mathrm{\Lambda }}_n(C,x):C\in \mathcal{C}\right\}$$

converges in law to a Gaussian process:

$$\left\{\mathrm{\Lambda }(C,x):C\in \mathcal{C}\right\},$$

which admits a version with uniformly bounded and uniformly continuous paths with respect to the ${\parallel ·\parallel }_2$-norm, with covariance ${\sigma }_{ij}\left(x\right)$ given in Theorem 2.

Remark 1.

An important consideration when working with infinite-dimensional data, particularly functional data, is the need to account for the local structures inherent in the data. However, the bandwidth (denoted by h) used in kernel-based estimators is fixed and does not depend on the covariate component x, meaning it fails to capture such local features. An alternative and widely used approach in nonparametric and semiparametric statistics incorporates ideas from k-nearest neighbors (kNN). This method is especially advantageous when the data exhibit local structures, as kNN-based estimators use a location-dependent bandwidth (i.e., a bandwidth that varies with x), allowing them to effectively capture local features. In other words, kNN provides location-adaptive methods. The kNN-based estimators offer at least two key practical advantages over kernel-based methods. First, although the number of neighbors k is fixed, the bandwidth $H_{n,k}\left(X\right)$ varies with x, which provides the local-adaptive property of kNN estimators, thus allowing adaptation to heterogeneous designs. Second, the computational cost associated with selecting the smoothing parameter k is lower than that of selecting h, since k takes values from the finite set $\{1,2,\cdots ,n\}$. However, the theoretical analysis of kNN statistics presents greater challenges, primarily because $H_{n,k}\left(X\right)$ is a random variable that depends on $X_i$ (for $i=1,\cdots ,n$), introducing additional technical complexities in the proofs. It is important to note that this paper extends the work of [29].

3.3. The Parameter k Selection Criterion

Various methods have been developed to establish bandwidth selection rules for nonparametric kernel estimators, with a focus on achieving asymptotic optimality, especially for the Nadaraya–Watson regression estimator. Notable contributions in this area include [68,69,70]. This parameter has been appropriately chosen either in finite-dimensional settings or infinite-dimensional settings, aiming to highlight good practical performance. To define the leave-out-$(X_i,Y_i)$ estimator for the regression function, we use the following expression:

$$\begin{array}{c}\hfill {\mathrm{\Gamma }}_n(C,x,k)={\displaystyle \frac{{\sum }_{i=1,i\ne j}^n{\mathbb{1}}_{\{Y_i\in C\}}K\left(\frac{d_{\mathcal{E}}(x,X_i)}{H_{n,k}\left(x\right)}\right)}{{\sum }_{i=1}^{n}K\left(\frac{d_{\mathcal{E}}(x,X_i)}{H_{n,k}\left(x\right)}\right)}},\end{array}$$

(12)

To minimize the quadratic loss function, we consider the following criterion for some known non-negative weight function $\mathcal{W}$:

$$\begin{array}{c}\hfill CV(C,k)={\displaystyle \frac{1}{n}}\sum _{j=1}^n{\left({\mathbb{1}}_{\{Y_j\in C\}}-{\mathrm{\Gamma }}_n(C,X_j,k)\right)}^2\mathcal{W}\left(X_j\right).\end{array}$$

(13)

This criterion is based on the practical approach developed by [69], where the smoothing parameter is chosen by minimizing the above criterion. The goal is to select ${\widehat{k}}_n$ that minimizes the following expression:

$$\underset{C\in \mathcal{C}}{sup}CV(C,k).$$

In our analysis, we aim to derive the asymptotic properties of our estimate, even when the bandwidth parameter is treated as a random variable, as suggested in the previous equation. We can refine (13) by compensating with the following:

$$\begin{array}{c}\hfill CV(C,k)={\displaystyle \frac{1}{n}}\sum _{j=1}^n{\left({\mathbb{1}}_{\{Y_j\in C\}}-{\mathrm{\Gamma }}_n(C,X_j,k)\right)}^2\widehat{\mathcal{W}}(X_j,x).\end{array}$$

(14)

In practice, the global uniform weights $\mathcal{W}\left(X_j\right)=1$ are typically used for $j=1,\cdots ,n$, and the local weights are defined as:

$$\widehat{\mathcal{W}}(X_j,x)=\left\{\begin{array}{cc}1\hfill & \mathrm{if}d(X_j,x)\le h,\hfill \\ \hfill \\ 0\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$

In this work, we adopt the popular method of cross-validation to select the bandwidth; though this approach can be extended to other bandwidth selection methods, such as those based on Bayesian ideas, as discussed in [71].

Remark 2.

Establishing the asymptotic theory for a random, cross-validated bandwidth requires a specialized stochastic equicontinuity argument. Cross-validation, as employed by [72], is used to investigate whether two functions (both unconditional and conditional) are equal in mixed categorical and continuous settings. Although cross-validation performs optimally for estimation, its optimality may not extend to nonparametric kernel testing. In the context of testing a parametric model for the conditional mean function against a nonparametric alternative, ref. [73] introduced an adaptive-rate-optimal rule. An alternative approach to bandwidth selection is proposed by [74], who leverage the Edgeworth expansion of the test’s asymptotic distribution to identify the bandwidth that maximizes test power while controlling its size.

4. Application

4.1. Statistical Classification

In this section, we apply the findings obtained in the previous sections to the problem of statistical classification. Given a sample of random elements $(X_1,Y_1),\dots ,(X_n,Y_n)$ drawn from the joint distribution of $(X,Y)$, where X takes values in a space $\mathcal{E}$ and Y in ${\mathbb{R}}^d$. In our classification, the objective is to predict the integer-valued label Y based on the covariate vector X. More formally, we aim to find a function (classifier) $\theta :\mathcal{E}⟶{\mathbb{R}}^d$ for which the probability of misclassification error (incorrect prediction), i.e., $\mathbb{P}\left(\theta \right(X)\ne Y)$, is minimized. Let

$${\tau }_k\left(x\right)=\mathbb{P}(Y=k\mid X=x),x\in \mathcal{E},1\le k\le n.$$

Demonstrating that the optimal classifier, i.e., the one with the minimum probability of error, is given by

$${\theta }_B\left(x\right)=arg\underset{1\le k\le n}{max}{\tau }_k\left(x\right),$$

i.e., the best classifier, ${\theta }_B$ satisfies

$$\underset{1\le k\le n}{max}{\tau }_k\left(x\right)={\tau }_{{\theta }_B\left(x\right)}\left(x\right).$$

As ${\theta }_B$ is unknown, the data are utilized to construct estimates of ${\theta }_B$. Specifically, let ${\mathbb{D}}_n=(X1,Y_1),\dots ,(X_n,Y_n)$ represent a random sample from the distribution of $(X,Y)$, where each $(X_i,Y_i)$ is fully observable. Let ${\widehat{\theta }}_n$ be any sample-based classifier. In other words, ${\widehat{\theta }}_n\left(X\right)$ is the predicted value of Y, based on ${\mathbb{D}}_n$ and X. Let

$$L_n\left({\widehat{\theta }}_n\right)=\mathbb{P}({\widehat{\theta }}_n\left(X\right)\ne Y\mid {\mathbb{D}}_n),$$

be the conditional probability of error of the sample-based classifier ${\widehat{\theta }}_n$. Then ${\widehat{\theta }}_n$ is said to be consistent if $L_n\left({\widehat{\theta }}_n\right)⟶L_n\left({\theta }_n\right)=\mathbb{P}({\theta }_B\left(X\right)\ne Y)$ as $n\to \infty$, for $k=1,\dots ,n$. Let ${\widehat{\tau }}_k\left(x\right)$ be any sample-based estimator of ${\tau }_k\left(x\right)=\mathbb{P}(Y=k\mid X=x)$ and define the classification rule ${\widehat{\theta }}_n$ by

$${\widehat{\theta }}_n\left(x\right)=arg\underset{1\le k\le n}{max}{\widehat{\tau }}_k\left(x\right).$$

In other words, ${\widehat{\theta }}_n$ satisfies

$$\underset{1\le k\le n}{max}{\widehat{\tau }}_k\left(x\right)={\widehat{\tau }}_{{\widehat{\theta }}_n\left(x\right)}\left(x\right),$$

To show

$$L_n\left({\widehat{\theta }}_n\right)-L_n\left({\theta }_B\right)⟶0,$$

we put for all ${\widehat{\tau }}_k\left(x\right)$, it is sufficient to show that

$${\widehat{\tau }}_k\left(x\right)-{\tau }_k\left(x\right)⟶0$$

We can rewrite the conditional distribution as

$$\begin{array}{c}\hfill {\mathrm{\Gamma }}_n(C,x)={\displaystyle \frac{{\displaystyle \sum _{i=1}^n{\widehat{\tau }}_k\left(x\right){\mathbb{1}}_{\{Y_i\in C\}}K\left(\frac{d_{\mathcal{E}}(x,X_i)}{H_{n,k}\left(x\right)}\right)}}{{\displaystyle \sum _{i=1}^{n}K\left(\frac{d_{\mathcal{E}}(x,X_i)}{H_{n,k}\left(x\right)}\right)}}}.\end{array}$$

(15)

Theorem 5.

Under the conditions of Theorem 1, we have the convergence

$$L_n\left({\widehat{\theta }}_n\right)-L_n\left({\theta }_B\right)⟶0.$$

4.2. Testing Conditional Independence

Conditional independence is an essential concept that unifies many diverse aspects of statistical inference; see [75]. The tasks of quantifying and detecting conditional dependence underlie a wide range of statistical problems, including limit theorems, Markov chain theory, sufficiency, and causality [76], among others. Conditional independence also plays a pivotal role in graphical models [77], causal inference [78], and artificial intelligence [79]; see also [80] for more recent developments. The notion of treating conditional independence as an abstract concept with a dedicated calculus was first introduced by [76], who demonstrated that many results and theorems related to statistical ideas (such as ancillarity, sufficiency, and causality) can be viewed as specific applications of the fundamental properties of conditional independence, extended to include both stochastic and non-stochastic variables. For more details, see [81].

Let ${\mathcal{C}}_1,{\mathcal{C}}_2$ be two classes of sets. In this section, we focus on a sample of random elements.

$$\left(X_1,Y_{1,1},Y_{1,2}\right),\dots ,\left(X_n,Y_{n,1},Y_{n,2}\right),$$

which are i.i.d. copies of $(X,Y_1,Y_2)$ taking values in $\mathcal{E}\times {\mathbb{R}}^{d_1}\times {\mathbb{R}}^{d_2}$. For $\left(C_1,C_2\right)\in {\mathcal{C}}_1\times {\mathcal{C}}_2$, define

$$\begin{array}{c}\hfill {\mathbb{G}}_n(C_1\times C_2,x)={\displaystyle \frac{{\displaystyle \sum _{i=1}^n{\mathbb{1}}_{\{Y_{i,1}\in C_1\}}{\mathbb{1}}_{\{Y_{i,2}\in C_2\}}K\left(\frac{d_{\mathcal{E}}(x,X_i)}{H_{n,k}\left(x\right)}\right)}}{{\displaystyle \sum _{i=1}^{n}K\left(\frac{d_{\mathcal{E}}(x,X_i)}{H_{n,k}\left(x\right)}\right)}}},\end{array}$$

(16)

$$\begin{array}{c}\hfill {\displaystyle {\mathbb{G}}_{n,1}(C_1,x)=\frac{{\displaystyle \sum _{i=1}^n{\mathbb{1}}_{\{Y_{i,1}\in C_1\}}K\left(\frac{d_{\mathcal{E}}(x,X_i)}{H_{n,k}\left(x\right)}\right)}}{{\displaystyle \sum _{i=1}^{n}K\left(\frac{d_{\mathcal{E}}(x,X_i)}{H_{n,k}\left(x\right)}\right)}},}\end{array}$$

(17)

$$\begin{array}{c}\hfill {\mathbb{G}}_{n,2}(C_2,x)={\displaystyle \frac{{\displaystyle \sum _{i=1}^n{\mathbb{1}}_{\{Y_{i,2}\in C_2\}}K\left(\frac{d_{\mathcal{E}}(x,X_i)}{H_{n,k}\left(x\right)}\right)}}{{\displaystyle \sum _{i=1}^{n}K\left(\frac{d_{\mathcal{E}}(x,X_i)}{H_{n,k}\left(x\right)}\right)}}}.\end{array}$$

(18)

We investigate the following empirical processes for $\left(C_1,C_2\right)\in {\mathcal{C}}_1\times {\mathcal{C}}_2$:

$${\widehat{\nu }}_n(C_1,C_2,x)=\sqrt{n\varphi \left(h_n\right)}\left({\mathbb{G}}_n(C_1\times C_2,x)-\mathbb{E}\left[{\mathbb{G}}_n(C_1,x)\right]\mathbb{E}\left[{\mathbb{G}}_n(C_2,x)\right]\right),$$

(19)

$${\breve{\nu }}_n(C_1,C_2,x)=\sqrt{n\varphi \left(h_n\right)}\left({\mathbb{G}}_n(C_1\times C_2,x)-{\mathbb{G}}_{n,1}(C_1,x){\mathbb{G}}_{n,2}(C_2,x)\right).$$

(20)

Observe that

$$\begin{array}{ccc}\hfill {\breve{\nu }}_n(C_1,C_2,x)& =& \sqrt{n\varphi \left(h_n\right)}\left({\mathbb{G}}_n(C_1\times C_2,x)-\mathbb{E}\left[{\mathbb{G}}_n(C_1,x)\right]\mathbb{E}\left[{\mathbb{G}}_n(C_2,x)\right]\right)\hfill \\ & & +\sqrt{n\varphi \left(h_n\right)}\mathbb{E}\left[{\mathbb{G}}_n(C_2,x)\right]\left({\mathbb{G}}_n(C_1,x)-\mathbb{E}\left[{\mathbb{G}}_n(C_1,x)\right]\right)\hfill \\ & & -\sqrt{n\varphi \left(h_n\right)}{\mathbb{G}}_n(C_1,x)\left({\mathbb{G}}_n(C_2,x)-\mathbb{E}\left[{\mathbb{G}}_n(C_2,x)\right]\right).\hfill \end{array}$$

Hence,

$$\begin{array}{ccc}\hfill {\breve{\nu }}_n(C_1,C_2,x)& \stackrel{d}{=}& \sqrt{n\varphi \left(h_n\right)}\left({\mathbb{G}}_n(C_1\times C_2,x)-\mathbb{E}\left[{\mathbb{G}}_n(C_1,x)\right]\mathbb{E}\left[{\mathbb{G}}_n(C_2,x)\right]\right)\hfill \\ & & +\sqrt{n\varphi \left(h_n\right)}\mathbb{E}\left[{\mathbb{G}}_n(C_2,x)\right]\left({\mathbb{G}}_n(C_1,x)-\mathbb{E}\left[{\mathbb{G}}_n(C_1,x)\right]\right)\hfill \\ & & -\sqrt{n\varphi \left(h_n\right)}\mathbb{E}\left[{\mathbb{G}}_n(C_1,x)\right]\left({\mathbb{G}}_n(C_2,x)-\mathbb{E}\left[{\mathbb{G}}_n(C_2,x)\right]\right)\hfill \\ & =& {\widehat{\nu }}_n(C_1,C_2,x)+\mathbb{E}\left[{\mathbb{G}}_n(C_2,x)\right]{\tilde{\nu }}_n(C_1,x)-\mathbb{E}\left[{\mathbb{G}}_n(C_1,x)\right]{\tilde{\nu }}_n(C_2,x),\hfill \end{array}$$

(21)

where ${\tilde{\nu }}_n(C_k,x)$ is defined analogously.

It can be shown that for $\left(A_1,B_1\right),\left(A_2,B_2\right)\in {\mathcal{C}}_1\times {\mathcal{C}}_2$,

$$\begin{array}{c}\hfill {\displaystyle \mathrm{cov}\left({\widehat{\nu }}_n(A_1,B_1,x),{\widehat{\nu }}_n(A_2,B_2,x)\right)}\\ \hfill & =& {\displaystyle \frac{C_2}{C_1^2{f}_1\left(x\right)}}\left(\mathbb{E}\left[{\mathbb{1}}_{\{Y\in A_1\cap A_2\}}\mid X=x\right]-\mathbb{E}\left[{\mathbb{1}}_{\{Y\in A_1\}}\mid X=x\right]\mathbb{E}\left[{\mathbb{1}}_{\{Y\in A_2\}}\mid X=x\right]\right)\hfill \\ & & \times \left(\mathbb{E}\left[{\mathbb{1}}_{\{Y\in B_1\cap B_2\}}\mid X=x\right]-\mathbb{E}\left[{\mathbb{1}}_{\{Y\in B_1\}}\mid X=x\right]\mathbb{E}\left[{\mathbb{1}}_{\{Y\in B_2\}}\mid X=x\right]\right),\hfill \end{array}$$

(22)

provided $f_1\left(x\right)>0$. Although the decomposition in (21) suggests the structure of ${\breve{\nu }}_n(C_1,C_2,x)$, its covariance is more delicate to derive. Let $\{\widehat{\nu }(C_1,C_2,x):(C_1,C_2)\in {\mathcal{C}}_1\times {\mathcal{C}}_2\}$ be a Gaussian process whose covariance is given by (22). Define the following limiting process for $\left(C_1,C_2\right)\in {\mathcal{C}}_1\times {\mathcal{C}}_2$:

$$\breve{\nu }(C_1,C_2,x)=\widehat{\nu }\left(C_1,C_2,x\right)+\mathbb{G}(C_2,x)\tilde{\nu }\left(C_1,x\right)-\mathbb{G}(C_1,x)\tilde{\nu }\left(C_2,x\right).$$

We aim to test the null hypothesis

$${\mathcal{H}}_0:Y_1\mathrm{and}{Y}_2\mathrm{are}\mathrm{conditionally}\mathrm{independent}\mathrm{given}X=x,$$

against the alternative

$${\mathcal{H}}_1:Y_1\mathrm{and}{Y}_2\mathrm{are}\mathrm{conditionally}\mathrm{dependent}.$$

To this end, one may use the following test statistics:

$$\begin{array}{ccc}\hfill S_{1,n}& =& \underset{(C_1,C_2)\in {\mathcal{C}}_1\times {\mathcal{C}}_2}{sup}\left|{\widehat{\nu }}_n(C_1,C_2,x)\right|,\hfill \end{array}$$

(23)

$$\begin{array}{ccc}\hfill S_{2,n}& =& \underset{(C_1,C_2)\in {\mathcal{C}}_1\times {\mathcal{C}}_2}{sup}\left|{\breve{\nu }}_n(C_1,C_2,x)\right|.\hfill \end{array}$$

(24)

By combining Theorem 4 with the continuous mapping theorem, we obtain the following result.

Theorem 6.

Under the conditions of Theorem 4, as $n\to \infty$, it holds that

$$\begin{array}{ccc}\hfill S_{1,n}& ⟶& \underset{(C_1,C_2)\in {\mathcal{C}}_1\times {\mathcal{C}}_2}{sup}\left|\widehat{\nu }\left(C_1,C_2,x\right)\right|,\hfill \end{array}$$

(25)

$$\begin{array}{ccc}\hfill S_{2,n}& ⟶& \underset{(C_1,C_2)\in {\mathcal{C}}_1\times {\mathcal{C}}_2}{sup}\left|\breve{\nu }\left(C_1,C_2,x\right)\right|.\hfill \end{array}$$

(26)

5. Concluding Remarks

In this paper, we present new limit theorems for an improved version of the set-indexed conditional empirical process, with particular attention to the interplay between mixing conditions and functional aspects of the data. A central feature of our contribution is the incorporation of the widely recognized k-nearest neighbors (k-NN) approach for estimating the regression function. Building on the results in [29], we establish analogous theorems on weak convergence in probability and asymptotic normality. Furthermore, our findings bolster existing research by refining the density properties inherent to the process under consideration.

The k-NN algorithm, valued for its non-parametric nature and relative simplicity, is well-suited to both classification and regression tasks. One principal advantage of applying k-NN in time series settings lies in its capacity to function effectively without strict assumptions regarding the underlying data distribution, thus making it particularly appropriate for non-stationary data. In addition to its methodological adaptability, k-NN is notable for its computational tractability and straightforward implementation. Consequently, the insights provided in this study hold the potential to guide future methodological developments. We also advocate for the utility of bootstrap procedures in functional data contexts, with the functional form of the wild bootstrap illustrating a promising avenue for selecting smoothing parameters. Nevertheless, the theoretical underpinnings that justify this bandwidth selection strategy via the functional wild bootstrap remain an open question.

It is widely recognized that bias constitutes a key challenge in kernel smoothing. Although local linear fitting offers a theoretical avenue for bias mitigation, there is a pressing need for more extensive investigation within the realm of nonparametric functional data analysis in order to comprehensively address bias-related issues. Empirical evidence suggests that bias correction strategies can markedly improve estimation accuracy. At the same time, further theoretical rigor is essential for understanding and controlling the asymptotic properties of these enhanced approaches. To the best of our knowledge, such formal examinations are still underrepresented in nonparametric functional data analysis, particularly in the context of locally stationary processes. Against this backdrop, a promising objective is to refine estimation procedures through the integration of local linear smoothing with established bias-correction techniques. Contributions from [82] and related foundational works [83] offer a natural basis for developing such integrative methods. Extending our current framework to simultaneously incorporate k-NN and single index models could generate more robust and practically relevant outcomes, thereby paving the way for subsequent empirical investigations.

It is worth noting that the notion of mixing, while commonly adopted due to its relative tractability, may be ill-suited for data exhibiting substantial dependence. Extending non-parametric functional methodologies to more general dependence structures remains an underexplored area. In this regard, ergodic frameworks avoid the stringent requirements of strong mixing assumptions and the intricate probabilistic arguments they necessitate. Nonetheless, adapting our approach to accommodate functional ergodic data would involve considerable mathematical complexity and thus exceeds the present scope.