Weak Convergence of the Conditional Set-Indexed Empirical Process for Missing at Random Functional Ergodic Data

Abstract

This work examines the asymptotic characteristics of a conditional set-indexed empirical process composed of functional ergodic random variables with missing at random (MAR). This paper’s findings enlarge the previous advancements in functional data analysis through the use of empirical process methodologies. These results are shown under specific structural hypotheses regarding entropy and under appealing situations regarding the model. The regression operator’s asymptotic $(1-\alpha )$-confidence interval is provided for $0<\alpha <1$ as an application. Additionally, we offer a classification example to demonstrate the practical importance of the methodology.

1. Introduction

There are several strategies for solving problems in statistics, among which empirical process techniques are considered the best. Historically, many limit theorems for the empirical process have been established in finite dimension frameworks (see, e.g., Refs. [1,2,3] for exhaustive, self-contained texts with a variety of statistical applications) together under mixing conditions and independent identically distributed framework, in the setting of independents variables [4] characterized modulo measurability, the classes $\mathcal{C}$ of sets for which the Glivenko–Cantelli theorem holds, we may also cite Refs. [5,6,7,8,9,10,11,12,13,14,15]. Under various mixing conditions, empirical processes based on dependent data have been investigated; for instance, the authors of Ref. [16] established the asymptotic normality of sequences undergoing $phi$-mixing. Regarding these areas of investigation concerning an alternative form of mixing, it is possible to refer to Refs. [17,18,19,20]. Nevertheless, the author of [21] identified a bracketing condition that could occur due to vigorous mixing. The function-indexed empirical procedure for beta-mixing sequences was investigated by Ref. [22]. Uniform convergence and asymptotic normality of a set-indexed conditional empirical process within a strictly stationary and strong mixing framework have been established by Ref. [23]. Over the past few decades, there has been a growing interest in the statistical literature regarding matters concerning functional random variables, which are variables with values that exist in an infinite-dimensional space. As is the case, for example, in meteorology, medicine, satellite imagery, and numerous other scientific disciplines, the proliferation of data collected on an ever-increasingly precise temporal and spatial grid has inspired the development of this research topic. Numerous complex theoretical and numerical inquiries were thus engendered by the statistical modeling of these data, which were perceived as stochastic functions. The monographs of Refs. [24,25] provide comprehensive surveys of functional data analysis, encompassing both theoretical and practical aspects. These monographs discuss linear models for random variables that take values in a Hilbert space, scalar-on-function and function-on-function linear models, parametric discriminant analysis, and functional principle component analysis, respectively. To access the most recent findings on FDA and related subjects, we may consult the bibliographic reviews provided by sources such as Refs. [26,27,28,29,30,31], among others. For scalar-on-function nonlinear regression models, the authors of [32] emphasized nonparametric techniques, particularly kernel-type estimation. Such tools were subsequently expanded to include discrimination and classification analysis. An intriguing statistical concept that was extended to the functional data framework was examined by Ref. [33]. These concepts included the portmanteau test, change detection, and goodness-of-fit tests. Good overviews of this literature can be found in Refs. [20,34,35,36,37,38,39,40,41], and, more recently, Ref. [42] gave the first results of the conditional set-indexed empirical process in functional data. Considerable effort has been devoted to developing a convergence theory for empirical processes involving functional random variables, although these topics are well beyond the purview of the paper discussed in Ref. [23]. A theoretical framework of this nature is imperative for contemporary statistical analysis. For over six decades, functional data analysis has been acknowledged in the statistical literature and has since become the focus of numerous works. We observe the extreme limitedness of the outcomes produced by empirical processes utilizing functional frameworks. We may refer for recent references to Refs, [43,44,45,46,47], who achieved numerous valuable outcomes regarding set-indexed conditional empirical processes inside the functional setting of the ergodic framework. One should avoid overlooking the possibility that some pairings of observations may be incomplete in numerous practical applications, including sampling surveys, pharmaceutical tracing tests, and reliability tests. Such instances are commonly referred to as “missing data”. Others in the fields of data science and analytics will attest to the fact that missing data is a common issue. MAR (Missing At Random) indicates that while there may be systematic differences between the missing and observed values, these discrepancies can be fully accounted for by other observed variables. The situation changes significantly when predictors are present; for instance, the authors of [48,49,50,51,52,53,54,55,56,57,58] provide some examples of this in finite dimensionality, as recent references to Refs. [59,60]. In a recent study, the authors of [61] examined the linear quantile regression model in the presence of missing response data that occur randomly. The study utilized the inverse probability weight method. The authors developed a mathematical equation for estimating unknown parameters using quantile regression. They also introduced a standard estimator for quantile regression. Simultaneously, they formulated the empirical likelihood (EL) ratio function for the unknown parameter and established a maximum EL estimator for the unknown parameter. There is a scarcity of work that examines the statistical characteristics of functional nonparametric models for missing data. The kernel estimator of the conditional quantile was introduced by Ref. [62] under the assumptions of ergodicity and random censorship. The author also demonstrated strong consistency (with rate) and defined the asymptotic distribution of the estimator. Additionally, they applied the estimator to forecast the peak electricity demand interval using smart meter data, details of which have been omitted. In their study, the authors of [63] developed a type of estimator for the regression operator in the context of functional stationary ergodic data with missing at random (MAR) responses. They also established the asymptotic properties of the estimator, including its convergence rate in probability and asymptotic normality. For further references, we suggest consulting Refs. [64,65].

Our findings extend upon a prior study [44] by establishing more precise limits under less stringent limitations. This offers a new perspective of the empirical processes theory for random variables with general dependencies. This work addresses a problem that has not been thoroughly examined thus far. The framework of ergodic functional data was introduced by Ref. [66], who established consistencies with rates along with the asymptotic normality of the regression function estimate and provided some examples. For recent papers on the subject, we refer to Ref. [43], where the authors extended Ref. [66] to a more general framework. Some motivations to consider ergodic dependence structure in the data rather than a mixing one are discussed in Refs. [67,68].

The objective of this study is to enhance the development of a practical methodology for addressing MAR samples in functional nonparametric situations. We want to examine the estimation of conditional set-indexed empirical processes in the presence of both missing at random (MAR) data and ergodicity.

The structure of this paper is outlined as follows. In Section 2, we introduce the notation and definitions, along with the conditional empirical process. Our main results are presented in Section 3. Section 3.1 is dedicated to discussing the procedure for selecting the bandwidth. In Section 4, we apply our main result to classification. Concluding remarks and potential future developments are discussed in Section 5. To maintain a smooth presentation flow, all proofs are consolidated in Appendix A.

2. The Set Indexed Conditional Empirical Process

To enhance clarity, let us delve into the definition of the ergodic property for processes. Consider a measurable space $(S,\mathcal{J})$, and denote by $S^{\mathbb{N}}$ the space of all functions $s:\mathbb{N}\to S$. If $s_j$ represents the value of the function s at $j\in \mathbb{N}$, define $H_j$ as the j-th coordinate map, i.e., $H_j\left(s\right)=s_j$. Now, consider $H_j^{-1}\left(\mathcal{J}\right)$ for $j\in \mathbb{N}$; a random process $Z=Z_j:j\in \mathbb{N}$ can be viewed as a random variable defined on the probability space $(\Omega ,\mathcal{A},\mathbb{P})$, taking values in $(S^{\mathbb{N}},{\mathcal{J}}^{\mathbb{N}})$. For any $B\in \mathcal{F}$, a set is termed invariant if there exists a set $\mathcal{A}\in {\mathcal{J}}^{\mathbb{N}}$ such that $B=(Z_n,Z_{n+1},\dots )\in \mathcal{A}$ holds for every $n\ge 1$. The process Z is then considered ergodic when, for any invariant set B, we have $\mathbb{P}\left(B\right)=0$ or $\mathbb{P}(\Omega \mid B)=0$. As per the ergodic theorem, it is well-known that for a stationary ergodic process Z, the following convergence holds almost surely:

$$\underset{n\to \infty }{lim}\frac{1}{n}\sum _{i=1}^n{Z}_i=\mathbb{E}\left(Z_1\right),\mathrm{almost}\mathrm{surely}.$$

(1)

Therefore, the ergodic property in our setting is formulated based on the statement (1). We consider a sample of random elements $(X_1,Y_1),\dots ,(X_n,Y_n)$, each drawn from the joint distribution of $(X,Y)$, where X takes values in a space $\mathcal{E}$ and Y in ${\mathbb{R}}^d$. The functional space $\mathcal{E}$ is endowed with a semi-metric $d_{\mathcal{E}}(·,·)$. Our goal is to investigate the relationships between X and Y by estimating functional operators associated with the conditional distribution of Y given X. One such operator is the regression operator for a measurable set C in a class of sets $\mathcal{C}$:

$$\mu (C\mid x)=\mathbb{E}\left({\mathbb{1}}_{\{Y\in C\}}\mid X=x\right).$$

To address this, we employ a Nadaraya–Watson-type conditional empirical distribution, as proposed by Refs. [42,44,69,70]. We introduce the term MAR (Missing mechanism with MAR) for the response variable. In an available incomplete sample of size from $(X,Y,\delta )$, denoted as $(X_i,Y_i,{\delta }_i),1\le i\le n$, $X_i$ is fully observed, ${\delta }_i=1$ if $Y_i$ is observed, and ${\delta }_i=0$ otherwise. The Bernoulli random variable $\delta$ satisfies:

$$\mathbb{P}(\delta =1\mid X=x;Y=y)=\mathbb{P}(\delta =1\mid X=x)=\mathcal{P}\left(x\right),$$

where $\mathcal{P}\left(x\right)$ is a function operator, termed the conditional probability of observing the response given the predictor, often unknown. This mechanism implies that $\delta$ and Y are conditionally independent given X, akin to the finite-dimensionality case in Ref. [48].

The Nadaraya–Watson-type conditional empirical distribution function is given by:

$$\begin{array}{c}\hfill {\mu }_n(C,x)=\frac{{\displaystyle \sum _{i=1}^n{\delta }_i{\mathbb{1}}_{\{Y_i\in C\}}K\left(\frac{d_{\mathcal{E}}(x,X_i)}{h_n}\right)}}{{\displaystyle \sum _{i=1}^n{\delta }_{i}K\left(\frac{d_{\mathcal{E}}(x,X_i)}{h_n}\right)}},\end{array}$$

(2)

where $K(·)$ is a real-valued kernel function from $[0,\infty )$ into $[0,\infty )$, $h_n$ is a smoothing parameter satisfying $h_n\to 0$ as $n\to \infty$, C is a measurable set, and $x\in \mathcal{E}$. When choosing $C=(-\infty ,z]$, where $z\in {\mathbb{R}}^d$, it reduces to the conditional empirical distribution function $F_n\left(z\right|x)={\mu }_n((-\infty ,z],x)$, as referenced in Refs. [71,72,73]. However, the corresponding class $\mathcal{C}$ is defined as $\left\{(-\infty ,z],z\in {\mathbb{R}}^d\right\}$. Regarding the semi-metric topology on $\mathcal{E}$, we introduce the notation

$$B(x,t)=\{x_1\in \mathcal{E}:d_{\mathcal{E}}(x_1,x)\le t\},$$

which denotes the ball in $\mathcal{E}$ with center x and radius t. This concept is commonly referred to as the small ball probability function in the literature, especially when t tends to zero. The significance of this notion is both theoretically and practically profound, as the concept of a ball is intricately connected with the semi-metric $d(·,·)$. The selection of this semi-metric becomes pivotal when dealing with data in infinite-dimensional spaces.

In many cases, the probability function for the small ball can be roughly represented as the multiplication of two independent functions with respect to variables x and h. This insight is illustrated in several examples found in Proposition 1 of [74]:

• $\varphi \left(h_n\right)=C{h}_n^{\upsilon }$ for some $\upsilon >0$ with ${\tau }_0\left(s\right)=s^{\upsilon }$;
• $\varphi \left(h_n\right)=C{h}_n^{\upsilon }exp(-C{h}_n^{-p})$ for some $\upsilon >0$ and $p>0$ with ${\tau }_0\left(s\right)$ is the Dirac’s function;
• $\varphi \left(h_n\right)=C{\left|ln\left(h_n\right)\right|}^{-1}$ with ${\tau }_0\left(s\right){=}_{]0,1]}\left(s\right)$ the indicator function in $]0,1]$.

Define the following $\sigma$-fields: ${\mathfrak{F}}_i$ and ${\mathfrak{G}}_i$ Let

$${\mathfrak{F}}_i=\sigma ((X_i,Y_i,{\delta }_i):0\le i\le n),$$

$${\mathfrak{G}}_i=\sigma ((X_i,Y_i,{\delta }_i):0\le i\le n),$$

where ${\mathfrak{F}}_i$ be the $\sigma$-filed generated by $((X_1,Y_1,{\delta }_1),\dots ,(X_i,Y_i,{\delta }_i))$ and ${\mathfrak{G}}_i$ that generated by $((X_1,Y_1,{\delta }_1),\dots ,(X_i,Y_i,{\delta }_i),X_{i+1})$. Let $B(x,u)$ be a ball centered at $x\in \mathcal{E}$ with radius u. Let $D_i=d(x,X_i)$ so that $D_i$ is a nonnegative real-valued random variable. Operating within the probability space $(\mathsf{\Omega },\mathcal{A},\mathbb{P})$, consider

$$F_x\left(u\right)=\mathbb{P}(D_i\le u)=\mathbb{P}(X_i\in B(x,u)),$$

and $F_x^{{\mathfrak{F}}_{i-1}}=\mathbb{P}(Xi\in B(x,u)\mid {\mathfrak{F}}_{i-1})$ to be the distribution function and the conditional distribution function, respectively, given the $\sigma$-field ${\mathfrak{F}}_{i-1}$ of ${\left(D_i\right)}_{i\ge 1}$. Here, $B(x,u)$ denotes the ball in the space $\mathcal{E}$ centered at x with radius u. Let $o_{a.s}\left(u\right)$ represent a real random function $l(·)$ such that $l\left(u\right)/u$ converges to zero almost surely as $u\to 0$. In a similar vein, define ${\mathcal{O}}_{a.s}\left(u\right)$ as a real random function $l(·)$ such that $l\left(u\right)/u$ is almost surely bounded. In what follows, we implicitly assume the ergodicity of the sequence of random elements $(X_i,Y_i),i=1,\dots ,n$.

2.1. Assumptions and Notation

In this paper, the variable x is a constant element within the functional space $\mathcal{E}$. We present the metric entropy with inclusion as a means to quantify the richness or complexity of the set class $\mathcal{C}$. For any given $\epsilon >0$, the covering number is defined as:

$$\begin{array}{ccc}\hfill \mathcal{N}& (\epsilon ,\mathcal{C},& \mu \left(·\mid x\right))\hfill \\ & =& inf\{n\in \mathbb{N}:\exists C_1,\dots ,C_n\in \mathcal{C}\mathrm{such}\mathrm{that}\forall C\in \mathcal{C}\exists 1\le i,j\le n\hfill \\ & & \mathrm{with}{C}_i\subset C\subset C_j\mathrm{and}\mu \left(C_j\setminus C_i\mid x\right)<\epsilon \}.\hfill \end{array}$$

The term $log\left(\mathcal{N}\left(\epsilon ,\mathcal{C},\mu \left(·\mid x\right)\right)\right)$ is referred to as the metric entropy with inclusion of $\mathcal{C}$ with respect to $\mu \left(·\mid x\right)$. For numerous classes, estimates for these covering numbers are well-documented; refer, for instance, to Ref. [75]. Below, we frequently make the assumption that either $log\mathcal{N}\left(\epsilon ,\mathcal{C},\mu \left(·\mid x\right)\right)$ or $\mathcal{N}\left(\epsilon ,\mathcal{C},\mu \left(·\mid x\right)\right)$ exhibit behaviors reminiscent of powers of ${\epsilon }^{-1}$. We affirm that condition ($R_{\gamma }$) is satisfied when

$$log\mathcal{N}\left(\epsilon ,\mathcal{C},\mu \left(·\mid x\right)\right)\le H_{\gamma }\left(\epsilon \right),\mathrm{for}\mathrm{all}\epsilon >0,$$

(3)

where

$$H_{\gamma }\left(\epsilon \right)=\left\{\begin{array}{ccc}log\left(A\epsilon \right)\hfill & \mathrm{if}& \hfill \gamma =0,\\ A{\epsilon }^{-\gamma }\hfill & \mathrm{if}& \hfill \gamma >0,\end{array}\right.$$

for some constants $A,r>0$. As emphasized in Ref. [23], it is notable that the condition (3), where $\gamma =0$, is fulfilled by intervals, rectangles, balls, ellipsoids, and by classes derived from these through finite set operations of union, intersection, and complement. The class of convex sets in ${\mathbb{R}}^d$ ($d\ge 2$) satisfies the condition (3) with $\gamma =(d-1)/2$. Various other sets that satisfy (3) with $\gamma >0$ are elaborated upon in Ref. [75]. We give now further notation. For $j\ge 1$, set

$${\displaystyle M_j=K^j\left(1\right)-{\int }_0^1{\left(K^j\right)}^{\prime }\left(u\right){\tau }_0\left(u\right)du.}$$

In this section, we establish the weak convergence of the process ${\nu }_n(C,x):C\in \mathcal{C}$ as defined by

$${\nu }_n(C,x):=\sqrt{n\varphi \left(h_n\right)}\left({\mu }_n(C,x)-\mathbb{E}{\mu }_n(C,x)\right).$$

(4)

In the course of our analysis, we will rely on the following assumptions.

(H1) 

For every $x\in \mathcal{E}$, there exists a sequence of nonnegative bounded random functionals $\left(f_{i,1}\right)i\ge 1$, a sequence of random functions $\left(g_{i,x}\right)i\ge 1$, a deterministic nonnegative bounded functional $f1$, and a nonnegative real function $\varphi$ such that $\varphi \left(h_n\right)\to 0$ as $h\to 0$, as $h\to 0$, such that

(i)

$F_x\left(u\right)=\varphi \left(u\right)f_1\left(x\right)+o\left(\varphi \left(u\right)\right)asu\to 0.$

(ii)

For any $i\in \mathbb{N},F_x^{{\mathfrak{F}}_{i-1}}\left(u\right)=\varphi \left(u\right)f_{i,1}\left(x\right)+g_{i,x}\left(u\right)$ with $g_{i,x}\left(u\right)=o_{a.s}\left(\varphi \left(u\right)\right)$ as $u\to 0,$ $g_{i,x}\left(u\right)/\varphi \left(u\right)$ almost surely bounded and ${\displaystyle n^{-1}\sum _{i=1}^n{g}_{i,x}^j\left(u\right)=o_{a.s}\left({\varphi }^j\left(u\right)\right)asn\to \infty ,j=1,2.}$

(iii)

${\displaystyle n^{-1}\sum _{i=1}^n{f}_{i,1}^j\left(x\right)\to f_1^j\left(x\right)}$ almost surely as $n\to \infty$, for $j=1,2.$

(iv)

There exists a nondecreasing bounded function ${\tau }_0\left(u\right)$ that uniformly holds for all $u\in (0,1)$.

$${\tau }_0\left(u\right)+o\left(1\right)=\frac{\varphi \left(ru\right)}{\varphi \left(r\right)},$$

as $r\downarrow 0$ and $1\le j\le 2+\delta with\delta >0$, ${\displaystyle {\int }_0^1{\left(K^j\left(u\right)\right)}^{\prime }{\tau }_0\left(u\right)du<\infty }$.

(H2) 

There exist positive constants $\beta >0$ and ${\eta }_1>0$ such that for all $x_1,x_2\in N_x$, a neighborhood of x, the following holds

$$|\mu (C\mid x_1)-\mu (C\mid x_2)|\le {\eta }_1{d}_{\mathcal{E}}^{\beta }(x_1,x_2).$$

(H3) 

(i)

The conditional mean of $\mathbb{1}{Y}_i\in C$ given the $\sigma$-field ${\mathfrak{G}}_{i-1}$ depends solely on $X_i$, meaning that for any $i\ge 1$, $\mathbb{E}\left(\mathbb{1}{Y}_1\in C\mid {\mathfrak{G}}_{\mathfrak{i}-\mathfrak{1}}\right)=\mu \left(X_i\right)$ almost surely. The conditional mean of $\mathbb{1}{Y}_i\in C$ given the $\sigma$-field ${\mathfrak{G}}_{i-1}$ also depends only on $X_i$, i.e., for any $i\ge 1$,

$$\mathbb{E}\left({\left({\mathbb{1}}_{\{Y_1\in C\}}-\mu \left(X_i\right)\right)}^2\mid {\mathfrak{G}}_{i-1}\right)=W\left(X_i\right),$$

almost surely.

(ii)

Furthermore, the functions $W(·)$ and $\mathcal{P}(·)$ are continuous in a neighborhood of x, namely,

$$\underset{\{u:d(x,u)\le h\}}{sup}\left|W\left(u\right)-W\left(x\right)\right|=o\left(1\right)ash\to 0,$$

$$\underset{\{u:d(x,u)\le h\}}{sup}\left|\mathcal{P}\left(u\right)-\mathcal{P}\left(x\right)\right|=o\left(1\right)ash\to 0.$$

(iii)

$\exists \delta >0$ such that we let

$${\overline{W}}_{2+\delta }\left(u\right)=\mathbb{E}\left(|({\mathbb{1}}_{\{Y_1\in C\}}-\mu \left(x\right)){|}^{2+\delta }\mid X_1=u\right)$$

be continuous in a neighborhood of x for $u\in \mathcal{E}.$

(H4) 

For any $(y_1,y_2)\in {\mathbb{R}}^{2d}$ and positive constants $b_3>0$ and ${\eta }_4>0$, the following holds for the conditional density $f(·)$ of Y given $X=x$:

$$\left|f\left(y_1\right)-f\left(y_2\right)\right|\le {\eta }_4{∥y_1-y_2∥}^{b_3}.$$

(H5) 

The kernel function $K(·)$ has support within the interval $(0,1)$ and possesses a continuous first derivative on $(0,1)$. It satisfies the condition $K^{\prime }\left(t\right)<0$ for all $t\in (0,1)$. Moreover,

$$\left|{\displaystyle {\int }_0^1{\left(K^j\right)}^{\prime }\left(u\right)du}\right|<\infty ,forj=1,2.$$

(H6) 

Suppose that the set class $\mathcal{C}$ adheres to condition (3);

(H7) 

The smoothing parameter ($h_n$) fulfills the following criterion: $h_n\to 0$ and $n\varphi \left(h_n\right)\to \infty$ as $n\to \infty$.

2.2. Comments on the Assumptions

The significance of condition (H1) extends to both the ergodic and functional aspects addressed in this paper. The condition utilized here shares similarities with that employed in Ref. [66]. The functions $f_{i,1}(·)$ and $f_1(·)$ play roles analogous to the conditional and unconditional densities in the finite-dimensional scenario. In the meantime, $\varphi \left(u\right)$ describes the influence of the radius u on the small ball probability as u tends to zero, as illustrated in Ref. [66]. Conditions (H2)(i) are standard in nonparametric regression estimation. (H3)(i) is essential for establishing consistency, reflecting the Markovian nature of the functionally stationary ergodic data. This condition aligns with that used in Ref. [63]. (H3)(ii,iii) serve as continuous local conditions, necessary for the main results and for conciseness in this paper. Condition (H4) on the density $f(·)$ conforms to a classical Lipschitz-type nonparametric functional model. Assumption (H5) relates to the choice of the kernel $K(·)$, a common practice in nonparametric functional estimation. It is worth noting that the Parzen symmetric kernel is unsuitable in this context due to the positivity of the random process $d\left(x,X\right)$. Hence, we consider $K(·)$ with support $[0,1]$, a natural generalization of the assumption usually made in the multivariate case, where $K(·)$ is expected to be a spherically symmetric density function. The conditions $K\left(1\right)>0$ and $K^{\prime }(·)<0$ ensure that $M_1>0$ for all limit functions ${\tau }_0$. The condition $K\left(1\right)>0$ is necessary for defining the moments $M_2$, which, in this case, are determined by the value $\mathcal{K}\left(1\right)$. (H7) provides a condition on the bandwidths, acknowledging that consistency cannot be guaranteed without it.

3. Main Results

Below, we note $Z\stackrel{\mathcal{D}}{=}\mathcal{N}(\mu ,{\sigma }^2)$ when the random variable Z is distributed according to a normal distribution with mean $\mu$ and variance ${\sigma }^2$. The symbol $\stackrel{\mathcal{D}}{\to }$ represents convergence in distribution, while $\stackrel{\mathbb{P}}{\to }$ indicates convergence in probability.

Theorem 1 

(Uniform Consistency). Assume that the conditions (H1)–(H7) are satisfied. Consider a class of measurable sets $\mathcal{C}$ for which

$$\mathcal{N}\left(\epsilon ,\mathcal{C},\mu \left(·\mid x\right)\right)<\infty ,$$

for any $\epsilon >0$. Moreover, assume that for every $C\in \mathcal{C}$

$$\left|\mu \right(C,y\left)f\right(y)-\mu (C,x\left)f\right(x\left)\right|⟶0,asy\to x.$$

If $n\varphi \left(h_n\right)\to \infty$ and $h_n\to 0$ as $n\to \infty$, then

$$\underset{C\in \mathcal{C}}{sup}\left|{\mu }_n(C,x)-\mathbb{E}\left({\mu }_n(C,x)\right)\right|\stackrel{\mathbb{P}}{⟶}0.$$

Note that the proof of Theorem 1 follows directly from the decomposition

$$\begin{array}{ccc}\hfill {\mu }_n(C,x)-\mathbb{E}\left({\mu }_n(C,x)\right)& =& \frac{1}{\mathbb{E}\left(\widehat{f_n}\left(x\right)\right)}\left[\widehat{{\phi }_n}(C,x)-\mathbb{E}\left(\widehat{{\phi }_n}(C,x)\right)\right]\hfill \\ & & -\frac{{\mu }_n(C,x)}{\mathbb{E}\left(\widehat{f_n}\left(x\right)\right)}\left[\widehat{f_n}\left(x\right)-\mathbb{E}\left(\widehat{f_n}\left(x\right)\right)\right],\hfill \\ & & =\frac{Q_n\left(x\right)}{\mathbb{E}\left(\widehat{f_n}\left(x\right)\right)},\hfill \end{array}$$

where

$$Q_n\left(x\right)=\left[\widehat{{\phi }_n}(C,x)-\mathbb{E}\left(\widehat{{\phi }_n}(C,x)\right)\right]-{\mu }_n(C,x)\left[\widehat{f_n}\left(x\right)-\mathbb{E}\left(\widehat{f_n}\left(x\right)\right)\right].$$

and

$$\begin{array}{ccc}\hfill \widehat{{\phi }_n}(C,x)& =& {\displaystyle \frac{1}{n\varphi \left(h_n\right)}\sum _{i=1}^n{\delta }_i{\mathbb{1}}_{\{Y_i\in C\}}K\left(\frac{d_{\mathcal{E}}(x,X_i)}{h_n}\right),}\hfill \\ \hfill \widehat{f_n}\left(x\right)& =& {\displaystyle \frac{1}{n\varphi \left(h_n\right)}\sum _{i=1}^n{\delta }_{i}K\left(\frac{d_{\mathcal{E}}(x,X_i)}{h_n}\right).}\hfill \end{array}$$

Let ${\Delta }_i\left(x\right)=K\left(\frac{d_{\mathcal{E}}(x,X_i)}{h_n}\right)$. We have

$$\begin{array}{ccc}\hfill \widehat{{\phi }_n}(C,x)& =& {\displaystyle \frac{1}{n\varphi \left(h_n\right)}\sum _{i=1}^n{\mathbb{1}}_{\{Y_i\in C\}}{\delta }_i{\Delta }_i\left(x\right),}\hfill \\ \hfill \widehat{f_n}\left(x\right)& =& {\displaystyle \frac{1}{n\varphi \left(h_n\right)}\sum _{i=1}^n{\delta }_i{\Delta }_i\left(x\right).}\hfill \end{array}$$

Henceforth, for $x\in \mathcal{E}$, let us denote

$${\displaystyle \mathbb{E}\left(\widehat{{\phi }_n}(C,x)\right)=\frac{1}{n\mathbb{E}\left({\Delta }_1\left(x\right)\right)}\sum _{i=1}^n\mathbb{E}\left({\delta }_i{\mathbb{1}}_{\{Y_i\in C\}}{\Delta }_i\left(x\right)\mid {\mathfrak{F}}_{i-1}\right),}$$

and

$${\displaystyle \mathbb{E}\left(\widehat{f_n}\left(x\right)\right)=\frac{1}{n\mathbb{E}\left({\Delta }_1\left(x\right)\right)}\sum _{i=1}^n\mathbb{E}({\delta }_i{\Delta }_i\left(x\right)\mid {\mathfrak{F}}_{i-1}),}$$

here, $\mathbb{E}(X\mid \mathfrak{F})$ represents the conditional expectation of the random variable X given the $\sigma$-field $\mathfrak{F}$.

To establish asymptotic normality, define the “bias” term as

$$\begin{array}{ccc}\hfill B_n\left(x\right)& =& \frac{\mathbb{E}\left(\widehat{f_n}\left(x\right)\right)-{\mu }_n(C,x)\mathbb{E}\left(\widehat{{\phi }_n}(C,x)\right)}{\mathbb{E}\left(\widehat{{\phi }_n}(C,x)\right)}\hfill \end{array}$$

The subsequent result presents the weak convergence. It is important to note that $f_1\left(x\right)$ is specified in (H1).

Theorem 2 

(Asymptotic normality). Assuming (H1)–(H7), as $n\to \infty$, for $m\ge 1$ and $C_1,\dots ,C_m\in \mathcal{C}$, we have

$$\left\{{\nu }_n{(C_i,x)}_{i=1,\dots ,m}\right\}\stackrel{\mathcal{D}}{⟶}\mathcal{N}(0,\Sigma ),$$

where $\Sigma ={\sigma }_{ij}\left(x\right),i,j=1,\dots ,m$ and

$${\sigma }_{ij}\left(x\right)=\frac{M_2}{\mathcal{P}\left(x\right)M_1^2{f}_1\left(x\right)}\left(\mathbb{E}({\mathbb{1}}_{\{Y\in C_i\cap C_j\}}\mid X=x)-\mathbb{E}({\mathbb{1}}_{\{Y\in C_i\}}\mid X=x)\mathbb{E}({\mathbb{1}}_{\{Y\in C_j\}}\mid X=x)\right),$$

whenever $f_1\left(x\right)>0$ and

$${\displaystyle M_1=K\left(1\right)-{\int }_0^1{K}^{\prime }\left(u\right){\tau }_0\left(u\right)d\left(u\right),M_2=K^2\left(1\right)-{\int }_0^1{\left(K^2\right)}^{\prime }\left(u\right){\tau }_0\left(u\right)du.}$$

To obtain the density of the process, it is essential to introduce the following function, which provides insights into the asymptotic behavior of the modulus of continuity:

$${\Lambda }_{\gamma }({\sigma }^2,n)=\left\{\begin{array}{ccc}{\displaystyle \sqrt{{\sigma }^{2}log\frac{1}{{\sigma }^2}},}\hfill & \mathrm{if}& \hfill \gamma =0;\\ {\displaystyle max\left({\left({\sigma }^2\right)}^{(1-\gamma )/2},n\varphi {\left(h_n\right)}^{(3\gamma -1)/\left(2\right(3\gamma +1\left)\right)}\right),}\hfill & \mathrm{if}& \hfill \gamma >0.\end{array}\right.$$

Theorem 3. 

Assume that (H1)–(H7) are satisfied. For every ${\sigma }^2>0$, consider ${\mathcal{C}}_{\sigma }\subset \mathcal{C}$ as a class of measurable sets with

$${\displaystyle \sum _{t=1}^n\underset{C\in {\mathcal{C}}_{\sigma }}{sup}\mu (C,x)\le {\sigma }^2\le 1,}$$

and suppose that $\mathcal{C}$ fulfils (3) with $\gamma \ge 0$. Additionally, we assume that $\varphi \left(h_n\right)\to 0$ and $n\varphi \left(h_n\right)\to +\infty$ as $n\to +\infty$, such that

$$n\varphi \left(h_n\right)\le {\left({\Lambda }_{\gamma }({\sigma }^2,n)\right)}^2,$$

and as $n\to +\infty ,$ we have

$$\frac{n\varphi {\left({\sigma }^{2}log\left(\frac{1}{{\sigma }^2}\right)\right)}^{1+\gamma }}{log\left(n\right)}\to \infty .$$

Furthermore, we assume that ${\sigma }^2\ge h^2$. For $\gamma >0$ and $d=1,2$, the latter has to be replaced by ${\sigma }^2\ge \varphi \left(h_n\right)log\left(\frac{1}{\varphi \left(h_n\right)}\right)$. Under the conditions of Theorem 2, the process converges in law to a Gaussian process $\left\{\nu (C,x):C\in \mathcal{C}\right\}$, which possesses a version with uniformly bounded and uniformly continuous paths with respect to the ${\parallel ·\parallel }_2-$norm. The covariance is given by ${\sigma }_{ij}\left(x\right)$ as specified in Theorem 2.

Remark 1. 

The distance of two measures ${\mu }_1$, ${\mu }_2$ in the Prokhorov metric is defined as (see, e.g., Refs. [76,77,78,79])

$${\rho }_{\mathrm{P}}\left({\mu }_1,{\mu }_2\right):=inf\left\{\epsilon >0\mid {\mu }_1\left(B\right)\le {\mu }_2\left(B^{\epsilon }\right)+\epsilon ,\forall \mathit{Borel}\mathit{sets}B\subset \Omega \right\}$$

Here $B^{\epsilon }=\{x\mid d(x,B)<\epsilon \}$, where $d(x,B)$ is the distance of x to B, i.e., $d(x,B)={inf}_{z\in B}\parallel x-z\parallel$. The distance of two random variables ${\xi }_1$, ${\xi }_2$ in the Ky Fan metric is defined as [80]

$${\rho }_{\mathrm{K}}\left({\xi }_1,{\xi }_2\right):=inf\left\{\epsilon >0\mid \mu \left\{\omega \in \Omega \mid d\left({\xi }_1\left(\omega \right),{\xi }_2\left(\omega \right)\right)>\epsilon \right\}<\epsilon \right\}.$$

It is worthwhile to establish an adequate link of our findings to these distances in the conditional setting.

Remark 2. 

Central limit theorems are frequently utilized to establish confidence intervals for the target being estimated. In the realm of non-parametric estimation, the asymptotic variance $\Sigma \left(x\right):={\sigma }_{i,j}\left(x\right)$ in the central limit depends on certain unknown functions. Consequently, in practical scenarios, only approximate confidence intervals can be derived, even when $\Sigma \left(x\right)$ is functionally specified. Notably, according to Theorem 2, the limiting variance incorporates the unknown function $f_1(·)$ and the normalization is contingent on the function $\varphi (·)$, which is not explicitly identifiable in practice. Furthermore, the quantities $W(·)$ and ${\tau }_0$ need to be estimated. The corollary below, a slight modification of Theorem 2, permits a practical form of the results to be used, as typically the conditional variance $W\left(x\right)$ is estimated similarly to what is obtained by Ref. [63].

Let

$$\begin{array}{ccc}\hfill W_n& =& \frac{{\displaystyle \sum _{i=1}^n{({\delta }_i{\mathbb{1}}_{\{Y_i\in C\}}-{\mu }_n\left(x\right))}^{2}K\left(\frac{d_{\mathcal{E}}(x,X_i)}{h}\right)}}{{\displaystyle \sum _{i=1}^n{\delta }_{i}K\left(\frac{d_{\mathcal{E}}(x,X_i)}{h}\right)}}\hfill \\ & =& \frac{{\displaystyle \sum _{i=1}^n{({\delta }_i{\mathbb{1}}_{\{Y_i\in C\}}-{\mu }_n\left(x\right))}^{2}K\left(\frac{d_{\mathcal{E}}(x,X_i)}{h}\right)}}{{\displaystyle \sum _{i=1}^n{\delta }_{i}K\left(\frac{d_{\mathcal{E}}(x,X_i)}{h}\right)}}-{\left({\mu }_n\left(x\right)\right)}^2\hfill \\ & =& \widehat{g_n}\left(x\right)-{\left({\mu }_n\left(x\right)\right)}^2.\hfill \end{array}$$

Let us introduce the following estimation

$${\mathcal{F}}_{x,n}\left(t\right)=\frac{1}{n}\sum _{i=1}^n{\mathbb{1}}_{\left\{d\left(x,X_i\right)\le t\right\}}.$$

By employing the decomposition of ${\tau }_0(·)$ in (H1)(i) and (H1)(i,iv), one can estimate ${\tau }_0(·)$ as

$${\tau }_n\left(t\right)=\frac{{\mathcal{F}}_{x,n}\left(th\right)}{{\mathcal{F}}_{x,n}\left(h\right)}.$$

Subsequently, for a given kernel $K(·)$ and the quantities $M_1$ and $M_2$ can be estimated as follows

$${\displaystyle M_{1,n}=K\left(1\right)-{\int }_0^1{K}^{\prime }\left(s\right){\tau }_n\left(s\right)ds,M_{2,n}=K^2\left(1\right)-{\int }_0^1{\left(K^2\right)}^{\prime }\left(s\right){\tau }_n\left(s\right)ds.}$$

Finally, the estimator of $\mathcal{P}\left(x\right)$ is denoted by

$${\mathcal{P}}_n\left(x\right)=\frac{{\displaystyle \sum _{i=1}^n{\delta }_{i}K\left(\frac{d_{\mathcal{E}}(x,X_i)}{h_n}\right)}}{{\displaystyle \sum _{i=1}^{n}K\left(\frac{d_{\mathcal{E}}(x,X_i)}{h_n}\right)}}.$$

Corollary 1. 

Suppose that conditions (H1)–(H7) are satisfied, where $K^{\prime }$ and ${\left(K^2\right)}^{\prime }$ are integrable functions. Additionally, assume that $n\mathcal{F}x\left(h\right)⟶\infty$ and $h^{\beta }{\left(n\mathcal{F}x\left(h\right)\right)}^{1/2}⟶0$ as $n\to \infty$. Then, for any $x\in \mathcal{E}$ such that $f_1\left(x\right)>0$, we have

$$\frac{M_{1,n}}{\sqrt{M_{2,n}}}\sqrt{\frac{n{\mathcal{F}}_{x,n}\left(h_n\right){\mathcal{P}}_n\left(x\right)}{W_n\left(x\right)}}\left({\mu }_n(C,x)-\mu (C,x)\right)\stackrel{\mathcal{D}}{⟶}\mathcal{N}(0,1).$$

Using Corollary (1) the asymptotic $100(1-\alpha )$ confidence band given by

$$\left[{\mu }_n(C,x)-c_{\alpha }\frac{M_{1,n}}{\sqrt{M_{2,n}}}\sqrt{\frac{W_n\left(x\right)}{n{\mathcal{F}}_{x,n}\left(h\right){\mathcal{P}}_n\left(x\right)}},{\mu }_n(C,x)+c_{\alpha }\frac{M_{1,n}}{\sqrt{M_{2,n}}}\sqrt{\frac{W_n\left(x\right)}{n{\mathcal{F}}_{x,n}\left(h\right){\mathcal{P}}_n\left(x\right)}}\right].$$

where $c_{\alpha }$ is the upper $\frac{\alpha }{2}$ quantile of the Normal distribution $\mathcal{N}(0,1)$

3.1. The Bandwidth Selection Criterion

Several approaches have been devised and refined to formulate asymptotically optimal bandwidth selection rules for nonparametric kernel estimators, particularly for the Nadaraya–Watson regression estimator. Some noteworthy contributions include [81,82,83,84,85,86,87]. Choosing this parameter appropriately is essential, whether in the conventional finite-dimensional case or within the infinite-dimensional framework, to guarantee favorable practical performance. Let us define the leave-out-$\left(X_j,Y_j,{\delta }_j\right)$ estimator for the regression function

$$\begin{array}{c}\hfill {\mu }_{n,j}(C,x)=\frac{{\displaystyle \sum _{i=1,i\ne j}^n{\delta }_i{\mathbb{1}}_{\{Y_i\in C\}}K\left(\frac{d_{\mathcal{E}}(x,X_i)}{h_n}\right)}}{{\displaystyle \sum _{i=1}^n{\delta }_{i}K\left(\frac{d_{\mathcal{E}}(x,X_i)}{h_n}\right)}}.\end{array}$$

(5)

To minimize the quadratic loss function, we introduce the following criterion, where we have a (known) nonnegative weight function $\mathcal{W}(·):$

$$CV\left(C,h\right):=\frac{1}{n}\sum _{j=1}^n{\left({\delta }_j{\mathbb{1}}_{\{Y_j\in C\}}-{\mu }_{n,j}(C,X_j)\right)}^2\mathcal{W}\left(X_j\right).$$

(6)

Building upon the concepts developed by Ref. [83], a natural approach for selecting the bandwidth is to minimize the preceding criterion. Thus, let us choose ${\widehat{h}}_n$, as the minimizer over h:

$$\underset{C\in \mathcal{C}}{sup}CV\left(C,h\right).$$

One can replace (6) by

$$CV\left(C,h_n\right):=\frac{1}{n}\sum _{j=1}^n{\left({\delta }_j{\mathbb{1}}_{\{Y_j\in C\}}-{\mu }_{n,j}(C,X_j)\right)}^2\widehat{\mathcal{W}}\left(X_j,x\right).$$

(7)

In practice, one takes, for $j=1,\dots ,n$, the uniform global weights $\mathcal{W}\left(X_j\right)=1$, and the local weights

$$\widehat{W}(X_j,x)=\left\{\begin{array}{ccc}1& if& d(X_j,x)\le h_n,\hfill \\ 0& & \mathrm{otherwise}.\hfill \end{array}\right.$$

For brevity, we have concentrated on the most popular method, namely, the cross-validated selected bandwidth. This approach can be extended to any other bandwidth selector, such as the bandwidth based on Bayesian ideas [88].

4. Applications to Classification with Partially Labeled Data

In this section, we apply the results developed in the previous sections to the problem of statistical classification. We consider a sample of random elements $(X_1,Y_1),\dots ,(X_n,Y_n)$ drawn from the joint distribution of $(X,Y)$, where X takes values in a space $\mathcal{E}$ and Y in ${\mathbb{R}}^d$. In classification, the objective is to predict the integer-valued label Y based on the covariate vector X. More formally, we aim to find a function (classifier) $\theta :\mathcal{E}⟶{\mathbb{R}}^d$ for which the probability of misclassification error (incorrect prediction), i.e., $\mathbb{P}\left(\theta \right(X)\ne Y)$, is minimized. Let

$${\mathcal{P}}_k\left(x\right)=\mathbb{P}(Y=k\mid X=x),x\in \mathcal{E},1\le k\le n.$$

Demonstrating that the optimal classifier, i.e., the one with the minimum probability of error, is given by

$${\theta }_B\left(x\right)=arg\underset{1\le k\le n}{max}{\mathcal{P}}_k\left(x\right),$$

i.e., the best classifier ${\theta }_B$ satisfies

$$\underset{1\le k\le n}{max}{\mathcal{P}}_k\left(x\right)={\mathcal{P}}_{{\theta }_B\left(x\right)}\left(x\right).$$

As ${\theta }_B$ is unknown, the data is utilized to construct estimates of ${\theta }_B$. Specifically, let ${\mathbb{D}}_n=(X1,Y_1),\dots ,(X_n,Y_n)$ represent a random sample from the distribution of $(X,Y)$, where each $(X_i,Y_i)$ is fully observable. Let ${\widehat{\theta }}_n$ be any sample-based classifier. In other words, $\widehat{\theta }n\left(X\right)$ is the predicted value of Y, based on ${\mathbb{D}}_n$ and X. Let

$$L_n\left({\widehat{\theta }}_n\right)=\mathbb{P}({\widehat{\theta }}_n\left(X\right)\ne Y\mid {\mathbb{D}}_n),$$

be the conditional probability of error of the sample-based classifier ${\widehat{\theta }}_n$. Then ${\widehat{\theta }}_n$ is said to be consistent if $L_n\left({\widehat{\theta }}_n\right)⟶L_n\left({\theta }_n\right)=\mathbb{P}({\theta }_B\left(X\right)\ne Y)$ as $n\to \infty$, for $k=1,\dots ,n$. Let ${\widehat{\mathcal{P}}}_k\left(x\right)$ be any sample-based estimators of ${\mathcal{P}}_k\left(x\right)=\mathbb{P}(Y=k\mid X=x)$ and define the classification rule ${\widehat{\theta }}_n$ by

$${\widehat{\theta }}_n\left(x\right)=arg\underset{1\le k\le n}{max}{\widehat{\mathcal{P}}}_k\left(x\right).$$

In other words, ${\widehat{\theta }}_n$ satisfies

$$\underset{1\le k\le n}{max}{\widehat{\mathcal{P}}}_k\left(x\right)={\widehat{\mathcal{P}}}_{{\widehat{\theta }}_n\left(x\right)}\left(x\right),$$

to show $L_n\left({\widehat{\theta }}_n\right)-L_n\left({\theta }_B\right)⟶0$ it is sufficient to show that ${\widehat{\mathcal{P}}}_k\left(x\right)-{\mathcal{P}}_k\left(x\right)⟶0$ by posing ${\delta }_i={\widehat{\mathcal{P}}}_k\left(x\right)$, we have

$$\begin{array}{c}\hfill {\mu }_n(C,x)=\frac{{\displaystyle \sum _{i=1}^n{\widehat{\mathcal{P}}}_k\left(x\right){\mathbb{1}}_{\{Y_i\in C\}}K\left(\frac{d_{\mathcal{E}}(x,X_i)}{h_n}\right)}}{{\displaystyle \sum _{i=1}^n{\widehat{\mathcal{P}}}_k\left(x\right)K\left(\frac{d_{\mathcal{E}}(x,X_i)}{h_n}\right)}}.\end{array}$$

(8)

Theorem 4. 

Under the conditions of Theorem 3, we have the convergence

$$L_n\left({\widehat{\theta }}_n\right)-L_n\left({\theta }_B\right)⟶0.$$

5. Concluding Remarks

In this investigation, we have examined the asymptotic properties of the conditional set-indexed empirical process involving ergodic functional data that are missing at random (MAR). Our findings are obtained under assumptions pertaining to the richness of the index class $\mathcal{C}$ of sets in terms of metric entropy with bracketing. Our contribution is two-fold: first, we have developed a functional methodology for addressing MAR samples in non-parametric problems, and second, we have extended our non-parametric conditional methodology by incorporating the ergodicity concepts introduced in Ref. [44]. Several challenging open questions remain in this context, including potential extensions to other types of non-parametric predictors such as functional local linear predictors, functional kNN predictors, and others. Furthermore, exploring extensions to problems beyond prediction, such as the estimation of variance error, is an interesting avenue for future research. Another direction for future exploration is the consideration of reducing the predictor’s dimensionality by employing a Single Functional Index Model (SFIM) to estimate the regression, as discussed in Refs. [89,90]. SFIM has shown its effectiveness in improving the consistency of the regression operator estimator.