Robust Inference after Random Projections via Hellinger Distance for Location-Scale Family

Abstract

Big data and streaming data are encountered in a variety of contemporary applications in business and industry. In such cases, it is common to use random projections to reduce the dimension of the data yielding compressed data. These data however possess various anomalies such as heterogeneity, outliers, and round-off errors which are hard to detect due to volume and processing challenges. This paper describes a new robust and efficient methodology, using Hellinger distance, to analyze the compressed data. Using large sample methods and numerical experiments, it is demonstrated that a routine use of robust estimation procedure is feasible. The role of double limits in understanding the efficiency and robustness is brought out, which is of independent interest.

1. Introduction

Streaming data are commonly encountered in several business and industrial applications leading to the so-called Big Data. These are commonly characterized using four V’s: velocity, volume, variety, and veracity. Velocity refers to the speed of data processing while volume refers to the amount of data. Variety refers to various types of data while veracity refers to uncertainty and imprecision in data. It is believed that veracity is due to data inconsistencies, incompleteness, and approximations. Whatever be the real cause, it is hard to identify and pre-process data for veracity in a big data setting. The issues are even more complicated when the data are streaming.

A consequence of the data veracity is that statistical assumptions used for analytics tend to be inaccurate. Specifically, considerations such as model misspecification, statistical efficiency, robustness, and uncertainty assessment-which are standard part of a statistical toolkit-cannot be routinely carried out due to storage limitations. Statistical methods that facilitate simultaneous addressal of twin problems of volume and veracity would enhance the value of the big data. While health care industry and financial industries would be the prime benefactors of this technology, the methods can be routinely applied in a variety of problems that use big data for decision making.

We consider a collection of n (n is of the order of at least ${10}^6$) observations, assumed to be independent and identically distributed (i.i.d.), from a probability distribution $f(·)$ belonging to a location-scale family; that is,

$$f(x;\mu ,\sigma )=\frac{1}{\sigma }f\left(\frac{x-\mu }{\sigma }\right),\mu \in \mathrm{I}\mathrm{R},\sigma >0.$$

We denote by $\mathsf{\Theta }$ the parameter space and without loss of generality take it as compact since otherwise it can be re-parametrized in a such a way that the resulting parameter space is compact (see [1]).

The purpose of this paper is to describe a methodology for joint robust and efficient estimation of $\mu$ and ${\sigma }^2$ that takes into account (i) storage issues, (ii) potential model misspecifications, and (iii) presence of aberrant outliers. These issues-which are more likely to occur when dealing with massive amounts of data-if not appropriately accounted in the methodological development, can lead to inaccurate inference and misleading conclusions. On the other hand, incorporating them in the existing methodology may not be feasible due to a computational burden.

Hellinger distance-based methods have long been used to handle the dual issue of robustness and statistical efficiency. Since the work of [1,2] statistical methods that invoke alternative objective functions which converge to the objective function under the posited model have been developed and the methods have been shown to possess efficiency and robustness. However, their routine use in the context of big data problems is not feasible due to the complexity in the computations and other statistical challenges. Recently, a class of algorithms-referred to as Divide and Conquer—have been developed to address some of these issues in the context of likelihood. These algorithms consist in distributing the data across multiple processors and, in the context of the problem under consideration, estimating the parameters from each processor separately and then combining them to obtain an overall estimate. The algorithm assumes availability of several processors, with substantial processing power, to solve the complex problem at hand. Since robust procedures involve complex iterative computations-invoking the increased demand for several high-speed processors and enhanced memory-routine use of available analytical methods in a big data setting is challenging. Maximum likelihood method of estimation in the context of location-scale family of distributions has received much attention in the literature ([3,4,5,6,7]). It is well-known that the maximum likelihood estimators (MLE) of location-scale families may not exist unless the defining function $f(·)$ satisfies certain regularity conditions. Hence, it is natural to ask if other methods of estimation such as minimum Hellinger distance estimator(MHDE) under weaker regularity conditions. This manuscript provides a first step towards addressing this question. Random projections and sparse random projections are being increasingly used to “compress data” and then use the resulting compressed data for inference. The methodology, primarily developed by computer scientists, is increasingly gaining attention among the statistical community and is investigated in a variety of recent work ([8,9,10,11,12]). In this manuscript, we describe a Hellinger distance-based methodology for robust and efficient estimation after the use of random projections for compressing i.i.d data belonging to the location-scale family. The proposed method consists in reducing the dimension of the data to facilitate the ease of computations and simultaneously maintain robustness and efficiency when the posited model is correct. While primarily developed to handle big and streaming data, the approach can also be used to handle privacy issues in a variety of applications [13].

The rest of the paper is organized as follows: Section 2 provides background on minimum Hellinger distance estimation; Section 3 is concerned with the development of Hellinger distance-based methods for compressed data obtained after using random projections; additionally, it contains the main results and their proofs. Section 4 contains results of the numerical experiments and also describes an algorithm for implementation of the proposed methods. Section 5 contains a real data example from financial analytics. Section 6 is concerned with discussions and extensions. Section 7 contains some concluding remarks.

2. Background on Minimum Hellinger Distance Estimation

Ref. [1] proposed minimum Hellinger distance (MHD) estimation for i.i.d. observations and established that MHD estimators (MHDE) are simultaneously robust and first-order efficient under the true model. Other researchers have investigated related estimators, for example, [14,15,16,17,18,19,20]. These authors establish that when the model is correct, the MHDE is asymptotically equivalent to the maximum likelihood estimator (MLE) in a variety of independent and dependent data settings. For a comprehensive discussion of minimum divergence theory see [21].

We begin by recalling that the Hellinger distance between two probability densities is the $L^2$ distance between the square root of the densities. Specifically, let, for $p\ge 1$, $\left|\right|·{\left|\right|}_p$ denote the $L^p$ norm defined by

$$\begin{array}{c}\hfill {\left|\right|h\left|\right|}_p={\left\{\int {|h|}^p\right\}}^{1/p}.\end{array}$$

The Hellinger distance between the densities $f(·)$ and $g(·)$ is given by

$$\begin{array}{c}\hfill H^2(f(·),g(·))=\left|\right|f^{1/2}(·)-g^{1/2}(·){\left|\right|}_2^2.\end{array}$$

Let $f(·|\mathit{\theta })$ denote the density of $\mathrm{I}{\mathrm{R}}^d$ valued independent and identically distributed random variables $X_1,\cdots ,X_n$, where $\mathit{\theta }\in \mathsf{\Theta }\subset \mathrm{I}{\mathrm{R}}^p$; let $g_n(·)$ be a nonparametric density estimate (typically a kernel density estimator). The Hellinger distance between $f(·|\mathit{\theta })$ and $g_n(·)$ is then

$$\begin{array}{c}\hfill H^2\left(f(·|\mathit{\theta }),g_n(·)\right)=\left|\right|f^{1/2}(·|\mathit{\theta })-g_n^{1/2}(·){\left|\right|}_2^2.\end{array}$$

The MHDE is a mapping $T(·)$ from the set of all densities to $\mathrm{I}{\mathrm{R}}^p$ defined as follows:

$$\begin{array}{c}\hfill {\mathit{\theta }}_g=T\left(g\right)=\underset{\mathit{\theta }\in \mathsf{\Theta }}{\mathrm{argmin}}{H}^2\left(f(·|\mathit{\theta }),g(·)\right).\end{array}$$

(1)

Please note that the above minimization problem is equivalent to maximizing $\mathcal{A}\left(f(·|\mathit{\theta }),g(·)\right)=\int f^{1/2}\left(x\right|\mathit{\theta })g^{1/2}\left(x\right)dx$. Hence MHDE can alternatively be defined as

$$\begin{array}{c}\hfill {\mathit{\theta }}_g=\underset{\mathit{\theta }\in \mathsf{\Theta }}{\mathrm{argmax}}\mathcal{A}\left(f(·|\mathit{\theta }),g(·)\right).\end{array}$$

To study the robustness of MHDE, ref. [1] showed that to assess the robustness of a functional with respect to the gross-error model it is necessary to examine the $\alpha$-influence curve rather than the influence curve, except when the influence curve provides a uniform approximation to the $\alpha$-influence curve. Specifically, the $\alpha$-influence function (${\mathrm{IF}}_{\alpha }(\mathit{\theta },z)$) is defined as follows: for $\mathit{\theta }\in \mathsf{\Theta }$, let $f_{\alpha ,\mathit{\theta },z}=(1-\alpha )f(·|\mathit{\theta })+\alpha {\eta }_z$, where ${\eta }_z$ denotes the uniform density on the interval $(z-ϵ,z+ϵ)$, where $ϵ>0$ is small, $\alpha \in (0,1)$, $z\in \mathrm{I}\mathrm{R}$; the $\alpha$-influence function is then defined to be

$$\begin{array}{c}\hfill {\mathrm{IF}}_{\alpha }(\mathit{\theta },z)=\frac{T\left(f_{\alpha ,\mathit{\theta },z}\right)-\mathit{\theta }}{\alpha },\end{array}$$

(2)

where $T\left(f_{\alpha ,\mathit{\theta },z}\right)$ is the functional for the model with density $f_{\alpha ,\mathit{\theta },z}(·)$. Equation (2) represents a complete description of the behavior of the estimator in the presence of contamination, up to the shape of the contaminating density. If ${\mathrm{IF}}_{\alpha }(\mathit{\theta },z)$ is a bounded function of z such that ${lim}_{z\to \infty }{\mathrm{IF}}_{\alpha }(\mathit{\theta },\mathit{z})=0,\mathrm{for}\mathrm{every}\mathit{\theta }\in \mathsf{\Theta }$, then the functional T is robust at $f(·|\mathit{\theta })$ against $100\%\alpha$ contamination by gross errors at arbitrary large value z. The influence function can be obtained by letting $\alpha \to 0$. Under standard regularity conditions, the minimum divergence estimators (MDE) are first order efficient and have the same influence function as the MLE under the model, which is often unbounded. Hence the robustness of these estimators cannot be explained through their influence functions. In contrast, the $\alpha$-influence function of the estimators are often bounded, continuous functions of the contaminating point. Finally, this approach often leads to high breakdown points in parametric estimation. Other explanations can also be found in [22,23].

Ref. [1] showed that the MHDE of location has a breakdown point equal to $50\%$. Roughly speaking, the breakdown point is the smallest fraction of data that, when strategically placed, can cause an estimator to take arbitrary values. Ref. [24] obtained breakdown results for MHDE of multivariate location and covariance. They showed that the affine-invariant MHDE for multivariate location and covariance has a breakdown point of at least $25\%$. Ref. [18] showed that the MHDE has $50\%$ breakdown in some discrete models.

3. Hellinger Distance Methodology for Compressed Data

In this section we describe the Hellinger distance-based methodology as applied to the compressed data. Since we are seeking to model the streaming independent and identically distributed data, we denote by J the number of observations in a fixed time-interval (for instance, every ten minutes, or every half-hour, or every three hours). Let B denote the total number of time intervals. Alternatively, B could also represent the number of sources from which the data are collected. Then, the incoming data can be expressed as $\{X_{jl},1\le j\le J;1\le l\le B\}$. Throughout this paper, we assume that the density of $X_{jl}$ belongs to a location-scale family and is given by $f(x;{\mathit{\theta }}^{*})=\frac{1}{{\sigma }^{*}}f(\frac{x-{\mu }^{*}}{{\sigma }^{*}})$, where ${\mathit{\theta }}^{*}=({\mu }^{*},{\sigma }^{*})$. A typical example is a data store receiving data from multiple sources, for instance financial or healthcare organizations, where information from multiple sources across several hours are used to monitor events of interest such as cumulative usage of certain financial instruments or drugs.

3.1. Random Projections

Let ${\mathit{R}}_l=\left(r_{ijl}\right)$ be a $S\times J$ matrix, where S is the number of compressed observations in each time interval, $S\ll J$, and $r_{ijl}$’s are independent and identically distributed random variables and assumed to be independent of $\{X_{jl},j=1,2,\cdots ,J;1\le l\le B\}$. Let

$$\begin{array}{c}\hfill {\tilde{Y}}_{il}=\sum _{j=1}^J{r}_{ijl}{X}_{jl}\end{array}$$

and set $\widetilde{{\mathit{Y}}_l}={({\tilde{Y}}_{1l},\cdots ,{\tilde{Y}}_{Sl})}^{\prime }$; in matrix form this can be expressed as ${\tilde{\mathit{Y}}}_l={\mathit{R}}_l{\mathit{X}}_l$. The matrix ${\mathit{R}}_l$ is referred to as the sensing matrix and $\{{\tilde{Y}}_{il},i=1,2\cdots ,S;l=1,2,\cdots ,B\}$ is referred to as the compressed data. The total number of compressed observations $m=SB$ is much smaller than the number of original observations $n=JB$. We notice here that ${\mathit{R}}_l$’s are independent and identically distributed random matrices of order $S\times J$. Referring to each time interval or a source as a group, the following

Table 1. Illustration of Data Reduction Mechanism, Here ${\mathit{r}}_{il}^{*}=(r_{i·l},{\omega }_{il})$.

	Grp 1	Grp 2	⋯	Grp B		Grp 1	Grp 2	⋯	Grp B
Original	$X_{11}$	$X_{12}$	⋯	$X_{1B}$	Compressed	$({\tilde{Y}}_{11},{\mathit{r}}_{11}^{*})$	$({\tilde{Y}}_{12},{\mathit{r}}_{12}^{*})$	⋯	$({\tilde{Y}}_{1B},{\mathit{r}}_{1B}^{*})$
Data	$X_{21}$	$X_{22}$	⋯	$X_{2B}$	Data	$({\tilde{Y}}_{21},{\mathit{r}}_{21}^{*})$	$({\tilde{Y}}_{22},{\mathit{r}}_{22}^{*})$	⋯	$({\tilde{Y}}_{2B},{\mathit{r}}_{2B}^{*})$
	⋮	⋮	⋮	⋮	$\stackrel{S\ll J}{⟹}$	⋮	⋮	⋮	⋮
	$X_{J1}$	$X_{J2}$	⋯	$X_{JB}$		$({\tilde{Y}}_{S1},{\mathit{r}}_{S1}^{*})$	$({\tilde{Y}}_{S2},{\mathit{r}}_{S2}^{*})$	⋯	$({\tilde{Y}}_{SB},{\mathit{r}}_{SB}^{*})$

is a tabular representation of the compressed data.

In random projections literature, the distribution of $r_{ijl}$ is typically taken to be Gaussian; but other distributions such as Rademacher distribution, exponential distribution and extreme value distributions are also used (for instance, see [25]). In this paper, we do not make any strong distributional assumptions on $r_{ijl}$. We only assume that $\mathit{E}\left[r_{ijl}\right]=1$ and $\mathit{Var}\left[r_{ijl}\right]={\gamma }_0^2$, where $\mathit{E}[·]$ represents the expectation of the random variable and $\mathit{Var}\left[·\right]$ represents the variance of the random variable. Additionally, we denote the density of $r_{ijl}$ by $q(·)$.

We next return to the storage issue. When $S=1$ and $r_{ijl}=1$, ${\tilde{Y}}_{il}$ is a sum of J random variables. In this case, one retains (stores) only the sum of J observations and robust estimates of ${\mathit{\theta }}^{*}$ are sought using the sum of observations. In other situations, that is when $r_{ijl}$ are not degenerate at 1, the distribution of ${\tilde{Y}}_{il}$ is complicated. Indeed, even if $r_{ijl}$ are assumed to be normally distributed, the marginal distribution of ${\tilde{Y}}_{il}$ is complicated. The conditional distribution is ${\tilde{Y}}_{il}$ (given $r_{ijl}$) is a weighted sum of location scale distributions and does not have a useful closed form expression. Hence, in general for these problems the MLE method is not feasible. We denote by ${\omega }_{il}^2={\sum }_{j=1}^J{r}_{ijl}^2$ and work with the random variables $Y_{il}\equiv {\omega }_{il}^{-1}{\tilde{Y}}_{il}.$ We denote the true density of $Y_{il}$ to be $h_J(·|{\mathit{\theta }}^{*},{\gamma }_0)$. Also, when ${\gamma }_0=0$ (which implies $r_{ijl}\equiv 1$) we denote the true density of $Y_{il}$ by $h^{*J}(·|{\mathit{\theta }}^{*})$ to emphasize that the true density is a convolution of J independent and identically distributed random variables.

3.2. Hellinger Distance Method for Compressed Data

In this section, we describe the Hellinger distance-based method for estimating the parameters of the location scale family using the compressed data. As described in the last section, let $\{X_{jl},j=1,2,\cdots ,J;l=1,2,\cdots ,B\}$ be a doubly indexed collection of independent and identically distributed random variables with true density $\frac{1}{{\sigma }^{*}}f\left(\frac{·-{\mu }^{*}}{{\sigma }^{*}}\right)$. Our goal is to estimate ${\mathit{\theta }}^{*}=({\mu }^{*},{\sigma }^{2*})$ using the compressed data $\{Y_{il},i=1,2,\cdots ,S;l=1,2,\cdots ,B\}$. We re-emphasize here that the density of $Y_{il}$ depends additionally on ${\gamma }_0$, the variance of the sensing random variables $r_{ijl}$.

To formulate the Hellinger-distance estimation method, let $\mathcal{G}$ be a class of densities metrized by the ${\mathit{L}}_1$ distance. Let $\{h_J(·|\mathit{\theta },{\gamma }_0);\mathit{\theta }\in \mathsf{\Theta }\}$ be a parametric family of densities. The Hellinger distance functional T is a measurable mapping mapping from $\mathcal{G}$ to $\mathsf{\Theta }$, defined as follows:

$$\begin{array}{ccc}\hfill T\left(g\right)& \equiv & \underset{\mathit{\theta }}{arg\; min}{\int }_{\mathrm{I}\mathrm{R}}{\left(g^{\frac{1}{2}}\left(y\right)-h_J^{\frac{1}{2}}\left(y\right|\mathit{\theta },{\gamma }_0)\right)}^{2}dy\hfill \\ & =& \underset{\mathit{\theta }}{arg\; min}H{D}^2\left(g,h_J(·|\mathit{\theta },{\gamma }_0)\right)={\mathit{\theta }}_g^{*}\left({\gamma }_0\right).\hfill \end{array}$$

When $g(·)=h_J(·|{\mathit{\theta }}^{*},{\gamma }_0)$, then under additional assumptions ${\mathit{\theta }}_g^{*}\left({\gamma }_0\right)={\mathit{\theta }}^{*}\left({\gamma }_0\right)$. Since minimizing the Hellinger-distance is equivalent to maximizing the affinity, it follows that

$$\begin{array}{c}\hfill T\left(g\right)=\underset{\mathit{\theta }}{arg\; max}\mathcal{A}\left(g,h_J(·|\mathit{\theta },{\gamma }_0)\right),\mathrm{where}\end{array}$$

$$\begin{array}{c}\hfill \mathcal{A}(g,h_J(·|\mathit{\theta },{\gamma }_0))\equiv {\int }_{\mathrm{I}\mathrm{R}}{g}^{\frac{1}{2}}\left(y\right)h_J^{\frac{1}{2}}\left(y\right|\mathit{\theta },{\gamma }_0)dy.\end{array}$$

It is worth noticing here that

$$\begin{array}{c}\hfill \mathcal{A}(g,h_J(·|\mathit{\theta },{\gamma }_0))=1-\frac{1}{2}H{D}^2(g,h_J(·|\mathit{\theta },{\gamma }_0)).\end{array}$$

(3)

To obtain the Hellinger distance estimator of the true unknown parameters ${\mathit{\theta }}^{*}$, expectedly we choose the parametric family $h_J(·|\mathit{\theta },{\gamma }_0)$ to be density of $Y_{il}$ and $g(·)$ to be a non-parametric ${\mathit{L}}_1$ consistent estimator $g_B(·)$ of $h_J(·|\mathit{\theta },{\gamma }_0)$. Thus, the MHDE of ${\mathit{\theta }}_B^{*}$ is given by

$$\begin{array}{ccc}\hfill {\widehat{\mathit{\theta }}}_B\left({\gamma }_0\right)& =& \underset{\mathit{\theta }}{arg\; max}\mathcal{A}\left(g_B,h_J(·|\mathit{\theta },{\gamma }_0)\right)=T\left(g_B\right).\hfill \end{array}$$

In the notation above, we emphasize the dependence of the estimator on the variance of the projecting random variables. We notice here that the solution to (1) may not be unique. In such cases, we choose one of the solutions in a measurable manner.

The choice of the density estimate, typically employed in the literature is the kernel density estimate. However, in the setting of the compressed data investigated here, there are S observations per group. These S observations are, conditioned on $r_{ijl}$ independent; however they are marginally dependent (if $S>1$). In the case when $S>1$, we propose the following formula for $g_B(·)$. First, we consider the estimator

$$\begin{array}{c}\hfill g_B^{\left(i\right)}\left(y\right)=\frac{1}{B{c}_B}\sum _{l=1}^{B}K\left(\frac{y-Y_{il}}{c_B}\right),i=1,2,\cdots ,S.\end{array}$$

With this choice, the MHDE of ${\mathit{\theta }}_B^{*}$ is given by, for $1\le i\le S$,

$$\begin{array}{c}\hfill {\widehat{\mathit{\theta }}}_{i,B}\left({\gamma }_0\right)=\underset{\mathit{\theta }}{arg\; max}\mathcal{A}\left(g_B^{\left(i\right)},h_J(·|\mathit{\theta },{\gamma }_0)\right).\end{array}$$

(4)

The above estimate of the density chooses $i^{th}$ observation from each group and obtains the kernel density estimator using the B independent and identically distributed compressed observations. This is one choice for the estimator. Of course, alternatively, one could obtain $S^B$ different estimators by choosing different combinations of observations from each group.

It is well-known that the estimator is almost surely ${\mathit{L}}_1$ consistent for $h_J(·|{\mathit{\theta }}^{*},{\gamma }_0)$ as long as $c_B\to 0$ and $B{c}_B\to \infty$ as $B\to \infty$. Hence, under additional regularity and identifiability conditions and further conditions on the bandwidth $c_B$, existence, uniqueness, consistency and asymptotic normality of ${\widehat{\mathit{\theta }}}_{i,B}\left({\gamma }_0\right)$, for fixed ${\gamma }_0$, follows from the existing results in the literature.

When ${\gamma }_0=0$ and $r_{ijl}\equiv 1$, as explained previously, the true density is a $J$–fold convolution of $f(·|{\mathit{\theta }}^{*})$, it is natural to ask the following question: if one lets ${\gamma }_0\to 0$, will the asymptotic results converge to what one would obtain by taking ${\gamma }_0=0$. We refer to this property as a continuity property in ${\gamma }_0$ of the procedure. Furthermore, it is natural to wonder if these asymptotic properties can be established uniformly in ${\gamma }_0$. If that is the case, then one can also allow ${\gamma }_0$ to depend on B. This idea has an intuitive appeal since one can choose the parameters of the sensing random variables to achieve an optimal inferential scheme. We address some of these issues in the next subsection.

Finally, we emphasize here that while we do not require $S>1$, in applications involving streaming data and privacy problems S tends to greater than one. In problems where the variance of sensing variables are large, one can obtain an overall estimator by averaging ${\widehat{\mathit{\theta }}}_{i,B}\left({\gamma }_0\right)$ over various choices of $1\le i\le S$; that is,

$$\begin{array}{c}\hfill {\widehat{\mathit{\theta }}}_B\left({\gamma }_0\right)=\frac{1}{S}\sum _{i=1}^S{\widehat{\mathit{\theta }}}_{i,B}\left({\gamma }_0\right).\end{array}$$

(5)

The averaging improves the accuracy of the estimator in small compressed samples (data not presented). For this reason, we provide results for this general case, even though our simulation and theoretical results demonstrate that for some problems considered in this paper, S can be taken to be one. We now turn to our main results which are presented in the next subsection. The following Figure 1 provides a overview of our work.

3.3. Main Results

In this section we state our main results concerning the asymptotic properties of the MHDE of compressed data $Y_{il}$. We emphasize here that we only store $\{({\tilde{Y}}_{il},r_{i·l},{\omega }_{il}^2):i=1,2,\cdots ,S;l=1,2,\cdots ,B\}.$ Specifically, we establish the continuity property in ${\gamma }_0$ of the proposed methods by establishing the existence of the iterated limits. This provides a first step in establishing the double limit. The first proposition is well-known and is concerned with the existence and uniqueness of MHDE for the location-scale family defined in () using compressed data.

Proposition 1.

Assume that $h_J(·|\mathit{\theta },{\gamma }_0)$ is a continuous density function. Assume further that if ${\mathit{\theta }}_1\ne {\mathit{\theta }}_2$. Then for every ${\gamma }_0\ge 0$, $h_J\left(y\right|{\mathit{\theta }}_1,{\gamma }_0)\ne h_J\left(y\right|{\mathit{\theta }}_2,{\gamma }_0)$ on a set of positive Lebesgue measure, the MHDE in (4) exists and is unique.

Proof. 

The proof follows from Theorem 2.2 of [20] since, without loss of generality, $\mathsf{\Theta }$ is taken to be compact and the density function $h_J(·|\mathit{\theta },{\gamma }_0)$ is continuous in $\mathit{\theta }$. □

Consistency: We next turn our attention to consistency. As explained previously, under regularity conditions for each fixed ${\gamma }_0$, the MHDE ${\widehat{\mathit{\theta }}}_{i,B}\left({\gamma }_0\right)$ is consistent for ${\mathit{\theta }}^{*}\left({\gamma }_0\right)$. The next result says that under additional conditions, the consistency property of MHDE is continuous in ${\gamma }_0$.

Proposition 2.

Let $h_J(·|\mathit{\theta },{\gamma }_0)$ be a continuous probability density function satisfying the conditions of Proposition 1. Assume that

$$\begin{array}{c}\hfill \underset{{\gamma }_0\to 0}{lim}\underset{\mathit{\theta }\in \mathsf{\Theta }}{sup}{\int }_{\mathrm{I}\mathrm{R}}|h_J\left(y\right|\mathit{\theta },{\gamma }_0)-h^{*J}\left(y\right|\mathit{\theta })|dy=0.\end{array}$$

(6)

Then, with probability one (wp1) the iterated limits also exist and equals ${\mathit{\theta }}^{*}$; that is, for $1\le i\le S$,

$$\begin{array}{c}\hfill \underset{B\to \infty }{lim}\underset{{\gamma }_0\to 0}{lim}{\widehat{\mathit{\theta }}}_{i,B}\left({\gamma }_0\right)=\underset{{\gamma }_0\to 0}{lim}\underset{B\to \infty }{lim}{\widehat{\mathit{\theta }}}_{i,B}\left({\gamma }_0\right)={\mathit{\theta }}^{*}.\end{array}$$

Proof. 

Without loss of generality let $\mathsf{\Theta }$ be compact since otherwise it can be embedded into a compact set as described in [1]. Since $f(·)$ is continuous in $\mathit{\theta }$ and $g(·)$ is continuous in ${\gamma }_0$, it follows that $h_J(·|\mathit{\theta },{\gamma }_0)$ is continuous in $\mathit{\theta }$ and ${\gamma }_0$. Hence by Theorem 1 of [1] for every fixed ${\gamma }_0\ge 0$ and $1\le i\le S$,

$$\begin{array}{c}\hfill \underset{B\to \infty }{lim}{\widehat{\mathit{\theta }}}_{i,B}\left({\gamma }_0\right)={\mathit{\theta }}^{*}\left({\gamma }_0\right).\end{array}$$

Thus, to verify the convergence of ${\mathit{\theta }}^{*}\left({\gamma }_0\right)$ to ${\mathit{\theta }}^{*}$ as ${\gamma }_0\to 0$, we first establish, using (6), that

$$\begin{array}{c}\hfill \underset{{\gamma }_0\to 0}{lim}\underset{\mathit{\theta }\in \mathsf{\Theta }}{sup}|\mathcal{A}(h_J(·|\mathit{\theta },{\gamma }_0),h^{*J}(·|\mathit{\theta }))-1|=0.\end{array}$$

To this end, we first notice that

$$\begin{array}{c}\hfill \underset{\mathit{\theta }\in \mathsf{\Theta }}{sup}H{D}^2(h_J(·|\mathit{\theta },{\gamma }_0),h^{*J}(·|\mathit{\theta }))\le \underset{\mathit{\theta }\in \mathsf{\Theta }}{sup}{\int }_{\mathrm{I}\mathrm{R}}\left|\right(h_J\left(y\right|\mathit{\theta },{\gamma }_0)-h^{*J}\left(y\right|\mathit{\theta })|dy.\end{array}$$

Hence, using (3),

$$\begin{array}{ccc}\hfill \underset{\mathit{\theta }\in \mathsf{\Theta }}{sup}|\mathcal{A}(h_J(·|\mathit{\theta },{\gamma }_0),h^{*J}(·|\mathit{\theta }))-1|& =& \frac{1}{2}\underset{\mathit{\theta }\in \mathsf{\Theta }}{sup}H{D}^2(h_J(·|\mathit{\theta },{\gamma }_0),h^{*J}(·|\mathit{\theta }))\hfill \\ & \to & 0\mathrm{as}{\gamma }_0\to 0.\hfill \end{array}$$

Hence,

$$\begin{array}{c}\hfill \underset{{\gamma }_0\to 0}{lim}\mathcal{A}(h_J(·|{\mathit{\theta }}^{*}\left({\gamma }_0\right),{\gamma }_0),h^{*J}(·|{\mathit{\theta }}^{*}\left({\gamma }_0\right)))=1.\end{array}$$

Also, by continuity,

$$\begin{array}{c}\hfill \underset{{\gamma }_0\to 0}{lim}\mathcal{A}(h^{*J}(·|{\mathit{\theta }}^{*}\left({\gamma }_0\right),{\gamma }_0),h^{*J}(·|{\mathit{\theta }}^{*}))=1,\end{array}$$

which, in turn implies that

$$\begin{array}{c}\hfill \underset{{\gamma }_0\to 0}{lim}\mathcal{A}(h_J(·|{\mathit{\theta }}^{*}\left({\gamma }_0\right),{\gamma }_0),h^{*J}(·|{\mathit{\theta }}^{*}))=1.\end{array}$$

Thus existence of the iterated limit first as $B\to \infty$ and then ${\gamma }_0\to 0$ follows using compactness of $\mathsf{\Theta }$ and the identifiability of the model. As for the other iterated limit, again notice notice that for each $1\le i\le S$, $\mathcal{A}(g_B^{\left(i\right)},h_J(·|\mathit{\theta },{\gamma }_0))$ converges to $\mathcal{A}(g_B^{\left(i\right)},h^{*J}(·|\mathit{\theta }))$ with probability one as ${\gamma }_0$ converges to 0. The result then follows again by an application of Theorem 1 of [20]. □

Remark 1.

Verification of condition (6) seems to be involved even in the case of standard Gaussian random variables and standard Gaussian sensing random variables. Indeed in this case, the density of $h_J(·|\mathit{\theta },{\gamma }_0)$ is a $J-$fold convolution of a Bessel function of second kind. It may be possible to verify the condition (6) using the properties of these functions and compactness of the parameter space Θ. However, if one is focused only on weak-consistency, it is an immediate consequence of Theorems 1 and 2 below and condition (6) is not required. Finally, it is worth mentioning here that the convergence in (6) without uniformity over Θ is a consequence of convergence in probability of $r_{ijl}$ to 1 and Glick’s Theorem.

Asymptotic limit distribution: We now proceed to investigate the limit distribution of ${\mathit{\theta }}_B^{*}\left({\gamma }_0\right)$ as $B\to \infty$ and ${\gamma }_0\to 0$. It is well-known that for fixed ${\gamma }_0\ge 0$, after centering and scaling, ${\mathit{\theta }}_B^{*}\left({\gamma }_0\right)$ has a limiting Gaussian distribution, under appropriate regularity conditions (see for instance [20]). However to evaluate the iterated limits as ${\gamma }_0\to 0$ and $B\to \infty$, additional refinements of the techniques in [20] are required. To this end, we start with additional notations. Let $s_J(·|\mathit{\theta },{\gamma }_0)=h_J^{\frac{1}{2}}(·|\mathit{\theta },{\gamma }_0)$ and let the score function be denoted by $u_J(·|\mathit{\theta },{\gamma }_0)\equiv \nabla log{h}_J(·|\mathit{\theta },{\gamma }_0)={\left(\frac{\partial log h_J(·|\mathit{\theta },{\gamma }_0)}{\partial \mu },\frac{\partial log h_J(·|\mathit{\theta },{\gamma }_0)}{\partial \sigma }\right)}^{\prime }$. Also, the Fisher information $I\left(\mathit{\theta }\left({\gamma }_0\right)\right)$ is given by

$$\begin{array}{c}\hfill I\left(\mathit{\theta }\left({\gamma }_0\right)\right)={\int }_{\mathrm{I}\mathrm{R}}{u}_J\left(y\right|\mathit{\theta },{\gamma }_0)u_J^{\prime }\left(y\right|\mathit{\theta },{\gamma }_0)h_J\left(y\right|\mathit{\theta },{\gamma }_0)dy.\end{array}$$

In addition, let ${\dot{s}}_J(·|\mathit{\theta },{\gamma }_0)$ be the gradient of $s_J(·|\mathit{\theta },{\gamma }_0)$ with respect to $\mathit{\theta }$, and ${\ddot{s}}_J(·|\mathit{\theta },{\gamma }_0)$ is the second derivative matrix of $s_J(·|\mathit{\theta },{\gamma }_0)$ with respect to $\mathit{\theta }$. In addition, let $t_J(·|\mathit{\theta })=h^{*J\frac{1}{2}}(·|\mathit{\theta })$ and $v_J(·|\mathit{\theta })=\nabla log{h}^{*J}(·|\mathit{\theta })$. Furthermore, let $Y_{il}^{*}$ denote $Y_{il}$ when ${\gamma }_0\equiv 0$. Please note that in this case, $Y_{il}=Y_{1l}$ for all $i=1,2,\cdots ,S.$ The corresponding kernel density estimate of $Y_{il}^{*}$ is given by

$$\begin{array}{c}\hfill g_B^{*}\left(y\right)=\frac{1}{B{c}_B}\sum _{l=1}^{B}K\left(\frac{y-Y_{il}^{*}}{c_B}\right).\end{array}$$

(7)

We emphasize here that we suppress i on the LHS of the above equation since $g_B^{\left(i\right)*}(·)$ are equal for all $1\le i\le S$.

The iterated limit distribution involves additional regularity conditions which are stated in the Appendix. The first step towards this aim is a representation formula which expresses the quantity of interest, viz., $\sqrt{B}({\widehat{\mathit{\theta }}}_{i,B}\left({\gamma }_0\right)-{\mathit{\theta }}^{*}\left({\gamma }_0\right))$ as a sum of two terms, one involving sums of compressed i.i.d. random variables and the other involving remainder terms that converge to 0 at a specific rate. This expression will appear in different guises in the rest of the manuscript and will play a critical role in the proofs.

3.4. Representation Formula

Before we state the lemma, we first provide two crucial assumptions that allow differentiating the objective function and interchanging the differentiation and integration:

Model assumptions on$h_J(·|\mathit{\theta },{\gamma }_0)$

(D1) $h_J(·|\mathit{\theta },{\gamma }_0)$ is twice continuously differentiable in $\mathit{\theta }$.

(D2) Assume further that $\left|\right|\nabla s_J(·|\mathit{\theta },{\gamma }_0){\left|\right|}_2$ is continuous and bounded.

Lemma 1.

Assume that the conditions (D1) and (D2) hold. Then for every $1\le i\le S$ and ${\gamma }_0\ge 0,$ the following holds:

$$\begin{array}{c}\hfill B^{\frac{1}{2}}{({\widehat{\mathit{\theta }}}_{i,B}\left({\gamma }_0\right)-{\mathit{\theta }}^{*}\left({\gamma }_0\right))}^{\prime }=A_{1B}\left({\gamma }_0\right)+A_{2B}\left({\gamma }_0\right),\mathit{where}\end{array}$$

(8)

$$\begin{array}{c}\hfill A_{1B}\left({\gamma }_0\right)=B^{\frac{1}{2}}{D}_B^{-1}({\tilde{\mathit{\theta }}}_{i,B}\left({\gamma }_0\right))T_B\left({\gamma }_0\right),A_{2B}\left({\gamma }_0\right)=B^{\frac{1}{2}}{D}_B^{-1}({\tilde{\mathit{\theta }}}_{i,B}\left({\gamma }_0\right))R_B\left({\gamma }_0\right),\end{array}$$

(9)

$$\begin{array}{c}\hfill {\tilde{\mathit{\theta }}}_{i,B}\left({\gamma }_0\right)\in U_B\left({\mathit{\theta }}^{\prime }\left({\gamma }_0\right)\right),U_B\left({\mathit{\theta }}^{\prime }\left({\gamma }_0\right)\right)=\{{\mathit{\theta }}^{\prime }:{\mathit{\theta }}^{\prime }\left({\gamma }_0\right)=t{\mathit{\theta }}^{*}\left({\gamma }_0\right)+(1-t){\widehat{\mathit{\theta }}}_{i,B}\left({\gamma }_0\right),t\in [0,1]\},\end{array}$$

(10)

$$\begin{array}{ccc}\hfill D_B\left(\mathit{\theta }\left({\gamma }_0\right)\right)& =& -\frac{1}{2}{\int }_{\mathrm{I}\mathrm{R}}{\dot{u}}_J\left(y\right|\mathit{\theta },{\gamma }_0)s_J\left(y\right|\mathit{\theta },{\gamma }_0)g_B^{\left(i\right)\frac{1}{2}}\left(y\right)dy\hfill \\ & & -\frac{1}{4}{\int }_{\mathrm{I}\mathrm{R}}{u}_J\left(y\right|\mathit{\theta },{\gamma }_0)u_J^{\prime }\left(y\right|\mathit{\theta },{\gamma }_0)s_J\left(y\right|\mathit{\theta },{\gamma }_0)g_B^{\left(i\right)\frac{1}{2}}\left(y\right)dy\hfill \\ & \equiv & D_{1B}\left(\mathit{\theta }\left({\gamma }_0\right)\right)+D_{2B}\left(\mathit{\theta }\left({\gamma }_0\right)\right),\hfill \end{array}$$

(11)

$$\begin{array}{c}\hfill T_B\left({\gamma }_0\right)\equiv \frac{1}{4}{\int }_{\mathrm{I}\mathrm{R}}{u}_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)\left(h_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)-g_B^{\left(i\right)}\left(y\right)\right)dy,\mathit{and}\end{array}$$

(12)

$$\begin{array}{c}\hfill R_B\left({\gamma }_0\right)\equiv \frac{1}{4}{\int }_{\mathrm{I}\mathrm{R}}{u}_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0){\left(h_J^{\frac{1}{2}}\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)-g_B^{\left(i\right)\frac{1}{2}}\left(y\right)\right)}^{2}dy.\end{array}$$

(13)

Proof. 

By algebra, note that ${\dot{s}}_J\left(y\right|\mathit{\theta },{\gamma }_0)=\frac{1}{2}{u}_J\left(y\right|\mathit{\theta },{\gamma }_0)s_J\left(y\right|\mathit{\theta },{\gamma }_0).$ Furthermore, the second partial derivative of $s_J(·|\mathit{\theta },{\gamma }_0)$ is given by ${\ddot{s}}_J\left(y\right|\mathit{\theta },{\gamma }_0)=\frac{1}{2}{\dot{u}}_J\left(y\right|\mathit{\theta },{\gamma }_0)s_J\left(y\right|\mathit{\theta },{\gamma }_0)+\frac{1}{4}{u}_J\left(y\right|\mathit{\theta },{\gamma }_0)u_J^{\prime }\left(y\right|\mathit{\theta },{\gamma }_0)s_J\left(y\right|\mathit{\theta },{\gamma }_0).$ Now using (D1) and (D2) and partially differentiating $H{D}_B^2\left(\mathit{\theta }\left({\gamma }_0\right)\right)\equiv H{D}^2(g_B^{\left(i\right)}(·),h_J(·|\mathit{\theta },{\gamma }_0))$ with respect to $\mathit{\theta }$ and setting it equal to 0, the estimating equations for ${\mathit{\theta }}^{*}\left({\gamma }_0\right)$ is

$$\begin{array}{c}\hfill \nabla H{D}_B^2\left({\mathit{\theta }}^{*}\left({\gamma }_0\right)\right)=0.\end{array}$$

(14)

Let ${\widehat{\mathit{\theta }}}_{i,B}\left({\gamma }_0\right)$ be the solution to (14). Now applying first order Taylor expansion of (14) we get

$$\begin{array}{c}\hfill \nabla H{D}_B^2\left({\mathit{\theta }}^{*}\left({\gamma }_0\right)\right)=\nabla H{D}_B^2({\widehat{\mathit{\theta }}}_{i,B}\left({\gamma }_0\right))+D_B({\tilde{\mathit{\theta }}}_{i,B}\left({\gamma }_0\right))({\widehat{\mathit{\theta }}}_{i,B}\left({\gamma }_0\right)-{\mathit{\theta }}^{*}\left({\gamma }_0\right)),\end{array}$$

where ${\tilde{\mathit{\theta }}}_{i,B}\left({\gamma }_0\right)$ is defined in (10), and $D_B(·)$ is given by

$$\begin{array}{ccc}\hfill D_B\left(\mathit{\theta }\left({\gamma }_0\right)\right)& =& -\frac{1}{2}{\int }_{\mathrm{I}\mathrm{R}}{\dot{u}}_J\left(y\right|\mathit{\theta },{\gamma }_0)s_J\left(y\right|\mathit{\theta },{\gamma }_0)g_B^{\left(i\right)\frac{1}{2}}\left(y\right)dy\hfill \\ & & -\frac{1}{4}{\int }_{\mathrm{I}\mathrm{R}}{u}_J\left(y\right|\mathit{\theta },{\gamma }_0)u_J^{\prime }\left(y\right|\mathit{\theta },{\gamma }_0)s_J\left(y\right|\mathit{\theta },{\gamma }_0)g_B^{\left(i\right)\frac{1}{2}}\left(y\right)dy\hfill \\ & \equiv & D_{1B}\left(\mathit{\theta }\left({\gamma }_0\right)\right)+D_{2B}\left(\mathit{\theta }\left({\gamma }_0\right)\right),\hfill \end{array}$$

and $\nabla H{D}_B^2(·)$ is given by

$$\begin{array}{c}\hfill \nabla H{D}_B^2\left(\mathit{\theta }\left({\gamma }_0\right)\right)=-\frac{1}{2}{\int }_{\mathrm{I}\mathrm{R}}{u}_J\left(y\right|\mathit{\theta },{\gamma }_0)s_J\left(y\right|\mathit{\theta },{\gamma }_0)\left(h_J^{\frac{1}{2}}\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)-g_B^{\left(i\right)\frac{1}{2}}\left(y\right)\right)dy.\end{array}$$

Thus,

$$\begin{array}{c}\hfill {\left({\widehat{\mathit{\theta }}}_{i,B}\left({\gamma }_0\right)-{\mathit{\theta }}^{*}\left({\gamma }_0\right)\right)}^{\prime }=D_B^{-1}\left({\tilde{\mathit{\theta }}}_{i,B}\left({\gamma }_0\right)\right)\nabla H{D}_B^2\left({\mathit{\theta }}^{*}\left({\gamma }_0\right)\right).\end{array}$$

By using the identity, $b^{\frac{1}{2}}-a^{\frac{1}{2}}={(2a^{\frac{1}{2}})}^{-1}((b-a)-{(b^{\frac{1}{2}}-a^{\frac{1}{2}})}^2)$, $\nabla H{D}_B^2\left({\mathit{\theta }}^{*}\left({\gamma }_0\right)\right)$ can be expressed as the difference of $T_B\left({\gamma }_0\right)$ and $R_B\left({\gamma }_0\right)$, where

$$\begin{array}{c}\hfill T_B\left({\gamma }_0\right)\equiv \frac{1}{4}{\int }_{\mathrm{I}\mathrm{R}}{u}_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)\left(h_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)-g_B^{\left(i\right)}\left(y\right)\right)dy,\end{array}$$

and

$$\begin{array}{c}\hfill R_B\left({\gamma }_0\right)\equiv \frac{1}{4}{\int }_{\mathrm{I}\mathrm{R}}{u}_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0){\left(h_J^{\frac{1}{2}}\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)-g_B^{\left(i\right)\frac{1}{2}}\left(y\right)\right)}^{2}dy.\end{array}$$

Hence,

$$\begin{array}{c}\hfill B^{\frac{1}{2}}{\left({\widehat{\mathit{\theta }}}_{i,B}\left({\gamma }_0\right)-{\mathit{\theta }}^{*}\left({\gamma }_0\right)\right)}^{\prime }=A_{1B}\left({\gamma }_0\right)+A_{2B}\left({\gamma }_0\right),\end{array}$$

where $A_{1B}\left({\gamma }_0\right)$ and $A_{2B}\left({\gamma }_0\right)$ are given in (9). □

Remark 2.

In the rest of the manuscript, we will refer to $A_{2B}\left({\gamma }_0\right)$ as the remainder term in the representation formula.

We now turn to the first main result of the manuscript, namely a central limit theorem for ${\widehat{\mathit{\theta }}}_{i,B}\left({\gamma }_0\right)$ as first $B\to \infty$ and then ${\gamma }_0\to 0$. As a first step, we note that the Fisher information of the density $h^{*J}(·|\mathit{\theta })$ is given by

$$\begin{array}{c}\hfill I\left(\mathit{\theta }\right)={\int }_{\mathrm{I}\mathrm{R}}{v}_J\left(y\right|\mathit{\theta })v_J^{\prime }\left(y\right|\mathit{\theta })h^{*J}\left(y\right|\mathit{\theta })dy.\end{array}$$

(15)

Next we state the assumptions needed in the proof of Theorem 1. We separate these conditions as (i) model assumptions, (ii) kernel assumptions, (iii) regularity conditions, (iV) conditions that allow comparison of original data and compressed data.

Model assumptions on$h^{*J}(·|\mathit{\theta })$

(D1’) $h^{*J}(·|\mathit{\theta })$ is twice continuously differentiable in $\mathit{\theta }$.

(D2’) Assume further that $\left|\right|\nabla t_J(·|\mathit{\theta }){\left|\right|}_2$ is continuous and bounded.

Kernel assumptions

(B1) $K(·)$ is symmetric about 0 on a compact support and bounded in ${\mathit{L}}_2.$ We denote the support of $K(·)$ by $Supp\left(K\right)$.

(B2) The bandwidth $c_B$ satisfies $c_B\to 0$, $B^{\frac{1}{2}}{c}_B\to \infty ,B^{\frac{1}{2}}{c}_B^2\to 0$.

Regularity conditions

(M1) The function $u_J(·|\mathit{\theta },{\gamma }_0)s_J(·|\mathit{\theta },{\gamma }_0)$ is continuously differentiable and bounded in ${\mathit{L}}_2$ at ${\mathit{\theta }}^{*}$.

(M2) The function ${\dot{u}}_J(·|\mathit{\theta },{\gamma }_0)s_J(·|\mathit{\theta },{\gamma }_0)$ is continuous and bounded in ${\mathit{L}}_2$ at ${\mathit{\theta }}^{*}$. In addition, assume that

$$\begin{array}{c}\hfill \underset{B\to \infty }{lim}{\int }_{\mathrm{I}\mathrm{R}}{\left({\dot{u}}_J\left(y\right|{\mathit{\theta }}_{i,B},{\gamma }_0)s_J\left(y\right|{\mathit{\theta }}_{i,B},{\gamma }_0)-{\dot{u}}_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)s_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)\right)}^{2}dy=0.\end{array}$$

(M3) The function $u_J(·|\mathit{\theta },{\gamma }_0)u_J^{\prime }(·|\mathit{\theta },{\gamma }_0)s_J(·|\mathit{\theta },{\gamma }_0)$ is continuous and bounded in ${\mathit{L}}_2$ at ${\mathit{\theta }}^{*}$; also,

$$\begin{array}{c}\hfill \underset{B\to \infty }{lim}{\int }_{\mathrm{I}\mathrm{R}}{(u_J(y|{\widehat{\mathit{\theta }}}_{i,B},{\gamma }_0)u_J^{\prime }(y|{\widehat{\mathit{\theta }}}_{i,B},{\gamma }_0)s_J(y|{\mathit{\theta }}_{i,B},{\gamma }_0)-u_J(y|{\mathit{\theta }}^{*},{\gamma }_0)u_J^{\prime }(y|{\mathit{\theta }}^{*},{\gamma }_0)s_J(y|{\mathit{\theta }}^{*},{\gamma }_0))}^{2}dy=0.\end{array}$$

(M4) Let $\{{\alpha }_B:B\ge 1\}$ be a sequence diverging to infinity. Assume that

$$\begin{array}{c}\hfill \underset{B\to \infty }{lim}B\underset{t\in Supp\left(K\right)}{sup}{\mathit{P}}_{{\mathit{\theta }}^{*}\left({\gamma }_0\right)}\left(|\Delta -c_{B}t|>{\alpha }_B\right)=0,\end{array}$$

where $Supp\left(\mathit{K}\right)$ is the support of the kernel density $K(·)$ and $\Delta$ is a generic random variable with density $h_J(·|{\mathit{\theta }}^{*},{\gamma }_0)$.

(M5) Let

$$\begin{array}{c}\hfill M_B=\underset{\left|y\right|\le {\alpha }_B}{sup}\underset{t\in Supp\left(K\right)}{sup}\left|\frac{h_J(y-t{c}_B|{\mathit{\theta }}^{*},{\gamma }_0)}{h_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)}\right|.\end{array}$$

Assume ${\displaystyle \underset{B\ge 1}{sup}}{M}_B<\infty$.

(M6) The score function has a regular central behavior relative to the smoothing constants, i.e.,

$$\begin{array}{c}\hfill \underset{B\to \infty }{lim}{(B^{\frac{1}{2}}{c}_B)}^{-1}{\int }_{-{\alpha }_B}^{{\alpha }_B}{u}_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)dy=0.\end{array}$$

Furthermore,

$$\begin{array}{c}\hfill \underset{B\to \infty }{lim}(B^{\frac{1}{2}}{c}_B^4){\int }_{-{\alpha }_B}^{{\alpha }_B}{u}_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)dy=0.\end{array}$$

(M7) The density functions are smooth in an ${\mathit{L}}_2$ sense; i.e.,

$$\begin{array}{c}\hfill \underset{B\to \infty }{lim}\underset{t\in Supp\left(K\right)}{sup}{\int }_{\mathrm{I}\mathrm{R}}{\left(u_J(y+c_{B}t|{\mathit{\theta }}^{*},{\gamma }_0)-u_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)\right)}^2{h}_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)dy=0.\end{array}$$

(M1’) The function $v_J(·|\mathit{\theta })t_J(·|\mathit{\theta })$ is continuously differentiable and bounded in ${\mathit{L}}_2$ at ${\mathit{\theta }}^{*}$.

(M2’) The function ${\dot{v}}_J(·|\mathit{\theta })t_J(·|\mathit{\theta })$ is continuous and bounded in ${\mathit{L}}_2$ at ${\mathit{\theta }}^{*}$. In addition, assume that

$$\begin{array}{c}\hfill \underset{B\to \infty }{lim}{\int }_{\mathrm{I}\mathrm{R}}{\left({\dot{v}}_J\left(y\right|{\mathit{\theta }}_B)t_J\left(y\right|{\mathit{\theta }}_B)-{\dot{v}}_J\left(y\right|{\mathit{\theta }}^{*})t_J\left(y\right|{\mathit{\theta }}^{*})\right)}^{2}dy=0.\end{array}$$

(M3’) The function $v_J(·|\mathit{\theta })v_J^{\prime }(·|\mathit{\theta })t_J(·|\mathit{\theta })$ is continuous and bounded in ${\mathit{L}}_2$ at ${\mathit{\theta }}^{*}$. also,

$$\begin{array}{c}\hfill \underset{B\to \infty }{lim}{\int }_{\mathrm{I}\mathrm{R}}{(v_J(y|{\widehat{\mathit{\theta }}}_{i,B})v_J^{\prime }(y|{\widehat{\mathit{\theta }}}_{i,B})t_J(y|{\widehat{\mathit{\theta }}}_{i,B})-v_J(y|{\mathit{\theta }}^{*})v_J^{\prime }(y|{\mathit{\theta }}^{*})t_J(y|{\mathit{\theta }}^{*}))}^{2}dy=0.\end{array}$$

Assumptions comparing models for original and compressed data

(O1) For all $\mathit{\theta }\in \mathsf{\Theta }$,

$$\begin{array}{c}\hfill \underset{{\gamma }_0\to 0}{lim}{\int }_{\mathrm{I}\mathrm{R}}{\left(u_J\left(y\right|\mathit{\theta },{\gamma }_0)u_J^{\prime }\left(y\right|\mathit{\theta },{\gamma }_0)s_J\left(y\right|\mathit{\theta },{\gamma }_0)-v_J\left(y\right|\mathit{\theta })v_J^{\prime }\left(y\right|\mathit{\theta })t_J\left(y\right|\mathit{\theta })\right)}^{2}dy=0.\end{array}$$

(O2) For all $\mathit{\theta }\in \mathsf{\Theta }$,

$$\begin{array}{c}\hfill \underset{{\gamma }_0\to 0}{lim}{\int }_{\mathrm{I}\mathrm{R}}{\left({\dot{u}}_J\left(y\right|\mathit{\theta },{\gamma }_0)s_J\left(y\right|\mathit{\theta },{\gamma }_0)-{\dot{v}}_J\left(y\right|\mathit{\theta })t_J\left(y\right|\mathit{\theta })\right)}^{2}dy=0.\end{array}$$

Theorem 1.

Assume that the conditions (B1)–(B2), (D1)–(D2) , (D1’)–(D2’), (M1)–(M7), (M1’)–(M3’), and (O1)–(O2) hold. Then, for every $1\le i\le S,$ the following holds:

$$\begin{array}{c}\hfill \underset{{\gamma }_0\to 0}{lim}\underset{B\to \infty }{lim}\mathit{P}\left(\sqrt{B}({\widehat{\mathit{\theta }}}_{i,B}\left({\gamma }_0\right)-{\mathit{\theta }}^{*}\left({\gamma }_0\right))\le x\right)=\mathit{P}\left(G\le x\right),\end{array}$$

where G is a bivariate Gaussian random variable with mean 0 and variance $I^{-1}\left({\mathit{\theta }}^{*}\right)$, where $I\left(\mathit{\theta }\right)$ is defined in (15).

Before we embark on the proof of Theorem 1, we first discuss the assumptions. Assumptions (B1) and (B2) are standard assumptions on the kernel and the bandwidth and are typically employed when investigating the asymptotic behavior of divergence-based estimators (see for instance [1]). Assumptions (M1)–(M7) and (M1’)–(M3’) are regularity conditions which are concerned essentially with ${\mathit{L}}_2$ continuity and boundedness of the scores and their derivatives. Assumptions (O1)–(O2) allow for comparison of $u_J(·|\mathit{\theta },{\gamma }_0)$ and $v_J(·|\mathit{\theta })$. Returning to the proof of Theorem 1, using representation formula, we will first show that ${lim}_{{\gamma }_0\to 0}{lim}_{B\to \infty }\mathit{P}\left(A_{1B}\left({\gamma }_0\right)\le x\right)=\mathit{P}\left(G\le x\right)$, and then prove that ${lim}_{{\gamma }_0\to 0}{lim}_{B\to \infty }{A}_{2B}\left({\gamma }_0\right)=0$ in probability. We start with the following proposition.

Proposition 3.

Assume that the conditions (B1), (D1)–(D2), (M1)–(M3), (M1’)–(M3’), (M7) and (O1)–(O2) hold. Then,

$$\begin{array}{c}\hfill \underset{{\gamma }_0\to 0}{lim}\underset{B\to \infty }{lim}\mathit{P}\left(A_{1B}\left({\gamma }_0\right)\le x\right)=\mathit{P}\left(G\le x\right),\end{array}$$

where G is given in Theorem 1.

We divide the proof of Proposition 3 into two lemmas. In the first lemma we will show that

$$\begin{array}{c}\hfill \underset{{\gamma }_0\to 0}{lim}\underset{B\to \infty }{lim}{D}_B({\tilde{\mathit{\theta }}}_{i,B}\left({\gamma }_0\right))=\frac{1}{4}I\left({\mathit{\theta }}^{*}\right).\end{array}$$

Next in the second lemma we will show that first letting $B\to \infty$ and then allowing ${\gamma }_0\to 0,$

$$\begin{array}{c}\hfill 4B^{\frac{1}{2}}{T}_B\left({\gamma }_0\right)\stackrel{d}{\to }N\left(0,I\left({\mathit{\theta }}^{*}\right)\right).\end{array}$$

We start with the first part.

Lemma 2.

Assume that the conditions (D1)–(D2), (D1’)–(D2’), (M1)–(M3), (M1’)–(M3’) and (O1)–(O2) hold. Then, with probability one, the following prevails:

$$\begin{array}{c}\hfill \underset{{\gamma }_0\to 0}{lim}\underset{B\to \infty }{lim}{D}_B({\tilde{\mathit{\theta }}}_{i,B}\left({\gamma }_0\right))=\frac{1}{4}I\left({\mathit{\theta }}^{*}\right).\end{array}$$

Proof. 

Using representation formula in Lemma 1. First fix ${\gamma }_0>0$. It suffices to show

$$\begin{array}{c}\hfill \underset{B\to \infty }{lim}{D}_{1B}({\tilde{\mathit{\theta }}}_{i,B}\left({\gamma }_0\right))=\frac{1}{2}I\left({\mathit{\theta }}^{*}\left({\gamma }_0\right)\right),\mathrm{and}\underset{B\to \infty }{lim}{D}_{2B}({\tilde{\mathit{\theta }}}_{i,B}\left({\gamma }_0\right))=-\frac{1}{4}I({\mathit{\theta }}^{*}\left({\gamma }_0\right)).\end{array}$$

We begin with $D_{1B}({\tilde{\mathit{\theta }}}_{i,B}\left({\gamma }_0\right))$. By algebra, $D_{1B}({\tilde{\mathit{\theta }}}_{i,B}\left({\gamma }_0\right))$ can be expressed as

$$\begin{array}{c}\hfill D_{1B}({\tilde{\mathit{\theta }}}_{i,B}\left({\gamma }_0\right))=D_{1B}^{\left(1\right)}({\tilde{\mathit{\theta }}}_{i,B}\left({\gamma }_0\right))+D_{1B}^{\left(2\right)}({\tilde{\mathit{\theta }}}_{i,B}\left({\gamma }_0\right))+D_{1B}^{\left(3\right)}({\mathit{\theta }}^{*}\left({\gamma }_0\right)),\mathrm{where}\end{array}$$

$$\begin{array}{c}\hfill D_{1B}^{\left(1\right)}({\tilde{\mathit{\theta }}}_{i,B}\left({\gamma }_0\right))=-\frac{1}{2}{\int }_{\mathrm{I}\mathrm{R}}{\dot{u}}_J(y|{\tilde{\mathit{\theta }}}_{i,B},{\gamma }_0)s_J(y|{\tilde{\mathit{\theta }}}_{i,B},{\gamma }_0)\left(g_B^{\left(i\right)\frac{1}{2}}\left(y\right)-s_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)\right)dy,\end{array}$$

$$\begin{array}{c}\hfill D_{1B}^{\left(2\right)}({\tilde{\mathit{\theta }}}_{i,B}\left({\gamma }_0\right))=-\frac{1}{2}{\int }_{\mathrm{I}\mathrm{R}}({\dot{u}}_J(y|{\tilde{\mathit{\theta }}}_{i,B},{\gamma }_0)s_J(y|{\tilde{\mathit{\theta }}}_{i,B},{\gamma }_0)-{\dot{u}}_J(y|{\mathit{\theta }}^{*},{\gamma }_0)s_J(y|{\mathit{\theta }}^{*},{\gamma }_0))h_J^{\frac{1}{2}}(y|{\mathit{\theta }}^{*},{\gamma }_0)dy,\end{array}$$

$$\begin{array}{c}\hfill \mathrm{and}{D}_{1B}^{\left(3\right)}({\mathit{\theta }}_B^{*}\left({\gamma }_0\right))=-\frac{1}{2}{\int }_{\mathrm{I}\mathrm{R}}{\dot{u}}_J(y|{\mathit{\theta }}^{*},{\gamma }_0)h_J(y|{\mathit{\theta }}^{*},{\gamma }_0)dy=\frac{1}{2}I\left({\mathit{\theta }}^{*}\left({\gamma }_0\right)\right).\end{array}$$

It suffices to show that as $B\to \infty$, $D_{1B}^{\left(1\right)}({\tilde{\mathit{\theta }}}_{i,B}\left({\gamma }_0\right))\to 0$, and $D_{1B}^{\left(2\right)}({\tilde{\mathit{\theta }}}_{i,B}\left({\gamma }_0\right))\to 0$. We first consider $D_{1B}^{\left(1\right)}({\tilde{\mathit{\theta }}}_{i,B}\left({\gamma }_0\right))$. By Cauchy-Schwarz inequality and assumption (M2), it follows that there exists $0<C_1<\infty$,

$$\begin{array}{ccc}\hfill \left|D_{1B}^{\left(1\right)}\left({\tilde{\mathit{\theta }}}_{i,B}\left({\gamma }_0\right)\right)\right|& \le & \frac{1}{2}{\left\{{\int }_{\mathrm{I}\mathrm{R}}{({\dot{u}}_J(y|{\tilde{\mathit{\theta }}}_{i,B},{\gamma }_0)s_J(y|{\tilde{\mathit{\theta }}}_{i,B},{\gamma }_0))}^{2}dy\right\}}^{\frac{1}{2}}{\left\{{\int }_{\mathrm{I}\mathrm{R}}{\left(g_B^{\left(i\right)\frac{1}{2}}\left(y\right)-s_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)\right)}^{2}dy\right\}}^{\frac{1}{2}}\hfill \\ & \le & C_1{\left\{{\int }_{\mathrm{I}\mathrm{R}}{\left(g_B^{\left(i\right)\frac{1}{2}}\left(y\right)-s_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)\right)}^{2}dy\right\}}^{\frac{1}{2}}\to 0,\hfill \end{array}$$

where the last convergence follows from the ${\mathit{L}}_1$ convergence of $g_B^{\left(i\right)}(·)$ and $h_J(·|{\mathit{\theta }}^{*},{\gamma }_0)$. Hence, as $B\to \infty$, $D_{1B}^{\left(1\right)}({\tilde{\mathit{\theta }}}_{i,B}\left({\gamma }_0\right))\to 0$. Next we consider $D_{1B}^{\left(2\right)}({\tilde{\mathit{\theta }}}_{i,B}\left({\gamma }_0\right))$. Again, by Cauchy-Schwarz inequality and assumption (M2), it follows that $D_{1B}^{\left(2\right)}({\tilde{\mathit{\theta }}}_{i,B}\left({\gamma }_0\right))\to 0$. Hence $D_{1B}({\tilde{\mathit{\theta }}}_{i,B}\left({\gamma }_0\right))\to \frac{1}{2}I({\mathit{\theta }}^{*}\left({\gamma }_0\right))$. Turning to $D_{2B}({\tilde{\mathit{\theta }}}_{i,B}\left({\gamma }_0\right))$, by similar argument, using Cauchy-Schwarz inequality and assumption (M3), it follows that $D_{2B}({\tilde{\mathit{\theta }}}_{i,B}\left({\gamma }_0\right))\to -\frac{1}{4}I({\mathit{\theta }}^{*}\left({\gamma }_0\right))$. Thus, to complete the proof, it is enough to show that

$$\begin{array}{c}\hfill \underset{{\gamma }_0\to 0}{lim}\underset{B\to \infty }{lim}{D}_{1B}({\tilde{\mathit{\theta }}}_{i,B}\left({\gamma }_0\right))=\frac{1}{2}I\left({\mathit{\theta }}^{*}\right)\mathrm{and}\underset{{\gamma }_0\to 0}{lim}\underset{B\to \infty }{lim}{D}_{2B}({\tilde{\mathit{\theta }}}_{i,B}\left({\gamma }_0\right))=-\frac{1}{4}I\left({\mathit{\theta }}^{*}\right).\end{array}$$

(16)

We start with the first term of (16). Let

$$\begin{array}{c}\hfill {\mathcal{J}}_1\left({\gamma }_0\right)={\int }_{\mathrm{I}\mathrm{R}}{\dot{u}}_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)h_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)dy-{\int }_{\mathrm{I}\mathrm{R}}{\dot{v}}_J\left(y\right|{\mathit{\theta }}^{*})h^{*J}\left(y\right|{\mathit{\theta }}^{*})dy.\end{array}$$

We will show that ${lim}_{{\gamma }_0\to 0}{\mathcal{J}}_1\left({\gamma }_0\right)=0$. By algebra, the difference of the above two terms can be expressed as the sum of ${\mathcal{J}}_{11}\left({\gamma }_0\right)$ and ${\mathcal{J}}_{12}\left({\gamma }_0\right)$, where

$$\begin{array}{c}\hfill {\mathcal{J}}_{11}\left({\gamma }_0\right)={\int }_{\mathrm{I}\mathrm{R}}\left({\dot{u}}_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)s_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)-{\dot{v}}_J\left(y\right|{\mathit{\theta }}^{*})t_J\left(y\right|{\mathit{\theta }}^{*})\right)s_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)dy,\mathrm{and}\end{array}$$

$$\begin{array}{c}\hfill {\mathcal{J}}_{12}\left({\gamma }_0\right)={\int }_{\mathrm{I}\mathrm{R}}{\dot{v}}_J\left(y\right|{\mathit{\theta }}^{*})t_J\left(y\right|{\mathit{\theta }}^{*})\left(s_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)-t_J\left(y\right|{\mathit{\theta }}^{*})\right)dy.\end{array}$$

${\mathcal{J}}_{11}\left({\gamma }_0\right)$ converges to zero by Cauchy-Schwarz inequality and assumption (O2), and ${\mathcal{J}}_{12}\left({\gamma }_0\right)$ converges to zero by Cauchy-Schwarz inequality, assumption (M2’) and Scheffe’s theorem. Next we consider the second term of (16). Let

$$\begin{array}{c}\hfill {\mathcal{J}}_2\left({\gamma }_0\right)={\int }_{\mathrm{I}\mathrm{R}}{u}_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)u_J^{\prime }\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)h_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)dy-{\int }_{\mathrm{I}\mathrm{R}}{v}_J\left(y\right|{\mathit{\theta }}^{*})v_J^{\prime }\left(y\right|{\mathit{\theta }}^{*})h^{*J}\left(y\right|{\mathit{\theta }}^{*})dy.\end{array}$$

We will show that ${lim}_{{\gamma }_0\to 0}{\mathcal{J}}_2\left({\gamma }_0\right)=0$. By algebra, the difference of the above two terms can be expressed as the sum of ${\mathcal{J}}_{21}\left({\gamma }_0\right)$ and ${\mathcal{J}}_{22}\left({\gamma }_0\right)$, where

$$\begin{array}{c}\hfill {\mathcal{J}}_{21}\left({\gamma }_0\right)={\int }_{\mathrm{I}\mathrm{R}}\left(u_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)u_J^{\prime }\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)s_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)-v_J\left(y\right|{\mathit{\theta }}^{*})v_J^{\prime }\left(y\right|{\mathit{\theta }}^{*})t_J\left(y\right|{\mathit{\theta }}^{*})\right)s_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)dy,\end{array}$$

$$\begin{array}{c}\hfill \mathrm{and}{\mathcal{J}}_{22}\left({\gamma }_0\right)={\int }_{\mathrm{I}\mathrm{R}}{v}_J\left(y\right|{\mathit{\theta }}^{*})v_J^{\prime }\left(y\right|{\mathit{\theta }}^{*})t_J\left(y\right|{\mathit{\theta }}^{*})\left(s_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)-t_J\left(y\right|{\mathit{\theta }}^{*})\right)dy.\end{array}$$

${\mathcal{J}}_{11}\left({\gamma }_0\right)$ converges to zero by Cauchy-Schwarz inequality and assumption (O1), and ${\mathcal{J}}_{12}\left({\gamma }_0\right)$ converges to zero by Cauchy-Schwarz inequality, assumption (M3’) and Scheffe’s theorem. Therefore the lemma holds. □

Lemma 3.

Assume that the conditions (B1), (D1)–(D2), (D1’)–(D2’), (M1)–(M3), (M3’), (M7) and (O1)–(O2) hold. Then, first letting $B\to \infty$, and then ${\gamma }_0\to 0,$

$$\begin{array}{c}\hfill 4B^{\frac{1}{2}}{T}_B\left({\gamma }_0\right)\stackrel{d}{\to }N\left(0,I\left({\mathit{\theta }}^{*}\right)\right).\end{array}$$

Proof. 

First fix ${\gamma }_0>0$. Please note that using ${\int }_{\mathrm{I}\mathrm{R}}{u}_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)h_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)dy=0$, we have that

$$\begin{array}{ccc}\hfill 4B^{\frac{1}{2}}{T}_B\left({\gamma }_0\right)& =& B^{\frac{1}{2}}{\int }_{\mathrm{I}\mathrm{R}}{u}_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)g_B^{\left(i\right)}\left(y\right)dy\hfill \\ & =& B^{\frac{1}{2}}{\int }_{\mathrm{I}\mathrm{R}}{u}_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)\frac{1}{B}\sum _{l=1}^B\frac{1}{c_B}K\left(\frac{y-Y_{il}}{c_B}\right)dy\hfill \\ & =& B^{\frac{1}{2}}\frac{1}{B}\sum _{l=1}^B{\int }_{\mathrm{I}\mathrm{R}}{u}_J(Y_{il}+c_{B}t|{\mathit{\theta }}^{*},{\gamma }_0)K\left(t\right)dt.\hfill \end{array}$$

Therefore,

$$\begin{array}{c}\hfill 4B^{\frac{1}{2}}{T}_B\left({\gamma }_0\right)-B^{\frac{1}{2}}\frac{1}{B}\sum _{l=1}^B{u}_J\left(Y_{il}\right|{\mathit{\theta }}^{*},{\gamma }_0)=B^{\frac{1}{2}}\frac{1}{B}\sum _{l=1}^B{\int }_{\mathrm{I}\mathrm{R}}\left(u_J(Y_{il}+c_{B}t|{\mathit{\theta }}^{*},{\gamma }_0)-u_J\left(Y_{il}\right|{\mathit{\theta }}^{*},{\gamma }_0)\right)K\left(t\right)dt.\end{array}$$

Since $Y_{il}$’s are i.i.d. across l, using Cauchy-Schwarz inequality and assumption (B1), we can show that there exists $0<C<\infty$,

$$\begin{array}{ccc}\hfill \mathit{E}{\left[4B^{\frac{1}{2}}{T}_B-B^{\frac{1}{2}}\frac{1}{B}\sum _{l=1}^B{u}_J\left(Y_{il}\right|{\mathit{\theta }}^{*},{\gamma }_0)\right]}^2& =& \mathit{E}{\left[{\int }_{\mathrm{I}\mathrm{R}}\left(u_J(Y_{i1}+c_{B}t|{\mathit{\theta }}^{*},{\gamma }_0)-u_J\left(Y_{i1}\right|{\mathit{\theta }}^{*},{\gamma }_0)\right)K\left(t\right)dt\right]}^2\hfill \\ & \le & C\mathit{E}{\left[{\left\{{\int }_{\mathrm{I}\mathrm{R}}{\left(u_J(Y_{i1}+c_{B}t|{\mathit{\theta }}^{*},{\gamma }_0)-u_J\left(Y_{i1}\right|{\mathit{\theta }}^{*},{\gamma }_0)\right)}^{2}dt\right\}}^{\frac{1}{2}}\right]}^2\hfill \\ & \le & C\mathit{E}\left[{\int }_{\mathrm{I}\mathrm{R}}{\left(u_J(Y_{i1}+c_{B}t|{\mathit{\theta }}^{*},{\gamma }_0)-u_J\left(Y_{i1}\right|{\mathit{\theta }}^{*},{\gamma }_0)\right)}^{2}dt\right]\hfill \\ & =& C{\int }_{\mathrm{I}\mathrm{R}}{\int }_{\mathrm{I}\mathrm{R}}{\left(u_J(y+c_{B}t|{\mathit{\theta }}^{*},{\gamma }_0)-u_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)\right)}^2{h}_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)dydt,\hfill \end{array}$$

converging to zero as $B\to \infty$ by assumption (M7). Also, the limiting distribution of $4B^{\frac{1}{2}}{T}_B\left({\gamma }_0\right)$ is $N(0,I\left({\mathit{\theta }}^{*}\left({\gamma }_0\right)\right))$ as $B\to \infty$. Now let ${\gamma }_0\to 0.$ It is enough to show that as ${\gamma }_0\to 0$ the density of $N(0,I\left({\mathit{\theta }}^{*}\left({\gamma }_0\right)\right))$ converges to the density of $N(0,I\left({\mathit{\theta }}^{*}\right))$. To this end, it suffices to show that ${lim}_{{\gamma }_0\to 0}I\left({\mathit{\theta }}^{*}\left({\gamma }_0\right)\right)=I\left({\mathit{\theta }}^{*}\right)$. However, this is established in Lemma 2. Combining the results, the lemma follows. □

Proof of Proposition 3.

The proof of Proposition 3 follows immediately by combining Lemmas 2 and 3. □

We now turn to establishing that the remainder term in the representation formula converges to zero.

Lemma 4.

Assume that the assumptions (B1)–(B2), (M1)–(M6) hold. Then

$$\begin{array}{c}\hfill \underset{{\gamma }_0\to 0}{lim}\underset{B\to \infty }{lim}{A}_{2B}\left({\gamma }_0\right)=0\mathit{in}\mathit{probability}.\end{array}$$

Proof. 

Using Lemma 2, it is sufficient to show that $B^{\frac{1}{2}}{R}_B$ converges to 0 in probability as $B\to \infty$. Let

$$\begin{array}{c}\hfill d_J\left(y\right|{\mathit{\theta }}^{*}\left({\gamma }_0\right))=g_B^{\left(i\right)\frac{1}{2}}\left(y\right)-s_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0).\end{array}$$

Please note that

$$\begin{array}{c}\hfill d_J^2\left(y\right|{\mathit{\theta }}^{*}\left({\gamma }_0\right))\le 2\left\{{\left(h_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)-\mathit{E}\left[g_B^{\left(i\right)}\left(y\right)\right]\right)}^2+{\left(\mathit{E}\left[g_B^{\left(i\right)}\left(y\right)\right]-g_B^{\left(i\right)}\left(y\right)\right)}^2\right\}{h}_J^{-1}\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0).\end{array}$$

Then

$$\begin{array}{ccc}\hfill |R_B\left({\gamma }_0\right)|& \le & \frac{1}{2}{\int }_{\mathrm{I}\mathrm{R}}\left|u_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)\right|d_J^2\left(y\right|{\mathit{\theta }}^{*}\left({\gamma }_0\right))dy\hfill \\ & \le & \frac{1}{2}{\int }_{-{\alpha }_B}^{{\alpha }_B}|u_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)|d_J^2\left(y\right|{\mathit{\theta }}^{*}\left({\gamma }_0\right))dy+\frac{1}{2}{\int }_{\left|y\right|\ge {\alpha }_B}\left|u_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)\right|d_J^2\left(y\right|{\mathit{\theta }}^{*}\left({\gamma }_0\right))dy\hfill \\ & \equiv & R_{1B}\left({\gamma }_0\right)+R_{2B}\left({\gamma }_0\right).\hfill \end{array}$$

We first deal with $R_{1B}\left({\gamma }_0\right)$, which can be expressed as the sum of $R_{1B}\left({\gamma }_0\right)$ and $R_{2B}\left({\gamma }_0\right)$, where

$$\begin{array}{c}\hfill R_{1B}^{\left(1\right)}\left({\gamma }_0\right)={\int }_{-{\alpha }_B}^{{\alpha }_B}\left|u_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)\right|{\left(h_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)-\mathit{E}\left[g_B^{\left(i\right)}\left(y\right)\right]\right)}^2{h}_J^{-1}\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)dy,\end{array}$$

(17)

$$\begin{array}{c}\hfill \mathrm{and}{R}_{1B}^{\left(2\right)}\left({\gamma }_0\right)={\int }_{-{\alpha }_B}^{{\alpha }_B}\left|u\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)\right|{\left(\mathit{E}\left[g_B^{\left(i\right)}\left(y\right)\right]-g_B^{\left(i\right)}\left(y\right)\right)}^2{h}_J^{-1}\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)dy.\end{array}$$

Now consider $R_{1B}^{\left(2\right)}$. Let $ϵ>0$ be arbitrary but fixed. Then, by Markov’s inequality,

$$\begin{array}{ccc}\hfill \mathit{P}\left(B^{\frac{1}{2}}{R}_{1B}^{\left(2\right)}>ϵ\right)& \le & {ϵ}^{-1}{B}^{\frac{1}{2}}\mathit{E}\left[R_{1B}^{\left(2\right)}\right]\hfill \\ & \le & {ϵ}^{-1}{B}^{\frac{1}{2}}{\int }_{{\alpha }_B}^{{\alpha }_B}\left|u_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)\right|\left(\mathit{Var}\left[g_B^{\left(i\right)}\left(y\right)\right]\right)h_J^{-1}\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)dy.\hfill \end{array}$$

(18)

Now since $Y_{il}^{\prime }s$ are independent and identically distributed across l, it follows that

$$\begin{array}{ccc}\hfill \mathit{Var}\left[g_B^{\left(i\right)}\left(y\right)\right]& \le & \frac{1}{B{c}_B}{\int }_{\mathrm{I}\mathrm{R}}{K}^2\left(t\right)h_J(y-t{c}_B|{\mathit{\theta }}^{*},{\gamma }_0)dt.\hfill \end{array}$$

(19)

Now plugging (19) into (18), interchanging the order of integration (using Tonelli’s Theorem), we get

$$\begin{array}{c}\hfill \mathit{P}\left(B^{\frac{1}{2}}{R}_{1B}^{\left(2\right)}>ϵ\right)\le C{\left(B^{\frac{1}{2}}{c}_B\right)}^{-1}{\int }_{-{\alpha }_B}^{{\alpha }_B}\left|u_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)\right|dy\to 0,\end{array}$$

where C is a universal constant, and the last convergence follows from conditions (M5)–(M6). We now deal with $R_{1B}^{\left(1\right)}$. To this end, we need to calculate ${\left(\mathit{E}\left[g_B^{\left(i\right)}\left(y\right)\right]-h_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)\right)}^2$. Using change of variables, two-step Taylor approximation, and assumption (B1), we get

$$\begin{array}{ccc}\hfill \mathit{E}\left[g_B^{\left(i\right)}\left(y\right)\right]-h_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)& =& {\int }_{\mathrm{I}\mathrm{R}}K\left(t\right)\left(h_J(y-t{c}_B|{\mathit{\theta }}^{*},{\gamma }_0)-h_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)\right)dt\hfill \\ & =& {\int }_{\mathrm{I}\mathrm{R}}K\left(t\right)\frac{{\left(t{c}_B\right)}^2}{2}{h}_J^{\prime \prime }\left(y_B^{*}\left(t\right)\right|{\mathit{\theta }}^{*},{\gamma }_0)dt.\hfill \end{array}$$

(20)

Now plugging in (20) into (17) and using conditions (M3) and (M6), we get

$$\begin{array}{c}\hfill B^{\frac{1}{2}}{R}_{1B}^{\left(1\right)}\left({\gamma }_0\right)\le C{B}^{\frac{1}{2}}{c}_B^4{\int }_{-{\alpha }_B}^{{\alpha }_B}\left|u_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)\right|dy.\end{array}$$

(21)

Convergence of (21) to 0 now follows from condition (M6). We next deal with $R_{2B}\left({\gamma }_0\right)$. To this end, by writing our the square term of $d_J(·|{\mathit{\theta }}^{*}\left({\gamma }_0\right))$, we have

$$\begin{array}{c}\hfill B^{\frac{1}{2}}{R}_{2B}\left({\gamma }_0\right)={\int }_{\left|y\right|\ge {\alpha }_B}\left|u_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)\right|\left(h_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)+g_B^{\left(i\right)}\left(y\right)-s_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)g_B^{\left(i\right)\frac{1}{2}}\left(y\right)\right)dy.\end{array}$$

(22)

We will show that the RHS of (22) converges to 0 as $B\to \infty .$ We begin with the first term. Please note that by Cauchy-Schwarz inequality,

$$\begin{array}{ccc}\hfill B{\left({\int }_{\left|y\right|\ge {\alpha }_B}\left|u_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)\right|h_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)dy\right)}^2& \le & \left\{{\int }_{\mathrm{I}\mathrm{R}}{u}_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)u_J^{\prime }\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)h_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)dy\right\}\hfill \\ & & \{B{P}_{{\mathit{\theta }}^{*}\left({\gamma }_0\right)}\left(\right|\Delta |\ge {\alpha }_B\left)\right\},\hfill \end{array}$$

the last term converges to 0 by (M4). As for the second term, note that, a.s., by Cauchy-Schwarz inequality,

$$\begin{array}{c}\hfill {\left({\int }_{\left|y\right|\ge {\alpha }_B}\left|u_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)\right|g_B^{\left(i\right)}\left(y\right)dy\right)}^2\le {\int }_{\left|y\right|\ge {\alpha }_B}{u}_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)u_J^{\prime }\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)g_B^{\left(i\right)}\left(y\right)dy.\end{array}$$

Now taking the expectation and using Cauchy-Schwarz inequality, one can show that

$$\begin{array}{c}\hfill B\mathit{E}{\left[{\int }_{\left|y\right|\ge {\alpha }_m}\left|u_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)\right|g_B^{\left(i\right)}\left(y\right)dy\right]}^2\le a_B{\int }_{\mathrm{I}\mathrm{R}}K\left(t\right){\int }_{\mathrm{I}\mathrm{R}}{u}_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)u_J^{\prime }\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)h_J(y-c_{B}t|{\mathit{\theta }}^{*},{\gamma }_0)dydt,\end{array}$$

where $a_B=B{\displaystyle \underset{z\in Supp\left(K\right)}{sup}}{P}_{{\mathit{\theta }}^{*}}\left(|\Delta -c_{B}z|>{\alpha }_B\right)$. The convergence to 0 of the RHS of above inequality now follows from condition (M4). Finally, by another application of the Cauchy-Schwarz inequality,

$$\begin{array}{c}\hfill B\mathit{E}\left[{\int }_{\left|y\right|\ge {\alpha }_m}\left|u_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)\right|g_B^{\left(i\right)\frac{1}{2}}\left(y\right)s_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)dy\right]\le a_B{\int }_{\mathrm{I}\mathrm{R}}{u}_J(y-c_{B}t|{\mathit{\theta }}^{*},{\gamma }_0)u_J^{\prime }(y-c_{B}t|{\mathit{\theta }}^{*},{\gamma }_0)h_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)dy.\end{array}$$

The convergence of RHS of above inequality to zero follows from (M4). Now the lemma follows. □

Proof of Theorem 1.

Recall that

$$\begin{array}{c}\hfill B^{\frac{1}{2}}{({\widehat{\mathit{\theta }}}_{i,B}\left({\gamma }_0\right)-{\mathit{\theta }}^{*}\left({\gamma }_0\right))}^{\prime }=A_{1B}\left({\gamma }_0\right)+A_{2B}\left({\gamma }_0\right),\end{array}$$

where $A_{1B}\left({\gamma }_0\right)$ and $A_{2B}\left({\gamma }_0\right)$ are given in (9). Proposition 3 shows that ${lim}_{{\gamma }_0\to 0}{lim}_{B\to \infty }{A}_{1B}\left({\gamma }_0\right)=N(0,I^{-1}\left({\mathit{\theta }}^{*}\right))$; while Lemma 4 shows that ${lim}_{{\gamma }_0\to 0}{lim}_{B\to \infty }{A}_{2B}\left({\gamma }_0\right)=0$ in probability. The result follows from Slutsky’s theorem. □

We next show that by interchanging the limits, namely first allowing ${\gamma }_0$ to converge to 0 and then letting $B\to \infty$ the limit distribution of ${\widehat{\mathit{\theta }}}_{i,B}\left({\gamma }_0\right)$ is Gaussian with the same covariance matrix as Theorem 1. We begin with additional assumptions required in the proof of the theorem.

Regularity conditions

(M4’) Let $\{{\alpha }_B:B\ge 1\}$ be a sequence diverging to infinity. Assume that

$$\begin{array}{c}\hfill \underset{B\to \infty }{lim}B\underset{t\in Supp\left(K\right)}{sup}{\mathit{P}}_{{\mathit{\theta }}^{*}}\left(|\Delta -c_{B}t|>{\alpha }_B\right)=0,\end{array}$$

where $Supp\left(\mathit{K}\right)$ is the support of the kernel density $K(·)$ and $\Delta$ is a generic random variable with density $h^{*J}(·|{\mathit{\theta }}^{*})$.

(M5’) Let

$$\begin{array}{c}\hfill M_B=\underset{\left|y\right|\le {\alpha }_B}{sup}\underset{t\in Supp\left(K\right)}{sup}\left|\frac{h^{*J}(y-t{c}_B|{\mathit{\theta }}^{*})}{h^{*J}\left(y\right|{\mathit{\theta }}^{*})}\right|.\end{array}$$

Assume that ${\displaystyle \underset{B\ge 1}{sup}}{M}_B<\infty$.

(M6’) The score function has a regular central behavior relative to the smoothing constants, i.e.,

$$\begin{array}{c}\hfill \underset{B\to \infty }{lim}{(B^{\frac{1}{2}}{c}_B)}^{-1}{\int }_{-{\alpha }_B}^{{\alpha }_B}{v}_J\left(y\right|{\mathit{\theta }}^{*})dy=0.\end{array}$$

Furthermore,

$$\begin{array}{c}\hfill \underset{B\to \infty }{lim}(B^{\frac{1}{2}}{c}_B^4){\int }_{-{\alpha }_B}^{{\alpha }_B}{v}_J\left(y\right|{\mathit{\theta }}^{*})dy=0.\end{array}$$

(M7’) The density functions are smooth in an ${\mathit{L}}_2$ sense; i.e.,

$$\begin{array}{c}\hfill \underset{B\to \infty }{lim}\underset{t\in Supp\left(K\right)}{sup}{\int }_{\mathrm{I}\mathrm{R}}{\left(v_J(y+c_{B}t|{\mathit{\theta }}^{*})-v_J\left(y\right|{\mathit{\theta }}^{*})\right)}^2{h}^{*J}\left(y\right|{\mathit{\theta }}^{*})dy=0.\end{array}$$

Assumptions comparing models for original and compressed data

(V1) Assume that ${lim}_{{\gamma }_0\to 0}{sup}_y|u_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)-v_J\left(y\right|{\mathit{\theta }}^{*})|=0.$

(V2) $v_J(·|\mathit{\theta })$ is ${\mathit{L}}_1$ continuous in the sense that $X_n\stackrel{p}{\to }X$ implies that $\mathit{E}\left[v_J\left(X_n\right|\mathit{\theta })-v_J\left(X\right|\mathit{\theta })\right]=0$, where the expectation is with respect to distribution $K(·)$.

(V3) Assume that for all $\mathit{\theta }\in \mathsf{\Theta }$, ${\int }_{\mathrm{I}\mathrm{R}}\nabla h^{*J}\left(y\right|\mathit{\theta })dy<\infty$.

(V4) Assume that for all $\mathit{\theta }\in \mathsf{\Theta }$, ${lim}_{{\gamma }_0\to 0}{sup}_y\left|\frac{s_J\left(y\right|\mathit{\theta },{\gamma }_0)}{t_J\left(y\right|\mathit{\theta })}-1\right|=0$.

Theorem 2.

Assume that the conditions (B1)–(B2), (D1’)–(D2’), (M1’)–(M7’), (O1)–(O2) and (V1)–(V4) hold. Then,

$$\begin{array}{c}\hfill \underset{B\to \infty }{lim}\underset{{\gamma }_0\to 0}{lim}\mathit{P}\left(\sqrt{B}({\widehat{\mathit{\theta }}}_{i,B}\left({\gamma }_0\right)-{\mathit{\theta }}^{*}\left({\gamma }_0\right))\le x\right)=\mathit{P}\left(G\le x\right),\end{array}$$

where G is a bivariate Gaussian random variable with mean 0 and variance $I^{-1}\left({\mathit{\theta }}^{*}\right)$.

We notice that in the above Theorem 2 that we use conditions (V2)–(V4) which are regularity conditions on the scores of the $J-$ fold convolution of $f(·)$ while $\left(\mathbf{V}\mathbf{1}\right)$ facilitates comparison of the scores of the densities of the compressed data and that of the $J-$fold convolution. As before, we will first establish (a):

$$\begin{array}{c}\hfill \underset{B\to \infty }{lim}\underset{{\gamma }_0\to 0}{lim}\mathit{P}\left(A_{1B}\left({\gamma }_0\right)\le x\right)=\mathit{P}\left(G\le x\right),\end{array}$$

and then (b): ${lim}_{B\to \infty }{lim}_{{\gamma }_0\to 0}{A}_{2B}\left({\gamma }_0\right)=0$ in probability. We start with the proof of (a).

Proposition 4.

Assume that the conditions (B1)–(B2), (D1’)–(D2’), (M1’)–(M3’), (M7’), (O1)–(O2), and (V1)–(V2) hold. Then,

$$\begin{array}{c}\hfill \underset{B\to \infty }{lim}\underset{{\gamma }_0\to 0}{lim}\mathit{P}\left(A_{1B}\left({\gamma }_0\right)\le x\right)=\mathit{P}\left(G\le x\right).\end{array}$$

We divide the proof of Proposition 4 into two lemmas. In the first lemma, we will show that

$$\begin{array}{c}\hfill \underset{B\to \infty }{lim}\underset{{\gamma }_0\to 0}{lim}{D}_B({\tilde{\mathit{\theta }}}_{i,B}\left({\gamma }_0\right))=\frac{1}{4}I\left({\mathit{\theta }}^{*}\right).\end{array}$$

In the second lemma, we will show that first let ${\gamma }_0\to 0$, then let $B\to \infty$,

$$\begin{array}{c}\hfill 4B^{\frac{1}{2}}{T}_B\left({\gamma }_0\right)\stackrel{d}{\to }N\left(0,I\left({\mathit{\theta }}^{*}\right)\right).\end{array}$$

Lemma 5.

Assume that the conditions (B1)–(B2), (D1’)–(D2’), (M1’)–(M3’), (O1)–(O2), and (V1)–(V2) hold. Then,

$$\begin{array}{c}\hfill \underset{B\to \infty }{lim}\underset{{\gamma }_0\to 0}{lim}{D}_B({\tilde{\mathit{\theta }}}_{i,B}\left({\gamma }_0\right))=\frac{1}{4}I\left({\mathit{\theta }}^{*}\right).\end{array}$$

(23)

Proof. 

First fix B. Recall that

$$\begin{array}{ccc}\hfill D_B\left(\mathit{\theta }\left({\gamma }_0\right)\right)& =& -\frac{1}{2}{\int }_{\mathrm{I}\mathrm{R}}{\dot{u}}_J\left(y\right|\mathit{\theta },{\gamma }_0)s_J\left(y\right|\mathit{\theta },{\gamma }_0){(g_B^{\left(i\right)}\left(y\right))}^{\frac{1}{2}}dy\hfill \\ & & -\frac{1}{4}{\int }_{\mathrm{I}\mathrm{R}}{u}_J\left(y\right|\mathit{\theta },{\gamma }_0)u_J^{\prime }\left(y\right|\mathit{\theta },{\gamma }_0)s_J\left(y\right|\mathit{\theta },{\gamma }_0){(g_B^{\left(i\right)}\left(y\right))}^{\frac{1}{2}}dy\hfill \\ & \equiv & D_{1B}\left(\mathit{\theta }\left({\gamma }_0\right)\right)+D_{2B}\left(\mathit{\theta }\left({\gamma }_0\right)\right).\hfill \end{array}$$

By algebra, $D_{1B}({\tilde{\mathit{\theta }}}_{i,B}\left({\gamma }_0\right))$ can be expressed as the sum of $H_{1B}^{\left(1\right)}$, $H_{1B}^{\left(2\right)}$, $H_{1B}^{\left(3\right)}$, $H_{1B}^{\left(4\right)}$ and $H_{1B}^{\left(5\right)}$, where

$$\begin{array}{c}\hfill H_{1B}^{\left(1\right)}=-\frac{1}{2}{\int }_{\mathrm{I}\mathrm{R}}[{\dot{u}}_J(y|{\tilde{\mathit{\theta }}}_{i,B},{\gamma }_0)s_J(y|{\tilde{\mathit{\theta }}}_{i,B},{\gamma }_0)-{\dot{v}}_J(y|{\tilde{\mathit{\theta }}}_{i,B})t_J(y|{\tilde{\mathit{\theta }}}_{i,B})]g_B^{\left(i\right)\frac{1}{2}}\left(y\right)dy,\end{array}$$

$$\begin{array}{c}\hfill H_{1B}^{\left(2\right)}=-\frac{1}{2}{\int }_{\mathrm{I}\mathrm{R}}[{\dot{v}}_J(y|{\tilde{\mathit{\theta }}}_{i,B})t_J(y|{\tilde{\mathit{\theta }}}_{i,B})-{\dot{v}}_J(y|{\mathit{\theta }}^{*})t_J(y|{\mathit{\theta }}^{*})]g_B^{\left(i\right)\frac{1}{2}}\left(y\right)dy,\end{array}$$

$$\begin{array}{c}\hfill H_{1B}^{\left(3\right)}=-\frac{1}{2}{\int }_{\mathrm{I}\mathrm{R}}{\dot{v}}_J\left(y\right|{\mathit{\theta }}^{*})t_J\left(y\right|{\mathit{\theta }}^{*})\left[g_B^{\left(i\right)\frac{1}{2}}\left(y\right)-h_J^{\frac{1}{2}}\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)\right]dy,\end{array}$$

$$\begin{array}{c}\hfill H_{1B}^{\left(4\right)}=-\frac{1}{2}{\int }_{\mathrm{I}\mathrm{R}}{\dot{v}}_J\left(y\right|{\mathit{\theta }}^{*})t_J\left(y\right|{\mathit{\theta }}^{*})\left[s_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)-t_J\left(y\right|{\mathit{\theta }}^{*})\right]dy,\mathrm{and}{H}_{1B}^{\left(5\right)}=\frac{1}{2}I\left({\mathit{\theta }}^{*}\right).\end{array}$$

We will show that

$$\begin{array}{c}\hfill \underset{{\gamma }_0\to 0}{lim}{D}_{1B}({\tilde{\mathit{\theta }}}_{i,B}\left({\gamma }_0\right))=H_{1B}^{\left(2\right)}+\underset{{\gamma }_0\to 0}{lim}{H}_{1B}^{\left(3\right)}+H_{1B}^{\left(5\right)},\end{array}$$

(24)

where

$$\underset{{\gamma }_0\to 0}{lim}{H}_{1B}^{\left(3\right)}=-\frac{1}{2}{\int }_{\mathrm{I}\mathrm{R}}{\dot{v}}_J\left(y\right|{\mathit{\theta }}^{*})t_J\left(y\right|{\mathit{\theta }}^{*})\left[g_B^{*\frac{1}{2}}\left(y\right)-t_J\left(y\right|{\mathit{\theta }}^{*})\right]dy\mathrm{and}$$

(25)

$g_B^{*}(·)$ is given in (7). First consider $H_{1B}^{\left(1\right)}$. it converges to zero as ${\gamma }_0\to 0$ by Cauchy-Schwarz inequality and assumption (O2). Next we consider $H_{1B}^{\left(3\right)}$. We will first show that

$$\begin{array}{c}\hfill \underset{{\gamma }_0\to 0}{lim}-\frac{1}{2}{\int }_{\mathrm{I}\mathrm{R}}{\dot{v}}_J\left(y\right|{\mathit{\theta }}^{*})t_J\left(y\right|{\mathit{\theta }}^{*})g_B^{\left(i\right)\frac{1}{2}}\left(y\right)dy=-\frac{1}{2}{\int }_{\mathrm{I}\mathrm{R}}{\dot{v}}_J\left(y\right|{\mathit{\theta }}^{*})t_J\left(y\right|{\mathit{\theta }}^{*})g_B^{*\frac{1}{2}}\left(y\right)dy.\end{array}$$

To this end, notice that by Cauchy-Schwarz inequality and boundedness of ${\dot{v}}_J\left(y\right|{\mathit{\theta }}^{*})t_J\left(y\right|{\mathit{\theta }}^{*})$ in ${\mathit{L}}_2$, it follows that there exists a constant C such that

$$\begin{array}{ccc}\hfill \left|{\int }_{\mathrm{I}\mathrm{R}}{\dot{v}}_J\left(y\right|{\mathit{\theta }}^{*})t_J\left(y\right|{\mathit{\theta }}^{*})\left[g_B^{\left(i\right)\frac{1}{2}}\left(y\right)-g_B^{*\frac{1}{2}}\left(y\right)\right]dy\right|& \le & C{\left\{{\int }_{\mathrm{I}\mathrm{R}}{\left(g_B^{\left(i\right)\frac{1}{2}}\left(y\right)-g_B^{*\frac{1}{2}}\left(y\right)\right)}^{2}dy\right\}}^{\frac{1}{2}}\hfill \\ & \le & C{\left\{{\int }_{\mathrm{I}\mathrm{R}}\left|g_B^{\left(i\right)}\left(y\right)-g_B^{*}\left(y\right)\right|dy\right\}}^{\frac{1}{2}}.\hfill \end{array}$$

It suffices to show that $g_B^{\left(i\right)}(·)$ converges to $g_B^{*}(·)$ in ${\mathit{L}}_1$. Since

$$\begin{array}{c}\hfill {\int }_{\mathrm{I}\mathrm{R}}|g_B^{\left(i\right)}\left(y\right)-g_B^{*}\left(y\right)|dy=2-2{\int }_{\mathrm{I}\mathrm{R}}min\left\{g_B^{\left(i\right)}\left(y\right),g_B^{*}\left(y\right)\right\}dy,\end{array}$$

and $min\left\{g_B^{\left(i\right)}\left(y\right),g_B^{*}\left(y\right)\right\}\le g_B^{*}\left(y\right)$, by dominated convergence theorem, $g_B^{\left(i\right)}(·)\stackrel{{\mathit{L}}_1}{\to }{g}_B^{*}(·)$. Next we will show that

$$\begin{array}{c}\hfill \underset{{\gamma }_0\to 0}{lim}-\frac{1}{2}{\int }_{\mathrm{I}\mathrm{R}}{\dot{v}}_J\left(y\right|{\mathit{\theta }}^{*})t_J\left(y\right|{\mathit{\theta }}^{*})s_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)dy=-\frac{1}{2}{\int }_{\mathrm{I}\mathrm{R}}{\dot{v}}_J\left(y\right|{\mathit{\theta }}^{*})t_J\left(y\right|{\mathit{\theta }}^{*})t_J\left(y\right|{\mathit{\theta }}^{*})dy.\end{array}$$

In addition, by Cauchy-Schwarz inequality, boundedness of ${\dot{v}}_J\left(y\right|{\mathit{\theta }}^{*})t_J\left(y\right|{\mathit{\theta }}^{*})$ in ${\mathit{L}}_2$ and Scheffe’s theorem, we have that ${\int }_{\mathrm{I}\mathrm{R}}{\dot{v}}_J\left(y\right|{\mathit{\theta }}^{*})h_J^{\frac{1}{2}}\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)\left(s_J\left(y\right|{\mathit{\theta }}^{*}\left|{\gamma }_0\right)-t_J\left(y\right|{\mathit{\theta }}^{*})\right)dy$ converges to zero as ${\gamma }_0\to 0.$ Next we consider $H_{1B}^{\left(4\right)}$. it converges to zero by Cauchy-Schwarz inequality and assumption (M2’). Thus (24) holds. Now let $B\to \infty ,$ we will show that ${lim}_{B\to \infty }{H}_{1B}^{\left(2\right)}=0$ and ${lim}_{B\to \infty }{lim}_{{\gamma }_0\to 0}{H}_{1B}^{\left(3\right)}=0.$ First consider ${lim}_{B\to \infty }{H}_{1B}^{\left(2\right)}$. It converges to zero by Cauchy-Schwarz inequality and assumption (M2’). Next we consider ${lim}_{B\to \infty }{lim}_{{\gamma }_0\to 0}{H}_{1B}^{\left(3\right)}$. It converges to zero by Cauchy-Schwarz inequality and ${\mathit{L}}_1$ convergence of $g_B^{*}(·)$ and $h^{*J}(·|{\mathit{\theta }}^{*})$. Therefore ${lim}_{B\to \infty }{lim}_{{\gamma }_0\to 0}{D}_{1B}({\tilde{\mathit{\theta }}}_{i,B}\left({\gamma }_0\right))=\frac{1}{2}I\left({\mathit{\theta }}^{*}\right)$.

We now turn to show that ${lim}_{B\to \infty }{lim}_{{\gamma }_0\to 0}{D}_{2B}({\tilde{\mathit{\theta }}}_{i,B}\left({\gamma }_0\right))=-\frac{1}{4}I\left({\mathit{\theta }}^{*}\right)$. First fix B and express $D_{2B}({\tilde{\mathit{\theta }}}_{i,B}\left({\gamma }_0\right))$ as the sum of $H_{2B}^{\left(1\right)}$, $H_{2B}^{\left(2\right)}$, $H_{2B}^{\left(3\right)}$, $H_{2B}^{\left(4\right)}$, and $H_{2B}^{\left(5\right)}$, where

$$\begin{array}{c}\hfill H_{2B}^{\left(1\right)}=-\frac{1}{4}{\int }_{\mathrm{I}\mathrm{R}}\left[u_J(y|{\tilde{\mathit{\theta }}}_{i,B},{\gamma }_0)u_J^{\prime }(y|{\tilde{\mathit{\theta }}}_{i,B},{\gamma }_0)s_J(y|{\tilde{\mathit{\theta }}}_{i,B},{\gamma }_0)-v_J(y|{\tilde{\mathit{\theta }}}_{i,B})v_J^{\prime }(y|{\tilde{\mathit{\theta }}}_{i,B})t_J(y|{\tilde{\mathit{\theta }}}_{i,B})\right]g_B^{\left(i\right)\frac{1}{2}}\left(y\right)dy,\end{array}$$

$$\begin{array}{c}\hfill H_{2B}^{\left(2\right)}=-\frac{1}{4}{\int }_{\mathrm{I}\mathrm{R}}\left[v_J(y|{\tilde{\mathit{\theta }}}_{i,B})v_J^{\prime }(y|{\tilde{\mathit{\theta }}}_{i,B})t_J(y|{\tilde{\mathit{\theta }}}_{i,B})-v_J(y|{\mathit{\theta }}^{*})v_J^{\prime }(y|{\mathit{\theta }}^{*})t_J(y|{\mathit{\theta }}^{*})\right]g_B^{\left(i\right)\frac{1}{2}}\left(y\right)dy,\end{array}$$

$$\begin{array}{c}\hfill H_{2B}^{\left(3\right)}=-\frac{1}{4}{\int }_{\mathrm{I}\mathrm{R}}{v}_J\left(y\right|{\mathit{\theta }}^{*})v_J^{\prime }\left(y\right|{\mathit{\theta }}^{*})t_J\left(y\right|{\mathit{\theta }}^{*})\left[g_B^{\left(i\right)\frac{1}{2}}\left(y\right)-h_J^{\frac{1}{2}}\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)\right]dy,\end{array}$$

$$\begin{array}{c}\hfill H_{2B}^{\left(4\right)}=-\frac{1}{4}{\int }_{\mathrm{I}\mathrm{R}}{v}_J\left(y\right|{\mathit{\theta }}^{*})v_J^{\prime }\left(y\right|{\mathit{\theta }}^{*})t_J\left(y\right|{\mathit{\theta }}^{*})\left[s_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)-t_J\left(y\right|{\mathit{\theta }}^{*})\right]dy,\mathrm{and}{H}_{2B}^{\left(5\right)}=-\frac{1}{4}I\left({\mathit{\theta }}^{*}\right).\end{array}$$

We will show that

$$\begin{array}{c}\hfill \underset{{\gamma }_0\to 0}{lim}{D}_{2B}({\tilde{\mathit{\theta }}}_{i,B}\left({\gamma }_0\right))=H_{2B}^{\left(2\right)}+\underset{{\gamma }_0\to 0}{lim}{H}_{2B}^{\left(3\right)}+H_{2B}^{\left(5\right)},\mathrm{where}\end{array}$$

(26)

$$\underset{{\gamma }_0\to 0}{lim}{H}_{2B}^{\left(3\right)}=-\frac{1}{2}{\int }_{\mathrm{I}\mathrm{R}}{v}_J\left(y\right|{\mathit{\theta }}^{*})v_J^{\prime }\left(y\right|{\mathit{\theta }}^{*})t_J\left(y\right|{\mathit{\theta }}^{*})\left[g_B^{*\frac{1}{2}}\left(y\right)-t_J\left(y\right|{\mathit{\theta }}^{*})\right]dy.$$

(27)

First consider $H_{2B}^{\left(1\right)}$. It converges to zero as ${\gamma }_0\to 0$ by Cauchy-Schwarz inequality and assumption (O1). Next consider $H_{2B}^{\left(3\right)}$. By similar argument as above and boundedness of $v_J^2\left(y\right|{\mathit{\theta }}^{*})t_J\left(y\right|{\mathit{\theta }}^{*})$, it follows that (27) holds. Next consider $H_{2B}^{\left(4\right)}$. It converges to zero as ${\gamma }_0\to 0$ by Cauchy-Schwarz inequality and assumption (M3’). Now let $B\to \infty ,$ we will show that ${lim}_{B\to \infty }{H}_{2B}^{\left(2\right)}=0$ and ${lim}_{B\to \infty }{lim}_{{\gamma }_0\to 0}{H}_{2B}^{\left(3\right)}=0$. First consider $H_{2B}^{\left(2\right)}$. It converges to zero by Cauchy-Schwarz inequality and assumption (M3’) as $B\to \infty$. Finally consider ${lim}_{B\to \infty }{lim}_{{\gamma }_0\to 0}{H}_{2B}^{\left(3\right)}$. It converges to zero by Cauchy-Schwarz inequality and ${\mathit{L}}_1$ convergence of $g_B^{*}(·)$ and $h^{*J}(·|{\mathit{\theta }}^{*})$. Thus ${lim}_{B\to \infty }{lim}_{{\gamma }_0\to 0}{D}_{2B}({\tilde{\mathit{\theta }}}_{i,B}\left({\gamma }_0\right))=-\frac{1}{4}I\left({\mathit{\theta }}^{*}\right)$. Now letting $B\to \infty$, the proof of (23) follows using arguments similar to the one in Lemma 2. □

Lemma 6.

Assume that the conditions (B1)–(B2),(D1’)–(D2’), (M1’)–(M3’), (M7’), (O1)–(O2), and (V1)–(V2) hold. Then, first letting $B\to \infty$, and then letting ${\gamma }_0\to 0,$

$$\begin{array}{c}\hfill 4B^{\frac{1}{2}}{T}_B\left({\gamma }_0\right)\stackrel{d}{\to }N\left(0,I\left({\mathit{\theta }}^{*}\right)\right).\end{array}$$

(28)

Proof. 

First fix B. We will show that as ${\gamma }_0\to 0,$

$$\begin{array}{c}\hfill 4B^{\frac{1}{2}}{T}_B\left({\gamma }_0\right)\stackrel{d}{\to }{\int }_{\mathrm{I}\mathrm{R}}{v}_J\left(y\right|{\mathit{\theta }}^{*})g_B^{*}\left(y\right)dy.\end{array}$$

First observe that

$$\begin{array}{ccc}\hfill 4B^{\frac{1}{2}}{T}_B\left({\gamma }_0\right)-{\int }_{\mathrm{I}\mathrm{R}}{v}_J\left(y\right|{\mathit{\theta }}^{*})g_B^{*}\left(y\right)dy& =& {\int }_{\mathrm{I}\mathrm{R}}\left[u_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)-v_J\left(y\right|{\mathit{\theta }}^{*})\right]g_B^{\left(i\right)}\left(y\right)dy\hfill \end{array}$$

(29)

$$\begin{array}{cc}& +{\int }_{\mathrm{I}\mathrm{R}}{v}_J\left(y\right|{\mathit{\theta }}^{*})\left[g_B^{\left(i\right)}\left(y\right)-g_B^{*}\left(y\right)\right]dy.\hfill \end{array}$$

(30)

We will show that the RHS of (29) converges to zero as ${\gamma }_0\to 0$ and the RHS of (30) converges to zero in probability as ${\gamma }_0\to 0$. First consider the RHS of (29). Since

$$\begin{array}{ccc}\hfill {\int }_{\mathrm{I}\mathrm{R}}\left[u_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)-v_J\left(y\right|{\mathit{\theta }}^{*})\right]g_B^{\left(i\right)}\left(y\right)dy& \le & {\int }_{\mathrm{I}\mathrm{R}}\underset{y}{sup}|u_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)-v_J\left(y\right|{\mathit{\theta }}^{*})|g_B^{\left(i\right)}\left(y\right)dy,\hfill \end{array}$$

which converges to zero as ${\gamma }_0\to 0$ by assumption (V1). Next consider the RHS of (30). Since

$$\begin{array}{ccc}\hfill {\int }_{\mathrm{I}\mathrm{R}}{v}_J\left(y\right|{\mathit{\theta }}^{*})\left[g_B^{\left(i\right)}\left(y\right)-g_B^{*}\left(y\right)\right]dy& =& \frac{1}{B}\sum _{l=1}^B{\int }_{\mathrm{I}\mathrm{R}}\left[v_J(Y_{il}+u{c}_B)-v_J(Y_{il}^{*}+u{c}_B)\right]K\left(u\right)du.\hfill \end{array}$$

By assumption (V2), it follows that as ${\gamma }_0\to 0$, (30) converges to zero in probability. Now letting $B\to \infty$, we have

$$\begin{array}{c}\hfill B^{\frac{1}{2}}{\int }_{\mathrm{I}\mathrm{R}}{v}_J\left(y\right|{\mathit{\theta }}^{*})g_B^{*}\left(y\right)dy-B^{\frac{1}{2}}\frac{1}{B}\sum _{l=1}^B{v}_J\left(Y_{il}^{*}\right|{\mathit{\theta }}^{*})=B^{\frac{1}{2}}\frac{1}{B}\sum _{l=1}^B{\int }_{\mathrm{I}\mathrm{R}}\left(v_J(Y_{il}^{*}+c_{B}t|{\mathit{\theta }}^{*})-v_J\left(Y_{il}^{*}\right|{\mathit{\theta }}^{*})\right)K\left(t\right)dt,\end{array}$$

and

$$\begin{array}{ccc}\hfill \mathit{E}{\left[B^{\frac{1}{2}}{\int }_{\mathrm{I}\mathrm{R}}{v}_J\left(y\right|{\mathit{\theta }}^{*})g_B^{*}\left(y\right)dy-B^{\frac{1}{2}}\frac{1}{B}\sum _{l=1}^B{v}_J\left(Y_{il}^{*}\right|{\mathit{\theta }}^{*})\right]}^2& =& \mathit{E}{\left[B^{\frac{1}{2}}\frac{1}{B}\sum _{l=1}^B{\int }_{\mathrm{I}\mathrm{R}}\left(v_J(Y_{il}^{*}+c_{B}t|{\mathit{\theta }}^{*})-v_J\left(Y_{il}^{*}\right|{\mathit{\theta }}^{*})\right)K\left(t\right)dt\right]}^2\hfill \\ & \le & C\mathit{E}\left[{\int }_{\mathrm{I}\mathrm{R}}{\left(v_J(Y_{i1}^{*}+c_{B}t|{\mathit{\theta }}^{*})-v_J\left(Y_{i1}^{*}\right|{\mathit{\theta }}^{*})\right)}^{2}dt\right]\hfill \\ & =& C{\int }_{\mathrm{I}\mathrm{R}}{\int }_{\mathrm{I}\mathrm{R}}{\left(v_J(y+c_{B}t|{\mathit{\theta }}^{*})-v_J\left(y\right|{\mathit{\theta }}^{*})\right)}^2{h}^{*J}\left(y\right|{\mathit{\theta }}^{*})dydt\hfill \\ & \to & 0\mathrm{as}B\to \infty ,\hfill \end{array}$$

where the last convergence follows by assumption (M7’). Hence, using the Central limit theorem for independent and identically distributed random variables it follows that the limiting distribution of $B^{\frac{1}{2}}{\int }_{\mathrm{I}\mathrm{R}}{v}_J\left(y\right|{\mathit{\theta }}^{*})g_B^{*}\left(y\right)dy$ is $N(0,I\left({\mathit{\theta }}^{*}\right))$, proving the lemma. □

Proof of Proposition 4.

The proof of Proposition 4 follows by combining Lemmas 5 and 6. □

Lemma 7.

Assume that the conditions (M1’)–(M6’) and (V1)–(V4) hold. Then,

$$\begin{array}{c}\hfill \underset{B\to \infty }{lim}\underset{{\gamma }_0\to 0}{lim}{A}_{2B}\left({\gamma }_0\right)=0\mathit{in}\mathit{probability}.\end{array}$$

Proof. 

First fix B. Let

$$\begin{array}{c}\hfill {\mathcal{H}}_B\left({\gamma }_0\right)={\int }_{\mathrm{I}\mathrm{R}}{u}_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0){\left[h_J^{\frac{1}{2}}\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)-g_B^{\left(i\right)}\left(y\right)\right]}^{2}dy-{\int }_{\mathrm{I}\mathrm{R}}{v}_J\left(y\right|{\mathit{\theta }}^{*}){\left[t_J\left(y\right|{\mathit{\theta }}^{*})-g_B^{*}\left(y\right)\right]}^{2}dy.\end{array}$$

we will show that as ${\gamma }_0\to 0$, ${\mathcal{H}}_B\left({\gamma }_0\right)\to 0$. By algebra, ${\mathcal{H}}_B\left({\gamma }_0\right)$ can be written as the sum of ${\mathcal{H}}_{1B}\left({\gamma }_0\right)$ and ${\mathcal{H}}_{2B}\left({\gamma }_0\right)$, where

$$\begin{array}{c}\hfill {\mathcal{H}}_{1B}\left({\gamma }_0\right)={\int }_{\mathrm{I}\mathrm{R}}\left(u_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)-v_J\left(y\right|{\mathit{\theta }}^{*})\right){\left[h_J^{\frac{1}{2}}\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)-g_B^{\left(i\right)}\left(y\right)\right]}^{2}dy,\mathrm{and}\end{array}$$

$$\begin{array}{c}\hfill {\mathcal{H}}_{2B}\left({\gamma }_0\right)={\int }_{\mathrm{I}\mathrm{R}}{v}_J\left(y\right|{\mathit{\theta }}^{*}){\left[h_J^{\frac{1}{2}}\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)-g_B^{\left(i\right)}\left(y\right)\right]}^{2}dy.\end{array}$$

First consider ${\mathcal{H}}_{1B}\left({\gamma }_0\right)$. It is bounded above by $C{sup}_y|u_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)-v_J\left(y\right|{\mathit{\theta }}^{*})|$, which converges to zero as ${\gamma }_0\to 0$ by assumption (V1), where C is a constant. Next consider ${\mathcal{H}}_{2B}\left({\gamma }_0\right)$. We will show that ${\mathcal{H}}_{2B}\left({\gamma }_0\right)$ converges to

$$\begin{array}{c}\hfill {\int }_{\mathrm{I}\mathrm{R}}{v}_J\left(y\right|{\mathit{\theta }}^{*}){\left[t_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)-g_B^{*}\left(y\right)\right]}^{2}dy.\end{array}$$

In fact, the difference of ${\mathcal{H}}_{2B}\left({\gamma }_0\right)$ and the above formula can be expressed as the sum of ${\mathcal{H}}_{2B}^{\left(1\right)}\left({\gamma }_0\right)$, ${\mathcal{H}}_{2B}^{\left(2\right)}\left({\gamma }_0\right),$ and ${\mathcal{H}}_{2B}^{\left(3\right)}\left({\gamma }_0\right)$, where

$$\begin{array}{c}\hfill {\mathcal{H}}_{2B}^{\left(1\right)}\left({\gamma }_0\right)={\int }_{\mathrm{I}\mathrm{R}}{v}_J\left(y\right|{\mathit{\theta }}^{*})\left(h_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)-h^{*J}\left(y\right|{\mathit{\theta }}^{*})\right)dy,\end{array}$$

$$\begin{array}{c}\hfill {\mathcal{H}}_{2B}^{\left(2\right)}\left({\gamma }_0\right)={\int }_{\mathrm{I}\mathrm{R}}{v}_J\left(y\right|{\mathit{\theta }}^{*})\left(g_B^{\left(i\right)}\left(y\right)-g_B^{*}\left(y\right)\right)dy,\mathrm{and}\end{array}$$

$$\begin{array}{c}\hfill {\mathcal{H}}_{2B}^{\left(3\right)}\left({\gamma }_0\right)={\int }_{\mathrm{I}\mathrm{R}}{v}_J\left(y\right|{\mathit{\theta }}^{*})\left(h_J^{\frac{1}{2}}\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)g_B^{\left(i\right)}\left(y\right)-t_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)g_B^{*}\left(y\right)\right)dy.\end{array}$$

First consider ${\mathcal{H}}_{2B}^{\left(1\right)}\left({\gamma }_0\right)$. Please note that

$$\begin{array}{ccc}\hfill \left|{\mathcal{H}}_{2B}^{\left(1\right)}\left({\gamma }_0\right)\right|& \le & {\int }_{\mathrm{I}\mathrm{R}}|\nabla h^{*J}\left(y\right|{\mathit{\theta }}^{*})|\left|\frac{h_J\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)}{h^{*J}\left(y\right|{\mathit{\theta }}^{*})}-1\right|dy\hfill \\ & \le & \left\{{\left(\underset{y}{sup}\left|\frac{s_J\left(y\right|\mathit{\theta },{\gamma }_0)}{t_J\left(y\right|\mathit{\theta })}-1\right|\right)}^2+2\underset{y}{sup}\left|\frac{s_J\left(y\right|\mathit{\theta },{\gamma }_0)}{t_J\left(y\right|\mathit{\theta })}-1\right|\right\}{\int }_{\mathrm{I}\mathrm{R}}|\nabla h^{*J}\left(y\right|{\mathit{\theta }}^{*})|dy,\hfill \end{array}$$

which converges to 0 as ${\gamma }_0\to 0$ by assumptions (V3) and (V4). Next we consider ${\mathcal{H}}_{2B}^{\left(2\right)}\left({\gamma }_0\right)$. Since

$$\begin{array}{c}\hfill {\mathcal{H}}_{2B}^{\left(2\right)}\left({\gamma }_0\right)=\frac{1}{B}\sum _{l=1}^B{\int }_{\mathrm{I}\mathrm{R}}\left(v_J(Y_{il}+u{c}_B|{\mathit{\theta }}^{*})-v_J(Y_{il}^{*}+u{c}_B|{\mathit{\theta }}^{*})\right)K\left(u\right)du,\end{array}$$

which converges to zero as ${\gamma }_0\to 0$ due to assumption (V2). Finally consider ${\mathcal{H}}_{2B}^{\left(3\right)}\left({\gamma }_0\right)$, which can be expressed as the sum of ${\mathcal{L}}_{1B}\left({\gamma }_0\right)$ and ${\mathcal{L}}_{2B}$, where

$$\begin{array}{c}\hfill {\mathcal{L}}_{1B}\left({\gamma }_0\right)={\int }_{\mathrm{I}\mathrm{R}}{v}_J\left(y\right|{\mathit{\theta }}^{*})\left(h_J^{\frac{1}{2}}\left(y\right|{\mathit{\theta }}^{*},{\gamma }_0)-t_J\left(y\right|{\mathit{\theta }}^{*})\right)g_B^{\left(i\right)\frac{1}{2}}\left(y\right)dy,\mathrm{and}\end{array}$$

$$\begin{array}{c}\hfill {\mathcal{L}}_{2B}={\int }_{\mathrm{I}\mathrm{R}}{v}_J\left(y\right|{\mathit{\theta }}^{*})t_J\left(y\right|{\mathit{\theta }}^{*})\left(g_B^{\left(i\right)\frac{1}{2}}\left(y\right)-g_B^{*\frac{1}{2}}\left(y\right)\right)dy.\end{array}$$

First consider ${\mathcal{L}}_{1B}\left({\gamma }_0\right)$. Notice that

$$\begin{array}{c}\hfill \left|{\mathcal{L}}_{1B}\left({\gamma }_0\right)\right|\le \underset{y}{sup}\left|\frac{s_J\left(y\right|\mathit{\theta },{\gamma }_0)}{t_J\left(y\right|\mathit{\theta })}-1\right|{\int }_{\mathrm{I}\mathrm{R}}{v}_J\left(y\right|{\mathit{\theta }}^{*})t_J\left(y\right|{\mathit{\theta }}^{*})g_B^{\left(i\right)\frac{1}{2}}\left(y\right)dy\to 0,\end{array}$$

where the last convergence follows by Cauchy-Schwarz inequality and assumption (V4). Next we consider ${\mathcal{L}}_{2B}$. By Cauchy-Schwarz inequality, it is bounded above by

$$\begin{array}{c}\hfill {\left\{{\int }_{\mathrm{I}\mathrm{R}}{v}_J\left(y\right|{\mathit{\theta }}^{*})v_J^{\prime }\left(y\right|{\mathit{\theta }}^{*})h^{*J}\left(y\right|{\mathit{\theta }}^{*})dy\right\}}^{\frac{1}{2}}{\left\{{\int }_{\mathrm{I}\mathrm{R}}{\left(g_B^{\left(i\right)\frac{1}{2}}\left(y\right)-g_B^{*\frac{1}{2}}\left(y\right)\right)}^{2}dy\right\}}^{\frac{1}{2}}.\end{array}$$

(31)

Equation (31) converges to zero as ${\gamma }_0\to 0$ by boundedness of ${\int }_{\mathrm{I}\mathrm{R}}{v}_J\left(y\right|{\mathit{\theta }}^{*})v_J^{\prime }\left(y\right|{\mathit{\theta }}^{*})h^{*J}\left(y\right|{\mathit{\theta }}^{*})dy$ and ${\mathit{L}}_1$ convergence between $g_B^{\left(i\right)}(·)$ and $g_B^{*}(·)$, where the ${\mathit{L}}_1$ convergence has already been established in Lemma 5. Now letting $B\to \infty$, following similar argument as Lemma 4 and assumptions (M1’)–(M6’), the lemma follows. □

Proof of Theorem 2.

Recall that

$$\begin{array}{c}\hfill B^{\frac{1}{2}}{({\widehat{\mathit{\theta }}}_{i,B}\left({\gamma }_0\right)-{\mathit{\theta }}^{*}\left({\gamma }_0\right))}^{\prime }=A_{1B}\left({\gamma }_0\right)+A_{2B}\left({\gamma }_0\right).\end{array}$$

Proposition 4 shows that first letting ${\gamma }_0\to 0,$ then $B\to \infty$, $A_{1B}\left({\gamma }_0\right)\stackrel{d}{\to }N(0,I^{-1}\left({\mathit{\theta }}^{*}\right))$; while Lemma 7 shows that ${lim}_{B\to \infty }{lim}_{{\gamma }_0\to 0}{A}_{2B}\left({\gamma }_0\right)=0$ in probability. The theorem follows from Slutsky’s theorem. □

Remark 3.

The above two theorems (Theorems 1 and 2) do not immediately imply the double limit exists. This requires stronger conditions and more delicate calculations and will be considered elsewhere.

3.5. Robustness of MHDE

In this section, we describe the robustness properties of MHDE for compressed data. Accordingly, let $h_{J,\alpha ,z}(·|\mathit{\theta },{\gamma }_0)\equiv (1-\alpha )h_J(·|\mathit{\theta },{\gamma }_0)+\alpha {\eta }_z$, where ${\eta }_z$ denotes the uniform density on the interval $(z-ϵ,z+ϵ)$, where $ϵ>0$ is small, $\mathit{\theta }\in \mathsf{\Theta }$, $\alpha \in (0,1)$, and $z\in \mathrm{I}\mathrm{R}$. Also, let $s_{J,\alpha ,z}\left(y\right|\mathit{\theta },{\gamma }_0)=h_{J,\alpha ,z}^{\frac{1}{2}}\left(y\right|\mathit{\theta },{\gamma }_0)$, $u_{J,\alpha ,z}\left(y\right|\mathit{\theta },{\gamma }_0)=\nabla log{h}_{J,\alpha ,z}\left(y\right|\mathit{\theta },{\gamma }_0)$, $h_{\alpha ,z}^{*J}(·|\mathit{\theta })\equiv (1-\alpha )h^{*J}(·|\mathit{\theta })+\alpha {\eta }_z$, $s_{\alpha ,z}^{*J}(·|\mathit{\theta })=h_{\alpha ,z}^{*J\frac{1}{2}}(·|\mathit{\theta })$, and $u_{\alpha ,z}^{*J}=\nabla log{h}_{\alpha ,z}^{*J}(·|\mathit{\theta })$. Before we state the theorem, we describe certain additional assumptions-which are essentially ${\mathit{L}}_2-$ continuity conditions-that are needed in the proof.

Model assumptions for robustness analysis

(O3) For $\alpha \in [0,1]$ and all $\mathit{\theta }\in \mathsf{\Theta }$,

$$\begin{array}{c}\hfill \underset{{\gamma }_0\to 0}{lim}{\int }_{\mathrm{I}\mathrm{R}}{\left({\dot{u}}_{J,\alpha ,z}\left(y\right|\mathit{\theta },{\gamma }_0)s_{J,\alpha ,z}\left(y\right|\mathit{\theta },{\gamma }_0)-{\dot{u}}_{\alpha ,z}^{*J}\left(y\right|\mathit{\theta })s_{\alpha ,z}^{*J}\left(y\right|\mathit{\theta })\right)}^{2}dy=0.\end{array}$$

(O4) For $\alpha \in [0,1]$ and all $\mathit{\theta }\in \mathsf{\Theta }$,

$$\begin{array}{c}\hfill \underset{{\gamma }_0\to 0}{lim}{\int }_{\mathrm{I}\mathrm{R}}{\left(u_{J,\alpha ,z}\left(y\right|\mathit{\theta },{\gamma }_0)u_{J,\alpha ,z}^{\prime }\left(y\right|\mathit{\theta },{\gamma }_0)s_{J,\alpha ,z}\left(y\right|\mathit{\theta },{\gamma }_0)-u_{\alpha ,z}^{*J}\left(y\right|\mathit{\theta })u_{\alpha ,z}^{*J^{\prime }}\left(y\right|\mathit{\theta })s_{\alpha ,z}^{*J}\left(y\right|\mathit{\theta })\right)}^{2}dy=0.\end{array}$$

Theorem 3.

(i) Let $\alpha \in (0,1),$ and assume that for all $\mathit{\theta }\in \mathsf{\Theta }$, and assume that the assumptions of Proposition 1 hold, also assume that $T\left(h_{J,\alpha ,z}(·|\mathit{\theta },{\gamma }_0)\right)$ is unique for all z. Then, $T\left(h_{J,\alpha ,z}(·|\mathit{\theta },{\gamma }_0)\right)$ is a bounded, continuous function of z and

$$\begin{array}{c}\hfill \underset{{\gamma }_0\to 0}{lim}\underset{\left|z\right|\to \infty }{lim}T\left(h_{J,\alpha ,z}(·|\mathit{\theta },{\gamma }_0)\right)=\mathit{\theta };\end{array}$$

(32)

(ii) Assume further that the conditions (V1), (M2)-(M3), and (O3)-(O4) hold. Then,

$$\begin{array}{c}\hfill \underset{{\gamma }_0\to 0}{lim}\underset{\alpha \to 0}{lim}{\alpha }^{-1}\left[T\left(h_{J,\alpha ,z}(·|\mathit{\theta },{\gamma }_0)\right)-\mathit{\theta }\right]={\left[I\left(\mathit{\theta }\right)\right]}^{-1}{\int }_{\mathrm{I}\mathrm{R}}\left[{\eta }_z\left(y\right)v_J\left(y\right|\mathit{\theta })\right]dy,\end{array}$$

Proof. 

Let ${\mathit{\theta }}_z\left({\gamma }_0\right)$ denote $T\left(h_{J,\alpha ,z}(·|\mathit{\theta },{\gamma }_0)\right)$ and let ${\mathit{\theta }}_z$ denote $T\left(h_{\alpha ,z}^{*J}(·|\mathit{\theta })\right)$ We first show that (32) holds. Let ${\gamma }_0\ge 0$ be fixed. Then, by triangle inequality,

$$\begin{array}{c}\hfill \underset{\left|z\right|\to \infty }{lim}|{\mathit{\theta }}_z\left({\gamma }_0\right)-\mathit{\theta }|\le \underset{\left|z\right|\to \infty }{lim}|{\mathit{\theta }}_z\left({\gamma }_0\right)-\mathit{\theta }\left({\gamma }_0\right)|+\underset{\left|z\right|\to \infty }{lim}|\mathit{\theta }\left({\gamma }_0\right)-\mathit{\theta }|.\end{array}$$

(33)

We will show that the first term of RHS of (33) is equal to zero. Suppose that it is not zero, without loss of generality, by going to a subsequence if necessary, we may assume that ${\mathit{\theta }}_z\to {\mathit{\theta }}_1\ne \mathit{\theta }$ as $\left|z\right|\to \infty$. Since ${\mathit{\theta }}_z\left({\gamma }_0\right)$ minimizes $H{D}^2(h_J(·|\mathit{\theta },{\gamma }_0),h_J(·|\mathit{\theta },{\gamma }_0))$, it follows that

$$\begin{array}{c}\hfill H{D}^2(h_{J,\alpha ,z}(·|\mathit{\theta },{\gamma }_0),h_J(·|{\mathit{\theta }}_z,{\gamma }_0))\le H{D}^2(h_{J,\alpha ,z}(·|\mathit{\theta },{\gamma }_0),h_J(·|{\mathit{\theta }}^{\prime },{\gamma }_0))\end{array}$$

(34)

for every ${\mathit{\theta }}^{\prime }\in \mathsf{\Theta }$. We now show that as $\left|z\right|\to \infty$,

$$\begin{array}{c}\hfill H{D}^2(h_{J,\alpha ,z}(·|\mathit{\theta },{\gamma }_0),h_J(·|{\mathit{\theta }}_z,{\gamma }_0))\to H{D}^2((1-\alpha )h_J(·|\mathit{\theta },{\gamma }_0),h_J(·|{\mathit{\theta }}_1,{\gamma }_0)).\end{array}$$

(35)

To this end, note that as $\left|z\right|\to \infty ,$ for every y,

$$\begin{array}{c}\hfill h_{J,\alpha ,z}\left(y\right|\mathit{\theta },{\gamma }_0)\to (1-\alpha )h_J\left(y\right|\mathit{\theta },{\gamma }_0),\mathrm{and}{h}_J\left(y\right|{\mathit{\theta }}_z,{\gamma }_0)\to h_J\left(y\right|{\mathit{\theta }}_1,{\gamma }_0)\end{array}$$

Therefore, as $\left|z\right|\to \infty ,$

$$\begin{array}{ccc}\hfill \left|H{D}^2(h_{J,\alpha ,z}(·|\mathit{\theta },{\gamma }_0),h_J(·|{\mathit{\theta }}_z,{\gamma }_0))-H{D}^2((1-\alpha )h_J(·|\mathit{\theta },{\gamma }_0),h_J(·|{\mathit{\theta }}_1,{\gamma }_0))\right|& \le & 2(Q_1+Q_2),\hfill \end{array}$$

where

$$\begin{array}{c}\hfill Q_1={\int }_{\mathrm{I}\mathrm{R}}\left|h_{J,\alpha ,z}^{\frac{1}{2}}\left(y\right|\mathit{\theta },{\gamma }_0)-{\left((1-\alpha )h_J\left(y\right|\mathit{\theta },{\gamma }_0)\right)}^{\frac{1}{2}}\right|{\left(h_J\left(y\right|{\mathit{\theta }}_z,{\gamma }_0))\right)}^{\frac{1}{2}}dy,\end{array}$$

$$\begin{array}{c}\hfill Q_2={\int }_{\mathrm{I}\mathrm{R}}\left|h_J^{\frac{1}{2}}\left(y\right|{\mathit{\theta }}_z,{\gamma }_0)-{\left(h_J\left(y\right|{\mathit{\theta }}_1,{\gamma }_0)\right)}^{\frac{1}{2}}\right|{\left((1-\alpha )h_J\left(y\right|\mathit{\theta },{\gamma }_0)\right)}^{\frac{1}{2}}dy.\end{array}$$

Now, by Cauchy-Schwarz inequality and Scheffe’s theorem, it follows that as $\left|z\right|\to \infty$, $Q_1\to 0$ and $Q_2\to 0$. Therefore, (35) holds. By Equations (34) and (35), we have

$$\begin{array}{c}\hfill H{D}^2((1-\alpha )h_J(·|\mathit{\theta },{\gamma }_0),h_J(·|{\mathit{\theta }}_1,{\gamma }_0))\le H{D}^2((1-\alpha )h_J(·|\mathit{\theta },{\gamma }_0),h_J(·|{\mathit{\theta }}^{\prime },{\gamma }_0))\end{array}$$

(36)

for every ${\mathit{\theta }}^{\prime }\in \mathsf{\Theta }$. Now consider

$$\begin{array}{c}\hfill HIF(\alpha ,h_J(·|\mathit{\theta },{\gamma }_0),h_J(·|{\mathit{\theta }}^{\prime },{\gamma }_0))\equiv {\int }_{\mathrm{I}\mathrm{R}}{\left({\left[(1-\alpha )\delta (h_J(·|\mathit{\theta },{\gamma }_0),h_J\left(y\right|{\mathit{\theta }}^{\prime },{\gamma }_0))+1\right]}^{\frac{1}{2}}-1\right)}^2{h}_J\left(y\right|{\mathit{\theta }}^{\prime },{\gamma }_0)dy,\end{array}$$

where $\delta (h_J(·|\mathit{\theta },{\gamma }_0),h_J\left(y\right|{\mathit{\theta }}^{\prime },{\gamma }_0))=\frac{h_J\left(y\right|\mathit{\theta },{\gamma }_0)}{h_J\left(y\right|{\mathit{\theta }}^{\prime },{\gamma }_0)}-1$. Since $G^{*}\left(\delta \right)={\left[{\left((1-\alpha )\delta +1\right)}^{\frac{1}{2}}-1\right]}^2$ is a non-negative and strictly convex function with $\delta =0$ as the unique point of minimum. Hence $HIF(\alpha ,h_J(·|\mathit{\theta },{\gamma }_0),h_J(·|{\mathit{\theta }}^{\prime },{\gamma }_0))>0$ unless $\delta (h_J(·|\mathit{\theta },{\gamma }_0),h_J\left(y\right|{\mathit{\theta }}^{\prime },{\gamma }_0))=0$ on a set of Lebesgue measure zero, which by the model identifiability assumption , is true if and only if ${\mathit{\theta }}^{\prime }=\mathit{\theta }$. Since ${\mathit{\theta }}_1\ne \mathit{\theta }$, it follows that

$$\begin{array}{c}\hfill HIF(\alpha ,h_J(·|\mathit{\theta },{\gamma }_0),h_J(·|{\mathit{\theta }}_1,{\gamma }_0))>HIF(\alpha ,h_J(·|\mathit{\theta },{\gamma }_0),h_J(·|\mathit{\theta },{\gamma }_0)).\end{array}$$

Since $HIF(\alpha ,h_J(·|\mathit{\theta },{\gamma }_0),h_J(·|{\mathit{\theta }}^{\prime },{\gamma }_0))=H{D}^2((1-\alpha )h_J(·|\mathit{\theta },{\gamma }_0),h_J(·|{\mathit{\theta }}^{\prime },{\gamma }_0))-\alpha$. This implies that

$$\begin{array}{c}\hfill H{D}^2((1-\alpha )h_J(·|\mathit{\theta },{\gamma }_0),h_J(·|{\mathit{\theta }}_1,{\gamma }_0))>H{D}^2((1-\alpha )h_J(·|\mathit{\theta },{\gamma }_0),h_J(·|{\mathit{\theta }}^{\prime },{\gamma }_0)),\end{array}$$

which contradicts (36). The continuity of ${\mathit{\theta }}_z$ follows from Proposition 2 and the boundedness follows from the compactness of $\mathsf{\Theta }$. Now let ${\gamma }_0\to 0$, the second term of RHS of (33) converges to zero by Proposition 2.

We now turn to part (ii) of the Theorem. First fix ${\gamma }_0\ge 0.$ Since ${\mathit{\theta }}_z$ minimizes $H^2(h_{J,\alpha ,z}(·|\mathit{\theta },{\gamma }_0),t\left({\gamma }_0\right))$ over $\mathsf{\Theta }$. By Taylor expansion of $H{D}^2(h_{J,\alpha ,z}(·|\mathit{\theta },{\gamma }_0),h_J(·|{\mathit{\theta }}_z,{\gamma }_0))$ around $\mathit{\theta }$, we get

$$\begin{array}{ccc}\hfill 0=\nabla H{D}^2(h_{J,\alpha ,z}(·|\mathit{\theta },{\gamma }_0),h_J(·|{\mathit{\theta }}_z,{\gamma }_0))& =& H{D}^2(h_{J,\alpha ,z}(·|\mathit{\theta },{\gamma }_0),h_J(·|\mathit{\theta },{\gamma }_0))\hfill \\ & & +({\mathit{\theta }}_z-\mathit{\theta })D(h_{J,\alpha ,z}(·|\mathit{\theta },{\gamma }_0),h_J(·|{\mathit{\theta }}_z^{*},{\gamma }_0)),\hfill \end{array}$$

where ${\mathit{\theta }}_z^{*}\left({\gamma }_0\right)$ is a point between $\mathit{\theta }$ and ${\mathit{\theta }}_z$,

$$\begin{array}{c}\hfill \nabla H{D}^2(h_{J,\alpha ,z}(·|\mathit{\theta },{\gamma }_0),h_J(·|\mathit{\theta },{\gamma }_0))=\frac{1}{2}{\int }_{\mathrm{I}\mathrm{R}}{u}_{J,\alpha ,z}\left(y\right|\mathit{\theta },{\gamma }_0)s_{J,\alpha ,z}\left(y\right|\mathit{\theta },{\gamma }_0)\left(s_{J,\alpha ,z}\left(y\right|\mathit{\theta },{\gamma }_0)-s_J\left(y\right|\mathit{\theta },{\gamma }_0)\right)dy,\end{array}$$

(37)

and $D(h_{J,\alpha ,z}(·|\mathit{\theta },{\gamma }_0),h_J(·|{\mathit{\theta }}^{\prime },{\gamma }_0))$ can be expressed the sum of $D_1(h_{J,\alpha ,z}(·|\mathit{\theta },{\gamma }_0),h_J(·|{\mathit{\theta }}^{\prime },{\gamma }_0))$ and $D_2(h_{J,\alpha ,z}(·|\mathit{\theta },{\gamma }_0),h_J(·|{\mathit{\theta }}^{\prime },{\gamma }_0))$, where

$$\begin{array}{c}\hfill D_1(h_{J,\alpha ,z}(·|\mathit{\theta },{\gamma }_0),h_J(·|{\mathit{\theta }}^{\prime },{\gamma }_0))=\frac{1}{2}{\int }_{\mathrm{I}\mathrm{R}}{\dot{u}}_{J,\alpha ,z}\left(y\right|\mathit{\theta },{\gamma }_0)s_{J,\alpha ,z}\left(y\right|\mathit{\theta },{\gamma }_0)s_J\left(y\right|{\mathit{\theta }}^{\prime },{\gamma }_0)dy\mathrm{and}\end{array}$$

(38)

$$\begin{array}{c}\hfill D_2(h_{J,\alpha ,z}(·|\mathit{\theta },{\gamma }_0),h_J(·|{\mathit{\theta }}^{\prime },{\gamma }_0))=\frac{1}{4}{\int }_{\mathrm{I}\mathrm{R}}{u}_{J,\alpha ,z}\left(y\right|\mathit{\theta },{\gamma }_0)u_{J,\alpha ,z}^{\prime }\left(y\right|\mathit{\theta },{\gamma }_0)s_{J,\alpha ,z}\left(y\right|\mathit{\theta },{\gamma }_0)s_J\left(y\right|{\mathit{\theta }}^{\prime },{\gamma }_0)dy.\end{array}$$

(39)

Therefore,

$$\begin{array}{c}\hfill {\alpha }^{-1}\left({\mathit{\theta }}_z-\mathit{\theta }\right)=-{\alpha }^{-1}{D}^{-1}(h_{J,\alpha ,z}(·|\mathit{\theta },{\gamma }_0),h_J(·|{\mathit{\theta }}_z^{*},{\gamma }_0))\nabla H{D}^2(h_{J,\alpha ,z}(·|\mathit{\theta },{\gamma }_0),h_J(·|\mathit{\theta },{\gamma }_0)).\end{array}$$

We will show that

$$\begin{array}{c}\hfill \underset{\alpha \to 0}{lim}D(h_{J,\alpha ,z}(·|\mathit{\theta },{\gamma }_0),h_J(·|{\mathit{\theta }}_z^{*},{\gamma }_0))=-\frac{1}{4}I\left(\mathit{\theta }\left({\gamma }_0\right)\right),\mathrm{and}\end{array}$$

(40)

$$\begin{array}{c}\hfill \underset{\alpha \to 0}{lim}{\alpha }^{-1}\nabla H{D}^2(h_{J,\alpha ,z}(·|\mathit{\theta },{\gamma }_0),h_J(·|\mathit{\theta },{\gamma }_0))=\frac{1}{4}{\int }_{\mathrm{I}\mathrm{R}}\left[{\eta }_z\left(y\right)u_J\left(y\right|\mathit{\theta },{\gamma }_0)\right]dy.\end{array}$$

(41)

We will first establish (40). Please note that as $\alpha \to 0,$ by definition ${\mathit{\theta }}_z\left(\alpha \right)\to \mathit{\theta }.$ Thus, ${lim}_{\alpha \to 0}{\mathit{\theta }}_z^{*}\left(\alpha \right)=\mathit{\theta }.$ In addition, by assumptions (O3) and (O4), $D(h_{J,\alpha ,z}(·|\mathit{\theta },{\gamma }_0),h_J(·|{\mathit{\theta }}_z,{\gamma }_0))$ is continuous in ${\mathit{\theta }}_z$. Therefore, to prove (40), it suffices to show that

$$\begin{array}{c}\hfill \underset{\alpha \to 0}{lim}{D}_1(h_{J,\alpha ,z}(·|\mathit{\theta },{\gamma }_0),h_J(·|\mathit{\theta },{\gamma }_0))=-\frac{1}{2}{\int }_{\mathrm{I}\mathrm{R}}{\dot{u}}_{J,\alpha ,z}\left(y\right|\mathit{\theta },{\gamma }_0)h_J\left(y\right|\mathit{\theta },{\gamma }_0)dy=-\frac{1}{2}I\left(\mathit{\theta }\left({\gamma }_0\right)\right),\mathrm{and}\end{array}$$

(42)

$$\begin{array}{c}\hfill \underset{\alpha \to 0}{lim}{D}_2(h_{J,\alpha ,z}(·|\mathit{\theta },{\gamma }_0),h_J(·|\mathit{\theta },{\gamma }_0))=\frac{1}{4}{\int }_{\mathrm{I}\mathrm{R}}{u}_{J,\alpha ,z}\left(y\right|\mathit{\theta },{\gamma }_0)u_{J,\alpha ,z}^{\prime }\left(y\right|\mathit{\theta },{\gamma }_0)h_J\left(y\right|\mathit{\theta },{\gamma }_0)dy=\frac{1}{4}I\left(\mathit{\theta }\left({\gamma }_0\right)\right).\end{array}$$

(43)

We begin with $D_2(h_{J,\alpha ,z}(·|\mathit{\theta },{\gamma }_0),h_J(·|\mathit{\theta },{\gamma }_0))$. Notice that

$$\begin{array}{c}\hfill \underset{\alpha \to 0}{lim}{s}_{J,\alpha ,z}\left(y\right|\mathit{\theta },{\gamma }_0)=s_J\left(y\right|\mathit{\theta },{\gamma }_0),\underset{\alpha \to 0}{lim}{u}_{J,\alpha ,z}\left(y\right|\mathit{\theta },{\gamma }_0)=u_J\left(y\right|\mathit{\theta },{\gamma }_0),\mathrm{and}\end{array}$$

$$\begin{array}{c}\hfill \underset{\alpha \to 0}{lim}{\dot{u}}_{J,\alpha ,z}\left(y\right|\mathit{\theta },{\gamma }_0)={\dot{u}}_J\left(y\right|\mathit{\theta },{\gamma }_0).\end{array}$$

Thus,

$$\begin{array}{c}\hfill \underset{\alpha \to 0}{lim}{\dot{u}}_{J,\alpha ,z}\left(y\right|\mathit{\theta },{\gamma }_0)s_{J,\alpha ,z}\left(y\right|\mathit{\theta },{\gamma }_0)s_J\left(y\right|\mathit{\theta },{\gamma }_0)={\dot{u}}_J\left(y\right|\mathit{\theta },{\gamma }_0)h_J\left(y\right|\mathit{\theta },{\gamma }_0).\end{array}$$

In addition, in order to pass the limit inside the integral, note that, for every component of matrix $u_{J,\alpha ,z}(·|\mathit{\theta },{\gamma }_0)u_{J,\alpha ,z}^{\prime }(·|\mathit{\theta },{\gamma }_0)$, we have

$$\begin{array}{ccc}\hfill \left|u_{J,\alpha ,z}\left(y\right|\mathit{\theta },{\gamma }_0)u_{J,\alpha ,z}^{\prime }\left(y\right|\mathit{\theta },{\gamma }_0)\right|& =& \left|\left(\frac{(1-\alpha )\nabla h_{J,\alpha ,z}\left(y\right|\mathit{\theta },{\gamma }_0)}{(1-\alpha )h_{J,\alpha ,z}\left(y\right|\mathit{\theta },{\gamma }_0)+\alpha {\eta }_z\left(y\right)}\right){\left(\frac{(1-\alpha )\nabla h_{J,\alpha ,z}\left(y\right|\mathit{\theta },{\gamma }_0)}{(1-\alpha )h_{J,\alpha ,z}\left(y\right|\mathit{\theta },{\gamma }_0)+\alpha {\eta }_z\left(y\right)}\right)}^{\prime }\right|\hfill \\ & =& \left|\left(\frac{\nabla h_{J,\alpha ,z}\left(y\right|\mathit{\theta },{\gamma }_0)}{h_{J,\alpha ,z}\left(y\right|\mathit{\theta },{\gamma }_0)+\frac{\alpha }{1-\alpha }{\eta }_z\left(y\right)}\right){\left(\frac{\nabla h_{J,\alpha ,z}\left(y\right|\mathit{\theta },{\gamma }_0)}{h_{J,\alpha ,z}\left(y\right|\mathit{\theta },{\gamma }_0)+\frac{\alpha }{1-\alpha }{\eta }_z\left(y\right)}\right)}^{\prime }\right|\hfill \\ & \le & \left|\left(\frac{\nabla h_{J,\alpha ,z}\left(y\right|\mathit{\theta },{\gamma }_0)}{h_{J,\alpha ,z}\left(y\right|\mathit{\theta },{\gamma }_0)}\right){\left(\frac{\nabla h_{J,\alpha ,z}\left(y\right|\mathit{\theta },{\gamma }_0)}{h_{J,\alpha ,z}\left(y\right|\mathit{\theta },{\gamma }_0)}\right)}^{\prime }\right|=\left|u_J\left(y\right|\mathit{\theta },{\gamma }_0)u_J^{\prime }\left(y\right|\mathit{\theta },{\gamma }_0)\right|,\hfill \end{array}$$

where $|·|$ represents the absolute function for each component of the matrix, and

$$\begin{array}{c}\hfill \left|s_{J,\alpha ,z}\left(y\right|\mathit{\theta },{\gamma }_0)\right|\le {\left[h_J\left(y\right|\mathit{\theta },{\gamma }_0)+{\eta }_z\left(y\right)\right]}^{\frac{1}{2}}.\end{array}$$

Now choosing the dominating function

$$\begin{array}{c}\hfill m_J^{\left(1\right)}\left(y\right|\mathit{\theta },{\gamma }_0)=\left|u_J\left(y\right|\mathit{\theta },{\gamma }_0)u_J^{\prime }\left(y\right|\mathit{\theta },{\gamma }_0)\right|{\left[h_J\left(y\right|\mathit{\theta },{\gamma }_0)+{\eta }_z\left(y\right)\right]}^{\frac{1}{2}}{s}_J\left(y\right|\mathit{\theta },{\gamma }_0)\end{array}$$

and applying Cauchy-Schwarz inequality, we obtain that there exists a constant C such that

$$\begin{array}{ccc}\hfill {\int }_{\mathrm{I}\mathrm{R}}\left|m_J^{\left(1\right)}\left(y\right|\mathit{\theta },{\gamma }_0)\right|dy& \le & C{\left\{{\int }_{\mathrm{I}\mathrm{R}}{\left(u_J\left(y\right|\mathit{\theta },{\gamma }_0)u_J^{\prime }\left(y\right|\mathit{\theta },{\gamma }_0)s_J\left(y\right|\mathit{\theta },{\gamma }_0)\right)}^{2}dy\right\}}^{\frac{1}{2}},\hfill \end{array}$$

which is finite by assumption (M2). Hence, by the dominated convergence theorem, (43) holds. Turning to (42), notice that for each component of the matrix ${\dot{u}}_{J,\alpha ,z}\left(y\right|\mathit{\theta },{\gamma }_0)$,

$$\begin{array}{ccc}\hfill \left|{\dot{u}}_{J,\alpha ,z}\left(y\right|\mathit{\theta },{\gamma }_0)\right|& =& \left|\frac{{\ddot{h}}_J\left(y\right|\mathit{\theta },{\gamma }_0)\left[h_J\left(y\right|\mathit{\theta },{\gamma }_0)+\frac{\alpha }{1-\alpha }{\eta }_z\left(y\right)\right]-\left(\nabla h_J\left(y\right|\mathit{\theta },{\gamma }_0)\right){\left(\nabla h_J\left(y\right|\mathit{\theta },{\gamma }_0)\right)}^{\prime }}{{\left(h_J\left(y\right|\mathit{\theta },{\gamma }_0)+\frac{\alpha }{1-\alpha }{\eta }_z\left(y\right)\right)}^2}\right|\hfill \\ & \le & \left|\frac{{\ddot{h}}_J\left(y\right|\mathit{\theta },{\gamma }_0)}{h_J\left(y\right|\mathit{\theta },{\gamma }_0)}\right|+\left|\frac{\left(\nabla h_J\left(y\right|\mathit{\theta },{\gamma }_0)\right){\left(\nabla h_J\left(y\right|\mathit{\theta },{\gamma }_0)\right)}^{\prime }}{h_J^2\left(y\right|\mathit{\theta },{\gamma }_0)}\right|,\hfill \end{array}$$

where $|·|$ denotes the absolute function for each component. Now choosing the dominating function

$$\begin{array}{c}\hfill m_J^{\left(2\right)}\left(y\right|\mathit{\theta },{\gamma }_0)=\left(\left|\frac{{\ddot{h}}_J\left(y\right|\mathit{\theta },{\gamma }_0)}{h_J\left(y\right|\mathit{\theta },{\gamma }_0)}\right|+\left|\frac{\left(\nabla h_J\left(y\right|\mathit{\theta },{\gamma }_0)\right){\left(\nabla h_J\left(y\right|\mathit{\theta },{\gamma }_0)\right)}^{\prime }}{h_J^2\left(y\right|\mathit{\theta },{\gamma }_0)}\right|\right){\left[h_J\left(y\right|\mathit{\theta },{\gamma }_0)+{\eta }_z\left(y\right)\right]}^{\frac{1}{2}}{s}_J\left(y\right|\mathit{\theta },{\gamma }_0),\end{array}$$

and applying the Cauchy-Schwarz inequality it follows, using (M3), that

$$\begin{array}{c}\hfill {\int }_{\mathrm{I}\mathrm{R}}\left|m_J^{\left(2\right)}\left(y\right|\mathit{\theta },{\gamma }_0)\right|dy<\infty .\end{array}$$

Finally, by the dominated convergence theorem, it follows that

$$\begin{array}{c}\hfill \underset{\alpha \to 0}{lim}{D}_1(h_{J,\alpha ,z}(·|\mathit{\theta },{\gamma }_0),h_J(·|{\mathit{\theta }}_z^{*},{\gamma }_0))=-\frac{1}{2}I\left(\mathit{\theta }\left({\gamma }_0\right)\right).\end{array}$$

Therefore (40) follows. It remains to show that (41) holds. To this end, note that

$$\begin{array}{ccc}\hfill \nabla H{D}^2(h_{J,\alpha ,z}(·|\mathit{\theta },{\gamma }_0),h_J(·|\mathit{\theta },{\gamma }_0))& =& -\frac{1}{2}{\int }_{\mathrm{I}\mathrm{R}}{s}_{J,\alpha ,z}\left(y\right|\mathit{\theta },{\gamma }_0)u_{J,\alpha ,z}\left(y\right|\mathit{\theta },{\gamma }_0)s_J\left(y\right|\mathit{\theta },{\gamma }_0)dy.\hfill \end{array}$$

Now taking partial derivative of $H{D}^2(h_{J,\alpha ,z}(·|\mathit{\theta },{\gamma }_0),h_J(·|\mathit{\theta },{\gamma }_0))$ with respect to $\alpha$, it can be expressed as the sum of ${\mathcal{U}}_1$, ${\mathcal{U}}_2$ and ${\mathcal{U}}_3$, where

$$\begin{array}{c}\hfill {\mathcal{U}}_1=-\frac{1}{4}{\int }_{\mathrm{I}\mathrm{R}}\frac{-h_J\left(y\right|\mathit{\theta },{\gamma }_0)+{\eta }_z\left(y\right)}{s_{J,\alpha ,z}\left(y\right|\mathit{\theta },{\gamma }_0)}{u}_{J,\alpha ,z}\left(y\right|\mathit{\theta },{\gamma }_0)s_J\left(y\right|\mathit{\theta },{\gamma }_0)dy,\end{array}$$

$$\begin{array}{c}\hfill {\mathcal{U}}_2=-\frac{1}{2}{\int }_{\mathrm{I}\mathrm{R}}{s}_{J,\alpha ,z}\left(y\right|\mathit{\theta },{\gamma }_0)\frac{-\nabla h_J\left(y\right|\mathit{\theta },{\gamma }_0)h_{J,\alpha ,z}\left(y\right|\mathit{\theta },{\gamma }_0)}{h_{J,\alpha ,z}^2\left(y\right|\mathit{\theta },{\gamma }_0)}{s}_J\left(y\right|\mathit{\theta },{\gamma }_0)dy,\mathrm{and}\end{array}$$

$$\begin{array}{c}\hfill {\mathcal{U}}_3=-\frac{1}{2}{\int }_{\mathrm{I}\mathrm{R}}{s}_{J,\alpha ,z}\left(y\right|\mathit{\theta },{\gamma }_0)\frac{-(1-\alpha )\nabla h_J\left(y\right|\mathit{\theta },{\gamma }_0)(-h_J\left(y\right|\mathit{\theta },{\gamma }_0)+{\eta }_z\left(y\right))}{h_{J,\alpha ,z}^2\left(y\right|\mathit{\theta },{\gamma }_0)}{s}_J\left(y\right|\mathit{\theta },{\gamma }_0)dy.\end{array}$$

By dominated convergence theorem (using similar idea as above to find dominating functions), we have

$$\begin{array}{c}\hfill \underset{\alpha \to 0}{lim}\frac{\partial \nabla H{D}^2(h_{J,\alpha ,z}(·|\mathit{\theta },{\gamma }_0),h_J(·|\mathit{\theta },{\gamma }_0))}{\partial \alpha }=\frac{1}{4}{\int }_{\mathrm{I}\mathrm{R}}{u}_J\left(y\right|\mathit{\theta },{\gamma }_0){\eta }_z\left(y\right)dy.\end{array}$$

Hence, by L’Hospital rule, (41) holds. It remains to show that

$$\begin{array}{c}\hfill \underset{{\gamma }_0\to 0}{lim}\underset{\alpha \to 0}{lim}D(h_{J,\alpha ,z}(·|\mathit{\theta },{\gamma }_0),h_J(·|\mathit{\theta },{\gamma }_0))=-\frac{1}{4}I\left(\mathit{\theta }\right),\mathrm{and}\end{array}$$

(44)

$$\begin{array}{c}\hfill \underset{{\gamma }_0\to 0}{lim}\underset{\alpha \to 0}{lim}{\alpha }^{-1}\nabla H{D}^2(h_{J,\alpha ,z}(·|\mathit{\theta },{\gamma }_0),h_J(·|\mathit{\theta },{\gamma }_0))=\frac{1}{4}{\int }_{\mathrm{I}\mathrm{R}}\left[{\eta }_z\left(y\right)v_J\left(y\right|\mathit{\theta })\right]dy.\end{array}$$

(45)

We start with (44). Since for fixed ${\gamma }_0\ge 0$, by the above argument, it follows that

$$\begin{array}{c}\hfill \underset{\alpha \to 0}{lim}D(h_{J,\alpha ,z}(·|\mathit{\theta },{\gamma }_0),h_J(·|\mathit{\theta },{\gamma }_0))=-\frac{1}{4}I\left(\mathit{\theta }\left({\gamma }_0\right)\right)=-\frac{1}{4}{\int }_{\mathrm{I}\mathrm{R}}{u}_J\left(y\right|\mathit{\theta },{\gamma }_0)u_J^{\prime }\left(y\right|\mathit{\theta },{\gamma }_0)h_J\left(y\right|\mathit{\theta },{\gamma }_0)dy,\end{array}$$

it is enough to show

$$\begin{array}{c}\hfill \underset{{\gamma }_0\to 0}{lim}{\int }_{\mathrm{I}\mathrm{R}}{u}_J\left(y\right|\mathit{\theta },{\gamma }_0)u_J^{\prime }\left(y\right|\mathit{\theta },{\gamma }_0)h_J\left(y\right|\mathit{\theta },{\gamma }_0)dy={\int }_{\mathrm{I}\mathrm{R}}{v}_J\left(y\right|\mathit{\theta })v_J^{\prime }\left(y\right|\mathit{\theta })h^{*J}\left(y\right|\mathit{\theta })dy,\end{array}$$

which is proved in Lemma 2. Hence (44) holds. Next we prove (45). By the argument used to establish (40), it is enough to show that

$$\begin{array}{c}\hfill \underset{{\gamma }_0\to 0}{lim}{\int }_{\mathrm{I}\mathrm{R}}\left[{\eta }_z\left(y\right)u_J\left(y\right|\mathit{\theta },{\gamma }_0)\right]dy={\int }_{\mathrm{I}\mathrm{R}}\left[{\eta }_z\left(y\right)v_J\left(y\right|\mathit{\theta })\right]dy.\end{array}$$

(46)

However,

$$\begin{array}{c}\hfill {\int }_{\mathrm{I}\mathrm{R}}{\eta }_z\left(y\right)\left[u_J\left(y\right|\mathit{\theta },{\gamma }_0)-v_J\left(y\right|\mathit{\theta })\right]dy\le \underset{y}{sup}\left|u_J\left(y\right|\mathit{\theta },{\gamma }_0)-v_J\left(y\right|\mathit{\theta })\right|,\end{array}$$

and the RHS of the above inequality converges to zero as ${\gamma }_0\to 0$ from assumption (V1). Hence (46) holds. This completes the proof. □

Our next result is concerned with the behavior of the $\alpha -$influence function when ${\gamma }_0\to 0$ first and then $\left|z\right|\to \infty$ or $\alpha \to 0$. The following three additional assumptions will be used in the proof of part (ii) of Theorem 4.

Model assumptions for robustness analysis

(O5) For $\alpha \in [0,1]$ and all $\mathit{\theta }\in \mathsf{\Theta }$, ${\dot{u}}_{\alpha ,z}^{*J}\left(y\right|\mathit{\theta })s_{\alpha ,z}^{*J}\left(y\right|\mathit{\theta })$ is bounded in ${\mathit{L}}_2$.

(O6) For $\alpha \in [0,1]$ and all $\mathit{\theta }\in \mathsf{\Theta }$, $u_{\alpha ,z}^{*J}\left(y\right|\mathit{\theta })u_{\alpha ,z}^{*J^{\prime }}\left(y\right|\mathit{\theta })s_{\alpha ,z}^{*J}\left(y\right|\mathit{\theta })$ is bounded in ${\mathit{L}}_2$.

(O7) For $\alpha \in [0,1]$ and all $\mathit{\theta }\in \mathsf{\Theta }$,

$$\begin{array}{c}\hfill \underset{{\gamma }_0\to 0}{lim}{\int }_{\mathrm{I}\mathrm{R}}{\left(s_{J,\alpha ,z}\left(y\right|\mathit{\theta },{\gamma }_0)u_{J,\alpha ,z}\left(y\right|\mathit{\theta },{\gamma }_0)-s_{\alpha ,z}^{*J}\left(y\right|\mathit{\theta })u_{\alpha ,z}^{*J}\left(y\right|\mathit{\theta })\right)}^{2}dy=0.\end{array}$$

Theorem 4.

(i) Let $\alpha \in (0,1),$ and assume that for all $\mathit{\theta }\in \mathsf{\Theta }$, assume that the assumptions of Proposition 1 hold, also assume that $T\left(h_{J,\alpha ,z}(·|\mathit{\theta },{\gamma }_0)\right)$ is unique for all z. Then, $T\left(h_{J,\alpha ,z}(·|\mathit{\theta },{\gamma }_0)\right)$ is a bounded, continuous function of z such that

$$\begin{array}{c}\hfill \underset{\left|z\right|\to \infty }{lim}\underset{{\gamma }_0\to 0}{lim}T\left(h_{J,\alpha ,z}(·|\mathit{\theta },{\gamma }_0)\right)=\mathit{\theta };\end{array}$$

(ii) Assume further that the conditions (O3)–(O7) hold. Then,

$$\begin{array}{c}\hfill \underset{\alpha \to 0}{lim}\underset{{\gamma }_0\to 0}{lim}{\alpha }^{-1}\left[T\left(h_{J,\alpha ,z}(·|\mathit{\theta },{\gamma }_0)\right)-\mathit{\theta }\right]={\left[I\left(\mathit{\theta }\right)\right]}^{-1}{\int }_{\mathrm{I}\mathrm{R}}\left[{\eta }_z\left(y\right)v_J\left(y\right|\mathit{\theta })\right]dy.\end{array}$$

Proof. 

Let ${\mathit{\theta }}_z\left({\gamma }_0\right)$ denote $T\left(h_{J,\alpha ,z}(·|\mathit{\theta },{\gamma }_0)\right)$ and let ${\mathit{\theta }}_z$ denote $T\left(h_{\alpha ,z}^{*J}(·|\mathit{\theta })\right)$. First fix $z\in \mathrm{I}\mathrm{R}$; then by the triangular inequality,

$$\begin{array}{c}\hfill \underset{{\gamma }_0\to 0}{lim}|{\mathit{\theta }}_z\left({\gamma }_0\right)-\mathit{\theta }|\le \underset{{\gamma }_0\to 0}{lim}|{\mathit{\theta }}_z\left({\gamma }_0\right)-{\mathit{\theta }}_z|+\underset{{\gamma }_0\to 0}{lim}|{\mathit{\theta }}_z-\mathit{\theta }|.\end{array}$$

(47)

The first term of RHS of (47) is equal to zero due to proposition 2. Now let $\left|z\right|\to \infty$, then the second term on the RHS of (47) converges to zero using similar argument as Theorem 3 with density converging to $h_{\alpha ,z}^{*J}(·|\mathit{\theta })$. This completes the proof of I). Turning to (ii), we will prove that

$$\begin{array}{c}\hfill \underset{\alpha \to 0}{lim}\underset{{\gamma }_0\to 0}{lim}D(h_{J,\alpha ,z}(·|\mathit{\theta },{\gamma }_0),h_J(·|\mathit{\theta },{\gamma }_0))=-\frac{1}{4}I\left(\mathit{\theta }\right),\end{array}$$

(48)

$$\begin{array}{c}\hfill \underset{\alpha \to 0}{lim}\underset{{\gamma }_0\to 0}{lim}{\alpha }^{-1}\nabla H{D}^2(h_{J,\alpha ,z}(·|\mathit{\theta },{\gamma }_0),h_J(·|\mathit{\theta },{\gamma }_0))=\frac{1}{4}{\int }_{\mathrm{I}\mathrm{R}}\left[{\eta }_z\left(y\right)v_J\left(y\right|\mathit{\theta })\right]dy.\end{array}$$

(49)

Recall from the proof of part (ii) of Theorem 3 that

$$\begin{array}{ccc}\hfill D(h_{J,\alpha ,z}(·|\mathit{\theta },{\gamma }_0),h_J(·|\mathit{\theta },{\gamma }_0))& =& \frac{1}{2}{\int }_{\mathrm{I}\mathrm{R}}{\dot{u}}_{J,\alpha ,z}\left(y\right|\mathit{\theta },{\gamma }_0)s_{J,\alpha ,z}\left(y\right|\mathit{\theta },{\gamma }_0)s_J\left(y\right|\mathit{\theta },{\gamma }_0)\hfill \\ & & +\frac{1}{4}{\int }_{\mathrm{I}\mathrm{R}}{u}_{J,\alpha ,z}\left(y\right|\mathit{\theta },{\gamma }_0)u_{J,\alpha ,z}^{\prime }\left(y\right|\mathit{\theta },{\gamma }_0)s_{J,\alpha ,z}\left(y\right|\mathit{\theta },{\gamma }_0)s_J\left(y\right|\mathit{\theta },{\gamma }_0)\hfill \\ & \equiv & D_1(h_{J,\alpha ,z}(·|\mathit{\theta },{\gamma }_0),h_J(·|\mathit{\theta },{\gamma }_0))+D_2(h_{J,\alpha ,z}(·|\mathit{\theta },{\gamma }_0),h_J(·|\mathit{\theta },{\gamma }_0)).\hfill \end{array}$$

We will now show that for fixed $\alpha \in (0,1)$

$$\begin{array}{c}\hfill \underset{{\gamma }_0\to 0}{lim}{D}_1(h_{J,\alpha ,z}(\mathit{\theta },{\gamma }_0),h_J(·|\mathit{\theta },{\gamma }_0))=\frac{1}{2}{\int }_{\mathrm{I}\mathrm{R}}{\dot{u}}_{\alpha ,z}^{*J}\left(y\right|\mathit{\theta })s_{\alpha ,z}^{*J}\left(y\right|\mathit{\theta })t_J\left(y\right|\mathit{\theta })dy,\mathrm{and}\end{array}$$

(50)

$$\begin{array}{c}\hfill \underset{{\gamma }_0\to 0}{lim}{D}_2(h_{J,\alpha ,z}(\mathit{\theta },{\gamma }_0),h_J(·|\mathit{\theta },{\gamma }_0))=\frac{1}{4}{\int }_{\mathrm{I}\mathrm{R}}{u}_{\alpha ,z}^{*J}\left(y\right|\mathit{\theta })u_{\alpha ,z}^{*J^{\prime }}\left(y\right|\mathit{\theta })s_{\alpha ,z}^{*J}\left(y\right|\mathit{\theta })t_J\left(y\right|\mathit{\theta })dy.\end{array}$$

(51)

We begin with (50). A standard calculation shows that $D_1(h_{J,\alpha ,z}(\mathit{\theta },{\gamma }_0),u_{\alpha ,z}^{*J}\left(y\right|\mathit{\theta }))$ can be expressed as the sum of ${\mathcal{D}}_{11}$, ${\mathcal{D}}_{12}$ and ${\mathcal{D}}_{13}$, where

$$\begin{array}{c}\hfill {\mathcal{D}}_{11}=\frac{1}{2}{\int }_{\mathrm{I}\mathrm{R}}\left({\dot{u}}_{J,\alpha ,z}\left(y\right|\mathit{\theta },{\gamma }_0)s_{J,\alpha ,z}\left(y\right|\mathit{\theta },{\gamma }_0)-{\dot{u}}_{\alpha ,z}^{*J}\left(y\right|\mathit{\theta })s_{\alpha ,z}^{*J}\left(y\right|\mathit{\theta })\right)s_J\left(y\right|\mathit{\theta },{\gamma }_0)dy,\end{array}$$

$$\begin{array}{c}\hfill {\mathcal{D}}_{12}=\frac{1}{2}{\int }_{\mathrm{I}\mathrm{R}}{\dot{u}}_{\alpha ,z}^{*J}\left(y\right|\mathit{\theta })s_{\alpha ,z}^{*J}\left(y\right|\mathit{\theta })\left(s_J\left(y\right|\mathit{\theta },{\gamma }_0)-t_J\left(y\right|\mathit{\theta })\right)dy,\mathrm{and}\end{array}$$

$$\begin{array}{c}\hfill {\mathcal{D}}_{13}=\frac{1}{2}{\int }_{\mathrm{I}\mathrm{R}}{\dot{u}}_{\alpha ,z}^{*J}\left(y\right|\mathit{\theta })s_{\alpha ,z}^{*J}\left(y\right|\mathit{\theta })t_J\left(y\right|\mathit{\theta })dy.\end{array}$$

It can be seen that ${\mathcal{D}}_{11}$ converges to zero as ${\gamma }_0\to 0$ by Cauchy-Schwarz inequality and assumption (O3); also, ${\mathcal{D}}_{12}$ converges to zero as ${\gamma }_0\to 0$ by Cauchy-Schwarz inequality, assumption (O5) and Scheffe’s theorem. Hence (50) follows. Similarly (51) follows as ${\gamma }_0\to 0$ by Cauchy-Schwarz inequality, assumption (O4), assumption (O6) and Scheffe’s theorem.

Now let $\alpha \to 0$. Using the same idea as in Theorem 3 to find dominating functions, one can apply the dominated convergence Theorem to establish that

$$\begin{array}{c}\hfill \underset{\alpha \to 0}{lim}\frac{1}{2}{\int }_{\mathrm{I}\mathrm{R}}{\dot{u}}_{\alpha ,z}^{*J}\left(y\right|\mathit{\theta })s_{\alpha ,z}^{*J}\left(y\right|\mathit{\theta })t_J\left(y\right|\mathit{\theta })dy=-\frac{1}{2}I\left(\mathit{\theta }\right),\mathrm{and}\end{array}$$

$$\begin{array}{c}\hfill \underset{\alpha \to 0}{lim}\frac{1}{4}{\int }_{\mathrm{I}\mathrm{R}}{u}_{\alpha ,z}^{*J}\left(y\right|\mathit{\theta })u_{\alpha ,z}^{*J^{\prime }}\left(y\right|\mathit{\theta })s_{\alpha ,z}^{*J}\left(y\right|\mathit{\theta })t_J\left(y\right|\mathit{\theta })dy=\frac{1}{4}I\left(\mathit{\theta }\right).\end{array}$$

Hence (48) follows. Finally, it remains to establish (49). First fix $\alpha \in (0,1)$; we will show that

$$\begin{array}{c}\hfill \underset{{\gamma }_0\to 0}{lim}\nabla H{D}^2(h_{J,\alpha ,z}(·|\mathit{\theta },{\gamma }_0),h_J(·|\mathit{\theta },{\gamma }_0))=-\frac{1}{2}{\int }_{\mathrm{I}\mathrm{R}}{s}_{\alpha ,z}^{*J}\left(y\right|\mathit{\theta })u_{\alpha ,z}^{*J}\left(y\right|\mathit{\theta })t_J\left(y\right|\mathit{\theta })dy.\end{array}$$

(52)

Please $\nabla H{D}^2(h_{J,\alpha ,z}(·|\mathit{\theta },{\gamma }_0),h_J(·|\mathit{\theta },{\gamma }_0))$ can be expressed as the sum of ${\mathcal{T}}_1$, ${\mathcal{T}}_2$ and ${\mathcal{T}}_3$, where

$$\begin{array}{c}\hfill {\mathcal{T}}_1=-\frac{1}{2}{\int }_{\mathrm{I}\mathrm{R}}\left(s_{J,\alpha ,z}\left(y\right|\mathit{\theta },{\gamma }_0)u_{J,\alpha ,z}\left(y\right|\mathit{\theta },{\gamma }_0)-s_{\alpha ,z}^{*J}\left(y\right|\mathit{\theta })u_{\alpha ,z}^{*J}\left(y\right|\mathit{\theta })\right)s_J\left(y\right|\mathit{\theta },{\gamma }_0)dy,\end{array}$$

$$\begin{array}{c}\hfill {\mathcal{T}}_2=-\frac{1}{2}{\int }_{\mathrm{I}\mathrm{R}}{s}_{\alpha ,z}^{*J}\left(y\right|\mathit{\theta })u_{\alpha ,z}^{*J}\left(y\right|\mathit{\theta })\left(s_J\left(y\right|\mathit{\theta },{\gamma }_0)-t_J\left(y\right|\mathit{\theta })\right)dy,\mathrm{and}\end{array}$$

$$\begin{array}{c}\hfill {\mathcal{T}}_3=-\frac{1}{2}{\int }_{\mathrm{I}\mathrm{R}}{s}_{\alpha ,z}^{*J}\left(y\right|\mathit{\theta })u_{\alpha ,z}^{*J}\left(y\right|\mathit{\theta })t_J\left(y\right|\mathit{\theta })dy.\end{array}$$

It can be seen thatr ${\mathcal{T}}_1$ converges to zero as ${\gamma }_0\to 0$ by Cauchy-Schwarz inequality and assumption (O7); ${\mathcal{T}}_2$ converges to zero as ${\gamma }_0\to 0$ by Cauchy-Schwarz inequality, boundedness of $u_{\alpha ,z}^{*J}(·)s_{\alpha ,z}^{*J}(·)$ in ${\mathit{L}}_2$, and Scheffe’s theorem. Therefore, (52) holds. Finally, letting $\alpha \to 0$ and using the same idea as in Theorem 3 to find the dominating function, it follows by the dominated convergence theorem and L’Hospital rule that (49) holds. This completes the proof of the Theorem. □

Remark 4.

Theorems 3 and 4 do not imply that the double limit exists. This is beyond the scope of this paper.

In the next section, we describe the implementation details and provide several simulation results in support of our methodology.

4. Implementation and Numerical Results

In this section, we apply the proposed MHD based methods to estimate the unknown parameters $\mathit{\theta }=(\mu ,{\sigma }^2)$ using the compressed data. We set J = 10,000 and $B=100$. All simulations are based on 5000 replications. We consider the Gaussian kernel and Epanechnikov kernel for the nonparametric density estimation. The Gaussian kernel is given by

$$\begin{array}{c}\hfill K\left(x\right)=\frac{1}{\sqrt{2\pi }}exp\left(-\frac{x^2}{2}\right),\end{array}$$

and the Epanechnikov kernel is given by

$$\begin{array}{c}\hfill K\left(x\right)=\frac{3}{4}\left(1-x^2\right)1_{\left(\left|x\right|\le 1\right)}.\end{array}$$

We generate $\mathit{X}$ and uncontaminated compressed data $\tilde{\mathit{Y}}$ in the following way:

• Step 1. Generate ${\mathit{X}}_l$, where $X_{jl}\stackrel{i.i.d.}{\sim }N(\mu ,{\sigma }^2)$.
• Step 2. Generate ${\mathit{R}}_l$, where $r_{ijl}\stackrel{i.i.d.}{\sim }N(1,{\gamma }_0^2)$.
• Step 3. Generate the uncontaminated ${\tilde{\mathit{Y}}}_l$ by calculating ${\tilde{\mathit{Y}}}_l={\mathit{R}}_l{\mathit{X}}_l$.

4.1. Objective Function

In practice, we store the compressed data $({\tilde{\mathit{Y}}}_l,{\mathit{r}}_{·l},{\mathit{\omega }}_l)$ for all $1\le l\le B$. Hence if $X_{jl}$ follows Normal distribution with mean $\mu$ and variance ${\sigma }^2$, the form of the marginal density of the compressed data, viz., $Y_{il}$ is complicated and does not have a closed form expression. However, for large J, using the local limit theorem its density can be approximated by Gaussian density with mean $\sqrt{J}\mu$ and variance ${\sigma }^2+{\gamma }_0^2({\mu }^2+{\sigma }^2)$. Hence, we work with $U_{il}$, where $U_{il}=\frac{{\tilde{Y}}_{il}-\mu r_{i·l}}{{\omega }_{il}}$. Please note that with this transformation, $\mathit{E}\left[U_{il}\right]=0$ and $\mathit{Var}\left[U_{il}\right]={\sigma }^2$. Hence, the kernel density estimate of the unknown true density is given by

$$\begin{array}{c}\hfill g_B^{\left(i\right)}\left(y\right|\mu )=\frac{1}{B{c}_B}\sum _{l=1}^{B}K\left(\frac{y-U_{il}}{c_B}\right).\end{array}$$

The difference between the kernel density estimate and the one proposed here is that we include the unknown parameter $\mu$ in the kernel. Additionally, this allows one to incorporate $({\mathit{r}}_{·r},{\mathit{\omega }}_l)$ into the kernel. Consequently, only the scale parameter $\sigma$ is part of the parametric model. Using the local limit theorem, we approximate the true parametric model by $\varphi (·|\sigma )$, where $\varphi (·|\sigma )$ is the density of $N(0,{\sigma }^2)$. Hence, the objective function is

$$\begin{array}{c}\hfill \mathsf{\Psi }(i,\mathit{\theta })\equiv \mathcal{A}(g_B^{\left(i\right)}\left(·\left|\mu \right),\varphi (·|\sigma )\right)={\int }_{\mathrm{I}\mathrm{R}}{g}_B^{\left(i\right)\frac{1}{2}}\left(y\right|\mu ){\varphi }^{\frac{1}{2}}\left(y\right|\sigma )dy;\end{array}$$

and, the estimator is given by

$$\begin{array}{c}\hfill {\widehat{\mathit{\theta }}}_B\left({\gamma }_0\right)=\frac{1}{S}\sum _{i=1}^S{\widehat{\mathit{\theta }}}_{i,B}\left({\gamma }_0\right),\mathrm{where}{\widehat{\mathit{\theta }}}_{i,B}\left({\gamma }_0\right)=\underset{\mathit{\theta }\in \mathsf{\Theta }}{\mathrm{argmax}}\mathsf{\Psi }(i,\mathit{\theta }).\end{array}$$

It is clear that ${\widehat{\mathit{\theta }}}_B\left({\gamma }_0\right)$ is a consistent estimator of ${\mathit{\theta }}^{*}.$ In the next subsection, we use Quasi-Newton method with Broyden-Fletcher-Goldfarb-Shanno (BFGS) update to estimate $\mathit{\theta }$. Quasi-Newton method is appealing since (i) it replaces the complicated calculation of the Hessian matrix with an approximation which is easier to compute (${\Delta }_k\left(\mathit{\theta }\right)$ given in the next subsection) and (ii) gives more flexible step size t (compared to the Newton-Raphson method), ensuring that it does not “jump” too far at every step and hence guaranteeing convergence of the estimating equation. The BFGS update ($H_k$) is a popular method for approximating the Hessian matrix via gradient evaluations. The step size t is determined using Backtracking line search algorithm described in Algorithm 2. The algorithms are given in detail in the next subsection. Our analysis also includes the case where $S\equiv 1$ and $r_{ijl}\equiv 1$. In this case, as explained previously, one obtains significant reduction in storage and computational complexity. Finally, we emphasize here that the density estimate contains $\mu$ and is not parameter free as is typical in classical MHDE analysis. In the next subsection, we describe an algorithm to implement our method.

4.2. Algorithm

As explained previously, we use the Quasi-Newton Algorithm with BFGS update to obtain ${\widehat{\mathit{\theta }}}_{\mathrm{MHDE}}$. To describe this method, consider the objective function (suppressing i) $\mathsf{\Psi }\left(\mathit{\theta }\right)$, which is twice continuously differentiable. Let the initial value of $\mathit{\theta }$ be ${\mathit{\theta }}^{\left(0\right)}=\left({\mu }^{\left(0\right)},{\sigma }^{\left(0\right)}\right)$ and $H_0=I$, where I is the identity matrix.

Algorithm 1: The Quasi-Newton Algorithm.

Set k = 1. repeat Calculate ${\Delta }_k\left(\mathit{\theta }\right)=-H_{k-1}^{-1}\nabla \mathsf{\Psi }({\mathit{\theta }}^{(k-1)})$, where $\nabla \mathsf{\Psi }(y;{\mathit{\theta }}^{k-1})$ is the first derivative of $\mathsf{\Psi }\left(\mathit{\theta }\right)$ with respect to $\mathit{\theta }$ at $(k-1)$th step. Determine the step length parameter t via backtracking line search. Compute ${\mathit{\theta }}^{\left(k\right)}={\mathit{\theta }}^{(k-1)}+t{\Delta }_k\left(\mathit{\theta }\right)$. Compute $H_k$, where the BFGS update is $$\begin{array}{c}\hfill H_k=H_{k-1}+\frac{q_{k-1}{q}_{k-1}^T}{q_{k-1}^T{d}_{k-1}}-\frac{H_{k-1}{d}_{k-1}{d}_{k-1}^T{H}_{k-1}^T}{d_{k-1}^T{H}_{k-1}{d}_{k-1}},\end{array}$$ where $$\begin{array}{c}{d}_{k-1}={\mathit{\theta }}^{\left(k\right)}-{\mathit{\theta }}^{(k-1)},\hfill \\ q_{k-1}=\nabla \mathsf{\Psi }({\mathit{\theta }}^{\left(k\right)})-\nabla \mathsf{\Psi }({\mathit{\theta }}^{(k-1)}).\end{array}$$ Compute $e_k=|\mathsf{\Psi }({\mathit{\theta }}^{\left(k\right)})-\mathsf{\Psi }({\mathit{\theta }}^{(k-1)})|$. Set $k=k+1$. until$\left(e_k\right)<\mathrm{threshold}$.

Remark 5.

In step 1, one can directly use the Inverse update for $H_k^{-1}$ as follows:

$$\begin{array}{c}\hfill H_k^{-1}=\left(I-\frac{d_{k-1}{q}_{k-1}^T}{q_{k-1}^T{d}_{k-1}}\right)H_{k-1}^{-1}\left(I-\frac{q_{k-1}{d}_{k-1}^T}{q_{k-1}^T{d}_{k-1}}\right)+\frac{d_{k-1}{d}_{k-1}^T}{q_{k-1}^T{d}_{k-1}}.\end{array}$$

Remark 6.

In step 2, the step size t should satisfy the Wolfe conditions:

$$\begin{array}{c}\hfill \begin{array}{c}\mathsf{\Psi }\left(y;{\mathit{\theta }}^{\left(k\right)}+t{\Delta }_k\right)\le \mathsf{\Psi }\left({\mathit{\theta }}^{\left(k\right)}\right)+u_{1}t\nabla {\mathsf{\Psi }}^T\left({\mathit{\theta }}^{\left(k\right)}\right){\Delta }_k,\hfill \\ \nabla \mathsf{\Psi }\left({\mathit{\theta }}^{\left(k\right)}+t{\Delta }_k\right)\ge u_2\nabla {\mathsf{\Psi }}^T\left({\mathit{\theta }}^{\left(k\right)}\right){\Delta }_k,\hfill \end{array}\end{array}$$

where $u_1$ and $u_2$ are constants with $0<u_1<u_2<1.$ The first condition requires that t sufficiently decrease the objective function. The second condition ensures that the step size is not too small. The Backtracking line search algorithm proceeds as follows (see [26]):

Algorithm 2: The Backtracking Line Search Algorithm.

Given a descent direction $\Delta \left(\mathit{\theta }\right)$ for $\mathsf{\Psi }$ at $\mathit{\theta }$, $\zeta \in (0,0.5),\kappa \in (0,1)$. $t:=1.$     while $\mathsf{\Psi }(\mathit{\theta }+t\Delta \mathit{\theta })>\mathsf{\Psi }\left(\mathit{\theta }\right)+\zeta t\nabla \mathsf{\Psi }{\left(\mathit{\theta }\right)}^T\Delta \mathit{\theta }$,     do         $t:=\kappa t.$     end while

4.3. Initial Values

The initial value for $\mathit{\theta }$ are taken to be

$$\begin{array}{c}\hfill \begin{array}{c}{\mu }^{\left(0\right)}=\mathrm{median}\left({\tilde{Y}}_{il}\right)/J,\hfill \\ {\sigma }^{\left(0\right)}=1.48\times \mathrm{median}\left(|{\tilde{Y}}_{il}-\mathrm{median}({\tilde{Y}}_{il})|\right)/B.\hfill \end{array}\end{array}$$

Another choice of the initial value for $\sigma$ is:

$$\begin{array}{c}\hfill {\widehat{\sigma }}^{\left(0\right)}=\sqrt{\frac{(\frac{\widehat{\mathit{Var}\left[{\tilde{Y}}_{il}\right]}}{J}-{\gamma }_0^2\mu )}{{\gamma }_0^2+{\mu }_0^2}},\end{array}$$

(53)

where $\widehat{\mathit{Var}[{\tilde{Y}}_{il}]}$ is an empirical estimate of the variance of ${\tilde{Y}}_1$.

Bandwidth Selection: A key issue in implementing the above method of estimation is the choice of the bandwidth. We express the bandwidth in the form $h_B=c_B{s}_B$, where $c_B\in \{0.3,0.4,0.5,0.7,0.9\}$, and $s_B$ is set equal to $1.48\times$ median $\left(|{\tilde{Y}}_{il}-\mathrm{median}({\tilde{Y}}_{il})|\right)/B$.

In all the tables below, we report the average (Ave), standard deviation (StD) and mean square error (MSE) to assess the performance of the proposed methods.

4.4. Analyses Without Contamination

From

Table 2. MHDE as the dimension S changes for compressed data $\tilde{\mathit{Y}}$ using Gaussian kernel.

	$\widehat{\mathit{\mu }}$			$\widehat{\mathit{\sigma }}$
	Ave	StD $\times {\mathbf{10}}^{\mathbf{3}}$	MSE $\times {\mathbf{10}}^{\mathbf{3}}$	Ave	StD $\times {\mathbf{10}}^{\mathbf{3}}$	MSE $\times {\mathbf{10}}^{\mathbf{3}}$
$S^{*}=1$	2.000	1.010	0.001	1.016	74.03	5.722
$S=1$	2.000	1.014	0.001	1.018	74.22	5.844
$S=2$	2.000	1.005	0.001	1.019	73.81	5.832
$S=5$	2.000	0.987	0.001	1.017	74.16	5.798
$S=10$	2.000	0.995	0.001	1.019	71.87	5.525

,

Table 3. MHDE as ${\gamma }_0$ changes for compressed data $\tilde{\mathit{Y}}$ using Gaussian kernel.

	$\widehat{\mathit{\mu }}$			$\widehat{\mathit{\sigma }}$
	Ave	StD $\times {\mathbf{10}}^{\mathbf{3}}$	MSE $\times {\mathbf{10}}^{\mathbf{3}}$	Ave	StD $\times {\mathbf{10}}^{\mathbf{3}}$	MSE $\times {\mathbf{10}}^{\mathbf{3}}$
${\gamma }_0=0.00$	2.000	1.010	0.001	1.016	74.03	5.722
${\gamma }_0=0.01$	2.000	1.017	0.001	1.015	74.83	5.814
${\gamma }_0=0.10$	2.000	1.023	0.001	1.021	72.80	5.717
${\gamma }_0=0.50$	2.000	1.119	0.001	1.076	72.59	11.08
${\gamma }_0=1.00$	2.000	1.399	0.002	1.226	82.21	57.75

,

Table 4. MHDE as the bandwidth $c_B$ changes for compressed data $\tilde{\mathit{Y}}$ using Gaussian kernel.

	$\widehat{\mathit{\mu }}$			$\widehat{\mathit{\sigma }}$
	Ave	StD $\times {\mathbf{10}}^{\mathbf{3}}$	MSE $\times {\mathbf{10}}^{\mathbf{3}}$	Ave	StD $\times {\mathbf{10}}^{\mathbf{3}}$	MSE $\times {\mathbf{10}}^{\mathbf{3}}$
$c_B^{*}=0.30$	2.000	1.010	0.001	1.016	74.03	5.722
$c_B=0.30$	2.000	1.014	0.001	1.018	74.22	5.844
$c_B=0.40$	2.000	1.015	0.001	1.063	79.68	10.26
$c_B=0.50$	2.000	1.014	0.001	1.108	82.33	18.33
$c_B=0.70$	2.000	1.004	0.001	1.212	93.96	53.64
$c_B=0.90$	2.000	1.009	0.001	1.346	110.5	132.2

and

Table 5. MHDE as B changes for compressed data $\tilde{\mathit{Y}}$ using Gaussian kernel with ${\gamma }_0=0.1$.

	$\widehat{\mathit{\mu }}$			$\widehat{\mathit{\sigma }}$
	Ave	StD $\times {\mathbf{10}}^{\mathbf{3}}$	MSE $\times {\mathbf{10}}^{\mathbf{3}}$	Ave	StD $\times {\mathbf{10}}^{\mathbf{3}}$	MSE $\times {\mathbf{10}}^{\mathbf{3}}$
$B=20$	2.000	2.205	0.005	1.739	378.5	688.6
$B=50$	2.000	1.409	0.002	1.136	125.2	34.17
$B=100$	2.000	1.010	0.001	1.016	74.03	5.722
$B=500$	2.000	0.455	0.000	0.972	32.63	1.873

, we let true $\mu =2,\sigma =1$, and take the kernel to be Gaussian kernel. In Table 2, we compare the estimates of the parameters as the dimension of the compressed data S increases. In this table, we allow S to take values in the set $\{1,2,5,10\}$. Also, we let the number of groups $B=100$, the bandwidth is chosen to be $c_B=0.3$, and ${\gamma }_0=0.1$. In addition, in Table 2, $S^{*}=1$ means that $S=1$ with ${\gamma }_0\equiv 0.$

From Table 2 we observe that as S increases, the estimates for $\mu$ and $\sigma$ remain stable. The case $S^{*}=1$ is interesting, since even by storing the sum we are able to obtain point estimates which are close to the true value. In Table 3, we choose $S=1,B=100$ and $c_B=0.3$ and compare the estimates as ${\gamma }_0$ changes from 0.01 to 1.00. We can see that as ${\gamma }_0$ increases, the estimate for $\mu$ remains stable, whereas the bias, standard deviation and MSE for $\sigma$ increase.

In Table 4, we fix $S=1,B=100$ and ${\gamma }_0=0.1$ and allow the bandwidth $c_B$ to increase. Also, $c_B^{*}=0.30$ means that the bandwidth is chosen as $0.30$ with ${\gamma }_0\equiv 0.$ Notice that in this case when $c_B=0.9$ $B^{\frac{1}{2}}{c}_B=9$ while $B^{\frac{1}{2}}{c}_B^2=8.1$ which is not small as is required in assumption (B2). We notice again that as $c_B$ decreases, the estimates of $\mu$ and $\sigma$ are close to the true value with small MSE and StD.

In Table 5, we let $S=1,c_B=0.3$ and ${\gamma }_0=0.1$ and let the number of groups B increase. This table implies that as B increases, the estimate performs better in terms of bias, standard deviation and MSE.

In

Table 6. MHDE as B changes for compressed data $\tilde{\mathit{Y}}$ using Gaussian kernel with ${\gamma }_0=0$.

	$\widehat{\mathit{\mu }}$			$\widehat{\mathit{\sigma }}$
	Ave	StD $\times {\mathbf{10}}^{\mathbf{3}}$	MSE $\times {\mathbf{10}}^{\mathbf{3}}$	Ave	StD $\times {\mathbf{10}}^{\mathbf{3}}$	MSE $\times {\mathbf{10}}^{\mathbf{3}}$
$B=20$	2.000	2.282	0.005	1.749	381.4	706.0
$B=50$	2.000	1.440	0.002	1.148	125.2	37.42
$B=100$	2.000	1.014	0.001	1.018	74.22	5.844
$B=500$	2.000	0.465	0.000	0.973	31.33	1.692

, we set ${\gamma }_0\equiv 0$ and keep other settings same as Table 5. This table implies that as B increases, the estimate performs better in terms of bias, standard deviation and MSE. Furthermore, the standard deviation and MSE are slightly smaller than the results in Table 5.

We next move on to investigating the effect of other sensing variables. In the following table, we use Gamma model to generate the additive matrix ${\mathit{R}}_l$. Specifically, the mean of Gamma random variable is set as ${\alpha }_0{\beta }_0=1$, and the variance $var\equiv {\alpha }_0{\beta }_0^2$ is chosen from the set $\{0,{0.01}^2,0.01,0.25,1.00\}$ which are also the variances in Table 3.

From

Table 7. MHDE as variance changes for compressed data $\tilde{\mathit{Y}}$ using Gaussian kernel under Gamma sensing variable.

	$\widehat{\mathit{\mu }}$			$\widehat{\mathit{\sigma }}$
	Ave	StD $\times {\mathbf{10}}^{\mathbf{3}}$	MSE $\times {\mathbf{10}}^{\mathbf{3}}$	Ave	StD $\times {\mathbf{10}}^{\mathbf{3}}$	MSE $\times {\mathbf{10}}^{\mathbf{3}}$
$var=0.00$	2.000	1.010	0.001	1.016	74.03	5.722
$var={0.01}^2$	2.000	1.005	0.001	1.016	74.56	5.806
$var=0.01$	2.000	1.006	0.001	1.018	73.70	5.762
$var=0.25$	2.000	1.120	0.001	1.078	73.70	11.56
$var=1.00$	2.000	1.438	0.001	1.228	81.94	58.48

, notice that using Gamma sensing variable yields similar results as Gaussian sensing variable. Our next example considers the case when the mean of the sensing variable is not equal to one and the sensing variable is taken to have a discrete distribution.Specifically, we use Bernoulli sensing variables with parameter p. Moreover, we fix $S=1$ and let $pJ=S$. Therefore $p=1/J$. Hence as J increases, the variance decreases. Now notice that in this case the mean of sensing variable is p instead of 1. In addition, $\mathit{E}[{\tilde{Y}}_{il}]=\mu$ and $\mathit{Var}[{\tilde{Y}}_{il}]={\sigma }^2+{\mu }^2(1-\frac{1}{J})$. Hence we set the initial value as

$$\begin{array}{c}\hfill \begin{array}{c}{\mu }^{\left(0\right)}=\mathrm{median}\left({\tilde{Y}}_{il}\right),\hfill \\ {\sigma }^{\left(0\right)}=1.48\times \mathrm{median}\left(|{\tilde{Y}}_{il}-\mathrm{median}({\tilde{Y}}_{il})|\right).\hfill \end{array}\end{array}$$

Additionally, we take $B=100$, $c_B=0.30$ and $s_B$ to be $1.48\times \mathrm{median}\left(|{\tilde{Y}}_{il}-\mathrm{median}({\tilde{Y}}_{il})|\right).$

Table 8. MHDE as J changes for compressed data $\tilde{\mathit{Y}}$ using Gaussian kernel under Bernoulli sensing variable.

	$\widehat{\mathit{\mu }}$			$\widehat{\mathit{\sigma }}$
	Ave	StD $\times {\mathbf{10}}^{\mathbf{3}}$	MSE $\times {\mathbf{10}}^{\mathbf{3}}$	Ave	StD $\times {\mathbf{10}}^{\mathbf{3}}$	MSE $\times {\mathbf{10}}^{\mathbf{3}}$
$J=10$	2.000	104.9	11.01	1.215	97.78	55.79
$J=100$	1.998	104.5	10.93	1.201	104.5	51.26
$J=1000$	1.998	104.7	10.96	1.195	106.6	49.36
$J=5000$	2.001	103.9	10.80	1.200	105.7	51.20
$J=10000$	1.996	105.1	11.07	1.196	104.4	49.16

shows that MHD method also performs well with Bernoulli sensing variable, although the bias of $\sigma$, standard deviation and mean squre error for both estimates are larger than those using Gaussian sensing variable and Gamma sensing variable.

4.5. Robustness and Model Misspecification

In this section, we provide a numerical assessment of the robustness of the proposed methodology. To this end, let

$$\begin{array}{c}\hfill f_{\alpha ,\eta }\left(x\right|\mathit{\theta })=(1-\alpha )f\left(x\right|\mathit{\theta })+\alpha \eta \left(x\right),\end{array}$$

where $\eta \left(x\right)$ is a contaminating component, $\alpha \in [0,1).$ We generate the contaminated reduced data $\mathit{Y}$ in the following way:

• Step 1. Generate ${\mathit{X}}_l$, where $X_{jl}\stackrel{i.i.d.}{\sim }N(2,1)$.
• Step 2. Generate ${\mathit{R}}_l$, where $r_{ijl}\stackrel{i.i.d.}{\sim }N(1,{\gamma }_0^2)$.
• Step 3. Generate uncontaminated ${\tilde{\mathit{Y}}}_l$ by calculating ${\tilde{\mathit{Y}}}_l={\mathit{R}}_l{\mathit{X}}_l$.
• Step 4. Generate contaminated ${\tilde{Y}}_{il}^c$, where ${\tilde{Y}}_{il}^c={\tilde{Y}}_{il}+\eta \left(x\right)$ with probability $\alpha$, and ${\tilde{Y}}_{il}^c={\tilde{Y}}_{il}$ with probability $1-\alpha$.

In the above description, the contamination with outliers is within blocks. A conceptual issue that one encounters is the meaning of outliers in this setting. Specifically, a data point which is an outlier in the original data set may not remain an outlier in the reduced data and vice-versa. Hence the concepts such as breakdown point and influence function need to be carefully studied. The tables below present one version of the robustness exhibited by the proposed method. In

Table 9. MHDE as $\alpha$ changes for contaminated data $\tilde{\mathit{Y}}$ using Gaussian kernel.

	$\widehat{\mathit{\mu }}$			$\widehat{\mathit{\sigma }}$
	Ave	StD $\times {\mathbf{10}}^{\mathbf{3}}$	MSE $\times {\mathbf{10}}^{\mathbf{3}}$	Ave	StD $\times {\mathbf{10}}^{\mathbf{3}}$	MSE $\times {\mathbf{10}}^{\mathbf{3}}$
${\alpha }^{*}=0.00$	2.000	1.010	0.001	1.016	74.03	5.722
$\alpha =0.00$	2.000	1.014	0.001	1.018	74.22	5.844
$\alpha =0.01$	2.000	1.002	0.001	1.022	74.89	6.079
$\alpha =0.05$	2.000	1.053	0.001	1.023	77.86	6.599
$\alpha =0.10$	2.000	1.086	0.001	1.034	79.30	7.350
$\alpha =0.20$	2.000	1.146	0.001	1.073	93.45	14.06
$\alpha =0.30$	2.001	7.205	0.054	1.264	688.2	542.5
$\alpha =0.40$	2.026	21.60	1.100	3.454	1861	9480
$\alpha =0.50$	2.051	14.00	2.600	4.809	1005	15513

and

Table 10. MHDE as $\alpha$ changes for contaminated data $\tilde{\mathit{Y}}$ using Epanechnikov kernel.

	$\widehat{\mathit{\mu }}$			$\widehat{\mathit{\sigma }}$
	Ave	StD $\times {\mathbf{10}}^{\mathbf{3}}$	MSE $\times {\mathbf{10}}^{\mathbf{3}}$	Ave	StD $\times {\mathbf{10}}^{\mathbf{3}}$	MSE $\times {\mathbf{10}}^{\mathbf{3}}$
${\alpha }^{*}=0.00$	2.000	0.972	0.001	1.008	73.22	5.425
$\alpha =0.00$	2.000	1.014	0.001	1.018	74.22	5.844
$\alpha =0.01$	2.000	0.978	0.001	1.028	107.4	12.19
$\alpha =0.05$	2.000	1.264	0.002	1.025	108.7	12.35
$\alpha =0.10$	2.000	1.202	0.001	1.008	114.7	13.09
$\alpha =0.20$	2.000	1.263	0.002	1.046	129.8	18.76
$\alpha =0.30$	2.001	5.098	0.026	1.104	557.8	318.9
$\alpha =0.40$	2.021	21.80	0.900	3.004	1973	7870
$\alpha =0.50$	2.051	10.21	3.000	4.893	720.4	15669

, we set $J={10}^4,B=100,S=1,{\gamma }_0=0.1,c_B=0.3,\eta =1000$. In addition, ${\alpha }^{*}=0$ means that $\alpha =0$ with ${\gamma }_0\equiv 0.$

From the above Table we observe that, even under $50\%$ contamination the estimate of the mean remains stable; however, the estimate of the variance is affected at high-levels of contamination (beyond $30\%$). An interesting and important issue is to investigate the role of ${\gamma }_0$ on the breakdown point of the estimator.

Finally, we investigate the bias in MHDE as a function of the values of the outlier. The graphs below (Figure 2) describe the changes to MHDE when outlier values ($\eta$) increase. Here we set $S=1,B=100,{\gamma }_0=0.1$. In addition, we let $\alpha =0.2,$ and $\eta$ to take values from $\{100,200,300,400,500,600,700,800,900,1000\}$. We can see that as $\eta$ increases, both $\widehat{\mu }$ and $\widehat{\sigma }$ increase up to $\eta =500$ then decrease, although $\widehat{\mu }$ does not change too much. This phenomenon is because when the outlier value is small (or closer to the observations), then it may not be considered as an “outlier” by the MHD method. However, as the outlier values move “far enough” from other values, then the estimate for $\mu$ and $\sigma$ remain the stable.

5. Example

In this section we describe an analysis of data from financial analytics, using the proposed methods. The data are from a bank (a cash and credit card issuer) in Taiwan and the targets of analyses were credit card holders of the bank. The research focused on the case of customers’ default payments. The data set (see [27] for details) contains 180,000 observations and includes information on twenty five variables such as default payments, demographic factors, credit data, history of payment, and billing statements of credit card clients from April 2005 to September 2005. Ref. [28] study machine learning methods for evaluating the probability of default. Here, we work with the first three months of data containing 90,000 observations concerning bill payments. For our analyses we remove zero payments and negative payment from the data set and perform a logarithmic transformation of the bill payments . Since the log-transformed data was multi-modal and exhibited features of a mixture of normal distributions, we work with the log-transformed data with values in the range (6.1, 13). Next, we performed the Box-Cox transformation to the log-transformed data. This transformation identifies the best transformation that yields approximately normal distribution (which belongs to the location-scale family). Specifically, let $\mathit{L}$ denote the log-transformed data in range (6.1, 13), then the data after Box-Cox transformation is given by $\mathit{X}=\left({\mathit{L}}^2-\mathit{1}\right)/19.9091$. The histogram for $\mathit{X}$ is given in Figure 3. The number of observations at the end of data processing was 70,000.

Our goal is to estimate the average bill payment during the first three months. For this, we will apply the proposed method. In this analysis, we assume that the target model for $\mathit{X}$ is Gaussian and split the data, randomly, into $B=100$ blocks yielding $J=700$ observations per block.

In

Table 11. MHDE from the real data analysis.

		$\widehat{\mathit{\mu }}$	$\widehat{\mathit{\sigma }}$
$S^{*}=1$	est	5.171	1.362
	$95\%$ CI	(4.904, 5.438)	(1.158, 1.540)
$S=1$	est	5.171	1.391
	$95\%$ CI	(4.898, 5.443)	(1.183, 1.572)
$S=5$	est	5.172	1.359
	$95\%$ CI	(4.905, 5.438)	(1.155, 1.535)
$S=10$	est	5.171	1.372
	$95\%$ CI	(4.902, 5.440)	(1.167, 1.551)
$S=20$	est	5.171	1.388
	$95\%$ CI	(4.899, 5.443)	(1.180, 1.569)

, “est” represents the estimator, “$95\%$ CI” stands for $95\%$ confidence interval for the estimator. When analyzing the whole data and choosing bandwidth as $c_n=0.30$, we get the MHDE of $\mu$ to be $\widehat{\mu }=5.183$ with $95\%$ confidence interval $(5.171,5.194)$, and the MHDE of $\sigma$ as $\widehat{\sigma }=1.425$ with confidence interval $(1.418,1.433)$.

In Table 11, we choose the bandwidth as $c_B=0.30.$ Also, $S^{*}=1$ represents the case where $S=1$ and ${\gamma }_0\equiv 0$. In all other settings, we keep ${\gamma }_0=0.1$. We observe that all estimates are similar as S changes.

Next we study the robustness of MHDE for this data by investigating the relative bias and studying the influence function. Specifically, we first reduce the dimension from $J=700$ to $S=1$ for each of the $B=100$ blocks and obtain the compressed data $\tilde{\mathit{Y}}$; next, we generate the contaminated reduced data ${\tilde{Y}}_{il}^c$ from step 4 in Section 4.5. Also, we set $\alpha =0.20,{\gamma }_0=0.20$; the kernel is taken to be to be Epanechnikov density with bandwidth $c_B=0.30$. $\eta \left(x\right)$ is assumed to takes values in $\{50,100,200,300,500,800,1000\}$ (note that the approximate mean of $\tilde{\mathit{Y}}$ is around 3600). Let $T_{\mathrm{MHD}}$ be the Hellinger distance functional. The influence function given by

$$\begin{array}{c}\hfill \begin{array}{c}\hfill \mathrm{IF}(\alpha ;T,\tilde{\mathit{Y}})=\frac{T_{\mathrm{MHD}}({\tilde{\mathit{Y}}}^c)-T_{\mathrm{MHD}}(\tilde{\mathit{Y}})}{\alpha },\end{array}\end{array}$$

which we use to assess the robustness. The graphs shown below (Figure 4) illustrate how the influence function changes as the outlier values increase. We observe that for both estimates ($\widehat{\mu }$ and $\widehat{\sigma }$), the influence function first increase and then decrease fast. From $\eta \left(x\right)=300$, the influence functions remain stable and are close to zero, which clearly indicate that MHDE is stable.

Additional Analyses: The histogram in Figure 3 suggests that, may be a mixture of normal distributions may fit the log and Box-Cox transformed data better than the normal distribution. For this reason, we calculated the Hellinger distance between four component mixture (chosen using BIC criteria) and the normal distribution and this was determined to be 0.0237, approximately. Thus, the normal distribution (which belongs to the location-scale family) can be viewed as a misspecified target distribution; admittedly, one does lose information about the components of the mixture distribution due to model misspecification. However, since our goal was to estimate the overall mean and variance the proposed estimate seems to possess the properties described in the manuscript.

6. Discussion and Extensions

The results in the manuscript focus on the iterated limit theory for MHDE of the compressed data obtained from a location-scale family. Two pertinent questions arise: (i) is it easy to extend this theory to MHDE of compressed data arising from non location-scale family of distributions? and (ii) is it possible to extend the theory from iterated limits to a double limit? Turning to (i), we note that the heuristic for considering the location-scale family comes from the fact that the first and the second moment are consistently estimable for partially observed random walks (see [29,30]). This is related to the size of J and can be of exponential order. For such large J, other moments may not be consistently estimable. Hence, the entire theory goes through as long as one is considering parametric models $f(·|\mathit{\theta })$, where $\mathit{\theta }=\mathcal{W}(\mu ,{\sigma }^2)$, for a known function $\mathcal{W}(·,·)$. The case in point is the Gamma distribution which can be re-parametrized in terms of the first two moments.

As for (ii), it is well-known that existence and equality of iterated limits for real sequences does not imply the existence of the double limit unless additional uniformity of convergence holds (see [31] for instance). Extension of this notion for distributional convergence requires additional assumptions and are investigated in a different manuscript wherein more general divergences are also considered.

7. Concluding Remarks

In this paper we proposed the Hellinger distance-based method to obtain robust estimates for mean and variance in a location-scale model using compressed data. Our extensive theoretical investigations and simulations show the usefulness of the methodology and hence can be applied in a variety of scientific settings. Several theoretical and practical questions concerning robustness in a big data setting arise. For instance, the effect of the variability in the $\mathit{R}$ matrix and its effect on outliers are important issues that need further investigation. Furthermore, statistical properties such as uniform consistency and uniform asymptotic normality under different choices for the distribution of $\mathit{R}$ would be useful. These are under investigation by the authors.