Composite Tests under Corrupted Data

Abstract

This paper focuses on test procedures under corrupted data. We assume that the observations $Z_i$ are mismeasured, due to the presence of measurement errors. Thus, instead of $Z_i$ for $i=1,\dots ,n$, we observe $X_i=Z_i+\sqrt{\delta }{V}_i$, with an unknown parameter $\delta$ and an unobservable random variable $V_i$. It is assumed that the random variables $Z_i$ are i.i.d., as are the $X_i$ and the $V_i$. The test procedure aims at deciding between two simple hyptheses pertaining to the density of the variable $Z_i$, namely $f_0$ and $g_0$. In this setting, the density of the $V_i$ is supposed to be known. The procedure which we propose aggregates likelihood ratios for a collection of values of $\delta$. A new definition of least-favorable hypotheses for the aggregate family of tests is presented, and a relation with the Kullback-Leibler divergence between the sets ${\left(f_{\delta }\right)}_{\delta }$ and ${\left(g_{\delta }\right)}_{\delta }$ is presented. Finite-sample lower bounds for the power of these tests are presented, both through analytical inequalities and through simulation under the least-favorable hypotheses. Since no optimality holds for the aggregation of likelihood ratio tests, a similar procedure is proposed, replacing the individual likelihood ratio by some divergence based test statistics. It is shown and discussed that the resulting aggregated test may perform better than the aggregate likelihood ratio procedure.

1. Introduction

A situation which is commonly met in quality control is the following: Some characteristic Z of an item is supposed to be random, and a decision about its distribution has to be made based on a sample of such items, each with the same distribution $F_0$ (with density $f_0$) or $G_0$ (with density $g_0$). The measurement device adds a random noise $V_{\delta }$ to each measurement, mutually independent and independent of the item, with a common distribution function $H_{\delta }$ and density $h_{\delta }$, where $\delta$ is an unknown scaling parameter. Therefore the density of the measurement $X:=Z+V_{\delta }$ is either $f_{\delta }:=f_0\ast h_{\delta }$ or $g_{\delta }:=g_0\ast h_{\delta }$, where ∗ denotes the convolution operation. We denote $F_{\delta }$ (respectively $G_{\delta }$) to be the distribution function with density $f_{\delta }$ (respectively $g_{\delta }$).

The problem of interest, studied in [1], is how the measurement errors can affect the conclusion of the likelihood ratio test with statistics

$$L_n:=\frac{1}{n}\sum log\frac{g_0}{f_0}\left(X_i\right).$$

For small $\delta$, the result of [2] enables us to estimate the true log-likelihood ratio (true Kullback-Leibler divergence) even when we only dispose of locally perturbed data by additive measurement errors. The distribution function $H_0$ of the measurement errors is considered unknown, up to zero expectation and unit variance. When we use the likelihood ratio test, while ignoring the possible measurement errors, we can incur a loss in both errors of the first and second kind. However, it is shown, in [1], that for small $\delta$ the original likelihood ratio test (LRT) is still the most powerful, only on a slightly changed significance level. The test problem leads to a composite of null and alternative classes ${\mathbf{H}}_0$ or ${\mathbf{H}}_1$ of distributions of random variables $Z+V_{\delta }$ with $V_{\delta }:=\sqrt{\delta }V$, where V has distribution $H_1.$ If those families are bounded by alternating Choquet capacities of order 2, then the minimax test is based on the likelihood ratio of the pair of the least-favorable distributions of ${\mathbf{H}}_0$ and ${\mathbf{H}}_1$, respectively (see Huber and Strassen [3]). Moreover, Eguchi and Copas [4] showed that the overall loss of power caused by a misspecified alternative equals the Kullback-Leibler divergence between the original and the corrupted alternatives. Surprisingly, the value of the overall loss is independent of the choice of null hypothesis. The arguments of [2] and of [5] enable us to approximate the loss of power locally, for a broad set of alternatives. The asymptotic behavior of the loss of power of the test based on sampled data is considered in [1], and is supplemented with numerical illustration.

Statement of the Test Problem

Our aim is to propose a class of statistics for testing the composite hypotheses ${\mathbf{H}}_0$ and ${\mathbf{H}}_1$, extending the optimal Neyman-Pearson LRT between $f_0$ and $g_0$. Unlike in [1], the scaling parameter $\delta$ is not supposed to be small, but merely to belong to some interval bounded away from 0.

We assume that the distribution H of the random variable (r.v.) V is known; indeed, in the tuning of the offset of a measurement device, it is customary to perform a large number of observations on the noise under a controlled environment.

Therefore, this first step produces a good basis for the modelling of the distribution of the density h. Although the distribution of V is known, under operational conditions the distribution of the noise is modified: For a given $\delta$ in $\left[{\delta }_{min},{\delta }_{max}\right]$ with ${\delta }_{min}>0$, denote by $V_{\delta }$ a r.v. whose distribution is obtained through some transformation from the distribution of V, which quantifies the level of the random noise. A classical example is when $V_{\delta }=\sqrt{\delta }V$, but at times we have a weaker assumption, which amounts to some decomposability property with respect to $\delta$: For instance, in the Gaussian case, we assume that for all $\delta ,\eta$, there exists some r.v. $W_{\delta ,\eta }$ such that $V_{\delta +\eta }{=}_d{V}_{\delta }+W_{\delta ,\eta }$, where $V_{\delta }$ and $W_{\delta ,\eta }$ are independent.

The test problem can be stated as follows: A batch of i.i.d. measurements $X_i:=Z_i+V_{\delta ,i}$ is performed, where $\delta >0$ is unknown, and we consider the family of tests of ${\mathbf{H}}_0$($\delta$)$:=$ [X has density $f_{\delta }$] vs. ${\mathbf{H}}_1$($\delta$)$:=$ [X has density $g_{\delta }$], with $\delta \in \Delta =\left[{\delta }_{min},{\delta }_{max}\right].$ Only the $X_i$ are observed. A class of combined tests of ${\mathbf{H}}_0$ vs. ${\mathbf{H}}_1$ is proposed, in the spirit of [6,7,8,9].

Under every fixed n, we assume that $\delta$ is allowed to run over a finite set $p_n$ of components of the vector ${\Delta }_n:=[{\delta }_{min}={\delta }_{0,n},\dots ,{\delta }_{p_n,n}={\delta }_{max}]$. The present construction is essentially non-asymptotic, neither on n nor on $\delta$, in contrast with [1], where $\delta$ was supposed to lie in a small neighborhood of 0. However, with increasing n, it would be useful to consider that the array ${\left({\delta }_{j,n}\right)}_{j=1}^{p_n}$ is getting dense in $\Delta =\left[{\delta }_{min},{\delta }_{max}\right]$ and that

$$\underset{n\to \infty }{lim}\frac{log{p}_n}{n}=0.$$

(1)

For the sake of notational brevity, we denote by $\Delta$ the above grid ${\Delta }_n$, and all suprema or infima over $\Delta$ are supposed to be over ${\Delta }_n$. For any event B and any $\delta$ in $\Delta$, $F_{\delta }\left(B\right)$ (respectively $G_{\delta }\left(B\right)$) designates the probability of B under the distribution $F_{\delta }$ (respectively $G_{\delta }$). Given a sequence of levels ${\alpha }_n$, we consider a sequence of test criteria $T_n:=T_n\left(X_1,\dots ,X_n\right)$ of ${\mathbf{H}}_0\left(\delta \right)$, and the pertaining critical regions

$$T_n\left(X_1,\dots ,X_n\right)>A_n,$$

(2)

such that

$$F_{\delta }\left(T_n\left(X_1,\dots ,X_n\right)>A_n\right)\le {\alpha }_n\forall \delta \in \Delta ,$$

leading to rejection of ${\mathbf{H}}_0$($\delta$) for at least some $\delta \in \Delta .$

In an asymptotic context, it is natural to assume that ${\alpha }_n$ converges to 0 as n increases, since an increase in the sample size allows for a smaller first kind risk. For example, in [8], ${\alpha }_n$ takes the form ${\alpha }_n:=exp\{-n{a}_n\}$ for some sequence $a_n\to \infty$.

In the sequel, the Kullback-Leibler discrepancy between probability measures Q and P, with respective densities p and q (with respect to the Lebesgue measure on $\mathbb{R}$), is denoted

$$K(Q,P):=\int log\frac{q\left(x\right)}{p\left(x\right)}q\left(x\right)dx$$

whenever defined, and takes value $+\infty$ otherwise.

The present paper handles some issues with respect to this context. In Section 2, we consider some test procedures based on the supremum of Likelihood Ratios (LR) for various values of $\delta$, and define $T_n$. The threshold for such a test is obtained for any level ${\alpha }_n$, and a lower bound for its power is provided. In Section 3, we develop an asymptotic approach to the Least Favorable Hypotheses (LFH) for these tests. We prove that asymptotically least-favorable hypotheses are obtained through minimization of the Kullback-Leibler divergence between the two composite classes $\mathbf{H}\mathbf{0}$ and $\mathbf{H}\mathbf{1}$ independently upon the level of the test.

We next consider, in Section 3.3, the performance of the test numerically; indeed, under the least-favorable pair of hypotheses we compare the power of the test (as obtained through simulation) with the theoretical lower bound, as obtained in Section 2. We show that the minimal power, as measured under the LFH, is indeed larger than the theoretical lower bound—this result shows that the simulation results overperform on theoretical bounds. These results are developed in a number of examples.

Since no argument plays in favor of any type of optimality for the test based on the supremum of likelihood ratios for composite testing, we consider substituting those ratios with other kinds of scores in the family of divergence-based concepts, extending the likelihood ratio in a natural way. Such an approach has a long history, stemming from the seminal book by Liese and Vajda [10]. Extensions of the Kullback-Leibler based criterions (such as the likelihood ratio) to power-type criterions have been proposed for many applications in Physics and in Statistics (see, e.g., [11]). We explore the properties of those new tests under the pair of hypotheses minimizing the Kullback-Leibler divergence between the two composite classes $\mathbf{H}\mathbf{0}$ and $\mathbf{H}\mathbf{1}$. We show that, in some cases, we can build a test procedure whose properties overperform the above supremum of the LRTs, and we provide an explanation for this fact. This is the scope of Section 4.

2. An Extension of the Likelihood Ratio Test

For any $\delta$ in $\Delta$, let

$$T_{n,\delta }:=\frac{1}{n}\sum _{i=1}^{n}log\frac{g_{\delta }}{f_{\delta }}\left(X_i\right),$$

(3)

and define

$$T_n:=\underset{\delta \in \Delta }{sup}{T}_{n,\delta }.$$

Consider, for fixed $\delta$, the Likelihood Ratio Test with statistics $T_{n,\delta }$ which is uniformly most powerful (UMP) within all tests of $\mathbf{H}\mathbf{0}\left(\delta \right):=p_T=f_{\delta }$ vs. $\mathbf{H}\mathbf{1}\left(\delta \right):=p_T=g_{\delta }$, where $p_T$ designates the distribution of the generic r.v. X. The test procedure to be discussed aims at solving the question: Does there exist some $\delta$, for which $\mathbf{H}\mathbf{0}$($\delta$) would be rejected vs. $\mathbf{H}\mathbf{1}$($\delta$), for some prescribed value of the first kind risk?

Whenever $\mathbf{H}\mathbf{0}$($\delta$) is rejected in favor of $\mathbf{H}\mathbf{1}$($\delta$), for some $\delta$, we reject $\mathbf{H}\mathbf{0}:=f_0=g_0$ in favor of $\mathbf{H}\mathbf{1}:=f_0\ne g_0$. A critical region for this test with level ${\alpha }_n$ is defined by

$$T_n>A_n,$$

with

$$\begin{array}{lll}{P}_{\mathbf{H}\mathbf{0}}\left(\mathbf{H}\mathbf{1}\right)& =& \underset{\delta \in \Delta }{sup}{F}_{\delta }\left(T_n>A_n\right)\hfill \\ & =& \underset{\delta \in \Delta }{sup}{F}_{\delta }\left(\bigcup _{{\delta }^{\prime }}{T}_{n,{\delta }^{\prime }}>A_n\right)\le {\alpha }_n.\hfill \end{array}$$

Since, for any sequence of events $B_1,\dots ,B_{p_n}$,

$$F_{\delta }\left(\bigcup _{k=1}^{p_n}{B}_k\right)\le p_n\underset{1\le k\le p_n}{max}{F}_{\delta }\left(B_k\right),$$

it holds that

$$P_{\mathbf{H}\mathbf{0}}\left(\mathbf{H}\mathbf{1}\right)\le p_n\underset{\delta \in \Delta }{max}\underset{{\delta }^{\prime }\in \Delta }{max}{F}_{\delta }\left(T_{n,{\delta }^{\prime }}>A_n\right).$$

(4)

An upper bound for $P_{\mathbf{H}\mathbf{0}}\left(\mathbf{H}\mathbf{1}\right)$ can be obtained, making use of the Chernoff inequality for the right side of (4), providing an upper bound for the risk of first kind for a given $A_n$. The correspondence between $A_n$ and this risk allows us to define the threshold $A_n$ accordingly.

Turning to the power of this test, we define the risk of second kind by

$$\begin{array}{ll}{P}_{H_1}\left(\mathbf{H}\mathbf{0}\right)& :=\underset{\eta \in \Delta }{sup}{G}_{\eta }\left(T_n\le A_n\right)\\ & =\underset{\eta \in \Delta }{sup}{G}_{\eta }\left(\underset{\delta \in \Delta }{sup}{T}_{n,\delta }\le A_n\right)\\ & =\underset{\eta \in \Delta }{sup}{G}_{\eta }\left(\bigcap _{\delta \in \Delta }{T}_{n,\delta }\le A_n\right)\\ & \le \underset{\eta \in \Delta }{sup}{G}_{\eta }\left(T_{n,\eta }\le A_n\right),\end{array}$$

(5)

a crude bound which, in turn, can be bounded from above through the Chernoff inequality, which yields a lower bound for the power of the test under any hypothesis $g_{\eta }$ in $\mathbf{H}\mathbf{1}$.

Let ${\alpha }_n$ denote a sequence of levels, such that

$$lim\underset{n\to \infty }{sup}{\alpha }_n<1.$$

We make use of the following hypothesis:

$$\underset{\delta \in \Delta }{sup}\underset{{\delta }^{\prime }\in \Delta }{sup}\int log\frac{f_{{\delta }^{\prime }}}{g_{{\delta }^{\prime }}}{f}_{\delta }<0.$$

(6)

Remark 1.

Since

$$\int log\frac{f_{{\delta }^{\prime }}}{g_{{\delta }^{\prime }}}{f}_{\delta }=K\left(F_{\delta },G_{{\delta }^{\prime }}\right)-K\left(F_{\delta },F_{{\delta }^{\prime }}\right),$$

making use of the Chernoff-Stein Lemma (see Theorem A1 in the Appendix A), Hypothesis (6) means that any LRT with H0: $p_T=f_{\delta }$ vs. H1: $p_T=g_{{\delta }^{\prime }}$ is asymptotically more powerful than any LRT with H0: $p_T=f_{\delta }$ vs. H1: $p_T=f_{{\delta }^{\prime }}.$

Both hypotheses (7) and (8), which are defined below, are used to provide the critical region and the power of the test.

For all $\delta ,{\delta }^{\prime }$ define

$$Z_{{\delta }^{\prime }}:=log\frac{g_{{\delta }^{\prime }}}{f_{{\delta }^{\prime }}}\left(X\right),$$

and let

$${\phi }_{\delta ,{\delta }^{\prime }}\left(t\right):=log{E}_{F_{\delta }}\left(exp\left(t{Z}_{{\delta }^{\prime }}\right)\right)=log\int {\left(\frac{g_{{\delta }^{\prime }}\left(x\right)}{f_{{\delta }^{\prime }}\left(x\right)}\right)}^t{f}_{\delta }\left(x\right)dx.$$

With ${\mathcal{N}}_{\delta ,{\delta }^{\prime }}$, the set of all t such that ${\phi }_{\delta ,{\delta }^{\prime }}\left(t\right)$ is finite, we assume

$${\mathcal{N}}_{\delta ,{\delta }^{\prime }}\mathrm{is}\mathrm{a}\mathrm{non}\mathrm{void}\mathrm{open}\mathrm{neighborhood}\mathrm{of}0.$$

(7)

Define, further,

$$J_{\delta ,{\delta }^{\prime }}\left(x\right):=\underset{t}{sup}tx-{\phi }_{\delta ,{\delta }^{\prime }}\left(t\right),$$

and let

$$J\left(x\right):=\underset{\left(\delta ,{\delta }^{\prime }\right)\in \Delta \times \Delta }{min}{J}_{\delta ,{\delta }^{\prime }}\left(x\right).$$

For any $\eta$, let

$$W_{\eta }:=-log\frac{g_{\eta }}{f_{\eta }}\left(X\right),$$

and let

$${\psi }_{\eta }\left(t\right):=log{E}_{G_{\eta }}\left(exp\left(t{W}_{\eta }\right)\right).$$

Let ${\mathcal{M}}_{\eta }$ be the set of all t such that ${\psi }_{\eta }\left(t\right)$ is finite. Assume

$${\mathcal{M}}_{\eta }\mathrm{is}\mathrm{a}\mathrm{non}\mathrm{void}\mathrm{neighborhood}\mathrm{of}0.$$

(8)

Let

$$I_{\eta }\left(x\right):=\underset{t}{sup}tx-log{E}_{G_{\eta }}\left(exp\left(t{W}_{\eta }\right)\right),$$

(9)

and

$$I\left(x\right):=\underset{\eta }{inf}{I}_{\eta }\left(x\right).$$

We also assume an accessory condition on the support of $Z_{{\delta }^{\prime }}$ and $W_{\eta }$, respectively under $F_{\delta }$ and under $G_{\eta }$ (see (A2) and (A5) in the proof of Theorem A1). Suppose the regularity assumptions (7) and (8) are fulfilled for all $\delta ,{\delta }^{\prime }$ and $\eta$. Assume, further, that $p_n$ fulfills (1).

The following result holds:

Proposition 2.

Whenever (6) holds, for any sequence of levels ${\alpha }_n$ bounded away from 1, defining

$$A_n:=J^{-1}\left(-\frac{1}{n}log\frac{{\alpha }_n}{p_n}\right),$$

it holds, for large n, that

$$P_{\mathbf{H}\mathbf{0}}\left(\mathbf{H}\mathbf{1}\right)=\underset{\delta \in \Delta }{sup}{F}_{\delta }\left(T_n>A_n\right)\le {\alpha }_n$$

and

$$P_{\mathbf{H}\mathbf{1}}\left(\mathbf{H}\mathbf{1}\right)=\underset{\delta \in \Delta }{sup}{G}_{\delta }\left(T_n>A_n\right)\ge 1-exp\left(-nI\left(A_n\right)\right).$$

3. Minimax Tests under Noisy Data, Least-Favorable Hypotheses

3.1. An Asymptotic Definition for the Least-Favorable Hypotheses

We prove that the above procedure is asymptotically minimax for testing the composite hypothesis ${\mathbf{H}}_0$ against the composite alternative ${\mathbf{H}}_1$. Indeed, we identify the least-favorable hypotheses, say $F_{{\delta }_{\ast }}\in {\mathbf{H}}_0$ and $G_{{\delta }_{\ast }}\in {\mathbf{H}}_1$, which lead to minimal power and maximal first kind risk for these tests. This requires a discussion of the definition and existence of such a least-favourable pair of hypotheses in an asymptotic context; indeed, for a fixed sample size, the usual definition only leads to an explicit definition in very specific cases. Unlike in [1], the minimax tests will not be in the sense of Huber and Strassen. Indeed, on one hand, hypotheses ${\mathbf{H}}_0$ and ${\mathbf{H}}_1$ are not defined in topological neighbourhoods of $F_0$ and $G_0$, but rather through a convolution under a parametric setting. On the other hand, the specific test of $\left\{{\mathbf{H}}_0\left(\delta \right),\delta \in \Delta \right\}$ against $\left\{{\mathbf{H}}_1\left(\delta \right),\delta \in \Delta \right\}$ does not require capacities dominating the corresponding probability measures.

Throughout the subsequent text, we shall assume that there exists ${\delta }_{\ast }$ such that

$$\underset{\delta \in \Delta }{min}K\left(F_{\delta },G_{\delta }\right)=K\left(F_{{\delta }_{\ast }},G_{{\delta }_{\ast }}\right).$$

(10)

We shall call the pair of distributions $\left(F_{\underline{\delta }},G_{\underline{\delta }}\right)$ least-favorable for the sequence of tests $1\left\{T_n>A_n\right\}$ if they satisfy

$$\begin{array}{c}{F}_{\delta }\left(T_n\le A_n\right)\ge F_{\underline{\delta }}\left(T_n\le A_n\right)\hfill \\ \hfill \ge G_{\underline{\delta }}\left(T_n\le A_n\right)\ge G_{\delta }\left(T_n\le A_n\right)& \hfill \end{array}$$

(11)

for all $\delta \in \Delta .$ The condition of unbiasedness of the test is captured by the central inequality in (11).

Because, for finite n, such a pair can be constructed only in few cases, we should take a recourse of (11) to the asymptotics $n\to \infty .$ We shall show that any pair of distributions $\left(F_{{\delta }_{\ast }}{G}_{{\delta }_{\ast }}\right)$ achieving (10) are least-favorable. Indeed, it satisfies the inequality (11) asymptotically on the logarithmic scale.

Specifically, we say that $\left(F_{\underline{\delta }},G_{\underline{\delta }}\right)$ is a least-favorable pair of distributions when, for any $\delta \in \Delta$,

$$\begin{array}{cc}lim\underset{n\to \infty }{inf}\frac{1}{n}log{F}_{\underline{\delta }}\left(T_n\le A_n\right)& \ge \underset{n\to \infty }{lim}\frac{1}{n}log{G}_{\underline{\delta }}\left(T_n\le A_n\right)\hfill \\ & \ge \underset{n\to \infty }{lim}sup\frac{1}{n}log{G}_{\delta }\left(T_n\le A_n\right).\hfill \end{array}$$

(12)

Define the total variation distance

$$d_{TV}\left(F_{\delta },G_{\delta }\right):=\underset{B}{sup}\left|F_{\delta }\left(B\right)-G_{\delta }\left(B\right)\right|,$$

where the supremum is over all Borel sets B of $\mathbb{R}$. We will assume that, for all n,

$${\alpha }_n<1-\underset{\delta \in \Delta }{sup}{d}_{TV}\left(F_{\delta },G_{\delta }\right).$$

(13)

We state our main result, whose proof is deferred to the Appendix B.

Theorem 3.

For any level ${\alpha }_n$ satisfying (13), the pair $\left(F_{{\delta }_{\ast }},G_{{\delta }_{\ast }}\right)$ is a least-favorable pair of hypotheses for the family of tests $1\left\{T_n\ge A_n\right\}$, in the sense of (12).

3.2. Identifying the Least-Favorable Hypotheses

We now concentrate on (10).

For given $\delta \in \left[{\delta }_{min},{\delta }_{max}\right]$ with ${\delta }_{min}>0$, the distribution of the r.v. $V_{\delta }$ is obtained through some transformation from the known distribution of $V.$ The classical example is $V_{\delta }=\sqrt{\delta }V$, which is a scaling, and where $\sqrt{\delta }$ is the signal to noise ratio. The following results state that the Kullback-Leibler discrepancy $K\left(F_{\delta },G_{\delta }\right)$ reaches its minimal value when the noise $V_{\delta }$ is “maximal”, under some additivity property with respect to $\delta .$ This result is not surprising: Adding noise deteriorates the ability to discriminate between the two distributions $F_0$ and $G_0$—this effect is captured in $K\left(F_{\delta },G_{\delta }\right)$, which takes its minimal value for the maximal $\delta .$

Proposition 4.

Assume that, for all $\delta ,\eta$, there exists some r.v. $W_{\delta ,\eta }$ such that $V_{\delta +\eta }{=}_d{V}_{\delta }+W_{\delta ,\eta }$ where $V_{\delta }$ and $W_{\delta ,\eta }$ are independent. Then

$${\delta }_{\ast }={\delta }_{max}.$$

This result holds as a consequence of Lemma A5 in the Appendix C.

In the Gaussian case, when h is the standard normal density, Proposition 4 holds, since $h_{\delta +\eta }=h_{\delta }\ast h_{\eta -\delta }$ with $h_{\epsilon }\left(x\right):=\left(1/\sqrt{\epsilon }\right)h\left(x/\sqrt{\epsilon }\right).$ In order to model symmetric noise, we may consider a symmetrized Gamma density as follows: Set $h_{\delta }\left(x\right):=\left(1/2\right){\gamma }^{+}(1,\delta )\left(x\right)+\left(1/2\right){\gamma }^{-}(1,\delta )\left(x\right)$, where ${\gamma }^{+}(1,\delta )$ designates the Gamma density with scale parameter 1 and shape parameter $\delta$, and ${\gamma }^{-}(1,\delta )$ the Gamma density on ${\mathbb{R}}^{-}$ with same parameter. Hence a r.v. with density $h_{\delta }$ is symmetrically distributed and has variance $2\delta .$ Clearly, $h_{\delta +\eta }\left(x\right)=h_{\delta }\ast h_{\eta }\left(x\right)$, which shows that Proposition 4 also holds in this case. Note that, except for values of $\delta$ less than or equal to 1, the density $h_{\delta }$ is bimodal, which does not play in favour of such densities for modelling the uncertainty, due to the noise. In contrast with the Gaussian case, $h_{\delta }$ cannot be obtained from $h_1$ by any scaling. The centred Cauchy distribution may help as a description of heavy tailed symmetric noise, and keeps uni-modality through convolution; it satisfies the requirements of Proposition 4 since $f_{\delta }\ast f_{\eta }\left(x\right)=f_{\delta +\eta }\left(x\right)$ where $f_{\epsilon }\left(x\right):=\epsilon /\pi \left(x^2+{\epsilon }^2\right)$. In this case, $\delta$ acts as a scaling, since $f_{\delta }$ is the density of $\delta X$ where X has density $f_1.$

In practice, the interesting case is when $\delta$ is the variance of the noise and corresponds to a scaling of a generic density, as occurs for the Gaussian case or for the Cauchy case. In the examples, which will be used below, we also consider symmetric, exponentially distributed densities (Laplace densities) or symmetric Weibull densities with a given shape parameter. The Weibull distribution also fulfills the condition in Proposition 4, being infinitely divisible (see [12]).

3.3. Numerical Performances of the Minimax Test

As frequently observed, numerical results deduced from theoretical bounds are of poor interest due to the sub-optimality of the involved inequalities. They may be sharpened on specific cases. This motivates the need for simulation. We study two cases, which can be considered as benchmarks.

• In the first case, $f_0$ is a normal density with expectation 0 and variance 1, whereas $g_0$ is a normal density with expectation $0.3$ and variance $1$.
• The second case handles a situation where $f_0$ and $g_0$ belong to different models: $f_0$ is a log-normal density with location parameter 1 and scale parameter $0.2$, whereas $g_0$ is a Weibull density on ${\mathbb{R}}^{+}$ with shape parameter 5 and scale parameter $3$. Those two densities differ strongly, in terms of asymptotic decay. They are, however, very close to one another in terms of their symmetrized Kullback-Leibler divergence (the so-called Jeffrey distance). Indeed, centering on the log-normal distribution $f_0$, the closest among all Weibull densities is at distance $0.10$—the density $g_0$ is at distance $0.12$ from $f_0.$

Both cases are treated, considering four types of distribution for the noise:

• The noise $h_{\delta }$ is a centered normal density with variance ${\delta }^2$;
• the noise $h_{\delta }$ is a centered Laplace density with parameter $\lambda \left(\delta \right)$;
• the noise $h_{\delta }$ is a symmetrized Weibull density with shape parameter $1.5$ and variable scale parameter $\beta \left(\delta \right)$; and
• the noise $h_{\delta }$ is Cauchy with density $h_{\delta }\left(x\right)=\gamma \left(\delta \right)/\pi (\gamma {\left(\delta \right)}^2+x^2)$.

In order to compare the performances of the test under those four distributions, we have adopted the following rule: The parameter of the distribution of the noise is tuned such that, for each value $\underline{\delta }$, it holds that $P\left(\left|V_{\underline{\delta }}\right|>\underline{\delta }\right)=\Phi \left(1\right)-\Phi (-1)\sim 0.65$, where $\Phi$ stands for the standard Gaussian cumulative function. Thus, distributions b to d are scaled with respect to the Gaussian noise with variance ${\delta }^2$.

In both cases A and B, the range of $\delta$ is $\Delta =\left({\delta }_{min}=0.1,{\delta }_{max}\right)$, and we have selected a number of possibilities for ${\delta }_{max}$, ranging from $0.2$ to $0.7$.

In case A, we selected $={\delta }_{max}^2=0.5$, which has a signal-to-noise ratio equal to $0.7$, a commonly chosen bound in quality control tests.

In case B, the variance of $f_0$ is roughly $0.6$ and the variance of $g_0$ is roughly $0.4.$ The maximal value of ${\delta }_{max}^2$ is roughly $0.5.$ This is thus a maximal upper bound for a practical modeling.

We present some power functions, making use of the theoretical bounds together with the corresponding ones based on simulation runs. As seen, the performances in the theoretical approach is weak. We have focused on simulation, after some comparison with the theoretical bounds.

3.3.1. Case A: The Shift Problem

In this subsection, we evaluate the quality of the theoretical power bound, defined in the previous sections. Thus, we compare the theoretical formula to the empirical lower performances obtained through simulations under the least-favorable hypotheses.

Theoretical Power Bound

While supposedly valid for finite n, the theoretical power bound given by (A8) still assumes some sort of asymptotics, since a good approximation of the bound implies a fine discretization of $\Delta$ to compute $I\left(A_n\right)={inf}_{\eta \in {\Delta }_n}{I}_{\eta }\left(A_n\right)$. Thus, by condition (1), n has to be large. Therefore, in the following, we will compute this lower bound for n sufficiently large (that is, at least 100 observations), which is also consistent with industrial applications.

Numerical Power Bound

In order to obtain a minimal bound for the power of the composite test, we compute the power of the test ${\mathbf{H}}_0\left({\delta }_{\ast }\right)$ against ${\mathbf{H}}_1\left({\delta }_{\ast }\right)$, where ${\delta }_{\ast }$ defines the LFH pair $\left(F_{{\delta }_{\ast }},G_{{\delta }_{\ast }}\right)$.

Following Proposition 4, the LFH for the test defined by $T_n$ when the noise follows a Gaussian, a Cauchy, or a symmetrized Weibull distribution is achieved for $\left(F_{{\delta }_{max}},G_{{\delta }_{max}}\right)$.

When the noise follows a Laplace distribution, the pair of LFH is the one that satisfies:

$$\left(F_{{\delta }_{\ast }},G_{{\delta }_{\ast }}\right)=arg\underset{(F_{\delta },G_{\delta }),\delta \in {\Delta }_n}{min}K(F_{\delta },G_{\delta }).$$

(14)

In both of the cases A and B, this condition is also satisfied for ${\delta }^{\ast }={\delta }_{max}$.

Comparison of the Two Power Curves

As expected, Figure 1, Figure 2 and Figure 3 show that the theoretical lower bound is always below the empirical lower bound, when n is high enough to provide a good approximation of $I\left(A_n\right)$. This is also true when the noise follows a Cauchy distribution, but for a bigger sample size than in the figures above ($n>250$).

In most cases, the theoretical bound tends to largely underestimate the power of the test, when compared to its minimal performance given by simulations under the least-favorable hypotheses. The gap between the two also tends to increase as n grows. This result may be explained by the large bound provided by (5), while the numerical performances are obtained with respect to the least-favorable hypotheses.

From a computational perspective, the computational cost of the theoretical bound is far higher than its numeric counterpart.

3.3.2. Case B: The Tail Thickness Problem

The calculation of the moment-generating function, appearing in the formula of $I_{\eta }\left(x\right)$ in (9), is numerically unstable, which renders the computation of the theoretical bound impossible. Thus, in the following sections, the performances of the test will be evaluated numerically, through Monte Carlo replications.

4. Some Alternative Statistics for Testing

4.1. A Family of Composite Tests Based on Divergence Distances

This section provides a similar treatment as above, dealing now with some extensions of the LRT test to the same composite setting. The class of tests is related to the divergence-based approach to testing, and it includes the cases considered so far. For reasons developed in Section 3.3, we argue through simulation and do not develop the corresponding large deviation approach.

The statistics $T_n$ can be generalized in a natural way, by defining a family of tests depending on some parameter $\gamma .$ For $\gamma \ne 0,1$, let

$${\varphi }_{\gamma }\left(x\right):=\frac{x^{\gamma }-\gamma x+\gamma -1}{\gamma (\gamma -1)}$$

be a function defined on $\left(0,\infty \right)$ with values in $\left(0,\infty \right)$, setting

$${\varphi }_0\left(x\right):=-logx+x-1$$

and

$${\varphi }_1\left(x\right):=xlogx-x+1.$$

For $\gamma \le 2$, this class of functions is instrumental in order to define the so-called power divergences between probability measures, a class of pseudo-distances widely used in statistical inference (see, for example, [13]).

Associated to this class, consider the function

$$\begin{array}{cc}{\phi }_{\gamma }\left(x\right)& :=-\frac{d}{dx}{\varphi }_{\gamma }\left(x\right)\hfill \\ & =\frac{1-x^{\gamma -1}}{\gamma -1}\mathrm{for}\gamma \ne 0,1.\hfill \end{array}$$

We also consider

$$\begin{array}{l}{\phi }_1\left(x\right):=-logx\\ {\phi }_0\left(x\right):=\frac{1}{x}-1,\end{array}$$

from which the statistics

$$T_{n,\delta }^{\gamma }:=\frac{1}{n}\sum _{i=1}^n{\phi }_{\gamma }\left(X_i\right)$$

and

$$T_n^{\gamma }:=\underset{\delta }{sup}{T}_{n,\delta }^{\gamma }$$

are well defined, for all $\gamma \le 2$. Figure 4 illustrates the functions ${\phi }_{\gamma }$, according to $\gamma$.

Fix a risk of first kind $\alpha$, and the corresponding power of the LRT pertaining to $H0\left({\delta }_{\ast }\right)$ vs. $H1\left({\delta }_{\ast }\right)$ by

$$1-\beta :=G_{{\delta }_{\ast }}\left(T_{n,{\delta }_{\ast }}^1>s_{\alpha }\right),$$

with

$$s_{\alpha }:=inf\left\{s:F_{{\delta }_{\ast }}\left(T_{n,{\delta }_{\ast }}^1>s\right)\le \alpha \right\}.$$

Define, accordingly, the power of the test, based on $T_n^{\gamma }$ under the same hypotheses, by

$$s_{\alpha }^{\gamma }:=inf\left\{s:F_{{\delta }_{\ast }}\left(T_n^{\gamma }>s\right)\le \alpha \right\}$$

and

$$1-{\beta }^{\prime }:=G_{{\delta }_{\ast }}\left(T_n^{\gamma }>s_{\alpha }^{\gamma }\right).$$

First, ${\delta }_{\ast }$ defines the pair of hypotheses $\left(F_{{\delta }_{\ast }},G_{{\delta }_{\ast }}\right)$, such that the LRT with statistics $T_{n,{\delta }_{\ast }}^1$ has maximal power among all tests $H0\left({\delta }_{\ast }\right)$ vs. $H1\left({\delta }_{\ast }\right).$ Furthermore, by Theorem A1, it has minimal power on the logarithmic scale among all tests $H0\left(\delta \right)$ vs. $H1\left(\delta \right).$

On the other hand, $\left(F_{{\delta }_{\ast }},G_{{\delta }_{\ast }}\right)$ is the LF pair for the test with statistics $T_n^1$ among all pairs $\left(F_{\delta },G_{\delta }\right).$

These two facts allow for the definition of the loss of power, making use of $T_n^1$ instead of $T_{n,{\delta }_{\ast }}^1$ for testing $H0\left({\delta }_{\ast }\right)$ vs. $H1\left({\delta }_{\ast }\right).$ This amounts to considering the price of aggregating the local tests $T_{n,\delta }^1$, a necessity since the true value of $\delta$ is unknown. A natural indicator for this loss consists in the difference

$${\Delta }_n^1:=G_{{\delta }_{\ast }}\left(T_{n,{\delta }_{\ast }}^1>s_{\alpha }\right)-G_{{\delta }_{\ast }}(T_n^1>s_{\alpha }^1)\ge 0.$$

Consider, now, aggregated test statistics $T_n^{\gamma }.$ We do not have at hand a similar result, as in Proposition 2. We, thus, consider the behavior of the test $H0\left({\delta }_{\ast }\right)$ vs. $H1\left({\delta }_{\ast }\right)$, although $\left(F_{{\delta }_{\ast }},G_{{\delta }_{\ast }}\right)$ may not be a LFH for the test statistics $T_n^{\gamma }.$ The heuristics, which we propose, makes use of the corresponding loss of power with respect to the LRT, through

$${\Delta }_n^{\gamma }:=G_{{\delta }_{\ast }}\left(T_{n,{\delta }_{\ast }}^1>s_{\alpha }\right)-G_{{\delta }_{\ast }}\left(T_n^{\gamma }>s_{\alpha }^{\gamma }\right).$$

We will see that it may happen that ${\Delta }_n^{\gamma }$ improves over ${\Delta }_n^1.$ We define the optimal value of $\gamma$, say ${\gamma }^{\ast }$, such that

$${\Delta }_n^{{\gamma }^{\ast }}\le {\Delta }_n^{\gamma },$$

for all $\gamma .$

In the various figures hereafter, NP corresponds to the LRT defined between the LFH’s $\left(F_{{\delta }_{\ast }},G_{{\delta }_{\ast }}\right),$ KL to the test with statistics $T_n^1$ (hence, as presented Section 2), HELL corresponds to $T_n^{1/2}$, which is associated to the Hellinger power divergence, and G = 2 corresponds to $\gamma =2.$

4.2. A Practical Choice for Composite Tests Based on Simulation

We consider the same cases A and B, as described in Section 3.3.

As stated in the previous section, the performances of the different test statistics are compared, considering the test of ${\mathbf{H}}_0\left({\delta }_{\ast }\right)$ against ${\mathbf{H}}_1\left({\delta }_{\ast }\right)$ where ${\delta }^{\ast }$ is defined, as explained in Section 3.3 as the LFH for the test $T_n^1$. In both cases A and B, this corresponds to ${\delta }^{\ast }={\delta }_{max}$.

4.2.1. Case A: The Shift Problem

Overall, the aggregated tests perform well, when the problem consists in identifying a shift in a distribution. Indeed, for the three values of $\gamma$ (0.5, 1, and 2), the power remains above 0.7 for any kind of noise and any value of ${\delta }_{\ast }$. Moreover, the power curves associated to $T_n^{\gamma }$ mainly overlap with the optimal test $T_{n,{\delta }_{\ast }}^1$.

• Under Gaussian noise, the power remains mostly stable over the values of ${\delta }_{\ast }$, as shown by Figure 5. The tests with statistics $T_n^1$ and $T_n^2$ are equivalently powerful for large values of ${\delta }_{\ast }$, while the first one achieves higher power when ${\delta }_{\ast }$ is small.
• When the noise follows a Laplace distribution, the three power curves overlap the NP power curve, and the different test statistics can be indifferently used. Under such a noise, the alternative hypotheses are extremely well distinguished by the class of tests considered, and this remains true as ${\delta }_{\ast }$ increases (cf. Figure 6).
• Under the Weibull hypothesis, $T_n^1$ and $T_n^2$ perform similarly well, and almost always as well as $T_{n,{\delta }_{\ast }}^1$, while the power curve associated to $T_n^{1/2}$ remains below. Figure 7 illustrates that, as ${\delta }_{max}$ increases, the power does not decrease much.
• Under a Cauchy assumption, the alternate hypotheses are less distinguishable than under any other parametric hypothesis on the noise, since the maximal power is about 0.84, while it exceeds 0.9 in cases a, b, and c (cf. Figure 5, Figure 6, Figure 7 and Figure 8). The capacity of the tests to discriminate between $\mathbf{H}\mathbf{0}\left({\delta }_{max}\right)$ and $\mathbf{H}\mathbf{1}\left({\delta }_{max}\right)$ is almost independent of the value of ${\delta }_{max}$, and the power curves are mainly flat.

4.2.2. Case B: The Tail Thickness Problem

• With the noise defined by case A (Gaussian noise), for KL ($\gamma =1$), ${\delta }_{\ast }={\delta }_{max}$ due to Proposition 4 and statistics $T_n^1$ provides the best power uniformly upon ${\delta }_{max}.$ Figure 9 shows a net decrease of the power as ${\delta }_{max}$ increases (recall that the power is evaluated under the least favorable alternative $G_{{\delta }_{max}}$).
• When the noise follows a Laplace distribution, the situation is quite peculiar. For any value of $\delta$ in $\Delta$, the modes $M_{G_{{\delta }_{max}}}$ and $M_{F_{{\delta }_{max}}}$ of the distributions of $\left(f_{\delta }/g_{\delta }\right)\left(X\right)$ under $G_{{\delta }_{max}}$ and under $F_{{\delta }_{max}}$ are quite separated; both larger than $1.$ Also, for $\delta$ all the values of $\left|{\varphi }_{\gamma }\right(M_{G_{{\delta }_{max}}})-{\varphi }_{\gamma }(M_{F_{{\delta }_{max}}}\left)\right|$ are quite large for large values of $\gamma .$ We may infer that the distributions of ${\varphi }_{\gamma }\left(\left(f_{\delta }/g_{\delta }\right)\left(X\right)\right)$ under $G_{{\delta }_{max}}$ and under $F_{{\delta }_{max}}$ are quite distinct for all $\delta$, which in turn implies that the same fact holds for the distributions of $T_n^{\gamma }$ for large $\gamma .$ Indeed, simulations presented in Figure 10 show that the maximal power of the test tends to be achieved when $\gamma =2.$
• When the noise follows a symmetric Weibull distribution, the power function when $\gamma =1$ is very close to the power of the LRT between $F_{{\delta }_{max}}$ and $G_{{\delta }_{max}}$ (cf. Figure 11). Indeed, uniformly on $\delta$, and on x, the ratio $\left(f_{\delta }/g_{\delta }\right)\left(x\right)$ is close to 1. Therefore, the distribution of $T_n^1$ is close to that of $T_{n,{\delta }_{max}}^1$, which plays in favor of the KL composite test.
• Under a Cauchy distribution, similarly to case A, Figure 12 shows that $T_n^{\gamma }$ achieves the maximal power for $\gamma =1$ and 2, closely followed by $\gamma =0.5$.

5. Conclusions

We have considered a composite testing problem, where simple hypotheses in either $\mathbf{H}\mathbf{0}$ and $\mathbf{H}\mathbf{1}$ were paired, due to corruption in the data. The test statistics were defined through aggregation of simple likelihood ratio tests. The critical region for this test and a lower bound of its power was produced. We have shown that this test is minimax, evidencing the least-favorable hypotheses. We have considered the minimal power of the test under such a least favorable hypothesis, both theoretically and by simulation, and for a number of cases (including corruption by Gaussian, Laplacian, symmetrized Weibull, and Cauchy noise). Whatever this noise, the actual minimal power, as measured through simulation, was quite higher than obtained through analytic developments. Least-favorable hypotheses were defined in an asymptotic sense, and were proved to be the pair of simple hypotheses in $\mathbf{H}\mathbf{0}$ and $\mathbf{H}\mathbf{1}$ which are closest, in terms of the Kullback-Leibler divergence; this holds as a consequence of the Chernoff-Stein Lemma. We, next, considered aggregation of tests where the likelihood ratio was substituted by a divergence-based statistics. This choice extended the former one, and may produce aggregate tests with higher power than obtained through aggregation of the LRTs, as examplified and analysed. Open questions are related to possible extensions of the Chernoff-Stein Lemma for divergence-based statistics.