Highly Efficient Robust and Stable M-Estimates of Location

Abstract

This article is partially a review and partially a contribution. The classical two approaches to robustness, Huber’s minimax and Hampel’s based on influence functions, are reviewed with the accent on distribution classes of a non-neighborhood nature. Mainly, attention is paid to the minimax Huber’s M-estimates of location designed for the classes with bounded quantiles and Meshalkin-Shurygin’s stable M-estimates. The contribution is focused on the comparative performance evaluation study of these estimates, together with the classical robust M-estimates under the normal, double-exponential (Laplace), Cauchy, and contaminated normal (Tukey gross error) distributions. The obtained results are as follows: (i) under the normal, double-exponential, Cauchy, and heavily-contaminated normal distributions, the proposed robust minimax M-estimates outperform the classical Huber’s and Hampel’s M-estimates in asymptotic efficiency; (ii) in the case of heavy-tailed double-exponential and Cauchy distributions, the Meshalkin-Shurygin’s radical stable M-estimate also outperforms the classical robust M-estimates; (iii) for moderately contaminated normal, the classical robust estimates slightly outperform the proposed minimax M-estimates. Several directions of future works are enlisted.

1. Introduction

Robust statistics, as a new field of mathematical statistics, originates from the pioneering works of John Tukey (1960) [1], Peter Huber (1964) [2], and Frank Hampel (1968) [3]. The term “robust” (Latin: strong, vigorous, sturdy, tough, powerful) was introduced into statistics by George Box (1953) [4].

The reasons of research in this field of statistics are of a general mathematical nature: the conceptions of “optimality” and “stability” are mutually complementary in performance evaluation for almost all mathematical procedures, and the trade-off between them is often a sought goal.

It is not rare that the performance of optimal solutions is rather sensitive to small violations of the assumed conditions of optimality. In statistics, the classical example of such unstable optimal procedure is given by the least squares estimates, in which performance under small deviations from normality can become disastrous [5].

Since the term “stability” is overloaded in mathematics, the term “robustness” being its synonym is at present conventionally used in statistics and in optimal control theory: in general, it means the stability of statistical inference under uncontrolled violations of accepted distribution models.

In present, there are two main approaches to robustness: historically, the first global minimax approach of Huber (quantitative robustness) [5] and the local approach of Hampel based on influence functions (qualitative robustness) [6]. Within the first approach, the least informative (favorable) distribution minimizing Fisher information over a certain distribution class is obtained with the subsequent use of the asymptotically optimal maximum likelihood parameter estimate for this distribution. In this case, the minimax approach gives the guaranteed accuracy of robust estimates, that is, the asymptotic variance of the optimal parameter estimate is upper-bounded for distributions from the aforementioned class.

Within the second approach, a parameter estimate is defined by its desired influence function, which determines the qualitative robustness properties of an estimate, such as its low sensitivity to the presence of gross outliers in the data, to the data rounding-off, to the missing data, etc.

In what follows, we consider these methodologies in detail focusing on the optimization and variational calculus tools used in both aforementioned approaches. Within Huber’s minimax approach, we review the conventional least informative (favorable) distributions obtained for the neighborhoods of a Gaussian [5], as well as those designed for a variety of the non-standard distribution classes of a non-neighborhood nature [7]. Within Hampel’s local approach [6], we mostly emphasize its recently developed stable estimation branch with the originally posed variational calculus problems and rather prospective results on their application to robust statistics [8].

While this paper focuses on particular topics in the field of robust statistics, it is worth noting a few comprehensive reviews also covering the present state of art in this field, namely Reference [9,10,11,12,13,14].

An outline of the remainder of the article is as follows. In Section 2, a general problem setting for the design of minimax variance M-estimates of location is recalled. In Section 3, the globally stable Meshalkin-Shurygin’s M-estimates are described. In Section 4, a comparative performance evaluation of the conventional robust M-estimates of location with the several novel proposed M-estimates is examined (univariate setting is considered throughout the paper), and several unforeseen and unexpected results have been obtained. In Section 5, some conclusions are given.

2. Huber’s Minimax Variance Robust $\mathbf{M}$-Estimates of Location

2.1. Preliminaries

The minimax principle aims at the worst case suggesting for it the best solution [2]; thus, this approach provides a guaranteed result [5]. However, being applied to the problem of estimation of location, it yields a robust version of the principle of the maximum likelihood [2]. Usually, estimation of location is of a primary interest, and, in this study, we focus on it.

Let $x_1,\dots ,x_n$ be a sample from a distribution with density $p(x-\theta )$ from a convex class $\mathcal{P}$, where $\theta$ is a location parameter. Further, we assume that p is a symmetric distribution density; hence, $\theta$ is the center of symmetry to be estimated.

An M-estimate $T_n$ of $\theta$ is a solution to the following minimization problem:

$$T_n=arg\underset{\theta }{min}\sum _{i=1}^n\rho (x_i-\theta ),$$

(1)

where $\rho \left(u\right)$ is called the function of contrast [15]: $\rho (x_i-\theta )$ is a measure of difference between the observation $x_i$ and the estimated center of symmetry. The following particular cases of (1) are of a particular interest: (i) for $\rho \left(u\right)=u^2$, we have the least squares method with the sample mean $\overline{x}$ as the estimate of location; (ii) for $\rho \left(u\right)=\left|u\right|$, we arrive at the least absolute values method with the sample median estimate $medx$; and (iii), mostly importantly, for a given density p, the choice $\rho \left(u\right)=-logp\left(u\right)$ yields the maximum likelihood (ML) estimate of location.

It is more convenient to formulate the properties of M-estimates in terms of the derivative of the function of contrast $\psi \left(u\right)={\rho }^{\prime }\left(u\right)$ called a score function. In this case, an M-estimate is defined as a solution to the following implicit estimating equation

$$\sum _{i=1}^n\psi (x_i-T_n)=0.$$

(2)

Under rather general conditions of regularity imposed on the class $\mathsf{\Psi }$ of score functions $\psi$ and on the class $\mathcal{P}$ of densities p (their various forms can be found in Reference [2,5,6]), M-estimates are consistent and asymptotically normal with the asymptotic variance $AV$:

$$Var\left(n^{1/2}{T}_n\right)=AV(\psi ,p)=\frac{A}{B^2}=\frac{\int {\psi }^2\left(x\right)p\left(x\right)dx}{{\left(\int {\psi }^{\prime }\left(x\right)p\left(x\right)dx\right)}^2},$$

(3)

where

$$A(\psi ,p)=\int {\psi }^2\left(x\right)p\left(x\right)dx,$$

$$B(\psi ,p)=\int {\psi }^{\prime }\left(x\right)p\left(x\right)dx.$$

For M-estimates (2), the following result holds [5].

Theorem 1.

(Huber, 1964) Under regularity conditions, M-estimates satisfy the minimax property

$$AV({\psi }^{*},p)\le AV({\psi }^{*},p^{*})=\underset{p\in \mathcal{P}}{sup}\underset{\psi \in \mathsf{\Psi }}{inf}AV(\psi ,p)\le AV(\psi ,p^{*}),$$

(4)

where $p^{*}\left(x\right)$ is the least informative (favorable) density $p^{*}$ minimizing Fisher information for location $I\left(p\right)$ over the class $\mathcal{P}$:

$$p^{*}=arg\underset{p\in \mathcal{P}}{min}I\left(p\right),I\left(p\right)=\int {\left[\frac{p^{\prime }\left(x\right)}{p\left(x\right)}\right]}^{2}p\left(x\right)dx.$$

(5)

From (4) and (5), it follows that the minimax function of contrast and score function are given by the maximum likelihood method for the least informative density $p^{*}$:

$${\rho }^{*}\left(x\right)=-log{p}^{*}\left(x\right),{\psi }^{*}\left(x\right)=-{p^{*}\left(x\right)}^{\prime }/p^{*}\left(x\right).$$

(6)

Thus, the pair $({\psi }^{*},p^{*})$ is the saddle-point of the functional $AV(\psi ,p)$. The right-hand part of inequality (4) is the Rao–Cramér inequality:

$$AV(\psi ,p^{*})\ge AV(-{p^{*}}^{\prime }/p^{*},p^{*})=1/\int {\left({p^{*}\left(x\right)}^{\prime }\right)}^2/p^{*}\left(x\right)dx=1/I\left(p^{*}\right),$$

whereas its left-hand part guarantees the asymptotic accuracy of robust minimax estimation with the following upper bound upon the asymptotic variance of the minimax variance robust M-estimate of location: $AV({\psi }^{*},p)\le 1/I\left(p^{*}\right)$.

The key point of the minimax approach is the solution of the variational problem (5): further, various classes $\mathcal{P}$ with the corresponding least informative densities $p^{*}$ and minimax estimates are enlisted.

Now, we recall the Huber’s classical solution for $\epsilon$-contaminated normal distributions (Tukey’s gross-error model):

$${\mathcal{P}}_{\epsilon }=\{p:p\left(x\right)\ge (1-\epsilon )\phi \left(x\right),0\le \epsilon <1\},$$

(7)

where $\phi \left(x\right)={\left(2\pi \right)}^{-1/2}exp(-x^2/2)$ is the standard normal distribution density.

Theorem 2.

(Huber, 1964) In the class ${\mathcal{P}}_{\epsilon }$, the least informative density $p^{*}$ and the optimal score function has the following form [2]:

$$p^{*}\left(x\right)=p_{Huber}\left(x\right)=\left\{\begin{array}{cc}(1-\epsilon )\phi \left(x\right),\hfill & \left|x\right|\le k,\hfill \\ Cexp(-D|x\left|\right),\hfill & \left|x\right|>k,\hfill \end{array}\right.$$

(8)

$${\psi }^{*}\left(x\right)={\psi }_{Huber}\left(x\right)=\left\{\begin{array}{cc}x,\hfill & \left|x\right|\le k,\hfill \\ CD\mathrm{sgn}\left(x\right),\hfill & \left|x\right|>k,\hfill \end{array}\right.$$

(9)

where the parameters C, D, and k satisfy the conditions of norming, continuity, and continuous differentiability of the solution at $x=k$:

$$\int p^{*}\left(x\right)dx=1,p^{*}(k-0)=p^{*}(k+0),{p^{*}}^{\prime }(k-0)={p^{*}}^{\prime }(k+0).$$

Finally, we get the linear bounded score ${\psi }_{Huber}\left(x\right)=max\{-k,min\{x,k\}\}$, where k depends on the value of the contamination parameter $\epsilon$, as follows:

$$\frac{2\phi \left(k\right)}{k}-2\mathsf{\Phi }(-k)=\frac{\epsilon }{1-\epsilon },\mathsf{\Phi }\left(x\right)={\int }_{-\infty }^x\phi \left(t\right)dt;$$

the values of the parameter $k=k\left(\epsilon \right)$ are tabulated in Reference [5].

The particular cases of this solution for $\epsilon =0$ and $\epsilon \to 1$ are given by $k\to \infty$ (the sample mean) and $k=0$ (the sample median), respectively.

The Huber’s score function ${\psi }_{Huber}\left(x\right)$ is a robust version of the ML estimation: in the center $|x_i-\theta |\le k$, the data are processed by the ML method, and they are trimmed within the exponential distribution tails $|x_i-\theta |>k$. In the limiting case of a completely unknown density as $\epsilon \to 1$, the minimax variance M-estimate of location tends to the sample median.

2.2. Free Extremals of the Basic Variational Problem

Consider the problem of minimization of Fisher information for location (5) under two basic side conditions of non-negativeness and norming: $p\left(x\right)\ge 0$, ${\int }_{-\infty }^{\infty }p\left(x\right)dx=1$.

Set $\sqrt{p\left(x\right)}=q\left(x\right)$, and rewrite this minimization problem as

$$\mathrm{minimize}I\left(p\right)=4{\int }_{-\infty }^{\infty }{\left(q^{\prime }\left(x\right)\right)}^{2}dx\mathrm{under}{\int }_{-\infty }^{\infty }{q}^2\left(x\right)dx=1.$$

Introducing the Lagrange multiplier $\lambda$ related to the norming condition, we obtain the following differential equation for the function $q\left(x\right)$: $4q^{\prime \prime }\left(x\right)+\lambda q\left(x\right)=0$.

The general solutions of this harmonic oscillator equation have the well-known forms depending on the sign of $\lambda =4k^2$: (i) the exponential $q\left(x\right)=C_1{e}^{kx}+C_2{e}^{-kx}$, the cosine $q\left(x\right)=C_{1}sinkx+C_{2}coskx$, and the linear $q\left(x\right)=C_1+C_{2}x$, where $k=\pm \sqrt{\lambda }/2$.

Further, we show that all these forms work in the structures of least informative distribution densities.

2.3. Least Informative Distributions

The neighborhoods of normal, generally, are not the only models of interest. In real-life applications, the information about the distribution central part tails, its moments, and/or subranges is rather often available. The empirical distribution and quantile functions, histograms, and kernel estimates, together with their tolerance limits, provide other examples. To enhance the efficiency of robust estimates, this information can be used in minimax settings.

Further, we deal with symmetric distribution densities $p(-x)=p\left(x\right)$. Evidently, distribution densities must also satisfy the non-negativeness and norming conditions common for all classes. For brevity, we do not write out all these conditions any time we define a distribution density class.

Now, we enlist several examples of distribution classes convenient for the description of a prior knowledge about data distributions [7].

(1)

The class of non-degenerate densities [15]: ${\mathcal{P}}_1=\left\{p:p\left(0\right)\ge 1/\left(2a\right)>0\right\}$. The parameter a of this class characterizes the dispersion of the central part of a distribution.

The least informative density in this class is given by the double-exponential or Laplace density [16]: $p_1^{*}\left(x\right)=L(x;0,a)={2a}^{-1}exp\left(-\left|x\right|/a\right)$. This result is quite natural if one recalls the exponential form of free extremals for the basic variational problem.

(2)

The class ${\mathcal{P}}_2$ of distribution densities with a bounded variance:

$${\mathcal{P}}_2=\left\{p:{\sigma }^2\left(p\right)={\int }_{-\infty }^{\infty }{x}^{2}p\left(x\right)dx\le {\overline{\sigma }}^2\right\}.$$

All distribution densities with bounded variances are the members of this class. Evidently, the Cauchy-type distributions do not belong to it.

The least informative density in this class is normal [17]:

$$p_2^{*}\left(x\right)=N(x;0,\overline{\sigma })=\frac{1}{\sqrt{2\pi }\overline{\sigma }}exp\left(-\frac{x^2}{2{\overline{\sigma }}^2}\right).$$

(3)

The class ${\mathcal{P}}_3$ of approximately normal distribution densities is defined by Equation (7): $\{p:p(x)\ge (1-\epsilon \left)\phi \right(x),0\le \epsilon <1\}$.

(4)

The class of finite distributions: ${\mathcal{P}}_4=\left\{p:{\int }_{-l}^{l}p\left(x\right)dx=1\right\}$. This class defines the boundaries of the data (i.e., the inequality $\left|X\right|\le l$ holds with probability one), and there is no more information about this distribution besides the boundary conditions of smoothness: $p(\pm l)=p^{\prime }(\pm l)=0$.

The least informative density in this class has the cosine-squared form [15]:

$$p_4^{*}\left(x\right)=\left\{\begin{array}{ccc}{\displaystyle \frac{1}{l}{cos}^2\left(\frac{\pi x}{2l}\right)}\hfill & \mathrm{for}\hfill & \left|x\right|\le l,\hfill \\ 0\hfill & \mathrm{for}\hfill & \left|x\right|>l.\hfill \end{array}\right.$$

(5)

The class of distributions with a bounded interquantile distribution mass:

$${\mathcal{P}}_5=\left\{p:{\int }_{-l}^{l}p\left(x\right)dx\ge 1-\beta \right\},0\le \beta <1.$$

The parameters l and $\beta$ are assumed given. The restriction upon the interquantile mass means that $P\left(\right|X|\le l)\ge 1-\beta$. We can redefine this class in a different way as the class with an upper-bounded interquantile range ${\displaystyle IQ{R}_{\beta }=P^{-1}\left(\frac{1+\beta }{2}\right)-P^{-1}\left(\frac{1-\beta }{2}\right)\le 2l}$, where $P\left(x\right)$ is a probability density function.

The least informative density in this class has both the cosine and exponential parts working at the center and tail areas, respectively [7],

$$p_5^{*}\left(x\right)=\left\{\begin{array}{ccc}{\displaystyle A_1{cos}^2\left(B_{1}x\right)}\hfill & \mathrm{for}\hfill & \left|x\right|\le l,\hfill \\ A_{2}exp(-B_2\left|x\right|)\hfill & \mathrm{for}\hfill & \left|x\right|>l,\hfill \end{array}\right.$$

where the constants $A_1$, $A_2$, $B_1$, and $B_2$ are determined from the simultaneous equations (restrictions) of the class ${\mathcal{P}}_5$, namely the conditions of normalization and upon the distribution interquantile range, and the conditions of continuity and continuous differentiability at $x=l$ (for details, see Reference [7]). It is worth noting that the classes ${\mathcal{P}}_1$ and ${\mathcal{P}}_4$ are the particular cases of the class ${\mathcal{P}}_5$ when $\beta \to 0$ and $\beta =1$, respectively.

3. Hampel’s Robust and Shurygin’s Stable Estimates of Location

Robust methods have lower sensitivity to possible departures from the accepted distribution models as compared to conventional statistical methods. To analyze the sensitivity of estimation, it is natural to have its specific indicator. In what follows, we introduce these indicators, namely the influence function and related to it measures.

3.1. Hampel’s Local Approach to Robustness

Let P be a distribution function corresponding to $p\in \mathcal{P}$, the class of distribution densities, and let $T\left(P\right)$ be a functional defined in a subset of all distribution functions. The natural estimate defined by T is $T_n=T\left(P_n\right)$, i.e., the functional computed at the sample distribution function $P_n$.

The influence function $IF(x;T,p)$ of this functional is defined as

$$IF(x;T,p)=\underset{t\to 0}{lim}\frac{T((1-t)P+t{\Delta }_x)-T\left(P\right)}{t},$$

(10)

where ${\Delta }_x$ is the degenerate distribution taking mass 1 at x [6].

The influence function measures the impact of an infinitesimal contamination at x on the value of an estimate, formally being the G$\widehat{\mathrm{a}}$teaux derivative of the functional $T\left(P\right)$. For an M-estimate with a score function $\psi$, the influence function is proportional to it: $IF(x;\psi ,p)=\psi \left(x\right)/B(\psi ,p)$, where the term $B(\psi ,p)$ stands in Equation (3) for the asymptotic variance of M-estimates.

Based on the influence function, several local measures of robustness are defined [6], including the gross-error sensitivity of T at p:

$${\gamma }^{*}(T,p)=\underset{x}{sup}\left|IF(x;T,p)\right|.$$

This indicator of sensitivity gives an upper bound upon the asymptotic estimate bias and measures the worst influence of an infinitesimal contamination on the value of an estimate. Maximizing the efficiency of an M-estimate of location under the condition of a bounded gross-error sensitivity at the normal distribution

$$\underset{\psi }{max}eff(\psi ,\phi )\mathrm{under}{\gamma }^{*}\le \overline{\gamma },$$

where ${\displaystyle eff(\psi ,p)=\frac{1}{AV(\psi ,p)I\left(p\right)}}$ formally leads to the Huber’s minimax linear bounded score ${\psi }_{Huber}\left(x\right)=max\{-k,min\{x,k\}\}$ in the class of contaminated normal distributions [6]. This particular result confirms the following general observation: the best estimates within both approaches, Huber’s minimax and Hampel’s local, are rather close in their performances.

3.2. Meshalkin-Shurygin’s Stable Estimates of Location

This topic is partially reversal to the conventional setting: the maximum of some measure of sensitivity is minimized under the guaranteed value of the estimate variance or efficiency.

The conventional point-wise local measures of sensitivity, such as the influence and change-of-variance functions [6] are not appropriate here—a global indicator of sensitivity is desirable. We show that a novel global indicator of robustness proposed by Shurygin [18], the estimate stability, is closely related to the classical variation of the functional of the estimate asymptotic variance. Although the related theory has been developed for stable estimation of an arbitrary parameter of the underlying distribution, here, we focus on stable estimation of location.

A measure of the M-estimate sensitivity called variance sensitivity ($VS$) is introduced in Reference [19]. Formally, it is defined as the Lagrange functional derivative of the asymptotic variance (3):

$$VS(\psi ,p)=\frac{\partial AV(\psi ,p)}{\partial p}=\frac{\partial }{\partial p}\left(\frac{\int {\psi }^2\left(x\right)p\left(x\right)dx}{{\left(\int {\psi }^{\prime }\left(x\right)p\left(x\right)dx\right)}^2}\right)=\frac{\int {\psi }^2\left(x\right)dx}{{\left(\int {\psi }^{\prime }\left(x\right)p\left(x\right)dx\right)}^2}.$$

(11)

Equation (11) gives a global measure of the stability of an M-estimate in a model, where the outliers occur uniformly anywhere on the real line. The boundness of the Lagrange derivative (11) holds under the condition of square integrability of $\psi$ with the corresponding redescending scores when $\psi \left(x\right)\to 0$ for $\left|x\right|\to \infty$.

In Reference [7], it is shown that the principal part of the variation $\delta AV(\psi ,p)$ of the asymptotic variance $AV(\psi ,p)$ with respect to $\left|\right|\delta p\left|\right|$ is proportional to the variance sensitivity (11) or to the Lagrange derivative of the asymptotic variance: $\delta AV(\psi ,p)\propto \partial AV(\psi ,p)/\partial p.$

Further, consider the following optimization problem: what is the minimum variance sensitive score function for a given distribution density p? The solution of this optimization problem is given by the minimum variance sensitive (MVS) score function:

$${\psi }_{MVS}\left(x\right)=arg\underset{\psi }{min}VS(\psi ,p)=-p^{\prime }\left(x\right).$$

(12)

The estimate with this optimal score function (12) is called as the estimate of minimum variance sensitivity with ${VS}_{min}=VS({\psi }_{MVS},p)$. We define a global measure of the stability of any M-estimate comparing an estimate variance sensitivity with its minimum

$$0\le stb(\psi ,p)={VS}_{min}\left(p\right)/VS(\psi ,p)\le 1.$$

A number of optimization criteria settings with different weights for efficiency and stability of M-estimates of location have been proposed and solved; in other examples, efficiency is maximized under guaranteed stability, and vice versa [18]. Practically in all these cases, we deal with the score functions ${\psi }_{opt}\left(x\right)$ with the following limiting forms: the maximum likelihood case ${\psi }_{opt}\left(x\right)\to {\psi }_{ML}\left(x\right)=-p^{\prime }\left(x\right)/p\left(x\right)$ when the requirement of high efficiency mostly matters and the opposite redescending case ${\psi }_{opt}\left(x\right)\to {\psi }_{MVS}\left(x\right)=-p^{\prime }\left(x\right)$ when the requirement of high stability is important.

A compromise case is given by a stable estimate called radical with equal efficiency and stability, $eff(\psi ,p)=stb(\psi ,p)$, desirably both highly efficient and stable: its score function is given by

$${\psi }_{rad}\left(x\right)={\psi }_{ML}\left(x\right)\sqrt{p\left(x\right)}=-p^{\prime }\left(x\right)/\sqrt{p\left(x\right)}.$$

(13)

Finally, it should be noted that the minimum sensitivity and radical estimates belong to the class of M-estimates with the exponentially weighted maximum likelihood score functions previously proposed by Meshalkin [20].

4. Asymptotic Efficiency of $\mathbf{M}$-Estimates: A Comparative Study

Here, we compare the asymptotic efficiency performance of various robust and stable estimates of location at the conventional in robustness studies distributions: some particular results can be found in Reference [19,21]. We mainly focus on the minimax variance estimates for distributions with bounded interquantile ranges, as until present, their performance has not been thoroughly studied. It is important that some obtained results are unexpected and surprising.

4.1. Robust and Stable M-Estimates of a Location Parameter

We test the following M-estimates of location: (i) the sample mean and median, (ii) the Huber’s minimax variance M-estimate with the linear bounded score ${\psi }_{Huber}\left(x\right)=max[-1.14,min(x,1.14)]$ optimal for $\epsilon$-contaminated standard normal distributions with the parameter of contamination $\epsilon =0.1$, (iii) the Hampel’s M-estimate with the redescending three-part score

$${\psi }_{Hampel}\left(x\right)=\left\{\begin{array}{cc}x\hfill & \mathrm{for}0\le \left|x\right|\le a,\hfill \\ a\mathrm{sgn}\left(x\right)\hfill & \mathrm{for}a\le \left|x\right|\le b,\hfill \\ {\displaystyle a\frac{r-\left|x\right|}{r-b}\mathrm{sgn}\left(x\right)}\hfill & \mathrm{for}a\le \left|x\right|\le b,\hfill \\ 0\hfill & \mathrm{for}r\le \left|x\right|,\hfill \end{array}\right.$$

where the parameters $a=1.31$, $b=2.039$, and $r=4$ (see Reference [6], pp. 166–167), (iv) the minimax variance M-estimates with the scores ${\psi }_{\beta }\left(x\right)$ for various values of the parameter $\beta :0.01,0.05,0.1,0.5,0.9,0.95,0.99$, and (v) the stable Shurygin’s minimum variance sensitivity and radical M-estimates with the scores ${\psi }_{MVS}\left(x\right)=-p^{\prime }\left(x\right)$ and ${\psi }_{rad}\left(x\right)=-p^{\prime }\left(x\right)/\sqrt{p\left(x\right)}$, respectively.

4.2. Data Distributions

In our study, the following distribution densities are used:

(i) the standard normal $p\left(x\right)=N(x;0,1)={\left(2\pi \right)}^{-1/2}exp(-x^2/2)$,

(ii) the standard Laplace $p\left(x\right)=L(x;0,1/\sqrt{2})=2^{-1/2}exp(-\sqrt{2}\left|x\right|)$,

(iii) the standard Cauchy $p\left(x\right)=C(x;0,1)={\pi }^{-1}{(1+x^2)}^{-1}$, and

(iv) the heavy-tailed Tukey gross-error model as the Cauchy contaminated normal density $p\left(x\right)=0.9N(x;0,1)+0.1C(x;0,1)$.

4.3. Asymptotic Efficiency

The asymptotic efficiency of M-estimates is numerically computed as follows:

$$eff\left(T_n\right)=\frac{Var\left(T_{ML}\right)}{Var\left(T_n\right)}=\frac{{\left(\int {\psi }^{\prime }\left(x\right)p\left(x\right)dx\right)}^2}{I\left(p\right)\int {\psi }^2\left(x\right)p\left(x\right)dx}.$$

5. Conclusions

From

Table 1. Asymptotic efficiency (two best values are boldfaced).

M-Estimate	Normal	Laplace	Cauchy	Tukey Gross Error
mean	1	0.500	0	0
median	0.636	1	0.811	0.678
Huber’s linear bounded	0.924	0.669	0.759	0.953
Hampel’s three-part	0.911	0.644	0.869	0.946
${\psi }_{0.01}$	0.895	0.310	0.250	0.747
${\psi }_{0.1}$	0.976	0.503	0.484	0.948
${\psi }_{0.5}$	0.844	0.799	0.765	0.962
${\psi }_{0.9}$	0.679	0.965	0.898	0.825
${\psi }_{0.99}$	0.641	0.995	0.859	0.727
MVS	0.650	0.750	0.800	0.710
radical	0.840	0.890	0.920	0.890

, it follows:

(1)

As usual, the performance of the sample mean under heavy-tailed Cauchy and contaminated normal distributions is awful. Designed for these models, Huber’s and Hampel’s M-estimates perform well except the Laplace distribution case. This distribution with moderately heavy tails against a sharp peak at the center is a rather tough test for the asymptotic performance of M-estimates of location, especially as compared to the Cauchy distribution case. Recall that the Laplace and distributions close to it are the least informative ones in wide classes of distributions, for instance, in the class ${\mathcal{P}}_{\beta }$ with the parameter $\beta$ close to unit (the corresponding minimax variance M-estimates perform quite well in these cases). For a statistical user, the version with a bounded IQR (interquartile range, $\beta =1/2$) seems a reasonable choice.

(2)

Surprisingly, the proposed minimax variance M-estimates with the scores ${\psi }_{\beta }$ outperform the classical robust Huber’s and Hampel’s M-estimates at the normal, although the shape of the least informative distribution is not at all normal: note that the Taylor expansion of the $cosin{e}^2$-bell shape is close to the exponential one with small values of $\beta$. Moreover, these M-estimates are better than the classical robust estimates in heavy-tailed distribution models. We explain this effect by the nature of the distribution class ${\mathcal{P}}_{\beta }$—it is one of the most wide possible distribution classes. Finally, these M-estimates and their statistical properties can be obtained in a closed analytical form.

(3)

The globally stable Meshalkin-Shurygin’s radical M-estimate also outperforms the classical robust Huber’s and Hampel’s M-estimates at the heavy-tailed Laplace and Cauchy distributions.

Finally, we outline the prospective future works: (i) an extension of the proposed minimax variance M-estimates to the multivariate case and the classes with simultaneously bounded subranges of different parameter $\beta$ values—in the latter case, we may expect a uniformly better performance; (ii) a generalization of the Meshalkin-Shurygin’s stable estimates to the multivariate case; and (iii) a thorough comparative study of the small sample performance of M-estimates of location—Monte Carlo experiments show a slightly better performance of the classical Huber’s and Hampel’s estimates in this case.