Higher Order Bias Correcting Moment Equation for M-Estimation and Its Higher Order Efficiency

Abstract

This paper studies an alternative bias correction for the M-estimator, which is obtained by correcting the moment equations in the spirit of Firth (1993). In particular, this paper compares the stochastic expansions of the analytically-bias-corrected estimator and the alternative estimator and finds that the third-order stochastic expansions of these two estimators are identical. This implies that at least in terms of the third-order stochastic expansion, we cannot improve on the simple one-step bias correction by using the bias correction of moment equations. This finding suggests that the comparison between the one-step bias correction and the method of correcting the moment equations or the fully-iterated bias correction should be based on the stochastic expansions higher than the third order.

1. Introduction

Asymptotic bias corrections are pursued to make estimators closer to the truth values. There are several ways of achieving this goal, including analytical corrections, jackknife and bootstrap methods (see, e.g., Quenouille (1956) [1], Hall (1992) [2], Shao and Tu (1995) [3], MacKinnon and Smith (1998) [4], Andrews (2002) [5], Hahn and Newey (2004) [6], Bun and Carree (2005) [7], Bao and Ullah (2007) [8,9], Bao (2013) [10] and Yang (2015) [11]). This variety of bias correction methods evokes the issue whether one method is preferable to others at least on asymptotic efficiency grounds (e.g., see Hahn et al. (2004) [12]). For the maximum likelihood (ML) estimation, they show that a method of bias correction does not affect the higher order efficiency of any estimator that is first-order efficient in parametric or semiparametric models. An ML estimator is a class of M-estimators, and this paper extends their intuition to a general class of M-estimators.1

Specifically, this paper considers an alternative bias correction for the M-estimator, which is achieved by correcting moment equations in the spirit of Firth (1993) [13]. In particular, we compare the stochastic expansions of the analytically-bias-corrected estimator (which is referred to one-step bias correction) and the alternative estimator and find that the third-order stochastic expansions of these two estimators are identical. This is a stronger result than comparing higher order variances, since it implies that these two estimators do not only have the same higher order variances, but would also agree upon more properties in terms of their stochastic expansions.2 We do not consider other bias correction methods, such as bootstrap and jackknife methods, in this paper.

In the literature (see Hahn and Newey (2004) [6] and Fernandez-Val (2004) [14] for nonlinear panel data models), it has been discussed that removing the bias directly from the moment equations has an attractive feature that it does not use pre-estimated parameters that are not bias corrected, though this alternative approach requires more intensive computations.3 Because the analytically-bias-corrected estimator is a two-step estimator, for which an initial estimator needs to be plugged in, while the bias-corrected moment equations estimator is a one-step estimator that does not need an initial estimator, the higher order asymptotic equivalence of these two estimators is not obvious. This paper, however, shows that at least for the third-order stochastic expansion, there is no benefit of using the bias correction of the moment equations over the simple one-step bias correction in the context of M-estimators. This finding suggests that the comparison between the one-step bias correction and the method of correcting the moment equations should be based on the stochastic expansions higher than the third order.

Examples of the M-estimation include maximum likelihood estimation (MLE), least squares and instrumental variable (IV) estimation. Many other useful estimators can also fit into the M-estimation framework with the appropriate definition of the moment equations. It includes some cases of the generalized method of moments (GMM; see examples in Rilstone et al. (1996) [15]) and two-step estimators (Newey (1984) [16]). We note that the generalized empirical likelihood (GEL) can also fit into this framework. This suggests that Firth (1993)’s [13] correcting moment equations approach can be an alternative to Newey and Smith’s approach to obtain the higher order bias and variance terms of GEL (2004) [17].

Our paper is organized as follows. In Section 2, we derive the higher order stochastic expansion of the M-estimator and consider the one-step bias correction. Section 3 introduces the bias-corrected moment equations estimator and derives its higher order stochastic expansion. Section 4 discusses the higher order efficiency properties of several analytically-bias-corrected estimators. We conclude in Section 5. Primitive conditions for the validity of the higher order stochastic expansions and mathematical details are gathered in Appendix A and Appendix B.

2. Higher Order Expansion for the M-Estimator

Consider a moment condition:

$$E\left[s\left(z_i,{\theta }_0\right)\right]=0$$

(1)

where $s(z_i,\theta )$ is a known $k\times 1$ vector-valued function of the data, and a parameter vector $\theta \in \mathrm{\Theta }\subset R^k$ and $z_i$ includes both endogenous and exogenous variables. The M-estimator is obtained by solving:

$$\frac{1}{n}\sum _{i=1}^{n}s\left(z_i,\widehat{\theta }\right)=0.$$

(2)

Examples for this class of estimators include MLE, least squares and IV estimation. In the MLE, $s(z_i,\theta )$ is the single observation score function. For the linear or nonlinear regression model of $y_i=f(X_i;{\theta }_0)+{\epsilon }_i$, we set $s\left(z_i,\theta \right)=\frac{\partial f(X_i;\theta )}{\partial \theta }\left(y_i-f(X_i;\theta )\right)$ and $z_i=(y_i$ $X_i^{\prime }{)}^{\prime }$ for a known function $f(·)$. In the linear IV model, we have $s\left(z_i,\theta \right)=w_i(y_i-X_i^{\prime }\theta )$ and $z_i=(y_i$ $X_i^{\prime }$ $w_i^{\prime }{)}^{\prime }$ for some instruments $w_i$ with $dim(w_i)=dim(\theta )$. Two-step estimators such as two-stage least squares, feasible generalized least squares (GLS) and Heckman (1979) [18]’s two-step estimator also fit into this framework (see Newey (1984) [16]). Rilstone et al. (1996) [15] provide some special cases of GMM estimators that can be put into the M-estimation, but the examples are not restricted to those. Partly motivated with this wide applicability, we study the stochastic expansion and the bias correction of the M-estimator.

We obtain the higher order stochastic expansion of the M-estimator using the iterative approach used in Rilstone et al. (1996) [15] up to a certain order. This approach is analytically convenient and straightforward to implement since the estimators are expressed as functions of the sums of random variables. Edgeworth expansion can be considered as an alternative whose validity has been derived in Bhattacharya and Ghosh (1978) [19], but the stochastic expansion approach is noted as a much simpler approach. Moreover, the main purpose of this paper is to provide the comparison of several estimators based on the higher order variance ($O(n^{-1})$ variance). Noting that rankings based on the higher order variances in a third-order stochastic expansion are equivalent to rankings based on the variances of an Edgeworth expansion as shown in Pfanzagl and Wefelmeyer (1978) [20] and Ghosh et al. (1980) [21] and as discussed in Rothenberg (1984) [22], it suffices to use the simple stochastic expansions for our purposes.

Here, we borrow Rilstone et al. (1996) [15]’s notation. We denote the matrix of υ-th order partial derivatives of a matrix $A(\theta )$ as ${\nabla }^{\upsilon }A(\theta )$. Specifically, if $A(\theta )$ is a $k\times 1$ vector function, $\nabla A(\theta )$ is the usual Jacobian whose l-th row contains the partial derivatives of the l-th element of $A(\theta )$. ${\nabla }^{\upsilon }A(\theta )$ (a $k\times k^{\upsilon }$ matrix) is defined recursively, such that the j-th element of the l-th row of ${\nabla }^{\upsilon }A(\theta )$ is the $1\times k$ vector $a_{lj}^{\upsilon }(\theta )=\partial a_{lj}^{\upsilon -1}(\theta )/\partial {\theta }^{\prime }$, where $a_{lj}^{\upsilon -1}$ is the l-th row and the j-th element of ${\nabla }^{\upsilon -1}A(\theta )$. We use ⊗ to denote a usual Kronecker product. Using this Kronecker product, we can express ${\nabla }^{\upsilon }A(\theta )=\frac{{\partial }^{\upsilon }A(\theta )}{\underset{\upsilon \mathrm{Kronecker}\mathrm{product}\mathrm{of}\partial {\theta }^{\prime }}{\underbrace{\partial {\theta }^{\prime }\otimes \partial {\theta }^{\prime }\otimes \dots \otimes \partial {\theta }^{\prime }}}}$. Finally, we use a matrix norm $∥A∥=\sqrt{tr(A^{\prime }A)}$ for a matrix A.

We first derive the higher order stochastic expansion of the M-estimator and consider the one-step bias correction here. In the next section, we introduce the bias-corrected moment equations estimator and derive its higher order stochastic expansion. Then, we compare these two approaches.

Before we derive the second-order expansion of the M-estimator to obtain the second-order bias analytically, we introduce simplifying notation. Let $H_1(\theta )=E\left[\nabla s(z_i,\theta )\right]$, $H_2(\theta )=E\left[{\nabla }^{2}s(z_i,\theta )\right]$, $Q(\theta )={\left(-E\left[\nabla s(z_i,\theta )\right]\right)}^{-1}$, and write $H_1=H_1({\theta }_0)$, $H_2=H_2({\theta }_0)$, $Q=Q({\theta }_0)$. Let ${\widehat{H}}_1(\theta )=\frac{1}{n}{\sum }_{i=1}^n\nabla s\left(z_i,\theta \right),$ ${\widehat{H}}_2(\theta )=\frac{1}{n}{\sum }_{i=1}^n{\nabla }^{2}s\left(z_i,\theta \right),$ $\widehat{Q}(\theta )={(-{\widehat{H}}_1(\theta ))}^{-1}$, ${\widehat{H}}_1={\widehat{H}}_1({\theta }_0)$, ${\widehat{H}}_2={\widehat{H}}_2({\theta }_0)$ and $\widehat{Q}=\widehat{Q}({\theta }_0)$. Furthermore, define $J\equiv \frac{1}{\sqrt{n}}{\sum }_{i=1}^{n}s\left(z_i,{\theta }_0\right)$, $V\equiv \frac{1}{\sqrt{n}}{\sum }_{i=1}^n\left(\nabla s\left(z_i,{\theta }_0\right)-E\left[\nabla s\left(z_i,{\theta }_0\right)\right]\right)$, $W\equiv \frac{1}{\sqrt{n}}{\sum }_{i=1}^n\left({\nabla }^{2}s\left(z_i,{\theta }_0\right)-E\left[{\nabla }^{2}s\left(z_i,{\theta }_0\right)\right]\right)$.

Lemma 1.

(Rilstone et al. (1996) [15]) Suppose ${\{z_i\}}_{i=1}^n$ are i.i.d.; ${\theta }_0$ is in the interior of Θ, and is the only $\theta \in \mathrm{\Theta }$ satisfying (1); and the M-estimator $\widehat{\theta }$ defined in (2) is consistent. Further suppose that: (i) $s(z,\theta )$ is κ-times continuously differentiable in the neighborhood of ${\theta }_0$, denoted by ${\mathrm{\Theta }}_0\subset \mathrm{\Theta }$ for all $z\in \mathcal{Z}\equiv Support(z_i)$, $\kappa \ge 3$ with probability one; (iia) ${\nabla }^{\upsilon }s(z,\theta )$ is integrable for each fixed $\theta \in$ ${\mathrm{\Theta }}_0$, $\upsilon =\{0,1,2,\dots \kappa \}$, $\kappa \ge 3$; and (iib) $E\left[{\nabla }^{3}s(z,\theta )\right]$ is continuous and bounded at ${\theta }_0$; (iii) $∥\frac{1}{n}{\sum }_{i=1}^n{\nabla }^{\upsilon }s\left(z_i,\overline{\theta }\right)-E\left[{\nabla }^{\upsilon }s\left(z_i,{\theta }_0\right)\right]∥=o_p(1)$ for $\overline{\theta }={\theta }_0+o_p(1)$ and $\upsilon =1,2;$ (iv) $\frac{1}{\sqrt{n}}{\sum }_{i=1}^n\left({\nabla }^{2}s\left(z_i,\overline{\theta }\right)-H_2(\overline{\theta })\right)-\frac{1}{\sqrt{n}}{\sum }_{i=1}^n\left({\nabla }^{2}s\left(z_i,{\theta }_0\right)-H_2\left({\theta }_0\right)\right)=o_p\left(1\right)$ for $\overline{\theta }={\theta }_0+o_p(1);$ (v) $Q({\theta }_0)$ exists, i.e., $E\left[\nabla s(z_i,{\theta }_0)\right]$ is nonsingular; (vi) $J=O_p(1);$ (vii) $V=O_p(1);$ (viii) $W=O_p(1)$. Then, we have $\sqrt{n}\left(\widehat{\theta }-{\theta }_0\right)=QJ+O_p\left(\frac{1}{\sqrt{n}}\right)$, and moreover, $\sqrt{n}\left(\widehat{\theta }-{\theta }_0\right)=QJ+\frac{1}{\sqrt{n}}Q\left(VQJ+\frac{1}{2}{H}_2\left(QJ\otimes QJ\right)\right)+O_p(n^{-1})$.

This lemma and the following Lemma 2 are available in Rilstone et al. (1996) [15], but we reproduce them since some of their results are useful to derive our new results. From Lemma 1, the higher order bias of $\widehat{\theta }$ is obtained as:

$$Bias(\widehat{\theta })\equiv \frac{1}{n}Q\left(E\left[VQJ\right]+\frac{1}{2}{H}_{2}E\left[\left(QJ\otimes QJ\right)\right]\right).$$

Defining $d_i(\theta )=Q(\theta )s(z_i,\theta )$ and $v_i(\theta )=\nabla s\left(z_i,\theta \right)-E\left[\nabla s\left(z_i,\theta \right)\right]$ and letting $d_i=d_i({\theta }_0)$ and $v_i=v_i({\theta }_0)$, it is not difficult to see $Q\left(E\left[VQJ\right]+\frac{1}{2}{H}_{2}E\left[\left(QJ\otimes QJ\right)\right]\right)=Q\left(E\left[v_i{d}_i\right]+\frac{1}{2}{H}_{2}E\left[\left(d_i\otimes d_i\right)\right]\right)$, as shown below. In this regard, we will write $B(\theta )\equiv Q(\theta )\left(E\left[v_i(\theta )d_i(\theta )\right]+\frac{1}{2}{H}_2(\theta )E\left[d_i(\theta )\otimes d_i(\theta )\right]\right)$.

Lemma 2.

(Rilstone et al. (1996) [15]) Suppose (1) holds and ${\left\{z_i\right\}}_{i=1}^n$ are i.i.d. Then, $E\left[VQJ\right]+\frac{1}{2}{H}_{2}E\left[QJ\otimes QJ\right]=E\left[v_i{d}_i\right]+\frac{1}{2}{H}_{2}E\left[d_i\otimes d_i\right],$ where $d_i=Qs(z_i,{\theta }_0)$ and $v_i=\nabla s\left(z_i,{\theta }_0\right)-E\left[\nabla s\left(z_i,{\theta }_0\right)\right]$.

Thus, we can eliminate the second-order bias of the M-estimator $\widehat{\theta }$ by subtracting a consistent estimator of the bias.4 Now, let ${\widehat{\theta }}_{bc}$ denote the bias-corrected estimator of this sort defined by:

$${\widehat{\theta }}_{bc}=\widehat{\theta }-\frac{1}{n}\widehat{B}(\widehat{\theta })$$

(3)

where the function $\widehat{B}(\theta )$, a consistent estimator of $B(\theta )$, is constructed as:

$$\widehat{Q}(\theta )\left(\frac{1}{n}\sum _{i=1}^n{\widehat{v}}_i(\theta ){\widehat{d}}_i(\theta )+\frac{1}{2}{\widehat{H}}_2(\theta )\frac{1}{n}\sum _{i=1}^n\left({\widehat{d}}_i(\theta )\otimes {\widehat{d}}_i(\theta )\right)\right)$$

(4)

for ${\widehat{d}}_i(\theta )=\widehat{Q}(\theta )s(z_i,\theta )$ and ${\widehat{v}}_i\left(\theta \right)=\nabla s\left(z_i,\theta \right)$. In particular, we can replace $\widehat{\theta }$ in $\widehat{B}(\widehat{\theta })$ with any $\sqrt{n}$-consistent estimator of ${\theta }_0$. In this sense, ${\widehat{\theta }}_{bc}$ is a two-step estimator.

To characterize the higher order efficiency based on the higher order variance ($O(n^{-1})$ variance) of the bias-corrected estimators, we need to expand the M-estimator to the third order. We use some additional simplifying terms: $H_3\left(\theta \right)=E[{\nabla }^{3}s(z,\theta )],$ ${\widehat{H}}_3\left(\theta \right)=\frac{1}{n}{\displaystyle \sum _{i=1}^n}{\nabla }^{3}s(z_i,\theta ),$ $H_3=H_3\left({\theta }_0\right)$, $W_3\equiv \frac{1}{\sqrt{n}}{\displaystyle \sum _{i=1}^n}\left({\nabla }^{3}s\left(z_i,{\theta }_0\right)-E\left[{\nabla }^{3}s\left(z_i,{\theta }_0\right)\right]\right)$. Furthermore, we write:

$$\begin{array}{cc}\hfill a_{-1/2}& =QJ,a_{-1}=Q\left(V{a}_{-1/2}+\frac{1}{2}{H}_2\left(a_{-1/2}\otimes a_{-1/2}\right)\right)\hfill \\ \hfill a_{-3/2}& =QV{a}_{-1}+\frac{1}{2}QW\left(a_{-1/2}\otimes a_{-1/2}\right)+\frac{1}{2}Q{H}_2\left(a_{-1/2}\otimes a_{-1}+a_{-1}\otimes a_{-1/2}\right)\hfill \\ & +\frac{1}{6}Q{H}_3\left(a_{-1/2}\otimes a_{-1/2}\otimes a_{-1/2}\right)\hfill \end{array}$$

for the ease of notation. We obtain:

Lemma 3.

Suppose ${\{z_i\}}_{i=1}^n$ are i.i.d., ${\theta }_0$ is in the interior of Θ, is the only $\theta \in \mathrm{\Theta }$ satisfying (1) and the M-estimator $\widehat{\theta }$ that solves (2) is consistent. Further suppose that: (i) $s(z,\theta )$ is κ-times continuously differentiable in a neighborhood of ${\theta }_0$, denoted by ${\mathrm{\Theta }}_0\subset \mathrm{\Theta }$ for all $z\in \mathcal{Z}$, $\kappa \ge 4$ with probability one; (iia) ${\nabla }^{\upsilon }s(z,\theta )$ is integrable for each fixed $\theta \in {\mathrm{\Theta }}_0$, $\upsilon =\{0,1,2,\dots \kappa \}$, $\kappa \ge 4$; (iib) $E[{\nabla }^{4}s(z,\theta )]$ is continuous and bounded at ${\theta }_0$; (iii) $\frac{1}{\sqrt{n}}{\sum }_{i=1}^n\left({\nabla }^{3}s\left(z_i,\overline{\theta }\right)-H_3\left(\overline{\theta }\right)\right)-\frac{1}{\sqrt{n}}{\sum }_{i=1}^n\left({\nabla }^{3}s\left(z_i,{\theta }_0\right)-H_3\left({\theta }_0\right)\right)=o_p\left(1\right)$ for $\overline{\theta }={\theta }_0+o_p(1);$ (iv) Q is nonsingular; (v) $J=O_p(1)$; (vi) $V=O_p(1)$; (vii) $W=O_p(1)$; (viii) $W_3=O_p(1);$ (ix) $\sqrt{n}(\widehat{\theta }-{\theta }_0)=a_{-1/2}+\frac{1}{\sqrt{n}}{a}_{-1}+O_p\left(\frac{1}{n}\right)$. Then, we have $\sqrt{n}\left(\widehat{\theta }-{\theta }_0\right)=a_{-1/2}+\frac{1}{\sqrt{n}}{a}_{-1}+\frac{1}{n}{a}_{-3/2}+O_p(n^{-3/2})$.

Note that the conditions in Lemma 3 are all standard regularity conditions.

In the following section, we propose an alternative one-step estimator that eliminates the second-order bias by adjusting the moment equations inspired by Firth (1993) [13].

3. Bias-Corrected Moment Equation

Here, we consider an alternative higher order bias reduced estimator that solves bias-corrected moment equations. This idea was proposed in Firth (1993) [13] for the ML with a fixed number of parameters and exploited in Hahn and Newey (2004) [6] and Fernandez-Val (2004) [14] for the nonlinear panel data models with individual specific effects. We refer to this estimator as Firth’s estimator.

To be precise, consider:

$$0=\frac{1}{n}\sum _{i=1}^{n}s\left(z_i,\theta \right)-\frac{1}{n}c(\theta )$$

for a known function $c(\theta )$ that is given by:

$$c(\theta )=Q{(\theta )}^{-1}B(\theta )=\frac{1}{2}{H}_2(\theta )E\left[Q(\theta )s(z_i,\theta )\otimes Q\left(\theta \right)s(z_i,\theta )\right]+E\left[\nabla s\left(z_i,\theta \right)Q(\theta )s(z_i,\theta )\right].$$

(5)

This correction term $c(\theta )$ is obtained following Firth (1993) [13] and using the bias term for the M-estimator. In the ML context, Firth (1993) [13] shows that by adjusting the score function (he refers to this as a modified score function) with the correction term defined by the product of the Fisher information matrix and the bias term, one can obtain a bias-corrected ML estimator. $c(\theta )$ has the same interpretation in the ML, since $-Q{(\theta )}^{-1}$ is the Hessian matrix, and hence, $Q{(\theta )}^{-1}$ is the Fisher information in the ML. Therefore, (5) is a generalization of Firth (1993) [13]’s approach to the M-estimation . In general $c(\theta )$ contains population terms, and hence, to implement this alternative estimator, we need to estimate the function $c\left(\theta \right)$. We use a sample analogue of (5) as:

$$\begin{array}{ccc}\hfill \widehat{c}(\theta )& =& \widehat{Q}{(\theta )}^{-1}\widehat{B}(\theta )\hfill \\ & =& \frac{1}{2}{\widehat{H}}_2(\theta )\left(\frac{1}{n}\sum _{i=1}^n\left[\widehat{Q}(\theta )s(z_i,\theta )\otimes \widehat{Q}(\theta )s(z_i,\theta )\right]\right)+\frac{1}{n}\sum _{i=1}^n\left[\nabla s\left(z_i,\theta \right)\widehat{Q}(\theta )s(z_i,\theta )\right].\hfill \end{array}$$

(6)

Now, we estimate ${\theta }_0$ by solving:

$$0=\frac{1}{n}\sum _{i=1}^{n}s\left(z_i,\theta \right)-\frac{1}{n}\widehat{c}(\theta ),$$

(7)

and claim that the solution of this modified moment condition eliminates the second-order bias of $\widehat{\theta }$ that solves the original moment condition (2).

Assumption 1.

(i) ${\left\{z_i\right\}}_{i=1}^n$ are i.i.d.; (ii) $s(z,\theta )$ is κ-times continuously differentiable in a neighborhood of ${\theta }_0$, denoted by ${\mathrm{\Theta }}_0$ for all $z\in \mathcal{Z}$, $\kappa \ge 4$; (iii) $E\left[{sup}_{\theta \in {\mathrm{\Theta }}_0}{∥{\nabla }^{\upsilon }s(z,\theta )∥}^2\right]<\infty$, $\upsilon =\{0,1,2,\dots \kappa \}$, $\kappa \ge 4$; (iv) Θ is compact; (v) ${\theta }_0$ is in the interior of Θ and is the only $\theta \in \mathrm{\Theta }$ satisfying (1); (vi) $E\left[{∥{\nabla }^{\overline{\upsilon }}s(z,{\theta }_0)∥}^4\right]<\infty$ for $\overline{\upsilon }=\{0,1,2,\dots ,\overline{\kappa }\},\overline{\kappa }\ge 3$.

Assumption 2.

For $\theta \in {\mathrm{\Theta }}_0$, $E\left[\frac{\partial s\left(z_i,\theta \right)}{\partial {\theta }^{\prime }}\right]$ is nonsingular.

Alternatively, we can assume the following instead of Assumption 1.

Assumption 3.

(i) ${\left\{z_i\right\}}_{i=1}^n$ are i.i.d.; (ii) ${\nabla }^{\upsilon }s(z,\theta )$ satisfies the Lipschitz condition in θ as:

$$∥{\nabla }^{\upsilon }s(z,{\theta }_1)-{\nabla }^{\upsilon }s(z,{\theta }_2)∥\le B_{\upsilon }(z)∥{\theta }_1-{\theta }_2∥\forall {\theta }_1,{\theta }_2\in {\mathrm{\Theta }}_0$$

for some function $B_{\upsilon }(·):\mathcal{Z}\to R$ and $E\left[B_{\upsilon }{(·)}^{2t+\delta }\right]<\infty$, $\upsilon =\{0,1,2,\dots \kappa \},$ with positive integer $t\ge 2$ and for some $\delta >0$ and $\kappa \ge 4$ in a neighborhood of ${\theta }_0$; (iii) $E\left[{sup}_{\theta \in {\mathrm{\Theta }}_0}{∥{\nabla }^{\upsilon }s(z,\theta )∥}^{2t+\delta }\right]<\infty$, $\upsilon =\{0,1,2,\dots \kappa \}$, $\kappa \ge 4$ with positive integer $t\ge 2$ and for some $\delta >0$; (iv) Θ is bounded; (v) ${\theta }_0$ is in the interior of Θ and is the only $\theta \in \mathrm{\Theta }$ satisfying (1).

Under Assumptions 1 and 2 or Assumptions 3 and 2, the following three conditions are satisfied (see Lemma A.9 in Appendix A).

Condition 1.

(i) $\widehat{c}({\theta }_0)=O_p(1)$; (ii) $\widehat{c}({\theta }_0)=c({\theta }_0)+O_p\left(\frac{1}{\sqrt{n}}\right)$.

Condition 2.

$\nabla \widehat{c}(\theta )=O_p(1)$ in the $n^{-1/2}$ neighborhood of ${\theta }_0$.

Condition 3.

${\nabla }^2\widehat{c}(\theta )=O_p(1)$ in the $n^{-1/2}$ neighborhood of ${\theta }_0$.

Note that these three conditions are required to control for the estimation error in $\widehat{c}(\theta )$ in the stochastic expansions. Now, we are ready to present one of our main results.

Proposition 1.

Suppose ${\theta }^{{}^{*}}$ solves (7) where $\widehat{c}(\theta )$ is given by (6) and that ${\theta }^{{}^{*}}$ is a consistent estimator of ${\theta }_0$. Further, suppose that Conditions 1–3 and Conditions (i)–(viii) in Lemma 1 are satisfied, then we have:

$$\sqrt{n}\left({\theta }^{{}^{*}}-{\theta }_0\right)=QJ+\frac{1}{\sqrt{n}}Q\left(VQJ+\frac{1}{2}{H}_2\left(QJ\otimes QJ\right)-c({\theta }_0)\right)+O_p\left(\frac{1}{n}\right),$$

where $c({\theta }_0)=\frac{1}{2}{H}_{2}E\left[Qs(z_i,{\theta }_0)\otimes Qs(z_i,{\theta }_0)\right]+E\left[\nabla s\left(z_i,{\theta }_0\right)Qs(z_i,{\theta }_0)\right]$, and hence, the second-order bias of ${\theta }^{{}^{*}}$ is $Bias({\theta }^{{}^{*}})\equiv \frac{1}{n}E\left[Q\left(VQJ+\frac{1}{2}{H}_2\left(QJ\otimes QJ\right)-c({\theta }_0)\right)\right]=0$.

This concludes that we can eliminate the second-order bias by adjusting the moment equations as (7), and it is a proper alternative to the analytic bias correction of (3).

4. Higher Order Efficiency

Asymptotic bias corrections can provide estimators that have better bias properties in the finite sample. There are several ways of achieving bias correction, including analytical corrections that we focus on in this paper, the jackknife and bootstrap methods. These abundant ways of bias correction evoke the issue of which method is preferable to others at least on asymptotic efficiency grounds. For the ML estimation, Hahn et al. (2004) [12] show that the method of bias correction does not affect the higher order efficiency of any bias-corrected estimator that is first-order efficient. Although the ML estimator is a class of the M-estimator we consider, it is not trivial to conjecture that the same equivalence result will hold for a general class of M-estimators because the equivalence in the ML can hold due to some specific properties of the ML estimator. In this section, we formally extend the equivalence result to a general M-estimator.

We compare the higher order efficiency of several first-order efficient bias-corrected estimators by comparing the higher order variances, which are defined by the $O\left(\frac{1}{n}\right)$ variance in a third-order stochastic expansion of an estimator.

4.1. Third-Order Expansion of the One-Step Bias-Corrected Estimator

To compare with the estimator of interest ${\theta }^{{}^{*}}$ in (7), first we consider a one-step bias-corrected estimator ${\widehat{\theta }}_{bc}$ defined in (3) as ${\widehat{\theta }}_{bc}=\widehat{\theta }-\frac{1}{n}\widehat{B}(\widehat{\theta })$ and observe that $\widehat{B}(\widehat{\theta })=\widehat{Q}(\widehat{\theta })\widehat{c}(\widehat{\theta })$ from (4) and (6). We also consider its infeasible version ${\widehat{\theta }}_b$ as ${\widehat{\theta }}_b=\widehat{\theta }-\frac{1}{n}B(\widehat{\theta })$, where the function $B(\widehat{\theta })$ is constructed as $B(\widehat{\theta })=Q(\widehat{\theta })c(\widehat{\theta })$, provided that both $\widehat{B}(\widehat{\theta })$ and $B(\widehat{\theta })$ are consistent estimators of the higher order bias term $B({\theta }_0)=Q({\theta }_0)c({\theta }_0)$. Note that for some $\tilde{\theta }$ between $\widehat{\theta }$ and ${\theta }_0$, a first-order Taylor expansion gives us:

$$c(\widehat{\theta })-c({\theta }_0)=\nabla c(\tilde{\theta })\left(\widehat{\theta }-{\theta }_0\right)=O_p(1)O_p\left(1/\sqrt{n}\right)=o_p\left(1\right)$$

under Condition 2 and because $\widehat{\theta }-{\theta }_0=O_p\left(\frac{1}{\sqrt{n}}\right)$. Furthermore, we have:

$$\begin{array}{ccc}\hfill ∥\widehat{c}(\widehat{\theta })-c({\theta }_0)∥& \le & ∥\widehat{c}(\widehat{\theta })-c(\widehat{\theta })∥+∥c(\widehat{\theta })-c({\theta }_0)∥\hfill \\ & \le & \underset{\theta \in {\mathrm{\Theta }}_0}{sup}∥\widehat{c}(\theta )-c(\theta )∥+∥c(\widehat{\theta })-c({\theta }_0)∥=o_p\left(1\right)+o_p(1)=o_p(1)\hfill \end{array}$$

by the triangle inequality, Lemma A.7 (in Appendix A) and the continuity of $c(\theta )$ at ${\theta }_0$ (applying the Slutsky theorem), hence, both $B(\widehat{\theta })$ and $\widehat{B}(\widehat{\theta })$ are indeed consistent estimators of the higher order bias noting that $Q(\widehat{\theta })=Q({\theta }_0)+o_p(1)$ by the continuity of $Q(\theta )$ at ${\theta }_0$ and $\widehat{Q}(\widehat{\theta })=Q({\theta }_0)+o_p(1)$.

Now, from the result of Lemma 3 and a second-order Taylor expansion of $B(\widehat{\theta })$, it follows that:

$$\begin{array}{ccc}\hfill \sqrt{n}({\widehat{\theta }}_b-{\theta }_0)& =& \sqrt{n}(\widehat{\theta }-{\theta }_0)-\frac{1}{\sqrt{n}}B(\widehat{\theta })\hfill \\ & =& \begin{array}{c}{a}_{-1/2}+\frac{1}{\sqrt{n}}{a}_{-1}+\frac{1}{n}{a}_{-3/2}+O_p(n^{-3/2})\hfill \\ -\frac{1}{\sqrt{n}}B({\theta }_0)-\frac{1}{\sqrt{n}}\nabla B({\theta }_0)(\widehat{\theta }-{\theta }_0)-\frac{1}{2\sqrt{n}}{\nabla }^{2}B(\tilde{\theta })((\widehat{\theta }-{\theta }_0)\otimes (\widehat{\theta }-{\theta }_0))\hfill \end{array}\hfill \end{array}$$

where $\tilde{\theta }$ is a point between $\widehat{\theta }$ and ${\theta }_0$, and hence:

$$\sqrt{n}({\widehat{\theta }}_b-{\theta }_0)=a_{-1/2}+\frac{1}{\sqrt{n}}\left(a_{-1}-B({\theta }_0)\right)+\frac{1}{n}\left(a_{-3/2}-\nabla B({\theta }_0)a_{-1/2}\right)+O_p(n^{-3/2}),$$

(8)

since $\sqrt{n}(\widehat{\theta }-{\theta }_0)=a_{-1/2}+O_p\left(\frac{1}{\sqrt{n}}\right)$ and ${\nabla }^{2}B(\tilde{\theta })={\nabla }^{2}B({\theta }_0)+o_p(1)=O_p(1)$ by the Slutsky theorem, from which we conclude $\frac{1}{2\sqrt{n}}{\nabla }^{2}B(\tilde{\theta })((\widehat{\theta }-{\theta }_0)\otimes (\widehat{\theta }-{\theta }_0))=O_p(n^{-3/2})$.

Now, similarly for ${\widehat{\theta }}_{bc}$, we obtain:

$$\begin{array}{ccc}\hfill \sqrt{n}({\widehat{\theta }}_{bc}-{\theta }_0)& =& \sqrt{n}(\widehat{\theta }-{\theta }_0)-\frac{1}{\sqrt{n}}\widehat{B}(\widehat{\theta })\hfill \\ & =& \begin{array}{c}{a}_{-1/2}+\frac{1}{\sqrt{n}}{a}_{-1}+\frac{1}{n}{a}_{-3/2}+O_p(n^{-3/2})\hfill \\ -\frac{1}{\sqrt{n}}\widehat{B}({\theta }_0)-\frac{1}{\sqrt{n}}\nabla \widehat{B}({\theta }_0)(\widehat{\theta }-{\theta }_0)-\frac{1}{2\sqrt{n}}{\nabla }^2\widehat{B}(\tilde{\theta })((\widehat{\theta }-{\theta }_0)\otimes (\widehat{\theta }-{\theta }_0)).\hfill \end{array}\hfill \end{array}$$

(9)

Then, applying the following three results (that hold under Assumptions 1 and 2 or 3 and 2, as shown in Lemma A.12 in Appendix A):

Condition 4.

$\widehat{B}({\theta }_0)=B({\theta }_0)+O_p\left(1/\sqrt{n}\right)$,

Condition 5.

$\nabla \widehat{B}({\theta }_0)=\nabla B({\theta }_0)+O_p\left(1/\sqrt{n}\right)$,

Condition 6.

${\nabla }^2\widehat{B}(\theta )=O_p(1)$ in the neighborhood of ${\theta }_0$,

we obtain:

$$\begin{array}{ccc}& & \sqrt{n}({\widehat{\theta }}_{bc}-{\theta }_0)\hfill \\ & =& a_{-1/2}+\frac{1}{\sqrt{n}}\left(a_{-1}-B({\theta }_0)\right)+\frac{1}{n}\left(a_{-3/2}-\nabla B({\theta }_0)a_{-1/2}-\sqrt{n}(\widehat{B}({\theta }_0)-B({\theta }_0))\right)+O_p(n^{-3/2})\hfill \end{array}$$

(10)

noting that $\frac{1}{\sqrt{n}}\nabla \widehat{B}({\theta }_0)(\widehat{\theta }-{\theta }_0)=\frac{1}{n}\nabla \widehat{B}({\theta }_0)\left(a_{-1/2}+O_p\left(\frac{1}{\sqrt{n}}\right)\right)=\frac{1}{n}\nabla B({\theta }_0)a_{-1/2}+O_p(n^{-3/2})$ by Condition 5 and that $\frac{1}{2\sqrt{n}}{\nabla }^2\widehat{B}(\tilde{\theta })((\widehat{\theta }-{\theta }_0)\otimes (\widehat{\theta }-{\theta }_0))=O_p(n^{-3/2})$ by Condition 6 and the fact that $\widehat{\theta }-{\theta }_0=O_p\left(1/\sqrt{n}\right)$.

4.2. Third-Order Expansion of the Bias-Corrected Moment Equations Estimator

Now, we derive the higher order expansion of the proposed bias-corrected estimator ${\theta }^{{}^{*}}$ up to the third order. For this, we need to verify an additional condition below, which is satisfied under Assumptions 1 and 2 or 3 and 2 with $\kappa \ge 5$ as shown in Lemma A.10 and A.11 in Appendix A.

Condition 7.

(i) $\nabla \widehat{c}({\theta }_0)=\nabla c({\theta }_0)+O_p\left(\frac{1}{\sqrt{n}}\right)$; (ii) ${\nabla }^3\widehat{c}(\theta )=O_p(1)$ in the $n^{-1/2}$ neighborhood of ${\theta }_0$.

Recall that $c(\theta )=Q^{-1}(\theta )B(\theta )$ and $\widehat{c}(\theta )=\widehat{Q}{(\theta )}^{-1}\widehat{B}(\theta )$, and we obtain:

Proposition 2.

Suppose ${\theta }^{{}^{*}}$ solves (7), where $\widehat{c}(\theta )$ is given in (6), and that ${\theta }^{{}^{*}}$ is consistent. Further, suppose that Conditions 1–7 and Conditions (i)–(viii) in Lemma 3 are satisfied, and assume $\sqrt{n}(\widehat{\theta }-{\theta }_0)=a_{-1/2}+\frac{1}{\sqrt{n}}\left(a_{-1}-B({\theta }_0)\right)+O_p\left(\frac{1}{n}\right)$. Then, we have:

$$\begin{array}{c}\sqrt{n}\left({\theta }^{{}^{*}}-{\theta }_0\right)\hfill \\ {=a}_{-1/2}+\frac{1}{\sqrt{n}}\left(a_{-1}-B({\theta }_0)\right)+\frac{1}{n}\left(a_{-3/2}-\nabla B({\theta }_0)a_{-1/2}-\sqrt{n}(\widehat{B}({\theta }_0)-B({\theta }_0))\right){+O}_p{(n}^{-3/2}).\hfill \end{array}$$

(11)

Comparing (10) and (11), we therefore conclude that $\sqrt{n}\left({\theta }^{{}^{*}}-{\theta }_0\right)$ and $\sqrt{n}\left({\widehat{\theta }}_{bc}-{\theta }_0\right)$ are identical up to $O_p(\frac{1}{n})$ order terms. This implies that ${\theta }^{{}^{*}}$ and ${\widehat{\theta }}_{bc}$ at least agree upon their higher order variances, as we discuss in the following section.

4.3. Higher Order Variances

For a three-term stochastic expansion of an estimator $\check{\theta }$, such as:

$$\sqrt{n}\left(\check{\theta }-{\theta }_0\right)=T_{-1/2}+\frac{1}{\sqrt{n}}{T}_{-1}+\frac{1}{n}{T}_{-3/2}+O_p(n^{-3/2}),$$

the higher order variance is given by:

$${\mathrm{\Lambda }}_{\check{\theta }}\equiv \Sigma +\frac{1}{n}\mathrm{\Xi },$$

with $\Sigma =$Var$[T_{-1/2}]$ and $\mathrm{\Xi }=\mathrm{Var}[T_{-1}]+E\left[\left(\sqrt{n}{T}_{-1}+T_{-3/2}\right)T_{-1/2}^{\prime }\right]+E\left[T_{-1/2}{\left(\sqrt{n}{T}_{-1}+T_{-3/2}\right)}^{\prime }\right]$. Then, from the third-order stochastic expansions of the bias-corrected estimators derived in (8), (10) and (11), we can obtain the higher order variances of three alternative estimators, denoted by ${\mathrm{\Lambda }}_{{\widehat{\theta }}_b}$, ${\mathrm{\Lambda }}_{{\widehat{\theta }}_{bc}}$, and ${\mathrm{\Lambda }}_{{\theta }^{{}^{*}}}$, respectively, as:5

$$\begin{array}{ccc}\hfill {\mathrm{\Lambda }}_{{\widehat{\theta }}_b}& =& \left\{\begin{array}{c}E\left[a_{-1/2}{a}_{-1/2}^{\prime }\right]+\frac{1}{n}E\left[\left(a_{-1}-B({\theta }_0)\right){\left(a_{-1}-B({\theta }_0)\right)}^{\prime }\right]\hfill \\ +\frac{1}{n}E\left[a_{-1/2}{\left(a_{-3/2}-\nabla B({\theta }_0)a_{-1/2}\right)}^{\prime }\right]+\frac{1}{n}E\left[\left(a_{-3/2}-\nabla B({\theta }_0)a_{-1/2}\right)a_{-1/2}^{\prime }\right]\hfill \\ +\frac{1}{n}E\left[\sqrt{n}{a}_{-1/2}{\left(a_{-1}-B({\theta }_0)\right)}^{\prime }\right]+\frac{1}{n}E\left[\sqrt{n}\left(a_{-1}-B({\theta }_0)\right)a_{-1/2}^{\prime }\right]\hfill \end{array}\right\}\hfill \end{array}$$

$$\begin{array}{ccc}\hfill {\mathrm{\Lambda }}_{{\widehat{\theta }}_{bc}}& =& {\mathrm{\Lambda }}_{{\widehat{\theta }}_b}-\frac{1}{n}E\left[a_{-1/2}\sqrt{n}{(\widehat{B}({\theta }_0)-B({\theta }_0))}^{\prime }\right]-\frac{1}{n}E\left[\sqrt{n}(\widehat{B}({\theta }_0)-B({\theta }_0))a_{-1/2}^{\prime }\right]\hfill \end{array}$$

(12)

$$\begin{array}{ccc}\hfill {\mathrm{\Lambda }}_{{\theta }^{{}^{*}}}& =& {\mathrm{\Lambda }}_{{\widehat{\theta }}_{bc}}.\hfill \end{array}$$

(13)

First note that the result of (12) reveals that the higher order variance of ${\widehat{\theta }}_{bc}$ has additional terms compared with ${\widehat{\theta }}_b$ (infeasible estimator) because we use the sample analogue of the second-order bias, unless $E\left[a_{-1/2}\sqrt{n}(\widehat{B}({\theta }_0)-B({\theta }_0))\right]=0$. These additional terms contribute to the cost of estimating the bias term $\widehat{B}(·)$ in the analytic bias correction approach. Now, the result of (13) tells that the higher order variances of two alternative bias-corrected estimators are the same, so on the grounds of this higher order variances’ comparison, we find that the bias correction method by adjusting moment equations does not improve over the analytic bias correction.

Indeed, it is more remarkable that comparing the third-order expansions of (10) and (11), we further find:

$$n^{3/2}({\widehat{\theta }}_{bc}-{\theta }^{{}^{*}})=o_p(1).$$

(14)

This is a stronger result than just comparing the higher order variances because it implies that these two estimators do not only have the same higher order variance, but also agree on more properties in terms of their stochastic expansions. In the literature, it has been argued that removing the bias directly from the moment equations has an attractive feature that it does not use pre-estimated parameters that are not bias corrected, though this alternative approach requires more intensive computations, since it requires solving some nonlinear equation. However, in view of the result (14), this paper concludes that at least for the third-order stochastic expansion comparison, there is no benefit of using such bias correction of the moment equations over the simple bias-corrected estimator.

4.4. Further Comparison of Alternative Bias Corrections

To have a better understanding of the equivalence result (14), here we compare several versions of bias-corrected estimators that are infeasible by their nature. First, let ${\theta }_1^{{}^{*}}$ be the solution of $0=\frac{1}{n}{\sum }_{i=1}^{n}s\left(z_i,\theta \right)-\frac{1}{n}c(\theta )$ where $c(\theta )$ is known. We also define two other bias-corrected estimators:

$${\widehat{\theta }}_2=\widehat{\theta }-\widehat{Q}(\widehat{\theta })c(\widehat{\theta })\mathrm{and}{\widehat{\theta }}_3=\widehat{\theta }-Q(\widehat{\theta })\widehat{c}(\widehat{\theta }),$$

so for ${\widehat{\theta }}_2$, $c(\theta )$ is known, but $Q(\theta )$ is estimated, while for ${\widehat{\theta }}_3$, $c(\theta )$ is estimated, but $Q(\theta )$ is known. For these estimators, we obtain the following results.6

$$\begin{array}{cc}\hfill \sqrt{n}({\theta }_1^{{}^{*}}-{\theta }_0)& =\sqrt{n}({\widehat{\theta }}_b-{\theta }_0)-\frac{1}{n}QVB({\theta }_0)+O_p(n^{-3/2}),\hfill \\ \hfill \sqrt{n}({\theta }_1^{{}^{*}}-{\theta }_0)& =\sqrt{n}({\widehat{\theta }}_2-{\theta }_0)+O_p(n^{-3/2}),\hfill \\ \hfill \sqrt{n}({\theta }^{{}^{*}}-{\theta }_0)& =\sqrt{n}({\widehat{\theta }}_3-{\theta }_0)+\frac{1}{n}QVB\left({\theta }_0\right)+O_p(n^{-3/2}),\hfill \\ \hfill \sqrt{n}({\theta }^{{}^{*}}-{\theta }_0)& =\sqrt{n}({\widehat{\theta }}_{bc}-{\theta }_0)+O_p(n^{-3/2}).\hfill \end{array}$$

The results illustrate that using $\widehat{Q}(·)$ rather than $Q(·)$ in the bias correction term plays a critical role for equating the stochastic expansions (up to the third order) of the one-step bias-corrected estimator and the bias-corrected moment equations estimator. To see the point compare ${\theta }_1^{{}^{*}}$ and ${\widehat{\theta }}_2$ and compare ${\theta }^{{}^{*}}$ and ${\widehat{\theta }}_{bc}$ where in the former $c(·)$ is known, and in the latter, $c(·)$ is also estimated.

Next, we consider the possible iteration of the bias correction. Hahn and Newey (2004) [6] discuss the relationship between the bias correction of moment equations and the iterated bias correction. The iteration idea is that one can update $\widehat{B}(·)$ several times using the previous estimator of $\widehat{\theta }$. To be precise, denoting $\widehat{B}(\theta )$ as a function of θ, we can write the one-step bias-corrected estimator as ${\widehat{\theta }}_{bc}^1=\widehat{\theta }-\widehat{B}(\widehat{\theta })/n$. The k-th iteration will give us ${\widehat{\theta }}_{bc}^k=\widehat{\theta }-\widehat{B}({\widehat{\theta }}_{bc}^{k-1})/n$ for $k\ge 2$ where ${\widehat{\theta }}_{bc}^1={\widehat{\theta }}_{bc}$. If we iterate this procedure until convergence, we will obtain ${\widehat{\theta }}_{bc}^{\infty }=\widehat{\theta }-\widehat{B}({\widehat{\theta }}_{bc}^{\infty })/n$, which implies that ${\widehat{\theta }}_{bc}^{\infty }$ solves (note $\widehat{B}(\theta )=\widehat{Q}(\theta )\widehat{c}(\theta )$):

$$0=\widehat{Q}{(\theta )}^{-1}(\widehat{\theta }-\theta )-\frac{1}{n}\widehat{c}(\theta )=\frac{1}{n}{\sum }_{i=1}^{n}s\left(z_i,\widehat{\theta }\right)+\widehat{Q}{(\theta )}^{-1}(\widehat{\theta }-\theta )-\frac{1}{n}\widehat{c}(\theta ),$$

(15)

where the second equality is from the definition of $\widehat{\theta }$ in (2). Observing that $\widehat{Q}{(\theta )}^{-1}=-\frac{1}{n}{\sum }_{i=1}^n\nabla s\left(z_i,\theta \right)$, if $s\left(z_i,\theta \right)$ is linear in θ, then we find that the Equation (15) is the same as Equation (7) for the bias-corrected moment equations; hence, ${\widehat{\theta }}_{bc}^{\infty }$ is exactly the same with ${\theta }^{{}^{*}}$. Otherwise, (15) is an approximation of (7). From this, we conclude that the fully-iterated bias-corrected estimator ${\widehat{\theta }}_{bc}^{\infty }$ can be interpreted as the solution to an approximation of the bias-corrected moment Equation (7). Similarly to (9), for $\tilde{\tilde{\theta }}$ between ${\widehat{\theta }}_{bc}^{\infty }$ and ${\theta }_0$, we can show that:

$$\begin{array}{c}\sqrt{n}({\widehat{\theta }}_{bc}^{\infty }-{\theta }_0)=\sqrt{n}(\widehat{\theta }-{\theta }_0)-\frac{1}{\sqrt{n}}\widehat{B}({\widehat{\theta }}_{bc}^{\infty })\hfill \\ =\begin{array}{c}{a}_{-1/2}+\frac{1}{\sqrt{n}}{a}_{-1}+\frac{1}{n}{a}_{-3/2}\hfill \\ -\frac{1}{\sqrt{n}}\widehat{B}({\theta }_0)-\frac{1}{\sqrt{n}}\nabla \widehat{B}({\theta }_0)({\widehat{\theta }}_{bc}^{\infty }-{\theta }_0)-\frac{1}{2\sqrt{n}}{\nabla }^2\widehat{B}(\tilde{\tilde{\theta }})(({\widehat{\theta }}_{bc}^{\infty }-{\theta }_0)\otimes ({\widehat{\theta }}_{bc}^{\infty }-{\theta }_0))+O_p(n^{-3/2})\hfill \end{array}\hfill \\ =a_{-1/2}+\frac{1}{\sqrt{n}}\left(a_{-1}-B({\theta }_0)\right)+\frac{1}{n}\left(a_{-3/2}-\nabla B({\theta }_0)a_{-1/2}-\sqrt{n}(\widehat{B}({\theta }_0)-B({\theta }_0))\right)+O_p(n^{-3/2})\hfill \end{array}$$

using Conditions 4, 5 and 6 and $\sqrt{n}\left({\widehat{\theta }}_{bc}^{\infty }-{\theta }_0\right)=QJ+O_p\left(1/\sqrt{n}\right)$. This result confirms that $\sqrt{n}({\widehat{\theta }}_{bc}^{\infty }-{\widehat{\theta }}_{bc})=O_p(n^{-3/2})$, which actually holds for all ${\widehat{\theta }}_{bc}^k$ ( $k\ge 2$).

From this equivalence of the higher order expansions for ${\widehat{\theta }}_{bc}^{\infty }$ and ${\widehat{\theta }}_{bc}$ at least up to the third order term, one would expect that the higher order expansion of ${\theta }^{{}^{*}}$ will be equivalent to that of ${\widehat{\theta }}_{bc}$ at least up to the third order, and we have verified that this intuition is correct. However, as observed in some Monte Carlo examples of Hahn and Newey (2004) [6] and Fernandez-Val (2004) [14], the iterative bias correction can lower the bias for small samples and so can the bias correction of the moment equations. This suggests that the comparison between the one-step bias correction and the method of correcting the moment equations (or the fully-iterated bias correction) should be based on the stochastic expansions higher than the third order. The comparison of the two alternative bias-corrected estimators over the fourth order or even higher order stochastic expansions can be challenging and is beyond the scope of this paper.

5. Conclusions

This paper considers an alternative bias correction for the M-estimator, which is achieved by correcting the moment equations in the spirit of Firth (1993) [13]. In particular, this paper compares the stochastic expansions of the analytically-bias-corrected estimator and the alternative estimator and finds that the third-order stochastic expansions of these two estimators are identical. This implies that these two estimators do not only have the same higher order variances, but also agree upon more properties in terms of their stochastic expansions.

We conclude that at least in terms of the third-order stochastic expansion, we cannot improve on the simple one-step bias-correction by using the bias correction of the moment equations. The intuition is that the fully-iterated bias-corrected estimator can be interpreted as the solution of an approximation to the bias-corrected moment equations, and the iteration will not improve the asymptotic properties in general; neither will the alternative estimator. We have verified this intuition in this paper.

multiple