A Simple Wide Range Approximation of Symmetric Binomial Distribution

Abstract

The paper gives a wide range, uniform, local approximation of symmetric binomial distribution. The result clearly shows how one has to modify the classical de Moivre–Laplace normal approximation in order to give an estimate at the tail as well as to minimize the relative error.

The topic of this paper is a wide range, uniform, local approximation of symmetric binomial distribution, an extension of the classical de Moivre–Laplace theorem in the symmetric case [1] (Ch. VII) [2]. In other words, I would like to approximate individual binomial probabilities not only in a classical neighborhood of the center, but at the tail as well. The result clearly shows how one has to modify the normal approximation in order to give a wide range estimate to minimize the relative error. The method will be somewhat similar to the ones applied by [3,4,5] or in the proof of Tusnády’s lemma, see, e.g., [6]; however, my task is much simpler than those. This simplification makes the proof short, transparent and natural. Moreover, the result is non-asymptotic, that is, it gives explicit, nearly optimal, upper and lower bounds for the relative error with a finite n. Thus, I hope that it can be used in both applications and teaching.

Let ${\left(X_r\right)}_{r\ge 1}$ be a sequence of independent, identically distributed random steps with $\mathbb{P}(X_r=\pm 1)={\displaystyle \frac{1}{2}}$ and $S_{\ell }={\sum }_{r=1}^{\ell }{X}_r$$(\ell \ge 1)$, $S_0=0$, be the corresponding simple, symmetric random walk. Then

$$\mathbb{P}(S_{\ell }=j)=\left({\displaystyle \genfrac{}{}{0pt}{}{\ell }{\frac{\ell +j}{2}}}\right)2^{-\ell }\left(\right|j|\le \ell ).$$

(1)

Here, we use the convention that the above binomial coefficient is zero whenever $\ell +j$ is not divisible by 2. Since $\mathbb{P}(S_{\ell }=j)=\mathbb{P}(S_{\ell }=-j)$ for any j, it is enough to consider only the case $j\ge 0$ whenever it is convenient.

First we consider the case when $\ell =2n$, even. Let us introduce the notation

$$a_{k,n}:=\mathbb{P}(S_{2n}=2k)=\left({\displaystyle \genfrac{}{}{0pt}{}{2n}{n+k}}\right)2^{-2n}\left(\right|k|\le n).$$

Also, introduce the notation

$$b_{k,n}:=n\left\{\left(1+{\displaystyle \frac{k+\frac{1}{2}}{n}}\right)log\left(1+{\displaystyle \frac{k}{n}}\right)+\left(1-{\displaystyle \frac{k-\frac{1}{2}}{n}}\right)log\left(1-{\displaystyle \frac{k}{n}}\right)\right\}$$

(2)

when $n\ge 1$ and $\left|k\right|<n$, and

$$b_{\pm n,n}:=\left(2n+{\displaystyle \frac{1}{2}}\right)log2-{\displaystyle \frac{1}{2}}log\left(2\pi n\right)(n\ge 1).$$

(3)

Theorem 1.

(a) For any $n\ge 1$ and $\left|k\right|\le n$, we have

$$a_{k,n}\le {\displaystyle \frac{1}{\sqrt{\pi n}}}{e}^{-b_{k,n}}.$$

(4)

(b) For any $n\ge 1$ and $\left|k\right|\le rn$, $r\in (0,1)$, we have

$$a_{k,n}>{\displaystyle \frac{1}{\sqrt{\pi n}}}{e}^{-b_{k,n}}exp\left(-{\displaystyle \frac{1}{7n}}-{\displaystyle \frac{r^4}{3{(1-r^2)}^{2}n}}\right).$$

(5)

(c) In accordance with the classical de Moivre–Laplace normal approximation, for $n\ge 3$ and $\left|k\right|\le n^{{\displaystyle \frac{2}{3}}}$ one has

$$1-2n^{-{\displaystyle \frac{1}{3}}}<{\displaystyle \frac{a_{k,n}}{\frac{1}{\sqrt{\pi n}}{e}^{-\frac{k^2}{n}}}}<1+2n^{-{\displaystyle \frac{1}{3}}}.$$

(6)

Proof. 

As usual, the first step is to estimate the central term

$$a_{0,n}=\left({\displaystyle \genfrac{}{}{0pt}{}{2n}{n}}\right)2^{-2n}={\displaystyle \frac{\left(2n\right)!}{{(n!)}^2}}{2}^{-2n}.$$

By Stirling’s formula, see, e.g., [1] (p. 54), we have:

$$\sqrt{2\pi n}{\left({\displaystyle \frac{n}{e}}\right)}^n{e}^{{\displaystyle \frac{1}{12n+1}}}<n!<\sqrt{2\pi n}{\left({\displaystyle \frac{n}{e}}\right)}^n{e}^{{\displaystyle \frac{1}{12n}}}(n\ge 1).$$

(7)

Thus, after simplification we obtain

$${\displaystyle \frac{1}{\sqrt{\pi n}}}{e}^{-{\displaystyle \frac{1}{7n}}}<a_{0,n}<{\displaystyle \frac{1}{\sqrt{\pi n}}}{e}^{-{\displaystyle \frac{1}{9n}}}(n\ge 1).$$

(8)

Second, also by the standard way, for $1\le k\le n$,

$$a_{k,n}=a_{0,n}{\displaystyle \frac{n(n-1)\cdots (n-k+1)}{(n+1)(n+2)\cdots (n+k)}}=a_{0,n}{\displaystyle \frac{\left(1-\frac{1}{n}\right)\left(1-\frac{2}{n}\right)\cdots \left(1-\frac{k-1}{n}\right)}{\left(1+\frac{1}{n}\right)\left(1+\frac{2}{n}\right)\cdots \left(1+\frac{k}{n}\right)}}.$$

So it follows that

$$\begin{array}{c}log{a}_{k,n}=log{a}_{0,n}-log\left(1+{\displaystyle \frac{k}{n}}\right)-2\sum _{j=1}^{k-1}{\displaystyle \frac{1}{2}}log{\displaystyle \frac{1+\frac{j}{n}}{1-\frac{j}{n}}}\\ =log{a}_{0,n}-log\left(1+{\displaystyle \frac{k}{n}}\right)-2\sum _{j=1}^{k-1}{tanh}^{-1}\left({\displaystyle \frac{j}{n}}\right).\end{array}$$

(9)

To approximate the sum here, let us introduce the integral

$$I\left(x\right):={\int }_0^x{tanh}^{-1}\left(t\right)\mathrm{d}t={\displaystyle \frac{1}{2}}(1+x)log(1+x)+{\displaystyle \frac{1}{2}}(1-x)log(1-x)$$

(10)

for $\left|x\right|<1$, and its approximation by a trapezoidal sum

$$T_{k,n}:={\displaystyle \frac{1}{n}}\left\{{\displaystyle \frac{1}{2}}{tanh}^{-1}\left(0\right)+\sum _{j=1}^{k-1}{tanh}^{-1}\left({\displaystyle \frac{j}{n}}\right)+{\displaystyle \frac{1}{2}}{tanh}^{-1}\left({\displaystyle \frac{k}{n}}\right)\right\},$$

(11)

where $0\le k<n$. It is well-known that the error of the trapezoidal formula for a function $f\in C^2\left([a,b]\right)$ is

$$T_n\left(f\right)-{\int }_a^{b}f\left(t\right)\mathrm{d}t={\displaystyle \frac{{(b-a)}^3}{12n^2}}{f}^{\prime \prime }\left(x\right),x\in [a,b].$$

Since ${\left({tanh}^{-1}\right)}^{″}\left(x\right)=2x{(1-x^2)}^{-2}$, we obtain that

$$0\le T_{k,n}-I\left({\displaystyle \frac{k}{n}}\right)\le {\displaystyle \frac{r^4}{6{(1-r^2)}^2{n}^2}},\mathrm{when}0\le k\le rn,r\in (0,1).$$

(12)

Let us combine Formulas (8)–(12):

$$log{a}_{k,n}=log{a}_{0,n}-2n{T}_{k,n}-{\displaystyle \frac{1}{2}}log\left(1+{\displaystyle \frac{k}{n}}\right)-{\displaystyle \frac{1}{2}}log\left(1-{\displaystyle \frac{k}{n}}\right),$$

thus

$$log{\displaystyle \frac{1}{\sqrt{\pi n}}}-b_{k,n}-{\displaystyle \frac{1}{7n}}-{\displaystyle \frac{r^4}{3{(1-r^2)}^{2}n}}<log{a}_{k,n}<log{\displaystyle \frac{1}{\sqrt{\pi n}}}-b_{k,n}-{\displaystyle \frac{1}{9n}},$$

(13)

where $0\le k\le rn$ and

$$b_{k,n}=2nI\left({\displaystyle \frac{k}{n}}\right)+{\displaystyle \frac{1}{2}}log\left(1-{\displaystyle \frac{k^2}{n^2}}\right).$$

(14)

Clearly, (14) is the same as (2). Thus, (13) proves (a) and (b) of the theorem.

Let us see now, using Taylor expansions, a series expansion of $b_{k,n}$ when $\left|k\right|<n$. First, for $\left|x\right|<1$,

$$I\left(x\right)={\displaystyle \frac{x^2}{1·2}}+{\displaystyle \frac{x^4}{3·4}}+{\displaystyle \frac{x^6}{5·6}}+{\displaystyle \frac{x^8}{7·8}}+\cdots .$$

Second, also for $\left|x\right|<1$,

$$log(1-x^2)=-x^2-{\displaystyle \frac{x^4}{2}}-{\displaystyle \frac{x^6}{3}}-{\displaystyle \frac{x^8}{4}}-\cdots .$$

So by (14), for $\left|k\right|<n$ we have a convergent series for $b_{k,n}$:

$$\begin{array}{cc}\hfill b_{k,n}& ={\displaystyle \frac{k^2}{n}}\left(1-{\displaystyle \frac{1}{2n}}\right)+{\displaystyle \frac{k^4}{2n^3}}\left({\displaystyle \frac{1}{3}}-{\displaystyle \frac{1}{2n}}\right)+{\displaystyle \frac{k^6}{3n^5}}\left({\displaystyle \frac{1}{5}}-{\displaystyle \frac{1}{2n}}\right)+\cdots \hfill \\ & =\sum _{j=1}^{\infty }{\left({\displaystyle \frac{k}{n}}\right)}^{2j}{\displaystyle \frac{1}{j}}\left({\displaystyle \frac{n}{2j-1}}-{\displaystyle \frac{1}{2}}\right).\hfill \end{array}$$

(15)

By (15), for $n\ge 3$ and $\left|k\right|\le n^{{\displaystyle \frac{2}{3}}}$ we obtain that

$$\begin{array}{c}\left|b_{k,n}-{\displaystyle \frac{k^2}{n}}\right|\le {\displaystyle \frac{n^{-\frac{2}{3}}}{2}}+\sum _{j=2}^{\infty }{n}^{-{\displaystyle \frac{2}{3}}j}{\displaystyle \frac{1}{j}}\left({\displaystyle \frac{n}{2j-1}}+{\displaystyle \frac{1}{2}}\right)\hfill \\ \hfill \le {\displaystyle \frac{n^{-\frac{2}{3}}}{2}}+{\displaystyle \frac{n}{2}}\left({\displaystyle \frac{1}{3}}+{\displaystyle \frac{1}{6}}\right)\sum _{j=2}^{\infty }{n}^{-{\displaystyle \frac{2}{3}}j}\le n^{-{\displaystyle \frac{1}{3}}}.\end{array}$$

Thus, by (4), for $n\ge 3$ and $\left|k\right|\le n^{{\displaystyle \frac{2}{3}}}$,

$$a_{k,n}\le {\displaystyle \frac{1}{\sqrt{\pi n}}}{e}^{-{\displaystyle \frac{k^2}{n}}}{e}^{n^{-1/3}}<{\displaystyle \frac{1}{\sqrt{\pi n}}}{e}^{-{\displaystyle \frac{k^2}{n}}}\left(1+2n^{-{\displaystyle \frac{1}{3}}}\right).$$

(16)

Similarly, by (5), for $n\ge 3$ and $\left|k\right|\le n^{{\displaystyle \frac{2}{3}}}$,

$$\begin{array}{c}{a}_{k,n}>{\displaystyle \frac{1}{\sqrt{\pi n}}}{e}^{-{\displaystyle \frac{k^2}{n}}}exp\left(-{\displaystyle \frac{1}{7n}}-{\displaystyle \frac{n^{-\frac{7}{3}}}{3{(1-n^{-\frac{2}{3}})}^2}}-n^{-{\displaystyle \frac{1}{3}}}\right)\hfill \\ \hfill \ge {\displaystyle \frac{1}{\sqrt{\pi n}}}{e}^{-{\displaystyle \frac{k^2}{n}}}{e}^{-1.21n^{-1/3}}>{\displaystyle \frac{1}{\sqrt{\pi n}}}{e}^{-{\displaystyle \frac{k^2}{n}}}\left(1-2n^{-1/3}\right).\end{array}$$

(17)

(16) and (17) together prove (c) of the theorem. □

The main moral of Theorem 1 is that the exponent $k^2/n$ in the exponent of the normal approximation is only a first approximation of the series (15). In (c), the bound $\left|k\right|\le n^{{\displaystyle \frac{2}{3}}}$ was chosen somewhat arbitrarily. It is clear from the series (15) that the bound should be $o(n^{{\displaystyle \frac{3}{4}}})$. The above given bound was picked because it seemed to be satisfactory for usual applications with large deviation and gave a nice relative error bound $2n^{-{\displaystyle \frac{1}{3}}}$.

It is not difficult to extend the previous results to the odd-valued case of the symmetric binomial probabilities

$$a_{k,n}^{*}:=\mathbb{P}(S_{2n-1}=2k-1)=\left({\displaystyle \genfrac{}{}{0pt}{}{2n-1}{n+k-1}}\right)2^{-2n+1}(-n+1\le k\le n).$$

Define

$$b_{k,n}^{*}:=n\left\{\left(1+{\displaystyle \frac{k-\frac{1}{2}}{n}}\right)log\left(1+{\displaystyle \frac{k}{n}}\right)+\left(1-{\displaystyle \frac{k-\frac{1}{2}}{n}}\right)log\left(1-{\displaystyle \frac{k}{n}}\right)\right\}$$

(18)

for $n\ge 1$ and $\left|k\right|<n$, and

$$b_{n,n}^{*}:=\left(2n-{\displaystyle \frac{1}{2}}\right)log2-{\displaystyle \frac{1}{2}}log\left(2\pi n\right)(n\ge 1).$$

(19)

Theorem 2.

(a) For any $n\ge 1$ and $-n+1\le k\le n$, we have

$$a_{k,n}^{*}<{\displaystyle \frac{1}{\sqrt{\pi n}}}{e}^{-b_{k,n}^{*}}exp\left({\displaystyle \frac{2}{3n}}\right).$$

(20)

(b) For any $n\ge 1$ and $\left|k\right|\le rn$, $r\in (0,1)$, we have

$$a_{k,n}^{*}>{\displaystyle \frac{1}{\sqrt{\pi n}}}{e}^{-b_{k,n}^{*}}exp\left(-{\displaystyle \frac{1}{n}}-{\displaystyle \frac{r^4}{3{(1-r^2)}^{2}n}}\right).$$

(21)

(c) In accordance with the classical de Moivre–Laplace normal approximation, for $n\ge 4$ and $\left|k\right|\le n^{{\displaystyle \frac{2}{3}}}$ one has

$$1-3n^{-{\displaystyle \frac{1}{3}}}<{\displaystyle \frac{a_{k,n}^{*}}{\frac{1}{\sqrt{\pi n}}{e}^{-\frac{k^2}{n}}}}<1+6n^{-{\displaystyle \frac{1}{3}}}.$$

(22)

Proof. 

Since this proof is very similar to the previous one, several details are omitted. First, by Stirling’s Formula (7), after simplifications we obtain that

$${\displaystyle \frac{1}{\sqrt{\pi n}}}{e}^{-{\displaystyle \frac{1}{n}}}<a_{0,n}^{*}<{\displaystyle \frac{1}{\sqrt{\pi n}}}{e}^{{\displaystyle \frac{2}{3n}}}(n\ge 1).$$

(23)

Second, similarly to (13), for $n\ge 1$ and $\left|k\right|\le rn$, $r\in (0,1)$, we obtain that

$$log{\displaystyle \frac{1}{\sqrt{\pi n}}}-b_{k,n}^{*}-{\displaystyle \frac{1}{n}}-{\displaystyle \frac{r^4}{3{(1-r^2)}^{2}n}}<log{a}_{k,n}^{*}<log{\displaystyle \frac{1}{\sqrt{\pi n}}}-b_{k,n}^{*}+{\displaystyle \frac{2}{3n}},$$

(24)

where

$$b_{k,n}^{*}:=2nI\left({\displaystyle \frac{k}{n}}\right)-{tanh}^{-1}\left({\displaystyle \frac{k}{n}}\right)=\sum _{j=1}^{\infty }{\left({\displaystyle \frac{k}{n}}\right)}^{2j-1}{\displaystyle \frac{1}{2j-1}}\left({\displaystyle \frac{k}{j}}-1\right).$$

(25)

(25) clearly agrees with (18). (24) proves (a) and (b).

By the series in (25), for $n\ge 4$ and $\left|k\right|\le n^{{\displaystyle \frac{2}{3}}}$ we obtain that

$$\left|b_{k,n}^{*}-{\displaystyle \frac{k^2}{n}}\right|<2n^{-{\displaystyle \frac{1}{3}}}.$$

This and (24) imply (c). □