A Simple Wide Range Approximation of Symmetric Binomial Distribution

Tamás Szabados

doi:10.3390/e27010021

Abstract

The paper gives a wide range, uniform, local approximation of symmetric binomial distribution. The result clearly shows how one has to modify the classical de Moivre–Laplace normal approximation in order to give an estimate at the tail as well as to minimize the relative error.

Keywords:

wide range; approximation; binomial distribution

MSC:

60E15; 60F05

The topic of this paper is a wide range, uniform, local approximation of symmetric binomial distribution, an extension of the classical de Moivre–Laplace theorem in the symmetric case [1] (Ch. VII) [2]. In other words, I would like to approximate individual binomial probabilities not only in a classical neighborhood of the center, but at the tail as well. The result clearly shows how one has to modify the normal approximation in order to give a wide range estimate to minimize the relative error. The method will be somewhat similar to the ones applied by [3,4,5] or in the proof of Tusnády’s lemma, see, e.g., [6]; however, my task is much simpler than those. This simplification makes the proof short, transparent and natural. Moreover, the result is non-asymptotic, that is, it gives explicit, nearly optimal, upper and lower bounds for the relative error with a finite n. Thus, I hope that it can be used in both applications and teaching.

Let

{(X_{r})}_{r \geq 1}

be a sequence of independent, identically distributed random steps with

P (X_{r} = \pm 1) = \frac{1}{2}

and

S_{ℓ} = \sum_{r = 1}^{ℓ} X_{r}

(ℓ \geq 1)

,

S_{0} = 0

, be the corresponding simple, symmetric random walk. Then

P (S_{ℓ} = j) = (\binom{ℓ}{\frac{ℓ + j}{2}}) 2^{- ℓ} (| j | \leq ℓ) .

(1)

Here, we use the convention that the above binomial coefficient is zero whenever

ℓ + j

is not divisible by 2. Since

P (S_{ℓ} = j) = P (S_{ℓ} = - j)

for any j, it is enough to consider only the case

j \geq 0

whenever it is convenient.

First we consider the case when

ℓ = 2 n

, even. Let us introduce the notation

a_{k, n} : = P (S_{2 n} = 2 k) = (\binom{2 n}{n + k}) 2^{- 2 n} (| k | \leq n) .

Also, introduce the notation

b_{k, n} : = n \{(1 + \frac{k + \frac{1}{2}}{n}) log (1 + \frac{k}{n}) + (1 - \frac{k - \frac{1}{2}}{n}) log (1 - \frac{k}{n})\}

(2)

when

n \geq 1

and

| k | < n

, and

b_{\pm n, n} : = (2 n + \frac{1}{2}) log 2 - \frac{1}{2} log (2 π n) (n \geq 1) .

(3)

Theorem 1.

(a) For any

n \geq 1

and

| k | \leq n

, we have

a_{k, n} \leq \frac{1}{\sqrt{π n}} e^{- b_{k, n}} .

(4)

(b) For any

n \geq 1

and

| k | \leq r n

,

r \in (0, 1)

, we have

a_{k, n} > \frac{1}{\sqrt{π n}} e^{- b_{k, n}} exp (- \frac{1}{7 n} - \frac{r^{4}}{3 {(1 - r^{2})}^{2} n}) .

(5)

(c) In accordance with the classical de Moivre–Laplace normal approximation, for

n \geq 3

and

| k | \leq n^{\frac{2}{3}}

one has

1 - 2 n^{- \frac{1}{3}} < \frac{a_{k, n}}{\frac{1}{\sqrt{π n}} e^{- \frac{k^{2}}{n}}} < 1 + 2 n^{- \frac{1}{3}} .

(6)

Proof.

As usual, the first step is to estimate the central term

a_{0, n} = (\binom{2 n}{n}) 2^{- 2 n} = \frac{(2 n)!}{{(n!)}^{2}} 2^{- 2 n} .

By Stirling’s formula, see, e.g., [1] (p. 54), we have:

\sqrt{2 π n} {(\frac{n}{e})}^{n} e^{\frac{1}{12 n + 1}} < n! < \sqrt{2 π n} {(\frac{n}{e})}^{n} e^{\frac{1}{12 n}} (n \geq 1) .

(7)

Thus, after simplification we obtain

\frac{1}{\sqrt{π n}} e^{- \frac{1}{7 n}} < a_{0, n} < \frac{1}{\sqrt{π n}} e^{- \frac{1}{9 n}} (n \geq 1) .

(8)

Second, also by the standard way, for

1 \leq k \leq n

,

a_{k, n} = a_{0, n} \frac{n (n - 1) \dots (n - k + 1)}{(n + 1) (n + 2) \dots (n + k)} = a_{0, n} \frac{(1 - \frac{1}{n}) (1 - \frac{2}{n}) \dots (1 - \frac{k - 1}{n})}{(1 + \frac{1}{n}) (1 + \frac{2}{n}) \dots (1 + \frac{k}{n})} .

So it follows that

\begin{matrix} log a_{k, n} = log a_{0, n} - log (1 + \frac{k}{n}) - 2 \sum_{j = 1}^{k - 1} \frac{1}{2} log \frac{1 + \frac{j}{n}}{1 - \frac{j}{n}} \\ = log a_{0, n} - log (1 + \frac{k}{n}) - 2 \sum_{j = 1}^{k - 1} {tanh}^{- 1} (\frac{j}{n}) . \end{matrix}

(9)

To approximate the sum here, let us introduce the integral

I (x) : = \int_{0}^{x} {tanh}^{- 1} (t) d t = \frac{1}{2} (1 + x) log (1 + x) + \frac{1}{2} (1 - x) log (1 - x)

(10)

for

| x | < 1

, and its approximation by a trapezoidal sum

T_{k, n} : = \frac{1}{n} \{\frac{1}{2} {tanh}^{- 1} (0) + \sum_{j = 1}^{k - 1} {tanh}^{- 1} (\frac{j}{n}) + \frac{1}{2} {tanh}^{- 1} (\frac{k}{n})\},

(11)

where

0 \leq k < n

. It is well-known that the error of the trapezoidal formula for a function

f \in C^{2} ([a, b])

is

T_{n} (f) - \int_{a}^{b} f (t) d t = \frac{{(b - a)}^{3}}{12 n^{2}} f^{''} (x), x \in [a, b] .

Since

{({tanh}^{- 1})}^{″} (x) = 2 x {(1 - x^{2})}^{- 2}

, we obtain that

0 \leq T_{k, n} - I (\frac{k}{n}) \leq \frac{r^{4}}{6 {(1 - r^{2})}^{2} n^{2}}, when 0 \leq k \leq r n, r \in (0, 1) .

(12)

Let us combine Formulas (8)–(12):

log a_{k, n} = log a_{0, n} - 2 n T_{k, n} - \frac{1}{2} log (1 + \frac{k}{n}) - \frac{1}{2} log (1 - \frac{k}{n}),

thus

log \frac{1}{\sqrt{π n}} - b_{k, n} - \frac{1}{7 n} - \frac{r^{4}}{3 {(1 - r^{2})}^{2} n} < log a_{k, n} < log \frac{1}{\sqrt{π n}} - b_{k, n} - \frac{1}{9 n},

(13)

where

0 \leq k \leq r n

and

b_{k, n} = 2 n I (\frac{k}{n}) + \frac{1}{2} log (1 - \frac{k^{2}}{n^{2}}) .

(14)

Clearly, (14) is the same as (2). Thus, (13) proves (a) and (b) of the theorem.

Let us see now, using Taylor expansions, a series expansion of

b_{k, n}

when

| k | < n

. First, for

| x | < 1

,

I (x) = \frac{x^{2}}{1 \cdot 2} + \frac{x^{4}}{3 \cdot 4} + \frac{x^{6}}{5 \cdot 6} + \frac{x^{8}}{7 \cdot 8} + \dots .

Second, also for

| x | < 1

,

log (1 - x^{2}) = - x^{2} - \frac{x^{4}}{2} - \frac{x^{6}}{3} - \frac{x^{8}}{4} - \dots .

So by (14), for

| k | < n

we have a convergent series for

b_{k, n}

:

\begin{matrix} b_{k, n} & = \frac{k^{2}}{n} (1 - \frac{1}{2 n}) + \frac{k^{4}}{2 n^{3}} (\frac{1}{3} - \frac{1}{2 n}) + \frac{k^{6}}{3 n^{5}} (\frac{1}{5} - \frac{1}{2 n}) + \dots \\ = \sum_{j = 1}^{\infty} {(\frac{k}{n})}^{2 j} \frac{1}{j} (\frac{n}{2 j - 1} - \frac{1}{2}) . \end{matrix}

(15)

By (15), for

n \geq 3

and

| k | \leq n^{\frac{2}{3}}

we obtain that

\begin{matrix} |b_{k, n} - \frac{k^{2}}{n}| \leq \frac{n^{- \frac{2}{3}}}{2} + \sum_{j = 2}^{\infty} n^{- \frac{2}{3} j} \frac{1}{j} (\frac{n}{2 j - 1} + \frac{1}{2}) \\ \leq \frac{n^{- \frac{2}{3}}}{2} + \frac{n}{2} (\frac{1}{3} + \frac{1}{6}) \sum_{j = 2}^{\infty} n^{- \frac{2}{3} j} \leq n^{- \frac{1}{3}} . \end{matrix}

Thus, by (4), for

n \geq 3

and

| k | \leq n^{\frac{2}{3}}

,

a_{k, n} \leq \frac{1}{\sqrt{π n}} e^{- \frac{k^{2}}{n}} e^{n^{- 1 / 3}} < \frac{1}{\sqrt{π n}} e^{- \frac{k^{2}}{n}} (1 + 2 n^{- \frac{1}{3}}) .

(16)

Similarly, by (5), for

n \geq 3

and

| k | \leq n^{\frac{2}{3}}

,

\begin{matrix} a_{k, n} > \frac{1}{\sqrt{π n}} e^{- \frac{k^{2}}{n}} exp (- \frac{1}{7 n} - \frac{n^{- \frac{7}{3}}}{3 {(1 - n^{- \frac{2}{3}})}^{2}} - n^{- \frac{1}{3}}) \\ \geq \frac{1}{\sqrt{π n}} e^{- \frac{k^{2}}{n}} e^{- 1.21 n^{- 1 / 3}} > \frac{1}{\sqrt{π n}} e^{- \frac{k^{2}}{n}} (1 - 2 n^{- 1 / 3}) . \end{matrix}

(17)

(16) and (17) together prove (c) of the theorem. □

The main moral of Theorem 1 is that the exponent

k^{2} / n

in the exponent of the normal approximation is only a first approximation of the series (15). In (c), the bound

| k | \leq n^{\frac{2}{3}}

was chosen somewhat arbitrarily. It is clear from the series (15) that the bound should be

o (n^{\frac{3}{4}})

. The above given bound was picked because it seemed to be satisfactory for usual applications with large deviation and gave a nice relative error bound

2 n^{- \frac{1}{3}}

.

It is not difficult to extend the previous results to the odd-valued case of the symmetric binomial probabilities

a_{k, n}^{*} : = P (S_{2 n - 1} = 2 k - 1) = (\binom{2 n - 1}{n + k - 1}) 2^{- 2 n + 1} (- n + 1 \leq k \leq n) .

Define

b_{k, n}^{*} : = n \{(1 + \frac{k - \frac{1}{2}}{n}) log (1 + \frac{k}{n}) + (1 - \frac{k - \frac{1}{2}}{n}) log (1 - \frac{k}{n})\}

(18)

for

n \geq 1

and

| k | < n

, and

b_{n, n}^{*} : = (2 n - \frac{1}{2}) log 2 - \frac{1}{2} log (2 π n) (n \geq 1) .

(19)

Theorem 2.

(a) For any

n \geq 1

and

- n + 1 \leq k \leq n

, we have

a_{k, n}^{*} < \frac{1}{\sqrt{π n}} e^{- b_{k, n}^{*}} exp (\frac{2}{3 n}) .

(20)

(b) For any

n \geq 1

and

| k | \leq r n

,

r \in (0, 1)

, we have

a_{k, n}^{*} > \frac{1}{\sqrt{π n}} e^{- b_{k, n}^{*}} exp (- \frac{1}{n} - \frac{r^{4}}{3 {(1 - r^{2})}^{2} n}) .

(21)

(c) In accordance with the classical de Moivre–Laplace normal approximation, for

n \geq 4

and

| k | \leq n^{\frac{2}{3}}

one has

1 - 3 n^{- \frac{1}{3}} < \frac{a_{k, n}^{*}}{\frac{1}{\sqrt{π n}} e^{- \frac{k^{2}}{n}}} < 1 + 6 n^{- \frac{1}{3}} .

(22)

Proof.

Since this proof is very similar to the previous one, several details are omitted. First, by Stirling’s Formula (7), after simplifications we obtain that

\frac{1}{\sqrt{π n}} e^{- \frac{1}{n}} < a_{0, n}^{*} < \frac{1}{\sqrt{π n}} e^{\frac{2}{3 n}} (n \geq 1) .

(23)

Second, similarly to (13), for

n \geq 1

and

| k | \leq r n

,

r \in (0, 1)

, we obtain that

log \frac{1}{\sqrt{π n}} - b_{k, n}^{*} - \frac{1}{n} - \frac{r^{4}}{3 {(1 - r^{2})}^{2} n} < log a_{k, n}^{*} < log \frac{1}{\sqrt{π n}} - b_{k, n}^{*} + \frac{2}{3 n},

(24)

where

b_{k, n}^{*} : = 2 n I (\frac{k}{n}) - {tanh}^{- 1} (\frac{k}{n}) = \sum_{j = 1}^{\infty} {(\frac{k}{n})}^{2 j - 1} \frac{1}{2 j - 1} (\frac{k}{j} - 1) .

(25)

(25) clearly agrees with (18). (24) proves (a) and (b).

By the series in (25), for

n \geq 4

and

| k | \leq n^{\frac{2}{3}}

we obtain that

|b_{k, n}^{*} - \frac{k^{2}}{n}| < 2 n^{- \frac{1}{3}} .

This and (24) imply (c). □

Funding

This research received no external funding.

Institutional Review Board Statement

Not applicable.

Data Availability Statement

No new data were created or analyzed in this study. Data sharing is not applicable to this article.

Conflicts of Interest

The author declares no conflicts of interest.

References

Feller, W. An Introduction to Probability Theory and Its Applications, 3rd ed.; Wiley: New York, NY, USA, 1968; Volume I. [Google Scholar]
Révész, P. Random Walk in Random and Non-Random Environments, 2nd ed.; World Scientific: Singapore, 2005. [Google Scholar]
Feller, W. On the normal approximation to the binomial distribution. Ann. Math. Statist. 1945, 16, 319–329. [Google Scholar] [CrossRef]
Littlewood, J.E. On the probability in the tail of a binomial distribution. Adv. Appl. Prob. 1969, 1, 43–72. [Google Scholar] [CrossRef]
McKay, B.D. On Littlewood’s estimate for the binomial distribution. Adv. Appl. Prob. 1989, 21, 475–478. [Google Scholar] [CrossRef]
Massart, P. Tusnady’s lemma, 24 years later. Ann. I. H. Poincaré—PR 2002, 38, 991–1007. [Google Scholar] [CrossRef]

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

© 2024 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

Article Metrics

Citations

Article Access Statistics

Journal Statistics

Multiple requests from the same IP address are counted as one view.