Building Test Batteries Based on Analyzing Random Number Generator Tests within the Framework of Algorithmic Information Theory

Abstract

The problem of testing random number generators is considered and a new method for comparing the power of different statistical tests is proposed. It is based on the definitions of random sequence developed in the framework of algorithmic information theory and allows comparing the power of different tests in some cases when the available methods of mathematical statistics do not distinguish between tests. In particular, it is shown that tests based on data compression methods using dictionaries should be included in test batteries.

1. Introduction

Random numbers play an important role in cryptography, gambling, Monte Carlo methods and many other applications. Nowadays, random numbers are generated using so-called random number generators (RNGs), and the “quality” of the generated numbers is evaluated using special statistical tests [1]. This problem is so important for applications that there are special standards for RNGs and for so-called test batteries, that is, sets of tests. The current practice for using an RNG is to verify the sequences it generates with tests from some battery (such as those recommended by [2,3] or other standards).

Many statistical tests are designed to test some deviations from randomness described as classes of random processes (e.g., Bernoulli process with unequal probabilities 0 and 1, Markov chains with some unknown parameters, stationary ergodic processes, etc.) [1,2,3,4,5].

A natural question is: how do we compare different tests and, in particular, create a suitable battery of tests? Currently, this question is mostly addressed experimentally: possible candidate tests are applied to a set of known RNGs and the tests that reject more (“bad”) RNGs are suitable candidates for the battery. In addition, researchers try to choose independent tests (i.e., those that reject different RNGs) and take into account other natural properties (e.g., testing speed, etc.) [1,2,3,4]. Obviously, such an approach depends significantly on the set of selected tests and RNGs pre-selected for consideration. It is worth noting that at present there are dozens of RNGs and tests, and their number is growing fast, so the recommended batteries of tests are rather unstable (see [4]).

It is clear that increasing the number of tests in a battery increases the total testing time or, conversely, if testing time is limited, increasing the number of tests causes the length of the binary sequence being examined to decrease and therefore the power of any battery test is reduced. Therefore, it is highly desirable to include in the battery powerful tests designed for different deviations from randomness.

The goal of this paper is to develop a theoretical framework for test comparison and illustrate it by comparing some popular tests. The main idea of the proposed approach is based on the definition of randomness developed in algorithmic information theory (AIT). Apparently, it is natural to use this theory, since it is the only mathematically correct theory that formally defines what a random binary sequence is, and by definition any RNG should generate such sequences. Similar to AIT, we extend the notion of “random sequence” to any statistical test T, and then compare the “size” of the set of random sequences corresponding to different tests. More precisely, let $R_{T_1}$ and $R_{T_2}$ be random sequences according to $T_1$ and $T_2$. Then, if $dim(R_{T_1}\setminus R_{T_2})$ $>0$, then $T_1$ accepts a large set of sequences as random, whereas $T_2$ rejects these sequences as non-random. So, in this sense, a $T_1$ test cannot replace $T_2$ in a battery of tests (here dim is the Hausdorff dimension.).

Based on this approach, we give some practical recommendations for building test batteries. In particular, we recommend including in the test batteries a test based on a dictionary data compressor, like Lempel–Ziv codes [6], grammar-based codes [7] and some others.

The rest of this paper consists is organized as follows. The next part contains definitions and preliminary information, the third part is a comparison of the test performance on Markov processes with different memories and general stationary processes, and the fourth part investigates tests based on Lempel–Ziv data compressors. The fifth part is a brief conclusion; some of the concepts used in this paper are given in the Appendix A.

2. Definitions and Preliminaries

2.1. Hypothesis Testing

In hypothesis testing, there is a main hypothesis $H_0= \{$the sequence x is random} and an alternative hypothesis $H_1=\neg H_0$. (In the probabilistic approach, $H_0$ is that the sequence is generated by a Bernoulli source with equal probabilities 0 and 1.) A test is an algorithm for which the input is the prefix $x_1\dots x_n$ (of the infinite sequence $x_1,\dots ,x_n,\dots$) and the output is one of two possible words: random or non-random (meaning that the sequence is random or non-random, respectively).

Let there be a hypothesis $H_0$, some alternative $H_1$, let T be a test and $\tau$ be a statistic, that is, a function on ${\{0,1\}}^n$ which is applied to a binary sequence $x=x_1\dots x_n$. Here and below ${\{0,1\}}^n$ is the set of all n-bit binary words, ${\{0,1\}}^{\infty }$ is the set of all infinite words $x_1{x}_2\dots ,x_i\in \{0,1\}$.

By definition, Type I error occurs if $H_0$ is true and $H_0$ is rejected; the significance level is defined as the probability of the Type I error. Denote the critical region of the test T for the significance level $\alpha$ by ${\overline{C}}_T(\alpha ,n)$ and let $C_T(\alpha ,n)$$={\{0,1\}}^n\setminus {\overline{C}}_T(\alpha ,n).$ Recall that, by definition, $H_0$ is rejected if and only if $x\in {\overline{C}}_T(\alpha ,n)$ and, hence,

$$|{\overline{C}}_T(\alpha ,n)|\le 2^n\alpha ,$$

(1)

see [8]. We also apply another natural limitation. We consider only tests T such that for all n and ${\alpha }_1<{\alpha }_2$${\overline{C}}_T({\alpha }_1,n)\subset {\overline{C}}_T({\alpha }_2,n)$. (Here and below, $\left|X\right|$ is the number of elements X if X is a set, and the length of X, if X is a word.)

A finite sequence $x_1\dots x_n$ is considered random for a given test T and the significance level $\alpha$ if it belongs to $C_T(\alpha ,n)$.

2.2. Batteries of Tests

Let us consider a situation where the randomness testing is performed by conducting a battery of statistical tests for randomness. Suppose that the battery $\widehat{T}$ contains a finite or countable set of tests $T_1,T_2,\dots$ and ${\alpha }_i$ is the significance level of i-th test, $i=1,2,\dots$. If the battery is applied in such a way that the hypothesis $H_0$ is rejected when at least one test in the battery rejects it, then the significance level $\alpha$ of this battery satisfies the following inequality:

$$\alpha \le \sum _{i=1}^{\infty }{\alpha }_i,$$

(2)

because $P(A+B)\le P\left(A\right)+P\left(B\right)$ for any events A and B (This inequality is a simple extension of the so-called Bonferroni correction, see [9]).

It will be convenient to formulate this inequality in a different way. Suppose there is some $\alpha \in (0,1)$ and a sequence $\omega$ of non-negative ${\omega }_i$ such that ${\sum }_{i=1}^{\infty }{\omega }_i\le 1$. For example, we can define the following sequence ${\omega }^{*}$:

$${\omega }_i^{*}=1/\left(i(i+1)\right)i=1,2,\dots .$$

(3)

If the significance level $T_i$ equals $\alpha {\omega }_i$, then the significance level of the battery $\widehat{T}$ is not grater than $\alpha$. (Indeed, from (2) we obtain ${\sum }_{i=1}{\alpha }_i={\sum }_{i=1}\left(\alpha {\omega }_i\right)$$=\alpha {\sum }_{i=1}{\omega }_i\le \alpha$.) Note that this simple observation makes it possible to treat a test battery as a single test.

2.2.1. Random and Non-Random Infinite Sequences

Kolmogorov complexity is one of the central notations of algorithmic information theory (AIT), see [10,11,12,13,14,15,16,17,18]. We will consider the so-called prefix-free Kolmogorov complexity $K\left(u\right)$, which is defined on finite binary words u and is closely related to the notion of randomness. More precisely, an infinite binary sequence $x=x_1{x}_2\dots$ is random if there exists a constant C such that

$$n-K(x_1\dots x_n)<C$$

(4)

for all n, see [19]. Conversely, the sequence x is non-random if

$$\forall C>0\exists n_C{n}_C-K(x_1\dots x_{n_C})\ge C$$

In some sense, Kolmogorov complexity is the length of the shortest lossless prefix-free code, that is, for any (algorithmically realisable) code f there exists a constant $c_f$ for which $K\left(u\right)\le \left|f\left(u\right)\right|+c_f$ [10,11,12,13,14,15,16]. Recall that a code f is lossless if there is a mapping $f^{-1}$ such that for any word u$f^{-1}\left(f\left(u\right)\right)=u$ and f is prefix-free (or unprefixed) if for any words $u,v$, $f\left(u\right)$ is not a prefix of $f\left(v\right)$ and $f\left(v\right)$ is not a prefix of $f\left(u\right)$.

Let f be a lossless prefix-free code defined for all finite words. Similarly to (4), we call it random with respect to f if there is a constant $C_f$ such that

$$n-|f(x_1\dots x_n)|<C_f$$

(5)

for all n. We call this difference the statistic corresponding to f and define

$${\tau }_f(x_1\dots x_n)=n-\left|f(x_1\dots x_n)\right|.$$

(6)

Similarly, the sequence x is non-random with respect to f if

$$\forall C>0\exists n_C{n}_C-\left|f\right(x_1\dots x_{n_C}|\ge C.$$

(7)

Informally, x is random with respect to f if the statistic ${\tau }_f$ is bounded by some constant on all prefixes $x_1\dots x_n$ and, conversely, x is non-random if ${\tau }_f$ is unbounded when the prefix length grows.

Based on these definitions, we can reformulate the concepts of randomness and non-randomness in a manner similar to what is customary in mathematical statistics. Namely, for any $\alpha \in (0,1)$ we define the set $\{y=y_1\dots y_n:{\tau }_f\left(y\right)\ge -log\alpha \}$. It is easy to see that (1) is valid and, therefore, this set represents the critical region ${\overline{C}}_T(\alpha ,n)$, where the test T is as follows: $T=\{x_1\dots x_n$: ${\tau }_f(x_1\dots x_n)<\alpha \}$.

Based on these consideration, (6) and the definitions of randomness (4), (5) we give the following definition of randomness and non-randomness for the statistic ${\tau }_f$ and corresponding test $T_f$. An infinite sequence $x=x_1{x}_2\dots$ is random according to the test $T_f$ if there exists such $\alpha >0$ that for any integer n and this $\alpha$ the word $x_1\dots x_n$ is random (according to the $T_f$ test). Otherwise, the sequence x is non-random.

Note that we can use the statistic

$${\tau }_f=n-\left|f(x_1\dots x_n)\right|$$

with the critical value $t_{\alpha }=n-log(1/\alpha )-1$, $\alpha \in (0,1)$, see [20,21]. So, there is no need to use the density distribution formula and it greatly simplifies the use of the test and makes it possible to use this test for any data compressor f.

It is important to note that there are tests developed within the AIT that can be used to test RNG [22,23].

2.2.2. Test Performance Comparison

For test T, let us define the set $R_T$ of all infinite sequences that are random for T.

We use this definition to compare the “effectiveness” of different tests as follows. The test $T_1$ is more efficient than $T_2$ if the size of the difference $R_{T_2}\setminus R_{T_1}$ is not equal to zero, where the size is measured by the Hausdorff dimension.

Informally, the “smallest” set of random sequences corresponds to a test based on Kolmogorov complexity (4) (corresponding set $R_K$ contains “truly” random sequences). For a given test $T_1$ we cannot calculate the difference $R_{T_1}\setminus R_K$ because the statistic (4) is noncomputabele, but in the case of two tests $T_1$ and $T_2$, where $dim(R_{T_2}\setminus R_{T_1})>0$, we can say that the set of sequences random according to $T_2$ contains clearly non-random sequences. So, in some sense, $T_1$ is more efficient than $T_2$. (Recall that we only consider computable tests.)

The definition of the Hausdorff dimension is given in the Appendix A, but here we briefly note that we use the Hausdorff dimension for it as follows: for any binary sequence $x_1{x}_2\dots$ we define a real number $\sigma \left(x\right)=0.$$x_1{x}_2\dots$ and for any set of infinite binary sequences S we denote the Hausdorff dimension of $\sigma \left(S\right)$ by $dimS$. So, a test $T_1$ is more efficient than $T_2$ (formally $T_1⪰T_2$) if $dim(R_{T_2}\setminus R_{T_1})>0$. Obviously, information about a test’s effectiveness can be useful to developers of the test’s batteries.

Also note that the Hausdorff dimension is widely used in information theory. Perhaps the first such use was due to Eggleston [24] (see also [25,26]), and later the Hausdorff dimension found numerous applications in AIT [27,28,29].

2.2.3. Shannon Entropy

In RNG testing, one of the popular alternative hypotheses ($H_1$) is that the considered sequence generated by Markov process of memory (or connectivity) $m,m>0,$ $\left(S_m\right)$, but the transition probabilities are unknown. ($S_0$, i.e., $m=0$, corresponds to the Bernoulli process). Another popular and perhaps the most general $H_1$ is that the sequence is generated by a stationary ergodic process ($S_{\infty }$) (excluding $H_0$).

Let us consider the Bernoulli process $\mu \in S_0$ for which $\mu \left(0\right)=p,\mu \left(1\right)=q,(p+q=1).$ By definition, the Shannon entropy $h\left(\mu \right)$ of this process is defined as $h\left(\mu \right)=-(plogp+qlogq)$ [30]. For any stationary ergodic process $\nu \in S$ the entropy of order k is defined as follows:

$$h_k\left(\nu \right)=E_{\nu }(-\sum _{u\in {\{0,1\}}^k}(\nu (0/u)log(0/u)+\nu (1/u)log\nu (1/u))),$$

where $E_{\nu }$ is the mathematical expectation according to $\nu$, $\nu (z/u)$ is the conditional probability $\nu (x_{i+1}=z|x_{i-k}\dots x_i=u)$, it does not depend on i due to stationarity [30].

It is known in Information Theory that for stationary ergodic processes (including $S_{\infty }$ and $S_m,m\ge 0)$ $h_k\ge h_{k+1}$ for $k\ge 0$ and there exists the limit Shannon entropy $h_{\infty }\left(\nu \right)=lim{h}_k\left(\nu \right)$. Besides, for $\nu \in S_m$$h_{\infty }=h_m$ [30].

Shannon entropy plays an important role in data compression because for any lossless and prefix-free code, the average codeword length (per letter) is at least as large as the entropy, and this limit can be reached. More precisely, let $\varphi$ be a lossless, prefix-free code defined on ${\{0,1\}}^n,n>0$, and let $\nu \in S$. Then, for any $\varphi$, $\nu$, and codewords of average length

$$E_n(\varphi ,\nu )=\frac{1}{n}\sum _{u\in {\{0,1\}}^n}\nu \left(u\right)\left|\varphi \left(u\right)\right|$$

(8)

$E_n(\varphi ,\nu )\ge h\left(\nu \right)$. In addition, there are codes ${\varphi }_1,{\varphi }_2,\dots$ such that ${lim}_{n\to \infty }{E}_n({\varphi }_n,\nu )=h\left(\nu \right)$ [30].

2.2.4. Typical Sequences and Universal Codes

The sequence $x_1{x}_2\dots$ is typical for the measure $\mu \in S_{\infty }$ if for any word $y_1\dots y_r$${lim}_{t\to \infty }{N}_{x_1\dots x_t}(y_1\dots y_r)/t=\mu \left(u\right)$, where $N_{x_1\dots x_t}(y_1\dots y_r)$ is the number of occurrences of a word $y_1\dots y_r$ in a word $x_1\dots x_t$.

Let us denote the set of all typical sequences as ${\mathbf{T}}_{\mu }$ and note that $\mu \left({\mathbf{T}}_{\mu }\right)=1$ [30]. This notion is deeply related to information theory. Thus, Eggleston proved the equality $dim{\mathbf{T}}_{\mu }=h\left(\mu \right)$ for Bernoulli processes ($\mu \in S_0$) [24], and later this was generalized for $\mu \in S_{\infty }$ [26,28].

By definition, a code $\varphi$ is universal for a set of processes S if for any $\mu \in S$ and any $x\in {\mathbf{T}}_{\mu }$

$$\underset{n\to \infty }{lim}\left|\varphi (x_1\dots x_n)\right|/n=h_{\infty }\left(\mu \right).$$

(9)

In 1968, R. Krichevsky [31] proposed a code ${\kappa }_m^t(x_1\dots x_t)$, $m\ge 0,t$ is an integer, whose redundancy, i.e., the average difference between the code length and Shannon entropy, is asymptotically minimal. This code and its generalisations are described in the Appendix A, but here we note the following main property. For any stationary ergodic process $\mu$, that is, $\mu \in S_{\infty }$ and typical $x\in {\mathbf{T}}_{\mu }$,

$$\underset{t\to \infty }{lim}\left|{\kappa }_m^t(x_1\dots x_t)\right|/t=h_m\left(\mu \right),$$

(10)

see [32].

Currently there are many universal codes which are based on different ideas and approaches, among which we note the PPM universal code [33], the arithmetic code [34], the Burrows–Wheeler transform [35], which is used along with the book-stack (or MTF) code [36,37,38], and some others [39,40,41].

The most interesting for us is the class of grammar-based codes suggested by Kieffer and Yang [7,42] which includes the Lempel–Ziv (LZ) codes [6] (note that perhaps the first grammar-based code was described in [43]).

The point is that all of them are universal codes and hence they “compress” stationary processes asymptotically to entropy and therefore cannot be distinguishable at $S_{\infty }$. On the other hand, we show that grammar-based codes can distinguish “large” sets of sequences as non-random beyond $S_{\infty }$.

2.2.5. Two-Faced Processes

The so-called two-faced processes are described in [20,21] and their definitions will be given in Appendix A. Here, we note some of their properties: the set of two-faced processes ${\mathsf{\Lambda }}_s\left(p\right)$ of order $s,s\ge 1$, and probability $p,p\in (0,1)$, contains the measures $\lambda$ from $S_s$ such that

$$h_0\left(\lambda \right)=h_1\left(\lambda \right)=\dots =h_{s-1}\left(\lambda \right)=1,$$

$$h_s\left(\lambda \right)=h_{\infty }\left(\lambda \right)=-(plogp+(1-p)log(1-p)).$$

(11)

Note that they are called two-faced because they appear to be truly random if we look at word frequencies whose length is less than s, but are “completely” non-random if the word length is equal to or greater than s (and p is far from $1/2$).

3. Comparison of the Efficiency of Tests for Markov Processes with Different Memories and General Stationary Processes

We now describe the statistical tests for Markov processes and stationary ergodic processes as follows. By (6), statistical definitions are as follows:

$${\tau }_{K_m^t}(x_1\dots x_n)=n-\left|{\widehat{\kappa }}_m^t(x_1\dots x_n)\right|,$$

$${\tau }_{R^t}(x_1\dots x_n)=n-\left|{\widehat{\rho }}^t(x_1\dots x_n)\right|$$

where ${\widehat{\kappa }}_m^t$ and ${\widehat{\rho }}^t$ are universal codes for $S_m$ and $S_{\infty }$ defined in the Appendix A, see (A4) and (A5). We also denote the corresponding tests by $T_{K_m}^t$ and $T_R^t$. The following statement compares the performance of these tests.

Theorem 1.

For any integers $m,s$ and $t=ms$

$$T_{K_m}^t⪯T_{K_{m+1}}^t,T_{K_m}^t⪯T_{K_R}^t.$$

Moreover, $dim(T_{K_m}^t\setminus T_{K_{m+1}}^t)=1$.

Proof. 

First, let us say a few words about the scheme of the proof. If we apply the $T_{K_m}^t$ test to typical sequences of a two-faced process $\lambda \in {\mathbf{T}}_{{\mathsf{\Lambda }}_{m+1\left(p\right)}},$ $p\ne 1/2$, they will appear random since $h_m\left(\lambda \right)=1$. So, the set of random sequences $R_{T_{K_m}^t}$ (i.e., random sequences according to $T_{K_m}^t$ test) contains the set of the typical sequences ${\mathbf{T}}_{{\mathsf{\Lambda }}_{m+1\left(p\right)}}$ for which $dim\left({\mathbf{T}}_{{\mathsf{\Lambda }}_{m+1\left(p\right)}}\right)$ equals the limit Shannon entropy $-(plogp+(1-p)log(1-p\left)\right)$. Hence, $dim\left(R_{T_{K_m}^t}\right)\ge$$dim\left({\mathbf{T}}_{{\mathsf{\Lambda }}_{m+1\left(p\right)}}\right)=$$-(plogp+(1-p)log(1-p\left)\right)$.

On the other hand, typical sequences of a two-faced process $\lambda \in {\mathbf{T}}_{{\mathsf{\Lambda }}_{m+1\left(p\right)}},p\ne 1/2$ are not random according to $T_{K_{m+1}}^t$ since $h_{m+1}\left(\lambda \right)$$=-(plogp+(1-p)log(1-p\left)\right)<1$ (11) and the test. $T_{K_m}^t$ “compresses” them till the Shannon entropy $-(plogp+(1-p)log(1-p\left)\right)$. So, $dim(R_{T_{K_m}^t}\setminus R_{T_{K_{m+1}}^t})\ge$$dim\left(R_{T_{K_m}^t}\right)$$\ge -(plogp+(1-p)log(1-p\left)\right)$. Then ${sup}_{p\in (0,1/2)}$$dim(T_{K_m}^t\setminus T_{K_{m+1}}^t)=1$.

More formally, consider a typical sequence x of ${\mathbf{T}}_{{\mathsf{\Lambda }}_{m+1\left(p\right)}}$, $p\ne 1/2$. So, ${lim}_{t\to \infty }-{\sum }_{u\in {\{0,1\}}^{m+1}}$$(N_{x_1\dots x_t}\left(u\right)/t)log(N_{x_1\dots x_t}\left(u\right)/t)=h_{\lambda }\left(m\right)=1,$ see (11), where the first equality is due to typicality, and the second to the property of two-faced processes (11).

From here and (A1), (A4) we obtain $E_{\lambda }(1/n)\left|{\widehat{\kappa }}_m^t(x_1\dots x_n)\right|$$=1+ϵ$, where $ϵ>0$. From this and typicality we can see that ${lim}_{n\to \infty }\left|{\widehat{\kappa }}_m^t(x_1\dots x_n)\right|/n$ $=1+ϵ$. Hence, there exists such $n_{\delta }$ that $1+ϵ-\delta <|{\widehat{\kappa }}_m^t(x_1\dots x_n)|/n<1+ϵ+\delta$, if $n>n_{\delta }$. So $n-|{\widehat{\kappa }}_m^t(x_1\dots x_n)|$$\le n-(n+ϵ-\delta )$. So, if we take $\delta =ϵ/2$, we can see that for $n>n_{\delta }$$n-|{\widehat{\kappa }}_m^t(x_1\dots x_n)|$ is negative. From this and the definition of randomness (5), we can see that typical sequences from ${\mathbf{T}}_{{\mathsf{\Lambda }}_{m+1\left(p\right)}}$ are random according to ${\widehat{\kappa }}_m^t(x_1\dots x_n)$, i.e., $T_{K_m}^t$. From this and (A6), we obtain $T_{K_{m+1}}^t⪯T_R^t$. □

4. Effectiveness of Tests Based on Lempel-Ziv Data Compressors

In this part we will describe a test that is more effective than $T_R^t$ and $T_{K_m}^t$ for any m.

First, we will briefly describe the LZ77 code based on the definition in [44]. Suppose there is a binary string ${\sigma }^{*}$ that is encoded using the code LZ77. This string is represented by a list of pairs $(p_1;l_1)\dots (p_s;l_s)$. Each pair $(p_i;l_i)$ represents a string, and the concatenation of these strings is ${\sigma }^{*}$. In particular, if $p_i=0$, then the pair represents the string $l_i$, which is a single terminal. If $p_i\ne 0$, then the pair represents a portion of the prefix of ${\sigma }^{*}$ that is represented by the preceding $i-1$ pairs; namely, the $l_i$ terminals beginning at position $p_i$ in ${\sigma }^{*}$; see ([44] part 3.1). The length of the codeword depends on the encoding of the sub-words $p_i,l_i$ which are integers. For this purpose we will use a prefix code C for integers, for which for any integer m

$$\left|C\right(m\left)\right|=logm+2loglog(m+1)+O\left(1\right).$$

(12)

Such codes are known in information theory; see, for example, ([30] part 7.2). Note that C is the prefix code and, hence, for any $r\ge 1$ the codeword $C\left(p_1\right)C\left(l_1\right)\dots C\left(p_r\right)C\left(l_r\right)$ can be decoded to $(p_1;l_1)\dots (p_r;l_r)$. There is the following upper bound for the length of the LZ77 code [30,44]: for any word $w_1{w}_2\dots .w_m$

$$|cod{e}_{LZ}(w_1{w}_2\dots w_m)|\le m(1+o\left(1\right)),$$

(13)

if $m\to \infty$.

We will now describe such sequences that, on the one hand, are not typical for any stationary ergodic measure and, on the other hand, are not random and will be rejected by the suggested test. Thus, the proposed model allows us to detect non-random sequences that are not typical for for any stationary processes.On the other hand, those sequences are recognized tests based on LZ77 as non-random. To do this, we take any random sequence $x=x_1{x}_2\dots$ (that is, for which (4) is valid) and define a new sequence $y\left(x\right)=y_1{y}_2\dots$ as follows. Let for $k=0,1,2,\dots$

$$u_k=x_{2^{2^k}-1}{x}_{2^{2^k}}{x}_{2^{2^k}+1}\dots x_{2^{2^{k+1}}-2}$$

$$y\left(x\right)=u_0{u}_0{u}_1{u}_1{u}_2{u}_2{u}_3{u}_3\dots$$

(14)

For example, $u_0=x_1{x}_2$, $u_1=x_3$$x_4$…$x_{14}$, $u_2=x_{15}\dots x_{254}$, $y\left(x\right)=x_1{x}_2$$x_1{x}_2{x}_3{x}_4\dots x_{14}$$x_3{x}_4\dots x_{14}$ $x_{15}\dots x_{254}$$x_{15}\dots x_{254}$….

The idea behind this sequence is quite clear. Firstly, it is obvious that the word y cannot be typical for a stationary ergodic source and, secondly, when $u_0{u}_0{u}_1{u}_1\dots u_k{u}_k$ is encoded the second subword $u_k$ will be encoded by a very short word (about $O(log|u_k\left|\right))$, since it coincides with the previous word $u_k$. So, for large k the length of the encoded word $LZ(u_0{u}_0{u}_1{u}_1\dots u_k{u}_k)$ will be about $|u_0{u}_0{u}_1{u}_1\dots u_k{u}_k|(1/2+o\left(1\right))$. So, $lim{inf}_{n\to \infty }\left|LZ(y_1{y}_2\dots y_n)\right|/n=1/2$. Hence, it follows that

$$dim\left(\right\{y\left(x\right):xisrandom\left\}\right)=1/2.$$

(15)

Here, we took into account that x is random and, $dim\{x:xisrandom\}=1$, see [28].) So, having taken into account the definitions of non-randomness (6) and (7), we can see that $y\left(x\right)$ is non random according to statistics $\tau =n-\left|LZ\right(y_1\dots y_n\left)\right|$. Denote this test by $T_{LZ}$.

Let us consider the test $T_{K_m}^t$, $m,t$ are integers. Having taken into account that the sequence x is random, we can see that ${lim}_{t\to \infty }$$|{\kappa }_m^t(x_i{x}_{i+1}\dots x_{i+t}|/t=1.$ So, from from (A4) we can see that for any n $|{\widehat{\kappa }}_m^t(x_1\dots x_n|/t=1+o\left(1\right)$. The same reasoning is true for the code ${\widehat{\rho }}^t$.

We can now compare the size of random sequence sets across different tests as follows:

$$R_{T_{K_m}^t}\setminus R_{T_{LZ}}\supset \{y\left(x\right):xisrandom\}.$$

Taking into account (15), we can see that

$$dim(R_{T_{K_m}^t}\setminus R_{T_{LZ}})\ge 1/2.$$

Likewise, the same is true for the $T_R$ test. From the latest inequality we obtain the following.

Theorem 2.

For any random (according to (4)) sequence x the sequence $y\left(x\right)$ is non-random for the test $T_{LZ}$, whereas this sequence is random for tests $T_R^t$ and $T_{K_m}^t$. Moreover, $T_R^t⪯T_{LZ}$ and $T_{K_m}^t⪯T_{LZ}$ for any $m,t$.

Comment. The sequence $y\left(x\right)$ is constructed by duplicating parts of x. This construction can be slightly modified as follows: instead of duplication (as $u_i{u}_i$), we can use $u_i{u}_i^{\gamma }$, where $u_i^{\gamma }$ contains $\gamma \left|u\right|$ the first letters of u, $\gamma <1/2$. In this case, $dim(R_{T_{K_m}^t}\setminus R_{T_{LZ}})\ge 1-\gamma$ and, therefore,

$$\underset{\gamma \in (0,1/2)}{sup}dim(R_{T_{K_m}^t}\setminus R_{T_{LZ}})=1.$$

5. Conclusions

Here, we describe some recommendations for the practical testing of RNGs, based on the method of comparing the power of different statistical tests. Based on Theorem 1, we can recommend to use several tests $T_{K_s^t}$, based on the analysis of occurrence frequencies of words of different length s. In addition, we recommend using tests for which s depends on the length n of the sequence under consideration. For example, $s_1=O(loglogn)$), $s_2=O(\sqrt{logn}$), etc. They can be included in the test battery directly or as the “mixture” $T_R$ with several non-zero $\beta$ coefficients, see (A2) in the Appendix A.

Theorem 2 shows that it is useful to include tests based on dictionary data compressors such as the Lempel–Ziv code. In such a case we can use the statistic

$${\tau }_{LZ}=n-\left|LZ(y_1\dots y_n)\right|$$

with the critical value $t_{\alpha }=n-log(1/\alpha )-1$, $\alpha \in (0,1)$, see [20,21]. Note that in this case, there is no need to use the density distribution formula, which greatly simplifies the use of the test and makes it possible to use a similar test for any grammar-based data compressor.