On the Entropy and Letter Frequencies of Powerfree Words

Abstract

We review the recent progress in the investigation of powerfree words, with particular emphasis on binary cubefree and ternary squarefree words. Besides various bounds on the entropy, we provide bounds on letter frequencies and consider their empirical distribution obtained by an enumeration of binary cubefree words up to length 80.

1. Introduction

The interest in combinatorics on words goes back to the work of Axel Thue at the beginning of the 20th century [1]. He showed, in particular, that the famous morphism

$$\rho :\begin{array}{ccc}0& \mapsto & 01\\ 1& \mapsto & 10\end{array},$$

(1)

called Thue-Morse morphism since the work of Morse [2], is cubefree. Its iteration on the initial word 0 produces an infinite cubefree word

$$0110100110010110100101100110100110010110011010010110100110010110\dots$$

over a binary alphabet, which means that it does not contain any subword of the form $0^3=000$, $1^3=111$, ${\left(01\right)}^3=010101$, ${\left(10\right)}^3=101010$ and so forth. Moreover, the statement that the morphism is cubefree means that it maps any cubefree word to a cubefree word, so it preserves this property. Generally, the iteration of a powerfree morphism is a convenient way to produce infinite powerfree words.

The investigation of powerfree, or more generally of pattern-avoiding words, is one particular aspect of combinatorics on words; we refer the reader to the book series [3,4,5] for a comprehensive overview of the area, including algebraic formulations and applications. The area has attracted considerable activity in the past decades [6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24], and continues to do so, see [25,26,27,28,29,30,31,32,33] for some recent work. Beyond the realm of combinatorics on words and coding theory, substitution sequences, such as the Thue-Morse sequence, have been investigated for instance in the context of symbolic dynamics [34,35,36] and aperiodic order [37], to name but two. In the latter case, one is interested in systems which display order without periodicity, and substitution sequences often provide paradigmatic models, which are used in many applications in physics and materials science. However, sequences produced by a substitution such as in Eq. (1) have subexponential complexity and hence zero combinatorial entropy, cf. Definition 12 below. A natural generalisation to interesting sets of positive entropy is provided by powerfree or pattern-avoiding words.

In this article, we review the recent progress on powerfree words, with emphasis on the two ‘classic’ cases of binary cubefree and ternary squarefree words. We include a summary of relevant results which are scattered over 25 years of literature, and also discuss some new results as well as conjectures on cubefree morphisms and letter frequencies in binary cubefree words.

The first term of interest is the combinatorial entropy of the set of powerfree words. Due to the fact that every subword of a powerfree word is again powerfree, the entropy of powerfree words exists as a limit. It is a measure for the exponential growth rate of the number of powerfree words of length n. Unfortunately, neither an explicit expression for the entropy of k-powerfree words nor an easy way to compute it numerically is known. Nevertheless, there are several strategies to derive upper and lower bounds for this limit. Upper bounds can be obtained, for example, by enumeration of all powerfree words up to a certain length, or by the derivation of generating functions for the number of powerfree words, see Section 4. Until recently all methods to achieve lower bounds relied on powerfree morphisms. However, the lower bounds obtained in this way are not particularly good, since they are considerably smaller than the upper bounds as well as reliable numerical estimates of the actual value of the entropy. A completely different approach introduced recently by Kolpakov [29], which amounts to choosing a parameter value to satisfy a number of inequalities derived from a Perron-Frobenius-type argument, provides surprisingly good lower bounds for the entropy of ternary squarefree and binary cubefree words.

In the following section, we briefly introduce the notation and basic terminology; see [3] for a more detailed introduction. We continue with a summary of results on k-powerfree morphisms, which can be used to derive lower bounds for the corresponding entropy. We then proceed by introducing the entropy of k-powerfree words and summarise the methods to derive upper and lower bounds in general, and for binary cubefree and ternary squarefree words in particular. We conclude with a discussion of the frequencies of letters in binary cubefree and ternary squarefree words.

2. Powerfree words and morphisms

Define an alphabet $\mathcal{A}$ as a finite non-empty set of symbols called letters. The cardinality of $\mathcal{A}$ is denoted by $\text{Card}\left(\mathcal{A}\right)$. Finite or infinite sequences of elements from $\mathcal{A}$ are called words. The empty word is denoted by ε. The set of all finite words, the operation of concatenation of words and the empty word ε form the free monoid ${\mathcal{A}}^{*}$. The free semigroup generated by $\mathcal{A}$ is ${\mathcal{A}}^{+}:=\mathcal{A}\setminus \{\epsilon \}$.

The length of a word $u\in {\mathcal{A}}^{*}$, denoted by $\left|u\right|$, is the number of letters that u consists of. The length of the empty word is $|\epsilon |:=0$.

For two words $u,v\in {\mathcal{A}}^{*}$, we say that v is a subword or a factor of u if there are words $x,y\in {\mathcal{A}}^{*}$ such that $u=xvy$. If $x=\epsilon$, the factor v is called a prefix of u, and if $y=\epsilon$, v is called a suffix of u. Given a set of words $X\subset {\mathcal{A}}^{*}$ (here and in what follows, the symbol ⊂ is meant to include the possibility that both sets are equal), the set of all factors of words in X is denoted by $\text{Fact}\left(X\right)$.

A map $\rho :{\mathcal{A}}^{*}\to {\mathcal{B}}^{*}$, where $\mathcal{A}$ and $\mathcal{B}$ are alphabets, is called a morphism if $\rho \left(uv\right)=\rho \left(u\right)\rho \left(v\right)$ holds for all $u,v\in {\mathcal{A}}^{*}$. Obviously, a morphism ρ is completely determined by $\rho \left(a\right)$ for all $a\in \mathcal{A}$, and satisfies $\rho (\epsilon )=\epsilon$. A morphism $\rho :{\mathcal{A}}^{*}\to {\mathcal{B}}^{*}$ is called ℓ-uniform, if $\left|\rho \right(a\left)\right|=\ell$ for all $a\in \mathcal{A}.$

For a word u, we define $u^0:=\epsilon$, $u^1:=u$ and, for an integer $k>1$, the power $u^k$ as the concatenation of k occurrences of the word u. If $u\ne \epsilon$, $u^k$ is called a k-power. A word v contains a k-power if at least one of its factors is a k-power. If a word does not contain any k-power as a factor, it is called k-powerfree. If a word does not contain the k-power of any word up to a certain length p as a factor, it is called length-p k-powerfree, i.e., $w=x{u}^{k}y$ implies that $u=\epsilon$ whenever $x,u,y\in {\mathcal{A}}^{*}$ with $\left|u\right|\le p$. We denote the set of k-powerfree words in an alphabet $\mathcal{A}$ by $F^{\left(k\right)}\left(\mathcal{A}\right)\subset {\mathcal{A}}^{*}$ and the set of length-p k-powerfree by $F^{(k,p)}\left(\mathcal{A}\right)\subset {\mathcal{A}}^{*}$. By definition, the empty word ε is k-powerfree for all k. A word $w\in {\mathcal{A}}^{*}$ is called primitive, if $w=v^n$, with $v\in {\mathcal{A}}^{*}$ and $n\in \mathbb{N}$, implies that $n=1$, meaning that w is not a proper power of another word v.

A morphism $\rho :{\mathcal{A}}^{*}\to {\mathcal{B}}^{*}$ is called k-powerfree, if $\rho \left(u\right)$ is k-powerfree for every k-powerfree word u. In other words, ρ is powerfree if $\rho \left(F^{\left(k\right)}\left(\mathcal{A}\right)\right)\subset F^{\left(k\right)}\left(\mathcal{B}\right)$. A test-set for k-powerfreeness of morphisms on an alphabet $\mathcal{A}$ is a set $T\subset {\mathcal{A}}^{*}$ such that, for any morphism $\rho :{\mathcal{A}}^{*}\to {\mathcal{B}}^{*}$, ρ is k-powerfree if and only if $\rho \left(T\right)$ is k-powerfree. A morphism is called powerfree if it is a k-powerfree morphism for every $k\ge 2$.

In particular, 2-powerfree and 3-powerfree words and morphisms are called squarefree and cubefree, respectively. A morphism from ${\mathcal{A}}^{*}$ to ${\mathcal{B}}^{*}$ with $\text{Card}\left(A\right)=2$ is also called a binary morphism. The notion of powerfreeness can be extended to non-integer powers; see, for instance, Ref. [25] for an investigation of k-powerfree binary words for $k\ge 2$. However, in this article we shall concentrate on the cases $k=2$ and $k=3$, and hence restrict the discussion to integer powers.

3. Characterisations of k-powerfree morphisms

In what follows, we summarise a number of relevant results on k-powerfree morphisms. In particular, we are interested in the question how to test a specified morphism for k-powerfreeness. We start with results relating to the case $k=2$.

3.1. Characterisations of squarefree morphisms

A sufficient (but in general not necessary) condition for the squarefreeness of a morphism is known since 1979.

Theorem 1

(Bean et al. [8]). A morphism $\rho :{\mathcal{A}}^{*}\to {\mathcal{B}}^{*}$ is squarefree if

(i)

$\rho \left(w\right)$ is squarefree for every squarefree word $w\in {\mathcal{A}}^{*}$ of length $\left|w\right|\le 3$;

(ii)

$a=b$ whenever $a,b\in \mathcal{A}$ and $\rho \left(a\right)$ is a factor of $\rho \left(b\right)$.

If the morphism ρ is uniform, this condition is in fact also necessary, because in this case $\rho \left(a\right)$ being a factor of $\rho \left(b\right)$ implies that $\rho \left(a\right)=\rho \left(b\right)$. If $a,b\in \mathcal{A}$ exist with $a\ne b$ and $\rho \left(a\right)=\rho \left(b\right)$, then clearly ρ is not squarefree since $\rho \left(ab\right)=\rho \left(a\right)\rho \left(b\right)$ is a square. This gives the following corollary.

Corollary 2.

A uniform morphism $\rho :{\mathcal{A}}^{*}\to {\mathcal{B}}^{*}$ is squarefree if and only if $\rho \left(w\right)$ is squarefree for every squarefree word $w\in {\mathcal{A}}^{*}$ of length $\left|w\right|\le 3$.

This corollary corresponds to Brandenburg’s Theorem 2 in Ref. [11] which only demands that $\rho \left(w\right)$ is squarefree for every squarefree word $w\in {\mathcal{A}}^{*}$ of length exactly 3. A short calculation reveals that this condition is equivalent to (i), because every squarefree word of length smaller than 3 occurs as a factor of a squarefree word of length 3.

For the next characterisation, we need the notion of a pre-square with respect to a morphism ρ. Let $\mathcal{A}$ be an alphabet, $w\in {\mathcal{A}}^{*}$ a squarefree word and $\rho :{\mathcal{A}}^{*}\to {\mathcal{B}}^{*}$ a morphism. A factor $u\ne \epsilon$ of $\rho \left(w\right)=\alpha u\beta$ is called a pre-square with respect to ρ, if there exists a word $w^{\prime }\in {\mathcal{A}}^{*}$ satisfying: $w{w}^{\prime }$ is squarefree and u is a prefix of $\beta \rho \left(w^{\prime }\right)$ or $w^{\prime }w$ is squarefree and u is a suffix of $\rho \left(w^{\prime }\right)\alpha$. Obviously, if u is a pre-square, then either $\rho \left(w{w}^{\prime }\right)$ or $\rho \left(w^{\prime }w\right)$ contains $u^2$ as a factor.

Theorem 3

(Crochemore [9]). A morphism $\rho :{\mathcal{A}}^{*}\to {\mathcal{B}}^{*}$ is squarefree if and only if

(i)

$\rho \left(w\right)$ is squarefree for every squarefree word $w\in {\mathcal{A}}^{*}$ of length $\left|w\right|\le 3$;

(ii)

for any$a\in \mathcal{A}$, $\rho \left(a\right)$ does not have any internal pre-squares.

It follows that, for a ternary alphabet $\mathcal{A}$, a finite test-set exists, as specified in the following corollary. However, the subsequent theorem shows that, as soon as we consider an alphabet with $\text{Card}\left(\mathcal{A}\right)>3$, no such finite test-sets exist, so the situation becomes more complex when considering larger alphabets.

Corollary 4

(Crochemore [9]). Let $\text{Card}\left(\mathcal{A}\right)=3$. A morphism $\rho :{\mathcal{A}}^{*}\to {\mathcal{B}}^{*}$ is squarefree if and only if $\rho \left(w\right)$ is squarefree for every squarefree word $w\in {\mathcal{A}}^{*}$ of length $\left|w\right|\le 5$.

Theorem 5

(Crochemore [9]). Let $\text{Card}\left(\mathcal{A}\right)>3$. For any integer n, there exists a morphism $\rho :{\mathcal{A}}^{*}\to {\mathcal{B}}^{*}$ which is not squarefree, but maps all squarefree words of length up to n on squarefree words.

3.2. Characterisations of cubefree and k-powerfree morphisms

We now move on to characterisations of cubefree and k-powerfree morphisms for $k>3$. We start with a recent result on cubefree binary morphisms.

Theorem 6

(Richomme, Wlazinski [23]). A set $T\subset {\{a,b\}}^{*}$ is a test-set for cubefree morphisms from ${\mathcal{A}}^{*}={\{a,b\}}^{*}$ to ${\mathcal{B}}^{*}$ with $\text{Card}\left(\mathcal{B}\right)\ge 2$ if and only if T is cubefree and $\text{Fact}\left(T\right)\supset T_{\min }$, where

$$T_{\min }:=\{abbabba,baabaab,ababba,babaab,abbaba,baabab,aabba,bbaab,abbaa,baabb,ababa,babab\}.$$

Obviously, the set $T_{\min }$ itself is a test-set for cubefree binary morphisms. Another test-set is the set of cubefree words of length 7, as each word of $T_{\min }$ appears as a factor of this set. There are even single words which contain all the elements of $T_{\min }$ as factors. For instance, the cubefree word $aabbababbabbaabaababaabb$ is one of the 56 words of length 24 which are test-sets for cubefree morphisms on $\{a,b\}$. The length of this word is optimal: no cube-free word of length 23 contains all the words of $T_{\min }$ as factors.

The following sufficient characterisation of k-powerfree morphisms generalises Theorem 1 to integer powers $k>2$.

Theorem 7

(Bean et al. [8]). Let $\rho :{\mathcal{A}}^{*}\to {\mathcal{B}}^{*}$ be a morphism for alphabets $\mathcal{A}$ and $\mathcal{B}$ and let $k>2$. Then ρ is k-powerfree if

(i)

$\rho \left(w\right)$ is k-powerfree whenever $w\in {\mathcal{A}}^{*}$ is k-powerfree and of length $\left|w\right|\le k+1$;

(ii)

$a=b$ whenever $a,b\in \mathcal{A}$ with $\rho \left(a\right)$ a factor of $\rho \left(b\right)$;

(iii)

the equality $x\rho \left(a\right)y=\rho \left(b\right)\rho \left(c\right)$, with $a,b,c\in \mathcal{A}$ and $x,y\in {\mathcal{B}}^{*}$, implies that either $x=\epsilon ,a=b$ or $y=\epsilon ,a=c$.

As in the squarefree case above, a uniform morphism ρ for which (i) holds also meets (ii), because uniformity implies that $\rho \left(a\right)=\rho \left(b\right)$. If $a\ne b$, the word $a^{k-1}b$ is k-powerfree but $\rho \left(a^{k-1}b\right)=\rho {\left(a\right)}^k$ is a k-power, which produces a contradiction. The condition (iii) means that, for all letters $a\in \mathcal{A}$, the images $\rho \left(a\right)$ do not occur as an inner factor of $\rho \left(bc\right)$ for any $b,c\in \mathcal{A}$. In general, this is not necessary for uniform morphisms; an example is given by the Thue-Morse morphism ρ of Eq. (1). For instance, $\rho \left(00\right)=0101=0\rho \left(1\right)1$, which violates condition (iii) in Theorem 7. Nevertheless, the Thue-Morse morphism is cubefree [1].

Alphabets with $\text{Card}\left(\mathcal{B}\right)<2$ only provide trivial results, because the only k-powerfree morphism from ${\mathcal{A}}^{*}$ to ${\{\epsilon \}}^{*}$ is the empty morphism ε, and for $\text{Card}\left(\mathcal{B}\right)=1$ the only additional morphism is the map for $\text{Card}\left(\mathcal{A}\right)=1$ that maps the single element in $\mathcal{A}$ to the single letter in $\mathcal{B}$. From now on, we consider alphabets with $\text{Card}\left(\mathcal{B}\right)\ge 2$. First, we deal with the case $\text{Card}\left(\mathcal{A}\right)\ge 3$.

Theorem 8

(Richomme, Wlazinski [23]). Given two alphabets $\mathcal{A}$ and $\mathcal{B}$ such that $\text{Card}\left(\mathcal{A}\right)\ge 3$ and $\text{Card}\left(\mathcal{B}\right)\ge 2$, and given any integer $k\ge 3$, there is no finite test-set for k-powerfree morphisms from ${\mathcal{A}}^{*}$ to ${\mathcal{B}}^{*}$.

This again is a negative result, which shows that the general situation is difficult to handle. In general, no finite set of words suffices to verify the k-powerfreeness of a morphism. The situation improves if we restrict ourselves to uniform morphisms, and look for test-sets for this restricted class of morphisms only. Here, a test-set for k-powerfreeness of uniform morphisms on ${\mathcal{A}}^{*}$ is a set $T\subset {\mathcal{A}}^{*}$ such that, for every uniform morphisms ρ on ${\mathcal{A}}^{*}$, ρ is k-powerfree if and only if $\rho \left(T\right)$ is k-powerfree.

The existence of finite test-sets of this type was recently established by Richomme and Wlazinski [28]. Let $\text{Card}\left(\mathcal{A}\right)\ge 2$ and $k\ge 3$ be an integer. Define

$$T^{\left(k\right)}\left(\mathcal{A}\right):=U^{\left(k\right)}\left(\mathcal{A}\right)\cup \left(F^{\left(k\right)}\left(\mathcal{A}\right)\cap V^{\left(k\right)}\left(\mathcal{A}\right)\right)$$

where $U^{\left(k\right)}\left(\mathcal{A}\right)$ is the set of k-powerfree words over $\mathcal{A}$ of length at most $k+1$, and $V^{\left(k\right)}\left(\mathcal{A}\right)$ is the set of words over $\mathcal{A}$ that can be written in the form $a_0{w}_1{a}_1{w}_2\dots a_{k-1}{w}_k{a}_k$ with letters $a_0,a_1,\dots ,a_k\in \mathcal{A}$ and words $w_1,w_2\dots w_k\in {\mathcal{A}}^{*}$ which contain every letter of $\mathcal{A}$ at most once and satisfy $\left||w_i|-|w_j|\right|\le 1$. Obviously, this set is finite and comprises words with a maximum length of

$$max\left\{\left|w\right||w\in T^{\left(k\right)}\left(\mathcal{A}\right)\right\}\le k\left(\text{Card}\left(\mathcal{A}\right|)+1\right)+1.$$

Theorem 9

(Richomme, Wlazinski [28]). Let $\text{Card}\left(\mathcal{A}\right)\ge 2$ and $k\ge 3$ be an integer. The finite set $T^{\left(k\right)}\left(\mathcal{A}\right)$ is a test-set for k-powerfreeness of uniform morphisms on ${\mathcal{A}}^{*}$.

Due to the upper bound on the maximum length of words in $T^{\left(k\right)}\left(\mathcal{A}\right)$, the following corollary is immediate.

Corollary 10

(Richomme, Wlazinski [28]). A uniform morphism ρ on ${\mathcal{A}}^{*}$ is k-powerfree for an integer power $k\ge 3$ if and only if $\rho \left(w\right)$ is k-powerfree for all k-powerfree words w of length at most $k\left(\text{Card}\left(\mathcal{A}\right)+1\right)+1$.

Although this result provides an explicit test-set for k-powerfreeness, it is of limited practical use, simply because the test-set becomes large very quickly. Already for $\text{Card}\left(\mathcal{A}\right)=4$ and $k=3$, the set $T^{\left(3\right)}\left(\mathcal{A}\right)$ has 26247020 elements. For comparison, the set of cubefree words in four letters of length 16, as required in Corollary 10, has 1939267560 elements, so is still much larger.

Finally, let us quote the following result of Keränen [38], which characterises k-powerfree binary morphisms and indicates that the test-set of Theorem 9 is far from optimal.

Theorem 11

(Keränen [38]). Let $\rho :\{a,b\}\to {\mathcal{B}}^{*}$ be a uniform morphism with $\rho \left(a\right)\ne \rho \left(b\right)$ and primitive words $\rho \left(a\right),\rho \left(b\right)$ and $\rho \left(ab\right)$. For every k-powerfree word $w\in {\{a,b\}}^{*}$, $\rho \left(w\right)$ is k-powerfree if and only if $\rho \left(v\right)$ is k-powerfree for every subword v of w with

$$\left|v\right|\le \left\{\begin{array}{cc}4\hfill & \text{for}3\le k\le 6;\hfill \\ \frac{2}{3}(k+1)\hfill & \text{for}k\ge 7.\hfill \end{array}\right.$$

4. Entropy of powerfree words

Let $\mathcal{A}$ be an alphabet. A subset $X\subset {\mathcal{A}}^{*}$ is called factorial if for any word $x\in X$ all factors of x are also contained in X. Define for a factorial subset $X\subset {\mathcal{A}}^{*}$ the number of words of length n occurring in X by $c_X^{}\left(n\right)$. This number gives some idea of the complexity of X: the larger the number of words of length n, the more diverse or complicated is the set. That is why $c_X^{}:\mathbb{N}\to \mathbb{N}$ is called the complexity function of X.

Definition 12.

The (combinatorial) entropy of an infinite factorial set $X\subset {\mathcal{A}}^{*}$ is defined by

$$h\left(X\right)=\underset{n\to \infty }{lim}\frac{1}{n}log{c}_X^{}\left(n\right).$$

The requirement that X is factorial ensures the existence of the limit, see for example [39, Lemma 1].

We note the following:

(i)

if $X\subset {\mathcal{A}}^{*}$ with $\text{Card}\left(\mathcal{A}\right)=r$, then $1\le c_X\left(n\right)\le r^n$ for all n which implies $0\le h\left(X\right)\le \text{log} r$.

(ii)

if $X\subset {\mathcal{A}}^{*}$ with $\text{Card}\left(\mathcal{A}\right)=r$, then $c_X\left(n\right)=r^n$ and $h\left(X\right)=\text{log} r$.

The set of k-powerfree words $F^{\left(k\right)}\left(\mathcal{A}\right)$ over an alphabet $\mathcal{A}$ is obviously a factorial subset of ${\mathcal{A}}^{*}$, which is infinite for suitable values of k for a given alphabet $\mathcal{A}$. The precise value of the corresponding entropy, which coincides with the topological entropy [40], is not known, but lower and upper bounds exist for many cases. Recently, much improved upper and lower bounds have been established for $h\left(F^{\left(2\right)}\left(\{0,1,2\}\right)\right)$ and $h\left(F^{\left(3\right)}\left(\{0,1\}\right)\right)$, which will be outlined below. Generally, it is easier to find upper bounds than to give lower bounds, due to the factorial nature of the set of k-powerfree words, so we start with describing several methods to produce upper bounds on the entropy.

4.1. Upper bounds for the entropy

A simple way to provide upper bounds is based on the enumeration of the set of k-powerfree words up to some length. Clearly, for the case of $r=\text{Card}\left(\mathcal{A}\right)$ letters, the number of words $c\left(n\right):=c_{F^{\left(k\right)}\left(\mathcal{A}\right)}^{}\left(n\right)$ is bounded by $r^n$, so the corresponding entropy is $h:=h\left(F^{\left(k\right)}\left(\mathcal{A}\right)\right)\le logr$, as mentioned above. Suppose we know the actual value of $c\left(n\right)$ for some fixed n. Then, due to the factorial nature of the set $F^{\left(k\right)}\left(\mathcal{A}\right)$,

$$c\left(mn\right)\le c{\left(n\right)}^m$$

for any $m\ge 1$. Hence

$$h=\underset{m\to \infty }{lim}\frac{logc\left(mn\right)}{mn}\le \frac{logc\left(n\right)}{n},$$

(2)

which, for any n, yields an upper bound for h. Obviously, the larger the value of n, the better the bound obtained in this way. In some cases, the bound can be slightly improved by considering words that overlap in a couple of letters; see [39] for an example.

Sharper upper bounds can be produced by following a different approach, namely by considering a set of words that do not contain k-powers of a fixed finite set of words, for instance k-powers of all words up to a given length. This limitation means that the number of forbidden words is finite, and that the resulting factorial set has a larger entropy than the set of k-powerfree words, so the latter provides an upper bound. Again, by increasing the number of forbidden words, the bounds can be systematically improved.

As Noonan and Zeilberger pointed out [41], it is possible to calculate the generating function for the numbers of words avoiding a finite set of forbidden words by solving a system of linear equations. The generating functions are rational functions, and the location of the pole closest to the origin determines the radius of convergence, and hence the entropy of the corresponding set of words. This approach has been applied in Ref. [26] to derive an upper bound for the set of squarefree words in three letters, and generating functions for cubefree words in two letters are discussed below.

A related, though computationally easier approach is based on a Perron-Frobenius argument. It is sometimes referred to as the ‘transfer matrix’ or the ‘cluster’ approach. Here, a matrix is constructed, which determines how k-powerfree words of a given length can be concatenated to form k-powerfree words, and the growth rate is then determined by the maximum eigenvalue of this matrix. Both methods yield upper bounds that can be improved by increasing the length of the words involved, and in principle can approximate the entropy arbitrarily well, though in practice this is limited by the computational problem of computing the leading eigenvalue of a large matrix, or solving a large system of linear equations; see, for instance, [27] for details.

4.2. Lower bounds for the entropy

Until very recently, all methods used to prove that the entropy of k-powerfree words is positive and to establish lower bounds on the entropy were based on k-powerfree morphisms. Clearly, a k-powerfree morphism, iterated on a single letter, produces k-powerfree words of increasing length and suffices to show the existence of infinite k-powerfree words. For example, the fact that the Thue-Morse morphism (1) is cubefree shows the existence of cubefree words of arbitrary length in two letters. To prove that the entropy is actually positive, one has to show that the number of k-powerfree words grows exponentially with their length. Essentially, this is achieved by considering k-powerfree morphisms from a larger alphabet. The following theorem is a generalisation of Brandenburg’s method, compare [11], and provides a path to produce lower bounds for the entropy of k-powerfree words.

Theorem 13.

Let $\mathcal{A}$ and $\mathcal{B}$ be alphabets with $\text{Card}\left(\mathcal{A}\right)=r\text{Card}\left(\mathcal{B}\right)$, where $r>1$ is an integer. If there exists an ℓ-uniform k-powerfree morphism $\rho :{\mathcal{A}}^{*}\to {\mathcal{B}}^{*}$, then

$$h\left(F^{\left(k\right)}\left(\mathcal{B}\right)\right)\ge \frac{logr}{\ell -1}.$$

Proof.

For this proof define $h:=h\left(F^{\left(k\right)}\left(\mathcal{B}\right)\right)$, $c\left(n\right):=c_{F^{\left(k\right)}\left(\mathcal{B}\right)}\left(n\right)$ and $s:=\text{Card}\left(\mathcal{B}\right)$. Label the elements of $\mathcal{A}$ as $\{a_{11},\dots ,a_{1r},a_{21},\dots ,a_{2r},\dots ,a_{s1},\dots ,a_{sr}\}$ and the elements of $\mathcal{B}$ as $\{b_1,\cdots ,b_s\}$. Define the map $\varphi :{\mathcal{A}}^{*}\to {\mathcal{B}}^{*}$ as $\varphi \left(a_{ij}\right):=b_i$ for $i=1,\dots ,s$ and $j=1,\dots ,r$. Hence $\text{Card}\left({\varphi }^{-1}\left(b_i\right)\right)=r$. Every k-powerfree word of length m over $\mathcal{B}$ has $r^m$ different preimages of ϕ which, by construction, consist only of k-powerfree words. These words are mapped by ρ, which is injective due to its k-powerfreeness, to different k-powerfree words of length $m\ell$ over $\mathcal{B}$. This implies the inequality

$$c\left(m\ell \right)\ge r^{m}c\left(m\right)$$

(3)

for any $m>0$. This means

$${\left(\frac{c\left(m\ell \right)}{c\left(m\right)}\right)}^{\frac{1}{m}}\ge r,$$

and hence

$$\ell \frac{logc\left(m\ell \right)}{m\ell }-\frac{logc\left(m\right)}{m}\ge logr$$

for any $m>0$. Taking the limit as $m\to \infty$ gives

$$(\ell -1)h\ge logr,$$

thus establishing the lower bound. ☐

This result means that, whenever we can find a uniform k-powerfree morphism from a sufficiently large alphabet, it provides a lower bound for the entropy. Clearly, the larger r and the smaller ℓ the better the bound, so one is particularly interested in uniform k-powerfree morphisms from large alphabets of minimal length.

Another method due to Brinkhuis [12], which is related to Brandenburg’s method, can be generalised as follows. Let again $\mathcal{B}=\{b_1,\dots ,b_s\}$ be an alphabet and $r\in \mathbb{N}$. For $i=1,\dots s$ let

$${\mathcal{U}}_i:=\{U_{i,1},U_{i,2},\dots ,U_{i,r}\}$$

with $U_{i,j}\subset F_{\ell }^{\left(k\right)}\left(\mathcal{B}\right)$, where the latter denotes the words in $F^{\left(k\right)}\left(\mathcal{B}\right)$ which have length ℓ. The set $\mathcal{U}=\{{\mathcal{U}}_1,\dots {\mathcal{U}}_n\}$ is called an $(k,\ell ,r)$-Brinkhuis-set if the ℓ-uniform substitution (in the context of formal language theory) ϕ from ${\mathcal{B}}^{*}$ to itself defined by

$$\varphi :b_i\mapsto {\mathcal{U}}_i\text{for}i=1,\dots ,s$$

has the property $\varphi \left(F^{\left(k\right)}\left(\mathcal{B}\right)\right)\subset F^{\left(k\right)}\left(\mathcal{B}\right)$. In other words $\mathcal{U}$ is an $(k,\ell ,r)$-Brinkhuis-set if the substitution of every letter $b_i$, occurring in a k-powerfree word, by an element of ${\mathcal{U}}_i$ results in a k-powerfree word over $\mathcal{B}$. The existence of a $(k,\ell ,r)$-Brinkhuis-set delivers the lower bound

$$h\left(F^{\left(k\right)}\left(\mathcal{B}\right)\right)\ge \frac{logr}{\ell -1}$$

because every k-powerfree word of length m is mapped to $r^m$ powerfree words of length $\ell m$; compare Eq. (3).

The method of Brinkhuis is stronger than the method of Brandenburg. Not every $(k,\ell ,r)$-Brinkhuis-set implies a map according to Theorem 13, see [42, p. 287] for an example. Conversely, if there exists an ℓ-uniform k-powerfree morphism $\rho :{\mathcal{A}}^{*}\to {\mathcal{B}}^{*}$ according to Theorem 13, then there exists a $(k,\ell ,r)$-Brinkhuis-set, namely ${\mathcal{U}}_i=\left\{\rho \left(a_{i1}\right),\dots ,\rho \left(a_{ir}\right)\right\}$ for $i=1,\dots ,s$, with the notation of Theorem 13.

Brinkhuis’ method was applied in Refs. [21,43,44]; see also below for a summary of bounds obtained for binary cubefree and ternary squarefree words. These bounds have in common that they are nowhere near the actual value of the entropy, and while a systematic improvement is possible by increasing the value of r in Theorem 13 (which, however, also means that one has to consider larger values of ℓ), it will always result in a much smaller growth rate, because only a subset of words is obtained in this way.

Recently, a different approach has been proposed [29], based essentially on the derivation of an inequality

$$S_m(n+1)\ge \alpha S_m\left(n\right)$$

(4)

for the weighted sum $S_m\left(n\right)$ of the number of elements in a certain subset (which depends on the choice of $m\in \mathbb{N}$) of squarefree (resp. cubefree) words of length n over a ternary (resp. binary) alphabet and a parameter $\alpha >1$ which satisfies two inequalities for $i=m,m+1,\dots ,n-1$. The estimation of $S_m(n+1)$ starts from a Perron-Frobenius argument and concludes with the observation that the order of growth of the number of squarefree (resp. cubefree) words cannot be less than the order of growth of $S_m\left(n\right)$, which is α. This implies

$$h\left(F^{\left(k\right)}\left(\mathcal{A}\right)\right)>log\left(\alpha \right)$$

for $k=2$ and $\mathcal{A}=\{0,1,2\}$ or $k=3$ and $\mathcal{A}=\{0,1\}$, with the corresponding values for α. In the end, this method leads to a recipe to check, with a computer, several conditions for the parameters (including m), which ensure that the inequality for $S_m$ holds. By increasing the parameter m, it appears to be possible to estimate the growth rate of cubefree and squarefree words with an arbitrary precision. For details, we refer the reader to Ref. [29].

5. Bounds on the entropy of binary cubefree and ternary squarefree words

We now consider the two main examples, binary cubefree and ternary squarefree words, in more detail, reviewing the bounds derived by the various approaches mentioned above. We start with the discussion of binary cubefree words, and then give a brief summary of the analogous results for ternary squarefree words.

5.1. Binary cubefree words

Define for this section $b\left(n\right):=c_{F^{\left(3\right)}\left(\{0,1\}\right)}^{}\left(n\right)$ as the number of binary cubefree words of length n and $h:=h\left(F^{\left(3\right)}\left(\{0,1\}\right)\right)$ as the entropy of cubefree words over the alphabet $\{0,1\}$. The values for $b\left(n\right)$ with $n\le 47$ are given in [45]; an extended list for $n\le 80$ is shown in Table 1. They were obtained by a straight-forward iterative construction of cubefree words, appending a single letter at a time. According to Eq. (2), the corresponding upper limit for the entropy h is

$$h\le \frac{logb\left(80\right)}{80}\simeq 0.389855.$$

For comparison, the limit obtained using the number of words of length 79 is $0.390020$, which indicates that these limits are still considerably larger than the actual value of h. As in the case of ternary squarefree words [26], the asymptotic behaviour of $b\left(n\right)$ fits a simple form $b\left(n\right)\sim A{x}_{\text{c}}^{-n}$ as $n\to \infty$, pointing at a simple pole as the dominating singularity of the corresponding generating function at $x=x_{\text{c}}$. The estimated values of the coefficients are $A\simeq 2.847$ and $x_{\text{c}}\simeq 1.4575773$, leading to a numerical estimate of $h=log\left(x_{\text{c}}\right)\simeq 0.3767757$ for the entropy.

Let us compare this with the upper limit derived from generating function of the number of binary length-p cubefree words. To this end, let $b_p\left(n\right):=c_{F^{(3,p)}\left(\{0,1\}\right)}^{}\left(n\right)$ denote the number of length-p cubefree words, and define

$$B_p\left(x\right)=\sum _{n=0}^{\infty }{b}_p\left(n\right)x^n$$

(5)

to be the generating function for the number of binary length-p cubefree words. These functions of x are rational [41]. The first few generating functions read

$$\begin{array}{ccc}\hfill B_0^{}\left(x\right)=& \frac{1}{1-2x}& =1+2x+4x^2+8x^3+16x^4+32x^5+64x^6+\dots ,\hfill \\ \hfill B_1^{}\left(x\right)=& \frac{1+x+x^2}{1-x-x^2}& =1+2x+4x^2+6x^3+10x^4+16x^5+26x^6+\dots ,\hfill \\ \hfill B_2^{}\left(x\right)=& \frac{1+2x+3x^2+3x^3+3x^4+3x^5+2x^6}{1-x^2-x^3-x^4-x^5}& =1+2x+4x^2+6x^3+10x^4+16x^5+24x^6+\dots \hfill \end{array}$$

The degrees of the numerator and denominator polynomials for $p\le 14$ are given in Table 2. The generating functions $B_p^{}\left(x\right)$ have a finite radius of convergence, determined by the location of the zero $x_{\mathrm{c}}^{}$ of its denominator polynomial which lies closest to the origin. A plot of the location of poles of $B_{14}^{}\left(x\right)$ is shown in Figure 1. It very much resembles the analogous distribution for ternary squarefree words [26]; again, the poles seem to accumulate, with increasing p, on or near the unit circle, which may indicate the presence of a natural boundary beyond which the generating function for cubefree binary words (corresponding to taking $p\to \infty$) cannot be analytically continued; see [26] for a discussion of this phenomenon in the case of ternary squarefree words.

Table 1. The number $b\left(n\right)$ of binary cube-free words of length n for $n\le 80$.

n	$b\left(n\right)$		n	$b\left(n\right)$		n	$b\left(n\right)$		n	$b\left(n\right)$
1	2		21	7754		41	14565048		61	27286212876
2	4		22	11320		42	21229606		62	39771765144
3	6		23	16502		43	30943516		63	57970429078
4	10		24	24054		44	45102942		64	84496383550
5	16		25	35058		45	65741224		65	123160009324
6	24		26	51144		46	95822908		66	179515213688
7	36		27	74540		47	139669094		67	261657313212
8	56		28	108664		48	203577756		68	381385767316
9	80		29	158372		49	296731624		69	555899236430
10	118		30	230800		50	432509818		70	810266077890
11	174		31	336480		51	630416412		71	1181025420772
12	254		32	490458		52	918879170		72	1721435861086
13	378		33	714856		53	1339338164		73	2509125828902
14	554		34	1041910		54	1952190408		74	3657244826158
15	802		35	1518840		55	2845468908		75	5330716904964
16	1168		36	2213868		56	4147490274		76	7769931925578
17	1716		37	3226896		57	6045283704		77	11325276352154
18	2502		38	4703372		58	8811472958		78	16507465616784
19	3650		39	6855388		59	12843405058		79	24060906866922
20	5324		40	9992596		60	18720255398		80	35070631260904

Table 1. The number $b\left(n\right)$ of binary cube-free words of length n for $n\le 80$.

n	$b\left(n\right)$		n	$b\left(n\right)$		n	$b\left(n\right)$		n	$b\left(n\right)$
1	2		21	7754		41	14565048		61	27286212876
2	4		22	11320		42	21229606		62	39771765144
3	6		23	16502		43	30943516		63	57970429078
4	10		24	24054		44	45102942		64	84496383550
5	16		25	35058		45	65741224		65	123160009324
6	24		26	51144		46	95822908		66	179515213688
7	36		27	74540		47	139669094		67	261657313212
8	56		28	108664		48	203577756		68	381385767316
9	80		29	158372		49	296731624		69	555899236430
10	118		30	230800		50	432509818		70	810266077890
11	174		31	336480		51	630416412		71	1181025420772
12	254		32	490458		52	918879170		72	1721435861086
13	378		33	714856		53	1339338164		73	2509125828902
14	554		34	1041910		54	1952190408		74	3657244826158
15	802		35	1518840		55	2845468908		75	5330716904964
16	1168		36	2213868		56	4147490274		76	7769931925578
17	1716		37	3226896		57	6045283704		77	11325276352154
18	2502		38	4703372		58	8811472958		78	16507465616784
19	3650		39	6855388		59	12843405058		79	24060906866922
20	5324		40	9992596		60	18720255398		80	35070631260904

As a consequence of Pringsheim’s theorem [46, Sec. 7.2], there is a dominant singularity on the positive real axis; we denote the position of the singularity by $x_{\mathrm{c}}^{}$. For the cases we considered, this simple pole appears to be the only dominant singularity. Since the radius of convergence of the power series $B_p^{}\left(x\right)$ is given by ${\left({lim\; sup}_{n\to \infty }\sqrt[n]{b_p^{}\left(n\right)}\right)}^{-1}$, the entropy $h_p$ of the set of binary length-p cubefree words is $h_p=-log{x}_{\mathrm{c}}$. Clearly, $h_p\ge h_{p^{\prime }}$ for $p\le p^{\prime }$, and $h={lim}_{p\to \infty }{h}_p$, so for any finite p the entropy $h_p$ provides an upper bound of the entropy h of binary cubefree words. The values of the entropy $h_p$ for $p\le 14$ are given in Table 2. As was observed for ternary squarefree words [26], the values appear to converge very quickly with increasing p, but it is difficult to extract a reliable estimate of the true value of the entropy without making assumptions on the asymptotic behaviour.

Already in 1983, Brandenburg [11] showed that

$$2·2^{\frac{n}{9}}\le b\left(n\right)\le 2·{1251}^{\frac{n}{17}}$$

which leads in our setting to $0.07701\le h\le 0.41952$. The currently best upper bounds are due to Edlin [45] and Ochem and Reix [27]. Analysing length-15 cubefree words up to a finite length, Edlin [45] arrives at the bound of $h\le 0.376777$ (which is what we would expect to find if we extended Table 2 to $n=15$, but this would require huge computational effort to compute the corresponding generating function completely), while using the transfer matrix (or cluster) approach described above, Ochem and Reix obtained an upper bound on the growth rate of $1.45758131$, which corresponds to the bound

$$h\le 0.3767784$$

on the entropy.

Table 2. The entropy $h_p$ of binary length-p cubefree words, obtained from the radius of convergence of the generating functions $B_p\left(x\right)$ of Eq. (5). Here, $d_{\mathrm{num}}$ and $d_{\mathrm{den}}$ denote the degree of the polynomial in the numerator and denominator of $B_p\left(x\right)$, respectively.

p	$d_{\mathrm{num}}$	$d_{\mathrm{den}}$	$h_p$
0	0	1	0.693147
1	2	2	0.481212
2	6	5	0.427982
3	21	13	0.394948
4	29	17	0.385103
5	43	25	0.380594
6	85	57	0.378213
7	127	99	0.377332
8	165	127	0.377179
9	300	254	0.376890
10	450	395	0.376835
11	569	513	0.376811
12	1098	1031	0.376790
13	1750	1656	0.376783
14	2627	2540	0.376779

Table 2. The entropy $h_p$ of binary length-p cubefree words, obtained from the radius of convergence of the generating functions $B_p\left(x\right)$ of Eq. (5). Here, $d_{\mathrm{num}}$ and $d_{\mathrm{den}}$ denote the degree of the polynomial in the numerator and denominator of $B_p\left(x\right)$, respectively.

p	$d_{\mathrm{num}}$	$d_{\mathrm{den}}$	$h_p$
0	0	1	0.693147
1	2	2	0.481212
2	6	5	0.427982
3	21	13	0.394948
4	29	17	0.385103
5	43	25	0.380594
6	85	57	0.378213
7	127	99	0.377332
8	165	127	0.377179
9	300	254	0.376890
10	450	395	0.376835
11	569	513	0.376811
12	1098	1031	0.376790
13	1750	1656	0.376783
14	2627	2540	0.376779

Figure 1. Location of poles of the generating function $B_{14}^{}\left(x\right)$.

Figure 1. Location of poles of the generating function $B_{14}^{}\left(x\right)$.

We now move on to the lower bound and cubefree morphisms. We already have seen one example above, the Thue-Morse morphism, which is a cubefree morphism from a binary alphabet to a binary alphabet. As explained above, it is also useful to find uniform cubefree morphisms from larger alphabets, because these provide lower bounds on the entropy. Clearly, if we have a uniform cubefree morphism $\rho :{\mathcal{A}}^{*}\to {\{0,1\}}^{*}$ of length ℓ, with $\text{Card}\left(\mathcal{A}\right)=r$, it is completely specified by the r words $w_i^{}$, $i=1,\dots ,r$, which are the images of the letters in $\mathcal{A}$. Since any permutation of the letters in $\mathcal{A}$ will again yield a uniform cubefree morphism, the set $\left\{w_1^{},\dots ,w_r^{}\right\}\subset {\{0,1\}}^{\ell }$ of generating words determines the morphism up to permutation of the letters in $\mathcal{A}$.

Moreover, the set $\left\{\overline{w_1^{}},\dots ,\overline{w_r^{}}\right\}$, where $\overline{w}$ denotes the image of w under the permutation $0\leftrightarrow 1$, also defines cubefree morphisms, as does $\left\{\widetilde{w_1^{}},\dots ,\widetilde{w_r^{}}\right\}$, where $\tilde{w}$ denotes the reversal of w, i.e., the words w read backwards. This is obvious because the test-sets of Theorem 9 are invariant under these operations. Unless the words are palindromic (which means that $\tilde{w}=w$), the set $\left\{w_1^{},\dots ,w_r^{}\right\}$ thus represents four different morphisms (not taking into account permutation of letters in $\mathcal{A}$), the forth obtained by performing both operations, yielding $\left\{\widetilde{\overline{w_1^{}}},\dots ,\widetilde{\overline{w_r^{}}}\right\}$.

For cubefree morphisms from a three-letter alphabet $\mathcal{A}$ to two letters one needs words of length at least six. For length six, there are twelve in-equivalent (with respect to the permutation of letters in $\mathcal{A}$) cubefree morphisms. The corresponding sets of generating words are

$$\{w_1^{},w_2^{},w_4^{}\},\{w_2^{},\overline{w_3^{}},\widetilde{\overline{w_3^{}}}\},\{w_2^{},\overline{w_3^{}},w_4^{}\},$$

and the corresponding images under the two operations explained above. Here, the four words are

$$w_1=001011,w_2=001101,w_3=010110,w_4=011001.$$

It turns out that none of these morphisms actually satisfy the sufficient criterion of Theorem 7, but cubefreeness was verified using the test-set of Theorem 9.

One has to go to length nine to find cubefree morphisms from four to two letters. There are 16 in-equivalent morphisms with respect to permutations of the four letters. Explicitly, they are given by the generating sets

$$\{w_1,w_2,\widetilde{\overline{w_2}},\widetilde{\overline{w_3}}\},\{w_4,\overline{w_6},\overline{w_7},\overline{w_9}\},\{w_5,\overline{w_5},w_8,\overline{w_8}\},\{w_5,\overline{w_5},\widetilde{w_8},\widetilde{\overline{w_8}}\},\{\overline{w_6},\widetilde{w_7},\widetilde{\overline{w_8}},w_9\}$$

(6)

with words

$$\begin{array}{cccccc}\hfill w_1& =001001101,\hfill & \hfill w_2& =001010011,\hfill & \hfill w_3& =001011001,\hfill \\ \hfill w_4& =001101001,\hfill & \hfill w_5& =010010110,\hfill & \hfill w_6& =010011010,\hfill \\ \hfill w_7& =010100110,\hfill & \hfill w_8& =011001001,\hfill & \hfill w_9& =011010110.\hfill \end{array}$$

Note that $w_9^{}=\widetilde{w_9^{}}$ is a palindrome, and that two of the five sets are invariant under the permutation $0\leftrightarrow 1$, which explains why they only represent 16 different morphisms.

Beyond four letters, the test-set of Theorem 9 becomes unwieldy, but the sufficient criterion of Theorem 7 can be used to obtain morphisms. However, these may not have the optimal length, as the examples here show – again for length nine all morphisms violate the conditions of Theorem 7. Still, this need not be the case; for instance, morphisms from a five-letter alphabet that satisfy the sufficient criterion exist for length 12, which in this case is the optimal length.

As a consequence of Theorem 13, the morphisms (6) from a four letter alphabet show that the entropy of cubefree binary words is positive, and that

$$h\ge \frac{log2}{8}\simeq 0.08664.$$

Using the sufficient condition, this bound can be improved. For instance, for length 15, one can find cubefree morphisms from 10 letters, which yields a lower bound of

$$h\ge \frac{log5}{14}\simeq 0.11496.$$

However, a large step to close the gap between these lower bounds and the upper bound was achieved by the work of Kolpakov [29]. With his approach, a lower bound of

$$h\ge 0.37676,$$

which is the best lower bound so far, has been established. The difference between this bound and the upper bound $0.3767784$ by Ochem and Reix [27] is just ${10}^{-5}$, showing the huge improvement over the previously available estimates.

5.2. Ternary squarefree words

Denote by $a\left(n\right):=c_{F^{\left(2\right)}\left(\{0,1,2\}\right)}^{}\left(n\right)$ the number of ternary squarefree words and by $a_p\left(n\right)$ the number of length-p squarefree words of length n. For this section let $h:=h\left(F^{\left(2\right)}\left(\{0,1,2\}\right)\right)$ be the entropy of squarefree words over the alphabet $\{0,1,2\}$. See [39] for a list of $a\left(n\right)$ for $n\le 90$ and [21] for $91\le n\le 110$. The generating functions are defined according to the binary cubefree case. The first four of them are stated in [26, Sec. 3], which also contains a list of their radii of convergence for $p\le 24$. Already in 1983 Brandenburg [11] showed that

$$6·2^{\frac{n}{22}}\le a\left(n\right)\le 6·{1172}^{\frac{n}{22}}$$

which leads in our setting to

$$0.03151\le h\le 0.32120.$$

In 1999, Noonan and Zeilberger [41] lowered the upper bound to $0.26391$ by means of generating functions for the number of words avoiding squares of up to length 23. Grimm and Richard [26] used the same method to improve the upper bound to $0.263855$. At the moment, the best known upper bound is $0.263740$ which was established by Ochem in 2006 using an approach based on the transfer matrix (or cluster) method, see [27] for details.

In 1998, Zeilberger showed that a Brinkhuis pair of length 18 exists, which by Theorem 13 implies that the entropy is bounded by $h\ge log\left(2\right)/17\simeq 0.04077$ [47]. By going to larger alphabets, this was subsequently improved to $h\ge log\left(65\right)/40\simeq 0.10436$ by Grimm [21] and $h\ge log\left(110\right)/42\simeq 0.11192$ by Sun [44]. Again, the recent work of Kolpakov [29] has made a large difference to the lower bounds; he achieved the best current lower bound which is $h\ge 0.26369$. The difference between the best known upper and lower bound is now just $5\times {10}^{-5}$.

6. Letter frequencies

For a finite word w of length n, the frequency of the letter a is ${\#}_a\left(w\right)/n\in [0,1]$, where ${\#}_a\left(w\right)$ denotes the number of occurrences of the letter a in w. In general, infinite k-powerfree words need not have well-defined letter frequencies. However, we can define upper and lower frequencies $f_a^{+}\ge f_a^{-}$ of a letter $a\in \mathcal{A}$ of a word $w\in {\mathcal{A}}^{*}$ as

$$f_a^{+}:=\underset{\left\{w_n\right\}}{sup}\underset{n\to \infty }{lim\; sup}\frac{{\#}_a\left(w_n\right)}{n},f_a^{-}:=\underset{\left\{w_n\right\}}{inf}\underset{n\to \infty }{lim\; inf}\frac{{\#}_a\left(w_n\right)}{n},$$

where $w_n$ is a n-letter subword of w. Here, we take the supremum and infimum over all sequences $\left\{w_n\right\}$. Alternatively, we can compute these frequencies from $a_n^{+}={max}_{w_n\subset w}{\#}_a\left(w_n\right)$ and $a_n^{-}={min}_{w_n\subset w}{\#}_a\left(w_n\right)$ by $f_a^{\pm }={lim}_{n\to \infty }{a}_n^{\pm }/n$. The limits exist due to the subadditivity of the sequences $\left\{a_n^{+}\right\}$ and $\{1-a_n^{-}\}$. If the infinite word w is such that $f_a^{+}=f_a^{-}=:f_a$, we call $f_a$ the frequency of the letter a in w.

The requirement that a word is k-powerfree for some k restricts the possible letter frequencies. For instance, for cubefree binary words, there cannot be three consecutive zeros, and hence the frequency of the letter 0 is certainly bounded from above by $2/3$. Due to symmetry under permutation of letters, it is bounded from below by $1/3$. In a similar way, considering maximum and minimum frequencies of letters in finite k-powerfree words produces bounds on the possible (upper and lower) frequencies of letters in infinite words. It is of interest, for which frequency of a letter k-powerfree words cease to exist, and how the entropy of k-powerfree words depends on the letter frequency. To answer these questions, k-powerfree morphisms are exploited once again, and in two ways. Firstly, the argument using frequencies in finite words only produces ‘negative’ results, in the sense that you can exclude the existence of k-powerfree words for certain ranges of the frequency. To show that k-powerfree words of a certain frequency actually exist, these are produced as fixed points of k-powerfree morphisms. The letter frequency for an infinite word obtained as a fixed point of a morphism ρ on the alphabet $\mathcal{A}=\{a_1,a_2,\dots ,a_m\}$ is well-defined, and obtained from the (statistically normalised) right Perron-Frobenius eigenvector of the associated $m\times m$ substitution matrix M with elements $M_{ij}={\#}_{a_i}\rho \left(a_j\right)$; see for instance [48]. For example, for the Thue-Morse morphism (1), the substitution matrix is $M=\left(\begin{array}{cc}1& 1\\ 1& 1\end{array}\right)$ with Perron-Frobenius eigenvalue 2 and corresponding eigenvector ${(\frac{1}{2},\frac{1}{2})}^T$, so both letters occur with frequency $1/2$ in the infinite Thue-Morse word.

To show that there exist exponentially many words with a given letter frequency, or, in other words, that the entropy of the set of k-powerfree words with a given letter frequency is positive, a variant of Theorem 13 is used.

Theorem 14.

Let $\mathcal{A}=\{a_{11},\dots ,a_{1r},a_{21},\dots ,a_{2r},\dots ,a_{s1},\dots ,a_{sr}\}$ and $\mathcal{B}=\{b_1,\dots ,b_s\}$ be alphabets with $\text{Card}\left(\mathcal{A}\right)=rs$ and $\text{Card}\left(\mathcal{B}\right)=s$, where $r,s>1$ are integers. Assume that there exists an ℓ-uniform k-powerfree morphism $\rho :{\mathcal{A}}^{*}\to {\mathcal{B}}^{*}$ with

$${\#}_b\rho \left(a_{ij}\right)={\#}_b\rho \left(a_{i{j}^{\prime }}\right)$$

for all $b\in \mathcal{B}$, $1\le i\le s$ and $1\le j,j^{\prime }\le r$. Define the $r\times r$ matrix M with elements

$$M_{ij}={\#}_{b_i}\rho \left(a_{j1}\right),$$

and denote its right Perron-Frobenius eigenvector (with eigenvalue ℓ) by ${(f_1,\dots ,f_r)}^T$, with statistical normalisation $f_1+\dots +f_r=1$. Then, the entropy h of the set of k-powerfree words in $\mathcal{B}$ with prescribed letter frequencies $f_i$ of $b_i$, $1\le i\le r$, is bounded by

$$h\ge \frac{logr}{\ell -1}.$$

Proof.

The bound is the same as in Theorem 13, and the statement thus follows by showing that the infinite words obtained from the uniform k-powerfree morphism ρ have letter frequency given by $f_1,\dots ,f_r$.

We again introduce the morphism $\varphi :{\mathcal{A}}^{*}\to {\mathcal{B}}^{*}$ by $\varphi \left(a_{ij}\right):=b_i$ for $i=1,\dots ,s$ and $j=1,\dots ,r$. Every k-powerfree word of length m over $\mathcal{B}$ has $r^m$ different preimages of ϕ which, by construction, consist only of k-powerfree words. These words are mapped by ρ, which is injective due to its k-powerfreeness, to different k-powerfree words of length $m\ell$ over $\mathcal{B}$. Due to the condition ${\#}_b\rho \left(a_{ij}\right)={\#}_b\rho \left(a_{i{j}^{\prime }}\right)$ on ρ, the letter statistics do not depend on the choice of the preimage under ϕ. The letter frequencies of words obtained by the procedure described in the proof of Theorem 13 are thus well defined, and given by the right Perron-Frobenius eigenvector of the $r\times r$ matrix M. ☐

Some results for binary cubefree words, as well as a discussion of the empirical frequency distribution of cubefree binary words obtained from the enumeration up to length 80, are detailed below.

6.1. Binary cubefree words

When counting the numbers $b\left(n\right)$ of binary cubefree words of length n shown in Table 1, we also counted the number $b(n,n_0^{})$ of words with $n_0^{}$ occurrences of the letter 0. Clearly, these numbers satisfy

$$b\left(n\right)=\sum _{n_0^{}=0}^{n}b(n,n_0^{})$$

and $b(n,n-n_0^{})=b(n,n_0^{})$ as a consequence of the symmetry under permutation of letters. Their values for $n=80$ are given in Table 3.

Table 3. The number of the binary cube-free words of length 80 with given excess $e=n_0^{}-40$ of the letter 0.

$\left|e\right|$	$b(80,40+e)$
$0$	9502419002570
$1$	7575510051076
$2$	3805516412947
$3$	1172047753336
$4$	210113470848
$5$	20038955440
$6$	866998237
$7$	12460464
$8$	26819
$\ge 9$	0

Table 3. The number of the binary cube-free words of length 80 with given excess $e=n_0^{}-40$ of the letter 0.

$\left|e\right|$	$b(80,40+e)$
$0$	9502419002570
$1$	7575510051076
$2$	3805516412947
$3$	1172047753336
$4$	210113470848
$5$	20038955440
$6$	866998237
$7$	12460464
$8$	26819
$\ge 9$	0

Obviously, there are at least 32 and at most 48 occurrences of the letter 0 in any cubefree binary word of length 80, so the frequency of a letter is bounded by $2/5\le f_0\le 3/5$. A stronger bound has been obtained by Ochem [30], who showed (amongst many results for a number of rational powers) that $f_0>\frac{115}{283}\simeq 0.40636$, using a backtracking algorithm.

One is interested to locate the minimum frequency $f_{\text{min}}$, such that infinite cubefree words with frequency $f_0=f_{\text{min}}$ exist, but not for any $f_0<f_{\text{min}}$. Clearly, the lower bound above is a lower bound for $f_{\text{min}}$. In order to obtain an upper bound, we need to prove the existence of an infinite binary cubefree words of a given letter frequency. This is again done by using a cubefree morphism, which provides an infinite word with well-defined letter frequencies. For instance,

$$\begin{array}{cc}\hfill 0& \mapsto 011011010110110011011010110\hfill \\ \hfill 1& \mapsto 011011010110110011010110110\hfill \end{array}$$

is a uniform morphism of length 27 with substitution matrix $\left(\begin{array}{cc}11& 11\\ 16& 16\end{array}\right)$, so the infinite fixed point word has letter frequencies $f_0=\frac{11}{27}$ and $f_1=\frac{16}{27}$. Hence we deduce that

$$0.406360\simeq \frac{115}{283}<f_{\text{min}}\le \frac{11}{27}\simeq 0.407407.$$

Using the data from our enumeration of binary cubefree words up to length 80, we can study the empirical distribution for small length, and try to conjecture the behaviour for large words. Figure 2 shows a plot of the normalised data $b(80,40+e)/b\left(80\right)$ of Table 3, compared with a Gaussian distribution, which appears to fit the data very well. Here, the Gaussian profile was determined from the variance ${\sigma }^2$ of the data points, which is approximately ${\sigma }^2\simeq 2.124$.

Figure 2. Distribution of cubefree words of length 80 as a function of the excess $e=n_0-40$, compared to a Gaussian distribution with the same variance ${\sigma }^2$.

Figure 2. Distribution of cubefree words of length 80 as a function of the excess $e=n_0-40$, compared to a Gaussian distribution with the same variance ${\sigma }^2$.

To draw any conclusions on the limit of large word length, we need to consider the scaling of the distribution with the word length n. The first step is to determine how the variance scales with n. A plot of the numerical data is given in Figure 3, which shows that, for large n, the variance appears to scale linearly with n. A least squares fit to the data points for $40\le n\le 80$ gives a slope of $0.021616$.

Figure 3. Variance of the distribution of the letter frequency in binary cubefree words of length n.

Figure 3. Variance of the distribution of the letter frequency in binary cubefree words of length n.

Assuming that the distribution for fixed n is Gaussian, the suitably re-scaled data

$$g_n\left(x\right)=\sqrt{n}\frac{b(n,\frac{n}{2}+e)}{b\left(n\right)},$$

considered as a function of the rescaled letter excess

$$x=\frac{e}{\sqrt{n}},$$

should approach a Gaussian distribution

$$G\left(x\right)=\frac{1}{\sqrt{2\pi {\sigma }^2}}exp\left(-\frac{x^2}{2{\sigma }^2}\right)$$

with variance ${\sigma }^2\simeq 0.021616$. Figure 4 shows a plot of this distribution, together with the data points obtained for $40\le n\le 80$. Clearly, there are some deviations, which has to be expected due to the fact that the relationship between the variance and the length shown in Figure 3, while being asymptotically linear, is not a proportionality; however, the overall agreement is reasonable. A plausible conjecture, therefore, is that the scaled distribution becomes Gaussian in the limit of large word length. In terms of the entropy, the observed concentration property is consistent with the entropy maximum occurring at letter frequency $1/2$, and a lower entropy for other letter frequencies. This is similar to the observed and conjectured behaviour for ternary squarefree words in Ref. [26].

Figure 4. The scaled data $g_n\left(x\right)$ for lengths $40\le n\le 80$, compared to a Gaussian distribution $G\left(x\right)$.

Figure 4. The scaled data $g_n\left(x\right)$ for lengths $40\le n\le 80$, compared to a Gaussian distribution $G\left(x\right)$.

By an application of Theorem 14, the cubefree morphisms of Eq. (6) show that the entropy for the case of letter frequency $f_0=f_1=1/2$ is positive. More interesting in the case of non-equal letter frequencies. As an example, consider the 13-uniform morphism

$$\begin{array}{cc}\hfill a_{11}& \mapsto 0010010110011\hfill \\ \hfill a_{12}& \mapsto 0010011010011\hfill \\ \hfill a_{21}& \mapsto 0010110010011\hfill \\ \hfill a_{22}& \mapsto 0100101001011,\hfill \end{array}$$

where all words on the right-hand side comprise seven letters 0 and six letters 1. One can check that this morphism satisfies the criterion of Theorem 9, hence is cubefree. Consequently, the matrix M of Theorem 14 is $M=\left(\begin{array}{cc}7& 7\\ 6& 6\end{array}\right)$, and the letter frequencies of any word constructed by this morphism are $f_0=7/13$ and $f_1=6/13$. Hence, the set of binary cubefree words with letter frequencies $f_0=7/13$ and $f_1=6/13$ has positive entropy bounded by $h\ge \frac{1}{6}log2\simeq 0.115525$ (and, by symmetry, this also holds for $f_0=6/13$ and $f_1=7/13$). Again, like in the case of ternary squarefree words discussed in Ref. [26], it is plausible to conjecture that the entropy is positive on an entire interval of letter frequencies around $1/2$ (where it is maximal), presumably on $(f_{\text{min}},1-f_{\text{min}})$.

6.2. Ternary squarefree words

Letter frequencies in ternary squarefree words were first studied by Tarannikov [49]. He showed that the minimal letter frequency $f_{\text{min}}$ is bounded by

$$0.274649\simeq \frac{1780}{6481}\le f_{\text{min}}\le \frac{64}{233}\simeq 0.274678,$$

see [49, Thm. 4.2]. These bounds have recently been improved by Ochem [30] to

$$0.2746498\simeq \frac{1000}{3641}\le f_{\text{min}}\le \frac{883}{3215}\simeq 0.2746501,$$

who also showed that the maximum frequency $f_{\text{max}}$ of a letter in a ternary squarefree word is bounded by

$$f_{\text{max}}\le \frac{255}{653}\simeq 0.390505;$$

see [30, Thm. 1]. Very recently, Khalyavin [33] proved that the minimum frequency is indeed equal to Ochem’s upper bound, so

$$f_{\text{min}}=\frac{883}{3215},$$

which finally settles this question.

By constructing suitable squarefree morphisms in accordance with Theorem 14, Richard and Grimm [26] showed that, for a number of letter frequencies, the number of ternary squarefree words grows exponentially. This has recently been further investigated by Ochem [32].

7. Summary and Outlook

In this paper, we reviewed recent progress on the combinatorics of k-powerfree words, with particular emphasis on the examples of binary cubefree and ternary squarefree words, which have attracted most attention over the years. Recent work in this area, using extensive computer searches, but also new methods, has led to a drastic improvement of the known bounds for the entropy of these sets. No analytic expression for the entropy is known to date, and the results on the generating function for the sets of length-p powerfree words indicate that this may be out of reach. However, considerable progress has been made on other combinatorial questions, such as letter frequencies, where again bounds have been improved, but eventually also a definite answer has emerged, in this case on the minimum letter frequency in ternary squarefree words.

We also presented some new results on binary cubefree words, including an enumeration of the number of words and their letter frequencies for length up to 80. The empirical distribution of the number of words as a function of the excess of one letter is investigated, and conjectured to become Gaussian in the limit of infinite word lengths after suitable scaling. We also found bounds on the letter frequency in binary squarefree words, and show that exponentially many words with unequal letter frequency exist, like in the case of ternary squarefree words. The analysis of the generating functions of length-p binary cubefree words, which we calculated for $p\le 14$, also shows striking similarity to the case of ternary squarefree words, suggesting that the observed behaviour may be generic for sets of k-powerfree words.

While a lot of progress has been made, there remain many open questions. For instance, is there an explanation for the observed accumulation of poles and zeros of the generating functions on or near the unit circle, and is it possible to prove what happens in the limit when $p\to \infty$? How does the entropy depend on the power, say for binary k-powerfree words? A partial answer to this question is given in Ref. [25], but it would be nice to show that, at least in some region, the entropy increases by a finite amount at any rational value of k, which you might expect to happen. Concerning powerfree words with given letter frequencies, how does the entropy vary as a function of the frequency? One might conjecture that the entropy changes continuously, but at present all we have are results that for some very specific frequencies, where powerfree morphisms have been found, the entropy is positive. Some of these questions may be too hard to hope for an answer in full generality, but the recent progress in the area shows that one should keep looking for alternative approaches which may succeed.