1. Introduction
One of the basic problems in the context of source coding is to assign a code
Cn = (
c1,
c2, ⋯,
cn) with codelength vector
Ln = (
ℓ1,
ℓ2, ⋯,
ℓn) to a memoryless source with probability vector
Pn = (
p1,
p2, ⋯,
pn). Decoding requirements often constrain us to choose a code
Cn from a specific class of codes, such as prefix-free codes, fix-free codes or symmetric fix-free codes. With a prefix-free code, no codeword is the prefix of another codeword. This property ensures that decoding in the forward direction can be done without any delay (instantaneously). Alternatively, with a fix-free code no codeword is the prefix or suffix of any other codeword [
1,
2]. Therefore, decoding of a fix-free code in both the forward and backward directions can be done without any delay. The ability to decode a fix-free code in both directions makes them more robust to transmission errors and faster decoding can be achieved compared to prefix-free codes. As a result, fix-free codes are used in video standards such as H.263+ and MPEG-4 [
3]. A symmetric fix-free code is a fix-free code whose codewords are symmetric. In general, decoder implementation for a fix-free code requires more memory compared to that for a prefix-free code. Although the decoder for a symmetric fix-free code is the same as for a fix-free code [
4], symmetric codes have greater redundancy in general.
Let
denote the Kraft sum of the codelength vector
Ln. A well-known necessary and sufficient condition for the existence of a prefix-free code with codelength vector
Ln is the Kraft inequality,
i.e.,
S(
Ln)
≤ 1 [
5]. However, this inequality is
only a necessary condition on the existence of a fix-free code. Some sufficient conditions on the existence of a fix-free code were introduced in [
6–
11].
The optimal code for a specific class of codes is defined as the code with the minimum average codelength,
i.e.,
among all codes in that class. The optimal prefix-free code can easily be obtained using the Huffman algorithm [
12]. Recently, two methods for finding the optimal fix-free code have been developed. One is based on the
A* algorithm [
13], while the other is based on the concept of dominant sequences [
14]. Compared to the Huffman algorithm, these methods are very complex.
A source with
n symbols having Huffman code with codelength vector
Ln = (1, 2, 3, ⋯,
n − 2,
n − 1,
n − 1) is called an anti-uniform source [
15,
16]. Such sources have been shown to correspond to particular probability distributions. For example, it was shown in [
17] and [
18], respectively, that the normalized tail of the Poisson distribution and the geometric distribution with success probability greater than some critical value are anti-uniform sources. It was demonstrated in [
15,
16] that a source with probability vector
Pn = (
p1,
p2, ⋯,
pn) where
p1 ≥ p2 ≥ ⋯
≥ pn, is anti-uniform if and only if
As mentioned above, finding an optimal fix-free or symmetric fix-free code is complex. Thus, in this paper optimal fix-free and symmetric fix-free codes are determined for anti-uniform sources. In particular, it is proven that
is an optimal fix-free code for this class of sources. Since
is symmetric, this code is also an optimal symmetric fix-free code. Although for an anti-uniform source, the difference between the average codelength of the optimal prefix-free code and
is small (it is exactly equal to
pn), it is not straightforward to prove that
is the optimal fix-free code. In [
19], the optimality of
among symmetric fix-free codes for a family of exponential probability distributions, which is an anti-uniform source, was discussed.
In [
20], another class of fix-free codes called weakly symmetric fix-free codes was examined. A fix-free code is weakly-symmetric if the reverse of each codeword is also a codeword. In fact, every symmetric fix-free code is a weakly symmetric fix-free code, and every weakly symmetric fix-free code is a fix-free code. Thus, since the optimal code among fix-free codes and symmetric fix-free codes for anti-uniform sources is
, this code is also optimal for weakly symmetric fix-free codes.
The remainder of this paper is organized as follows. In Section 2, a sketch of the proofs of the main theorems, i.e., Theorems 1 and 3, is provided, followed by the main results of the paper. Then detailed proofs of these results are given in Section 3.
2. A Sketch of the Proofs
Since a fix-free code is also a prefix-free code, the Kraft sum of an optimal fix-free code is not greater than 1. Therefore, the Kraft sum of this code is either equal to 1 or smaller than 1.
Proposition 1. Ifthen we have It can be inferred from Proposition 1 that if the Kraft sum of an optimal fix-free code is smaller than 1, then the average codelength of this code is not better than the codelength vector
. The optimal prefix-free code for an anti-uniform source has codelength vector (1, 2, ⋯, n − 1, n − 1). Therefore, the optimal codelength vector with Kraft sum smaller than 1 for an anti-uniform source is the codelength vector (1, 2, ⋯, n−1, n). Further, the codelength vector of
is (1, 2, ⋯, n−1, n). Thus, if the Kraft sum of the optimal fix-free code for an anti-uniform source is smaller than 1, then the code
is optimal.
Proposition 2. There is no symmetric fix-free code with Kraft sum 1 for n > 2.
According to Proposition 2, the Kraft sum for an optimal symmetric fix-free code is smaller than 1. Thus, Propositions 1 and 2 prove the following theorem.
Theorem 1. The optimal symmetric fix-free code for an anti-uniform source Pn is the code.
There exist fix-free codes with Kraft sum 1, for example (00, 01, 10, 11) and (01, 000, 100, 110, 111, 0010, 0011, 1010, 1011) [
21]. Therefore, proving that the code
is the optimal fix-free code for anti-uniform sources requires that the average codelength for this code be better than every possible codelength for a fix-free code. To achieve this, we use the following theorem which was proven in [
21].
Theorem 2. [
21]
Let Ln = (
ℓ1, ⋯,
ℓn),
Mi(
Ln) = |{
j|ℓj =
i}|
for 1
≤ i ≤ max
1≤j≤n ℓj and where (
i, j)
denotes the greatest common divisor of i and j. If S(
Ln) = 1,
Mi(
Ln)
> Hi for some i and|{
ℓ1, ⋯,
ℓn}| > 1,
then no fix-free code exists with codelength vector Ln. According to the definition of
Hi, we have that
H1 = 0 and
H2 = 1. Therefore from Theorem 2, for
Ln with Kraft sum 1 and
or
there is no fix-free code.
Definition 1. For a given n, let From Theorem 2, if the Kraft sum of the optimal fix-free code is equal to 1, then the average codelength for this code is not smaller than that of the optimal codelength vector among those in
for n > 4. It can easily be verified that
for n < 7. For anti-uniform sources, the following proposition characterizes the optimal codelength vector in
for n ≥ 7.
Proposition 3. Let Pn = (
p1, ⋯,
pn−1,
pn)
be the probability vector of an anti-uniform source with p1 ≥ ⋯
≥ pn−1 ≥ pn. Then we have The last step requires that the average codelength of
is better than that of the given codelength vector in Proposition 3. This is given in the proof of the following theorem.
Theorem 3. The optimal fix-free code for an anti-uniform source Pn is for n > 4.
Note that Theorem 3 is not true for n = 4. For example, for
which is the probability vector of an anti-uniform source, the average codelength of the fix-free code (00, 01, 10, 11) is better than that of
.
3. Proofs of the Results in Section 2
Proof of Proposition 1: Let
Ln = (
ℓ1, ⋯,
ℓn−1,
ℓn) with
ℓ1 ≤ ⋯ ≤
ℓn−1 ≤
ℓn and 1 >
S(
Ln). Thus, we have
, and consequently
Therefore, we can write
Let
such that
for 1
≤ i ≤ n − 1, and
. From (
2), we have that
. According to the definition of
, we can write
and consequently
This shows that the average codelength of
is better than any other codelength vector, say
Ln, with Kraft sum smaller than 1.
Proof of Proposition 2: Suppose that the Kraft sum of Ln, which is the codelength vector of the code Cn, is equal to 1. Let codeword c = x1x2·⋯ x ℓ−10 (resp. c = x1x2·⋯ xℓ−11) with length ℓ (ℓ > 1), be the longest codeword of Cn. Since the Kraft sum of Ln is equal to 1, the codeword c′ = x1x2·⋯ xℓ−11 (resp. c′ = x1x2·⋯ xℓ−10) belongs to Cn. However, both c and c′ cannot be symmetric because x1 = 0 and x1 = 1 cannot both be true. Thus, Cn is not a symmetric fix-free code.
The following lemma will be used in the proof of Proposition 3.
Lemma 1. For n ≥ 7,
let Pn = (
p1, ⋯,
pn−1,
pn)
with p1 ≥ ⋯
≥ pn−1 ≥ pn andFurther, suppose thatand for n > 7
Then we have Proof. It can easily be verified that
consists of all permutations of 2, 3, 3, 3, 3, 3, 3. Thus,
p1 ≥ ⋯
≥ p6 ≥ p7 completes the proof for
n = 7. To prove the lemma for
n > 7, we consider two cases: (
1)
, and (
2)
. First, note that
, because
p1 ≥ ⋯
≥ pn−1 ≥ pn.
: It can easily be shown that (3, 3, 3, 3, 3, 3, 3, 3) is the only codelength vector in
with maximum codelength which is a not greater than 3. Therefore, to prove the lemma in this case it is enough to show that
. According to the first argument in this proof, the codelength vector
is a permutation of 2, 3, 3, 3, 3, 3, 3. Thus, proving that
p7 +
p8 is maximum over all probabilities in
,
i.e.,
p7 +
p8 ≥ p1, completes the proof for this case. If
p7 +
p8 < p1, then the average codelength of
, is better than that of
, because
where (a) follows from
p7 +
p8 < p1. Thus, the codelength vector (3, 3, 3, 3, 3, 3, 3) is not optimal, which is a contradiction. Therefore,
p7 +
p8 ≥ p1 and the proof for this case is complete.
: The proof for this case is similar to the proof of the Huffman algorithm. Let
,
i.e.,
S(
Ln) = 1,
M1(
Ln) = 0 and
M2(
Ln)
≤ 1, with
ℓ1 ≤ ⋯
≤ ℓn and 3
< ℓn, and let
. It can easily be shown that
S(
Ln) = 1 implies that
ℓn =
ℓn−1. Since
ℓn =
ℓn−1, we have
, and consequently
. Further, since
and
ℓn > 3,
M1(
Ln) = 0 and
M2(
Ln)
≤ 1 imply that
and
, which gives
, and we can write
where (a) follows from
, (b) follows from the definition of
and (
3), and (c) follows from (
4). Therefore, for
n > 7, since the average codelength of the given codelength vector in (
5) is equal to
, this codelength vector is optimal and the proof is complete.
□
Proof of Proposition 3. The proposition is proved by induction on
n. According to Lemma 1, the base of induction,
i.e.,
n = 7, is true. Assume that the proposition is true for all anti-uniform sources with
n−1 symbols. Let
Pn = (
p1, ⋯,
pn) be the probability vector of an anti-uniform source. Also, suppose that
. From (
1), it is obvious that
is the probability vector of an anti-uniform source and
. Since we have
, where
, from the induction assumption we can write
Therefore, we have
, and Lemma 1 completes the proof.
Proof of Theorem 3. We have that
. Therefore, for
n = 5, 6 the proof is complete. The proof for
n = 7 is the same as the proof for
n > 7, and so is omitted. Now suppose that
n > 7. In the following, it is proven that the average codelength vector of
is greater than or equal to that of
,
i.e.,
, which completes the proof.
where (a) follows from the fact that
Pn is the probability vector of an anti-uniform source,
i.e.,
for
n − 3
≥ i ≥ 1.□