Next Article in Journal
Interactive System for Similarity-Based Inspection and Assessment of the Well-Being of mHealth Users
Next Article in Special Issue
The Listsize Capacity of the Gaussian Channel with Decoder Assistance
Previous Article in Journal
Hypothetical Control of Fatal Quarrel Variability
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Encoding Individual Source Sequences for the Wiretap Channel

The Viterbi Faculty of Electrical and Computer Engineering, Technion-Israel Institute of Technology, Technion City, Haifa 3200003, Israel
Entropy 2021, 23(12), 1694; https://doi.org/10.3390/e23121694
Submission received: 9 November 2021 / Revised: 5 December 2021 / Accepted: 15 December 2021 / Published: 17 December 2021
(This article belongs to the Special Issue Wireless Networks: Information Theoretic Perspectives Ⅱ)

Abstract

:
We consider the problem of encoding a deterministic source sequence (i.e., individual sequence) for the degraded wiretap channel by means of an encoder and decoder that can both be implemented as finite-state machines. Our first main result is a necessary condition for both reliable and secure transmission in terms of the given source sequence, the bandwidth expansion factor, the secrecy capacity, the number of states of the encoder and the number of states of the decoder. Equivalently, this necessary condition can be presented as a converse bound (i.e., a lower bound) on the smallest achievable bandwidth expansion factor. The bound is asymptotically achievable by Lempel–Ziv compression followed by good channel coding for the wiretap channel. Given that the lower bound is saturated, we also derive a lower bound on the minimum necessary rate of purely random bits needed for local randomness at the encoder in order to meet the security constraint. This bound too is achieved by the same achievability scheme. Finally, we extend the main results to the case where the legitimate decoder has access to a side information sequence, which is another individual sequence that may be related to the source sequence, and a noisy version of the side information sequence leaks to the wiretapper.

1. Introduction

In his seminal paper, Wyner [1] introduced the wiretap channel as a model of secure communication over a degraded broadcast channel, without using a secret key, where the legitimate receiver has access to the output of the good channel and the wiretapper receives the output of the bad channel. The main idea is that the excess noise at the output of the wiretapper channel is utilized to secure the message intended to the legitimate receiver. Wyner fully characterized the best achievable trade-off between reliable communication to the legitimate receiver and the equivocation rate at the wiretapper, which was quantified in terms of the conditional entropy of the source given the output of the wiretapper channel. One of the most important concepts, introduced by Wyner, was the secrecy capacity, that is, the supremum of all coding rates that allow both reliable decoding at the legitimate receiver and full secrecy, where the equivocation rate saturates at the (unconditional) entropy rate of the source, or equivalently, the normalized mutual information between the source and the wiretap channel output is vanishingly small for large blocklength. The idea behind the construction of a good code for the wiretap channel is basically the same as the idea of binning: one designs a big code, that can be reliably decoded at the legitimate receiver, which is subdivided into smaller codes that are fed by purely random bits that are unrelated to the secret message. Each such sub-code can be reliably decoded individually by the wiretapper to its full capacity, thus leaving no further decoding capability for the remaining bits, which all belong to the real secret message.
During the nearly five decades that have passed since [1] was published, the wiretap channel model was extended and further developed in many aspects. We mention here just a few. Three years after Wyner, Csiszár and Körner [2] extended the wiretap channel to a general broadcast channel that is not necessarily degraded, allowing also a common message intended to both receivers. In the same year, Leung-Yan-Cheong and Hellman [3], studied the Gaussian wiretap channel, and proved, among other things, that its secrecy capacity is equal to the difference between the capacity of the legitimate channel and that of the wiretap channel. In [4], Ozarow and Wyner considered a somewhat different model, known as the type II wiretap channel, where the channel to the legitimate receiver is clean (noiseless), and the wiretapper can access a subset of the coded bits. In [5], Yamamoto extended the wiretap channel to include two parallel broadcast channels that connect one encoder and one legitimate decoder, and both channels are wiretapped by wiretappers that do not cooperate with each other. A few years later, the same author [6] further developed the scope of [1] in two ways: first, by allowing a private secret key to be shared between the encoder and the legitimate receiver, and secondly, by allowing a given distortion in the reproducing the source at the legitimate receiver. The main coding theorem of [6] suggests a three-fold separation principle, which asserts that no asymptotic optimality is lost if the encoder first applies a good lossy source code, then encrypts the compressed bits, and finally, applies a good channel code for the wiretap channel. In [7], this model in turn was generalized to allow source side information at the decoder and at the wiretapper in a degraded structure with application to systematic coding for the wiretap channel. The Gaussian wiretap channel model of [3] was also extended in two ways: the first is the Gaussian multiple access wiretap channel of [8], and the second is Gaussian interference wiretap channel of [9,10], where the encoder has access to the interference signal as side information. Wiretap channels with feedback were considered in [11], where it was shown that feedback is best used for the purpose of sharing a secret key as in [6,7]. More recent research efforts were dedicated to strengthening the secrecy metric from weak secrecy to strong secrecy, where the mutual information between the source and the wiretap channel output vanishes, even without normalization by the blocklength, as well as to semantic security, which is similar but refers even to the worst-case message source distribution; see, for example, Refs. [12,13,14,15,16], (Section 3.3 in [14]).
In this work, we look at Wyner’s wiretap channel model from a different perspective. Following the individual sequence approach pioneered by Ziv in [17,18,19], and continued in later works, such as [20,21], we consider the problem of encoding a deterministic source sequence (i.e., an individual sequence) for the degraded wiretap channel using finite-state encoders and finite-state decoders. One of the non-trivial issues associated with individual sequences, in the context of the wiretap channel, is how to define the security metric, as there is no probability distribution assigned to the source, and therefore, the equivocation, or the mutual information between the source and the wiretap channel output, cannot be well defined. In [20], a similar dilemma was encountered in the context of private key encryption of individual sequences, and in the converse theorem therein, it was assumed that the system is perfectly secure in the sense that the probability distribution of the cryptogram does not depend on the source sequence. In principle, it is possible to apply the same approach here, where the word ‘cryptogram’ is replaced by the ‘wiretap channel output’. However, in order to handle residual dependencies, which will always exist, it would be better to use a security metric that quantifies those small dependencies. To this end, it makes sense to adopt the above-mentioned maximum mutual information security metric (or, equivalently the semantic security metric), where the maximum is over all input assignments. After this maximization, this quantity depends only on the ‘channel’ between the source and the wiretap channel output.
Our first main result is a necessary condition (i.e., a converse to a coding theorem) for both reliable and secure transmission, which depends on: (i) the given individual source sequence, (ii) the bandwidth expansion factor, (iii) the secrecy capacity, (iv) the number of states of the encoder, (v) the number of states of the decoder, (vi) the allowed bit error probability at the legitimate decoder and (vii) the allowed maximum mutual information secrecy. Equivalently, this necessary condition can be presented as a converse bound (i.e., a lower bound) to the smallest achievable bandwidth expansion factor. The bound is asymptotically achievable by Lempel–Ziv (LZ) compression followed by a good channel coding scheme for the wiretap channel. Given that this lower bound is saturated, we then derive also a lower bound on the minimum necessary rate of purely random bits needed for adequate local randomness at the encoder, in order to meet the security constraint. This bound too is achieved by the same achievability scheme, a fact which may be of independent interest regardless of individual sequences and finite-state encoders and decoders (i.e., also for ordinary block codes in the traditional probabilistic setting). Finally, we extend the main results to the case where the legitimate decoder has access to a side information sequence, which is another individual sequence that may be related to the source sequence, and where a noisy version of the side information sequence leaks to the wiretapper. It turns out that in this case, the best strategy is the same as if one assumes that the wiretapper sees the clean side information sequence. While this may not be surprising as far as sufficiency is concerned (i.e., as an achievability result), it is less obvious in the context of necessity (i.e., a converse theorem).
The remaining part of this article is organized as follows. In Section 2, we establish the notation, provide some definitions and formalize the problem setting. In Section 3, we provide the main results of this article and discuss them in detail. In Section 4, the extension that incorporates side information is presented. Finally, in Section 5, the proofs of the main theorems are given.

2. Notation, Definitions, and Problem Setting

2.1. Notation

Throughout this paper, random variables are denoted by capital letters; specific values they may take are denoted by the corresponding lower case letters; and their alphabets are denoted by calligraphic letters. Random vectors, their realizations, and their alphabets are denoted, respectively, by capital letters, the corresponding lower case letters and calligraphic letters, all superscripted by their dimensions. For example, the random vector X n = ( X 1 , , X n ) , (n – positive integer) may take a specific vector value x n = ( x 1 , , x n ) in X n , the n-th order Cartesian power of X , which is the alphabet of each component of this vector. Infinite sequences are denoted using the bold face font, e.g., x = ( x 1 , x 2 , ) . Segments of vectors are denoted by subscripts and superscripts that correspond to the start and the end locations; for example, x i j , for i < j integers, denotes ( x i , x i + 1 , , x j ) . When i = 1 , the subscript is omitted.
Sources and channels are denoted by the letter P or Q, subscripted by the names of the relevant random variables/vectors and their conditionings, if applicable, following the standard notation conventions, e.g., Q X , P Y | X , and so on, or by abbreviated names that describe their functionality. When there is no room for ambiguity, these subscripts are omitted. The probability of an event E will be denoted by Pr { E } , and the expectation operator with respect to (w.r.t.) a probability distribution P is denoted by E P { · } . Again, the subscript is omitted if the underlying probability distribution is clear from the context or explicitly explained in the following text. The indicator function of an event E is denoted by 1 { E } , that is, 1 { E } = 1 if E occurs; otherwise, 1 { E } = 0 .
Throughout considerably large parts of the paper, the analysis is carried out w.r.t. joint distributions that involve several random variables. Some of these random variables are induced from empirical distributions of deterministic sequences, while others are ordinary random variables. Random variables from the former kind are denoted with ‘hats’. As a simple example, consider a deterministic sequence, x n , that is fed as an input to a memoryless channel defined by a single-letter transition matrix, { P Y | X , x X , y Y } , and let y n denote a realization of the corresponding channel output. Let P X ^ Y ^ ( x , y ) = 1 n i = 1 n 1 { x i = x , y i = y } denote the joint empirical distribution induced from ( x n , y n ) . In addition to P X ^ Y ^ ( x , y ) , we also define P X ^ Y ( x , y ) = E { P X ^ Y ^ ( x , y ) } , where now Y is an ordinary random variable. Clearly, the relation between the two distributions is given by P X ^ Y ( x , y ) = P X ^ ( x ) · P Y | X ( y | x ) , where P X ^ ( x ) = y P X ^ Y ^ ( x , y ) is the empirical marginal of X ^ . Such mixed joint distributions underlie certain information-theoretic quantities, for example, I ( X ^ ; Y ) and H ( Y | X ^ ) denote, respectively, the mutual information between X ^ and Y and the conditional entropy of Y given X ^ , both induced from P X ^ Y . The same notation rules are applicable in more involved situations too.

2.2. Definitions and Problem Setting

We consider the system configuration of the degraded wiretap channel, depicted in Figure 1. Let u = ( u 1 , u 2 , ) be a deterministic source sequence (i.e., individual sequence), whose symbols take values in a finite alphabet, U , of size α . This source sequence is to be conveyed reliably to a legitimate decoder while keeping it secret from a wiretapper, as described below. The encoding mechanism is as follows. The source sequence, u is first is divided into chunks of length k, u ˜ i = u i k + 1 i k + k U k , i = 0 , 1 , 2 , , which are fed into a stochastic finite-state encoder, defined by the following equations:
Pr { X ˜ i = x ˜ | u ˜ i = u ˜ , s i e = s } = P ( x ˜ | u ˜ , s ) , i = 0 , 1 , 2 ,
s i + 1 e = h ( u ˜ i , s i e ) , i = 0 , 1 , 2 , .
We allow a stochastic encoder, in view of the fact that even the traditional, probabilistic setting, optimal coding for the wiretap channel (see [1] and later works) must be randomized in order to meet the security requirements. Here, X ˜ i is a random vector taking values in X m , X being the β -ary input alphabet of the channel and m being a positive integer; x ˜ X m is a realization of X ˜ i ; and s i e is the state of the encoder at time i, which designates the memory of the encoder with regard to the past of the source sequence. In other words, at time instant i, whatever the encoder ‘remembers’ from ( u 1 , u 2 , , u i 1 ) is stored in the variable s i e (for example, in the case of trellis coding, it can be a shift register of finite length, say p, that stores the most recent symbols of p, ( u i p , u i p + 1 , , u i 1 ) , or, in block coding, it can be the contents of the current block starting from its beginning up to the present). The state, s i e , takes values in a finite set of states, S e , of size q e . In the above equation, the variable u ˜ is any member of U k . The function h : U k × S e S e is called the next-state function of the encoder. (More generally, we could define both s i + 1 e and x ˜ i to be random functions of the ( u ˜ i , s i e ) by a conditional joint distribution, Pr { X ˜ i = x ˜ , s i + 1 e = s | u ˜ i = u ˜ , s i e = s } . However, it makes sense to let the encoder state sequence evolve deterministically in response to the input u since the state designates the memory of the encoder to past inputs.) Finally, P ( x ˜ | u ˜ , s ) , u ˜ U k , s S e , x ˜ X m , is a conditional probability distribution function, i.e., { P ( x ˜ | u ˜ , s ) } are all non-negative and x ˜ P ( x ˜ | u ˜ , s ) = 1 for all ( u ˜ , s ) U k × S e . The vector x ˜ i designates the current output vector from the encoder in response to the current input source vector, u ˜ i and its current state, s i e . Without loss of generality, we assume that the initial state of the encoder, s 0 e , is some fixed member of S e . The ratio
λ = m k
is referred to as the bandwidth expansion factor. It should be pointed out that the parameters k and m are fixed integers, which are not necessarily large (e.g., k = 2 and m = 3 are valid values of k and m). The concatenation of the output vectors from the encoder, x ˜ 0 , x ˜ 1 , , is viewed as a sequence chunks of channel input symbols, x 1 , x 2 , , with x ˜ i = x i m + 1 i m + m , similarly to the above-defined partition of the source sequence.
The sequence of encoder outputs, x 1 , x 2 , , is fed into a discrete memoryless channel (DMC), henceforth referred to as the main channel, whose corresponding outputs, y 1 , y 2 , , are generated according to
Pr { Y N = y N | X N = x N } = Q M ( y N | x N ) = i = 1 N Q M ( y i | x i ) ,
for every positive integer N and every x N X N and y N Y N . The channel output symbols, { y i } , take values in a finite alphabet, Y , of size γ .
The sequence of channel outputs, y 1 , y 2 , , is divided into chunks of length m, y ˜ i = y i m + 1 i m + m , i = 0 , 1 , 2 , , which are fed into a deterministic finite-state decoder, defined according to the following recursive equations:
v ˜ i = f ( y ˜ i , s i d )
s i + 1 d = g ( y ˜ i , s i d ) ,
where the variables in the equations are defined as follows: { s i d } is the sequence of states of the decoder (which again, designate the finite memory, this time, at the decoder). Each s i d takes values in a finite set, S d of size q d . The variable v ˜ i U k is the i-th chunk of k source reconstruction symbols, i.e., v ˜ i = v i k + 1 i k + k , i = 0 , 1 , , which form the decoder output. The function f : Y m × S d U k is called the output function of the decoder and the function g : Y m × S d S d is the next-state function of the decoder. The concatenation of the decoder output vectors, v ˜ 0 , v ˜ 1 , , forms the entire stream of reconstruction symbols, v 1 , v 2 , .
The output of the main channel, y 1 , y 2 , , is fed into another DMC, henceforth referred to as the wiretap channel, which generates, in response, a corresponding sequence, z 1 , z 2 , , according to
Pr { Z N = z N | Y N = y N } = Q W ( z N | y N ) = i = 1 N Q W ( z i | y i ) ,
where { Z i } and { z i } take values in a finite alphabet Z . We denote the cascade of channels Q M and Q W by Q M W , that is
Q M W ( z | x ) = y Y Q M ( y | x ) Q W ( z | y ) .
We seek a communication system ( P , h , f , g ) which satisfies two requirements:
  • For a given ϵ r > 0 , the system satisfies the following reliability requirement: The bit error probability is guaranteed to be less than ϵ r , i.e.,
    P b = 1 k i = 1 k Pr { V i u i } ϵ r
    for every ( u 1 , , u k ) and every combination of initial states of the encoder and the decoder, where Pr { · } is defined w.r.t. the randomness of the encoder and the main channel.
  • For a given ϵ s > 0 , the system satisfies the following security requirement: For every sufficiently large positive integer n,
    max μ I μ ( U n ; Z N ) n ϵ s ,
    where N = n λ and I μ ( U n ; Z N ) is the mutual information between U n and Z N , induced by an input distribution μ = { μ ( u n ) , u n U n } and the system, { P ( z N | u n ) , u n U n , z N Z N } .
As for the reliability requirement, note that the larger k is, the less stringent the requirement becomes. Concerning the security requirement, ideally, we would like to have perfect secrecy, which means that P ( z N | u n ) would be independent of u n (see also [20]), but it is more realistic to allow a small deviation from this idealization. This security metric is actually the maximum mutual information metric, or equivalently (see [15]) the semantic security, as mentioned in the Introduction.

2.3. Preliminaries and Background

We need two more definitions along with some background associated with them. The first is the secrecy capacity [1,14], which is the supremum of all coding rates for which there exist block codes that maintain both an arbitrarily small error probability at the legitimate decoder and an equivocation arbitrarily close to the unconditional entropy of the source. The secrecy capacity is given by
C s = max P X I ( X ; Y | Z ) = max P X [ I ( X ; Y ) I ( X ; Z ) ] ,
with P X Y Z ( x , y , z ) = P X ( x ) × Q M ( y | x ) Q W ( z | y ) for all ( x , y , z ) X × Y × Z .
The second quantity we need to define is the LZ complexity [22]. In their famous paper [22], Ziv and Lempel actually developed a deterministic counterpart of source coding theory, where instead of imposing assumptions on probabilistic mechanisms that generate the data (i.e., memoryless sources, Markov sources, and general stationary sources), whose relevance to real-world data compression may be subject to dispute, they considered arbitrary, deterministic source sequences (i.e., individual sequences, in their terminology), but they imposed instead a limitation on the resources of the encoder (or the data compression algorithm): they assumed that it has limited storage capability (i.e., limited memory) of past data when encoding the current source symbol. This limited storage was modeled in terms of a finite-state machine, where the state variable of the encoder evolves recursively in time in response to the input and designates the information that the encoder ‘remembers’ from the past input (just like in the model description in Section 2.2 above). As mentioned earlier, a simple example of such a state variable can be the contents of a finite shift register, fed sequentially by the source sequence in which case the state contains a finite number of the most recent source symbols. This individual-sequence approach is appealing, because it is much more realistic to assume practical limitations on the encoder (which is under the control of the system designer) than to make assumptions on the statistics of the data to be compressed.
Ziv and Lempel developed an asymptotically optimal, practical compression algorithm (which is used in almost every computer), that is well known as the Lempel–Ziv (LZ) algorithm. This algorithm has several variants. One of them, which is called the LZ78 algorithm (where ‘78’ designates the year 1978), is based on the notion of incremental parsing: given the the source vector, u n , the incremental parsing procedure sequentially parses this sequence into distinct phrases such that each new parsed phrase is the shortest string that has not been obtained before as a phrase, with a possible exception of the last phrase, which might be incomplete. Let c ( u n ) denote the number of resulting phrases. For example, if n = 10 and u 10 = ( 0000110110 ) , then incremental parsing (from left to right) yields ( 0 , 00 , 01 , 1 , 011 , 0 ) and so, c ( u 10 ) = 6 . We define the LZ complexity of the individual sequence, u n , as
ρ LZ ( u n ) = c ( u n ) log c ( u n ) n .
As was shown by Ziv and Lempel in their seminal paper [22], for large n, the LZ complexity, ρ LZ ( u n ) , is essentially the best compression ratio that can be achieved by any information lossless, finite-state encoder (up to some negligibly small terms, for large n), and it can be viewed as the individual-sequence analogue of the entropy rate.

3. Results

Before moving on to present our first main result, a simple comment is in order. Even in the traditional probabilistic setting, given a source with entropy H and a channel with capacity C, reliable communication cannot be accomplished unless H λ C , where λ is the bandwidth expansion factor. Since both H and C are given and only λ is under the control of the system designer, it is natural to state this condition as a lower bound to bandwidth expansion factor, i.e., λ H / C . By the same token, in the presence of a secrecy constraint, λ must not fall below H / C s . Our converse theorems for individual sequences are presented in the same spirit, where the entropy H at the numerator is replaced by an expression whose main term is the Lempel–Ziv compressibility.
We assume, without essential loss of generality, that k divides n (otherwise, omit the last ( n mod k ) symbols of u n and replace n by k · n / k without affecting the asymptotic behavior as n ). Our first main result is the following.
Theorem 1.
Consider the problem setting defined in Section 2. If there exists a stochastic encoder with q e states and a decoder with q d states that together satisfy the reliability constraint (9) and the security constraint (10), then the bandwidth expansion factor λ must be lower bounded as follows.
λ ρ L Z ( u n ) Δ ( ϵ r ) ϵ s ζ n ( q d , k ) C s ,
where
Δ ( ϵ r ) = h 2 ( ϵ r ) + ϵ r · log ( α 1 ) ,
with h 2 ( ϵ r ) = ϵ r log ϵ r ( 1 ϵ r ) log ( 1 ϵ r ) being the binary entropy function, and
ζ n ( q d , k ) = min { d i v i d e s n / k } log q d + 1 k + 2 k ( log α + 1 ) 2 ( 1 ϵ n ) log n + 2 k α 2 k log α n ,
with ϵ n 0 as n .
The proof of Theorem 1, like all other proofs in this article, is deferred to Section 5.
Discussion. 
A few comments are in order with regard to Theorem 1.
1.
Irrelevance of q e . It is interesting to note that as far as the encoding and decoding resources are concerned, the lower bound depends on k and q d , but not on the number of states of the encoder, q e . This means that the same lower bound continues to hold, even if the encoder has an unlimited number of states. Pushing this to the extreme, even if the encoder has room to store the entire past, the lower bound of Theorem 1 would remain unaltered. The crucial bottleneck is, therefore, in the finite memory resources associated with the decoder, where the memory may help to reconstruct the source by exploiting empirical dependencies with the past. The dependence on q e , however, appear later when we discuss local randomness resources as well as in the extension to the case of decoder side information.
2.
The redundancy term ζ n ( q d , k ) . A technical comment is in order concerning the term ζ n ( q d , k ) , which involves minimization over all divisors of n / k , where we have already assumed that n / k is integer. Strictly speaking, if n / k happens to be a prime, this minimization is not very meaningful, as ζ n ( q d , k ) would be relatively large. If this the case, a better bound is obtained if one omits some of the last symbols of u n , thereby reducing n to, say, n so that n / k has a richer set of factors. Consider, for example, the choice = n = log n (instead of minimizing over ) and replace n / k by the n / k ( n / k mod n ) , without essential loss of tightness. This way, ζ n ( q d , k ) would tend to zero as n , for fixed k and q d .
3.
Achievability. Having established that ζ n ( q d , k ) 0 , and given that ϵ r and ϵ s are small, it is clear that the main term at the numerator of the lower bound of Theorem 1 is the term ρ LZ ( u n ) , which is, as mentioned earlier, the individual-sequence analogue of the entropy of the source [22]. In other words, λ cannot be much smaller than λ L ( u n ) = ρ LZ ( u n ) / C s . A matching achievability scheme would most naturally be based on separation: first apply variable-rate compression of u n to about n ρ LZ ( u n ) bits using the LZ algorithm [22], and then feed the resulting compressed bit-stream into a good code for the wiretap channel [1] with codewords of length about
N = n λ L ( u n ) n ρ LZ ( u n ) C s ( 1 δ ) ,
where δ is an arbitrarily small (but positive) margin to keep the coding rate strictly smaller than C s . However, to this end, the decoder must know N. One possible solution is that before the actual encoding of each u n , one would use a separate, auxiliary fixed code that encodes the value of the number of compressed bits, n ρ LZ ( u n ) , using log ( n log α ) bits (as n log α is about the number of possible values that n ρ LZ ( u n ) can take) and protect it using a channel code of rate less than C s ( 1 δ ) . Since the length of this auxiliary code grows only logarithmically with n (as opposed to the ‘linear’ growth of n ρ LZ ( u n ) ), the overhead in using the auxiliary code is asymptotically negligible. The auxiliary code and the main code are used alternately: first the auxiliary code, and then the main code for each n-tuple of the source. The main channel code is actually an array of codes, one for each possible value of n ρ LZ ( u n ) . Once the auxiliary decoder has decoded this number, the corresponding main decoder is used. Overall, the resulting bandwidth expansion factor is about
λ n ρ LZ ( u n ) + log ( n log α ) n C s ( 1 δ ) = ρ LZ ( u n ) C s ( 1 δ ) + O log n n .
Another, perhaps simpler and better, approach is to use the LZ algorithm in the mode of a variable-to-fixed length code: let the length of the channel codeword, N, be fixed, and start to compress u = ( u 1 , u 2 , ) until obtaining n ρ LZ ( u n ) = N · C s ( 1 δ ) compressed bits. Then,
λ = N n = ρ LZ ( u n ) C s ( 1 δ ) .
Of course, these coding schemes require decoder memory that grows exponentially in n, and not just a fixed number, q d , and therefore strictly speaking, there is a gap between the achievability and the converse result of Theorem 2. However, this gap is closed asymptotically, once we take the limit of q d 0 after the limit n , and we consider successive application of these codes over many blocks. The same approach appears also in [17,18,19,22] as well as in later related work.
This concludes the discussion on Theorem 1. □
We next focus on local randomness resources that are necessary when the full secrecy capacity is exploited. Specifically, suppose that the stochastic encoder { P ( x ˜ | u ˜ , s ) , x ˜ X n , u ˜ U k , s S e } is implemented as a deterministic encoder with an additional input of purely random bits, i.e.,
x ˜ i = a ( u ˜ i , s i e , b ˜ i ) ,
where b ˜ i = b i j + 1 i j + j is a string of j purely random bits. The question is the following: how large must j be in order to achieve full secrecy? Equivalently, what is the minimum necessary rate of random bits for local randomness at the encoder for secure coding at the maximum reliable rate? In fact, this question may be interesting on its own right, regardless of the individual-sequence setting and finite-state encoders and decoders, but even for ordinary block coding (which is the special case of q e = q d = 1 ) and in the traditional probabilistic setting. The following theorem answers this question.
Theorem 2.
Consider the problem setting defined in Section 2 and let λ meet the lower bound of Theorem 1. If there exists an encoder (19) with q e states and a decoder with q d states that jointly satisfy the reliability constraint (9) and the security constraint (10), then
j m I ( X ; Z ) k ϵ s log q e
where X is the random variable that achieves C s and ℓ is the achiever of ζ n ( q d , k ) .
Note that the lower bound of Theorem 2 depends on q e , as opposed to Theorem 1, where it depends only on q d . Since ϵ s is assumed small and , it is clear that main term is m I ( X ; Z ) , i.e., the bit rate must be essentially at least as large as I ( X ; Z ) random bits per channel use, or equivalently, λ I ( X ; Z ) bits per source symbol. It is interesting to note that Wyner’s code [1] asymptotically achieves this bound when the coding rate saturates the secrecy capacity because the subcode that can be decoded by the wiretapper (within each given bin) is of the rate of about I ( X ; Z ) , and it encodes just the bits of the local randomness. So when working at the full secrecy capacity, Wyner’s code is optimal not only in terms of the optimal trade-off between reliability and security, but also in terms of minimum consumption of local, purely random bits.

4. Side Information at the Decoder with Partial Leakage to the Wiretapper

Consider next an extension of our model to the case where there are side information sequences, w n = ( w 1 , , w n ) and w ˙ n = ( w ˙ 1 , , w ˙ n ) , available to the decoder and the wiretapper, respectively; see Figure 2. For the purpose of a converse theorem, we assume that w n is available to the encoder too, whereas in the achievability part, we comment also on the case where it is not. We assume that w n is a deterministic sequence, but w ˙ n is a realization of a random vector W ˙ n = ( W ˙ 1 , , W ˙ n ) , which is a noisy version of w n . In other words, it is generated from w n by another memoryless channel, Q W ˙ n | W n ( w ˙ n | w n ) = i = 1 n Q W ˙ | W ( w ˙ i | w i ) . The symbols of { w i } and { w ˙ i } take values in finite alphabets, W and W ˙ , respectively. There are two extreme important special cases: (i) W ˙ n = w n almost surely, which is the case of totally insecure side information that fully leaks to the wiretapper, and (ii) W ˙ n is degenerated (or independent of w n ), which is the case of secure side information with no leakage to the wiretapper. Every intermediate situation between these two extremes is a situation of partial leakage. The finite-state encoder model is now re-defined according to
Pr { X ˜ i = x ˜ | u ˜ i = u ˜ , w ˜ i = w ˜ , s i e = s } = P ( x ˜ | u ˜ , w ˜ , s ) , i = 0 , 1 , 2 ,
s i + 1 e = h ( u ˜ i , w ˜ i , s i e ) , i = 0 , 1 , 2 , ,
where w ˜ i = w i k + 1 i k + k , i = 0 , 1 , , n / k 1 . Likewise, the decoder is given by
v ˜ i = f ( y ˜ i , w ˜ i , s i d )
s i + 1 d = g ( y ˜ i , w ˜ i , s i d ) ,
and the wiretapper has access to Z N and W ˙ n . Accordingly, the security constraint is modified as follows: for a given ϵ s > 0 and for every sufficiently large n,
max μ I μ ( U n ; Z N | W ˙ n ) n ϵ s ,
where I μ ( U n ; Z N | W ˙ n ) is the conditional mutual information between U n and Z N given W ˙ n , induced by μ = { μ ( u n , w ˙ n ) , u n U n , w ˙ n W ˙ n } and the system, { P ( z N | u n ) , u n U n , z N Z N } , where μ ( u n , w ˙ n ) = w n μ ( u n , w n ) Q W ˙ n | W n ( w ˙ n | w n ) .
In order to present the extension of Theorem 1 to incorporate side information, we first need to define the extension of the LZ complexity to include side information, namely, to define the conditional LZ complexity (see also [23]). Given u n and w n , let us apply the incremental parsing procedure of the LZ algorithm to the sequence of pairs ( ( u 1 , w 1 ) , ( u 2 , w 2 ) , , ( u n , w n ) ) . According to this procedure, all phrases are distinct with a possible exception of the last phrase, which might be incomplete. Let c ( u n , w n ) denote the number of distinct phrases. As an example (which appears also in [23]), if
u 6 = 0 | 1 | 0 0 | 0 1 | w 6 = 0 | 1 | 0 1 | 0 1 |
then c ( u 6 , w 6 ) = 4 . Let c ( w n ) denote the resulting number of distinct phrases of w n , and let w ( l ) denote the l-th distinct w-phrase, l = 1 , 2 , . . . , c ( w n ) . In the above example, c ( w 6 ) = 3 . Denote by c l ( u n | w n ) the number of occurrences of w ( l ) in the parsing of w n , or equivalently, the number of distinct u-phrases that jointly appear with w ( l ) . Clearly, l = 1 c ( w n ) c l ( u n | w n ) = c ( u n , w n ) . In the above example, w ( 1 ) = 0 , w ( 2 ) = 1 , w ( 3 ) = 01 , c 1 ( u 6 | w 6 ) = c 2 ( u 6 | w 6 ) = 1 , and c 3 ( u 6 | w 6 ) = 2 . Now, the conditional LZ complexity of u n given w n is defined as
ρ L Z ( u n | w n ) = 1 n l = 1 c ( w n ) c l ( u n | w n ) log c l ( u n | w n ) .
We are now ready to present the main result of this section.
Theorem 3.
Consider the problem setting defined in Section 2 along with the above–mentioned modifications to incorporate side information. If there exists a stochastic encoder with q e states and a decoder with q d states that together satisfy the reliability constraint (9) and the security constraint (25), then its bandwidth expansion factor λ must be lower bounded as follows.
λ ρ L Z ( u n | w n ) Δ ( ϵ r ) ϵ s η n ( q e · q d , k ) C s ,
where
η n ( q e · q d , k ) = min { d i v i d e s n / k } log ( q d q e ) + 1 k + log ( 4 A 2 ) ( 1 ϵ n ) log n + A 2 log ( 4 A 2 ) n ,
with ϵ n 0 as n and A = [ ( α ω ) k + 1 1 ] / [ α ω 1 ] , ω being the size of W .
Note that the lower bound of Theorem 3 does not depend on the noisy side information at the wiretapper or on the channel Q W ˙ | W that generates it from w n . It depends only on u n and w n in terms of the data available in the system. Clearly, as it is a converse theorem, if it allows the side information to be available also at the encoder, then it definitely applies also to the case where the encoder does not have access to w n . Interestingly, the encoder and the legitimate decoder act as if the wiretapper has the clean side information, w n . While it is quite obvious that protection against availability of w n at the wiretapper is sufficient for protection against availability of W ˙ n (as W ˙ n is a degraded version of w n ), it is not quite trivial that this should be also necessary, as the above converse theorem asserts. It is also interesting to note that here, the bound depends also on q e , and not only q d , as in Theorem 1. However, this dependence on q e disappears in the special case where W ˙ n = w n with probability one.
We next discuss the achievability of the lower bound of Theorem 3. If the encoder has access to w n , then the first step would be to apply the conditional LZ algorithm (see ([23], proof of Lemma 2) [24]), thus compressing u n to about n ρ LZ ( u n | w n ) bits. The second step would be good channel coding for the wiretap channel, using the same methods as described in the previous section. If, however, the encoder does not have access to w n , the channel coding part is still as before, but the situation with the source coding part is somewhat more involved since neither the encoder nor the decoder can calculate the target bit rate, ρ LZ ( u n | w n ) , as neither party has access to both u n and w n . However, this source coding rate can essentially be achieved, provided that there is a low-rate noiseless feedback channel from the legitimate decoder to the encoder. The following scheme is in the spirit of the one proposed by Draper [25], but with a few modifications.
The encoder implements random binning for all source sequences in U n , that is, for each member of U n an index is drawn independently, under the uniform distribution over { 0 , 1 , 2 , , α n 1 } , which is represented by its binary expansion, b ( u n ) , of length n log α bits. We select a large positive integer r, but keep r n (say, r = n or r = log 2 n ). The encoder transmits the bits of b ( u n ) incrementally, r bits at a time, until it receives from the decoder ACK. Each chunk of r bits is fed into a good channel code for the wiretap channel, at a rate slightly less than C s . At the decoder side, this channel code is decoded (correctly, with high probability, for large r). Then, for each i ( i = 1 , 2 , ), after having decoded the i-th chunk of r bits of b ( u n ) , the decoder creates the list A i ( u n ) = { u ˙ n : [ b ( u ˙ n ) ] i r = [ b ( u n ) ] i r } , where [ b ( u ˙ n ) ] l denotes the string formed by the first l bits of b ( u ˙ n ) . For each u ˙ n A i ( u n ) , the decoder calculates ρ LZ ( u ˙ n | w n ) . We fix an arbitrarily small δ > 0 , which controls the trade-off between error probability and compression rate. If n ρ LZ ( u ˙ n | w n ) i · r n δ for some u ˙ n A i ( u n ) , the decoder sends ACK on the feedback channel and outputs the reconstruction, u ˙ n , with the smallest ρ LZ ( u ˙ n | w n ) among all members of A i ( u n ) . If no member of A i ( u n ) satisfies n ρ LZ ( u ˙ n | w n ) i · r n δ , the receiver waits for the next chunk of r compressed bits, and it does not send ACK. The probability of source-coding error after the i-th chunk is upper bounded by
P e ( i ) ( a ) | { u ˙ n u n : n ρ LZ ( u ˙ n | w n ) i · r n δ } | · 2 i · r ( b ) exp 2 i · r n δ + O n log ( log n ) log n · 2 i · r = exp 2 n δ + O n log ( log n ) log n 0 a s n ,
where in (a), the factor 2 i · r is the probability that [ b ( u ˙ n ) ] i r = [ b ( u n ) ] i r for each member of the set { u ˙ n u n : n ρ LZ ( u ˙ n | w n ) i · r n δ } and (b) is based on ([23], Equation (A.13)). Clearly, it is guaranteed that an ACK is received at the encoder (and hence the transmission stops), no later than after the transmission of chunk no. i , where i is the smallest integer i such that i · r n ρ LZ ( u n | w n ) + n δ , namely, i = [ n ρ LZ ( u n | w n ) + n δ ] / r , which is the stage at which at least the correct source sequence begins to satisfy the condition n ρ LZ ( u n | w n ) i · r n δ . Therefore, the compression ratio is no worse than i · r / n = n [ ρ LZ ( u n | w n ) + δ ] / r · r / n ρ LZ ( u n | w n ) + δ + r / n . The overall probability of source-coding error is then upper bounded by
P e = Pr i = 1 i { error   at   state i } i = 1 i P e ( i ) n log α r + 1 · exp 2 n δ + O n log ( log n ) log n ,
which still tends to zero as n . As for channel-coding errors, the probability that at least one chunk is decoded incorrectly is upper bounded by ( n log α r + 1 ) · e r E , where E is an achievable error exponent of channel coding at the given rate. Thus, if r grows at any rate faster than logarithmic, but sub-linearly in n, then the overall channel-coding error probability tends to zero and, at the same time, the compression redundancy, r / n , tends to zero too.
To show that the security constraint (25) is satisfied too, consider an arbitrary assignment μ of random vectors ( U n , W n ) , and let us denote by B the string of I ( X N ; Z N ) N ϵ bits of local randomness in Wyner’s code [1]. Then,
I ( X N ; Z N ) = H ( Z N ) H ( Z N | X N ) ( a ) H ( Z N ) H ( Z N | U n , B ) ( b ) H ( Z N | W ˙ n ) H ( Z N | U n , B ) = ( c ) H ( Z N | W ˙ n ) H ( Z N | U n , B , W ˙ n ) = I ( U n , B ; Z N | W ˙ n ) = H ( U n , B | W ˙ n ) H ( U n , B | Z n , W ˙ n ) = H ( U n | W ˙ n ) + H ( B | U n , W ˙ n ) H ( U n | Z N , W ˙ n ) H ( B | Z N , W ˙ n , U n ) = ( d ) H ( U n | W ˙ n ) + H ( B ) H ( U n | Z N , W ˙ n ) H ( B | Z N , W ˙ n , U n ) ( e ) H ( U n | W ˙ n ) + H ( B ) H ( U n | Z N , W ˙ n ) H ( B | Z N , U n ) ( f ) H ( U n | W ˙ n ) + [ I ( X N ; Z N ) N ϵ ] H ( U n | Z N , W ˙ n ) n δ n = I ( X N ; Z N ) + I μ ( U n ; Z N | W ˙ n ) n ( λ ϵ + δ m ) ,
where (a) is due to ( U n , B ) X N Z N being a Markov chain, (b) is due to conditioning reducing entropy, (c) is due to W ˙ n ( U n , B ) Z N being a Markov chain, (d) is due to B being independent of ( U n , W ˙ n ) , (e) is due to conditioning reducing entropy, and (f) is due to, in Wyner coding, B being able to be reliably decoded given that ( Z N , U n ) ( δ n is understood to be small, and recall that W n is not needed in the channel decoding phase, but only in the Slepian–Wold decoding phase), and that the length of B is chosen to be I ( X N ; Z N ) N ϵ . Comparing the right-most side to the left-most side, we readily obtain
I μ ( U n ; Z N | W ˙ n ) n ( λ ϵ + δ n ) ,
which can be made arbitrarily small.

5. Proofs

We begin this section by establishing more notation conventions to be used throughout all proofs.
Let n k be a positive integer and let be such that K = · k divides n. Consider the partition of u n into n / K non-overlapping blocks of length K,
( u ˜ 0 , u ˜ 1 , , u ˜ 1 ) , ( u ˜ , u ˜ + 1 , , u ˜ 2 1 ) , , ( u ˜ n / k , u ˜ n / k + 1 , u ˜ n / k 1 ) = ( u 1 K , u K + 1 2 K , , u n K + 1 n )
and apply the same partition to v n . The corresponding channel input and output sequences are of length N = n λ . Let M = · m = K λ and consider the parallel partition of the channels input and output sequences according to
( x ˜ 0 , x ˜ 1 , , x ˜ 1 ) , ( x ˜ , x ˜ + 1 , , x ˜ 2 1 ) , , ( x ˜ N / m , x ˜ N / m + 1 , , x ˜ N / m 1 ) ( y ˜ 0 , y ˜ 1 , , y ˜ 1 ) , ( y ˜ , y ˜ + 1 , , y ˜ 2 1 ) , , ( y ˜ N / m , y ˜ N / m + 1 , , y ˜ N / m 1 ) ( z ˜ 0 , z ˜ 1 , , z ˜ 1 ) , ( z ˜ , z ˜ + 1 , , z ˜ 2 1 ) , , ( z ˜ N / m , z ˜ N / m + 1 , , z ˜ N / m 1 ) .
For the sake of brevity, we henceforth denote ( u ˜ i , , u ˜ ( i + 1 ) 1 ) by u ˜ i ( i + 1 ) 1 and use the same notation rule for all other sequences. Next, define the joint empirical distribution
P U ^ K X ^ M Y ^ M Z ^ M S ^ e S ^ d ( u K , x M , y M , z M , s e , s d ) = K n i = 0 n / K 1 δ { u ˜ i ( i + 1 ) 1 = u K , x ˜ i ( i + 1 ) 1 = x M , y ˜ i ( i + 1 ) 1 = y M , z ˜ i ( i + 1 ) 1 = z M , s i + 1 e = s e , s i + 1 d = s d } ,
and
P U ^ K X M Y M Z M S ^ e S d ( u K , x M , y M , z M , s e , s d ) = E P U ^ K X ^ M Y ^ M Z ^ M S ^ e S ^ d ( u K , x M , y M , z M , s e , s d ) ,
where the expectation is w.r.t. both the randomness of the encoder and the randomness of both channels. Note that
P U ^ K X M Y M Z M S ^ e ( u K , x M , y M , z M , s e ) = P U ^ K S ^ e ( u K , s e ) P ( x M | u K , s e ) Q M ( y M | x M ) Q W ( z M | y M ) .
where
P ( x M | u K , s e ) = j = 0 1 P ( x ˜ j | u ˜ j , s j e ) , s 0 e = s e
Q M ( y M | x M ) = j = 0 M 1 Q M ( y i | x i )
Q W ( z M | y M ) = j = 0 M 1 Q M ( z i | y i ) .
Note also that the bit error probability (in the absence of side information) under this distribution is
1 K E { d H ( U ^ K , f ( Y M , S d ) ) } = 1 K u K , y M , s e , s d P U ^ K Y M S e S d ( u K , y M , s e , s d ) d H ( u K , f ( y M , s d ) ) = 1 K u K , y M , s d K n i = 0 n / K 1 E δ { u ˜ i ( i + 1 ) 1 = u K , s i + 1 e = s e , y ˜ i ( i + 1 ) 1 = y M , s i + 1 d = s d × d H ( u K , f ( y M , s d ) ) = 1 n i = 0 n / K 1 y M , s d F ( y M , s d | u i K + 1 i K + K , s i + 1 e ) d H ( u i K + 1 i K + K , f ( y M , s d ) ) = 1 n i = 1 n E { d H ( u i , V i ) } ,
where f ( Y M , S d ) is induced by successive applications of the decoder output function with inputs Y m , Y m + 1 2 m , , Y M m + 1 M and the initial state S d , and where
F ( y M , s d | u K , s e ) = x M P ( x M | u K , s e ) Q M ( y M | x M ) P S d | Y M ( s d | y M ) .

5.1. Proof of Theorem 1

Beginning with the reliability constraint, we have
I ( U ^ K ; Y M , S d ) = H ( U ^ K ) H ( U ^ K | Y M , S d ) = H ( U ^ K ) H ( U ^ K | Y M ) + I ( S d ; U ^ K | Y M ) I ( U ^ K ; Y M ) + H ( S d | Y M ) I ( X M ; Y M ) + log q d .
On the other hand,
I ( U ^ K ; Y M , S d ) = H ( U ^ K ) H ( U ^ K | Y M , S d ) H ( U ^ K ) K Δ ( ϵ r ) ,
and so,
I ( X M ; Y M ) H ( U ^ K ) K Δ ( ϵ r ) log q d = K · R ( u n , q d , ϵ r ) = M · R ( u n , q d , ϵ r ) λ .
Following [1], we define the function
Γ [ R ] = max { P X : I ( X ; Y ) R } I ( X ; Y | Z ) = max { P X : I ( X ; Y ) R } [ I ( X ; Y ) I ( X ; Z ) ] ,
which is monotonically non–increasing and concave ([1], Lemma 1). Regarding the security constraint,
H ( U ^ K ) K ϵ s ( a ) H ( U ^ K ) max μ I μ ( U K ; Z M ) H ( U ^ K ) I ( U ^ K ; Z M ) = H ( U ^ K | Z M ) H ( U ^ K | Y M , Z M , S d ) + H ( U ^ K | Y M , Z M , S d ) = H ( U ^ K | Z M ) H ( U ^ K | Y M , Z M ) + I ( S d ; U ^ K | Y M , Z M ) + H ( U ^ K | Y M , Z M , S d ) ( b ) I ( U ^ K ; Y M | Z M ) + log q d + K Δ ( ϵ r ) ( c ) I ( X M ; Y M | Z M ) + log q d + K Δ ( ϵ r ) ( d ) i = 1 M I ( X i ; Y i | Z i , Y i 1 ) + log q d + K Δ ( ϵ r ) = i = 1 M y i 1 P Y i 1 ( y i 1 ) I ( X i ; Y i | Z i , Y i 1 = y i 1 ) + log q d + K Δ ( ϵ r ) ( e ) M · 1 M i = 1 M y i 1 P Y i 1 ( y i 1 ) Γ [ I ( X i ; Y i | Y i 1 = y i 1 ) ] + log q d + K Δ ( ϵ r ) ( f ) M · Γ 1 M i = 1 M y i 1 P Y i 1 ( y i 1 ) I ( X i ; Y i | Y i 1 = y i 1 ) + log q d + K Δ ( ϵ r ) = M · Γ 1 M i = 1 M I ( X i ; Y i | Y i 1 ) + log q d + K Δ ( ϵ r ) = M · Γ 1 M i = 1 M H ( Y i | Y i 1 ) H ( Y i | X i , Y i 1 ) + log q d + K Δ ( ϵ r ) = M · Γ 1 M i = 1 M H ( Y i | Y i 1 ) H ( Y i | X i ) + log q d + K Δ ( ϵ r ) = M · Γ 1 M H ( Y M ) H ( Y M | X M ) + log q d + K Δ ( ϵ r ) = M · Γ I ( X M ; Y M ) M + log q d + K Δ ( ϵ r ) ( g ) M · Γ R ( u n , q d , ϵ r ) λ + log q d + K Δ ( ϵ r ) M · Γ R ( u n , q d , ϵ r ) ϵ s λ + log q d + K Δ ( ϵ r ) ,
where P Y i 1 ( y i 1 ) = y i M P Y M ( y M ) , (a) is due to the security constraint, (b) follows from Fano’s inequality and the fact that I ( S d ; U ^ K | Y M , Z M ) H ( S d ) log q d , (c) is due to the data processing inequality and the fact that U ^ K X M Y M is a Markov chain given Z M , (d) is as in ([1], Equation (37)), (e) is by the definition of Wyner’s function Γ ( · ) , (f) is by the concavity of this function, and (g) is by (45) and the decreasing monotonicity of the function Γ ( · ) . Thus,
R ( u n , q d , ϵ r ) ϵ s λ Γ R ( u n , q d , ϵ r ) ϵ s λ
or
R ( u n , q d , ϵ r ) ϵ s λ C s
which is
R ( u n , q d , ϵ r ) λ C s + ϵ s
or, equivalently,
H ( U ^ K ) K λ C s + ϵ s + Δ ( ϵ r ) + log q d K .
Finally, we apply the inequality ([20], Equation (18))
H ( U ^ K ) K ρ LZ ( u n ) 2 K ( log α + 1 ) 2 ( 1 ϵ n ) log n 2 K α 2 K log α n 1 K ,
to obtain
ρ LZ ( u n ) λ C s + ϵ s + Δ ( ϵ r ) + ζ n ( q d , k ) ,
which completes the proof of Theorem 1.

5.2. Proof of Theorem 2

Consider the following extension of the joint distribution to include a random variable that represents { b i } , as follows:
P U ^ K B J X M Y M Z M S ^ e S d ( u K , b J , x M , y M , z M , s e , s d ) = K n i = 0 n / K 1 E [ δ { u ˜ i ( i + 1 ) 1 = u K , b ˜ i ( i + 1 ) 1 = b J , x ˜ i ( i + 1 ) 1 = x M , y ˜ i ( i + 1 ) 1 = y M , z ˜ i ( i + 1 ) 1 = z M , s i + 1 e = s e , s i + 1 d = s d } ] ,
where J = j and b ˜ i ( i + 1 ) 1 = ( b ˜ i , b ˜ i + 1 , , b ˜ ( i + 1 ) 1 ) . Next, consider the following chain of inequalities
K ϵ s max μ I μ ( U K ; Z M ) I ( U ^ K ; Z M ) = I ( U ^ K , B J , S e ; Z M ) I ( B J , S e ; Z M | U ^ K ) = ( a ) I ( X M ; Z M ) I ( B J , S e ; Z M | U ^ K ) I ( X M ; Z M ) H ( B J , S e | U ^ K ) I ( X M ; Z M ) H ( B J , S e ) I ( X M ; Z M ) H ( B J ) H ( S e ) I ( X M ; Z M ) J log q e ,
where (a) is due to the fact that, on the one hand, X M is a deterministic function of ( U ^ K , B J , S e ) , which implies that I ( U ^ K , B J , S e ; Z M ) I ( X M ; Z M ) , but on the other hand, ( U ^ K , B J , S e ) X M Z M is a Markov chain and so, I ( U ^ K , B J , S e ; Z M ) I ( X M ; Z M ) , hence the equality. Thus,
J I ( X M ; Z M ) K ϵ s log q e ,
or
j I ( X M ; Z M ) k ϵ s log q e m I ( X M ; Z M ) M k ϵ s log q e .
The meaning of this result is the following: once one finds a communication system that complies with both the security constraint and the reliability constraint, then the amount of local randomization is lower bounded in terms of the induced mutual information, I ( X M ; Z M ) , as above. By the hypothesis of Theorem 2, the secrecy capacity is saturated, and hence P X M must coincide with the product distribution, [ P X ] M , yielding I ( X M ; Z M ) / M = I ( X ; Z ) . Thus,
j m I ( X ; Z ) k ϵ s log q e .
This completes the proof of Theorem 2.

5.3. Outline of the Proof of Theorem 3

The proof follows essentially the same steps as those of the proof of Theorem 1, except that everything should be conditioned on the side information, but there are also some small twists. We, therefore, only provide a proof outline and highlight the differences.
The auxiliary joint distribution is now extended to read
P U ^ K W ^ K W ˙ K X N Y N Z N S ^ e S d ( u K , w K , w ˙ K , x N , y N , z N , s e , s d ) = K m i = 0 m / K 1 E [ δ { u ˜ i ( i + 1 ) 1 = u K , w ˜ i ( i + 1 ) 1 = w K , w ˙ ˜ i ( i + 1 ) 1 = w ˙ K , x ˜ i ( i + 1 ) 1 = x M , y ˜ i ( i + 1 ) 1 = y M , z ˜ i ( i + 1 ) 1 = z M , s i + 1 e = s e , s i + 1 d = s d } ] .
Note that
P U ^ K W ^ K W ˙ K Z M S ^ e ( u k , w k , w ˙ k , z M , s e ) = K n i = 0 n / K 1 E δ { u ˜ i ( i + 1 ) 1 = u K , w ˜ i ( i + 1 ) 1 = w K , w ˙ ˜ i ( i + 1 ) 1 = w ˙ K , z ˜ i ( i + 1 ) 1 = z M , s i + 1 e = s e } = K n i = 0 n / K 1 δ { u ˜ i ( i + 1 ) 1 = u K , w ˜ i ( i + 1 ) 1 = w K , s i + 1 e = s e } · Q W ˙ K | W K ( w ˙ K | w K ) · G ( z M | u K , s e ) = P U ^ K W ^ K S ^ e ( u K , w K , s e ) · Q W ˙ K | W K ( w ˙ K | w K ) · G ( z M | u K , s e ) ,
where
G ( z M | u K , s e ) = x M P ( x M | u K , s e ) Q M W ( z M | x M ) .
It follows that W ˙ K W ^ K ( U ^ K , S ^ e ) Z M is a Markov chain under P U ^ K W ^ K W ˙ K Z M S ^ e . In other words, the legitimate decoder has side information of better quality than that of the wiretapper. First, observe that
I μ ( U n ; Z N | W n ) = H μ ( Z N | W n ) H μ ( Z N | W n , U n ) H μ ( Z N | W n ) H μ ( Z N | W n , U n , S e ) = H μ ( Z N | W n ) H μ ( Z N | U n , S e ) H μ ( Z N | W ˙ n ) H μ ( Z N | U n , S e ) = H μ ( Z N | W ˙ n ) H μ ( Z N | W ˙ n , U n , S e ) H μ ( Z N | W ˙ n ) H μ ( Z N | W ˙ n , U n ) + log q e = I μ ( U n ; Z N | W ˙ n ) + log q e .
The reliability constraint is handled exactly as in the proof of Theorem 1, except that everything should be conditioned on W ^ K . The result of this is
I ( X M ; Y M | W ^ K ) H ( U ^ K | W ^ K ) K Δ ( ϵ r ) log q d = K · R ( u n , w n , q d , ϵ r ) = M · R ( u n , w n , q d , ϵ r ) λ .
Regarding the security constraint, we begin with the following manipulation.
H ( U ^ K | Z M , W ˙ K ) = H ( U ^ K | W ˙ K ) I ( U ^ K ; Z M | W ˙ K ) = H ( U ^ K | W ˙ K ) H ( U ^ K | W ^ K ) + H ( U ^ K | W ^ K ) I ( U ^ K ; Z M | W ˙ K ) ( a ) H ( U ^ K | W ˙ K ) H ( U ^ K | W ^ K ) + H ( U ^ K | W ^ K ) I ( U ^ K ; Z M | W ^ K ) + log q e = H ( U ^ K | W ˙ K ) H ( U ^ K | W ^ K ) + H ( U ^ K | Z M , W ^ K ) + log q e = H ( U ^ K | W ˙ K ) H ( U ^ K | W ^ K ) + H ( U ^ K | Z M , W ^ K ) H ( U ^ K | Y M , Z M , S d , W ^ K ) + H ( U ^ K | Y M , Z M , S d , W ^ K ) + log q e = H ( U ^ K | W ˙ K ) H ( U ^ K | W ^ K ) + H ( U ^ K | Z M , W ^ K ) H ( U ^ K | Y M , Z M , W ^ K ) + I ( S d ; U ^ K | Y M , Z M , W ^ K ) + H ( U ^ K | Y M , Z M , S d , W ^ K ) + log q e ( b ) H ( U ^ K | W ˙ K ) H ( U ^ K | W ^ K ) + I ( U ^ K ; Y M | Z M , W ^ K ) + log q d + K Δ ( ϵ r ) + log q e H ( U ^ K | W ˙ K ) H ( U ^ K | W ^ K ) + I ( U ^ K , S ^ e ; Y M | Z M , W ^ K ) + log q d + K Δ ( ϵ r ) + log q e ( c ) H ( U ^ K | W ˙ K ) H ( U ^ K | W ^ K ) + I ( X M ; Y M | Z M , W ^ K ) + log ( q e q d ) + K Δ ( ϵ r ) ,
where in (a) we used Equation (62), in (b) we used Fano’s inequality, and in (c) we used the data processing inequality as ( U ^ K , S ^ e ) X M Y M is a Markov chain (also conditioned on ( W ^ K , Z M ) ). The next step is to further upper bound the term I ( X M ; Y M | Z M , W ^ K ) . This is carried out very similarly as in the proof of Theorem 1, except that everything is conditioned also on W ^ K . We then obtain
H ( U ^ K | Z M , W ˙ K ) H ( U ^ K | W ˙ K ) H ( U ^ K | W ^ K ) + M · Γ R ( u n , w n , q d , ϵ r ) λ + log ( q e q d ) + K Δ ( ϵ r ) ,
or, equivalently,
H ( U ^ K | W ^ K ) M · Γ R ( u n , w n , q d , ϵ r ) λ H ( U ^ K | W ˙ K ) H ( U ^ K | Z M , W ˙ K ) + log ( q e q d ) + K Δ ( ϵ r ) = I ( U ^ K ; Z M | W ˙ K ) + log ( q e q d ) + K Δ ( ϵ r ) K ϵ s + log ( q e q d ) + K Δ ( ϵ r ) ,
or
R ( u n , w n , q e · q d , ϵ r ) λ · Γ R ( u n , w n , q d , ϵ r ) λ λ · Γ R ( u n , w n , q e · q d , ϵ r ) λ
which is the same as
R ( u n , w n , q e · q d , ϵ r ) λ · C s .
or
H ( U ^ K | W ^ K ) K λ · C s + ϵ s + Δ ( ϵ r ) + log ( q e · q d ) K .
The proof is completed by combining the last inequality with the following inequality ([26] Equations (17)–(19), [27] Equations (55)–(57)):
H ( U ^ K | W ^ K ) K ρ LZ ( u n | w n ) log ( 4 A 2 ) ( 1 ϵ n ) log n A 2 log ( 4 A 2 ) n 1 K ,
where A = [ ( α ω ) K + 1 1 ] / [ α ω 1 ] , ω being the alphabet size of W .

Funding

This research received no external funding.

Data Availability Statement

Data sharing not applicable.

Acknowledgments

Interesting discussions with Alejandro Cohen are acknowledged with thanks.

Conflicts of Interest

The author declares no conflict of interest.

References

  1. Wyner, A.D. The wire-tap channel. Bell Syst. Tech. J. 1975, 54, 1355–1387. [Google Scholar] [CrossRef]
  2. Csiszár, I.; Körner, J. Broadcast channels with confidential messages. IEEE Trans. Inform. Theory 1978, 24, 339–348. [Google Scholar] [CrossRef] [Green Version]
  3. Leung-Yan-Cheong, S.K.; Hellman, M.E. The Gaussian wire-tap channel. IEEE Trans. Inform. Theory 1978, 24, 451–456. [Google Scholar] [CrossRef]
  4. Ozarow, L.H.; Wyner, A.D. Wire–tap channel II. In Proceedings of the Eurocrypt 84, Workshop on Advances in Cryptology: Theory and Applications of Cryptographic Techniques, Paris, France, 9–11 April 1985; pp. 33–51. [Google Scholar]
  5. Yamamoto, H. Coding theorems for secret sharing communication systems with two noisy channels. IEEE Trans. Inform. Theory 1989, 35, 572–578. [Google Scholar] [CrossRef]
  6. Yamamoto, H. Rate–distortion theory for the Shannon cipher system. IEEE Trans. Inform. Theory 1997, 43, 827–835. [Google Scholar] [CrossRef] [Green Version]
  7. Merhav, N. Shannon’s secrecy system with informed receivers an its application to systematic coding for wiretapped channels. IEEE Trans. Inform. Theory 2008, 54, 2723–2734. [Google Scholar] [CrossRef] [Green Version]
  8. Tekin, E.; Yener, A. The Gaussian multiple access wire–Tap channel. IEEE Trans. Inform. Theory 2008, 54, 5747–5755. [Google Scholar] [CrossRef] [Green Version]
  9. Mitrpant, C. Information Hiding—An Application of Wiretap Channels with Side Information. Ph.D. Thesis, der Universitaet Duisburg–Essen, Essen, Germany, 2003. [Google Scholar]
  10. Mitrpant, C.; Vinck, A.J.H.; Luo, Y. An achievable region for the Gaussian wiretap channel with side information. IEEE Trans. Inform. Theory 2006, 52, 2181–2190. [Google Scholar] [CrossRef]
  11. Ardestanizadeh, E.; Franceschetti, M.; Javidi, T.; Kim, Y.-H. Wiretap channel with secure rate–Limited feedback. IEEE Trans. Inform. Theory 2009, 55, 5353–5361. [Google Scholar] [CrossRef]
  12. Hayashi, M. Upper bounds of eavesdropper’s performances in finite-length code with the decoy method. Phy. Rev. A 2007, 76, 012329, Erratum in Phys. Rev. A 2009, 79, 019901E. [Google Scholar] [CrossRef] [Green Version]
  13. Hayashi, M.; Matsumoto, R. Secure multiplex coding with dependent and non-uniform multiple messages. IEEE Trans. Inform. Theory 2016, 62, 2355–2409. [Google Scholar] [CrossRef] [Green Version]
  14. Bloch, M.; Barros, J. Physical-Layer Security: From Information Theory to Security Engineering; Cambridge University Press: New York, NY, USA, 2011. [Google Scholar]
  15. Bellare, M.; Tessaro, S.; Vardy, A. Semantic security for the wiretap channel. In Advances in Cryptology—CRYPTO 2012; Safavi-Naini, R., Canetti, R., Eds.; CRYPTO 2012 Lecture Notes in Computer Science; Springer: Berlin/Heidelberg, Germany, 2012; Volume 7417. [Google Scholar] [CrossRef] [Green Version]
  16. Goldfeld, Z.; Cuff, P.; Permuter, H.H. Semantic security capacity for wiretap channels of type II. In Proceedings of the 2016 IEEE International Symposium on Information Theory (ISIT 2016), Barcelona, Spain, 10–15 July 2016; pp. 2799–2803. [Google Scholar]
  17. Ziv, J. Coding theorems for individual sequences. IEEE Trans. Inform. Theory 1978, 24, 405–412. [Google Scholar] [CrossRef]
  18. Ziv, J. Distortion–rate theory for individual sequences. IEEE Trans. Inform. Theory 1980, 26, 137–143. [Google Scholar] [CrossRef]
  19. Ziv, J. Fixed–rate encoding of individual sequences with side information. IEEE Trans. Inform. Theory 1984, 30, 348–352. [Google Scholar] [CrossRef]
  20. Merhav, N. Perfectly secure encryption of individual sequences. IEEE Trans. Inform. Theory 2013, 58, 1302–1310. [Google Scholar] [CrossRef]
  21. Merhav, N. On the data processing theorem in the semi-deterministic setting. IEEE Trans. Inform. Theory 2014, 60, 6032–6040. [Google Scholar] [CrossRef] [Green Version]
  22. Ziv, J.; Lempel, A. Compression of individual sequences via variable–Rate coding. IEEE Trans. Inform. Theory 1978, 24, 530–536. [Google Scholar] [CrossRef] [Green Version]
  23. Ziv, J. Universal decoding for finite-state channels. IEEE Trans. Inform. Theory 1985, 31, 453–460. [Google Scholar] [CrossRef]
  24. Uyematsu, T.; Kuzuoka, S. Conditional Lempel–Ziv complexity and its application to source coding theorem with side information. IEICE Trans. Fundam. 2003, E86-A, 2615–2617. [Google Scholar]
  25. Draper, S. Universal incremental Slepian–Wolf Coding. In Proceedings of the 43rd Annual Allerton Conference on Communication, Control, and Computing, Monticello, IL, USA, 29 September–1 October 2004; pp. 1332–1341. [Google Scholar]
  26. Merhav, N. Universal detection of messages via finite–State channels. IEEE Trans. Inform. Theory 2000, 46, 2242–2246. [Google Scholar] [CrossRef]
  27. Merhav, N. Guessing individual sequences: Generating randomized guesses using finite—State machines. IEEE Trans. Inform. Theory 2020, 66, 2912–2920. [Google Scholar] [CrossRef] [Green Version]
Figure 1. Wiretap channel model. Since the source and the channel may operate at different rates ( λ channel symbols per source symbol), the time variables associated with source-related sequences and channel-related sequences are denoted differently, i.e., i and t, respectively.
Figure 1. Wiretap channel model. Since the source and the channel may operate at different rates ( λ channel symbols per source symbol), the time variables associated with source-related sequences and channel-related sequences are denoted differently, i.e., i and t, respectively.
Entropy 23 01694 g001
Figure 2. Wiretap channel model with side information.
Figure 2. Wiretap channel model with side information.
Entropy 23 01694 g002
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Merhav, N. Encoding Individual Source Sequences for the Wiretap Channel. Entropy 2021, 23, 1694. https://doi.org/10.3390/e23121694

AMA Style

Merhav N. Encoding Individual Source Sequences for the Wiretap Channel. Entropy. 2021; 23(12):1694. https://doi.org/10.3390/e23121694

Chicago/Turabian Style

Merhav, Neri. 2021. "Encoding Individual Source Sequences for the Wiretap Channel" Entropy 23, no. 12: 1694. https://doi.org/10.3390/e23121694

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop