Compressed Secret Key Agreement:Maximizing Multivariate Mutual Information per Bit

Abstract

The multiterminal secret key agreement problem by public discussion is formulated with an additional source compression step where, prior to the public discussion phase, users independently compress their private sources to filter out strongly correlated components in order to generate a common secret key. The objective is to maximize the achievable key rate as a function of the joint entropy of the compressed sources. Since the maximum achievable key rate captures the total amount of information mutual to the compressed sources, an optimal compression scheme essentially maximizes the multivariate mutual information per bit of randomness of the private sources, and can therefore be viewed more generally as a dimension reduction technique. Single-letter lower and upper bounds on the maximum achievable key rate are derived for the general source model, and an explicit polynomial-time computable formula is obtained for the pairwise independent network model. In particular, the converse results and the upper bounds are obtained from those of the related secret key agreement problem with rate-limited discussion. A precise duality is shown for the two-user case with one-way discussion, and such duality is extended to obtain the desired converse results in the multi-user case. In addition to posing new challenges in information processing and dimension reduction, the compressed secret key agreement problem helps shed new light on resolving the difficult problem of secret key agreement with rate-limited discussion by offering a more structured achieving scheme and some simpler conjectures to prove.

1. Introduction

In information-theoretic security, the problem of secret key agreement by public discussion concerns a group of users discussing in public to generate a common secret key that is independent of their discussion. The problem was first formulated by Maurer [1] and Ahlswede and Csiszár [2] under a private source model involving two users who observe some correlated private sources. Rather surprisingly, public discussion was shown to be useful in generating the secret key; i.e., it strictly increases the maximum achievable secret key rate, called the secrecy capacity. This phenomenon was also discovered in [3] in a different formulation. Furthermore, the secrecy capacity was given an information-theoretically appealing characterization—it is equal to Shannon’s mutual information [4] between the two private sources, assuming the wiretapper can listen to the entire public discussion but not observe any other side information of the private sources. It was also shown that the capacity can be achieved by one-way public discussion (i.e., with only one of the users discussing in public).

As a simple illustration, let ${\mathrm{X}}_0$, ${\mathrm{X}}_1$, and $\mathrm{J}$ be three uniformly random independent bits, and suppose user 1 observes ${\mathrm{Z}}_1:=({\mathrm{X}}_0,{\mathrm{X}}_1)$ privately while user 2 observes ${\mathrm{Z}}_2:=({\mathrm{X}}_{\mathrm{J}},\mathrm{J})$, where ${\mathrm{X}}_{\mathrm{J}}={\mathrm{X}}_0$ when $\mathrm{J}=0$ but ${\mathrm{X}}_{\mathrm{J}}={\mathrm{X}}_1$ when $\mathrm{J}=1$. If user 2 reveals $\mathrm{J}$ in public, then user 1 can recover ${\mathrm{X}}_{\mathrm{J}}$ and therefore ${\mathrm{Z}}_2$. Furthermore, since ${\mathrm{X}}_{\mathrm{J}}$ is independent of $\mathrm{J}$, it can serve as a secret key bit that is recoverable by both users but remains perfectly secret to a wiretapper who observes only the public message $\mathrm{J}$. This scheme achieves a secrecy capacity equal to the mutual information $I({\mathrm{Z}}_1\wedge {\mathrm{Z}}_2)=1$ roughly because user 2 reveals $H({\mathrm{Z}}_2|{\mathrm{Z}}_1)=1$ bit in public so there is $H({\mathrm{Z}}_2)-H({\mathrm{Z}}_2|{\mathrm{Z}}_1)=I({\mathrm{Z}}_1\wedge {\mathrm{Z}}_2)$ bits of randomness left for the secret key. However, if no public discussion is allowed, it follows from the work of Gác and Körner [5] that no common secret key bit can be extracted from the sources. In particular, ${\mathrm{X}}_{\mathrm{J}}$ cannot be used as a secret key because user 1 does not know whether ${\mathrm{X}}_{\mathrm{J}}$ is ${\mathrm{X}}_0$ or ${\mathrm{X}}_1$. ${\mathrm{X}}_0$ and ${\mathrm{X}}_1$ cannot be used as a secret key either because they may not be observed by user 2 when $\mathrm{J}=1$ and $\mathrm{J}=0$, respectively. It can be seen that while the private sources are clearly statistically dependent, public discussion is needed to consolidate the mutual information of the sources into a common secret key.

The secret key agreement formulation was subsequently extended to the multi-user case by Csiszár and Narayan [6]. Some users are also allowed to act as helpers who can participate in the public discussion but need not share the secret key. The designated set of users who need to share the secret key are referred to as the active users. In contrast to the two-user case, one-way discussion may not achieve the secrecy capacity when there are more than two users. Instead, an omniscience strategy was considered in [6] in which the users first communicate minimally in public until omniscience; that is, the users discuss in public at the smallest total rate until every active user can recover all the private sources. The scheme was shown to achieve the secrecy capacity in the case when the wiretapper only listens to the public discussion. However, this assumes that the public discussion is lossless and unlimited in rate, and the sources take values from finite alphabet sets. If the sources are continuous or if the public discussion is limited to a certain rate, it may be impossible to attain omniscience.

This work is motivated by the search for a better alternative to the omniscience strategy for multiterminal secret key agreement. A prior work of Csiszár and Narayan [7] considered secret key agreement under rate-limited public discussion. The model involves two users and a helper observing correlated discrete memoryless sources. The public discussion of the users is conducted in a particular order and direction. While the region of achievable secret key rate and discussion rates remains unknown, single-letter characterizations involving two auxiliary random variables were given for many special cases, including the two-user case with two rounds of interactive public discussion, where each user speaks once in sequence, with the last public message possibly depending on the first. By further restricting to one-way public discussion, the characterization involves only one auxiliary random variable and was extended to continuous sources by Watanabe and Oohama in [8], where they also gave an explicit characterization without any auxiliary random variable for scalar Gaussian sources in [8]. For vector Gaussian sources, the characterization by the same authors in [9] involving some matrix optimization was further improved in [10] to a more explicit formula. However, if the discussion is allowed to be two-way and interactive, Tyagi [11] showed with a concrete two-user example that the minimum total discussion rate required—called the communication complexity—can be strictly reduced. Using the technique of Kaspi [12], multi-letter characterizations were given in [11] for the communication complexity, and similarly by Liu et al. in [13] for the region of achievable secret key rate. The characterization was further simplified in [13] using the idea of convex envelope using the technique by Ma et al. [14]. While these characterizations provide many new insights and properties, they are not considered computable compared to the usual single-letter and explicit characterizations. Further extension to the multi-user case also appears difficult, as the converse can be seen to rely on the Csiszár sum identity [2] (Lemma 4.1), which does not appear to extend beyond the two-user case.

Nevertheless, partial solutions under more restrictive public discussion constraints were possible. By simplifying the problem to the right extent, new results were discovered in the multi-user case, which has led to the formulation in this work. For instance, Gohari and Anantharam [15] characterized the secrecy capacity in the multi-user case under the simpler vocality constraint where some users have to remain silent throughout the public discussion. Using this result, simple necessary and sufficient conditions can be derived as to whether a user can remain silent without diminishing the maximum achievable key rate [16,17,18]. This is a simpler result than characterizing the achievable rate region because it does not say how much discussion is required if a user must discuss. Another line of work [19,20,21,22] follows [11] to characterize the communication complexity, but in the multi-user case. Courtade and Halford [19] characterized the communication complexity under a special non-asymptotic hypergraphical source model with linear discussion. A multi-letter lower bound was obtained in [21] for the communication complexity for the asymptotic general source model. It also gave a precise and simple condition under which the omniscience strategy for secret key agreement is optimal for a special source model called the pairwise independent network (PIN) [23], which is a special hypergraphical source model [24]. In [22,25], some single-letter and more easily computable explicit lower bounds were also derived. These bounds also lead to conditions for the omniscience strategy to be optimal under the hypergraphical source model, which covers the PIN model as a special case. The more general problem of characterizing the multiterminal secrecy capacity under rate-limited public discussion was considered in [26]. In particular, an objective of [26] is to characterize the constrained secrecy capacity, defined as the maximum achievable key rate as a function of the total discussion rate. This covers the communication complexity as a special case when further increase in the public discussion rate does not increase the secrecy capacity. While only single-letter bounds were derived for the general source model, a surprisingly simple explicit formula was derived for the PIN model [26]. The optimal scheme in [26] follows the tree-packing protocol in [27]. It turns out to belong to the more general approach of decremental secret key agreement in [28,29] inspired by the achieving scheme in [19] and the notion of excess edge in [24]. More precisely, the omniscience strategy is applied after some excess or less-useful edge random variables are removed (decremented) from the source. Because the entropy of the decremented source is smaller, the discussion required to attain omniscience of the decremented source is also less. Such decremental secret key agreement approach applies to hypergraphical sources more generally, and it results in one of the best upper bounds in [20] for communication complexity. However, for more general source models that are not necessarily hypergraphical, the approach does not directly apply.

The objective of this work is to formalize and extend the idea of decremental secret key agreement beyond the hypergraphical source model. More precisely, the secret key agreement problem is considered with an additional source compression step before public discussion, in which each user independently compresses their private source component to filter away less correlated randomness that does not contribute much to the achievable secret key rate. The compression is such that the entropy rate of the compressed sources is reduced to under a certain specified level. In particular, the edge removal process in decremental secret key agreement can be viewed as a special case of source compression, and the more general problem is referred to as compressed secrecy key agreement. The objective is to characterize the achievable secret key rate maximized over all valid compression schemes. For simplicity, this work will focus on the case without helpers—that is, when all users are active and want to share a common secret key. A closely related formulation is given by Nitinawarat and Narayan [30], which characterized the maximum achievable key rate for the two-user case under the scalar Gaussian source model where one of the users is required to quantize the source to within a given rate. The formulation and techniques in [30] was also extended in [31] to the multi-user case where every user can quantize their sources individually to a certain rate. The compression considered in this work is more general than quantizations for Gaussian sources, and the new results are meaningful beyond continuous sources.

The compressed secret key agreement problem is also motivated by the study of multivariate mutual information (MMI) [32]—that is, an extension of Shannon’s mutual information to the multivariate case involving possibly more than two random variables. The unconstrained secrecy capacity in the no-helper case has been viewed as a measure of mutual information in [32,33], not only because of its mathematically appealing interpretations such as the residual independence relation and data processing inequalities in [32], but also because of its operational significance in undirected network coding [34,35], data clustering [36], and feature selection [37] (cf. [38]). The optimal source compression scheme that achieves the compressed secrecy capacity can be viewed more generally as an optimal dimension reduction procedure that maximizes the MMI per bit of randomness, which is an extension of the information bottleneck problem [39] to the multivariate case. However, different from the multivariate extension in [40], the MMI is used instead of Watanabe’s total correlation [41], and so it captures only the information mutual to all the random variables rather than the information mutual to any subsets of the random variables. Furthermore, the compression is on each random variable rather than subsets of random variables.

The paper is organized as follows. The problem of compressed secret key agreement is formulated in Section 2. Preliminary results of the secret key agreement are given in Section 3. The main results are motivated in Section 4 and presented in Section 5, followed by the conclusion and some discussions on potential extensions in Section 6.

2. Problem Formulation

Similarly to the multiterminal secret key agreement problem [6] without helpers or wiretappers’ side information, the setting of the problem involves a finite set V of $|V|>1$ users, and a discrete memoryless multiple source:

$$\begin{array}{c}{\mathrm{Z}}_V:=({\mathrm{Z}}_i|i\in V)\sim P_{{\mathrm{Z}}_V}\mathrm{taking}\mathrm{values}\mathrm{from}\hfill \\ Z_V:={\prod }_{i\in V}{Z}_i(\mathrm{not}\mathrm{necessarily}\mathrm{finite}).\hfill \end{array}$$

Note that letters in sans serif font are used for random variables and the corresponding capital letters in the usual math italic font denote the alphabet sets. $P_{{\mathrm{Z}}_V}$ denotes the joint distribution of ${\mathrm{Z}}_i$’s.

A secret key agreement protocol with source compression can be broken into the following phases:

• Private observation: Each user $i\in V$ observes an n-sequence:

  $${\mathrm{Z}}_i^n:=({\mathrm{Z}}_{it}|t\in \left[n\right])=({\mathrm{Z}}_{i1},{\mathrm{Z}}_{i2},\dots ,{\mathrm{Z}}_{in})$$

  i.i.d. generated from the source ${\mathrm{Z}}_i$ for some block length n. For convenience, $\left[n\right]$ denotes the set of positive integers up to n (i.e., $1,\dots ,n$).
• Private randomization: Each user $i\in V$ generates a random variable ${\mathrm{U}}_i$ independent of the private source; i.e.,

  $$\begin{array}{c}\hfill H({\mathrm{U}}_V|{\mathrm{Z}}_V)=\sum _{i\in V}H({\mathrm{U}}_i).\end{array}$$

  (1)
• Source compression: Each user $i\in V$ computes

  $$\begin{array}{c}\hfill {\tilde{\mathrm{Z}}}_i={\zeta }_i({\mathrm{U}}_i,{\mathrm{Z}}_i^n)\end{array}$$

  (2)

  for some function ${\zeta }_i$ that maps to a finite set. ${\tilde{\mathrm{Z}}}_V$ is referred to as the compressed source.
• Public discussion: Using a public authenticated noiseless channel, a user $i_t\in V$ is chosen in round $t\in \left[\ell \right]$ to broadcast a message

  $$\begin{array}{ccc}\hfill {\tilde{\mathrm{F}}}_t:={\tilde{f}}_t({\tilde{\mathrm{Z}}}_{i_t},{\tilde{\mathrm{F}}}^{t-1})& \hfill & \mathrm{where}\hfill \end{array}$$

  (3a)

  ℓ is a positive integer denoting the number of rounds and ${\tilde{\mathrm{F}}}^{t-1}$ denotes all the messages broadcast in the previous rounds. If the dependency on ${\tilde{\mathrm{F}}}^{t-1}$ is dropped, the discussion is said to be non-interactive. The discussion is said to be one-way (from user i) if $\ell =1$ (and $i_1=1$). For convenience,

  $$\begin{array}{c}\hfill {\mathrm{F}}_i:=({\tilde{\mathrm{F}}}_t|t\in \left[\ell \right],i_t=i)\end{array}$$

  (3b)

  $$\begin{array}{c}\hfill \mathrm{F}:={\tilde{\mathrm{F}}}^{\ell }={\mathrm{F}}_V\end{array}$$

  (3c)

  denote the aggregate message from user $i\in V$ and the aggregation of the messages from all users, respectively.
• Key generation: A random variable $\mathrm{K}$, called the secret key, is required to satisfy the recoverability constraint that

  $$\underset{n\to \infty }{lim}\mathrm{Pr}(\exists i\in V,\mathrm{K}\ne {\theta }_i({\tilde{\mathrm{Z}}}_i,\mathrm{F}))=0,$$

  (4)

  for some function ${\theta }_i$, and the secrecy constraint:

  $$\underset{n\to \infty }{lim}\frac{1}{n}[log|K|-H(\mathrm{K}|\mathrm{F})]=0,$$

  (5)

  where K denotes the finite alphabet set of possible key values.

Note that, unlike [11], non-interactive discussion is considered different from one-way discussion in the two-user case, since both users are allowed to discuss even though their messages cannot depend on each other. In contrast to [42], there is an additional source compression phase, after which the protocol can only depend on the original sources through the compressed sources.

The objective is to characterize the maximum achievable secret key rate for a continuum of different levels of source compression:

Definition 1.

The compressed secrecy capacity with a joint entropy limit $\alpha \ge 0$ is defined as

$$\begin{array}{c}\hfill {\tilde{C}}_{\mathrm{S}}(\alpha ):=sup\underset{n\to \infty }{lim\; inf}\frac{1}{n}log|K|\end{array}$$

(6)

where the supremum is over all possible compressed secret key agreement schemes satisfying

$$\begin{array}{c}\hfill \underset{n\to \infty }{lim\; sup}\frac{1}{n}H({\tilde{\mathrm{Z}}}_V)-\alpha \le 0.\end{array}$$

(7)

This constraint limits the joint entropy rate of the compressed source.

Note that instead of the joint entropy limit, one may also consider entropy limits on some subset $B\subseteq V$ that

$$\begin{array}{c}\hfill \underset{n\to \infty }{lim\; sup}\frac{1}{n}H({\tilde{\mathrm{Z}}}_B)-\alpha \le 0.\end{array}$$

(8)

If multiple entropy limits are imposed, ${\tilde{C}}_{\mathrm{S}}$ will be a higher-dimensional surface instead of a one-dimensional curve. For example, in the two-user case under the scalar Gaussian source model, the entropy limit was imposed on only one of the users in [30]. In [31], the multi-user case under the Gaussian Markov tree model was considered under the symmetric case where the entropy limit is imposed on every user.

However, for simplicity, the joint entropy constraint (7) will be the primary focus in this work. It will be shown that ${\tilde{C}}_{\mathrm{S}}(\alpha )$ is closely related to the constrained secrecy capacity$C_{\mathrm{S}}(R)$ defined as [26]:

$$\begin{array}{c}\hfill C_{\mathrm{S}}(R):=sup\underset{n\to \infty }{lim\; inf}\frac{1}{n}log|K|\mathrm{for}R\ge 0,\end{array}$$

(9)

with ${\tilde{\mathrm{Z}}}_i:=({\mathrm{U}}_i,{\mathrm{Z}}_i^n)$ instead of (2) (i.e., without compression), and the entropy limit (7) replaced by the constraint on the total discussion rate:

$$\begin{array}{c}\hfill R\ge \underset{n\to \infty }{lim\; sup}\frac{1}{n}log|F|=\underset{n\to \infty }{lim\; sup}\frac{1}{n}\sum _{i\in V}log|F_i|.\end{array}$$

(10)

It follows directly from the result of [6] that ${\tilde{C}}_{\mathrm{S}}(\alpha )$ remains unchanged whether or not the discussion is interactive. Indeed, the relation between ${\tilde{C}}_{\mathrm{S}}(\alpha )$ and $C_{\mathrm{S}}(R)$ to be shown in this work will not be affected either. Therefore, for notational simplicity, $C_{\mathrm{S}}(R)$ may refer to the case with or without interaction, even though $C_{\mathrm{S}}(R)$ may be smaller with non-interactive discussion.

It is easy to show that $C_{\mathrm{S}}(R)$ is continuous, non-decreasing, and concave in R [26] (Proposition 3.1). As R goes to $\infty$, the secrecy capacity

$$\begin{array}{c}\hfill C_{\mathrm{S}}(\infty ):=\underset{R\to \infty }{lim\; inf}{C}_{\mathrm{S}}(R)\end{array}$$

(11)

is the usual unconstrained secrecy capacity defined in [6] without the discussion rate constraint (10). The smallest discussion rate that achieves the unconstrained secrecy capacity is the communication complexity denoted by

$$\begin{array}{c}\hfill R_{\mathrm{S}}:=inf\{R\ge 0\mid C_{\mathrm{S}}(R)=C_{\mathrm{S}}(\infty )\}.\end{array}$$

(12)

Similar to $C_{\mathrm{S}}(R)$, the following basic properties can be shown for ${\tilde{C}}_{\mathrm{S}}(\alpha )$:

Proposition 1.

${\tilde{C}}_{\mathrm{S}}(\alpha )$ is continuous, non-decreasing, and concave in $\alpha \ge 0$. Furthermore,

$$\begin{array}{c}\hfill C_{\mathrm{S}}(\infty )=\underset{\alpha \to \infty }{lim\; inf}{\tilde{C}}_{\mathrm{S}}(\alpha ),\end{array}$$

(13)

achieving the unconstrained secrecy capacity in the limit.

Proof. 

Continuity, monotonicity, and (13) follow directly from the definition of ${\tilde{C}}_{\mathrm{S}}(\alpha )$. Concavity follows from the usual time-sharing argument; i.e., for any $\lambda \in [0,1]$, ${\alpha }^{\prime },{\alpha }^{″}>0$, a secret key rate of $\lambda {\tilde{C}}_{\mathrm{S}}({\alpha }^{\prime })+(1-\lambda ){\tilde{C}}_{\mathrm{S}}(1-{\alpha }^{\prime })$ is achievable with the entropy limit $\alpha :=\lambda {\alpha }^{\prime }+(1-\lambda ){\alpha }^{″}$ by applying the optimal scheme that achieves ${\tilde{C}}_{\mathrm{S}}({\alpha }^{\prime })$ for the first $n^{\prime }:=\lfloor \lambda n\rfloor$ samples of ${\mathrm{Z}}_V^n$ and applying the optimal scheme that achieves ${\tilde{C}}_{\mathrm{S}}({\alpha }^{″})$ for the remaining $n^{″}:=n-n^{\prime }$ samples. ☐

Because of (13), a quantity playing the same role of $R_{\mathrm{S}}$ for $C_{\mathrm{S}}$ can be defined for ${\tilde{C}}_{\mathrm{S}}(\alpha )$ as follows.

Definition 2.

The smallest entropy limit that achieves the unconstrained secrecy capacity is defined as

$$\begin{array}{c}\hfill {\alpha }_{\mathrm{S}}:=inf\{\alpha \mid {\tilde{C}}_{\mathrm{S}}(\alpha )=C_{\mathrm{S}}(\infty )\}\end{array}$$

(14)

and referred to as the minimum admissible joint entropy.

One may also consider both the entropy limit (7) and discussion rate constraint (10) simultaneously, and define the secrecy capacity as a function of $\alpha$ and R. However, for simplicity, we will not consider this case but instead focus on the relationship between ${\tilde{C}}_{\mathrm{S}}(\alpha )$ and $C_{\mathrm{S}}(R)$.

The following example illustrates the problem formulation. It will be revisited at the end of Section 5 (Example 3) to illustrate the main results.

Example 1.

Consider $V:=\{1,2,3\}$ and

$$\begin{array}{c}\hfill {\mathrm{Z}}_1:=({\mathrm{X}}_a,{\mathrm{X}}_b),{\mathrm{Z}}_2:=({\mathrm{X}}_a,{\mathrm{X}}_b,{\mathrm{X}}_c),and{\mathrm{Z}}_3:=({\mathrm{X}}_a,{\mathrm{X}}_c),\end{array}$$

(15)

where ${\mathrm{X}}_a,{\mathrm{X}}_b$, and ${\mathrm{X}}_c$ are uniformly random and independent bits. It is easy to argue that

$$\begin{array}{c}\hfill {\tilde{C}}_{\mathrm{S}}(\alpha )\ge \alpha \mathit{for}\alpha \in [0,1].\end{array}$$

(16a)

To see this, notice that ${\mathrm{X}}_a$ is observed by every user. Any choice of $\mathrm{K}=\theta ({\mathrm{X}}_a^n)$ can therefore be recovered by every user without any discussion, satisfying the recoverability constraint (4) trivially. Since there is no public discussion required, the secrecy constraint (5) also holds immediately by taking a portion of the bits from ${\mathrm{X}}_a^n$ to be the key bits in $\mathrm{K}$. Finally, setting ${\tilde{\mathrm{Z}}}_i={\zeta }_i({\mathrm{X}}_a^n)=\theta ({\mathrm{X}}_{\alpha }^n)$ for all $i\in V$ ensures $H({\tilde{\mathrm{Z}}}_V)\le H(\mathrm{K})$, satisfying the entropy limit (7) with $\alpha$ equal to the key rate. Hence, ${\tilde{C}}_{\mathrm{S}}(\alpha )\ge \alpha$, as desired. Indeed, we will show (by Proposition 5) that the reverse inequality holds in general, and so we have equality for $\alpha \in [0,1]$ for this example.

For $\alpha =H({\mathrm{Z}}_V)=H({\mathrm{X}}_a,{\mathrm{X}}_b,{\mathrm{X}}_c)=3$, every user can simply retain their source without compression; that is, with ${\tilde{\mathrm{Z}}}_i={\mathrm{Z}}_i$ for $i\in V$ while satisfying the entropy limit (7). Now, with $\mathrm{K}=({\mathrm{X}}_a^n,{\mathrm{X}}_b^n)$ and $\mathrm{F}={\mathrm{F}}_2={\mathrm{X}}_b^n\oplus {\mathrm{X}}_c^n$ where ⊕ is the elementwise XOR, it can be shown that both the recoverability (4) and secrecy (5) constraints hold. This is because user 3 can recover ${\mathrm{X}}_b$ from the XOR ${\mathrm{X}}_b\oplus {\mathrm{X}}_c$ with the side information ${\mathrm{X}}_c$. Furthermore, the XOR bit is independent of $({\mathrm{X}}_a,{\mathrm{X}}_b)$ and therefore does not leak any information about the key bits. With this scheme, ${\tilde{C}}_{\mathrm{S}}(3)\ge 2$. By the usual time-sharing argument, we have

$$\begin{array}{c}\hfill {\tilde{C}}_{\mathrm{S}}(\alpha )\ge \left\{\begin{array}{cc}\frac{1+\alpha }{2}\hfill & \mathit{for}\alpha \in [1,3]\hfill \\ 2\hfill & \mathit{for}\alpha \ge 3.\hfill \end{array}\right.\end{array}$$

(16b)

Indeed, the reverse inequality can be argued using one of the main results (Theorem 1) and so the minimum admissible joint entropy will turn out to be ${\alpha }_{\mathrm{S}}=3$.

3. Preliminaries

In this section, a brief summary of related results for the secrecy capacity and communication complexity will be given. The results for the two-user case are introduced first, followed by the more general results for the multi-user case, and the stronger results for the special hypergraphical source model. An example will also be given at the end to illustrate some of the results.

3.1. Two-User Case

As mentioned in the introduction, no single-letter characterization is known for $C_{\mathrm{S}}(R)$ and ${\tilde{C}}_{\mathrm{S}}(\alpha )$, even in the two-user case where $V:=1,2$. Furthermore, while multi-letter characterizations for $R_{\mathrm{S}}$ and $C_{\mathrm{S}}(R)$ were given in [11] and [13], respectively, in the two-user case under interactive discussion, no such multi-letter characterization is known for the case with non-interactive discussion. Nevertheless, if one-way discussion from user 1 is considered, then the result of [7] (Theorem 2.4) and its extension [8] to continuous sources gave the following characterization of $C_{\mathrm{S}}(R)$:

$$\begin{array}{c}\hfill C_{\mathrm{S},1}(R):=supI({\mathrm{Z}}_1^{\prime }\wedge {\mathrm{Z}}_2)\mathrm{where}\end{array}$$

(17a)

$$\begin{array}{c}I({\mathrm{Z}}_1^{\prime }\wedge {\mathrm{Z}}_1)-I({\mathrm{Z}}_1^{\prime }\wedge {\mathrm{Z}}_2)\le R\hfill \end{array}$$

(17b)

$$\begin{array}{c}I({\mathrm{Z}}_1^{\prime }\wedge {\mathrm{Z}}_2|{\mathrm{Z}}_1)=0.\hfill \end{array}$$

(17c)

The last constraint (17c) corresponds to the Markov chain ${\mathrm{Z}}_1^{\prime }-{\mathrm{Z}}_1-{\mathrm{Z}}_2$ and so the supremum is taken over the choices of the conditional distribution $P_{{\mathrm{Z}}_1^{\prime }|{\mathrm{Z}}_1}=P_{{\mathrm{Z}}_1^{\prime }|{\mathrm{Z}}_1,{\mathrm{Z}}_2}$. Using the double Markov property as in [11], it follows that $C_{\mathrm{S}}(0)$ can be characterized more explicitly by the Gács–Körner common information

$$\begin{array}{c}\hfill J_{\mathrm{GK}}({\mathrm{Z}}_1\wedge {\mathrm{Z}}_2):=sup\{H(\mathrm{U})\mid H(\mathrm{U}|{\mathrm{Z}}_1)=H(\mathrm{U}|{\mathrm{Z}}_2)=0\}\end{array}$$

(18)

where $\mathrm{U}$ is a discrete random variable. If (18) is finite, a unique optimal solution $\mathrm{U}$ exists and is called the maximum common function of ${\mathrm{Z}}_1$ and ${\mathrm{Z}}_2$ because any common function of ${\mathrm{Z}}_1$ and ${\mathrm{Z}}_2$ must be a function of $\mathrm{U}$. The communication complexity also has a more explicit characterization [11] (Equation (44))

$$\begin{array}{cc}\hfill R_{\mathrm{S},1}& =J_{\mathrm{W},1}({\mathrm{Z}}_1\wedge {\mathrm{Z}}_2)-I({\mathrm{Z}}_1\wedge {\mathrm{Z}}_2)\mathrm{where}\hfill \end{array}$$

(19)

$$\begin{array}{c}{J}_{\mathrm{W},1}({\mathrm{Z}}_1\wedge {\mathrm{Z}}_2):=inf\{H(\mathrm{W})\mid H(\mathrm{W}|{\mathrm{Z}}_1)=0,I({\mathrm{Z}}_1\wedge {\mathrm{Z}}_2|\mathrm{W})=0\}\hfill \end{array}$$

(20)

and $\mathrm{W}$ is a discrete random variable. If $J_{\mathrm{W},1}({\mathrm{Z}}_1\wedge {\mathrm{Z}}_2)$ is finite, a unique optimal solution $\mathrm{W}$ exists and is called the minimum sufficient statistics of ${\mathrm{Z}}_1$ for ${\mathrm{Z}}_2$ since ${\mathrm{Z}}_2$ can only depend on ${\mathrm{Z}}_1$ through $\mathrm{W}$.

In Section 4, the expression $C_{\mathrm{S},1}(R)$ will be related to the compressed secret key agreement restricted to the two-user case when the entropy limit is imposed only on user 1. This duality relationship in the two-user case will serve as the motivation of the main results for the multi-user case. Indeed, the desired characterization of ${\tilde{C}}_{\mathrm{S}}(\alpha )$ for the two-user case has appeared in [30] (Lemma 4.1) for the scalar Gaussian source model:

$$\begin{array}{c}\hfill {\tilde{C}}_{\mathrm{S},1}(\alpha ):=supI({\mathrm{Z}}_1^{\prime }\wedge {\mathrm{Z}}_2)\mathrm{where}\end{array}$$

(21a)

$$\begin{array}{c}I({\mathrm{Z}}_1^{\prime }\wedge {\mathrm{Z}}_1)\le \alpha \hfill \end{array}$$

(21b)

$$\begin{array}{c}I({\mathrm{Z}}_1^{\prime }\wedge {\mathrm{Z}}_2|{\mathrm{Z}}_1)=0.\hfill \end{array}$$

(21c)

For the general source model, the expression (21) has also appeared before with other information-theoretic interpretations, as mentioned in [43]. In particular, the Lagrangian dual of (21) reduces to the dimension reduction technique called the information bottleneck method in [39], where ${\mathrm{Z}}_1$ is an observable used to predict the target ${\mathrm{Z}}_2$, and ${\mathrm{Z}}_1^{\prime }$ is a feature of ${\mathrm{Z}}_1$ that captures as much mutual information with the target variable as possible per bit of mutual information with the observable. Interestingly, the principal of the information bottleneck method was also proposed in [44,45] as a way to understand deep learning, since the best prediction of ${\mathrm{Z}}_2$ from ${\mathrm{Z}}_1$ is nothing but a particular feature of ${\mathrm{Z}}_1$ sharing a lot of mutual information with ${\mathrm{Z}}_2$.

3.2. General Source with Finite Alphabet Set

Consider the multi-user case where $|V\ge 2|$. If ${\mathrm{Z}}_V$ takes values from a finite set, then the unconstrained secrecy capacity was shown in [6] to be achievable via communication for omniscience (CO) and equal to

$$\begin{array}{c}\hfill C_{\mathrm{S}}(\infty )=H({\mathrm{Z}}_V)-R_{\mathrm{CO}},\end{array}$$

(22)

where $R_{\mathrm{CO}}$ is the smallest rate of CO [6] characterized by the linear program

$$\begin{array}{c}\hfill R_{\mathrm{CO}}=\underset{r_V}{min}r(V)\mathrm{such}\mathrm{that}\end{array}$$

(23a)

$$\begin{array}{c}\hfill r(B)\ge H({\mathrm{Z}}_B|{\mathrm{Z}}_{V\backslash B})\forall B⊊V,\end{array}$$

(23b)

where $r(B)$ denotes the sum ${\sum }_{i\in B}{r}_i$. Further, $R_{\mathrm{CO}}$ can be achieved by non-interactive discussion. It follows that

$$\begin{array}{ccc}\hfill R_{\mathrm{S}}\le R_{\mathrm{CO}},& \hfill & \mathrm{or}\mathrm{equivalently}\hfill \end{array}$$

(24a)

$$\begin{array}{cc}\hfill C_{\mathrm{S}}(R)=C_{\mathrm{S}}(\infty )& R\ge R_{\mathrm{CO}}.\hfill \end{array}$$

(24b)

It was also pointed out in [6] that private randomization does not increase $C_{\mathrm{S}}(\infty )$. Hence, if $Z_V$ is finite, we have

$$\begin{array}{c}\hfill {\alpha }_{\mathrm{S}}\le H({\mathrm{Z}}_V)\end{array}$$

(25)

because $C_{\mathrm{S}}(\infty )$ can be achieved with ${\tilde{\mathrm{Z}}}_i={\mathrm{Z}}_i$. While it seems plausible that randomization does not decrease $\mathrm{S}$ nor increase $C_{\mathrm{S}}(R)$ for any $R\ge 0$, a rigorous proof remains elusive. Similarly, it appears plausible that neither ${\alpha }_{\mathrm{S}}$ nor ${\tilde{C}}_{\mathrm{S}}(\alpha )$ are affected by randomization, but, again, no proof is known yet.

An alternative characterization of $C_{\mathrm{S}}(\infty )$ was established in [24,33] by showing that the divergence bound in [6] is tight in the case without helpers. More precisely, with ${\mathsf{\Pi }}^{\prime }(V)$ defined as the set of partitions of V into at least two non-empty disjoint sets, then

$$\begin{array}{c}\hfill C_{\mathrm{S}}(\infty )=I({\mathrm{Z}}_V):=\underset{\mathcal{P}\in {\mathsf{\Pi }}^{\prime }(V)}{min}{I}_{\mathcal{P}}({\mathrm{Z}}_V),\mathrm{where}\end{array}$$

(26a)

$$\begin{array}{cc}\hfill I_{\mathcal{P}}({\mathrm{Z}}_V)& :=\frac{1}{|\mathcal{P}|-1}D\left(P_{{\mathrm{Z}}_V}\left|\right|\prod _{C\in \mathcal{P}}{P}_{{\mathrm{Z}}_C}\right)\hfill \\ & =\frac{1}{|\mathcal{P}|-1}[{\sum }_{C\in \mathcal{P}}H({\mathrm{Z}}_C)-H({\mathrm{Z}}_V)].\hfill \end{array}$$

(26b)

In the bivariate case for which $V=\{1,2\}$, $I({\mathrm{Z}}_V)$ reduces to Shannon’s mutual information $I({\mathrm{Z}}_1\wedge {\mathrm{Z}}_2)$. It was further pointed out in [32] that $I({\mathrm{Z}}_V)$ is the minimum solution $\gamma$ to the residual independence relation

$$\begin{array}{c}\hfill H({\mathrm{Z}}_V)-\gamma =\sum _{C\in \mathcal{P}}[H({\mathrm{Z}}_C)-\gamma ]\end{array}$$

(27)

for some $\mathcal{P}\in {\mathsf{\Pi }}^{\prime }(V)$. To get an intuition of the above relation, notice that $\gamma =0$ is a solution when the joint entropy $H({\mathrm{Z}}_V)$ on the left is equal to the sum of entropies $H({\mathrm{Z}}_C)$’s on the right for some partition $\mathcal{P}$ . In other words, the MMI is the smallest value of $\gamma$ removal of which leads to an independence relation; i.e., the total residual randomness on the left is equal to the sum of individual residual randomness on the right according to some partitioning of the random variables. It was further shown in [32] that there is a unique finest optimal partition to (26a) with a clustering interpretation in [36]. The MMI is also computable in polynomial time, following the result of Fujishige [46].

In the opposite extreme with $R\to 0$, it is easy to argue that

$$\begin{array}{c}\hfill C_{\mathrm{S}}(0)\ge J_{\mathrm{GK}}({\mathrm{Z}}_V)\end{array}$$

(28)

where $J_{\mathrm{GK}}({\mathrm{Z}}_V)$ is the multivariate extension of the Gács–Körner common information in (18)

$$\begin{array}{c}\hfill J_{\mathrm{GK}}({\mathrm{Z}}_V):=sup\{H(\mathrm{U})\mid H(\mathrm{U}|{\mathrm{Z}}_i)=0\forall i\in V\}\end{array}$$

(29)

with $\mathrm{U}$ again chosen as a discrete random variable. Note that even without any public discussion, every user can compress their source independently to ${\mathrm{U}}^n$ where $\mathrm{U}$ is the maximum common function if $J_{\mathrm{GK}}({\mathrm{Z}}_V)$ is finite. Hence, it is easy to achieve a secret key rate of $H(\mathrm{U})=J_{\mathrm{GK}}({\mathrm{Z}}_V)$ without any discussion. The reverse inequality of (28) seems plausible, but has not yet been proven, except in the two-user case. The technique in [7] which relies on the Csiszár sum identity does not appear to extend to the multi-user case to give a matching converse.

3.3. Hypergraphical Sources

Stronger results have been derived for the following special source model:

Definition 3 

(Definition 2.4 of [24]). ${\mathrm{Z}}_V$ is a hypergraphical source w.r.t. a hypergraph $(V,E,\zeta )$ with edge functions $\zeta :E\to 2^V\backslash \left\{\varnothing \right\}$ iff, for some independent edge variables ${\mathrm{X}}_e$ for $e\in E$ with $H({\mathrm{X}}_e)>0$,

$${\mathrm{Z}}_i:=({\mathrm{X}}_e\mid e\in E,i\in \zeta (e))\mathit{for}i\in V.$$

(30)

In the special case for which the hypergraph is a graph (i.e., $|\zeta (e)|=2$), the model reduces to the pairwise independent network (PIN) model in [23]. The hypergrahical source can also be viewed as a special case of the finite linear source considered in [47] if the edge random variables take values from a finite field.

For hypergraphical sources, various bounds on $R_{\mathrm{S}}$ and $C_{\mathrm{S}}(R)$ have been derived in [20,21,22,26]. The achieving scheme makes use of the idea of decremental secret key agreement [28,29], where the redundant or less useful edge variables are removed or reduced before public discussion. This is a special case of the compressed secret key agreement, where the compression step simply selects the more useful edge variables up to the joint entropy limit.

For the PIN model, it turns out that decremental secret key agreement is optimal, leading to a single-letter characterization of $R_{\mathrm{S}}$ and $C_{\mathrm{S}}(R)$ in [26]:

$$\begin{array}{c}\hfill R_{\mathrm{S}}=(|V|-2)C_{\mathrm{S}}(\infty ).\end{array}$$

(31a)

$$\begin{array}{c}\hfill C_{\mathrm{S}}(R)=min\left\{\frac{R}{|V|-2},C_{\mathrm{S}}(\infty )\right\}\mathrm{for}R\ge 0.\end{array}$$

(31b)

It can be verified that (31a) is the smallest value of R such that $C_{\mathrm{S}}(R)=C_{\mathrm{S}}(\infty )$ using (31b). While the proof of the converse (i.e., ≤ for (31b)) is rather involved, the achievability is by a simple tree packing protocol, which belongs to the decremental secret key agreement approach that removes excess edges unused for the maximum tree packing. In other words, the achieving scheme is a compressed secret key agreement scheme. This connection will lead to a single-letter characterization of ${\tilde{C}}_{\mathrm{S}}(\alpha )$ for the PIN model (in Theorem 2).

To illustrate the above results, a single-letter characterization for $C_{\mathrm{S}}(R)$ is derived in the following for the source in Example 1. It also demonstrates how an exact characterization for $C_{\mathrm{S}}(R)$ can be extended from a PIN model to a hypergraphical model via some contrived arguments. The characterization is also useful later in Example 3 to give an exact characterization of ${\tilde{C}}_{\mathrm{S}}(\alpha )$.

Example 2.

The source defined in (15) in Example 1, for instance, is a hypergraphical source with $E=\{a,b,c\}$, $\zeta (a)=\{1,2,3\}$, $\zeta (b)=\{1,2\}$ and $\zeta (c)=\{2,3\}$. By (23), we have $R_{\mathrm{CO}}=1$ with the optimal solution $r_1=r_3=0$ and $r_2=1$. This means that user 2 needs to discuss 1 bit to attain omniscience. In particular, user 2 can reveal the XOR ${\mathrm{X}}_b\oplus {\mathrm{X}}_c$ so that users 1 and 3 can recover ${\mathrm{X}}_c$ and ${\mathrm{X}}_b$, respectively, from their observations. By (24b), we have

$$\begin{array}{c}\hfill C_{\mathrm{S}}(R)=C_{\mathrm{S}}(\infty )=H({\mathrm{Z}}_V)-R_{\mathrm{CO}}=2\mathit{for}R\ge R_{\mathrm{CO}}=1.\end{array}$$

(32)

It can also be checked that the alternative characterization of $C_{\mathrm{S}}(\infty )$ in (26) gives

$$\begin{array}{c}\hfill C_{\mathrm{S}}(\infty )=I({\mathrm{Z}}_V)=\frac{1}{2}[H({\mathrm{Z}}_1)+H({\mathrm{Z}}_2)+H({\mathrm{Z}}_3)-H({\mathrm{Z}}_{\{1,2,3\}})]=2.\end{array}$$

Next, we argue that

$$\begin{array}{c}\hfill C_{\mathrm{S}}(R)=1+R\mathit{for}R\in [0,1].\end{array}$$

(33)

The achievability (i.e., the inequality $C_{\mathrm{S}}(R)\ge 1+R$) is by the usual time-sharing argument. In particular, the bound $C_{\mathrm{S}}(0.5)\ge 1.5$, for example, can be achieved by the compressed secret key agreement scheme in Example 1 with $\alpha =2$ (i.e., by time-sharing the compressed secret key agreement schemes for $\alpha =1$ and for $\alpha =3$ equally). More precisely, we set ${\tilde{\mathrm{Z}}}_1=({\mathrm{X}}_a^n,{\mathrm{X}}_b^{\lfloor n/2\rfloor })$, ${\tilde{\mathrm{Z}}}_2=({\mathrm{X}}_a^n,{\mathrm{X}}_b^{\lfloor n/2\rfloor },{\mathrm{X}}_c^{\lfloor n/2\rfloor })$, ${\tilde{\mathrm{Z}}}_3=({\mathrm{X}}_a^n,{\mathrm{X}}_c^{\lfloor n/2\rfloor })$, $\mathrm{K}=({\mathrm{X}}_a^n,{\mathrm{X}}_b^{\lfloor n/2\rfloor })$, and $\mathrm{F}={\mathrm{F}}_2={\mathrm{X}}_b^{\lfloor n/2\rfloor }\oplus {\mathrm{X}}_c^{\lfloor n/2\rfloor }$. It follows that the public discussion rate is ${lim\; sup}_{n\to \infty }\frac{1}{n}log|F|=0.5$.

Now, to prove the reverse inequality ≤ for (33), we modify the source ${\mathrm{Z}}_V$ to another source ${\mathrm{Z}}_V^{\prime }$ defined as follows with an additional uniformly random and independent bit ${\mathrm{X}}_d$:

$$\begin{array}{c}\hfill {\mathrm{Z}}_1^{\prime }:=({\mathrm{X}}_a,{\mathrm{X}}_b),{\mathrm{Z}}_2^{\prime }:=({\mathrm{X}}_a,{\mathrm{X}}_b,{\mathrm{X}}_c,{\mathrm{X}}_d),\mathit{and}{\mathrm{Z}}_3^{\prime }:=({\mathrm{X}}_c,{\mathrm{X}}_d).\end{array}$$

Note that ${\mathrm{Z}}_V^{\prime }$ is different from ${\mathrm{Z}}_V$; namely, ${\mathrm{Z}}_2^{\prime }$ is obtained from ${\mathrm{Z}}_2$ by adding ${\mathrm{X}}_d$, and ${\mathrm{Z}}_3^{\prime }$ is obtained from ${\mathrm{Z}}_3$ by adding ${\mathrm{X}}_d$ and removing ${\mathrm{X}}_a$. It follows that ${\mathrm{Z}}_V^{\prime }$ is a PIN. By (26) and (31b), the constrained secrecy capacity for the modified source ${\mathrm{Z}}_V^{\prime }$ is

$$\begin{array}{c}\hfill C_{\mathrm{S}}^{\prime }(R)=min\left\{R,2\right\}.\end{array}$$

The desired inequality is proved if we can show that

$$\begin{array}{c}\hfill C_{\mathrm{S}}^{\prime }(R+1)\ge C_{\mathrm{S}}(R).\end{array}$$

To argue this, we note that, if user 2 reveals ${\mathrm{F}}_2^{\prime }={\mathrm{X}}_a\oplus {\mathrm{X}}_d$ in public, then user 3 can recover ${\mathrm{X}}_a$. Furthermore, ${\mathrm{F}}_2^{\prime }$ does not leak any information about ${\mathrm{X}}_a$, and so the source ${\mathrm{Z}}_V^{\prime }$ effectively emulates the source ${\mathrm{Z}}_V$. Consequently, any optimal discussion scheme ${\mathrm{F}}_V$ that achieves $C_{\mathrm{S}}(R)$ for ${\mathrm{Z}}_V$ can be used to achieve the same secret key rate but after an additional bit of discussion ${\mathrm{F}}_2^{\prime }$. This gives the desired inequality that establishes (33).

4. Multi-Letter Characterization

We start with a simple multi-letter characterization of the compressed secrecy capacity in terms of the MMI (26).

Proposition 2.

For any $\alpha \ge 0$, we have

$$\begin{array}{c}\hfill {\tilde{C}}_{\mathrm{S}}(\alpha )=sup\underset{n\to \infty }{lim}\frac{1}{n}I({\tilde{\mathrm{Z}}}_V)\end{array}$$

(34)

where the supremum is over all valid compressed source ${\tilde{\mathrm{Z}}}_V$ satisfying the joint entropy limit (7).

Proof. 

This is because the compressed secrecy capacity is simply the secret key agreement on a compressed source. Hence, by (26), the MMI on the compressed source gives the compressed secrecy capacity. ☐

The characterization in (34) is simpler than the formulation in (6) because it does not involve the random variables $\mathrm{F}$ and $\mathrm{K}$, nor the recoverability (4) and secrecy (5) constraints. Although such a multi-letter expression is not computable and therefore not accepted as a solution to the problem, it serves as an intermediate step that helps derive further results. More precisely, consider the bivariate case where $V=\{1,2\}$. Then, (34) becomes

$$\begin{array}{c}\hfill {\tilde{C}}_{\mathrm{S}}(\alpha )=sup\underset{n\to \infty }{lim}\frac{1}{n}I({\tilde{\mathrm{Z}}}_1\wedge {\tilde{\mathrm{Z}}}_2)\mathrm{where}\end{array}$$

(35a)

$$\begin{array}{c}\underset{n\to \infty }{lim\; sup}\frac{1}{n}H({\tilde{\mathrm{Z}}}_1,{\tilde{\mathrm{Z}}}_2)-\alpha \le 0.\hfill \end{array}$$

(35b)

If in addition the joint entropy constraint (35b) is replaced by the entropy constraint on user 1 only, i.e.,

$$\begin{array}{c}\hfill \underset{n\to \infty }{lim\; sup}\frac{1}{n}H({\tilde{\mathrm{Z}}}_1)-\alpha \le 0,\end{array}$$

(35c)

then ${\tilde{C}}_{\mathrm{S}}(\alpha )$ can be single-letterized by standard techniques as in [7] to ${\tilde{C}}_{\mathrm{S},1}(\alpha )$ defined in (21). The following gives a simple upper bound that is tight for sufficiently small $\alpha$.

Proposition 3.

${\tilde{C}}_{\mathrm{S},1}(\alpha )$ defined in (21) is continuous, non-decreasing, and concave in $\alpha \ge 0$ with

$$\begin{array}{c}\hfill {\tilde{C}}_{\mathrm{S},1}(\alpha )\le \alpha .\end{array}$$

(36)

Furthermore, equality holds iff $\alpha \le J_{\mathrm{GK}}({\mathrm{Z}}_1\wedge {\mathrm{Z}}_2)$.

Proof. 

Monotonicity is obvious. Continuity and concavity can be shown by the usual time-sharing argument as in Proposition 1. (36) follows directly from the data processing inequality that $I({\mathrm{Z}}_1^{\prime }\wedge {\mathrm{Z}}_2)\le I({\mathrm{Z}}_1^{\prime }\wedge {\mathrm{Z}}_1)$ under the Markov chain ${\mathrm{Z}}_1^{\prime }-{\mathrm{Z}}_1-{\mathrm{Z}}_2$ required in (21c). If $\alpha \le J_{\mathrm{GK}}({\mathrm{Z}}_1\wedge {\mathrm{Z}}_2)$, then there exists a feasible solution $\mathrm{U}$ to (18) (a common function of ${\mathrm{Z}}_1$ and ${\mathrm{Z}}_2$) with $H(\mathrm{U})\ge \alpha$, and so the compressed sources ${\tilde{\mathrm{Z}}}_1$ and ${\tilde{\mathrm{Z}}}_2$ can be chosen as a function of ${\mathrm{U}}^n$ to achieve the equality for (36). Conversely, suppose $J_{\mathrm{GK}}({\mathrm{Z}}_1\wedge {\mathrm{Z}}_2)$ is finite and (36) is satisfied with equality. Then, in addition to ${\mathrm{Z}}_1^{\prime }-{\mathrm{Z}}_1-{\mathrm{Z}}_2$, we also have ${\mathrm{Z}}_1^{\prime }-{\mathrm{Z}}_2-{\mathrm{Z}}_1$, which implies by the double Markov property that for the maximum common function $\mathrm{U}$ achieving $J_{\mathrm{GK}}({\mathrm{Z}}_1\wedge {\mathrm{Z}}_2)$ defined in (18),

$$\begin{array}{c}\hfill I({\mathrm{Z}}_1^{\prime }\wedge {\mathrm{Z}}_1,{\mathrm{Z}}_2|\mathrm{U})=0(\mathrm{or}{\mathrm{Z}}_1^{\prime }-\mathrm{U}-({\mathrm{Z}}_1,{\mathrm{Z}}_2)).\end{array}$$

In other words, the optimal ${\mathrm{Z}}_1^{\prime }$ is a stochastic function of the maximum common function of ${\mathrm{Z}}_1$ and ${\mathrm{Z}}_2$, and so $\alpha =I({\mathrm{Z}}_1^{\prime }\wedge {\mathrm{Z}}_2)\le J_{\mathrm{GK}}({\mathrm{Z}}_1\wedge {\mathrm{Z}}_2)$ as desired. ☐

We will show that the above upper bound in (36) extends to the multi-user case (in Proposition 5). However, for $\alpha \ge J_{\mathrm{GK}}({\mathrm{Z}}_1\wedge {\mathrm{Z}}_2)$, the above upper bound is not tight even in the two-user case. To improve the upper bound, the following duality between ${\tilde{C}}_{\mathrm{S},1}$ and $C_{\mathrm{S},1}$ will be used and extended to the multi-user case (in Theorem 1).

Proposition 4.

For $\alpha \ge J_{\mathrm{GK}}({\mathrm{Z}}_1\wedge {\mathrm{Z}}_2)$,

$$\begin{array}{c}\hfill {\tilde{C}}_{\mathrm{S},1}(\alpha )=C_{\mathrm{S},1}(\alpha -{\tilde{C}}_{\mathrm{S},1}(\alpha )).\end{array}$$

(37)

Furthermore, the set of optimal solutions to the left (achieving ${\tilde{C}}_{\mathrm{S},1}(\alpha )$ defined in (21)) is the same as the set of optimal solutions to the right (achieving $C_{\mathrm{S},1}(R)$ in (17) with $R=\alpha -{\tilde{C}}_{\mathrm{S},1}(\alpha )$). It follows that the minimum admissible entropy (12) but with the entropy constraint on user 1 instead is

$$\begin{array}{c}\hfill {\alpha }_{\mathrm{S},1}=R_{\mathrm{S},1}+I({\mathrm{Z}}_1\wedge {\mathrm{Z}}_2)=J_{\mathrm{W},1}({\mathrm{Z}}_1\wedge {\mathrm{Z}}_2)\end{array}$$

(38)

where $R_{\mathrm{S},1}$ and $J_{\mathrm{W},1}({\mathrm{Z}}_1\wedge {\mathrm{Z}}_2)$ are defined in (19) and (20), respectively.

Proof. 

Set $R=\alpha -{\tilde{C}}_{\mathrm{S},1}(\alpha )$. Consider first an optimal solution ${\mathrm{Z}}_1^{\prime }$ to ${\tilde{C}}_{\mathrm{S},1}(\alpha )$ and show that it is also an optimal solution to $C_{\mathrm{S},1}(R)$. By optimality,

$$\begin{array}{c}\hfill I({\mathrm{Z}}_1^{\prime }\wedge {\mathrm{Z}}_2)={\tilde{C}}_{\mathrm{S},1}(\alpha ).\end{array}$$

(39)

By the constraint (21b), $I({\mathrm{Z}}_1^{\prime }\wedge {\mathrm{Z}}_1)\le \alpha$. It follows that the constraint (17b) holds, and so ${\mathrm{Z}}_1^{\prime }$ is a feasible solution to $C_{\mathrm{S},1}(R)$; i.e., we have ≥ for (37) that

$$\begin{array}{c}\hfill {\tilde{C}}_{\mathrm{S},1}(\alpha )\ge C_{\mathrm{S},1}(\alpha -{\tilde{C}}_{\mathrm{S},1}(\alpha )).\end{array}$$

(40)

To show that ${\mathrm{Z}}_1^{\prime }$ is also optimal to $C_{\mathrm{S},1}(R)$, suppose to the contrary that there exists a strictly better solution ${\mathrm{Z}}_1^{″}$ to $C_{\mathrm{S},1}(R)$; i.e., with

$$\begin{array}{c}\hfill I({\mathrm{Z}}_1^{″}\wedge {\mathrm{Z}}_2)>I({\mathrm{Z}}_1^{\prime }\wedge {\mathrm{Z}}_2)={\tilde{C}}_{\mathrm{S},1}(\alpha ).\end{array}$$

(41)

It follows that

$$\begin{array}{c}\hfill I({\mathrm{Z}}_1^{″}\wedge {\mathrm{Z}}_1)>I({\mathrm{Z}}^{\prime }\wedge {\mathrm{Z}}_1)=\alpha .\end{array}$$

(42)

The last equality means that the constraint (21b) is satisfied with equality. If on the contrary that the equality does not hold, setting ${\mathrm{Z}}_1^{\prime }$ to be ${\mathrm{Z}}_1^{″}$ for some fraction $\lambda >0$ of time gives a better solution to $C_{\mathrm{S},1}(R)$, contradicting the optimality of ${\mathrm{Z}}_1^{\prime }$. The first inequality can also be argued similarly by the optimality of ${\mathrm{Z}}_1^{\prime }$. Now, we have

$$\begin{array}{c}\hfill \frac{I({\mathrm{Z}}_1^{″}\wedge {\mathrm{Z}}_2)-I({\mathrm{Z}}_1^{\prime }\wedge {\mathrm{Z}}_2)}{I({\mathrm{Z}}_1^{″}\wedge {\mathrm{Z}}_1)-I({\mathrm{Z}}_1^{\prime }\wedge {\mathrm{Z}}_1)}\stackrel{(a)}{\le }\frac{I({\mathrm{Z}}_1^{\prime }\wedge {\mathrm{Z}}_2)}{I({\mathrm{Z}}_1^{\prime }\wedge {\mathrm{Z}}_1)}\stackrel{(b)}{\le }1,\end{array}$$

where (a) is by the concavity of ${\tilde{C}}_{\mathrm{S},1}(\alpha )$; and (b) is by the upper bound ${\tilde{C}}_{\mathrm{S},1}(\alpha )\le \alpha$ in (36). We note that equality cannot hold simultaneously for (a) and (b) because, otherwise, we have $\frac{I({\mathrm{Z}}_1^{″}\wedge {\mathrm{Z}}_2)}{I({\mathrm{Z}}_1^{″}\wedge {\mathrm{Z}}_1)}=1$, which, together with (41) and (42), contradicts the result in Proposition 3 that ${\tilde{C}}_{\mathrm{S},1}(\alpha )<\alpha$ (with strict inequality) for $\alpha >J_{\mathrm{GK}}({\mathrm{Z}}_1\wedge {\mathrm{Z}}_2)$. Hence,

$$\begin{array}{c}\hfill \frac{I({\mathrm{Z}}_1^{″}\wedge {\mathrm{Z}}_2)-I({\mathrm{Z}}_1^{\prime }\wedge {\mathrm{Z}}_2)}{I({\mathrm{Z}}_1^{″}\wedge {\mathrm{Z}}_1)-I({\mathrm{Z}}_1^{\prime }\wedge {\mathrm{Z}}_1)}<1,\end{array}$$

which, together with (41) and (42), implies

$$\begin{array}{c}\hfill I({\mathrm{Z}}_1^{″}\wedge {\mathrm{Z}}_1)-I({\mathrm{Z}}_1^{″}\wedge {\mathrm{Z}}_2)>\alpha -{\tilde{C}}_{\mathrm{S},1}(\alpha )=R\end{array}$$

contradicting even the feasibility of ${\mathrm{Z}}_1^{″}$ to $C_{\mathrm{S},1}(R)$; namely, the constraint (17b) with ${\mathrm{Z}}_1^{\prime }$ replaced with ${\mathrm{Z}}_1^{″}$. This completes the proof of the optimality of ${\mathrm{Z}}_1^{\prime }$ to $C_{\mathrm{S},1}(R)$.

Next, consider showing that an optimal solution ${\mathrm{Z}}_1^{\prime }$ to $C_{\mathrm{S},1}(R)$ is also optimal to ${\tilde{C}}_{\mathrm{S},1}(\alpha )$. Then,

$$\begin{array}{c}\hfill I({\mathrm{Z}}_1^{\prime }\wedge {\mathrm{Z}}_1)\le R+I({\mathrm{Z}}_1^{\prime }\wedge {\mathrm{Z}}_2)=\alpha -{\tilde{C}}_{\mathrm{S},1}(\alpha )+C_{\mathrm{S},1}(R)\le \alpha \end{array}$$

where the first inequality is by (17b); the second equality is by the optimality of ${\mathrm{Z}}_1^{\prime }$; and the last inequality follows from (40). Hence, the constraint (21b) holds and so ${\mathrm{Z}}_1^{\prime }$ is a feasible solution for ${\tilde{C}}_{\mathrm{S},1}(\alpha )$. If on the contrary that we have a better solution ${\mathrm{Z}}_1^{″}$ for ${\tilde{C}}_{\mathrm{S},1}(\alpha )$, then ${\mathrm{Z}}_1^{″}$ can be shown to be a feasible solution for $C_{\mathrm{S},1}(R)$, contradicting the optimality of ${\mathrm{Z}}_1^{\prime }$. ☐

5. Main Results

The following extends the single-letter upper bound (36) in Proposition 3 to the multi-user case.

Proposition 5.

${\tilde{C}}_{\mathrm{S}}(\alpha )\le \alpha$ with equality if $\alpha \le J_{\mathrm{GK}}({\mathrm{Z}}_V)$.

Proof. 

The upper bound ${\tilde{C}}_{\mathrm{S}}(\alpha )\le \alpha$ is because $n{\tilde{C}}_{\mathrm{S}}(\alpha )$ cannot exceed the unconstrained secrecy capacity for the compressed source ${\tilde{\mathrm{Z}}}_V$, which, by (22) and (7), is upper-bounded by $H({\tilde{\mathrm{Z}}}_V)\le n[\alpha +{\delta }_n]$ for some ${\delta }_n\to 0$ as $n\to \infty$.

Next, to prove the equality condition is sufficient, suppose $\alpha \le J_{\mathrm{GK}}({\mathrm{Z}}_V)$. Then, each user can compress their source directly to a common secret key at rate $\alpha$ without any public discussion. Hence, ${\tilde{C}}_{\mathrm{S}}(\alpha )=\alpha$ as desired. ☐

Note that unlike the two-user case in Proposition 3, the equality condition above in terms of the multivariate Gács–Körner common information is sufficient but not shown to be necessary. Nevertheless, necessity seems very plausible, as there seems to be no counter-example that suggests otherwise.

As in Proposition 4, a duality can be proved in the multi-user case, relating the compressed secret key agreement problem to the constrained secrecy key agreement problem.

Theorem 1.

With $C_{\mathrm{S}}(R)$ and $\mathrm{S}$ defined in (9) and (12) respectively, we have

$$\begin{array}{c}\hfill {\alpha }_{\mathrm{S}}\ge \mathrm{S}+C_{\mathrm{S}}(\infty )\end{array}$$

(43a)

$$\begin{array}{c}\hfill {\tilde{C}}_{\mathrm{S}}(\alpha )\le C_{\mathrm{S}}(\alpha -{\tilde{C}}_{\mathrm{S}}(\alpha ))\end{array}$$

(43b)

for all $\alpha \ge 0$.

Proof. 

Equation (43a) can be obtained from (43b) by setting $\alpha ={\alpha }_{\mathrm{S}}$ as follows:

$$\begin{array}{c}\hfill C_{\mathrm{S}}(\infty )\stackrel{(a)}{\ge }{C}_{\mathrm{S}}({\alpha }_{\mathrm{S}}-{\tilde{C}}_{\mathrm{S}}({\alpha }_{\mathrm{S}}))\stackrel{(b)}{\ge }{\tilde{C}}_{\mathrm{S}}({\alpha }_{\mathrm{S}})\stackrel{(c)}{=}{C}_{\mathrm{S}}(\infty )\end{array}$$

where (b) is given by (43b) with $\alpha ={\alpha }_{\mathrm{S}}$; while (a) and (c) follows directly from (11), (13), and monotonicity. It follows that the inequalities (a) and (b) hold with equality. In particular, equality for (a) means that $C_{\mathrm{S}}(R)=C_{\mathrm{S}}(\infty )$ for $R\ge {\alpha }_{\mathrm{S}}-{\tilde{C}}_{\mathrm{S}}({\alpha }_{\mathrm{S}})={\alpha }_{\mathrm{S}}-C_{\mathrm{S}}(\infty )$, implying (43a) as desired.

To show (43b), we consider an optimal compressed secret key agreement scheme achieving ${\tilde{C}}_{\mathrm{S}}(\alpha )$ with an arbitrary entropy limit $\alpha$. It suffices to show that the discussion rate need not be larger than $\alpha -{\tilde{C}}_{\mathrm{S}}(\alpha )$. Letting ${\tilde{\mathrm{Z}}}_V$ be the optimal compressed source and ${\tilde{R}}_{\mathrm{CO}}$ be the smallest rate of communication for omniscience of ${\tilde{\mathrm{Z}}}_V$, which is given by (23) with ${\mathrm{Z}}_V$ replaced by ${\tilde{\mathrm{Z}}}_V$, the discussion rate for the omniscience strategy is

$$\begin{array}{c}\hfill \frac{1}{n}{\tilde{R}}_{\mathrm{CO}}=\frac{1}{n}[H({\tilde{\mathrm{Z}}}_V)-I({\tilde{\mathrm{Z}}}_V)]\end{array}$$

by (22). This simplifies to $\alpha -{\tilde{C}}_{\mathrm{S}}(\alpha )$ as desired in the limit $n\to \infty$. Note that since the omniscience strategy is non-interactive, the desired hold even if $C_{\mathrm{S}}$ and $R_{\mathrm{S}}$ are defined with non-interactive discussion. ☐

While it is obvious from the above proof that a compressed secret key agreement scheme can be used as a constrained secret key agreement scheme, yielding one of the best lower bounds for $C_{\mathrm{S}}(R)$ in [26], the above result also means that a converse result for constrained secret key agreement can be applied to compressed secret key agreement. Upper bounds on ${\tilde{C}}_{\mathrm{S}}(\alpha )$ may be obtained from the upper bounds for $C_{\mathrm{S}}(R)$ such as those in [26]. It turns out that this approach can give better upper bounds which, surprisingly, are tight for the PIN model as mentioned in Section 3.3. This leads to the following exact single-letter characterization of ${\tilde{C}}_{\mathrm{S}}(\alpha )$.

Theorem 2.

For the PIN model in Definition 3,

$$\begin{array}{c}\hfill {\alpha }_{\mathrm{S}}=(|V|-1)C_{\mathrm{S}}(\infty )\end{array}$$

(44a)

$$\begin{array}{c}\hfill {\tilde{C}}_{\mathrm{S}}(\alpha )=min\left\{\frac{\alpha }{|V|-1},C_{\mathrm{S}}(\infty )\right\}\end{array}$$

(44b)

for all $\alpha \ge 0$.

Proof. 

Equation (44a) follows easily from (44b) by setting the two terms in the minimization to be equal and solving for $\alpha$. To show (44b), note that by (31b) we have

$$\begin{array}{c}\hfill C_{\mathrm{S}}^{-1}(\gamma )=(|V|-2)\gamma \forall \gamma <C_{\mathrm{S}}(\infty )\end{array}$$

because $C_{\mathrm{S}}(R)$ is non-decreasing and concave, and thus it must be strictly non-decreasing before it reaches $C_{\mathrm{S}}(\infty )=C_{\mathrm{S}}(\infty )$. Now, by (43b),

$$\begin{array}{cc}\hfill \alpha -{\tilde{C}}_{\mathrm{S}}(\alpha )& \ge C_{\mathrm{S}}^{-1}({\tilde{C}}_{\mathrm{S}}(\alpha ))\hfill \\ & =(|V|-2){\tilde{C}}_{\mathrm{S}}(\alpha )\hfill \end{array}$$

for any $\alpha \ge 0$ such that ${\tilde{C}}_{\mathrm{S}}(\alpha )<C_{\mathrm{S}}(\infty )$; that is, for $\alpha \le {\alpha }_{\mathrm{S}}$, and thus ${\tilde{C}}_{\mathrm{S}}(\alpha )\le \frac{\alpha }{|V|-1}$. This implies ≤ for (44b). The bound is achievable by the same achieving scheme as in [26] (Theorem 4.4) along the idea of decremental secrecy key agreement and the tree packing protocol in [27]. More precisely, every $(|V|-1)$ bits of edge variable forming a spanning tree are turned into a secret key bit by the tree packing protocol. This results in the factor of $(|V|-1)$ in (44), which corresponds to the number of edges in a spanning tree.  ☐

For the more general source model, the idea of decremental secret key agreement needs to be refined, because there need not be any edge variables to remove. The following is a simple extension that leads to a single-letter lower bound on ${\tilde{C}}_{\mathrm{S}}(\alpha )$.

Theorem 3.

A single-letter lower bound on ${\tilde{C}}_{\mathrm{S}}(\alpha )$ is

$$\begin{array}{c}\hfill {\tilde{C}}_{\mathrm{S}}(\alpha )\ge I({\mathrm{Z}}_V^{\prime }|\mathrm{Q})\end{array}$$

(45)

for any random vector $(\mathrm{Q},{\mathrm{Z}}_V^{\prime })$ taking values from a finite set and satisfying

$$\begin{array}{c}\hfill I(\mathrm{Q}\wedge {\mathrm{Z}}_V)=0\end{array}$$

(46a)

$$\begin{array}{c}\hfill H({\mathrm{Z}}_i^{\prime }|{\mathrm{Z}}_i,\mathrm{Q})=0\forall i\in V\end{array}$$

(46b)

$$\begin{array}{c}\hfill H({\mathrm{Z}}_V^{\prime }|\mathrm{Q})\le \alpha .\end{array}$$

(46c)

Furthermore, it is admissible to have $|Q|\le 3$.

Proof. 

By (46b), we have ${\mathrm{Z}}_i^{\prime }={\zeta }_i({\mathrm{Z}}_i,\mathrm{Q})$ for some function ${\zeta }_i$. W.l.o.g., we let $Q:=1,\dots ,k$ for some integer $k>0$. We choose ${\tilde{\mathrm{Z}}}_i$ to be the following function of ${\mathrm{Z}}_i^n$:

$$\begin{array}{c}{\tilde{\mathrm{Z}}}_i=(({\zeta }_i({\mathrm{Z}}_{i\tau },q)\mid n_{q-1}<\tau \le n_q)\mid 1\le q\le k)\mathrm{where}\hfill \\ n_0=0\mathrm{and}{n}_q=\lfloor n\sum _{j=1}^q{P}_{\mathrm{Q}}(j)\rfloor \mathrm{for}1\le q\le k.\hfill \end{array}$$

Essentially, $\mathrm{Q}$ acts as a time-sharing random variable, where $P_{\mathrm{Q}}(q)$ is the fraction of time the source ${\mathrm{Z}}_i$ is processed to ${\mathrm{Z}}_i^{(q)}:=x_i({\mathrm{Z}}_i,q)$, for $1\le q\le k$. More precisely, we have that $\frac{n_q-n_{q-1}}{n}$ converges to $P_{\mathrm{Q}}(q)$, and thus

$$\begin{array}{cc}\hfill \frac{1}{n}I({\tilde{\mathrm{Z}}}_V)& =\sum _{q=1}^{k}I({\mathrm{Z}}_V^{(q)})\frac{n_q-n_{q-1}}{n}\hfill \\ & \stackrel{n\to \infty }{\to }I({\mathrm{Z}}_V^{\prime }|\mathrm{Q}).\hfill \end{array}$$

Similarly,

$$\begin{array}{cc}\hfill \frac{1}{n}H({\tilde{\mathrm{Z}}}_V)& \stackrel{n\to \infty }{\to }H({\mathrm{Z}}_V^{\prime }|\mathrm{Q})\le \alpha \hfill \end{array}$$

by (46c), satisfying the entropy limit of (7). Hence, ${\tilde{\mathrm{Z}}}_V$ is a valid compressed source, the unconstrained capacity of which is $I({\mathrm{Z}}_V^{\prime }|\mathrm{Q})$, leading to the desired lower bound of (45).

The condition that $|Q\le 3|$ is admissible follows from the usual argument by the well-known Eggleston–Carathéodory theorem. More precisely, by letting

$$\begin{array}{cc}\hfill \mathcal{F}:=\{(I({\mathrm{Z}}_V^{\prime }|\mathrm{Q}=q),H({\mathrm{Z}}_V^{\prime }|\mathrm{Q}=q))& \mid P_{{\mathrm{Z}}_V|\mathrm{Q}=q}=P_{{\mathrm{Z}}_V},\hfill \\ & H({\mathrm{Z}}_i^{\prime }|{\mathrm{Z}}_i,\mathrm{Q}=q)=0\}.\hfill \end{array}$$

It can be seen that the conditions above are equivalent to (46a) and (46b), respectively, and thus the set of feasible values to (46), namely

$$\begin{array}{c}\hfill (I({\mathrm{Z}}_V^{\prime }|\mathrm{Q}),H({\mathrm{Z}}_V^{\prime }|\mathrm{Q}))=\sum _{q\in Q}{P}_{\mathrm{Q}}(q)(I({\mathrm{Z}}_V^q),H({\mathrm{Z}}_V^q)),\end{array}$$

is equal to the convex hull of $\mathcal{F}$. Because the dimension of $\mathcal{F}$ is at most 2, the pair $({\tilde{C}}_{\mathrm{S}}({\mathrm{Z}}_V),\alpha )$ can be obtained as a convex combination of at most three points in $\mathcal{F}$ as desired by the Eggleston–Carathéodory theorem. ☐

The main results above can be illustrated as follows using the hypergraphical source in Example 1 given earlier. In particular, an exact single-letter characterization of ${\tilde{C}}_{\mathrm{S}}(\alpha )$ will be derived, even though such an exact characterization is not known for general hypergraphical sources.

Example 3.

Consider the source defined in (15) in Example 1. It is shown that (16a) and (16b) are satisfied with equality, which gives the desired single-letter characterization of ${\tilde{C}}_{\mathrm{S}}(\alpha )$.

It is easy to show that $J_{\mathrm{GK}}({\mathrm{Z}}_V)=1$ as ${\mathrm{X}}_a$ is the maximum common function of ${\mathrm{Z}}_1$, ${\mathrm{Z}}_2$, and ${\mathrm{Z}}_3$. Hence, the reverse inequality of (16a) follows from Proposition 5.

The reverse inequality for (16b) can be argued using the bound in Theorem 1 by $C_{\mathrm{S}}(R)$ and the characterization of $C_{\mathrm{S}}(R)$ in Example 2. More precisely, by (32), the unconstrained secrecy capacity $C_{\mathrm{S}}(\infty )=2$. Then, by (33)), we have $C_{\mathrm{S}}^{-1}(\gamma )\le \gamma -1$ for all $\gamma \le C_{\mathrm{S}}(\infty )=2$. Now, by (43b),

$$\begin{array}{cc}\hfill \alpha -{\tilde{C}}_{\mathrm{S}}(\alpha )& \le C_{\mathrm{S}}^{-1}({\tilde{C}}_{\mathrm{S}}(\alpha ))\le {\tilde{C}}_{\mathrm{S}}(\alpha )-1\hfill \end{array}$$

and thus ${\tilde{C}}_{\mathrm{S}}(\alpha )\le \frac{1+\alpha }{2}$ for ${\tilde{C}}_{\mathrm{S}}(\alpha )\le 2$. This completes the proof.

6. Conclusion and Extensions

Inspired by the idea of decremental secret key agreement and its application to the constrained secret key agreement problem, we have formulated a multiterminal secret key agreement problem with a more general source compression step that applies beyond the hypergraphical source model. This formulation allows us to separate and compare the issues of source compression and discussion rate constraint in secret key agreement. While a single-letter characterization of the compressed secrecy capacity and admissible entropy limit remains unknown, single-letter bounds have been derived and they are likely to be tight for the hypergraphical model, and possibly more general source models such as the finite linear source model [47]. For the PIN model in particular, the bounds are tight, giving rise to a complete characterization of the capacity in Theorem 2. One way to improve the current converse results is to show whether the equality condition in Proposition 5 is necessary; that is, ${\tilde{C}}_{\mathrm{S}}(\alpha )<\alpha$ for $\alpha >J_{\mathrm{GK}}({\mathrm{Z}}_V)$. By the duality in Theorem 1, the condition is necessary if one can show that $C_{\mathrm{S}}(0)=J_{\mathrm{GK}}({\mathrm{Z}}_V)$; i.e., (28) holds with equality. Such equality can be proved for hypergraphical as well as finite linear sources by extending the lamination techniques in [26]. It is hopeful that a complete solution can be given for the finite linear source model and the well-known jointly Gaussian source model. The bounds (43) in the duality result may plausibly be tight for these special sources, in which case non-interactive discussion suffices to achieve the constrained secrecy capacity. The current achievability results may also be improved. In particular, for the two-user case with joint entropy constraint (35), the lower bound in (45) can be improved to ${\tilde{C}}_{\mathrm{S}}(\alpha )\ge maxI({\mathrm{Z}}_1^{\prime }\wedge {\mathrm{Z}}_2^{\prime })$ where $I({\mathrm{Z}}_1^{\prime }\wedge {\mathrm{Z}}_1)+I({\mathrm{Z}}_2^{\prime }\wedge {\mathrm{Z}}_2)\le \alpha$ and ${\mathrm{Z}}_1^{\prime }-{\mathrm{Z}}_1-{\mathrm{Z}}_2-{\mathrm{Z}}_2^{\prime }$. Whether this improvement is strict or is the best possible is not yet clear, but an extension to the multi-user case seems possible. A related open problem is to characterize the $C_{\mathrm{S}}(R)$ in the two-user case with two-way non-interactive discussion. A simpler question is whether two-way non-interactive discussion can be strictly better than one-way discussion.

As pointed out before, by regarding the secrecy capacity as a measure of mutual information, an optimal source compression scheme translates to a dimension reduction technique which is potentially useful for machine learning. A closely related line of work is the study of the strong data processing inequality in [43,48,49]; in particular, the ratio $s^{*}({\mathrm{Z}}_1;{\mathrm{Z}}_2):=sup\frac{I({\mathrm{Z}}_1^{\prime }\wedge {\mathrm{Z}}_2)}{I({\mathrm{Z}}_1^{\prime }\wedge {\mathrm{Z}}_1)}$ where—as in (21)—the supremum is taken over the choice of the conditional distribution $P_{{\mathrm{Z}}_1^{\prime }|{\mathrm{Z}}_1,{\mathrm{Z}}_2}$ such that ${\mathrm{Z}}_1^{\prime }-{\mathrm{Z}}_1-{\mathrm{Z}}_2$ forms a Markov chain and $I({\mathrm{Z}}_1^{\prime }\wedge \mathrm{Z})>0$. It is straightforward to show that ${sup}_{\alpha \ge 0}\frac{{\tilde{C}}_{\mathrm{S}}(\alpha )}{\alpha }$ for the two-user case in (35) is upper bounded by $s^{*}({\mathrm{Z}}_1;{\mathrm{Z}}_2)$ and $s^{*}({\mathrm{Z}}_2;{\mathrm{Z}}_1)$. However, a sharper bound and a more precise mathematical connection may be possible, and the result may be extended to the multivariate case. Furthermore, the linearization considered in [50] may potentially be adopted to provide a single-letter lower bound on the compressed secrecy capacity. As in [13,48], the problem may also be related to a notion of maximum correlation appropriately extended to the multivariate case.