On the Optimal Error Exponent of Type-Based Distributed Hypothesis Testing ^†

Abstract

Distributed hypothesis testing (DHT) has emerged as a significant research area, but the information-theoretic optimality of coding strategies is often typically hard to address. This paper studies the DHT problems under the type-based setting, which is requested from the popular federated learning methods. Specifically, two communication models are considered: (i) DHT problem over noiseless channels, where each node observes i.i.d. samples and sends a one-dimensional statistic of observed samples to the decision center for decision making; and (ii) DHT problem over AWGN channels, where the distributed nodes are restricted to transmit functions of the empirical distributions of the observed data sequences due to practical computational constraints. For both of these problems, we present the optimal error exponent by providing both the achievability and converse results. In addition, we offer corresponding coding strategies and decision rules. Our results not only offer coding guidance for distributed systems, but also have the potential to be applied to more complex problems, enhancing the understanding and application of DHT in various domains.

1. Introduction

Distributed hypothesis testing (DHT) is a significant problem in the field of information theory [1]. In this problem, each distributed node observes partial data generated from the joint distribution and transmits an encoded message through a communication channel to a decision center, aiming to detect the true hypothesis. The primary goal of DHT is to maximize the decision error exponent in the asymptotic regime, where many different communication models [2,3,4,5,6] were considered in the previous literature. The main challenges of the DHT arise in two respects. Firstly, due to the intricate distributed structures, most of the existing works have focused on demonstrating achievability results, with converse results being limited to specific cases, such as the 1-bit [3], ${log}_{2}3$-bit [7], and $O\left({log}_{2}n\right)$-bit [1] communication channels. Secondly, many of the achievability results were established using random coding with auxiliary random variables [8], which are difficult to implemen in real systems.

Notice that the distributed encoders in many real applications are required to process high-dimensional data [9], such as images, texts, and audios. Consequently, many of the federated learning algorithms focus on computing the quantities, such as the statistics, empirical risks, and gradient of data [10], which can be viewed as certain functions of the empirical distribution (type) of the data (for example, given the data $x_1,\dots ,x_n$ and feature function $f\left(x\right)$, the statistic $\frac{1}{n}{\sum }_{i=1}^{n}f\left(x_i\right)={\sum }_x{\widehat{P}}_X\left(x\right)f\left(x\right)$ is a linear function of the empirical distribution ${\widehat{P}}_X$).

Motivated by this observation, we investigate the optimal decision error exponent of DHT based on the empirical distributions (type-based) under two common communication models. The first problem considers a noiseless channel, which is the typical mathematical model in real federated learning scenarios. It comes from the reality that federated learning often assumes that the nodes and the center machine can exchange information precisely; however, the dimensionalities of the transmitted signals are limited [9]. Specifically, it is assumed that each node can only transmit the empirical mean of a one-dimensional feature, and such settings have gained significant attention recently in federated and multi-modal machine learning [9,11]. The second problem assumes that the signal of each node, encoded with the empirical distribution, is transmitted over an additive white Gaussian noise (AWGN) channel, which is a widely-used mathematical model for real-world channels [12]. The main goal of this paper is to establish the optimal error exponent for the aforementioned two problems by presenting: (i) the converse bound for the error exponent; and (ii) a practical coding strategy that achieves the converse bound.

The contributions of this paper are summarized as follows. First, in Section 4.1, we demonstrate the optimal error exponent for the type-based hypothesis testing over noiseless channels, where one-dimensional functions for all nodes and the corresponding decision rule are provided. Moreover, by applying the information geometric approach in [13], the hypotheses and the feature functions of each node can be modeled as vectors in the joint and marginal distribution spaces, respectively. In Section 4.3, the optimal feature function of each node can be interpreted as a decomposition of the hypothesis vector in the joint distribution space into vectors in the marginal distribution spaces, where each decomposed component indicates the contribution of the corresponding node in making the inference.

Second, we establish the optimal achievable error of the type-based hypothesis testing over AWGN channels by presenting both the achievability and converse results. In particular, the achievability part is based on a mixture coding strategy of both the amplify-and-forward and decode-and-forward strategies. Specifically, when the observed empirical distribution at a distributed node is sufficiently close to one of the true marginal distributions with respect to the two hypotheses, the node is confident of the true hypothesis. Then, we apply the decode-and-forward strategy, which first estimates the true hypothesis based on the observed empirical distribution, and then we apply the binary phase shift keying (BPSK) to transmit the decoded bit to the decision center. On the other hand, when the observed empirical distribution is far from both true marginal distributions, we apply the amplify-and-forward strategy to encode and transmit the observed empirical distribution by the pulse amplitude modulation (PAM) to the decision center. By applying the proposed coding strategy and conducting the log-likelihood ratio test at the decision center, we show in Section 5.2 the achievable error exponent. Finally, we demonstrate the converse results of the error exponent in Section 5.3 based on a genie-aided approach. The main idea is to add additional information to the distributed nodes. By either leveraging the true hypothesis to the distributed nodes or eliminating the channel noises, we show that the error exponent in Section 5.2 is also an upper bound of the optimal error exponent, which establishes the optimality.

2. Problem Formulations

Suppose that there are K random variables $X^K\triangleq (X_1,\dots ,X_K)$. In this paper, we consider the binary hypothesis testing problem, and the two hypotheses ${\mathsf{H}}_0$ and ${\mathsf{H}}_1$ are defined as:

$$\begin{array}{cc}\hfill & {\mathsf{H}}_0:(x_1^{\left(1\right)},\cdots ,x_K^{\left(1\right)}),\cdots ,(x_1^{\left(n\right)},\cdots ,x_K^{\left(n\right)})\stackrel{\mathrm{i}.\mathrm{i}.\mathrm{d}.}{\sim }{P}_{X^K}^{\left(0\right)},\hfill \\ \hfill & {\mathsf{H}}_1:(x_1^{\left(1\right)},\cdots ,x_K^{\left(1\right)}),\cdots ,(x_1^{\left(n\right)},\cdots ,x_K^{\left(n\right)})\stackrel{\mathrm{i}.\mathrm{i}.\mathrm{d}.}{\sim }{P}_{X^K}^{\left(1\right)},\hfill \end{array}$$

(1)

where the observable data are i.i.d. generated according to either $P_{X^K}^{\left(0\right)}$ or $P_{X^K}^{\left(1\right)}$ from the alphabet set $({\mathcal{X}}_1,\cdots ,{\mathcal{X}}_K)$. In addition, we assume that there are K distributed nodes, where the k-th ($k=1,\cdots ,K$) node can only observe the samples ${\mathsf{X}}_k\triangleq \{x_k^{\left(1\right)},\dots ,x_k^{\left(n\right)}\}$. To facilitate clarity in our illustration, we concentrate on the discrete case, assuming that each alphabet ${\mathcal{X}}_k$ is discrete, and $\mathcal{X}\triangleq {\mathcal{X}}_1\times \cdots \times {\mathcal{X}}_K$. In addition, for a joint distribution $Q_{X^K}\in {\mathcal{P}}_{\mathcal{X}}$, we use ${\left[Q_{X^K}\right]}_{X_k}$ to denote its marginal distribution with respect to $X_k$. We also denote $P_{X_1}^{\left(i\right)},\cdots ,P_{X_K}^{\left(i\right)}$ as the marginal distributions of $P_{X^K}^{\left(i\right)}$, for $i=0,1$. In the distributed hypothesis testing problem, we introduce a common assumption in the distributed setup [14] that the generating distributions $P_{X^K}^{\left(0\right)}$ and $P_{X^K}^{\left(1\right)}$ satisfy $D(P_{X^K}^{\left(1\right)}\parallel P_{X^K}^{\left(0\right)})<\infty ,D(P_{X^K}^{\left(0\right)}\parallel P_{X^K}^{\left(1\right)})<\infty$, to avoid the trivial irregularities. Due to the type-based restriction, we further assume that $P_{X_k}^{\left(0\right)}\ne P_{X_k}^{\left(1\right)}$, $k=1,\cdots ,K$. Otherwise, the transmitted message as a function of the empirical distribution would be uninformative for distinguishing the hypotheses. In the following, we denote ${\widehat{P}}_{X_k}$ as the empirical distributions of ${\mathsf{X}}_k$, defined as:

$$\begin{array}{c}\hfill {\widehat{P}}_{X_k}\left(x_k\right)\triangleq \frac{1}{n}\sum _{i=1}^n\mathbb{1}\left\{x_k=x_k^{\left(i\right)}\right\}.\end{array}$$

(2)

2.1. Type-Based Hypothesis Testing over Noiseless Channels

As shown in Figure 1, node k ($k=1,\cdots ,K$) can encode the observed data ${\mathsf{X}}_k$ and transmit a scalar signal by function $u_k$. Due to the computational requirement as introduced in Section 1, we impose a restriction whereby the encoder $u_k$ is explicitly dependent on the empirical distribution ${\widehat{P}}_{X_k}$, i.e., $u_k:{\mathcal{P}}_{{\mathcal{X}}_k}\mapsto \mathbb{R}$, and ${\mathcal{P}}_{{\mathcal{X}}_k}$ denotes the set of probability distributions defined on the alphabet ${\mathcal{X}}_k$. For the most direct method, we can transmit the emprical distributions by encoding them into the real space, which can lead to computational difficulty for federated learning data. In this paper, we further consider one of the most commonly used approaches in federated learning [15,16] and assume that $u_k$ computes a one-dimensional statistic

$$\begin{array}{c}\hfill u_k\left({\widehat{P}}_{X_k}\right)=\frac{1}{n}\sum _{i=1}^n{f}_k\left(x_k^{\left(i\right)}\right)={\mathbb{E}}_{{\widehat{P}}_{X_k}}\left[f_k\left(X_k\right)\right],\end{array}$$

(3)

where feature function $f_k:{\mathcal{X}}_k\mapsto \mathbb{R}$. Then, the decision center collects statistics ${\left\{u_k\left({\widehat{P}}_{X_k}\right)\right\}}_{k=1}^K$, and makes a decision $\widehat{\mathsf{H}}$ on the true hypothesis. We prove in Section 4 that the further restrictions of computing the empirical means of features are without a loss of generality, where we can make good decisions as we observe the types. Additionally, the error probability is defined as

$$\begin{array}{c}\hfill {\mathbb{P}}_n(\widehat{\mathsf{H}}\ne \mathsf{H})\triangleq \sum _{i\in \{0,1\}}{\mathbb{P}}_{\mathsf{H}}\left({\mathsf{H}}_i\right){\mathbb{P}}_n(\widehat{\mathsf{H}}\ne \mathsf{H}|\mathsf{H}={\mathsf{H}}_i),\end{array}$$

where $\mathsf{H}$ denotes the true hypothesis, ${\mathbb{P}}_{\mathsf{H}}\left({\mathsf{H}}_0\right)$ and ${\mathbb{P}}_{\mathsf{H}}\left({\mathsf{H}}_1\right)$ are the prior distributions, and ${\mathbb{P}}_n(·)$ is the probability measure defined from the data sampling process (1). In particular, we focus on the asymptotic error decaying rate, i.e., the error exponent, defined as

$$\begin{array}{c}\hfill E\triangleq \underset{n\to \infty }{lim}-\frac{1}{n}log{\mathbb{P}}_n(\widehat{\mathsf{H}}\ne \mathsf{H}),\end{array}$$

(4)

where all logarithms are base e unless otherwise specified. The goal is to find the maximal error exponent of (4) and design the feature functions $f_1,\cdots ,f_k$ and the detailed decision rule such that this error exponent can be achieved based on the log-likelihood ratio test (LLRT).

2.2. Type-Based Hypothesis Testing over AWGN Channels

As depicted in Figure 2, we employ the identical hypothesis testing formulation as presented in (1). In this context, it is assumed that nodes 1 through K encode and transmit a length-m sequence using functions ${\mathit{g}}_1,\cdots ,{\mathit{g}}_K$, which operate based on their respective observations through additive white Gaussian noise (AWGN) channels to the central decision center. To accommodate the computational constraints, we restrict that the encoder ${\mathit{g}}_k$ ($k=1,\cdots ,K$) is a function of the empirical distribution ${\widehat{P}}_{X_k}$, i.e.,

$$\begin{array}{c}\hfill {\mathit{g}}_k:{\mathcal{P}}_{{\mathcal{X}}_k}\mapsto {\mathbb{R}}^m,k=1,\cdots ,K.\end{array}$$

(5)

Moreover, the averaged power constraints of the AWGN channels are:

$$\begin{array}{c}\hfill \frac{1}{m}\mathbb{E}\left[{∥{\mathit{g}}_k\left({\widehat{P}}_{X_k}\right)∥}^2\right]\le p_k,k=1,\cdots ,K,\end{array}$$

(6)

where the expectations are taken over the data sampling process defined in (1). Then, the decision center makes a decision $\widehat{\mathsf{H}}$ based on the received signals ${\mathit{g}}_1\left({\widehat{P}}_{X_1}\right)+{\mathit{Z}}_1,\cdots ,{\mathit{g}}_K\left({\widehat{P}}_{X_K}\right)+{\mathit{Z}}_K$, where the noises are drawn from

$$\begin{array}{c}\hfill {\mathit{Z}}_k\sim \mathcal{N}\left(\mathbf{0},{\sigma }_k^2{\mathbf{I}}_m\right),k=1,\cdots ,K,\end{array}$$

(7)

and ${\mathbf{I}}_m$ denotes the $m\times m$ identity matrix.

Additionally, we make the following assumption to make the errors arising from the AWGN channels and the decision process comparable, so that the trade-off between them can be described. In detail, we assume that the sequence length m also increases with n, and there exists a positive constant $\mu$ such that

$$\begin{array}{c}\hfill \underset{n\to \infty }{lim}\frac{n}{m\left(n\right)}=\mu .\end{array}$$

(8)

Our goal is to design the optimal encoders ${\mathit{g}}_1,\cdots ,{\mathit{g}}_K$, subjected to the constraints (5) and (6), as well as the decision rule $\widehat{\mathsf{H}}$, where we have assumed ${\mathbb{P}}_{\mathsf{H}}\left({\mathsf{H}}_0\right)={\mathbb{P}}_{\mathsf{H}}\left({\mathsf{H}}_1\right)=\frac{1}{2}$ for explicit mathematical expression, such that the error exponent as defined in (4) is maximized.

3. Related Works

Distributed hypothesis testing problems, also known as multiterminal hypothesis testing [1,3,14] or decentralized detection [17,18], have been extensively explored in the literature. In scenarios where each node can observe a single observation and send an encoded message to the central machine, the authors of [17] demonstrated that determining the optimal coding scheme is NP-hard, while [18,19] provided characterizations for the minimum decoding error rate and the optimal coding scheme for conditionally independent nodes.

Furthermore, in situations where each node can observe n samples and transmit an encoded message to the decision center, [3,5,14,20] investigated the optimal decoding error exponents for the case of $K=2$ nodes, with [21] generalizing the results to $K>2$ nodes. Additionally, the author of [5] studied the Neyman–Pearson-like test, which further constrained the encoded messages to being an empirical functional mean, and provided optimal functions for the scenario with $K=2$ nodes. The outcome presented in Section 4 can be perceived as a generalization of such setups to the case with $K>2$ nodes.

On the other hand, DHT over noisy channels represents a novel and highly significant sub-problem within the broader context. While current research has primarily focused on transmission over discrete memoryless channels, certain aspects of this sub-problem have been investigated. For instance, some studies have explored scenarios involving side information [22] and cases that counteract independence assumptions [23]. Additionally, optimal Type-II error considerations have been examined [24], along with investigations into the optimal pairs of Type-I and Type-II errors [25].

Diverging from the existing literature, the present paper delves into the DHT problem in the context of widely considered AWGN channels while also addressing the implications of computational demands. This novel approach fills a critical research gap and extends the understanding of DHT to a broader set of channel conditions, thus contributing to the advancement of the field.

4. Type-Based Hypothesis Testing over Noiseless Channels

In this section, we present the optimal error exponent along with the corresponding decision rule for the type-based hypothesis testing over noiseless channels. We commence by introducing the optimal error exponent under the condition that the decision center has access to the empirical distributions from different nodes.

Definition 1.

The quantities $D_i^{*}(R_{X_1},\cdots ,R_{X_K})$, for $i=0,1$, are defined as

$$\begin{array}{c}\hfill D_i^{*}(R_{X_1},\cdots ,R_{X_K})\triangleq \underset{Q_{X^K}\in \mathcal{S}}{min}D(Q_{X^K}\parallel P_{X^K}^{\left(i\right)}),\end{array}$$

(9)

where

$$\begin{array}{cc}\hfill \mathcal{S}\triangleq & \left\{Q_{X^K}:{\left[Q_{X^K}\right]}_{X_k}=R_{X_k},k=1,\cdots ,K\right\},\hfill \end{array}$$

which represents the set of all distributions with given marginals $R_{X_1},\cdots ,R_{X_K}$.

The following result provides the operational meaning of (9), which can be proved by Sanov’s theorem [12].

Lemma 1.

When ${\mathsf{H}}_i$ is the true hypothesis, the probability that nodes $1,\cdots ,K$ observe the empirical distributions ${\widehat{P}}_{X_1},\cdots ,{\widehat{P}}_{X_K}$, respectively, is given by

$$\begin{array}{c}\hfill {\mathbb{P}}_n({\widehat{P}}_{X_1},\cdots ,{\widehat{P}}_{X_K}|\mathsf{H}={\mathsf{H}}_i)\doteq exp\left(-n{D}_i^{*}({\widehat{P}}_{X_1},\cdots ,{\widehat{P}}_{X_K})\right),i=0,1,\end{array}$$

where ≐ is the conventional dot-equal notation, i.e., we denote $f_n\doteq g_n$ when ${lim}_{n\to \infty }\frac{1}{n}log{f}_n={lim}_{n\to \infty }\frac{1}{n}log{g}_n$. In addition, by applying the log-likelihood ratio test to detect the true hypothesis, the optimal decision error exponent based on the empirical distributions is

$$\begin{array}{c}\hfill E^{*}\triangleq \underset{R_{X_1},\cdots ,R_{X_K}}{min}\underset{i\in \{0,1\}}{max}{D}_i^{*}(R_{X_1},\cdots ,R_{X_K}).\end{array}$$

(10)

Note that the type-based hypothesis testing problem assumes that the signal from each node is a function of the empirical distribution. Hence, the optimal error exponent in (4) will not exceed $E^{*}$. In the following, we prove that error exponent $E^{*}$ can be achieved and provide the corresponding decision rule.

4.1. Optimal Feature

First, we introduce the following definitions of exponential and linear families, which will be useful for delineating our results.

Definition 2

(Exponential family). Given distribution $P_Z\left(z\right)$, and a function $T:\mathcal{Z}\to \mathbb{R}$, we define the distribution ${\tilde{P}}_Z^{\left(\lambda \right)}(·;T,P_Z)$ as

$$\begin{array}{c}\hfill {\tilde{P}}_Z^{\left(\lambda \right)}(z;T,P_Z)\triangleq P_Z\left(z\right)exp(\lambda T\left(z\right)-\alpha \left(\lambda \right)),\mathit{for}\mathit{all}z\in \mathcal{Z},\end{array}$$

(11)

with $\alpha \left(\lambda \right)\triangleq log{\sum }_{z^{\prime }\in \mathcal{Z}}{P}_Z\left(z^{\prime }\right)exp\left(\lambda T\left(z^{\prime }\right)\right)$. In addition, we use

$$\begin{array}{c}\hfill {\mathcal{E}}_{\mathcal{Z}}(T,P_Z)\triangleq \left\{{\tilde{P}}_Z^{\left(\lambda \right)}(·;T,P_Z):\lambda \in \mathbb{R}\right\}\end{array}$$

(12)

to denote the exponential family passing through $P_Z$ with T being the natural statistic.

Definition 3

(Linear family). Given a function $h:\mathcal{Z}\to \mathbb{R}$, we define the linear family ${\mathcal{L}}_{\mathcal{Z}}\left(h\right)$ as

$$\begin{array}{c}\hfill {\mathcal{L}}_{\mathcal{Z}}\left(h\right)\triangleq \left\{Q_Z\in {\mathcal{P}}^{\mathcal{Z}}:{\mathbb{E}}_{Q_Z}\left[h\left(Z\right)\right]=0\right\}.\end{array}$$

(13)

In addition, we define the half-spaces ${\mathcal{S}}_{\mathcal{Z}}^{\left(0\right)}\left(h\right)$ and ${\mathcal{S}}_{\mathcal{Z}}^{\left(1\right)}\left(h\right)$ as

$$\begin{array}{c}{\mathcal{S}}_{\mathcal{Z}}^{\left(0\right)}\left(h\right)\triangleq \left\{Q_Z\in {\mathcal{P}}^{\mathcal{Z}}:{\mathbb{E}}_{Q_Z}\left[h\left(Z\right)\right]\le 0\right\},\\ {\mathcal{S}}_{\mathcal{Z}}^{\left(1\right)}\left(h\right)\triangleq \left\{Q_Z\in {\mathcal{P}}^{\mathcal{Z}}:{\mathbb{E}}_{Q_Z}\left[h\left(Z\right)\right]\ge 0\right\}.\end{array}$$

Then, for $i=0,1$ and $t>0$, we define the sets

$$\begin{array}{cc}\hfill {\mathcal{D}}_i\left(t\right)& \triangleq \{(R_{X_1},\cdots ,R_{X_K}):D_i^{*}(R_{X_1},\cdots ,R_{X_K})<t\}.\hfill \end{array}$$

We also define $\mathcal{D}\left(t\right)\triangleq {\mathcal{D}}_0\left(t\right)\cap {\mathcal{D}}_1\left(t\right)$. It can be verified that, for all $t\ge 0$, both ${\mathcal{D}}_0\left(t\right)$ and ${\mathcal{D}}_1\left(t\right)$ are convex subsets of ${\mathcal{P}}_{{\mathcal{X}}_1}\times \cdots \times {\mathcal{P}}_{{\mathcal{X}}_K}$, and thus $\mathcal{D}\left(t\right)$ is also convex. In addition, we have the following lemma.

Lemma 2.

For $E^{*}$ as defined in (10), we have $\mathcal{D}\left(t\right)=⌀$ for all $t\in [0,E^{*}]$ and $\mathcal{D}\left(t\right)\ne ⌀$ for all $t>E^{*}$. Additionally, a unique $({\tilde{R}}_{X_1},\cdots ,{\tilde{R}}_{X_K})\in {\mathcal{P}}_{{\mathcal{X}}_1}\times \cdots \times {\mathcal{P}}_{{\mathcal{X}}_K}$ exists such that

$$\begin{array}{c}\hfill D_0^{*}({\tilde{R}}_{X_1},\cdots ,{\tilde{R}}_{X_K})=D_1^{*}({\tilde{R}}_{X_1},\cdots ,{\tilde{R}}_{X_K})=E^{*}.\end{array}$$

(14)

Proof. 

See Appendix A. □

Based on Lemma 2, it follows from the separating hyperplane theorem (see, e.g., Section 2.5.1 of [26]) that functions $(f_1^{*},\cdots ,f_K^{*})$, where $f_k^{*}:{\mathcal{X}}_k\to \mathbb{R}$, $k=1,\cdots ,K$ exist, such that for all $(R_{X_1},\cdots ,R_{X_K})\in {\mathcal{D}}_0\left(E^{*}\right)$,

$$\begin{array}{c}\hfill \sum _{i=1}^K\sum _{x_i\in {\mathcal{X}}_i}{R}_{X_i}\left(x_i\right)f_i^{*}\left(x_i\right)=\sum _{i=1}^K{\mathbb{E}}_{R_{X_i}}\left[f_i^{*}\left(X_i\right)\right]\le 0,\end{array}$$

(15)

and for all $(R_{X_1},\cdots ,R_{X_K})\in {\mathcal{D}}_1\left(E^{*}\right)$,

$$\begin{array}{c}\hfill \sum _{i=1}^K{\mathbb{E}}_{R_{X_i}}\left[f_i^{*}\left(X_i\right)\right]\ge 0.\end{array}$$

(16)

Furthermore, we denote

$$\begin{array}{c}\hfill h^{*}\left(x^K\right)\triangleq \sum _{i=1}^K{f}_i^{*}\left(x_i\right),\end{array}$$

(17)

and then we have the following proposition. Given $P_Z\in {\mathcal{P}}_{\mathcal{Z}}$ and $\mathcal{S}\subset {\mathcal{P}}_{\mathcal{Z}}$, we adopt the notation [27,28] $D(\mathcal{S}\parallel P_Z)\triangleq {inf}_{Q_Z\in \mathcal{S}}D(Q_Z\parallel P_Z)$, where ${\mathcal{P}}_{\mathcal{Z}}$ denotes the set of all distributions supported on $\mathcal{Z}$.

Proposition 1.

The optimal exponent $E^{*}$ as defined in (10) satisfies

$$\begin{array}{c}\hfill E^{*}=D\left({\mathcal{S}}_{\mathcal{X}}^{\left(0\right)}\left(h^{*}\right)\parallel P_{X^K}^{\left(1\right)}\right)=D\left({\mathcal{S}}_{\mathcal{X}}^{\left(1\right)}\left(h^{*}\right)\parallel P_{X^K}^{\left(0\right)}\right).\end{array}$$

(18)

Proof. 

See Appendix B. □

Consequently, we establish the optimality of $E^{*}$ and provide the corresponding decision rule.

Theorem 1.

Let $f_1^{*},\cdots ,f_K^{*}$ denote the features as defined in (15) and (16). The optimal error exponent of (4) is given by

$$\begin{array}{c}\hfill \underset{n\to \infty }{lim}-\frac{1}{n}log{\mathbb{P}}_n(\widehat{\mathsf{H}}\ne \mathsf{H})=E^{*},\end{array}$$

(19)

where $E^{*}$ is defined in (10). In addition, the corresponding decision rule $\widehat{\mathsf{H}}$ is

$$\begin{array}{c}\hfill \sum _{k=1}^K{\mathbb{E}}_{{\widehat{P}}_{X_k}}\left[f_k^{*}\left(X_k\right)\right]\underset{\widehat{\mathsf{H}}={\mathsf{H}}_0}{\overset{\widehat{\mathsf{H}}={\mathsf{H}}_1}{\gtrless }}0.\end{array}$$

(20)

Proof. 

See Appendix C. □

4.2. General Geometric Structure

The geometry associated with Proposition 1 and Theorem 1 is depicted in Figure 3. In this figure, each point represents a distribution in ${\mathcal{P}}_{\mathcal{X}}$, and the decision boundary (20) corresponds to the linear family ${\mathcal{L}}_{\mathcal{X}}\left(h^{*}\right)$ defined as in (13). In addition, from Corollary 3.1 of [27], ${\lambda }_0,{\lambda }_1\in \mathbb{R}$ exist such that

$$\begin{array}{c}\hfill Q_{X^K}^{\left(i\right)}\triangleq {\tilde{P}}_{X^K}^{\left({\lambda }_i\right)}(·;h^{*},P_{X^K}^{\left(i\right)}),i=0,1,\end{array}$$

(21)

satisfy

$$\begin{array}{c}\hfill D\left({\mathcal{S}}_{\mathcal{X}}^{(1-i)}\left(h^{*}\right)\parallel P_{X^K}^{\left(i\right)}\right)=D\left(Q_{X^K}^{\left(i\right)}\parallel P_{X^K}^{\left(i\right)}\right),\end{array}$$

(22)

where ${\tilde{P}}_{X^K}^{\left({\lambda }_i\right)}(·;h^{*},P_{X^K}^{\left(i\right)}),i=0,1$ are as defined in (11). In this context, $Q_{X^K}^{\left(0\right)}$ and $Q_{X^K}^{\left(1\right)}$ in (21) are the I-projections [27] of $P_{X^K}^{\left(0\right)}$ and $P_{X^K}^{\left(1\right)}$ onto this linear family, respectively, which also induces the two exponential families ${\mathcal{E}}_{\mathcal{X}}\left(h^{*},P_{X^K}^{\left(0\right)}\right)$ and ${\mathcal{E}}_{\mathcal{X}}\left(h^{*},P_{X^K}^{\left(1\right)}\right)$ with $h^{*}$ as their common natural statistic. Additionally, all the points in ${\mathcal{D}}_0\left(E^{*}\right)$ and ${\mathcal{D}}_1\left(E^{*}\right)$ are divided by the the linear family ${\mathcal{L}}_{\mathcal{X}}\left(h^{*}\right)$.

4.3. Local Information Geometric Analysis

Although an explicit information geometry has been shown, we apply the local information geometric framework [13] to provide fundamental insights into this problem. Some useful notations and definitions in local information geometry are introduced as follows.

Definition 4

($ϵ$-neighborhood). Given a finite alphabet $\mathcal{Z}$, and letting $R_Z$ be a distribution supported on $\mathcal{Z}$ with all entries being positive, its ϵ-neighborhood ${\mathcal{N}}_{ϵ}^{\mathcal{Z}}\left(R_Z\right)$ is defined as

$$\begin{array}{c}\hfill {\mathcal{N}}_{ϵ}^{\mathcal{Z}}\left(R_Z\right)\triangleq \left\{P_Z\in {\mathcal{P}}_{\mathcal{Z}}:\sum _{z\in \mathcal{Z}}\frac{{(P_Z\left(z\right)-R_Z\left(z\right))}^2}{R_Z\left(z\right)}\le {ϵ}^2\right\}.\end{array}$$

Then, with $R_Z$ used as the reference distribution, each distribution $P_Z\in {\mathcal{P}}_{\mathcal{Z}}$ can be equivalently expressed as a vector $\varphi \in {\mathbb{R}}^{\left|\mathcal{Z}\right|}$ or a function $f:\mathcal{Z}\to \mathbb{R}$ with

$$\begin{array}{c}\hfill \varphi \left(z\right)\triangleq \frac{P_Z\left(z\right)-R_Z\left(z\right)}{\sqrt{R_Z\left(z\right)}},f\left(z\right)\triangleq \frac{\varphi \left(z\right)}{\sqrt{R_Z\left(z\right)}},\forall z\in \mathcal{Z},\end{array}$$

(23)

referred to as the information vector and feature function associated with $P_Z$, respectively. This provides a three way correspondence $P_Z\leftrightarrow \varphi \leftrightarrow f$, which will be useful in our derivations. Based on Definition 4, we introduce the local assumption that

$$\begin{array}{c}\hfill P_{X^k}^{\left(i\right)}\in {\mathcal{N}}_{ϵ}^{\mathcal{X}}\left(P_{X^k}\right),\mathrm{for}i=0,1,\end{array}$$

(24)

We use ${\psi }^{\left(i\right)}\leftrightarrow P_{X^K}^{\left(i\right)},i=0,1$ to represent the corresponding information vectors [cf. (23)]. For each $k=1,\cdots ,K$, and given feature $f_k:{\mathcal{X}}_k\to \mathbb{R}$, we define the corresponding information vector ${\varphi }_k\in {\mathbb{R}}^{|{\mathcal{X}}_k|}$, where $P_{X_k}\triangleq {\left[P_{X^K}\right]}_{X_k}$ is used as the reference distribution. Note that for $i=0,1$, the correspondence $B_k^{\mathrm{T}}{\psi }^{\left(i\right)}\leftrightarrow P_{X_k}^{\left(i\right)}$ exists, where $P_{X_k}^{\left(i\right)}\triangleq {\left[P_{X^K}^{\left(i\right)}\right]}_{X_k}$ represents the corresponding marginal distributions. Specifically, $B_k$ is an $\left|\mathcal{X}\right|\times |{\mathcal{X}}_k|$ dimensional matrix with entries [29]

$$\begin{array}{c}\hfill B_k(x^K,{\widehat{x}}_k)\triangleq \sqrt{\frac{P_{X^K}\left(x^K\right)}{P_{X_k}\left({\widehat{x}}_k\right)}}{\delta }_{x_k{\widehat{x}}_k},\end{array}$$

(25)

where ${\delta }_{x_k{\widehat{x}}_k}$ represents the Kronecker delta.

Moreover, the feature $f_k$ defined on ${\mathcal{X}}_k$, when considered as a mapping from $\mathcal{X}$ to $\mathbb{R}$, corresponds to the information vector $B_k{\varphi }_k$ in ${\mathbb{R}}^{\left|\mathcal{X}\right|}$. Leveraging this correspondence, we can further establish the information vector for $h\left(x^K\right)={\sum }_{k=1}^K{f}_k\left(x_k\right)$ as

$$\begin{array}{c}\hfill \sum _{i=1}^K{B}_i{\varphi }_i=B_0{\varphi }_0\in {\mathbb{R}}^{\left|\mathcal{X}\right|},\end{array}$$

(26)

where we have defined

$$\begin{array}{c}\hfill B_0\triangleq \left[\begin{array}{ccc}{B}_1& \cdots & B_K\end{array}\right]\mathrm{and}{\varphi }_0\triangleq \left[\begin{array}{c}{\varphi }_1\\ ⋮\\ {\varphi }_K\end{array}\right],\end{array}$$

(27)

and where for each $k=1,\cdots ,K$, ${\varphi }_k\in {\mathbb{R}}^{|{\mathcal{X}}_k|}$ denotes the information vector corresponding to $f_k$.

Additionally, given a matrix $A\in {\mathbb{R}}^{m_1\times m_2}$, we use $A^{†}$ to denote its Moore–Penrose inverse [30], and we define the associated column space $\mathcal{R}\left(A\right)\triangleq \{Ax:x\in {\mathbb{R}}^{m_2}\}$ and projection matrix ${\Pi }_A\triangleq A{A}^{†}$. Then, we can establish the local counterpart of $E^{*}$ in Theorem 1 as follows.

Theorem 2.

Under the local assumption (24), let ${\psi }^{\left(i\right)}\leftrightarrow P_{X^K}^{\left(i\right)}$, $i=0,1$ denote the corresponding information vectors. Then, for $h^{*}$ as defined in (17), we have the correspondence $h^{*}\leftrightarrow B_0{\varphi }_0^{*}$, where

$$\begin{array}{c}\hfill {\varphi }_0^{*}\triangleq B_0^{†}\left({\psi }^{\left(1\right)}-{\psi }^{\left(0\right)}\right),\end{array}$$

(28)

and where $B_0$ is defined in (27). In addition, the optimal exponent $E^{*}$ in (10) can be expressed as

$$\begin{array}{c}\hfill E^{*}=\frac{1}{8}{∥B_0{\varphi }_0^{*}∥}^2+o\left({ϵ}^2\right).\end{array}$$

(29)

Proof. 

See Appendix D. □

Note that from Theorem 2, we have

$$\begin{array}{c}\hfill h^{*}\leftrightarrow B_0{B}_0^{†}({\psi }^{\left(1\right)}-{\psi }^{\left(0\right)})={\Pi }_{B_0}({\psi }^{\left(1\right)}-{\psi }^{\left(0\right)}),\end{array}$$

where ${\Pi }_{B_0}$ is the projection matrix associated with the subspace $\mathcal{R}\left(B_0\right)$. The optimal feature $B_0{\varphi }_0^{*}$ in (26) corresponds to the projection of the sufficient statistic $f_{\mathrm{LLR}}\leftrightarrow ({\psi }^{\left(1\right)}-{\psi }^{\left(0\right)})$ onto the function space that encompasses all possible h’s satisfying the form $h\left(x^K\right)={\sum }_{k=1}^K{f}_k\left(x_k\right)$. In other words, $B_0{\varphi }_0^{*}$ represents the best approximation of $f_{\mathrm{LLR}}$ within the function space of interest, which leads to the optimal decision error exponent $E^{*}$ as shown in (29).

Moreover, from (26), this optimal feature can be decomposed to K components in subspaces $\mathcal{R}\left(B_k\right)$, for $k=1,\cdots ,K$,

$$\begin{array}{c}\hfill B_0{\varphi }_0^{*}=\sum _{k=1}^K{B}_k{\varphi }_k^{*},\end{array}$$

(30)

where ${\varphi }_0^{*}$ is stacked by ${\varphi }_k^{*}\in {\mathbb{R}}^{|{\mathcal{X}}_k|}$, $k=1,\cdots ,K$, as in (27). This decomposition structure can be depicted as Figure 4 for the case $K=2$.

Remark 1.

The vectors $B_i{\varphi }_k^{*}$ are not simply the orthogonal projections of $B_0{\varphi }_0^{*}$ onto the subspaces $\mathcal{R}\left(B_k\right)$ since these subspaces, for $k=1,\dots ,K$, are not mutually orthogonal. Therefore, the decomposition of $B_0{\varphi }_0^{*}$ will depend on the Gram matrix [30] of the subspaces $\mathcal{R}\left(B_k\right)$, as illustrated in Figure 4. Furthermore, it is noteworthy that the orthogonal projection of $B_0{\varphi }_0^{*}$ onto the subspaces $\mathcal{R}\left(B_k\right)$ can be interpreted as characterizing the optimal error exponent of the binary hypothesis testing problem solely with the observations of ${\mathsf{X}}_k$ [12]. When the subspaces $\mathcal{R}\left(B_k\right)$ are orthogonal to each other, the optimal inference approach is straightforward, involving the extraction of the optimal information from each node by orthogonal projection. However, when the subspaces $\mathcal{R}\left(B_k\right)$ are not orthogonal, different nodes may share various forms of common information. Our result fundamentally demonstrates how to handle this shared information and extract the optimal features through the decomposition of the information vector over non-orthogonal subspaces. This insight provides a novel approach to address the challenges posed by the non-orthogonal subspaces and reveals how to extract the most informative features effectively, ultimately leading to improved performance in the distributed hypothesis testing problem.

5. Type-Based Hypothesis Testing over AWGN Channels

This section presents the optimal error exponent of the type-based hypothesis testing problem over AWGN channels, along with the corresponding coding strategy. To begin, we introduce several notations that will help in the presentation of the results.

Definition 5.

Let $\left[K\right]\triangleq \{1,2,\cdots ,K\}$, and for subset $\omega \subseteq \left[K\right]$, $i=0,1$, we define

$$\begin{array}{c}\hfill D_i^{\omega }\left({\left\{R_{X_k}\right\}}_{k\in \omega }\right)\triangleq \underset{Q_{X^K}\in {\mathcal{S}}_{\omega }}{min}D(Q_{X^K}\parallel P_{X^K}^{\left(i\right)}),\end{array}$$

(31)

where

$$\begin{array}{cc}\hfill {\mathcal{S}}_{\omega }\triangleq & \left\{Q_{X^K}:{\left[Q_{X^K}\right]}_{X_k}=R_{X_k},k\in \omega \right\}.\hfill \end{array}$$

It would be easy to find that $D_i^{\left[K\right]}(·)=D_i^{*}(·)$, and $D_i^{*}(·)$ is as defined in (9). Moreover, we define the following error exponent with respect to $\omega \subseteq \left[K\right]$.

$$\begin{array}{cc}\hfill E_{\omega }\triangleq \underset{{\left\{R_{X_k}\right\}}_{k\in \omega },{\left\{{\theta }_k\right\}}_{k\in \left[K\right]\setminus \omega }}{min}max\{& D_0^{\omega }\left({\left\{R_{X_k}\right\}}_{k\in \omega }\right)+\sum _{k\in \left[K\right]\setminus \omega }\frac{{({\theta }_k-\sqrt{p_k})}^2}{2\mu {\sigma }_k^2},\hfill \\ \hfill & D_1^{\omega }\left({\left\{R_{X_k}\right\}}_{k\in \omega }\right)+\sum _{k\in \left[K\right]\setminus \omega }\frac{{({\theta }_k+\sqrt{p_k})}^2}{2\mu {\sigma }_k^2}\},\hfill \end{array}$$

(32)

where we have used $A\setminus B$ to represent the relative complement of set B in set A, and where $\mu$ is as defined in (8). We can also find $E_{\left[K\right]}=E^{*}$ and $E^{*}$ is as defined in (10). Finally, we define the quantity $E^{\odot }$, which will be shown as the optimal error exponent

$$\begin{array}{c}\hfill E^{\odot }\triangleq \underset{\omega \in \Im \left(\right[K\left]\right)}{min}{E}_{\omega },\end{array}$$

(33)

where $\Im \left(\right[K\left]\right)$ denotes the power set of $\left[K\right]$.

Theorem 3.

The optimal error exponent of (4) is given by

$$\begin{array}{c}\hfill \underset{n\to \infty }{lim}-\frac{1}{n}log{\mathbb{P}}_n(\widehat{\mathsf{H}}\ne \mathsf{H})=E^{\odot }.\end{array}$$

(34)

In the following, we prove Theorem 3 by both the achievability and converse result.

5.1. The Coding Strategy for Distributed Nodes

First, we define the different regimes of empirical distributions, for each $k=1,\cdots ,K$ and for some $\gamma \in (0,1)$. Basically, the specific choice of $\gamma$ does not effect the achievable error exponent as long as $\gamma \in (0,1)$. It helps conduct the decode-and-forward and amplify-and-forward coding strategies as introduced in Section 1.

• Decode-and-forward regime:

$$\begin{array}{cc}\hfill & {\mathcal{M}}_k^{\left(0\right)}\triangleq \left\{R_{X_k}:D(R_{X_k}\parallel P_{X_k}^{\left(0\right)})<n^{-\gamma }\right\},\hfill \\ \hfill & {\mathcal{M}}_k^{\left(1\right)}\triangleq \left\{R_{X_k}:D(R_{X_k}\parallel P_{X_k}^{\left(1\right)})<n^{-\gamma }\right\}.\hfill \end{array}$$

• Amplify-and-forward regime:

$$\begin{array}{c}\hfill {\mathcal{M}}_k^{\mathsf{c}}\triangleq \left\{R_{X_k}:min\left\{D(R_{X_k}\parallel P_{X_k}^{\left(0\right)}),D(R_{X_k}\parallel P_{X_k}^{\left(1\right)})\right\}\ge n^{-\gamma }\right\}.\end{array}$$

(35)

Note that for each $k=1,\cdots ,K$, the probability of the empirical distribution ${\widehat{P}}_{X_k}$ in ${\mathcal{M}}_k^{\mathsf{c}}$ is $exp\left(-n^{1-\gamma }\right)$. Consequently, in the amplify-and-forward regime, we can transmit such empirical distributions with exponentially large power by Pulse Amplitude Modulation (PAM) while still satisfying the power constraint. Specifically, let ${\mathcal{P}}_{{\mathcal{X}}_k}^{\left(n\right)}$ be the set of all possible empirical distributions of $X_k$ with n samples, and denote ${\eta }_k\triangleq |{\mathcal{P}}_{{\mathcal{X}}_k}^{\left(n\right)}\cap {\mathcal{M}}_k^{\mathsf{c}}|$. We define the bijective function ${\xi }_k:{\mathcal{P}}_{{\mathcal{X}}_k}^{\left(n\right)}\cap {\mathcal{M}}_k^{\mathsf{c}}\mapsto \{1,\dots ,{\eta }_k\}$ as the indices of empirical distributions. Then, according to the observed empirical distribution, the encoder of node k ($k=1,\cdots ,K$) is designed to transmit the signal

$$\begin{array}{c}\hfill {\mathit{Q}}_k\left({\widehat{P}}_{X_k}\right)\triangleq {\xi }_k\left({\widehat{P}}_X\right)·exp\left(n^{\frac{1-\gamma }{2}}\right).\end{array}$$

(36)

Furthermore, if the empirical distributions are in the decode-and-forward regimes, we initially detect the true hypothesis and then transmit the bit using Binary Phase Shift Keying (BPSK) with the appropriate power. By employing these strategies, the achievability result can be obtained through repeated transmissions from all the distributed nodes. In other words, the resulting encoder for node k is defined as follows:

$$\begin{array}{c}\hfill {\mathit{g}}_k^{*}=\left[g_k^{*},\cdots ,g_k^{*}\right],k=1,\cdots ,K,\end{array}$$

(37)

where

$$\begin{array}{c}\hfill g_k^{*}\left({\widehat{P}}_{X_k}\right)\triangleq \left\{\begin{array}{cc}\sqrt{p_k-\delta (n,\gamma )},\hfill & \mathrm{if}{\widehat{P}}_{X_k}\in {\mathcal{M}}_k^{\left(0\right)}\hfill \\ -\sqrt{p_k-\delta (n,\gamma )},\hfill & \mathrm{if}{\widehat{P}}_{X_k}\in {\mathcal{M}}_k^{\left(1\right)}\hfill \\ {\mathit{Q}}_k\left({\widehat{P}}_{X_k}\right),\hfill & \mathrm{if}{\widehat{P}}_{X_k}\in {\mathcal{M}}_k^{\mathsf{c}}\hfill \end{array}\right.,\end{array}$$

(38)

and where

$$\begin{array}{c}\hfill \delta (n,\gamma )\triangleq \underset{k\in \left[K\right]}{max}\frac{{\mathbb{P}}_n({\widehat{P}}_{X_k}\in {\mathcal{M}}_k^{\mathsf{c}})}{{\mathbb{P}}_n({\widehat{P}}_{X_k}\notin {\mathcal{M}}_k^{\mathsf{c}})}·{(n+1)}^{2|{\mathcal{X}}_k|}·exp\left(2n^{\frac{1-\gamma }{2}}\right).\end{array}$$

(39)

Proposition 2.

The encoders as defined in (38) satisfy the power constraint (6), and

$$\begin{array}{c}\hfill \underset{n\to \infty }{lim}\delta (n,\gamma )=0.\end{array}$$

(40)

Proof. 

See Appendix E. □

5.2. Decision Rule and Achievable Error Exponent

After the decision center receives the output signals ${\mathit{g}}_1^{*}\left({\widehat{P}}_{X_1}\right)+{\mathit{Z}}_1,\cdots ,{\mathit{g}}_K^{*}\left({\widehat{P}}_{X_K}\right)+{\mathit{Z}}_K$, we then compute

$$\begin{array}{c}\hfill {\theta }_k\triangleq \frac{1}{m}\sum _{i=1}^m{[{\mathit{g}}_k^{*}\left({\widehat{P}}_{X_k}\right)+{\mathit{Z}}_k]}_i,k=1,\cdots ,K,\end{array}$$

where ${[·]}_i$ denotes the i-th entry of a given vector. Then, we conduct the log-likelihood ratio test (LLRT) to detect the true hypothesis:

$$\begin{array}{c}\hfill log\frac{{\mathbb{P}}_n({\theta }_1,\cdots ,{\theta }_K|\mathsf{H}={\mathsf{H}}_0)}{{\mathbb{P}}_n({\theta }_1,\cdots ,{\theta }_K|\mathsf{H}={\mathsf{H}}_1)}\underset{\widehat{\mathsf{H}}={\mathsf{H}}_1}{\overset{\widehat{\mathsf{H}}={\mathsf{H}}_0}{⋛}}0.\end{array}$$

(41)

Note that exponentially large power is allocated for the empirical distributions in the amplify-and-forward regime (cf. (35), (36)); the decision center can correctly detect the coding regime of the nodes with super-exponentially high probability, i.e., for $k=1,\cdots ,K$,

$$\begin{array}{cc}\hfill & \underset{n\to \infty }{lim}-\frac{1}{n}log{\mathbb{P}}_n\left({\widehat{P}}_{X_k}\in {\mathcal{M}}_k^{\mathsf{c}}|{\theta }_k\le exp\left(n^{\frac{1-\gamma }{4}}\right)\right)=\infty ,\hfill \\ \hfill & \underset{n\to \infty }{lim}-\frac{1}{n}log{\mathbb{P}}_n\left({\widehat{P}}_{X_k}\notin {\mathcal{M}}_k^{\mathsf{c}}|{\theta }_k>exp\left(n^{\frac{1-\gamma }{4}}\right)\right)=\infty .\hfill \end{array}$$

(42)

Therefore, we can assume that the decision center knows the coding regime of the nodes and define the following regime of the received signals with respect to subset $\omega \subseteq \left[K\right]$.

$$\begin{array}{cc}\hfill & {\Theta }_{\omega }\triangleq \left\{({\theta }_1,\cdots ,{\theta }_K):{\theta }_k>exp\left(n^{\frac{1-\gamma }{4}}\right),\forall k\in \omega ,\mathrm{and}{\theta }_{k^{\prime }}\le exp\left(n^{\frac{1-\gamma }{4}}\right),\forall k^{\prime }\in \left[K\right]\setminus \omega \right\},\hfill \end{array}$$

for all $\omega \in \Im \left(\right[K\left]\right)$. When the received signals $({\theta }_1,\cdots ,{\theta }_K)\in {\Theta }_{\omega }$, the decision center can recover the empirical distributions ${\widehat{P}}_{X_k}$ ($k\in \omega$) from the received signals ${\theta }_k$ by the decoder:

$$\begin{array}{cc}\hfill & {\mathit{Q}}_k^{-1}\left({\theta }_k\right)\triangleq {\xi }_k^{-1}\left(⌊{\theta }_k/exp\left(n^{\frac{1-\gamma }{2}}\right)+0.5⌋\right),\hfill \end{array}$$

(43)

where $\lfloor ·\rfloor$ denotes the floor function [31]. The following result shows that decoding error of (43) can be neglected.

Proposition 3.

For all ${\widehat{P}}_{X_k}\in {\mathcal{P}}_{{\mathcal{X}}_k}^{\left(n\right)}\cup {\mathcal{M}}_k^{\mathsf{c}}$, $k=1,\cdots ,K,$

$$\begin{array}{c}\hfill \underset{n\to \infty }{lim}-\frac{1}{n}log\mathbb{P}({\mathit{Q}}_k^{-1}\left({\theta }_k\right)\ne {\widehat{P}}_{X_k})=\infty .\end{array}$$

(44)

Proof. 

See Appendix F. □

In the following, we denote $p_k^{\prime }\triangleq p_k-\delta$, for $k=1,\cdots ,K$ and discuss the decision error exponent when the received signals are in ${\Theta }_{\omega }$. For $k\in \omega$, the empirical distribution ${\widehat{P}}_{X_k}$ can be recovered by (43), and for $k\in \left[K\right]\setminus \omega$, node k detects the hypothesis according to the observed empirical distribution and transmits the detected bit by BPSK (cf. (38)) through the AWGN channel. Then, the decision center detects the true hypothesis from the received signals by LLRT (41), which can be reduced to

$$\begin{array}{c}\hfill {\tilde{E}}_0^{\omega }({\theta }_1,\cdots ,{\theta }_K)\underset{\widehat{\mathsf{H}}={\mathsf{H}}_0}{\overset{\widehat{\mathsf{H}}={\mathsf{H}}_1}{⋛}}{\tilde{E}}_1^{\omega }({\theta }_1,\cdots ,{\theta }_K),\end{array}$$

(45)

where for $i=0,1$,

$$\begin{array}{cc}\hfill & {\tilde{E}}_i^{\omega }({\theta }_1,\cdots ,{\theta }_K)\hfill \\ \hfill & \triangleq \underset{\overline{\omega }\in \Im (\left[K\right]\setminus \omega )}{min}{D}_i^{*}({\overline{P}}_{X_1},\cdots ,{\overline{P}}_{X_K})+\sum _{k\in \overline{\omega }}\frac{{({\theta }_k-\sqrt{p_k^{\prime }})}^2}{2\mu {\sigma }_k^2}+\sum _{k^{\prime }\in \left[K\right]\setminus (\omega \cup \overline{\omega })}\frac{{({\theta }_{k^{\prime }}+\sqrt{p_{k^{\prime }}^{\prime }})}^2}{2\mu {\sigma }_{k^{\prime }}^2},\hfill \end{array}$$

where $\Im \left(\right[K]\setminus \omega )$ denotes the power set of $\left[K\right]\setminus \omega$, and where for $k=1,\cdots ,K$,

$$\begin{array}{c}\hfill {\overline{P}}_{X_k}\triangleq \left\{\begin{array}{cc}{\mathit{Q}}_k^{-1}\left({\theta }_k\right),\hfill & \mathrm{if}k\in \omega \hfill \\ P_{X_k}^{\left(0\right)},\hfill & \mathrm{if}k\in \overline{\omega }\hfill \\ P_{X_k}^{\left(1\right)},\hfill & \mathrm{if}k\in \left[K\right]\setminus (\omega \cup \overline{\omega })\hfill \end{array}\right..\end{array}$$

(46)

Consequently, the decision error exponent is characterized by the following proposition.

Proposition 4.

For any $ϵ>0$ and $\omega \in \Im \left(\right[K\left]\right)$, the decision error exponent by the decision rule (45) satisfies

$$\begin{array}{c}\hfill \underset{n\to \infty }{lim}-\frac{1}{n}log{\mathbb{P}}_n\left(\widehat{\mathsf{H}}\ne \mathsf{H},({\theta }_1,\cdots ,{\theta }_K)\in {\Theta }_{\omega }\right)\ge E^{\odot }-ϵ,\end{array}$$

(47)

where $E^{\odot }$ is as defined in (33).

Proof. 

See Appendix G. □

Noticing that the overall decision error probability is

$$\begin{array}{c}\hfill {\mathbb{P}}_n(\widehat{\mathsf{H}}\ne \mathsf{H})=\sum _{\omega \in \Im \left(\right[K\left]\right)}\mathbb{P}(\widehat{\mathsf{H}}\ne \mathsf{H},({\theta }_1,\cdots ,{\theta }_K)\in {\Theta }_{\omega }),\end{array}$$

the following proposition establishes the achievable error exponent by the coding strategy (38).

Proposition 5.

By using the encoders ${\mathit{g}}_1^{*},\cdots ,{\mathit{g}}_K^{*}$ as defined in (38), and the decision rules $\widehat{\mathsf{H}}$ from (41), the achievable error exponent is given by $E^{\odot }$, i.e.,

$$\begin{array}{c}\hfill \underset{n\to \infty }{lim}-\frac{1}{n}log{\mathbb{P}}_n(\widehat{\mathsf{H}}\ne \mathsf{H})\ge E^{\odot },\end{array}$$

(48)

where $E^{\odot }$ is as defined in (33).

5.3. The Converse Result

In this section, we show that $E^{\odot }$ is indeed an upper bound of (4), which establishes Theorem 3. Our main technique is to apply a genie-aided approach, which provides different kinds of additional information to both nodes and computes the corresponding error exponents under additional information. As depicted in Figure 5, given index set $\omega \in \Im \left(\right[K\left]\right)$, suppose that for all $k\in \omega$, node k can know and cancel the channel noise in advance; then, the channel is noiseless, and the decision center can perfectly receive the empirical distribution ${\widehat{P}}_{X_k}$. On the other hand, suppose that for all $k^{\prime }\in \left[K\right]\setminus \omega$, we can leverage the true hypothesis $\mathsf{H}$ to node $k^{\prime }$; then, with such additional information, we can establish the following upper bound of (4) (cf. (33)).

Proposition 6.

Given index set $\omega \in \Im \left(\right[K\left]\right)$, suppose that for all $k\in \omega$, the decision center can obtain ${\widehat{P}}_{X_k}$ perfectly. Additionally, for all $k^{\prime }\in \left[K\right]\setminus \omega$, node $k^{\prime }$ can obtain the true hypothesis $\mathsf{H}$. The resulting optimal decision error exponent is

$$\begin{array}{c}\hfill \underset{n\to \infty }{lim}-\frac{1}{n}log{\mathbb{P}}_n(\widehat{\mathsf{H}}\ne \mathsf{H})=E_{\omega },\end{array}$$

(49)

where $E_{\omega }$ is as defined in (32).

Proof. 

See Appendix H. □

Notice that Proposition 6 is verified for all $\omega \in \Im \left(\right[K\left]\right)$, and we cannot obtain a better performance than Proposition 6 for the DHT over AWGN channels without the additional information. We then conclude the following error exponent upper bound.

Proposition 7.

For all possible encodes ${\mathit{g}}_1,\cdots ,{\mathit{g}}_K$ under the power constraint (6), the corresponding error exponent with respect to the LLRT decision rule satisfies

$$\begin{array}{c}\hfill \underset{n\to \infty }{lim}-\frac{1}{n}log{\mathbb{P}}_n(\widehat{\mathsf{H}}\ne \mathsf{H})\le E^{\odot },\end{array}$$

(50)

where $E^{\odot }$ is as defined in (33).

Finally, by combining Propositions 5 and 7, Theorem 3 is proved.

Remark 2

(Local-geometric interpretation). Note that the expression of the optimal error exponent $E^{\odot }$ as defined in (33) is quite intricate, which could limit our understanding. To simplify the analysis, we introduce the local geometry assumption as given in (24). In Appendix I, we demonstrate that the error exponent corresponds to a more manageable expression

$$\begin{array}{c}\hfill E^{\odot }=\underset{\omega \in \Im \left(\right[K\left]\right)}{min}\frac{1}{8}{∥B_{\omega }{B}_{\omega }^{†}\left({\psi }_{\omega }^{\left(1\right)}-{\psi }_{\omega }^{\left(0\right)}\right)∥}^2+\sum _{k\in \left[K\right]\setminus \omega }\frac{p_k}{2\mu {\sigma }_k^2}+o\left({ϵ}^2\right),\end{array}$$

(51)

where for $\omega =\{i_1,\cdots ,i_{\left|\omega \right|}\}$, we have defined

$$\begin{array}{c}\hfill B_{\omega }\triangleq \left[\begin{array}{ccc}{B}_{i_1}& \cdots & B_{i_{\left|\omega \right|}}\end{array}\right],\end{array}$$

(52)

and ${\psi }_{\omega }^{\left(i\right)}\leftrightarrow {\left[P_{X^K}^{\left(i\right)}\right]}_{X_{i_1}\cdots X_{i_{\left|\omega \right|}}},i=0,1$. Given $\omega \in \Im \left(\right[K\left]\right)$, the first term in (51) represents the optimal error exponent (cf. (29)) when the decision center can access the empirical distributions ${\widehat{P}}_{X_k}$, $k\in \omega$. The second term corresponds to the optimal error exponent when node k, $k\in \left[K\right]\setminus \omega$ can know the true hypothesis $\mathsf{H}$ and transmit the bit using BPSK modulation. The total error exponent is the sum of these two parts, and $E^{\odot }$ aims to determine the minimum sum among all possible splits of the index set $\left[K\right]$. In other words, $E^{\odot }$ finds the optimal trade-off between accessing empirical distributions at the decision center and having individual nodes transmit bits with BPSK modulation.

6. Discussion

This paper discusses the DHT problem over two communication models.The first is the noiseless channel, which is mostly considered in current distributed learning and federated learning systems [9,11]. For the noiseless channels, we show that by using one-dimensional statistics from different nodes, it is possible to achieve the same error exponent when the decision center has knowledge of the corresponding empirical distributions. This result is significant as it simplifies the coding process at distributed nodes, allowing them to transmit only the necessary statistics rather than the entire empirical distribution, which provides a practical implementation of the result in [5]. This finding proves the rationality of transmitting statistics as the most widely-used strategy in distributed learning and federated learning [11].

For the AWGN channels, this paper introduces a novel coding strategy, which cleverly combines decode-and-forward and amplify-and-forward techniques. The underlying concept of this coding strategy is based on the observation that the probability of the empirical distribution deviating significantly from the true marginal distribution diminishes exponentially. Consequently, by employing sufficiently large power, we can transmit the empirical distribution almost perfectly to the decision center while satisfying the averaged power constraint. When the prior distributions are not $1/2$, the strategy still work for the optimal error exponent, and the only difference is to adjust the BPSK points for two hypotheses according to the power constaint. The demonstrated optimality of the achieved decision error exponent further indicates that the proposed coding strategy is highly effective and successfully approaches the theoretical limit within the given constraints of the problem.

7. Conclusions

This paper focuses on investigating DHT problems over both noiseless channels and AWGN channels, where the distributed nodes are constrained to encoding the received empirical distributions, driven by practical computational considerations. In the first problem, we demonstrate that utilizing one-dimensional statistics of distributed nodes and simply summing them up as the decision rule can lead to the optimal error exponent. For the second problem, we propose a coding strategy that combines decode-and-forward and amplify-and-forward techniques. We further introduce a genie-aided approach to establish the optimality of the achieved decision error exponent. Overall, our findings offer valuable insights into coding techniques for distributed nodes, and the established strategies can be extended to more general scenarios, broadening the applicability of DHT in diverse settings.