On Continuous-Time Gaussian Channels ^†

Abstract

A continuous-time white Gaussian channel can be formulated using a white Gaussian noise, and a conventional way for examining such a channel is the sampling approach based on the Shannon–Nyquist sampling theorem, where the original continuous-time channel is converted to an equivalent discrete-time channel, to which a great variety of established tools and methodology can be applied. However, one of the key issues of this scheme is that continuous-time feedback and memory cannot be incorporated into the channel model. It turns out that this issue can be circumvented by considering the Brownian motion formulation of a continuous-time white Gaussian channel. Nevertheless, as opposed to the white Gaussian noise formulation, a link that establishes the information-theoretic connection between a continuous-time channel under the Brownian motion formulation and its discrete-time counterparts has long been missing. This paper is to fill this gap by establishing causality-preserving connections between continuous-time Gaussian feedback/memory channels and their associated discrete-time versions in the forms of sampling and approximation theorems, which we believe will play important roles in the long run for further developing continuous-time information theory. As an immediate application of the approximation theorem, we propose the so-called approximation approach to examine continuous-time white Gaussian channels in the point-to-point or multi-user setting. It turns out that the approximation approach, complemented by relevant tools from stochastic calculus, can enhance our understanding of continuous-time Gaussian channels in terms of giving alternative and strengthened interpretation to some long-held folklore, recovering “long-known” results from new perspectives, and rigorously establishing new results predicted by the intuition that the approximation approach carries. More specifically, using the approximation approach complemented by relevant tools from stochastic calculus, we first derive the capacity regions of continuous-time white Gaussian multiple access channels and broadcast channels, and we then analyze how feedback affects their capacity regions: feedback will increase the capacity regions of some continuous-time white Gaussian broadcast channels and interference channels, while it will not increase capacity regions of continuous-time white Gaussian multiple access channels.

1. Introduction

Continuous-time Gaussian channels were considered at the very inception of information theory. In his celebrated paper [1] birthing information theory, Shannon studied the following point-to-point continuous-time white Gaussian channels:

$$Y(t)=X(t)+Z(t),t\in \mathbb{R},$$

(1)

where $X(t)$ is the channel input with average power limit P, $Z(t)$ is the white Gaussian noise with flat power spectral density 1 and $Y(t)$ are the channel output. Shannon actually only considered the case that the channel has bandwidth limit $\omega$, namely, the channel input X and the noise Z, and therefore the output Y all have bandwidth limit $\omega$ (alternatively, as in ($9.54$) of [2], this can be interpreted as the original channel (1) concatenated with an ideal bandpass filter with bandwidth limit $\omega$). Using the celebrated Shannon–Nyquist sampling theorem [3,4], the continuous-time channel (1) can be equivalently represented by a parallel Gaussian channel:

$$Y_n^{(\omega )}=X_n^{(\omega )}+Z_n^{(\omega )},n\in \mathbb{Z},$$

(2)

where the noise process $\left\{Z_n^{(\omega )}\right\}$ is i.i.d. with variance 1 [2]. Regarding the “space” index n as time, the above parallel channel can be interpreted as a discrete-time Gaussian channel associated with the continuous-time channel (1). It is well known from the theory of discrete-time Gaussian channels that the capacity of the channel (2) can be computed as

$$C^{(\omega )}=\omega \log \left(1+\frac{P}{2\omega }\right).$$

(3)

Then, the capacity C of the channel (1) can be computed by taking the limit of the above expression as $\omega$ tends to infinity:

$$C=\underset{\omega \to \infty }{lim}{C}^{(\omega )}=P/2.$$

(4)

The sampling approach consisting of (1)–(4) as above, which serves as a link between the continuous-time channel (1) and the discrete-time channel (2), typifies a conventional way to examine continuous-time Gaussian channels: convert them into associated discrete-time Gaussian channels, for which we have ample ammunition at hands. Note that when P tends to 0, using the fact that when P is “close” to 0,

$$\omega \log \left(1+\frac{P}{2\omega }\right)\mathrm{is} “\mathrm{close}” \mathrm{to} \frac{P}{2},$$

one also reaches (4), which roughly explains the following long-held folklore within the information theory community:

a continuous-time infinite-bandwidth Gaussian channel without feedback or memory is “equivalent” to a discrete-time Gaussian channel without feedback or memory at low signal-to-noise ratio (SNR).

(A)

Moments of reflection, however, reveals that the sampling approach for the channel capacity (with bandwidth limit or not) is heuristic in nature: For one thing, a bandwidth-limited signal cannot be time-limited, which renders it infeasible to define the data transmission rate if assuming a channel has bandwidth limit. In this regard, rigorous treatments coping with this issue and other technicalities can be found in [5,6]; see also [7] for a relevant in-depth discussion. Another issue is that, even disregarding the above technical nuisance arising from the bandwidth limit assumption, the sampling approach only gives a lower bound for the capacity of (1): it shows that $P/2$ is achievable via a class of special coding schemes, but it is not clear that why transmission rate higher than $P/2$ cannot be achieved by other coding schemes. The capacity of (1) was rigorously studied in [8,9], and a complete proof establishing $P/2$ as its de facto capacity can be found in [10,11].

Alternatively, the continuous-time white Gaussian channel (1) can be examined [12] under the Brownian motion formulation:

$$Y(t)={\int }_0^{t}X(s)ds+B(t),$$

(5)

where, slightly abusing the notation, we still use $Y(t)$ to denote the output corresponding to the input $X(s)$, and $B(t)$ denotes the standard Brownian motion ($Z(t)$ can be viewed as a generalized derivative of $B(t)$); equivalently, the channel (5) can be seen as the original channel (1) concatenated with an integrator circuit. As opposed to white Gaussian noises, which only exist as generalized functions [13], Brownian motions are well-defined stochastic processes and have been extensively studied in probability theory. Here we remark that, via a routine orthonormal decomposition argument, both channels are equivalent to a parallel channel consisting of infinitely many Gaussian sub-channels [14].

An immediate and convenient consequence of such a formulation is that many notions in discrete time, including mutual information and typical sets, carry over to the continuous-time setting, which will rid us of the nuisances arising from the bandwidth limit assumption. Indeed, such a framework yields a fundamental formula for the mutual information of the channel (5) [15,16] and a clean and direct proof [16] that the capacity of (5) is $P/2$; moreover, as evidenced by numerous results collected in [12] and some recent representative work [17,18] on point-to-point Gaussian channels, the use of Brownian motions elevates the level of rigor of our treatment, and equip us with a wide range of established techniques and tools from stochastic calculus. Here we remark that Girsanov’s theorem, one of the most important theorems in stochastic calculus, lays the foundation of our rigourous treatment; for those who are interested in the technical details in our proofs, we refer to [12,19], where Girsanov’s theorem (and its numerous variants) and its wide range of applications in information theory is discussed in great details.

Furthermore, as elaborated in Remark 4, the Brownian motion formulation is also versatile enough to accommodate feedback and memory; in particular, the point-to-point continuous-time white Gaussian memory/feedback channel can be characterized by the following stochastic differential equation:

$$Y(t)={\int }_0^{t}g(s,W_0^s,Y_0^s)ds+B(t),t\in [0,T],$$

(6)

where g is a function from $[0,T]\times C[0,T]\times C[0,T]$ to $\mathbb{R}$. Note that (6) can be interpreted

(1)

either as a feedback channel, where $W_0^s\triangleq \{W(r):0\le r\le s\}$ can be rewritten as M, interpreted as the message to be transmitted through the channel, and $g(s)$ can be rewritten as $X(s)$, interpreted as the channel input, which depends on M and $Y_0^s$, the channel output up to time s that is fed back to the sender,

(2)

or as a memory channel, where $W_0^s$ can rewritten as $X_0^s$, interpreted as the channel input, g is “part” of the channel, and $Y(t)$, the channel output at time t, depends on $X_0^t$ and $Y_0^t$, the channel input and output up to time t that are present in the channel as memory, respectively.

Note that, strictly speaking, the third parameter of g in (6) should be $Y_0^{s-}$, which, however, can be equivalently replaced by $Y_0^s$ due to the continuity of sample paths of $\{Y(t)\}$. Note that, with the presence of feedback/memory, the existence and uniqueness of Y is in fact a tricky mathematical problem; however, we will in this paper simply assume that the input X is appropriately chosen such that Y uniquely exists. For more detailed discussion about the Brownian motion formulation of continuous-time Gaussian channels and preliminaries and known results thereof, we refer the reader to [12].

As opposed to the white Gaussian noise formulation, under the Brownian motion formulation, memory and feedback can be naturally translated to the discrete-time setting: the pathwise continuity of a Brownian motion allows the inheritance of temporal causality when the channel is sampled (see Section 2) or approximated (see Section 3). On the other hand, the white Gaussian noise formulation is facing inherent difficulty as far as inheriting temporal causality is concerned: in converting (1) to (2), while $X_n^{(w)}$ are obtained as “time” samples of $X(t)$, $Z_n^{(w)}$ are in fact “space” samples of $Z(t)$, as they are merely the coefficients of the (extended) Karhunen–Loeve decomposition of $Z(t)$ [20,21,22]; see also [23] for an in-depth discussion on this.

On the other hand, though, as opposed to the white Gaussian noise formulation, a link that establishes the information-theoretic connection between the continuous-time channel (6) and its discrete-time counterparts has long been missing, which may explain why discrete-time and continuous-time information theory (under the Brownian motion formulation) have largely gone separate ways with little interaction for the past several decades. In this paper, we will fill this gap by establishing causality-preserving connections between the channel (5) and its associated discrete-time versions in the forms of sampling and approximation theorems, which we believe will serve as the above-mentioned missing links and play important roles in the long run for further developing continuous-time information theory, particularly for the communication scenarios when feedback/memory is present.

As an immediate application of the approximation theorem, we propose the approximation approach to examine continuous-time Gaussian feedback channels with the average power constraint and infinite bandwidth (again, by comparison, the conventional sampling approach cannot handle feedback). It turns out that this approach, when complemented by relevant tools from stochastic calculus, can greatly enhance our understanding of continuous-time Gaussian channels in terms of giving alternative and strengthened interpretations to the low SNR equivalence in (A), recovering “long known” results (Theorems 7 (for the non-feedback case), 8 and 10) from new and rigorous perspectives, and deriving new results (Theorems 7 (for the feedback case), 9, 11 and 12) inspired by the intuition that the approximation approach carries.

Below, we summarize the contributions of this paper in greater details.

In Section 2, we prove Theorems 1 and 2, sampling theorems for a continuous-time Gaussian feedback/memory channel, which naturally connect such a channel with their sampled discrete-time versions. In addition, in Section 3, we prove Theorems 3 and 4, the so-called approximation theorems, which connect a continuous-time Gaussian feedback/memory channel with its approximated discrete-time versions (in the sense of the Euler-Maruyama approximation [24]). Roughly speaking, a sampling theorem says that a time-sampled channel is “close” to the original channel if the sampling is fine enough, and an approximation theorem says that an approximated channel is “close” to the original channel if the approximation is fine enough, both in an information-theoretic sense. Note that, as elaborated in Remark 3, certain version of the approximation theorem boils down to the sampling theorem when there is no memory and feedback in the channel.

Apparently a sampling theorem, whose spirit is in line with the Shannon–Nyquist sampling theorem, is of practical and theoretical value due to the fact it deals with the “real” values of the channel output; and, as will be elaborated later, approximation theorems seem to be surprisingly useful in a number of respects despite the fact it only deals with the “approximated” values of the channel output: it can certainly provide alternative rigorous tools in translating results from discrete time to continuous time; more importantly, as elaborated in Section 4, it lays the foundation for the approximation approach, which gives us the intuition in the point-to-point continuous-time setting, which will further help us to deliver rigorous treatments of multi-user continuous-time Gaussian channels in Section 5.

More specifically, in Section 5, we derive the capacity regions of a continuous-time white Gaussian multiple access channel (Theorem 7), a continuous-time white Gaussian interference channel (Theorem 8), and a continuous-time white Gaussian broadcast channel (Theorem 10). Here, we note that when there is no feedback, as discussed in Remark 5, the results above are “long known” in the sense that they are roughly suggested by the conventional sampling approach, or alternatively, the low SNR equivalence in (A). However, to the best of our knowledge, explicit formulations and statements of such results are missing in the literature and their rigourous proofs are non-trivial (for instance, when establishing Theorem 10, we have to resort to the continuous-time I-minimum mean square error (MMSE) relationship [25], which has been established only recently). By comparison, the presence of feedback necessitates the use of the approximation approach, which help us to connect relevant results and proofs in discrete time to analyze how feedback affects the capacity regions of families of continuous-time multi-user one-hop Gaussian channels: feedback will increase the capacity regions of some continuous-time Gaussian broadcast channels (Theorem 12) and interference channels (Theorem 9), while it will not increase capacity regions of a continuous-time physically degraded Gaussian broadcast channel (Theorem 11) and a continuous-time Gaussian multiple access channels (Theorem 7).

2. Sampling Theorems

A very natural question is whether, similarly for the white Gaussian noise formulation, sampling theorems hold for continuous-time white Gaussian channels under Brownian motion formulation. In this section, we will establish sampling theorems for the channel (6), which naturally connect such channels with their discrete-time versions obtained by sampling.

Consider the following regularity conditions for channel (6):

(a)

The solution $\{Y(t)\}$ to the stochastic differential Equation (6) uniquely exists;

(b)

$$\mathbb{P}\left({\int }_0^T{g}^2(t,W_0^t,Y_0^t)dt<\infty \right)=\mathbb{P}\left({\int }_0^T{g}^2(t,W_0^t,B_0^t)dt<\infty \right)=1;$$

(c)

$${\int }_0^T\mathbb{E}[|g(t,W_0^t,Y_0^t)|]dt<\infty .$$

Note that all the three above conditions are rather weak: Condition (a) is necessary for the channel to be meaningful, and Conditions (b) and (c) are very mild integrability assumptions.

Now, for any $n\in \mathbb{N}$, choose time points $t_{n,0},t_{n,1},\dots ,t_{n,n}\in \mathbb{R}$ such that

$$0=t_{n,0}<t_{n,1}<\cdots <t_{n,n-1}<t_{n,n}=T,$$

and let ${\Delta }_n\triangleq \{t_{n,0},t_{n,1},\dots ,t_{n,n}\}$. Sampling the channel (6) over the time interval $[0,T]$ with respect to ${\Delta }_n$, we obtain its sampled discrete-time version as follows:

$$Y(t_{n,i})={\int }_0^{t_{n,i}}g(s,W_0^s,Y_0^s)ds+B(t_{n,i}),i=0,1,\dots ,n.$$

(7)

For any time-point sequence ${\Delta }_n$, we will use ${\delta }_{{\Delta }_n}$ to denote its minimal stepsize, namely,

$${\delta }_{{\Delta }_n}\triangleq \underset{i=1,2,\cdots ,n}{max}(t_{n,i}-t_{n,i-1}).$$

${\Delta }_n$ is said to be evenly spaced if $t_{n,i}-t_{n,i-1}=T/n$ for all feasible i, and we will use the shorthand notation ${\delta }_n$ to denote its stepsize, i.e., ${\delta }_n\triangleq t_{n,1}-t_{n,0}=T/n$. Apparently, evenly spaced time-point sequences are natural candidates with respect to which a continuous-time Gaussian channel can be sampled.

We are primarily concerned with the mutual information for the channel (6), whose standard definition (see, e.g., [12,26]) is given below:

$$I(W_0^T;Y_0^T)=\left\{\begin{array}{cc}\mathbb{E}\left[\log \frac{d{\mu }_{WY}}{d{\mu }_W\times {\mu }_Y}(W_0^T,Y_0^T)\right],\hfill & \mathrm{if}\frac{d{\mu }_{WY}}{d{\mu }_W\times {\mu }_Y}\mathrm{exists},\hfill \\ \infty ,\hfill & \mathrm{otherwise},\hfill \end{array}\right.$$

(8)

where the subscripted $\mu$ denotes the measure induced on $C[0,T]$ or $C[0,T]\times C[0,T]$ by the corresponding stochastic process and $d{\mu }_{WY}/d{\mu }_W\times {\mu }_Y$ denotes the Radon-Nikodym derivative of ${\mu }_{WY}$ with respect to $d{\mu }_W\times {\mu }_Y$. Here we note that the same definition with appropriately modified sample spaces applies to the degenerated case when $W_0^T$ and/or $Y_0^T$ are random variables.

Roughly speaking, the following sampling theorem states that for any sequence of “increasingly refined” samplings, the mutual information of the sampled discrete-time channel (7) will converge to that of the original channel (6).

Theorem 1.

Assume Conditions (a)–(c). Suppose that ${\Delta }_n\subset {\Delta }_{n+1}$ for all n and that ${\delta }_{{\Delta }_n}\to 0$ as n tends to infinity. Then, we have

$$\underset{n\to \infty }{lim}I(W_0^T;Y({\Delta }_n))=I(W_0^T;Y_0^T),$$

where $Y({\Delta }_n)\triangleq \{Y(t_{n,0}),Y(t_{n,1}),\dots ,Y(t_{n,n})\}$.

Proof. 

The proof is rather technical and thereby postponed to Appendix A. ☐

Regarding the assumptions of Theorem 1, as mentioned before, Conditions (a)–(c) are very weak, but the condition that “${\Delta }_n\subset {\Delta }_{n+1}$ for all n” is somewhat restrictive, which, in particular, is not satisfied by the set $\left\{{\Delta }_n\right\}$ of all evenly spaced time-point sequences. We next show that this condition can be replaced by some extra regularity conditions: The same theorem holds as long as the stepsize of the sampling tends to 0, which, in particular, is satisfied by the set of all evenly spaced sampling sequences.

Below and hereafter, defining the distance $\parallel U_0^s-V_0^t\parallel$ between $U_0^s$ and $V_0^t$ with $0\le s\le t$ as

$$\parallel U_0^s-V_0^t\parallel \triangleq \underset{r\in [0,s]}{sup}|U(r)-V(r)|+\underset{r\in [s,t]}{sup}|U(s)-V(r)|,$$

(9)

we may assume the following three regularity conditions for the channel (6):

(d)

Uniform Lipschitz condition: There exists a constant $L>0$ such that for any $0\le s_1,s_2,s_3,t_1,t_2,t_3\le T$, any $U_0^T,V_0^T,Y_0^T$ and $Z_0^T$,

$$|g(s_1,U_0^{s_2},Y_0^{s_3})-g(t_1,V_0^{t_2},Z_0^{t_3})|\le L(|s_1-t_1|+\parallel U_0^{s_2}-V_0^{t_2}\parallel +\parallel Y_0^{s_3}-Z_0^{t_3}\parallel );$$

(e)

Uniform linear growth condition: There exists a constant $L>0$ such that for any $W_0^T$ and any $Y_0^T$,

$$|g(t,W_0^t,Y_0^t)|\le L(1+\parallel W_0^t\parallel +\parallel Y_0^t\parallel ),$$

where

$$\parallel W_0^t\parallel =\underset{r\in [0,t]}{sup}|W(r)|,\parallel Y_0^t\parallel =\underset{r\in [0,t]}{sup}|Y(r)|;$$

(f)

Regularity conditions on W: There exists $\epsilon >0$ such that

$$\mathbb{E}[e^{\epsilon \parallel W_0^T{\parallel }^2}]<\infty ,$$

and for any $K>0$, there exists ${\epsilon }^{\prime }>0$ such that

$$\mathbb{E}[e^{K{sup}_{|s-t|\le {\epsilon }^{\prime }}{(W(s)-W(t))}^2}]<\infty ,$$

and there exists a constant $L>0$ such that for any ${\epsilon }^{″}>0$,

$$\mathbb{E}[{sup}_{|s-t|\le {\epsilon }^{″}}{(W(s)-W(t))}^4]\le L{({\epsilon }^{″})}^2.$$

The following lemma, whose proof is postponed to Appendix B, says that Conditions (d)–(f) are stronger than Conditions (a)–(c). We however remark that Conditions (d)–(f) are still rather mild assumptions: The uniform Lipschitz condition, uniform linear growth condition and their numerous variants are typical assumptions that can guarantee the existence and uniqueness of the solution to a given stochastic differential equation. In theory, these two conditions are considered mild in the sense there are examples that the corresponding stochastic differential equation may not have solutions at all if these two conditions are not satisfied (see, e.g., [27]). Note that the third condition is a mild integrability condition; as a matter of fact, for a feedback channel where W is interpreted as the message, this condition is trivially satisfied. All three conditions above will be taken for granted in most practical communication situations: as might be expected, the signals employed in practice will be much better-behaving.

Lemma 1.

Assume Conditions (d)–(f). Then, there exists a unique strong solution of (6) with initial value $Y(0)=0$. Moreover, there exists $\epsilon >0$ such that

$$\mathbb{E}[e^{\epsilon \parallel Y_0^T{\parallel }^2}]<\infty ,$$

(10)

which immediately implies Conditions (b) and (c).

Roughly speaking, the following sampling theorem states that if the stepsizes of the samplings tend to 0, the mutual information of the channel (7) will converge to that of the channel (6). Note that in this theorem, we do not need the assumption that “${\Delta }_n\subset {\Delta }_{n+1}$ for all n”, which is required in Theorem 1.

Theorem 2.

Assume Conditions (d)–(f). For any sequence $\left\{{\Delta }_n\right\}$ with ${\delta }_{{\Delta }_n}\to 0$ as n tends to infinity, we have

$$\underset{n\to \infty }{lim}I(W_0^T;Y({\Delta }_n))=I(W_0^T;Y_0^T).$$

Proof. 

The proof is rather technical and lengthy, and thereby postponed to Appendix C. We note that, as detailed in Remark A1, the arguments in the proof can be adapted to yield a sampling theorem in estimation theory. ☐

3. Approximation Theorems

In this section, we will establish approximation theorems for the channel (6), which naturally connect such channels with their discrete-time versions obtained by approximation. As elaborated in later sections, the approximation theorem will underpin the approximation approach that will be introduced in Section 4.

An application of the Euler-Maruyama approximation [24] with respect to ${\Delta }_n$ to (6) will yield a discrete-time sequence $\{Y^{(n)}(t_{n,i}):i=0,1,\cdots ,n\}$ and a continuous-time process $\{Y^{(n)}(t):t\in [0,T]\}$, a linear interpolation of $\left\{Y(t_{n,i})\right\}$, as follows: Initializing with $Y^{(n)}(0)=0$, we recursively compute, for each $i=0,1,\cdots ,n-1$,

$$Y^{(n)}(t_{n,i+1})=Y^{(n)}(t_{n,i})+{\int }_{t_{n,i}}^{t_{n,i+1}}g(s,W_0^{t_{n,i}},Y_0^{(n),t_{n,i}})ds+B(t_{n,i+1})-B(t_{n,i}),$$

(11)

$$Y^{(n)}(t)=Y^{(n)}(t_{n,i})+\frac{t-t_{n,i}}{t_{n,i+1}-t_{n,i}}(Y^{(n)}(t_{n,i+1})-Y^{(n)}(t_{n,i})),t_{n,i}\le t\le t_{n,i+1}.$$

(12)

We are now ready to prove the following theorem:

Theorem 3.

Assume Conditions (d)–(f). Then, we have

$$\underset{n\to \infty }{lim}I(W_0^T;Y^{(n)}({\Delta }_n))=I(W_0^T;Y_0^T),$$

where $Y^{(n)}({\Delta }_n)\triangleq \{Y^{(n)}(t_{n,0}),Y^{(n)}(t_{n,1}),\dots ,Y^{(n)}(t_{n,n})\}$.

Proof. 

The proof is rather technical and lengthy, and thereby postponed to Appendix D. We note that, as detailed in Remark A2, the arguments in the proof can be adapted to yield an approximation theorem in estimation theory. ☐

For any $\left\{{\Delta }_n\right\}$, let $W^{(n)}(t)$ denote the piecewise linear version of $W_0^T$ with respect to ${\Delta }_n$; more precisely, for any $i=0,1,\cdots ,n$, $W^{(n)}(t_{n,i})=W(t_{n,i})$, and for any $t_{n,i-1}<s<t_{n,i}$ with $s=\lambda t_{n,i-1}+(1-\lambda )t_{n,i}$ where $0<\lambda <1$, $W^{(n)}(s)=\lambda W(t_{n,i-1})+(1-\lambda )W(t_{n,i})$. The following modified Euler-Maruyama approximation with respect to ${\Delta }_n$ applied to the channel (6) yields a discrete-time sequences $\{Y^{(n)}(t_{n,i}):i=0,1,\cdots ,n\}$ and a continuous-time processes $\{Y^{(n)}(t):t\in [0,T]\}$ as follows: Initializing with $Y^{(n)}(0)=0$, we recursively compute, for each $i=0,1,\cdots ,n-1$,

$$Y^{(n)}(t_{n,i+1})=Y^{(n)}(t_{n,i})+{\int }_{t_{n,i}}^{t_{n,i+1}}g(s,W_0^{(n),t_{n,i}},Y_0^{(n),t_{n,i}})ds+B(t_{n,i+1})-B(t_{n,i}),$$

(13)

$$Y^{(n)}(t)=Y^{(n)}(t_{n,i})+\frac{t-t_{n,i}}{t_{n,i+1}-t_{n,i}}(Y^{(n)}(t_{n,i+1})-Y^{(n)}(t_{n,i})),t_{n,i}\le t\le t_{n,i+1}.$$

(14)

Now, using a parallel argument in the proof of Theorem 3, we have the following approximation theorem.

Theorem 4.

Assume Conditions (d)–(f). Then, we have

$$\underset{n\to \infty }{lim}I(W^{(n)}({\Delta }_n);Y^{(n)}({\Delta }_n))=I(W_0^T;Y_0^T).$$

Remark 1.

When the channel (6) is interpreted as a feedback channel, both $W^{(n)}$ and W are precisely M. When the channel (6) is interpreted as a memory channel, Theorem 4 states that the mutual information between its input and output is the limit of that of its approximated input and output (in the sense of the above-mentioned modified Euler-Maruyama approximation).

Other variants of the Euler-Maruyama approximation can also be applied to the channel to yield variants of the approximation theorem. For instance, under Conditions (d)–(f), for the following variant of the Euler-Maruyama approximation,

$$Y^{(n)}(t_{n,i+1})=Y^{(n)}(t_{n,i})+{\int }_{t_{n,i}}^{t_{n,i+1}}g(t_{n,i},W_0^{t_{n,i}},Y_0^{(n),t_{n,i}})ds+B(t_{n,i+1})-B(t_{n,i}),$$

(15)

a parallel argument as in the proof of Theorem 3 will give the following variant of Theorem 3:

Theorem 5.

Assume Conditions (d)–(f). Then, we have

$$\underset{n\to \infty }{lim}I(W_0^T;Y^{(n)}({\Delta }_n))=I(W_0^T;Y_0^T).$$

(16)

Moreover, for

$$Y^{(n)}(t_{n,i+1})=Y^{(n)}(t_{n,i})+{\int }_{t_{n,i}}^{t_{n,i+1}}g(t_{n,i},W_0^{(n),t_{n,i}},Y_0^{(n),t_{n,i}})ds+B(t_{n,i+1})-B(t_{n,i}),$$

(17)

we have the following variant of Theorem 4:

Theorem 6.

Assume Conditions (d)–(f). Then, we have

$$\underset{n\to \infty }{lim}I(W^{(n)}({\Delta }_n);Y^{(n)}({\Delta }_n))=I(W_0^T;Y_0^T).$$

(18)

Regarding the approximation theorem and its variants, we make the following several remarks.

Remark 2.

Continuous-time directed information has been defined in [18] for continuous-time white Gaussian channels with positively delayed feedback. In this remark, we show that our approximation theorem can be used to give an alternative definition of continuous-time directed information, even for the case that the feedback is instantaneous.

Consider the following continuous-time Gaussian feedback channel:

$$Y(t)={\int }_0^{t}X(s,M,Y_0^s)ds+B(t),t\in [0,T].$$

(19)

For any ${\Delta }_n$, we define ${\tilde{X}}^{(n)}(·)$ as follows: for any t with $t_{n,i}\le t<t_{n,i+1}$,

$${\tilde{X}}^{(n)}(t)=\sum _{j=0}^{i-1}{\int }_{t_{n,j}}^{t_{n,j+1}}X(s,M,Y_0^{(n),t_{n,j}})ds+{\int }_{t_{n,i}}^{t}X(s,M,Y_0^{(n),t_{n,i}})ds,$$

Writing $\tilde{X}(t,M,Y_0^{(n),t})$ as ${\tilde{X}}^{(n)}(t)$ for simplicity, (11) can be rewritten as

$$Y^{(n)}(t_{n,i+1})=Y^{(n)}(t_{n,i})+{\tilde{X}}^{(n)}(t_{n,i+1})-{\tilde{X}}^{(n)}(t_{n,i})+B(t_{n,i+1})-B(t_{n,i}),$$

for which it can be readily checked that

$$I({\tilde{X}}^{(n)}({\Delta }_n)\to Y^{(n)}({\Delta }_n))=I(M;Y^{(n)}({\Delta }_n)).$$

(20)

Theorem 3 and the above observation can be used to define continuous-time directed mutual information. To be more precise, the continuous-time directed information from $X_0^T$ to $Y_0^T$ of the channel (19) can be defined as

$$I(X_0^T\to Y_0^T)\triangleq \underset{n\to \infty }{lim}I({\tilde{X}}^{(n)}({\Delta }_n)\to Y^{(n)}({\Delta }_n)).$$

(21)

Consider the following continuous-time Gaussian channel with possibly delayed feedback:

$$Y(t)={\int }_0^{t}X(s,M,Y_0^{s-D})ds+B(t),t\in [0,T],$$

(22)

where $D\ge 0$ denotes the delay of the feedback. In [18], the notion of continuous-time directed information from $X_0^T$ to $Y_0^T$ is defined as follows:

$$I_D(X_0^T\to Y_0^T)=\underset{{\Delta }_n}{inf}\sum _{i=1}^{n}I(X_{t_{n,0}}^{t_{n,i}};Y_{t_{n,i-1}}^{t_{n,i}}|Y_{t_{n,0}}^{t_{n,i-1}}).$$

(23)

It is proven that for the case $D>0$, using this notion, a connection between information theory and estimation theory can be established as follows:

$$I_D(X_0^T\to Y_0^T)=\frac{1}{2}{\int }_0^T\mathbb{E}[{(X(t)-\mathbb{E}[X(t)|Y_0^t])}^2]dt.$$

(24)

On the other hand though, it is easy to see that for the case $D=0$, i.e., there is no delay in the feedback as in (19), the definition in (23) and the equality as in (24) may run into some problems: Consider the extreme scenario and choose $X(t)=-Y(t)$ for any feasible t, then clearly the right-hand side of (24) should be equal to 0. On the other hand, though, for the left-hand side, each small interval in (23) will yield

$$I_{D=0}(X_{t_{n,0}}^{t_{n,i+1}};Y_{t_{n,i}}^{t_{n,i+1}}|Y_{t_{n,0}}^{t_{n,i}})=I_{D=0}(Y_{t_{n,i}}^{t_{n,i+1}};Y_{t_{n,i}}^{t_{n,i+1}}|Y_{t_{n,0}}^{t_{n,i}})=I_{D=0}(Y_{t_{n,i}}^{t_{n,i+1}};Y_{t_{n,i}}^{t_{n,i+1}}|Y(t_{n,i})),$$

where for the last equality, we have used the fact that under the assumption that $X(t)=-Y(t)$, $\{Y(t)\}$ is an Ornstein–Uhlenbeck process, which is a Gaussian Markov process. Noting that given $Y(t_i)=y(t_i)$, the Radon-Nikodym derivative $d{\mu }_{Y_{t_i}^{t_{i+1}}{Y}_{t_i}^{t_{i+1}}}/d{\mu }_{Y_{t_i}^{t_{i+1}}}\times {\mu }_{Y_{t_i}^{t_{i+1}}}$ does not exist, we conclude, by referring to the definition in (8), that

$$I_{D=0}(Y_{t_{n,i}}^{t_{n,i+1}};Y_{t_{n,i}}^{t_{n,i+1}}|Y(t_{n,i}))=\infty ,and thereby,I_{D=0}(X_{t_{n,0}}^{t_{n,i+1}};Y_{t_{n,i}}^{t_{n,i+1}}|Y_{t_{n,0}}^{t_{n,i}})=\infty ,$$

which further implies that, $I_{D=0}(X_0^T\to Y_0^T)$, the left-hand side of (24) at $D=0$ is infinite, a contradiction. On the other hand, be it the case $D>0$ or $D=0$, with the definition in (21), Theorem 3 however promises:

$$I(X_0^T\to Y_0^T)=I(M_0^T;Y_0^T)=\frac{1}{2}{\int }_0^T\mathbb{E}[{(X(t)-\mathbb{E}[X(t)|Y_0^t])}^2]dt.$$

Remark 3.

When there is no feedback or memory, Theorem 3 boils down to Theorem 2: obviously we will have for any feasible i

$$Y^{(n)}(t_{n,i})=Y(t_{n,i}),$$

which means that Theorem 3 actually states

$$\underset{n\to \infty }{lim}I(W_0^T;Y({\Delta }_n))=I(W_0^T;Y_0^T),$$

which is precisely the conclusion of Theorem 2. Moreover, by Remark 2, we also have

$$\underset{n\to \infty }{lim}I({\tilde{X}}^{(n)}({\Delta }_n);Y({\Delta }_n))=\underset{n\to \infty }{lim}I({\tilde{X}}^{(n)}({\Delta }_n)\to Y({\Delta }_n))=I(W_0^T;Y_0^T).$$

Remark 4.

In this remark, we briefly discuss the possible applications of our sampling and approximation theorems, both of which we believe will important roles in the long run for further developing continuous-time information theory, particularly for scenarios where feedback and memory are present.

Taking advantage of the pathwise continuity of a Brownian motion, our sampling theorems, Theorems 1 and 2, naturally connect continuous-time Gaussian memory/feedback channels with their discrete-time counterparts, whose outputs are precisely sampled outputs of the original continuous-time Gaussian channel. In discrete time, the Shannon–McMillan–Breiman theorem provides an effective way to approximate the entropy rate of a stationary ergodic process, and numerical computation and optimization of mutual information of discrete-time channel using the Shannon–McMillan–Breiman theorem and its extensions have been extensively studied (see, e.g., [28,29] and references therein), which suggests our sampling theorems may well serve as a bridge to capitalize on relevant results in discrete time to numerically compute and optimize the mutual information of continuous-time Gaussian channels. In short, despite numerous technical barriers that one needs to overcome, we believe that in the long run the sampling theorems can help us in terms of numerically computing the mutual information and capacity of continuous-time Gaussian channels.

By comparison, our approximation theorems, Theorems 3 and 4, are somewhat “artificial” in the sense that the outputs of the associated discrete-time channels are only approximated outputs of the original continuous-time channels. Nonetheless, as the Euler-Maruyama approximation of a continuous-time channel yields the form of a discrete-time channel typically takes, our approximation theorems allow a smooth translation from the results and ideas from the discrete-time setting to the continuous-time setting. As a result, the approximation theorems underpin the so-called approximation approach (to be introduced in Section 4) and readily yield results for continuous-time Gaussian channels in the multi-user setting, which will be elaborated in the following sections.

4. The Approximation Approach

Consider the following continuous-time white Gaussian channel with feedback

$$Y(t)={\int }_0^{t}X(s,M,Y_0^s)ds+B(t),t\ge 0,$$

(25)

satisfying the power constraint: there exists $P>0$ such that for any T, with probability 1

$$\frac{1}{T}{\int }_0^T{X}^2(s,M,Y_0^s)ds\le P.$$

(26)

As mentioned in Section 1, it is well known that the capacity of the above channel is $P/2$ (The same result can be established under alternative power constraints; see, e.g., [12]).

As elaborated in Section 1, when there is no feedback in the channel, i.e., the channel (25) is actually equivalent to (1), and one can “derive” the non-feedback capacity heuristically using the conventional sampling approach as in (1)–(4). But this approach is unable to tackle feedback since an application of the Shannon–Nyquist sampling theorem will destroy the temporal causality.

In this section, we use our approximation theorems to give an alternative way to “derive” the capacity of (25), which will be referred to as the approximation approach in the remainder of the paper. Compared to the sampling approach, the approximation approach can handle feedback due to the fact the Euler-Maruyama approximation preserves temporal causality. Below we briefly explain this new approach, which will be further developed and used, either heuristically or rigorously, in Section 5, where multiple users may be involved in a communication system.

For fixed $T>0$, consider the evenly spaced sequence ${\Delta }_n$ with stepsize ${\delta }_n=T/n$. Applying the Euler-Maruyama approximation (15) to the channel (25) over the time window $[0,T]$, we obtain

$$Y^{(n)}(t_{n,i+1})=Y^{(n)}(t_{n,i})+{\int }_{t_{n,i}}^{t_{n,i+1}}X(t_{n,i},M,Y_0^{(n),t_{n,i}})ds+B(t_{n,i+1})-B(t_{n,i}).$$

(27)

By Theorem 5, we have

$$I(M;Y_0^T)=\underset{n\to \infty }{lim}I(M;Y^{(n)}({\Delta }_n)).$$

(28)

Our strategy is to “establish” the capacity for the discrete-time channel (27) first, and then the capacity for the continuous-time channel (25) using the “closeness” between the two channels, as claimed by approximation theorems.

For the converse part, we first note that

$$\begin{array}{cc}\hfill I(M;Y^{(n)}({\Delta }_n))& =\sum _{i=1}^{n}h(Y^{(n)}(t_{n,i})-Y^{(n)}(t_{n,i-1})|Y_{t_{n,0}}^{(n),t_{n,i-2}})-\sum _{i=1}^{n}h(B(t_{n,i})-B(t_{n,i-1}))\hfill \\ & \le \sum _{i=1}^{n}h(Y^{(n)}(t_{n,i})-Y^{(n)}(t_{n,i-1}))-\sum _{i=1}^{n}h(B(t_{n,i})-B(t_{n,i-1})).\hfill \end{array}$$

It then follows from the fact

$$\begin{array}{cc}\hfill Var(Y^{(n)}(t_{n,i})-Y^{(n)}(t_{n,i-1}))& =\mathbb{E}[{(Y^{(n)}(t_{n,i})-Y^{(n)}(t_{n,i-1}))}^2]\hfill \\ & =\mathbb{E}[{\delta }_n^2{(X^{(n)}(t_{n,i-1}))}^2]+\mathbb{E}[{(B(t_{n,i})-B(t_{n,i-1}))}^2]\hfill \\ & =\mathbb{E}[{\delta }_n^2{(X^{(n)}(t_{n,i-1}))}^2]+{\delta }_n,\hfill \end{array}$$

that

$$\begin{array}{cc}\hfill I(M;Y_{{\Delta }_n}^{(n)})& \le \frac{1}{2}\sum _{i=0}^n\log (1+{\delta }_{n}E[{(X^{(n)}(t_{n,i}))}^2])\hfill \end{array}$$

(29)

$$\begin{array}{c}\le \frac{1}{2}\sum _{i=0}^n{\delta }_{n}E[{(X^{(n)}(t_{n,i}))}^2],\hfill \end{array}$$

(30)

which, by (28), immediately yields

$$I(M;Y_0^T)\le \frac{1}{2}{\int }_0^{T}E[X^2(s)]ds\le \frac{PT}{2},$$

(31)

which establishes the converse part.

For the availability part, note that if we assume all $X^{(n)}(t_{n,i-1})$ are independent of the Brownian motion B with $\mathbb{E}[{(X^{(n)}(t_{n,i-1}))}^2]=P$, then the inequalities in (29) and (30) will become equalities. The part then follows from a usual random coding argument with codes generated by the distribution of $X^{(n)}$ (or more precisely, a linear interpolation of $X^{(n)}$). It is clear that as n tends to infinity, the process $X^{(n)}$ behaves increasingly like a white Gaussian process. This observation echoes Theorem $\mathrm{6.4.1}$ in [12], whose proof rigorously shows that an Ornstein–Uhlenbeck process that oscillates “extremely” fast will achieve the capacity of (25).

Roughly speaking, similar to the conventional sampling approach, the above approximation approach establishes a continuous-time Gaussian feedback channel as the limit of the associated discrete-time channels as the SNR for each channel use shrink to zero proportionately (note that in the above arguments, the SNR for each channel use is ${\delta }_n$). In other words, we have strengthened the low SNR equivalence in (A) as follows:

a continuous-time infinite-bandwidth Gaussian channel with feedback is “equivalent” to a discrete-time Gaussian channel with feedback at low SNR.

(B)

We remark however that for the purpose of deriving the capacity of (25) though, the approximation approach, like the conventional sampling approach, is heuristic in nature: Theorem 3 does require Conditions (d)–(f), which are much stronger than the power constraint (26). Nevertheless, this approach is of fundamental importance to our treatment of continuous-time Gaussian channels: as elaborated in Section 5, not only can it channel the ideas and techniques in discrete time to rigorously establish new results in continuous time, more importantly, it can also provide insights and intuition for our rigorous treatments where we will employ established tools and develop new tools in stochastic calculus.

5. Continuous-Time Multi-User Gaussian Channels

Extending Shannon’s fundamental theorems on point-to-point communication channels to general networks with multiple sources and destinations, network information theory aims to establish the fundamental limits on information flows in networks and the optimal coding schemes that achieve these limits. The vast majority of researches on network information theory to date have been focusing on networks in discrete time. In a way, this phenomenon can find its source from Shannon’s original treatment of continuous-time point-to-point channels, where such channels were examined through their associated discrete-time versions. This insightful viewpoint has exerted major influences on the bulk of the related literature on continuous-time Gaussian channels, oftentimes prompting a model shift from the continuous-time setting to the discrete-time one right from the beginning of a research attempt.

The primary focus of this section is to illustrate the possible applications of the approximation approach: (1) Guided by this approach, we will rigorously derive the capacity regions of families of continuous-time multi-user one-hop white Gaussian channels, including continuous-time multi-user white Gaussian multiple access channels (MACs) and broadcast channels (BCs). To deliver the rigourous proofs of our results, we will directly work within the continuous-time setting, employing established tools and developing new tools (see Theorems A3 and A5) in stochastic calculus to complement the approximation approach; (2) We can also rigorously apply this approach to examine how feedback affects the capacity regions of the above-mentioned channels via translations of results and techniques in discrete time. It turns out that some results can be translated from the discrete-time setting to the continuous-time setting, such as that feedback increases the capacity region of Gaussian BCs, and that feedback does not increase the capacity region of physically degraded BCs. Nevertheless, there is a seeming “exception”: as opposed to discrete-time Gaussian MACs, feedback does not increase the capacity region some of continuous-time Gaussian MACs, which, somewhat surprisingly, can also be explained by the approximation approach as well.

Below, we summarize the results in this section. To put our results into a relevant context, we will first list some related results in discrete time, and for obvious reasons, we can only list those that are most relevant to ours.

Gaussian MACs. When there is no feedback, the capacity region of a discrete-time memoryless MAC is relatively better understood: a single-letter characterization has been established by Ahlswede [30] and the capacity region of a Gaussian MAC was explicitly derived in Wyner [31] and Cover [32]. On the other hand, the capacity region of MACs with feedback still demands more complete understanding, despite several decades of great effort by many authors: Cover and Leung [33] derived an achievable region for a memoryless MAC with feedback. In [34], Willems showed that Cover and Leung’s region is optimal for a class of memoryless MACs with feedback where one of the inputs is a deterministic function of the output and the other input. More recently, Bross and Lapidoth [35] improved Cover and Leung’s region, and Wu et al. [36] extended Cover and Leung’s region for the case where non-causal state information is available at both senders. An interesting result has been obtained by Ozarow [37], who derived the capacity region of a memoryless Gaussian MAC with two users via a modification of the Schalkwijk-Kailath scheme [38]; moreover, Ozarow’s result showed that in general, the capacity region for a discrete memoryless MAC is increased by feedback.

In Section 5.1, guided by the approximation approach, we first establish Lemma A3, a key lemma which roughly says that “other users can be simply treated as noises”, and we then employ established tools from stochastic calculus to derive the capacity region of a continuous-time white Gaussian MAC with m senders and with/without feedback. It turns out that for such a channel, feedback does not increase the capacity region, which, at first sight, may seem at odds with the aforementioned Ozarow’s result and the conclusion of our approximation theorems. This however can be roughly explained by the well-known fact that “>” may become “=” when taking the limit (indeed, $a_n>b_n$ does not necessarily imply ${lim}_{n\to \infty }{a}_n>{lim}_{n\to \infty }{b}_n$); see Remark 6 for a more detailed explanation.

Gaussian ICs. The capacity regions of discrete-time Gaussian ICs are largely unknown except for certain special scenarios: The capacity region of Gaussian ICs with strong interference has been established in Sato [39], Han and Kobayashi [40]. The sum-capacity of Gaussian ICs with weak interference has been simultaneously derived in [41,42,43]. The half-bit theorem on the tightness of the Han-Kobayashi bound [40] was proven in [44]. The approximation of the Gaussian IC by the q-ary expansion deterministic channel was first proposed by Avestimehr, Diggavi, and Tse [45]. Outer and inner bounds on the feedback capacity region of Gaussian interference channels are established by Suh and Tse [46]. Note that all the above-mentioned work deal with ICs with two pairs of senders and receivers. For more than two user pairs, special classes of Gaussian ICs have been examined using the scheme of interference alignment; see an extensive list of references in [47].

In Section 5.2, using a similar approach that we developed for continuous-time Gaussian MACs, we derive the capacity region of a continuous-time white Gaussian IC with m pairs of senders and receivers and without feedback. In addition, we also use a translated version of the argument in [46] and the approximation approach to show that feedback does increase the capacity region of certain continuous-time white Gaussian IC.

Gaussian BCs. The capacity regions of discrete-time Gaussian BCs without feedback are well known [48,49]. In addition, it has been shown by El Gamal [50] that feedback cannot increase the capacity region of a physically degraded Gaussian BC. On the other hand, it was shown by Ozarow and Leung [51] that feedback can increase the capacity of stochastically degraded Gaussian BCs, whose capacity regions are far less understood.

In Section 5.3, we first establish a continuous-time version of entropy power inequality (Theorem A5) and then derive the capacity region of a continuous-time Gaussian non-feedback BC with m receivers. Employing the approximation approach, we use a modified argument in [50] to show that feedback does not increase the capacity region of a physically degraded continuous-time Gaussian BC, and on the other hand, a translated version of the argument in [51] to show that feedback does increase the capacity region of certain continuous-time Gaussian BC.

Here we remark that the above-mentioned capacity results for the non-feedback case (Theorems 7 (for the non-feedback case) and 10) are “long known” in the sense that they are “predicted” by the conventional sampling approach and their proofs follow from the usual framework. On the other hand, though, explicit formulations and statements of these results and their rigorous and complete proofs, to the best of our knowledge, do not exist in the literature. The reason, we believe, is that there are a number of technical difficulties that one has to overcome to prove such results: Lemmas A3 and A5 (which is based on the I-MMSE relationship that has only been established in [25]) are newly developed in this work and their proofs are non-trivial.

In contrast, the approximation approach can be applied to continuous-time Gaussian feedback channels, either heuristically or rigorously. More specifically, it can be heuristically applied to “explain” Theorems 7 (on feedback capacity) and 10) and give us intuition (as elaborated in Remark 6, it helps “predict” the optimal channel input distribution, which has been made rigorous in Lemma A3), and it can also be applied to establish Theorems 11 and 12.

5.1. Gaussian MACs

Consider a continuous-time white Gaussian MAC with m users, which can be characterized by

$$Y(t)={\int }_0^t{X}_1(s,M_1,Y_0^s)ds+{\int }_0^t{X}_2(s,M_2,Y_0^s)ds+\cdots +{\int }_0^t{X}_m(s,M_m,Y_0^s)ds+B(t),t\ge 0,$$

(32)

where $X_i$ is the channel input from sender i, which depends on $M_i$, the message sent from sender i, which is independent of all messages from other senders, and possibly on the feedback $Y_0^s$, the channel output up to time s.

For $T,R_1,\dots ,R_m,P_1,\dots ,P_m>0$, a $(T,(e^{T{R}_1},\dots ,e^{T{R}_m}),(P_1,\dots ,P_m))$-code for the MAC (32) consists of m sets of integers ${\mathcal{M}}_i=\{1,2,\dots ,e^{T{R}_i}\}$, the message alphabet for user i, $i=1,2,\dots ,m$, and m encoding functions, $X_i:{\mathcal{M}}_i\to C[0,T]$, which satisfy the following power constraint: for any $i=1,2,\dots ,m$, with probability 1,

$$\frac{1}{T}{\int }_0^T{X}_i^2(s,M_i,Y_0^s)ds\le P_i,$$

(33)

and a decoding function,

$$g:C[0,T]\to {\mathcal{M}}_1\times {\mathcal{M}}_2\times \cdots \times {\mathcal{M}}_m.$$

The average probability of error for the above code is defined as

$$P_e^{(T)}=\frac{1}{e^{T({\sum }_{i=1}^m{R}_i)}}{\sum }_{(M_1,M_2,\dots ,M_m)\in {\mathcal{M}}_1\times {\mathcal{M}}_2\times \cdots \times {\mathcal{M}}_m}P\{g(Y_0^T)\ne (M_1,M_2,\dots ,M_m)|(M_1,M_2,\dots ,M_m)\mathrm{sent}\}.$$

A rate tuple $(R_1,R_2,\dots ,R_m)$ is said to be achievable for the MAC if there exists a sequence of $(T,(e^{T{R}_1},\dots ,e^{T{R}_m}),(P_1,\dots ,P_m))$-codes with $P_e^{(T)}\to 0$ as $T\to \infty$. The capacity region of the MAC is the closure of the set of all the achievable $(R_1,R_2,\dots ,R_m)$ rate tuples.

The following theorem, whose proof is postponed to Appendix E, gives an explicit characterization of the capacity region of (32).

Theorem 7.

Whether there is feedback or not, the capacity region of the continuous-time white Gaussian MAC (32) is

$$\{(R_1,R_2,\dots ,R_m)\in {\mathbb{R}}_{+}^m:R_i\le P_i/2,i=1,2,\dots ,m\}.$$

Remark 5.

When there is no feedback, Theorem 7 can be heuristically explained using the sampling approach as in (2)–(4) (this heuristic approach in this example should be well known; see, e.g., Exercise 15.26 in [2]).

For simplicity only, we consider the following continuous-time white Gaussian multiple access channel with two senders:

$$Y(t)=X_1(t)+X_2(t)+Z(t),t\in \mathbb{R},$$

(34)

where $X_i$, $i=1,2$, is the input from the i-th user with average power limit $P_i$. Juat as before, consider its associated discrete-time version corresponding to bandwidth limit ω:

$$Y_n=X_{1,n}^{(\omega )}+X_{2,n}^{(\omega )}+Z_n^{(\omega )},n\in \mathbb{Z}.$$

Then, it is well known [47] that the outer bound on the capacity region can be computed as

$$\left\{(R_1,R_2)\in {\mathbb{R}}_{+}^2:R_1\le W\log \left(1+\frac{P_1}{2\omega }\right),R_2\le W\log \left(1+\frac{P_2}{2\omega }\right)\right\},$$

and the inner bound as

$$\left\{(R_1,R_2)\in {\mathbb{R}}_{+}^2:R_1\le \omega \log \left(1+\frac{P_1}{2\omega }\right),R_2\le \omega \log \left(1+\frac{P_2}{2\omega }\right),R_1+R_2\le \omega \log \left(1+\frac{P_1+P_2}{2\omega }\right)\right\}.$$

(Here, it is known [31,32] that the outer bound can be tightened to coincide with the inner bound, which, however, is not needed for this example.) It is easy to verify that the two bounds also collapse into the same region as ω tends to infinity:

$$\left\{(R_1,R_2)\in {\mathbb{R}}_{+}^2:R_1\le P_1/2,R_2\le P_2/2\right\},$$

which is “expected” to be the capacity region of (34); or alternatively, one can apply the low SNR equivalence in (A) and take the limit as P tends to 0, reaching the same conclusion. Note that similar arguments hold for more than two senders as well through a parallel extension.

Remark 6.

When the feedback is present in the channel, the approximation approach, rather than the conventional sampling approach, is necessary to explain Theorem 7.

Again, for simplicity only, we consider the following continuous-time Gaussian MAC with two senders:

$$Y(t)={\int }_0^t{X}_1(s,M_1,Y_0^s)ds+{\int }_0^t{X}_2(s,M_2,Y_0^s)ds+B(t),t\ge 0,$$

(35)

with the power constraints: there exist $P_1,P_2>0$ such that for all T,

$${\int }_0^T{X}_1^2(s,M_1,Y_0^s)ds\le P_{1}T,{\int }_0^T{X}_2^2(s,M_2,Y_0^s)ds\le P_{2}T.$$

(36)

Applying the Euler-Maruyama approximation to the above channel over the time window $[0,T]$ with respect to the evenly spaced ${\Delta }_n$ with ${\delta }_n=T/n$, we obtain

$$Y^{(n)}(t_{n,i+1})=Y^{(n)}(t_{n,i})+{\int }_{t_{n,i}}^{t_{n,i+1}}{X}_1(s,M_1,Y_0^{(n),t_{n,i}})ds+{\int }_{t_{n,i}}^{t_{n,i+1}}{X}_2(s,M_2,Y_0^{(n),t_{n,i}})ds+B(t_{n,i+1})-B(t_{n,i}).$$

Now, straightforward computations and a usual concavity argument then yields that for large n,

$$\begin{array}{cc}\hfill I(M_1;Y^{(n)}({\Delta }_n)|M_2)& =h(Y^{(n)}({\Delta }_n)|M_2)-h(Y^{(n)}({\Delta }_n)|M_1,M_2)\hfill \\ & =\sum _{i=1}^{n}h(Y^{(n)}(t_{n,i})|Y_{t_{n,0}}^{(n),t_{n,i-1}},M_2)-\sum _{i=1}^{n}h(Y^{(n)}(t_{n,i})|Y_{t_{n,0}}^{(n),t_{n,i-1}},M_1,M_2)\hfill \\ & \le \sum _{i=1}^n\frac{1}{2}\log \left(\mathbb{E}{\left({\int }_{t_{n,i-1}}^{t_{n,i}}{X}_1(s,M_1,Y_0^{(n),t_{n,i-1}})ds\right)}^2+{\delta }_n\right)-\frac{1}{2}\log ({\delta }_n)\hfill \\ & \le \sum _{i=1}^n\frac{1}{2}\log \left(\left({\int }_{t_{n,i-1}}^{t_{n,i}}\mathbb{E}{X}_1{(s,M_1,Y_0^{(n),t_{n,i-1}})}^{2}ds\right){\delta }_n+{\delta }_n\right)-\frac{1}{2}\log ({\delta }_n)\hfill \\ & =\sum _{i=1}^n\frac{1}{2}\log \left({\int }_{t_{n,i-1}}^{t_{n,i}}\mathbb{E}{X}_1{(s,M_1,Y_0^{(n),t_{n,i-1}})}^{2}ds+1\right)\hfill \\ & \le \sum _{i=1}^n\sum _{i=1}^n\frac{1}{2}{\int }_{t_{n,i-1}}^{t_{n,i}}\mathbb{E}{X}_1{(s,M_1,Y_0^{(n),t_{n,i-1}})}^{2}ds\hfill \\ & \approx \sum _{i=1}^n\sum _{i=1}^n\frac{1}{2}{\int }_{t_{n,i-1}}^{t_{n,i}}\mathbb{E}{X}_1{(s,M_1,Y_0^s)}^{2}ds\hfill \\ & \le \frac{P_{1}T}{2}.\hfill \end{array}$$

A completely parallel argument will yield that

$$I(M_2;Y^{(n)}({\Delta }_n)|M_1)\le \frac{P_{2}T}{2}.$$

It then follows from Theorem 3 the region below give an outer bound of the capacity region:

$$\{(R_1,R_2):0\le R_1\le P_1/2,0\le R_2\le P_2/2\}.$$

(37)

To see that this outer bound can be achieved, set $X_1(s),X_2(s)$, $t_{n,i}\le s\le t_{n,i+1}$, in (35) to be independent Gaussian random variables with variances $P_1$, $P_2$, respectively. Then, one verifies that for large n,

$$\begin{array}{cc}\hfill I(M_1;Y({\Delta }_n))& =h(Y({\Delta }_n))-h(Y({\Delta }_n)|M_1)\hfill \\ & =\sum _{i=1}^{n}h(Y(t_{n,i}))-\sum _{i=1}^{n}h(Y(t_{n,i})|M_1)\hfill \\ & =\sum _{i=1}^n\frac{1}{2}\log (P_1{\delta }_n^2+P_2{\delta }_n^2+{\delta }_n)-\frac{1}{2}\log (P_2{\delta }_n^2+{\delta }_n)\hfill \end{array}$$

(38)

$$\begin{array}{c}=\sum _{i=1}^n\frac{1}{2}\log \left(1+\frac{P_1{\delta }_n^2}{P_2{\delta }_n^2+{\delta }_n}\right)\hfill \end{array}$$

(39)

$$\begin{array}{c}\approx P_{1}T/2,\hfill \end{array}$$

(40)

where we have used the fact that ${\delta }_n$ is “close” to 0 for large enough n for (40), which, parallelly as in Remark 5, can be alternatively explained by taking $P_1$ to 0 and then applying the low SNR equivalence in (B). With a similar argument, one can prove that

$$I(M_2;Y({\Delta }_n))\approx P_{2}T/2.$$

It then follows that the outer bound in (37) can be achieved.

Here we remark that similarly as in Section 4, for n large enough, the constructed processes $X_1$ and $X_2$ behave like “fast-oscillating” Ornstein–Uhlenbeck processes, and moreover, from (38) and (39), one can tell that for one user to achieve the maximum transmission rate, the other user can simply be ignored. Predicting the optimal channel input, these facts echo Remark A3 and give another explanation to Lemma A3, a key lemma in our rigourous proof of Theorem 7 in Appendix E.

5.2. Gaussian ICs

Consider the following continuous-time white Gaussian interference channel having no feedback and with m pairs of senders and receivers: for $i=1,2,\dots ,m$,

$$\begin{array}{c}\hfill Y_i(t)=a_{i1}{\int }_0^t{X}_1(s,M_1)ds+a_{i2}{\int }_0^t{X}_2(s,M_2)ds+\cdots +a_{im}{\int }_0^t{X}_m(s,M_m)ds+B_i(t),t\ge 0,\end{array}$$

(41)

where $X_i$ is the channel input from sender i, which depends on $M_i$, the message sent from sender i, which is independent of all messages from other senders, and $a_{ij}\in \mathbb{R}$, $i,j=1,2,\dots ,m$, is the channel gain from sender j to receiver i, all $B_i(t)$ are (possibly correlated) standard Brownian motions.

For $T,R_1,\dots ,R_m,P_1,\dots ,P_m>0$, a $(T,(e^{T{R}_1},\dots ,e^{T{R}_m}),(P_1,\dots ,P_m))$-code for the IC (41) consists of m sets of integers ${\mathcal{M}}_i=\{1,2,\dots ,e^{T{R}_i}\}$, the message alphabet for user i, $i=1,2,\dots ,m$, and m encoding functions, $X_i:{\mathcal{M}}_i\to C[0,T]$ satisfying the following power constraint: for any $i=1,2,\dots ,m$, with probability 1,

$$\frac{1}{T}{\int }_0^T{X}_i^2(s,M_i)ds\le P_i,$$

(42)

and m decoding functions, $g_i:C[0,T]\to {\mathcal{M}}_i$, $i=1,2,\dots ,m$.

The average probability of error for the $(T,(e^{T{R}_1},\dots ,e^{T{R}_m}),(P_1,\dots ,P_m))$-code is defined as

$$P_e^{(T)}=\frac{1}{e^{T({\sum }_{i=1}^m{R}_i)}}{\sum }_{(M_1,M_2,\dots ,M_m)\in {\mathcal{M}}_1\times {\mathcal{M}}_2\times \cdots \times {\mathcal{M}}_m}P\{g_i(Y_{i,0}^T)\ne M_i,i=1,2,\dots ,m|(M_1,M_2,\dots ,M_m)\mathrm{sent}\}.$$

A rate tuple $(R_1,R_2,\dots ,R_m)$ is said to be achievable for the IC if there exists a sequence of $(T,(e^{T{R}_1},\dots ,e^{T{R}_m}),(P_1,\dots ,P_m))$-codes with $P_e^{(T)}\to 0$ as $T\to \infty$. The capacity region of the IC is the closure of the set of all the achievable $(R_1,R_2,\dots ,R_m)$ rate tuples.

The following theorem explicitly characterizes the capacity region of the above IC, whose proof has been postponed to Appendix F.

Theorem 8.

The capacity region of the continuous-time white Gaussian IC (41) is

$$\{(R_1,R_2,\dots ,R_m)\in {\mathbb{R}}_{+}^m:R_i\le a_{ii}^2{P}_i/2,i=1,2,\dots ,m\}.$$

Remark 7.

Theorem 8 can be heuristically derived using a similar argument employing the approximation approach as in Remark 6.

With the explicit non-feedback capacity region stated in Theorem 8, we are now ready to use the approximation approach to analyze the effects of feedback on continuous-time Gaussian ICs.

The following theorem says that feedback does help continuous-time Gaussian ICs, whose proof uses a translated version of the argument in [46] coupled with the approximation approach as in Section 4, and so we only provide a sketch of the proof.

Theorem 9.

Feedback strictly increases the capacity region of certain continuous-time Gaussian interference channel.

Proof. 

Consider the following symmetric continuous-time Gaussian interference channel with two pairs of senders and receivers:

$$Y_1(t)=\sqrt{snr}{\int }_0^t{X}_1(s)ds+\sqrt{inr}{\int }_0^t{X}_2(s)ds+B_1(t),$$

$$Y_2(t)=\sqrt{inr}{\int }_0^t{X}_1(s)ds+\sqrt{snr}{\int }_0^t{X}_2(s)ds+B_2(t),$$

where $snr,inr$ denote the signal-to-noise, interference-to-noise ratios, respectively, $B_1(t),B_2(t)$ are independent standard Brownian motions, and the average power of $X_1,X_2$ are assumed to be 1.

Following [46], we consider the following coding scheme over two stages, each of length $T_0$. In the first stage, transmitters 1 and 2 send codewords $X_{1,0}^{T_0}$ and $X_{2,0}^{T_0}$ with rates $R_1$ and $R_2$, respectively. In the second stage, using feedback, transmitters 1 and 2 decode $X_{2,0}^{T_0}$ and $X_{1,0}^{T_0}$, respectively. This can be decoded if

$$R_1,R_2\le \frac{inr}{2}.$$

Then, transmitters 1 and 2 send $X_{1,T_0}^{2T_0}$ and $X_{2,T_0}^{2T_0}$, respectively such that for any $0\le t\le T_0$,

$$X_1(T_0+t)=X_2(t),X_2(T_0+t)=-X_1(t).$$

Then during the two stages, receiver 1 receives

$$Y_1(t)=\sqrt{snr}{\int }_0^t{X}_1(s)ds+\sqrt{inr}{\int }_0^t{X}_2(s)ds+B_1(t),0\le t\le T_0,$$

and

$$Y_1(T_0+t)=\sqrt{snr}{\int }_0^{T_0+t}{X}_1(s)ds+\sqrt{inr}{\int }_0^{T_0+t}{X}_2(s)ds+B_1(T_0+t),0\le t\le T_0,$$

which immediately gives rise to

$$\begin{array}{cc}\hfill Y_1(T_0+t)-Y_1(T_0)& =\sqrt{snr}{\int }_{T_0}^{T_0+t}{X}_1(s)ds+\sqrt{inr}{\int }_{T_0}^{T_0+t}{X}_2(s)ds+B_1(T_0+t)-B_1(T_0)\hfill \\ & =\sqrt{snr}{\int }_0^t{X}_2(s)ds-\sqrt{inr}{\int }_0^t{X}_1(s)ds+B_1(T_0+t)-B_1(T_0).\hfill \end{array}$$

We then have that for any $0\le t\le T_0$,

$$\sqrt{snr}{Y}_1(t)-\sqrt{inr}(Y_1(T_0+t)-Y_1(T_0))=(snr+inr){\int }_0^t{X}_1(s)ds+\sqrt{snr}{B}_1(t)-\sqrt{inr}(B_1(T_0+t)-B(T_0)),$$

which means the codeword $X_{1,0}^{T_0}$ can be decoded at the second stage if

$$R_1\le \frac{snr+inr}{2}.$$

A completely parallel argument yields that the codeword $X_{2,0}^{T_0}$ can be decoded at the second stages if

$$R_2\le \frac{snr+inr}{2}.$$

All in all, after the two stages, the two codewords $X_{1,0}^{T_0}$ and $X_{2,0}^{T_0}$ can be decoded as long as

$$R_1,R_2\le \frac{inr}{2};$$

in other words, coding rate $(\frac{inr}{2},\frac{inr}{2})$ is achievable, which, if assuming $inr>snr$, will imply that feedback strictly increases the capacity region. ☐

5.3. Gaussian BCs

In this section, we consider a continuous-time white Gaussian BC with m receivers, which is characterized by: for $i=1,2,\dots ,m$,

$$Y_i(t)=\sqrt{sn{r}_i}{\int }_0^{t}X(s,M_1,M_2,\dots ,M_m)ds+B_i(t),t\ge 0,$$

(43)

where X is the channel input, which depends on $M_i$, the message sent from sender i, which is uniformly distributed over a finite alphabet ${\mathcal{M}}_i$ and independent of all messages from other senders, $sn{r}_i$ is the signal-to-noise ratio in the channel for user i, $B_i(t)$ are (possibly correlated) standard Brownian motions.

For $T,R_1,R_2,\dots ,R_m,P>0$, a $(T,(e^{T{R}_1},\dots ,e^{T{R}_m}),P)$-code for the BC (43) consists of m set of integers ${\mathcal{M}}_i=\{1,2,\dots ,e^{T{R}_i}\}$, the message set for receiver i, $i=1,2,\dots ,m$, and an encoding function, $X:{\mathcal{M}}_1\times {\mathcal{M}}_2\times \cdots \times {\mathcal{M}}_m\to C[0,T]$, which satisfies the following power constraint: with probability 1,

$$\frac{1}{T}{\int }_0^T{X}^2(s,M_1,M_2,\dots ,M_m)ds\le P,$$

(44)

and m decoding functions, $g_i:C[0,T]\to {\mathcal{M}}_i$, $i=1,2,\dots ,m$.

The average probability of error for the $(T,(e^{T{R}_1},e^{T{R}_2},\dots ,e^{T{R}_m}),P)$-code is defined as

$$P_e^{(T)}=\frac{1}{e^{T({\sum }_{i=1}^m{R}_i)}}{\sum }_{(M_1,M_2,\dots ,M_m)\in {\mathcal{M}}_1\times {\mathcal{M}}_2\times \cdots \times {\mathcal{M}}_m}P\{g_i(Y_0^T)\ne M_i,i=1,2,\dots ,m|(M_1,M_2,\dots ,M_m)\mathrm{sent}\}.$$

A rate tuple $(R_1,R_2,\dots ,R_m)$ is said to be achievable for the BC if there exists a sequence of $(T,(e^{T{R}_1},e^{T{R}_2},\dots ,e^{T{R}_m}),P)$-codes with $P_e^{(T)}\to 0$ as $T\to \infty$. The capacity region of the BC is the closure of the set of all the achievable $(R_1,R_2,\dots ,R_m)$ rate tuples.

The following theorem explicitly characterizes the capacity region of the above BC, whose proof is postponed to Appendix G.

Theorem 10.

The capacity region of the continuous-time white Gaussian BC (43) is

$$\left\{(R_1,R_2,\dots ,R_m)\in {\mathbb{R}}_{+}^m:\frac{R_1}{sn{r}_1}+\frac{R_2}{sn{r}_2}+\cdots +\frac{R_m}{sn{r}_m}\le \frac{P}{2}\right\}.$$

Remark 8.

Theorem 10 can be heuristically derived using a similar argument employing the approximation approach as in Remark 6.

We are now ready to use the approximation approach to analyze the effects of feedback on continuous-time Gaussian BCs.

The following theorem says that feedback does not help physically degraded Gaussian BCs, whose proof is inspired by the ideas in Section 4 and parallels the argument in [50].

Theorem 11.

Consider the following continuous-time physically degraded Gaussian broadcast channel with one sender and two receivers:

$$Y_1(t)={\int }_0^{t}X(s,M_1,Y_{1,0}^s,Y_{2,0}^s)ds+\sqrt{N_1}{B}_1(t),$$

$$Y_2(t)={\int }_0^{t}X(s,M_2,Y_{1,0}^s,Y_{2,0}^s)ds+\sqrt{N_1}{B}_1(t)+\sqrt{N_2}{B}_2(t),$$

where $N_1,N_2>0$, and $B_1,B_2$ are independent standard Brownian motions, and the channel input $X(s)$ is assumed to satisfy Conditions (d)–(f). Then, feedback does not increase the capacity region of the above channel.

Proof. 

Let X be a $(T,(e^{T{R}_1},e^{T{R}_2}),P)$-code. By the code construction, for $i=1,2$, it is possible to estimate the messages $M_i$ from the channel output $Y_{i,0}^T$ with an arbitrarily low probability of error. Hence, by Fano’s inequality, for $i=1,2$,

$$H(M_i|Y_{i,0}^T)\le T{R}_i{P}_e^{(T)}+H(P_e^{(T)})=T{\epsilon }_{i,T},$$

where ${\epsilon }_{i,T}\to 0$ as $T\to \infty$. It then follows that

$$T{R}_1=H(M_1)=H(M_1|M_2)\le I(M_1;Y_{1,0}^T|M_2)+T{\epsilon }_{1,T},$$

$$T{R}_2=H(M_2)\le I(M_2;Y_{2,0}^T)+T{\epsilon }_{2,T}.$$

Now the Euler-Maruyama approximation with respect to the evenly spaced ${\Delta }_n$ of stepsize ${\delta }_n=T/n$ applied to the continuous-time physically degraded Gaussian BC yields:

$$Y_1^{(n)}(t_{n,i})-Y_1^{(n)}(t_{n,i-1})={\int }_{t_{n,i-1}}^{t_{n,i}}X(s,M,Y_{1,t_{n,0}}^{(n),t_{n,i-1}},Y_{2,t_{n,0}}^{(n),t_{n,i-1}})ds+\sqrt{N_1}{B}_1(t_{n,i})-\sqrt{N_1}{B}_1(t_{n,i-1}),$$

$$Y_2^{(n)}(t_{n,i})-Y_2^{(n)}(t_{n,i-1})={\int }_{t_{n,i-1}}^{t_{n,i}}X(s,M,Y_{1,t_{n,0}}^{(n),t_{n,i-1}},Y_{2,t_{n,0}}^{(n),t_{n,i-1}})ds+\sqrt{N_1}{B}_1(t_{n,i})-\sqrt{N_1}{B}_1(t_{n,i-1})+\sqrt{N_2}{B}_2(t_{n,i})-\sqrt{N_2}{B}_2(t_{n,i-1}).$$

Then, by Theorem 3, we have

$$\begin{array}{cc}\hfill I(M_2;Y_{2,0}^T)& =\underset{n\to \infty }{lim}I(M_2;Y_2^{(n)}({\Delta }_n))\hfill \\ & =\underset{n\to \infty }{lim}I(M_2;\Delta Y_2^{(n)}({\Delta }_n))\hfill \\ & =\underset{n\to \infty }{lim}h(\Delta Y_2^{(n)}({\Delta }_n))-h(\Delta Y_2^{(n)}({\Delta }_n)|M_2),\hfill \end{array}$$

where $\Delta Y_2^{(n)}({\Delta }_n)\triangleq \{Y_2^{(n)}(t_{n,i})-Y_2^{(n)}(t_{n,i-1}):i=1,2,\cdots ,n\}$. Note that

$$H(\Delta Y_2^{(n)}({\Delta }_n))\le \sum _{i=1}^n\log (2\pi e(P{\delta }_n^2+N_2{\delta }_n)),$$

and

$$\begin{array}{cc}\hfill H(\Delta Y_2^{(n)}({\Delta }_n)|M_2)& =\sum _{i=1}^{n}h(Y_2^{(n)}(t_{n,i})-Y_2^{(n)}(t_{n,i-1})|Y_{2,t_{n,0}}^{(n),t_{n,i-1}},M_2)\hfill \\ & \ge \sum _{i=1}^{n}h(\sqrt{N_2}{B}_2(t_{n,i})-\sqrt{N_2}{B}_2(t_{n,i-1}))\hfill \\ & =\sum _{i=1}^n\log (2\pi e{N}_2{\delta }_n),\hfill \end{array}$$

which implies that there exists an $\alpha \in [0,1]$ such that

$$h(\Delta Y_2^{(n)}({\Delta }_n)|M)=\sum _{i=1}^n\frac{n}{2}\log (2\pi e(\alpha P{\delta }_n^2+N_2{\delta }_n)).$$

It then follows from Theorem 3 that

$$I(M_2;Y_{2,0}^T)\le \frac{1}{2}\underset{n\to \infty }{lim}\sum _{i=1}^n\log \frac{P{\delta }_n^2+N_2{\delta }_n}{\alpha P{\delta }_n^2+N_2{\delta }_n}=\frac{(1-\alpha )PT}{2N_2}.$$

Next we consider

$$\begin{array}{cc}\hfill I(M_1;Y_1^{(n)}({\Delta }_n)|M_2)& =h(Y_1^{(n)}({\Delta }_n)|M_2)-h(Y_1^{(n)}({\Delta }_n)|M_1,M_2)\hfill \\ & =h(Y_1^{(n)}({\Delta }_n)|M_2)-\sum _{i=1}^{n}h(Y_1^{(n)}(t_{n,i})|M_1,M_2,Y_{1,t_{n,0}}^{(n),t_{n,i-1}})\hfill \\ & \le h(Y_1^{(n)}({\Delta }_n)|M_2)-\sum _{i=1}^{n}h(Y_1^{(n)}(t_{n,i})|M_1,M_2,Y_{1,t_{n,0}}^{(n),t_{n,i-1}},Y_{1,t_{n,0}}^{(n),t_{n,i-1}})\hfill \\ & =h(Y_1^{(n)}({\Delta }_n)|M_2)-\frac{1}{2}\sum _{i=1}^n\log (2\pi e{N}_1{\delta }_n).\hfill \end{array}$$

Now, using Lemma 1 in [50] (an extension of the entropy power inequality), we obtain

$$\begin{array}{c}\hfill h(Y^{(n)}({\Delta }_n)|M_2)\ge \frac{n}{2}\log (2^{2h(Y_1^{(n)}({\Delta }_n))|M_2)/n}+2\pi e(N_2-N_1){\delta }_n),\end{array}$$

which immediately implies that

$$h(Y_{1,t_{n,0}}^{(n),t_{n,i}}|M_2)\le \frac{1}{2}\sum _{i=1}^n\log (2\pi e(\alpha P{\delta }_n^2+N_1{\delta }_n))$$

and furthermore, by Theorem 3,

$$\begin{array}{cc}\hfill I(M_1;Y_{1,0}^T|M_2)& \le \underset{n\to \infty }{lim}\frac{1}{2}\sum _{i=1}^n\log (2\pi e(\alpha P{\delta }_n^2+N_1{\delta }_n))-\frac{1}{2}\sum _{i=1}^n\log (2\pi e{N}_1{\delta }_n)\hfill \\ & =\underset{n\to \infty }{lim}\frac{1}{2}\sum _{i=1}^n\log \left(1+\frac{\alpha P{\delta }_n}{N_1}\right)\hfill \\ & =\frac{\alpha PT}{2N_1}.\hfill \end{array}$$

Now, by Theorem 10, we conclude that feedback capacity region is exactly the same non-feedback capacity region; in other words, feedback does not increase the capacity region of a physically degraded continuous-time Gaussian BC. ☐

The following theorem says that feedback does help some stochastically degraded Gaussian BCs, whose proof, instead of directly employing the approximation theorem, uses the connections between continuous-time and discrete-time Gaussian channels and the notion of continuous-time directed information in Remark 2, both of which can find their source from the approximation theorem. We only provide the sketch of the proof, since it is largely based on a translated version of the argument in [51].

Theorem 12.

Feedback increases the capacity region of certain continuous-time stochastically degraded Gaussian broadcast channel.

Proof. 

Consider the following symmetric continuous-time Gaussian broadcast channel:

$$Y_1(t)={\int }_0^{t}X(s)ds+B_1(t),$$

$$Y_2(t)={\int }_0^{t}X(s)ds+B_2(t),$$

where $B_1,B_2$ are independent standard Brownian motions, and X satisfies the average power constraint P. By Theorem 10, without feedback, the capacity region is the set of rate pairs $(R_1,R_2)$ such that

$$R_1+R_2\le \frac{P}{2}.$$

(45)

With feedback, one can use the following variation [51] of the Schalkwijk-Kailath coding scheme [38] over $[0,T]$ at discrete-time points $\left\{t_{n,i}\right\}$ that form an evenly spaced ${\Delta }_n$ of stepsize ${\delta }_n$: For the channel input, after some proper initialization, at time $t\in [t_{n,i},t_{n,i+1})$, we send $X^{(n)}(t)=X_1^{(n)}(t)+X_2^{(n)}(t)$, where

$$X_1^{(n)}(t)={\gamma }_i(X_1^{(n)}(t_{n,i-1})-\mathbb{E}[X_1^{(n)}(t_{n,i-1})|Y_1^{(n)}(t_{n,i-1})]),$$

$$X_2^{(n)}(t)=-{\gamma }_i(X_2^{(n)}(t_{n,i-1})-\mathbb{E}[X_2^{(n)}(t_{n,i-1})|Y_2^{(n)}(t_{n,i-1})]),$$

where ${\gamma }_i$ is chosen so that $\mathbb{E}[X_i^2(t)]=P$ for each i; and for the channel outputs, we have, for any $t\in [t_{n,i},t_{n,i+1}]$,

$$Y_1^{(n)}(t)=Y_1^{(n)}(t_{n,i})+{\int }_{t_{n,i}}^t{X}^{(n)}(t_{n,i})ds+B_1(t)-B_1(t_{n,i}),$$

(46)

and

$$Y_2^{(n)}(t)=Y_2^{(n)}(t_{n,i})+{\int }_{t_{n,i}}^t{X}^{(n)}(t_{n,i})ds+B_2(t)-B_2(t_{n,i}).$$

(47)

Going through a completely parallel argument as in [51] and capitalizing on the fact that the SNR in the channels (46) and (47) tend to 0 as n tends to infinity, we derive that

$$\underset{n\to \infty }{lim}I(X^{(n)}({\Delta }_n)\to Y_1^{(n)}({\Delta }_n))=\frac{1}{2}\sum _{n\to \infty }\sum _{i=1}^n\frac{\log \left(1+\frac{P{\delta }_n(1+{\rho }^{*})/2}{(1+P{\delta }_n(1-{\rho }^{*}))/2}\right)}{T}=\frac{P(1+{\rho }^{*})}{4},$$

and parallelly,

$$\underset{n\to \infty }{lim}I(X^{(n)}({\Delta }_n)\to Y_2^{(n)}({\Delta }_n))=\frac{P(1+{\rho }^{*})}{4},$$

where ${\rho }^{*}>0$ satisfies the condition

$${\rho }^{*}(1+(P+1)(1+P(1-{\rho }^{*})/2))=\frac{P(P+2)(1-{\rho }^{*})}{2}.$$

Note that, by Remark 2, we have

$$I(X_0^{(n),T}\to Y_{1,0}^{(n),T})\ge I(X^{(n)}({\Delta }_n)\to Y_1^{(n)}({\Delta }_n)),I(X_0^{(n),T}\to Y_{2,0}^{(n),T})\ge I(X^{(n)}({\Delta }_n)\to Y_2^{(n)}({\Delta }_n)),$$

which immediately implies that

$$R_1=R_2=\frac{P(1+{\rho }^{*})}{4}$$

(48)

are achievable. The claim that feedback strictly increases the capacity region then follows from (45), and (48) and the fact that ${\rho }^{*}>0$. ☐

6. Conclusions and Future Work

For a continuous-time white Gaussian channel without feedback, the classical Shannon–Nyquist sampling theorem can convert it to a discrete-time Gaussian channel; however such a link has long been missing when feedback/memory is present in the channel. In this paper, we establish sampling and approximation theorems as the missing links, which we believe will play important roles in the long run for further developing continuous-time information theory, particularly for the communication scenarios where feedback/memory is present.

As an immediate application of our approximation theorem, we propose the approximation approach, an analog of the conventional sampling approach, for Gaussian feedback channels. It turns out that, like its non-feedback counterpart, the approximation approach can bring insights and intuition to investigation of continuous-time Gaussian channels with possible feedback, and moreover, when complemented with relevant tools from stochastic calculus, can deliver rigorous treatments in the point-to-point or multi-user setting.

On the other hand, though, there are many questions that remain unanswered and a number of directions that need to be further explored. Below we list a number of research directions that look promising in the near future.

(1) The first direction is to strengthen and generalize our sampling and approximation theorems.

Note that both Theorems 2 and 3 require Conditions (d)–(f), which are stronger than the typical average power constraint. While Conditions (d)–(f) are rather mild for practical considerations, the stronger assumptions in our theorems will narrow their reach in some theoretical situations. For instance, despite the fact that our approximation theorem gives intuitive explanations to the rigorous treatment of continuous-time multi-user Gaussian channels in Section 5, it fails to rigorously establish Theorems 7 and 10. The stochastic calculus approach employed in Section 5 requires only the power constraints, which can be loosely explained by the fact that Girsanov’s theorem (or, more precisely, its several variants) only requires as weak conditions. It is certainly worthwhile to explore whether the assumptions in our sampling and approximation theorems can be relaxed either in general or for some special settings.

Another topic in this direction is the rate of convergence in the sampling and approximation theorems. While the current versions of our theorems have merely established some limits, the rate of convergence will certainly yield a more quantitative description of how fast those limits will be approached.

One can also consider generalizing these two theorems to general Gaussian channels [52,53]. For this topic, note that there exist in-depth studies [54,55,56,57,58,59,60,61] on continuous-time point-to-point general Gaussian channels with possible feedback, for which information-theoretic connections with the discrete-time setting are somehow lacking. A first step in this direction can be establishing sampling or approximation theorems for stationary Gaussian processes. Obviously, such theorems for stationary Gaussian processes can connect continuous-time stationary Gaussian channels to their discrete-time counterparts, for which the variational formulation of discrete-time stationary Gaussian feedback capacity in [62] proves to rather effective.

(2) The second direction is to further explore the possible applications of our sampling and approximation theorems in the following respects.

We have shown that feedback may increase the capacity region of some continuous-time Gaussian BC, but the capacity regions of such channels remain unknown in general. An immediate problem is to explicitly find the exact capacity regions of continuous-time Gaussian BCs using the approach employed in this work, as we have done for continuous-time Gaussian MACs. Of course, further topics also include exploring whether the ideas and techniques in this paper can be applied to other families of continuous-time multi-user Gaussian channels with possible feedback.

So far we have implicitly assumed infinite bandwidth and average power constraints, but our theorems can certainly go beyond these assumptions. For instance, one can consider examining continuous-time Gaussian channels with both bandwidth limit and peak power constraint, which are more reasonable assumptions for many practical communication scenarios as they give a more accurate description of the limitations of the communication system. Little is known about the capacity of continuous-time Gaussian channels with such constraints except some upper and lower bounds established in [63,64]. In stark contrast, discrete-time peak power constrained channels (including, but not limited to Gaussian channels) have been better investigated: there has been a series of work on their capacity, such as [65,66,67,68,69,70,71,72], which feature relatively thorough discussions about different aspects of channel capacity including capacity achieving distribution, bounds and asymptotics of capacity, and numerical computation of capacity. An immediate question is to explore whether the approximation approach can translate the aforementioned existing results in discrete time, or more probably, help channel the ideas and techniques therein to the continuous-time setting. A next question is to explore whether there exists any randomized algorithm for computation of the capacity of such a channel, for which, as discussed in Remark 4, we believe our sampling theorems can be particularly helpful in terms of numerically computing and optimizing the mutual information of a continuous-time Gaussian channel with bandwidth limit and peak power constraint.