Analysis of Deep Convolutional Neural Networks Using Tensor Kernels and Matrix-Based Entropy

Abstract

Analyzing deep neural networks (DNNs) via information plane (IP) theory has gained tremendous attention recently to gain insight into, among others, DNNs’ generalization ability. However, it is by no means obvious how to estimate the mutual information (MI) between each hidden layer and the input/desired output to construct the IP. For instance, hidden layers with many neurons require MI estimators with robustness toward the high dimensionality associated with such layers. MI estimators should also be able to handle convolutional layers while at the same time being computationally tractable to scale to large networks. Existing IP methods have not been able to study truly deep convolutional neural networks (CNNs). We propose an IP analysis using the new matrix-based Rényi’s entropy coupled with tensor kernels, leveraging the power of kernel methods to represent properties of the probability distribution independently of the dimensionality of the data. Our results shed new light on previous studies concerning small-scale DNNs using a completely new approach. We provide a comprehensive IP analysis of large-scale CNNs, investigating the different training phases and providing new insights into the training dynamics of large-scale neural networks.

1. Introduction

Although deep neural networks (DNNs) are at the core of most state-of-the art systems in computer vision, the theoretical understanding of such networks is still not at a satisfactory level [1]. In order to provide insight into the inner workings of DNNs, the prospect of utilizing the mutual information (MI), a measure of dependency between two random variables, has recently garnered a significant amount of attention [1,2,3,4,5,6,7,8]. Given the input variable X and the desired output Y for a supervised learning task, a DNN is viewed as a transformation of X into a representation that is favorable for obtaining a good prediction of Y. By treating the output of each hidden layer as a random variable T, one can model the MI $I(X;T)$ between X and T. Likewise, the MI $I(T;Y)$ between T and Y can be modeled. The quantities $I(X;T)$ and $I(T;Y)$ span what is referred to as the information plane (IP). Several works have unveiled interesting properties of the training dynamics through IP analysis of DNNs [4,7,9,10,11]. Figure 1, produced using our proposed measure, illustrates one such insight that is similar to the observations of [1], where training can be separated into two distinct phases: the fitting phase and the compression phase.

The claim of a fitting and compression phase has been highly debated as subsequent research has linked the compression phase to the saturation of neurons [5] or clustering of the hidden representations [9]. Recent studies [1,5,7,9] have been focused on small-scale networks or non-convolutional networks, as the current MI estimators cannot tackle the tensor representations produced by convolutional layers or do not scale well to convolutional layers with many filters [6]. This severely limits the scope of IP analysis, as most real-world applications rely on large-scale CNNs. In this work, we continue the IP line of research through a new matrix-based entropy functional [6,7,12]. We provide insights on this functional by linking the functional to well-understood measures from the kernel literature and propose a new kernel tensor-based approach to the matrix-based entropy functional. Furthermore, we give a new formulation of the matrix-based entropy that is closely connected to measures from quantum information theory. Using the proposed estimator, we provide an analysis of large-scale DNNs and give a new information theoretic understanding of the training procedure of DNNs. Using the proposed measure, we investigate the claim of [3] that the entropy $H\left(X\right)\approx I(T;X)$ and $H\left(Y\right)\approx I(T;Y)$ in high dimensions (in which case MI-based analysis would be meaningless). The contributions of this work can be summarized as:

• We propose a kernel tensor-based approach to the matrix-based entropy functional that is designed for measuring MI in large-scale convolutional neural networks (CNNs).
• We provide new insights on the matrix-based entropy functional by showing its connection to well-known quantities in the kernel literature such as the kernel mean embedding and maximum mean discrepancy. Furthermore, we show that the matrix-based entropy functional is closely linked with the von Neuman entropy from quantum information theory.
• Our results indicate that the compression phase is apparent mostly for the training data and less so for the test data, particularly for more challenging datasets. When using a technique such as early stopping to avoid overfitting, training tends to stop before the compression phase occurs (see Figure 1).

2. Related Work

Analyzing DNNs in the IP was first proposed by [13] and later demonstrated by [1]. Among other results, the authors studied the evolution of the IP during the training process of DNNs and noted that the process was composed of two different phases. First, there is an initial fitting phase where $I(T;Y)$ increases, which is followed by a phase of compression where $I(X;T)$ decreases. These results were later questioned by [5], who argued that the compression phase is not a general property of the DNN training process but rather an effect of different activation functions. However, a recent study by [4] seems to support the claim of a compression phase regardless of the activation function. The authors argue that the base estimator of MI utilized by [5] might not be accurate enough and demonstrate that a compression phase does occur, but the amount of compression can vary between different activation functions. Another recent study by [10] also reported a compression phase but highlighted the importance of adaptive MI estimators. They also showed that when L2 regularization was included in the training, compression was observed regardless of the activation function. In addition, some recent studies have discussed the limitations of the IP framework for analysis and optimization for particular types of DNN [14,15]. Furthermore, ref. [16] investigated similarities between hidden layers and between hidden layers of different networks, but they did so only for the representation obtained after the networks were fully trained. The dynamics of large-scale DNNs was investigated [17] using MINE [18]. A number of information plane-related studies have also been discussed in [19].

On a different note, Ref. [3] proposed an evaluation framework for DNNs based on the IP and demonstrated that MI can be used to infer the capability of DNNs to recognize objects for an image classification task. Furthermore, the authors argue that when the number of neurons in a hidden layer grows large, $I(T;X)$ and $I(Y;T)$ barely change and are, using [3] terminology, approximately deterministic, i.e., $I(T;X)\approx H\left(X\right)$ and $I(T;Y)\approx H\left(Y\right)$. Therefore, they only model the MI between X and the last hidden layer—that is, the output of the network—and the last hidden layer and Y.

Ref. [7] investigated a matrix-based measure of MI for analyzing different data processing inequalities in feed-forward stacked autoencoders (SAEs), concluding that the compression phase in the IP of SAEs is determined by the values of the SAE bottleneck layer size and the intrinsic dimensionality of the given data. In follow-up work, a simplistic analysis of small convolutional neural networks (CNNs) was provided in [6]; however, it was based on a multivariate extension [20] of matrix-based Renyi entropy that does not scale well numerically or computationally in the number of feature maps. The interested reader can find additional information on the multivariate extension by [20] in Appendix A.

Recently, the information plane of quantized neural networks was modeled [8], which allowed for an exact analysis of its dynamics. In addition, log-determinant entropy has been used for information plane analysis [11]. Lastly, information plane analysis has also been used to improve the understanding of graph convolutional neural networks [21].

In this paper, we continue the recent trend of leveraging the new matrix-based measures of entropy. We contribute both new insight to the definition of these measures (see Section 3.2.1), and importantly, we extend matrix-based measures of entropy to exploit tensor kernels to enable the first IP analysis of large-scale CNNs by treating feature maps as tensors (Section 3.3).

3. Materials and Methods

3.1. Preliminaries on Matrix-Based Information Measures

For the benefit of the reader, we review in this section first the theory underlying the recent matrix-based measures of entropy and mutual information. Thereafter, we contribute a new special case interpretation of the definition of matrix-based Renyi entropy (Section 3.2.1) and give new insights on the link to other measures in the kernel literature before presenting our new tensor-based approach.

3.1.1. Matrix-Based Entropy and Mutual Information

The matrix-based measure of entropy, originally proposed by [12], is built on kernel matrices obtained from raw data, involving no explicit density estimation or binning procedure:

Definition 1 

([12]). Let ${\mathbf{x}}_i\in \mathcal{X},i=1,2,\dots ,N$ denote data points and let $\kappa :\mathcal{X}\times \mathcal{X}\mapsto \mathbb{R}$ be an infinitely divisible positive definite kernel [22]. Given the kernel matrix $\mathbf{K}\in {\mathbb{R}}^{N\times N}$ with elements ${\left(\mathbf{K}\right)}_{ij}=\kappa ({\mathbf{x}}_i,{\mathbf{x}}_j)$ and the matrix $\mathbf{A},{\left(\mathbf{A}\right)}_{ij}=\frac{1}{N}\frac{{\left(\mathbf{K}\right)}_{ij}}{\sqrt{{\left(\mathbf{K}\right)}_{ii}{\left(\mathbf{K}\right)}_{jj}}}$, the matrix-based Rényi’s α-order entropy is given by

$$\begin{array}{cc}\hfill S_{\alpha }\left(\mathbf{A}\right)& =\frac{1}{1-\alpha }{log}_2\left(tr\left({\mathbf{A}}^{\alpha }\right)\right)\hfill \\ \hfill & =\frac{1}{1-\alpha }{log}_2\left[\sum _{i=1}^N{\lambda }_i{\left(\mathbf{A}\right)}^{\alpha }\right].\hfill \end{array}$$

(1)

Here, ${\lambda }_i\left(\mathbf{A}\right)$ denotes the ith eigenvalue of the matrix $\mathbf{A}$. Equation (1) is a measure of an entropy-like quantity that satisfies Renyi’s axiomatic characterization of entropy [23], which is referred to as matrix-based Renyi entropy. In addition to the matrix-based entropy, ref. [12] also defined the matrix-based joint entropy between $\mathbf{x}\in \mathcal{X}$ and $\mathbf{y}\in \mathcal{Y}$ as

$$S_{\alpha }({\mathbf{A}}_{\mathcal{X}},{\mathbf{A}}_{\mathcal{Y}})=S_{\alpha }\left(\frac{{\mathbf{A}}_{\mathcal{X}}\circ {\mathbf{A}}_{\mathcal{Y}}}{\mathrm{tr}({\mathbf{A}}_{\mathcal{X}}\circ {\mathbf{A}}_{\mathcal{Y}})}\right),$$

(2)

where ${\mathbf{x}}_i$ and ${\mathbf{y}}_i$ are two different representations of the same object and ∘ denotes the Hadamard product. Finally, the MI is, similar to Shannon’s formulation, defined as

$$I_{\alpha }({\mathbf{A}}_{\mathcal{X}};{\mathbf{A}}_{\mathcal{Y}})=S_{\alpha }\left({\mathbf{A}}_{\mathcal{X}}\right)+S_{\alpha }\left({\mathbf{A}}_{\mathcal{Y}}\right)-S_{\alpha }({\mathbf{A}}_{\mathcal{X}},{\mathbf{A}}_{\mathcal{Y}}).$$

(3)

The properties of these quantities were analyzed in detail in [12], but we want to highlight some important properties and provide links with other measures in the kernel literature.

Information theoretic measures are developed in such a way that they satisfy certain axioms. The measures presented above satisfy the axioms put forth by [23] and are closely connected with quantum information theory [24]. In quantum statistical mechanics, the Von Neumann’s entropy [24] is defined as

$$S\left(\rho \right)=-tr(\rho log\rho ),$$

(4)

where $\rho$ is a density matrix that described a quantum mechanical system. If $\rho$ is written in terms of its eigenvalues, ${\lambda }_i\left(\rho \right)$, then Equation (4) can also be formulated as

$$S\left(\rho \right)=-\sum _{i=1}^N{\lambda }_i\left(\rho \right)log\left[{\lambda }_i\left(\rho \right)\right].$$

(5)

In addition, the quantum extensions of Renyi’s entropy [25] that is defined as

$$S_{\alpha }\left(\rho \right)=\frac{1}{1-\alpha }log[tr\left({\rho }^{\alpha }\right)],$$

(6)

bears a close resemblance to the matrix-based definition of entropy in Equation (1). While some properties of Equations (4) and (6) can also be extended to Equation (1), it is important to note that the two approaches are very different since the matrix-based framework is built around kernel matrices obtained directly from raw data.

3.1.2. Bound on Matrix-Based Entropy Measure

Not all measures of entropy have the same properties. Many of the estimators used and developed for Shannon suffer from the curse of dimensionality [26]. In contrast, Renyi’s entropy measures have the same functional form of the statistical quantity in a reproducing kernel Hilbert space (RKHS), thus capturing properties of the data population. Essentially, we are projecting marginal distribution to an RKHS in order to measure entropy and MI. This is similar to the approach of maximum mean discrepancy and the kernel mean embedding [27,28]. The connection with the data population can be shown via the theory of covariance operators. The covariance operator $G:\mathcal{H}\to \mathcal{H}$ is defined through the bilinear form

$$\begin{array}{cc}\hfill \mathcal{G}(f,g)=〈f,Gg〉& ={\int }_{\mathcal{X}}〈f,\psi \left(\mathbf{x}\right)〉〈\psi \left(\mathbf{x}\right),g〉d{\mathbb{P}}_{\mathcal{X}}\left(\mathbf{x}\right)\hfill \\ \hfill & =E_{\mathcal{X}}\left\{f\left(X\right),g\left(Y\right)\right\}\hfill \end{array}$$

(7)

where ${\mathbb{P}}_{\mathcal{X}}$ is a probability measure and $f,g\in \mathcal{H}$. Based on the empirical distribution ${\mathbb{P}}_N=\frac{1}{N}{\sum }_{i=1}^N{\delta }_{{\mathbf{x}}_i}\left(\mathbf{x}\right)$, the empirical version $\widehat{G}$ of G obtained from a sample ${\mathbf{x}}_i$ of size N is given by:

$$\begin{array}{cc}\hfill 〈f,{\widehat{G}}_{Ng}〉=\widehat{\mathcal{G}}(f,g)& ={\int }_{\mathcal{X}}〈f,\psi \left(\mathbf{x}\right)〉〈\psi \left(\mathbf{x}\right),g〉d{\mathbb{P}}_{\mathcal{X}}\left(\mathbf{x}\right)\hfill \\ \hfill & =\frac{1}{N}\sum _{i=1}^N〈f,\psi \left({\mathbf{x}}_i\right)〉〈\psi \left({\mathbf{x}}_i\right),g〉\hfill \end{array}$$

(8)

By analyzing the spectrum of $\widehat{G}$ and G, Ref. [12] showed that the difference between $tr\left(G\right)$ and $tr\left(\widehat{G}\right)$ can be bounded, as stated in the following proposition:

Proposition 2. 

Let ${\mathbb{P}}_N=\frac{1}{N}{\sum }_{i=1}^N{\delta }_{{\mathbf{x}}_i}\left(\mathbf{x}\right)$ be the empirical distribution. Then, as a consequence of Proposition 6.1 in [12], $tr\left[{\widehat{G}}_N^{\alpha }\right]=tr\left[{\left(\frac{1}{N}\mathbf{K}\right)}^{\alpha }\right]$. The difference between $tr\left(G\right)$ and $tr\left(\widehat{G}\right)$ can be bounded under the conditions of Theorem 6.2 in [12] and for $\alpha >1$, with probability 1-δ

$$\left|tr\left(G^{\alpha }\right)-tr\left({\widehat{G}}_N^{\alpha }\right)\right|\le \alpha C\sqrt{\frac{2log\frac{2}{\delta }}{N}}$$

(9)

where C is a compact self-adjoint operator.

3.2. Analysis of Matrix-Based Information Measures

In this section, we present new theoretical insights into the Matrix-Based Information Measures.

3.2.1. A New Special-Case Interpretation of the Matrix-Based Renyi Entropy Definition

In previous works, the $\alpha$ in Equation (1) has been ad hoc set to a value of 1.01 in order to approximate Shannon’s entropy [6,7]. For $\alpha =1$, both the denominator and the numerator become zero, so Equation (1) cannot be used directly in this case. However, as a contribution to the matrix-based Renyi entropy theory, we show here that for the case $\alpha \to 1$, Equation (1) can be expressed similarly to the matrix-based Von Neumann’s entropy [24], resembling Shannon’s definition over probability states and expressed as

$$\underset{\alpha \to 1}{lim}{S}_{\alpha }\left(\mathbf{A}\right)=-\sum _{i=1}^N{\lambda }_i\left(\mathbf{A}\right){log}_2\left[{\lambda }_i\left(\mathbf{A}\right)\right].$$

(10)

Equation (1) can be proved using L’Hôpital’s rule as follows:

Proof. 

$$\underset{\alpha \to 1}{lim}{S}_{\alpha }\left(\mathbf{A}\right)=\underset{\alpha \to 1}{lim}\frac{1}{1-\alpha }{log}_2\left(\sum _{i=1}^n{\lambda }_i^{\alpha }\right)\to \frac{0}{0},$$

(11)

since ${\sum }_{i=1}^N{\lambda }_i=tr\left(\mathbf{A}\right)=1$. L’Hôpital’s rule yields

$$\begin{array}{cc}\hfill \underset{\alpha \to 1}{lim}{S}_{\alpha }\left(\mathbf{A}\right)& =\underset{\alpha \to 1}{lim}\frac{\frac{\partial }{\partial \alpha }{log}_2\left[{\sum }_{i=1}^n{\lambda }_i{\left(\mathbf{A}\right)}^{\alpha }\right]}{\frac{\partial }{\partial \alpha }(1-\alpha )}\hfill \\ \hfill & =-\frac{1}{ln2}\underset{\alpha \to 1}{lim}\frac{{\sum }_{i=1}^n{\lambda }_i{\left(\mathbf{A}\right)}^{\alpha }ln\left[{\lambda }_i\left(\mathbf{A}\right)\right]}{|{\sum }_{i=1}^n{\lambda }_i{\left(\mathbf{A}\right)}^{\alpha }|}\hfill \\ \hfill & =-\sum _{i=1}^n{\lambda }_i\left(\mathbf{A}\right){log}_2\left[{\lambda }_i\left(\mathbf{A}\right)\right].\hfill \end{array}$$

(12)

□

3.2.2. Link to Measures in Kernel Literature and Validation on High-Dimensional Synthetic Data

An interesting aspect of the matrix-based measure of entropy is the special case connection with the theory of maximum mean discrepancy and Hilbert–Schmidt norms via covariance operators [29]. Let G be the covariance operator (see [28] for details), then, for the particular case of $\alpha =2$, the empirical trace of the covariance operator, $tr\left(G^2\right)$, is given by $tr\left({\mathbf{A}}^2\right)$. Furthermore,

$$\begin{array}{cc}\hfill tr\left(G^2\right)& =〈{\int }_{\mathcal{X}}\kappa (·,\mathbf{x})d{\mathbb{P}}_{\mathcal{X}}\left(\mathbf{x}\right),{\int }_{\mathcal{X}}\kappa (·,\mathbf{y})d{\mathbb{P}}_{\mathcal{X}}\left(\mathbf{y}\right),〉\hfill \\ \hfill & =\left|\right|{\mu }_{\mathcal{X}}{\left|\right|}_{\mathcal{K}}^2,\hfill \end{array}$$

(13)

where ${\mu }_{\mathcal{X}}={\int }_{\mathcal{X}}\kappa (·,\mathbf{x})d{\mathbb{P}}_{\mathcal{X}}\left(\mathbf{x}\right)$ is the kernel mean map [30], i.e., an embedding of the probability measure ${\mathbb{P}}_{\mathcal{X}}\left(\mathbf{x}\right)$ in a reproducing kernel Hilbert space (RKHS). Thus, the matrix $\mathbf{A}$ can be related to an empirical covariance operator on embeddings of probability distributions in an RKHS. Moreover, ref. [12] showed that under certain conditions, Equation (1) converges to the trace of the underlying covariance operator, as shown in Proposition 2 in Section 3.1.2. Notice that the dimensionality of the data does not appear in Proposition 2. This means that $S_{\alpha }\left(\mathbf{A}\right)$ captures properties of the distribution with a certain robustness with respect to high-dimensional data. This is a beneficial property compared to KNN and KDE-based information measures used in previous works [5,10], which have difficulties handling high-dimensional data [26]. Some measures of entropy developed for measuring the Shannon entropy suffer from the curse of dimensionality [26]. In addition, there is no need for any binning procedure utilized in previous works [1], which are known to struggle with the ReLU activation function commonly used in DNNs [5]. While Equation (13) is not explicitly used in the remainder of our manuscript, we believe that these insights provides a deeper understanding of the inner workings of the matrix-based entropy measure.

To examine the behavior of the matrix-based measures described in Section 3.1, we have conducted a simple experiment on measuring entropy and mutual information in high-dimensional data following a normal distribution with known mean and covariance matrix. In the particular case of the normal distribution, the entropy and mutual information can be calculated analytically. The entropy can be calculated as:

$$H\left({\mathcal{N}}_0\right)=\frac{1}{2}log\left({\left(2\pi e\right)}^{d}det\left({\mathbf{\Sigma }}_0\right)\right),$$

(14)

where ${\mathcal{N}}_0$ denotes a normal distribution with mean vector ${\mathit{\mu }}_0$ covariance matrix ${\mathbf{\Sigma }}_0$, and dimensionality d. For mutual information, we use the experimental setup considered in [31]. Let Z have a $d+1$ dimensional Gaussian distribution with covariance matrix ${\mathbf{\Sigma }}_z$. Next, let $X=(Z_1,\dots ,Z_d)$ and $Y=Z_{d+1}$. Then, their mutual information satisfies:

$$\begin{array}{cc}I(X,Y)& =I(Z_1,\dots ,Z_{d+1})-I(Z_1,\dots ,Z_d)\hfill \end{array}$$

(15)

$$\begin{array}{cc}& =-\frac{1}{2}log\left(\frac{det\left({\mathbf{\Sigma }}_z\right)}{det\left({\mathbf{\Sigma }}_x\right)}\right),\hfill \end{array}$$

(16)

where ${\mathbf{\Sigma }}_x$ is the covariance matrix of X. We consider five cases of $(d+1)$-dimensional Gaussian distributions with mean zero, unit variance, and an increasing amount of dependence. The unit variance covariance matrices for the five cases are given as follows:

$$\left\{\begin{array}{cc}{\mathbf{\Sigma }}_A(i,j)=0.1,\hfill & \mathrm{for}i\ne j,\hfill \\ {\mathbf{\Sigma }}_B(i,j)=0.25,\hfill & \mathrm{for}i\ne j,\hfill \\ {\mathbf{\Sigma }}_C(i,j)=0.5,\hfill & \mathrm{for}i\ne j,\hfill \\ {\mathbf{\Sigma }}_D(i,j)=0.75,\hfill & \mathrm{for}i\ne j,\hfill \\ {\mathbf{\Sigma }}_E(i,j)=0.9,\hfill & \mathrm{for}i\ne j.\hfill \end{array}\right.$$

The left plot in Figure 2 displays the entropy of a 100-dimensional normal distribution with zero mean and isotropic covariance matrix, which is calculated using Equation (14). The right plot in Figure 2 displays the estimated entropy using Equation (1) on 500 randomly drawn samples from ${\mathcal{N}}_0$, which are computed on all 500 samples and in batches of 100. The results show how the estimated entropy follows the same trend as the analytically computed entropy. We quantitativly evaluate the correlation between the analytic quantity and the estimated quantity by calculating Pearson’s correlation coefficient and find that that both the full data and batch-wise estimation are highly correlated with the analytic calculation (correlation $\approx 0.99$, p-value $\le 0.01$). For mutual information, we generate 500 samples from a 100 + 1 dimensional Gaussian distribution and compare the exact and estimated mutual information, which are both based on all 500 samples and in the batch-wise setting. The left plot of Figure 3 shows the mutual information between X and Y calculated using Equation (16) for the five cases described above. The right part of Figure 3 shows the estimated mutual information using Equation (3), which is computed on all 500 samples and in batches of 100. Again, the result shows how the estimated values follow the same trend as the exact mutual information values. Similarly as with the entropy, we quantitativly evaluate the correlation between the analytic quantity and the estimated quantity by calculating Pearson’s correlation coefficient and find that that both the full data and batch-wise estimation are highly correlated with the analytic calculation (correlation $\approx 0.99$, p-value $\le 0.01$). Note that the exact value of both the entropy and mutual information is different between the exact and estimated quantities. This is expected, as the matrix-based entropy estimators measure information theoretic quantities in RKHS without the need for explicit PDF but with similar properties as common information theoretic measures. The kernel width was selected by taking the median distance between all samples.

3.3. Novel Tensor-Based Matrix-Based Renyi Information Measures

To invoke information theoretic quantities of features produced by convolutional layers and to address the limitations discussed above, we introduce in this section our novel tensor-based approach for measuring entropy and MI in DNNs. This enables for the first time an IP analysis of large-scale CNNs.

3.4. Tensor Kernels for Measuring Mutual Information

The output of a convolutional layer is represented as a tensor ${\mathbb{X}}_i\in {\mathbb{R}}^C\otimes {\mathbb{R}}^H\otimes {\mathbb{R}}^W$ for a data point i. As discussed above, the matrix-based Rényi’s $\alpha$-entropy cannot include tensor data without modifications. To handle the tensor-based nature of convolutional layers, we propose to utilize tensor kernels [32] to produce a kernel matrix for the output of a convolutional layer. A tensor formulation of the radial basis function (RBF) kernel can be stated as

$${\kappa }_{\mathrm{ten}}({\mathbb{X}}_i,{\mathbb{X}}_j)=e^{-\frac{1}{{\sigma }^2}{\parallel {\mathbb{X}}_i-{\mathbb{X}}_j\parallel }_F^2},$$

(17)

where ${\parallel ·\parallel }_F$ denotes the Hilbert–Frobenius norm [32] and $\sigma$ is the kernel width parameter. In practice, the tensor kernel in Equation (17) can be computed by reshaping the tensor into a vectorized representation while replacing the Hilbert–Frobenius norm with a Euclidean norm. We compute the MI in Equation (3) by replacing the matrix $\mathbf{A}$ with

$$\begin{array}{cc}\hfill {\left({\mathbf{A}}_{\mathrm{ten}}\right)}_{ij}& =\frac{1}{N}\frac{{\left({\mathbf{K}}_{\mathrm{ten}}\right)}_{ij}}{\sqrt{{\left({\mathbf{K}}_{\mathrm{ten}}\right)}_{ii}{\left({\mathbf{K}}_{\mathrm{ten}}\right)}_{jj}}}\hfill \\ \hfill & =\frac{1}{N}{\kappa }_{\mathrm{ten}}({\mathbb{X}}_i,{\mathbb{X}}_j).\hfill \end{array}$$

(18)

While Equation (17) provides the simplest and most intuitive approach for using kernels with tensor data, it does have its limitations. Namely, a tensor kernel that simply vectorizes the tensor ignores the inter-component structures within and between the respective tensor [32]. For simple tensor data, such structures might not be present and a tensor kernel as described above can suffice; however, other tensor kernels do exist, such as for instance the matricization-based tensor kernels [32]. In this work, we have chosen the tensor kernel defined in Equation (17) for its simplicity and computational benefits, which come from the fact that the entropy and joint entropy are computed batch-wise by finding the eigenvalues of a kernel matrix, or the eigenvalues of the Hadamard product of two kernel matrices, and utilizing Equation (1). Nevertheless, exploring structure-preserving kernels can be an interesting research path in future works. In Appendix C, we have included a simple example toward this direction, where the tensor kernel described in this paper is compared to a matricization-based tensor kernel.

3.4.1. Choosing the Kernel Width

With methods involving RBF kernels, the choice of the kernel width parameter, $\sigma$, is always critical. For supervised learning, one might choose this parameter by cross-validation based on validation accuracy, while in unsupervised problems, one might use a rule of thumb [33,34,35]. However, in the case of measuring MI in DNNs, the data are often high dimensional, in which case unsupervised rules of thumb often fail [34].

In this work, we choose $\sigma$ based on an optimality criterion. Intuitively, one can make the following observation: a good kernel matrix should reveal the class structures present in the data. This can be accomplished by maximizing the so-called kernel alignment loss [36] between the kernel matrix of a given layer, ${\mathbf{K}}_{\sigma }$, and the label kernel matrix, ${\mathbf{K}}_y$. The kernel alignment loss is defined as

$$A({\mathbf{K}}_a,{\mathbf{K}}_b)=\frac{{\langle {\mathbf{K}}_a,{\mathbf{K}}_b\rangle }_F}{\parallel {\mathbf{K}}_a{\parallel }_F{\parallel {\mathbf{K}}_b\parallel }_F},$$

(19)

where ${\parallel ·\parallel }_F$ and ${\langle ·,·\rangle }_F$ denote the Frobenius norm and inner product, respectively. Thus, we choose our optimal $\sigma$ as

$${\sigma }^{*}={arg\; max}_{\sigma }A({\mathbf{K}}_{\sigma },{\mathbf{K}}_y).$$

To stabilize the $\sigma$ values across mini batches, we employ an exponential moving average, such that in layer ℓ at iteration t, we have

$${\sigma }_{𝓁,t}=\beta {\sigma }_{𝓁,t-1}+(1-\beta ){\sigma }_{𝓁,t}^{*},$$

where $\beta \in [0,1]$ and ${\sigma }_{𝓁,1}={\sigma }_{𝓁,1}^{*}$.

4. Results

We evaluate our approach by comparing it to previous results obtained on small networks by considering the MNIST dataset and a Multilayer Perceptron (MLP) architecture that was inspired by [5]. We further compare to a small CNN architecture similar to that of [4] before considering large networks, namely VGG16, and a more challenging dataset, namely CIFAR-10. Note that unless stated otherwise, we use CNN to denote the small CNN architecture. Details about the MLP and the CNN utilized in these experiments can be found in Appendix D. All MI measures were computed using Equations (2), (3) and (10) and the tensor approach described in Section 4, which amounts to setting $\alpha =1$. Furthermore, the MI estimates showed in all plots are averages across multiple training runs. Code is available online (https://github.com/Wickstrom/InformationTheoryExperiment, accessed on 1 June 2023).

Since the MI is computed at the mini-batch level, a certain degree of noise is present. To smooth the MI measures, we employ a moving average approach where each sample is averaged over k mini-batches. For the MLP and CNN experiments, we use $k=10$, and for the VGG16, we use $k=50$. We use a batch size of 100 samples and determine the kernel width using the kernel alignment loss defined in Equation (19). For each hidden layer, we chose the kernel width that maximizes the kernel alignment loss in the range 0.1 and 10 times the mean distance between the samples in one mini-batch. Initially, we sample 75 equally spaced values for the kernel width in the given range for the MLP and CNN and 300 values for the VGG16 network. During training, we dynamically reduce the number of samples to 50 and 100, respectively, to reduce computational complexity, which is motivated by the fact that the kernel width remains relatively stable during the latter part of training (illustrated in Section 5). We chose the ranges 0.1 and 10 times the mean distance between the samples in one mini-batch to avoid the kernel width becoming too small and to ensure that we cover a wide enough range of possible values. For the input kernel width, we empirically evaluated values in the range 2–16 and found consistent results for values in the range 4–12. All our experiments were conducted with an input kernel width of 8. For the label kernel matrix, we want a kernel width that is as small as possible to approach an ideal kernel matrix while at the same time large enough to avoid numerical instabilities. For all our experiments, we use a value of 0.1 for the kernel width of the label kernel matrix.

Comparison to previous approaches First, we study the IP of the MLP similar to the one examined in previous works on DNN analysis using information theory [4,5]. We utilize stochastic gradient descent, a cross-entropy loss function, and repeat the experiment 5 times. Figure 1 displays the IP of the MLP with a ReLU activation function in each hidden layer. MI was measured using the training data of the MNIST dataset. A similar experiment was performed with the tanh activation function, obtaining similar results. The interested reader can find these results in Appendix E.

From Figure 1, one can clearly observe a fitting phase, where both $I(T;X)$ and $I(Y;T)$ increase rapidly, followed by a compression phase where $I(T;X)$ decrease and $I(Y;T)$ remains unchanged. In addition, note that $I(Y;T)$ for the output layer (layer 5 in Figure 1) stabilizes at an approximate value of ${log}_2\left(10\right)$. The following analysis shows that this is to be expected. When the network achieves approximately $100\%$ accuracy, $I(Y;\widehat{Y})\approx S\left(Y\right)$, where $\widehat{Y}$ denotes the output of the network, since Y and $\widehat{Y}$ will be approximately identical and the MI between a variable and itself is just the entropy of the variable. The entropy of Y is measured using Equation (10), which requires the computation of the eigenvalues of the label kernel matrix $\frac{1}{N}{\mathbf{K}}_y$. For the ideal case, where ${\left({\mathbf{K}}_y\right)}_{ij}=1$ if $y_i=y_j$ and zero otherwise, ${\mathbf{K}}_y$ is a rank K matrix, where K is the number of classes in the data. Thus, $\frac{1}{N}{\mathbf{K}}_y$ has K non-zero eigenvalues which are given by ${\lambda }_k\left(\frac{1}{N}{\mathbf{K}}_y\right)=\frac{1}{N}{\lambda }_k\left({\mathbf{K}}_y\right)=\frac{N_{c_k}}{N}$, where $N_{c_k}$ is the number of datapoints in class $k,k=1,2,\dots ,K$. Furthermore, if the dataset is balanced, we have $N_{c_1}=N_{c_2}=\dots =N_{c_K}\equiv N_c$. Then, ${\lambda }_k\left(\frac{1}{N}{\mathbf{K}}_y\right)=\frac{N_c}{N}=\frac{1}{K}$, which gives us the entropy measure

$$\begin{array}{cc}\hfill S\left(\frac{1}{N}{\mathbf{K}}_y\right)& =-\sum _{k=1}^K{\lambda }_k\left(\frac{1}{N}{\mathbf{K}}_y\right){log}_2\left[{\lambda }_k\left(\frac{1}{N}{\mathbf{K}}_y\right)\right]\hfill \\ \hfill & =-\sum _{k=1}^K\frac{1}{K}{log}_2\left[\frac{1}{K}\right]\hfill \\ \hfill & ={log}_2\left[K\right].\hfill \end{array}$$

(20)

Next, we examine the IP of a CNN, similar to that studied by [4], with a similar experimental setup as for the MLP experiment. Figure 4 displays the IP of the CNN with a ReLU activation function in all hidden layers. A similar experiment was conducted using the tanh activation function and can be found in Appendix F. While the output layer behaves similarly to that of the MLP, the preceding layers show much less movement. In particular, no fitting phase is observed, which we hypothesize is a result of the convolutional layers being able to extract the necessary information in very few iterations. Note that the output layer is again settling at the expected value of ${log}_2\left(10\right)$, similar to the MLP, as it also achieves close to $100\%$ accuracy.

Increasing DNN size We analyze the IP of the VGG16 network on the CIFAR-10 dataset with the same experimental setup as in the previous experiments. To our knowledge, this is the first time that the full IP has been modeled for such a large-scale network. Figure 5 and Figure 6 show the IP when measuring the MI for the training dataset and the test dataset, respectively. For the training dataset, we can clearly observe the same trend as for the smaller networks, where layers experience a fitting phase during the early stages of training and a compression phase in the later stage. Note that the compression phase is less prominent for the testing dataset. Note also the difference between the final values of $I(Y;T)$ for the output layer measured using the training and test data, which is a result of the different accuracy achieved on the training data (≈100%) and test data (≈90%). Ref. [3] claims that $I(T;X)\approx H\left(X\right)$ and $I(Y;T)\approx H\left(Y\right)$ for high-dimensional data, and they highlight particular difficulties with measuring the MI between convolutional layers and the input/output. However, this statement is dependent on their particular measure for the MI, and the results presented in Figure 5 and Figure 6 demonstrate that neither $I(T;X)$ nor $I(Y;T)$ is deterministic for our proposed measure. Furthermore, other measures of MI have also demonstrated that both $I(T;X)\approx H\left(X\right)$ and $I(Y;T)\approx H\left(Y\right)$ evolve during training [4,18].

Another type of widely used DNNs is residual networks [37]. While these networks typically have less parameters than the VGG16, they usually have more layers. This increase in the number of layers is enabled by skip-connections that allow data to flow through the network without loss of information. This complicates the information theoretic analysis, as the dynamics between the layers change and information do not need to decrease in between the layers. While our proposed estimator is computationally capable of handling residual networks, an extensive analysis would be required to understand the added complexity that is introduced by the lossless flow of information in these networks. We consider such an analysis as outside the scope of this paper but an interesting avenue of future research.

Effect of early stopping We also investigate the effect of using early stopping on the IP described above. Early stopping is a regularization technique where the validation accuracy is monitored and training is stopped if the validation accuracy does not increase for a set number of iterations, which is often referred to as the patience hyperparameter. Figure 1 displays the results of monitoring where the training would stop if the early stopping procedure was applied for different values of patience. For a patience of five iterations, the network training would stop before the compression phase takes place for several of the layers. For larger patience values, the effects of the compression phase can be observed before training is stopped. Early stopping is a procedure intended to prevent the network from overfitting, which may imply that the compression phase observed in the IP of DNNs can be related to overfitting. However, recent research on the so-called double descent phenomenon has shown that longer training might be necessary for good performance for overparameterized DNNs [38,39]. In such settings, early stopping might not be as applicable. We describe the double descent phenomenon and investigate its possible connection with the IP in Appendix H.

Data processing inequality The data processing inequality (DPI) is a concept in information theory which states that the amount of information cannot increase in a chain of transformations. A good information theoretic estimator should tend to uphold the DPI. DNN consists of a chain of mappings from the input through the hidden layers and to the output. One can interpret a DNN as a Markov chain [1,7] that defines an information path [1], which should satisfy the DPI [40]:

$$I(X;T_1)\ge I(X;T_2)\ge \dots \ge I(X;T_L),$$

(21)

where L is the number of layers in the network. An indication of a good MI measure is that it tends to uphold the DPI. Figure 7 illustrates the mean difference in MI between two subsequent layers in the MLP and VGG16 networks. Positive numbers indicate that MI decreases, thus indicating compliance with the DPI. We observe that our measure complies with the DPI for all layers in the MLP and all except one in the VGG16 network. Furthermore, we also model the DPI for a simple MLP using the EDGE MI estimator, which has shown encouraging results on several MI estimation tasks [4]. The DPI for the EDGE estimator is shown in Figure A4 of Appendix G, which shows that the EDGE estimator also upholds the DPI. This agrees with our results with regard to the information flow in neural networks. However, a limitation of the EDGE estimator is that it is not differentiable, which can be beneficial if MI estimates are to be included in the training [41].

5. Kernel Width Sigma

We further evaluate our dynamic approach of finding the kernel width $\sigma$. Figure 8 shows the variation of $\sigma$ in each layer for the MLP, the small CNN and the VGG16 network. We observe that the optimal kernel width for each layer (based on the criterion in Section 3.4.1) stabilizes reasonably quickly and remains relatively constant during training. This illustrates that decreasing the sampling range is a useful approach to decreasing computational complexity.

6. Discussion and Conclusions

In this work, we propose a novel framework for analyzing DNNs from an information theoretic perspective using a tensor-based measure of the matrix-based approach of [12]. Our experiments illustrate that the proposed approach scales to large DNNs, which allows us to provide insights into the training dynamics. We observe that the compression phase in neural network training tends to be more prominent when MI is measured on the training set and that commonly used early-stopping criteria tend to stop training before or at the onset of the compression phase. This could imply that the compression phase is linked to overfitting. However, recent research on the double descent phenomenon has shown that a longer training time might be beneficial for generalization [38,39]. In Appendix H, we perform a preliminary study that examines a potential connection between the compression phase and the recent epoch-wise double descent phenomenon. Furthermore, we showed that for our tensor-based approach, the claim that $H\left(X\right)\approx I(T;X)$ and $H\left(Y\right)\approx I(T;Y)$ does not hold. We believe that our proposed approach can provide new insight and facilitate a more theoretical understanding of DNNs.