Integrated Information in the Spiking–Bursting Stochastic Model

Abstract

Integrated information has been recently suggested as a possible measure to identify a necessary condition for a system to display conscious features. Recently, we have shown that astrocytes contribute to the generation of integrated information through the complex behavior of neuron–astrocyte networks. Still, it remained unclear which underlying mechanisms governing the complex behavior of a neuron–astrocyte network are essential to generating positive integrated information. This study presents an analytic consideration of this question based on exact and asymptotic expressions for integrated information in terms of exactly known probability distributions for a reduced mathematical model (discrete-time, discrete-state stochastic model) reflecting the main features of the “spiking–bursting” dynamics of a neuron–astrocyte network. The analysis was performed in terms of the empirical “whole minus sum” version of integrated information in comparison to the “decoder based” version. The “whole minus sum” information may change sign, and an interpretation of this transition in terms of “net synergy” is available in the literature. This motivated our particular interest in the sign of the “whole minus sum” information in our analytical considerations. The behaviors of the “whole minus sum” and “decoder based” information measures are found to bear a lot of similarity—they have mutual asymptotic convergence as time-uncorrelated activity increases, and the sign transition of the “whole minus sum” information is associated with a rapid growth in the “decoder based” information. The study aims at creating a theoretical framework for using the spiking–bursting model as an analytically tractable reference point for applying integrated information concepts to systems exhibiting similar bursting behavior. The model can also be of interest as a new discrete-state test bench for different formulations of integrated information.

1. Introduction

Integrated information (II) [1,2,3,4] is a measure of internal information exchange in complex systems and it has recently attracted a lot of interest, because initially it was proposed to quantify consciousness [5]. Despite the fact that this initial aim is still a matter of research and debate [6,7,8,9], the II concept itself is by now a widely acknowledged tool in the field of complex dynamics analysis [10,11,12]. The general concept gave rise to specific “empirical” formalizations of II [13,14,15,16] aimed at computability from empirical probability distributions based on real data. For a systematic taxonomy of II measures, see [17], and a comparative study of empirical II measures in application to Gaussian autoregressive network models has been recently done in [18].

Our recent study [19] addressed the role of astrocytic regulation of neurotransmission [20,21,22] in generating positive II via small networks of brain cells—neurons and astrocytes. Empirical “whole minus sum” II, as defined in [13], was calculated in [19] from the time series produced by a biologically realistic model of neuro-astrocytic networks. A simplified, analytically tractable stochastic “spiking–bursting” model (in complement to the realistic one) was designed to describe a specific type of activity in neuro-astrocytic networks which manifests itself as a sequence of intermittent system-wide excitations of rapid pulse trains (“bursts”) on the background of random “spiking” activity in the network [23,24]. The spiking–bursting model is a discrete-time, discrete-state stochastic process which mimics the main features of this behavior. The model was successfully used in [19] to produce semi-analytical estimates of II in good agreement with direct computation of II from time series of the biologically realistic network model. We have suggested a possible explanation that a generation of positive II was the reason why mammal brain evolved to develop an astrocyte network to overlap with a network of neurons, but, still, it remained unclear what are the underlying mechanisms driving a complex neural behavior to generate positive II. In this paper we address this challenging question.

The present study aims at creating a theoretical formalism for using the spiking–bursting model of [19] as an analytically tractable reference point for applying integrated information concepts to systems exhibiting similar bursting behavior (in particular, to other neuron–astrocyte networks). The analytical treatment is based on exact and asymptotic expressions for integrated information in terms of exactly known probability distributions for the spiking–bursting model. The model is constructed as the simplest possible (although essentially non-Gaussian) to reflect the features of neuron–astrocyte network dynamics which lead to generating positive II. We also aim at extending the knowledge of comparative features of different empirical II measures, which are currently available mainly in application to Gaussian autoregressive models [17,18], by applying two such measures [13,16] to our discrete-state model.

In Section 2 and Section 3 we specify the definitions of the II measures used and the model. Specific properties of the model which lead to redundancy in its parameter set are addressed in Section 4. In Section 5 we provide an analytical treatment for the empirical “whole minus sum” [13] version of II in application to our model. This choice among other empirical II measures is inherited from the preceding study [19] and is in part due to its easy analytical tractability, and also due to its ability to change sign, which naturally identifies a transition point in the parameter space. This property may be considered a violation of the natural non-negativeness requirement for II [16]; on the other hand, the sign of the “whole minus sum” information has been given interpretation in terms of “net synergy” [25] as a degree of redundancy in the evolution of a system [18]. In this sense this transition may be viewed as a useful marker in its own right in the tool-set of measures for complex dynamics. This motivates our particular focus on identifying the sign transition of the “whole minus sum” information in the parameter space of the model. We also identify a scaling of II with a small parameter which determines time correlations in the bursting (astrocytic) subsystem.

In Section 6 we compare the outcome of the “whole minus sum” II measure [13] to that of the “decoder based” measure ${\Phi }^{*}$, which was specifically designed in [16] to satisfy the non-negativeness property. We compute ${\Phi }^{*}$ directly by definition from known probability distributions of the model. Despite their inherent difference consisting in changing or not changing sign, the two compared measures are shown to bear similarities in their dependence upon model parameters, including the same scaling with the time correlation parameter.

2. Definition of II Measures in Use

The empirical “whole minus sum” version of II is formulated according to [13] as follows. Consider a stationary stochastic process $\xi \left(t\right)$ (binary vector process), whose instantaneous state is described by N binary digits (bits), each identified with a node of the network (neuron). The full set of N nodes (“system”) can be split into two non-overlapping non-empty subsets (“subsystems”) A and B; such a splitting is referred to as bipartition $AB$. Denote by $x=\xi \left(t\right)$ and $y=\xi (t+\tau )$ two states of the process separated by a specified time interval $\tau \ne 0$. The states of the subsystems are denoted as $x_A$, $x_B$, $y_A$, $y_B$.

Mutual information between x and y is defined as

$$I_{xy}=H_x+H_y-H_{xy},$$

(1)

where

$$H_x=-\sum _{x}p\left(x\right){log}_{2}p\left(x\right)$$

(2)

is entropy (base 2 logarithm gives result in bits); summation is hereinafter assumed to be taken over the whole range of the index variable (here x), $H_y=H_x$, due to assumed stationarity.

Next, a bipartition $AB$ is considered, and “effective information” ${\Phi }_{\mathrm{eff}}$ as a function of the particular bipartition is defined as

$${\Phi }_{\mathrm{eff}}\left(AB\right)=I_{xy}-I_{x_A,y_A}-I_{x_B,y_B}.$$

(3)

Finally, “whole minus sum” II denoted as $\Phi$ is defined as effective information calculated for a specific bipartition $A{B}^{\mathrm{MIB}}$ (“minimum information bipartition”) which minimizes specifically normalized effective information:

$$\begin{array}{c}\Phi ={\Phi }_{\mathrm{eff}}\left(A{B}^{\mathrm{MIB}}\right),\end{array}$$

(4a)

$$\begin{array}{c}A{B}^{\mathrm{MIB}}={\mathrm{argmin}}_{AB}\left[\frac{{\Phi }_{\mathrm{eff}}\left(AB\right)}{min\{H\left(x_A\right),H\left(x_B\right)\}}\right].\end{array}$$

(4b)

Note that this definition prohibits positive II, whenever ${\Phi }_{\mathrm{eff}}$ turns out to be zero or negative for at least one bipartition $AB$.

We compare the result of the “whole minus sum” effective information (3) to the “decoder based” information measure ${\Phi }^{*}$, which is modified from its original formulation of [16] by setting the logarithms base to 2 for consistency:

$${\Phi }^{*}\left(AB\right)=I_{xy}-I_{xy}^{*}\left(AB\right),$$

(5a)

where

$$I_{xy}^{*}\left(AB\right)=\underset{\beta }{max}\left[-\sum _{y}p\left(y\right){log}_2\sum _{x}p\left(x\right)q_{AB}{\left(y\right|x)}^{\beta }+\sum _{xy}p\left(xy\right){log}_2{q}_{AB}{\left(y\right|x)}^{\beta }\right],$$

(5b)

$$q_{AB}\left(y\right|x)=p\left(y_A\right|x_A)p\left(y_B\right|x_B)=\frac{p\left(x_A{y}_A\right)p\left(x_B{y}_B\right)}{p\left(x_A\right)p\left(x_B\right)}.$$

(5c)

3. Spiking–Bursting Stochastic Model

Physiologically, spikes are short (about 1 millisecond in duration) pulses of voltage (action potential) across the neuronal membrane. Bursts are rapid sequences of spikes. The main feature of the neuron–astrocyte network model in [19] is the presence of network-wide coordinated bursts, when all neurons are rapidly spiking in the same time window. Such bursts are coordinated by the astrocytic network and occur on the background of weakly correlated spiking activity of individual neurons. The spiking–bursting model was suggested in [19] as the simplest mathematical description of this behavior. In this model, time is discretized into small bins, and neurons are represented by binary digits taking on values 0 or 1, denoting the absence or the presence of at least one spike within the specific time bin. Respectively, a network-wide burst is represented by a time interval during which all neurons are locked at value 1 (which corresponds to a train of rapid spiking in the underlying biological system). The idea behind the model is illustrated by the graphical representation of its typical time evolution, as shown in Figure 1. The graphs of the model dynamics can be seen as envelopes of respective time recordings of membrane voltage in actual neurons: each short rectangular pulse of the model is assumed to correspond to at least one narrow spike of voltage, and a prolonged pulse (several discrete time bins in duration) represents a spike train (burst).

Mathematically, this “spiking–bursting” model is a stochastic model, which produces a binary vector valued, discrete-time stochastic process. In keeping with [19], the model is defined as a combination $M=\{V,S\}$ of a time-correlated dichotomous component V which turns on and off system-wide bursting (that mimics global bursting of a neuronal network, when each neuron produces a train of pulses at a high rate [19]), and a time-uncorrelated component S describing spontaneous (spiking) activity (corresponding to a background random activity in a neural network characterized by relatively sparse random appearance of neuronal pulses—spikes [19]) occurring in the absence of a burst. The model mimics the spiking–bursting type of activity which occurs in a neuro-astrocytic network, where the neural subsystem normally exhibits time-uncorrelated patterns of spiking activity, and all neurons are under the common influence of the astrocytic subsystem, which is modeled by the dichotomous component V and sporadically induces simultaneous bursting in all neurons. A similar network architecture with a “master node” spreading its influence on subordinated nodes was considered, for example, in [1] (Figure 4b therein).

The model is defined as follows. At each instance of (discrete) time the state of the dichotomous component can be either “bursting” with probability $p_b$, or “spontaneous” (or “spiking”) with probability $p_s=1-p_b$. While in the bursting mode, the instantaneous state of the resulting process $x=\xi \left(t\right)$ is given by all ones: $x=11..1$ (further abbreviated as $x=1$). In cases of spiking, the state x is a (time-uncorrelated) random variate, which is described by a discrete probability distribution $s_x$ (where an occurrence of “1” in any bit is referred to as a “spike”), so that the resulting one-time state probabilities read

$$\begin{array}{cc}\hfill p(x\ne 1)& =p_s{s}_x,\hfill \end{array}$$

(6a)

$$\begin{array}{cc}\hfill p(x=1)& =p_1,p_1=p_s{s}_1+p_b,\hfill \end{array}$$

(6b)

where $s_1$ is the probability of spontaneous occurrence of $x=1$ (hereafter referred to as a system-wide simultaneous spike) in the absence of a burst. (In a real network, “simultaneous” implies occurring within the same time discretization bin [19]).

To describe two-time joint probabilities for $x=\xi \left(t\right)$ and $y=\xi (t+\tau )$, consider a joint state $xy$ which is a concatenation of bits in x and y. The spontaneous activity is assumed to be uncorrelated in time, which leads to the factorization

$$s_{xy}=s_x{s}_y.$$

(7)

The time correlations of the dichotomous component are described by a $2\times 2$ matrix

$$p_{q\in \{s,b\},r\in \{s,b\}}=\left(\begin{array}{cc}{p}_{ss}& p_{sb}\\ p_{bs}& p_{bb}\end{array}\right)$$

(8)

whose components are joint probabilities to observe the respective spiking (index “s”) and/or bursting (index “b”) states in x and y. (In a neural network these correlations are conditioned by burst duration [19]; e.g., if this (in general, random) duration mostly exceeds $\tau$, then the correlation is positive.) The probabilities obey $p_{sb}=p_{bs}$ (due to stationarity), $p_b=p_{bb}+p_{sb}$, $p_s=p_{ss}+p_{sb}$, thereby allowing one to express all one- and two-time probabilities describing the dichotomous component in terms of two independent quantities, which for example, can be a pair $\{p_s,p_{ss}\}$; then

$$\begin{array}{cc}\hfill p_{sb}& =p_{bs}=p_s-p_{ss},\hfill \end{array}$$

(9a)

$$\begin{array}{cc}\hfill p_{bb}& =1-(p_{ss}+2p_{sb}),\hfill \end{array}$$

(9b)

or $\{p_b,\rho \}$ as in [19], where $\rho$ is the Pearson correlation coefficient defined by

$$p_{sb}=p_s{p}_b(1-\rho ).$$

(10)

In Section 4 we justify the use of another effective parameter $ϵ$ (13) instead of $\rho$ to determine time correlations in the dichotomous component.

The two-time joint probabilities for the resulting process are then expressed as

$$\begin{array}{c}\begin{array}{cc}\hfill p(x\ne 1,y\ne 1)& =p_{ss}{s}_x{s}_y,\hfill \\ \hfill p(x\ne 1,y=1)& =\pi s_x,p(x=1,y\ne 1)=\pi s_y,\hfill \\ \hfill p(x=1,y=1)& =p_{11},\hfill \end{array}\end{array}$$

(11a)

$$\begin{array}{c}\pi =p_{ss}{s}_1+p_{sb},p_{11}=p_{ss}{s}_1^2+2p_{sb}{s}_1+p_{bb}.\end{array}$$

(11b)

Note that the above notation can be applied to any subsystem instead of the whole system (with the same dichotomous component, as it is system-wide anyway).

The mentioned probabilities can be interpreted in terms of the underlying biological system as follows (see details in [19]): $p_b$ is the probability of observing the astrocytic subsystem in the excited (high calcium concentration) state, which induces global bursting activity in all neurons, within a specific time discretization bin; $p_{bb}$ is the probability of observing the mentioned state in two time bins separated by the time lag $\tau$, and $\rho$ is the respective time-delayed Pearson correlation coefficient of the astrocytic activity; $s_x$ is the probability of observing a specific spatial pattern of spiking x within one time bin in spontaneous neuronal activity (in the absence of an astrocyte-induced burst), and in particular $s_1$ is the probability that all neurons fire a spike within one time bin in spontaneous activity. In this sense $s_1$ measures the overall strength of spontaneous activity of the neuronal subsystem. When spiking activity is independent across neurons, the set of parameters $\{s_1,p_b,\rho \}$ fully determines the “whole minus sum” II in the spiking–bursting model. In [19] these parameters were fitted to match (in the least-squares sense) the two-time probability distribution (11) to the respective “empirical” (numerically obtained) probabilities for the biologically realistic model of the neuron–astrocyte network. This fitting produced the dependence of the spiking–bursting parameters $\{s_1,p_b,\rho \}$ upon the biological parameters; see Figure 7 in [19].

4. Model Parameter Scaling

The spiking–bursting stochastic model, as described in Section 3, is redundant in the following sense. In terms of the model definition, there are two distinct states of the model which equally lead to observing the same one-time state of the resultant process with 1s in all bits: firstly—a burst, and secondly—a system-wide simultaneous spike in the absence of a burst, which are indistinguishable by one-time observations. Two-time observations reveal a difference between system-wide spikes on one hand and bursts on the other, because the latter are assumed to be correlated in time, unlike the former. That said, the “labeling” of bursts versus system-wide spikes exists in the model (by the state of the dichotomous component), but not in the realizations. Proceeding from the realizations, it must be possible to relabel a certain fraction of system-wide spikes into bursts (more precisely, into a time-uncorrelated portion thereof). Such relabeling would change both components of the model $\{V,S\}$ (dichotomous and spiking processes), in particular, diluting the time correlations of bursts, without changing the actual realizations of the resultant process. This implies the existence of a transformation of model parameters which keeps realizations (i.e., the stochastic process as such) invariant. The derivation of this transformation is presented in Appendix A and leads to the following scaling.

$$\begin{array}{cc}\hfill s_{x\ne 1}& =\alpha s_{x\ne 1}^{\prime },\hfill \end{array}$$

(12a)

$$\begin{array}{cc}\hfill 1-s_1& =\alpha (1-s_1^{\prime }),\hfill \end{array}$$

(12b)

$$\begin{array}{cc}\hfill p_{s^{\prime }}& =\alpha p_s,\hfill \end{array}$$

(12c)

$$\begin{array}{cc}\hfill p_{s^{\prime }{s}^{\prime }}& ={\alpha }^2{p}_{ss},\hfill \end{array}$$

(12d)

where $\alpha$ is a positive scaling parameter, and all other probabilities are updated according to Equation (9).

The mentioned invariance in particular implies that any characteristic of the process must be invariant to the scaling (12a–d). This suggests a natural choice of a scaling-invariant effective parameter $ϵ$ defined by

$$p_{ss}=p_s^2(1+ϵ)$$

(13)

to determine time correlations in the dichotomous component. In conjunction with a second independent parameter of the dichotomous process, for which a straightforward choice is $p_s$, and with full one-time probability table for spontaneous activity $s_x$, these constitute a natural full set of model parameters $\{s_x,p_s,ϵ\}$.

The two-time probability table (8) can be expressed in terms of $p_s$ and $ϵ$ by substituting Equation (13) into Equation (9):

$$p_{q\in \{s,b\},r\in \{s,b\}}=\left(\begin{array}{cc}\hfill p_s^2+ϵp_s^2& \hfill p_s{p}_b-ϵp_s^2\\ \hfill p_s{p}_b-ϵp_s^2& \hfill p_b^2+ϵp_s^2\end{array}\right).$$

(14)

The requirement of non-negativeness of probabilities imposes simultaneous constraints

$$\begin{array}{c}ϵ\ge -1\end{array}$$

(15a)

and

$$\begin{array}{c}\hfill p_s\le p_{smax}=\left\{\begin{array}{cc}\frac{1}{1+ϵ}\left(1-\sqrt{\left|ϵ\right|}\right)\hfill & \mathrm{if}-1\le ϵ<0,\hfill \\ \frac{1}{1+ϵ}\hfill & \mathrm{if}ϵ\ge 0,\hfill \end{array}\right.\end{array}$$

(15b)

or equivalently,

$$-{ϵ}_{max}^2\le ϵ\le {ϵ}_{max}=\frac{p_b}{p_s}.$$

(16)

Comparing the off-diagonal term $p_{sb}$ in (14) to the definition of the Pearson’s correlation coefficient $\rho$ in (10), we get

$$ϵ=\rho \frac{p_b}{p_s}=\rho {ϵ}_{max};$$

(17)

thus, the sign of $ϵ$ has the same meaning as that of $\rho$. Hereinafter we limit ourselves to non-negative correlations $ϵ\ge 0$.

5. Analysis of the Empirical “Whole Minus Sum” Measure for the Spiking–Bursting Process

In this Section we analyze the behavior of the “whole minus sum” empirical II [13] defined by Equations (3) and (4) for the spiking–bursting model in dependence of the model parameters, particularly focusing on its transition from negative to positive values.

5.1. Expressing the “Whole Minus Sum” Information

Mutual information $I_{xy}$ for two time instances x and y of the spiking–bursting process is expressed by inserting all one- and two-time probabilities of the process according to (6), (11) into the definition (1), (2). The full derivation is given in Appendix B and leads to an expression which was used in [19]

$$I_{xy}=2(1-s_1)\left\{p_s\right\}+2\left\{p_1\right\}-{(1-s_1)}^2\left\{p_{ss}\right\}-2(1-s_1)\left\{\pi \right\}-\left\{p_{11}\right\},$$

(18)

where we denote for compactness

$$\left\{q\right\}=-q{log}_{2}q.$$

(19)

We exclude from further consideration the following degenerate cases which automatically give $I_{xy}=0$ by definition (1):

$$s_1=1,\mathrm{or}{p}_s=0,\mathrm{or}{p}_s=1,\mathrm{or}\rho =ϵ=0,$$

(20)

where the former two correspond to a deterministic “always 1” state for which all entropies in (1) are zero, and the latter two produce no predictability, which implies $H_{xy}=H_x+H_y$.

The particular case $s_1=0$ in (18) reduces to

$$I_{xy}{|}_{s_1=0}=2\left(\left\{p_s\right\}+\left\{p_b\right\}\right)-\left(\left\{p_{ss}\right\}+2\left\{p_{sb}\right\}+\left\{p_{bb}\right\}\right),$$

(21a)

which coincides with mutual information for the dichotomous component taken alone and can be seen as a function of just two independent parameters of the dichotomous component, for which we chose $p_s$ and $ϵ$ as suggested in Section 4. Using the expressions for the two-time probabilities (14), we rewrite (21a) in the form

$$\begin{array}{cc}\hfill I_{xy}{|}_{s_1=0}& =2\left(\left\{p_s\right\}+\left\{p_b\right\}\right)-\left(\{p_s^2+ϵp_s^2\}+2\{p_s{p}_b-ϵp_s^2\}+\{p_b^2+ϵp_s^2\}\right),\mathrm{where}{p}_b=1-p_s,\hfill \\ \hfill & =I_0(p_s,ϵ).\hfill \end{array}$$

(21b)

Expression (21b) explicitly defines a function $I_0(p_s,ϵ)$, which turns out to be a universal function allowing one to express mutual information (18) and effective information (3) in terms of the model parameters, as we show below. Typical plots of $I_0(p_s,ϵ)$ versus $p_s$ at several fixed values of $ϵ$ are shown with blue solid lines in Figure 2.

The formula (18) can be recovered back from (21a,b) by virtue of the scaling (12a–d), by assuming $s_1^{\prime }=0$ in (21b) and substituting the corresponding scaled value $p_{s^{\prime }}=(1-s_1)p_s$ as per (12c) in place of the first argument of function $I_0(p_{s^{\prime }},ϵ)$ defined in (21b), while parameter $ϵ$ remains invariant to the scaling. This produces a simplified expression

$$I_{xy}=I_0\left((1-s_1)p_s,ϵ\right),$$

(22)

which is exactly equivalent to (18) for any $s_1$. We emphasize that hereinafter expressions containing $I_0(·,·)$—(22), (23), (30b), etc.—imply that $p_s$ in (21b) must be substituted with the actual first argument of $I_0(·,·)$, e.g., by $(1-s_1)p_s$ in (22). The same applies when the approximate expression for $I_0(·)$ (35) is used.

Given a bipartition $AB$ (see Section 2), this result is applicable as well to any subsystem A (B), with $s_1$ replaced by $s_A$ ($s_B$) which denote the probability of a subsystem-wide simultaneous spike $x_A=1$ ($x_B=1$) in the absence of a burst, and with same parameters of the dichotomous component (here $p_s$, $ϵ$). Then effective information (3) is expressed as

$${\Phi }_{\mathrm{eff}}=I_0\left((1-s_1)p_s,ϵ\right)-I_0\left((1-s_A)p_s,ϵ\right)-I_0\left((1-s_B)p_s,ϵ\right).$$

(23)

Hereafter in this section we assume the independence of spontaneous activity across the network nodes (neurons), which implies

$$s_A{s}_B=s_1,$$

(24)

then (23) turns into

$${\Phi }_{\mathrm{eff}}=f\left(s_A\right),$$

(25a)

where

$$f\left(s\right)=I_0\left((1-s_1)p_s,ϵ\right)-I_0\left((1-s)p_s,ϵ\right)-I_0\left((1-s_1/s)p_s,ϵ\right).$$

(25b)

Essentially, according to (25a,b), the function $f\left(s\right)$ shows the dependence of effective information ${\Phi }_{\mathrm{eff}}$ upon the choice of the bipartition, which is characterized by the value of $s_A=s$ (if A is any non-empty subsystem, then $s_A$ is defined as the probability of spontaneous occurrence of 1s in all bits in A in the same instance of the discrete time), while the function parameter $s_1$ determines the intensity of spontaneous spiking activity. Note that the function $I_0(·,·)$ in (21b) is defined only when the first argument is in the range $(0,1)$; thus, the definition domain of $f\left(s\right)$ in (25b) is

$$s_1<s<1.$$

(26)

5.2. Determining the Sign of the “Whole Minus Sum” Information

According to (4), the necessary and sufficient condition for the “whole minus sum” empirical II to be positive is the requirement that ${\Phi }_{\mathrm{eff}}$ be positive for any bipartition $AB$. Due to (25a,b), this requirement can be written in the form

$$\underset{s\in \left\{s_A\right\}}{min}f\left(s\right)>0,$$

(27)

where $\left\{s_A\right\}$ is the set of $s_A$ values for all possible bipartitions $AB$.

Expanding the set of s in (27) to the whole definition domain of $f\left(s\right)$ (26) leads to a sufficient (generally, stronger) condition for positive II

$$f\left(s\right)>0\mathrm{for}\mathrm{all}s\in (s_1,1).$$

(28)

Note that $f\left(s\right)$ by definition (25b) satisfies $f(s=s_1)=f(s=1)=0$, $f^{\prime }(s=s_1)>0$ and (due to the invariance to mutual renaming of subsystems A and B) $f(s_1/s)=f\left(s\right)$. (All mentioned properties and subsequent reasoning can be observed in Figure 3, which shows a few sample plots of $f\left(s\right)$). The latter symmetry implies that the quantity of extrema of $f\left(s\right)$ on $s\in (s_1,1)$ must be odd, one of them always being at $s=\sqrt{s_1}$. If the latter is the only extremum, then it is a positive maximum, and (28) is thus fulfilled automatically. In case of three extrema, $f\left(\sqrt{s_1}\right)$ is a minimum, which can change sign. In both these cases the condition (28) is equivalent to the requirement

$$f\left(\sqrt{s_1}\right)>0,$$

(29)

which can be rewritten as

$$g\left(s_1\right)>0,$$

(30a)

where

$$g\left(s_1\right)=f\left(\sqrt{s_1}\right)=I_0\left((1-s_1)p_s,ϵ\right)-2I_0\left((1-\sqrt{s_1})p_s,ϵ\right).$$

(30b)

The reasoning above essentially reduces the problem of determining the sign of II to determining the sign of the extremum $f\left(\sqrt{s_1}\right)$.

The equivalence of (29) to (28) could be broken if $f\left(s\right)$ had five or more extrema. As suggested by the numerical calculation on a grid of $p_s\in [0.01,0.99]$ and $\rho \in [0.01,1]$, both with step 0.01, this exception never holds, although we did not prove this rigorously. Based on the reasoning above, in the following we assume the equivalence of (29) (and (30)) to (28).

A typical scenario of transformations of $f\left(s\right)$ with the change of $s_1$ is shown in Figure 3. Here the extremum $f\left(\sqrt{s_1}\right)$ (shown with a dot) transforms with the decrease of $s_1$ from a positive maximum into a minimum, which in turn decreases from positive through zero into negative values.

Note that by construction, the function $g\left(s_1\right)$ defined in (30b) expresses effective information ${\Phi }_{\mathrm{eff}}$ from (3) for the hypothetic bipartition characterized by $s_A=s_B=\sqrt{s_1}$, which may or may not exist in the actual particular system. If such “symmetric” bipartition exists, then the value $s_A=\sqrt{s_1}$ belongs to the set $\left\{s_A\right\}$ in (27), which implies that (29) (same as (30)) is equivalent not only to (28), but also to the necessary and sufficient condition (27). Otherwise, (28) (equivalently, (29) or (30)), formally being only sufficient, still may produce a good estimate of the necessary and sufficient condition in cases when $\left\{s_A\right\}$ contains values which are close to $\sqrt{s_1}$ (corresponding to nearly symmetric partitions, if such exist).

Except for the degenerate cases (20), $g\left(s_1\right)$ is negative at $s_1=0$

$$g(s_1=0)=-I_0(p_s,ϵ)<0$$

(31)

and has a limit $g(s_1\to 1-0)\to +0$ ( $-0$ and $+0$ denote the left and right one-sided limits), because

$$\underset{s_1\to 1-0}{lim}\frac{I_0\left((1-s_1)p_s,ϵ\right)}{2I_0\left((1-\sqrt{s_1})p_s,ϵ\right)}=2;$$

(32)

hence, $g\left(s_1\right)$ changes sign at least once on $s_1\in (0,1)$. According to numerical evidence, we assume that $g\left(s_1\right)$ changes sign exactly once on $(0,1)$ without providing a rigorous proof for the latter statement (it was confirmed up to machine precision for each combination of $p_s\in [0.01,0.99]$ and $\rho \in [0.01,1]$, both with step 0.01; also note that for the asymptotic case (38) this statement is rigorous). In line with the above, the solution to (30a) has the form

$$s_1^{min}(p_s,ϵ)<s_1<1,$$

(33)

where $s_1^{min}(p_s,ϵ)$ is the unique root of $g\left(s_1\right)$ on $(0,1)$. Several plots of $s_1^{min}(p_s,ϵ)$ versus $p_s$ at $ϵ$ fixed and versus $ϵ$ at $p_s$ fixed, which are obtained by numerically solving for the zero of $g\left(s_1\right)$, are shown in Figure 4 with blue solid lines.

This result identifies a region in the parameter space of the model, where the “whole minus sum” information is positive. From the viewpoint of the underlying biological system, the quantity $s_1^{min}$ determines the minimal sufficient intensity of spontaneous neuronal spiking activity for positive II. According to the result in Figure 4, within the assumption of independent spiking across the network (24), values $s_1\gtrsim 0.17$ lead to positive II regardless of other parameter values, and this threshold decreases further when $p_s$ is increased, which implies decreasing the frequency of occurrence of astrocyte-activated global bursting $p_b=1-p_s$.

5.3. Asymptotics for Weak Correlations in Time

Further insight into the dependence of mutual information $I_{xy}$ (and, consequently, of ${\Phi }_{\mathrm{eff}}$ and II) upon parameters can be obtained by expanding the definition of $I_0(p_s,ϵ)$ in (21b) in powers of $ϵ$ (limit of weak correlations in time), which yields

$$I_0(p_s,ϵ)=\frac{1}{2log2}{\left(\frac{p_s}{1-p_s}\right)}^2·{ϵ}^2+O\left({ϵ}^3\right).$$

(34)

Estimating the residual term (see details in Appendix C) indicates that the approximation by the leading term

$$I_0(p_s,ϵ)\approx \frac{{ϵ}^2}{2log2}{\left(\frac{p_s}{1-p_s}\right)}^2$$

(35)

is valid when

$$\begin{array}{cc}\hfill \left|ϵ\right|& \ll 1,\hfill \end{array}$$

(36a)

$$\begin{array}{cc}\hfill \left|ϵ\right|& \ll {\left(\frac{p_b}{p_s}\right)}^2={ϵ}_{max}^2.\hfill \end{array}$$

(36b)

Solving (36b) for $p_s$ rewrites it in the form of an upper bound for $p_s$

$$p_s<\frac{1}{1+\sqrt{\left|ϵ\right|}}$$

(36c)

(the use of “≪” sign is not appropriate in (36c), because this inequality does not imply a small ratio between its left-hand and right-hand parts). Note that the inequalities (36b), (36c) are not weaker than the formal upper bounds ${ϵ}_{max}$ in (16) and $p_{smax}$ in (15) which arise from the definition of $ϵ$ (13) due to the requirement of positive probabilities.

Approximation (35) is plotted in Figure 2 with red dashed lines along with corresponding upper bounds of approximation applicability range (36c) denoted by red dots (note that large $ϵ$ violates (36a) anyway, thus in this case (36c) has no effect). Mutual information (35) scales with $ϵ$ within range (36) as ${ϵ}^2$ and vanishes with $ϵ\to 0$. The same holds for effective information (23). Since the normalizing denominator in (4b) contains one-time entropies which do not depend on $ϵ$ at all, this scaling of ${\Phi }_{\mathrm{eff}}$ does not change the minimum information bipartition, finally implying that II also scales as ${ϵ}^2$. That said, as factor ${ϵ}^2$ does not affect the sign of ${\Phi }_{\mathrm{eff}}$, the lower bound $s_1^{min}$ in (33) exists and is determined only by $p_s$ in this limit.

Substituting the approximation (35) for $I_0(·,·)$ into the definition of $g\left(s_1\right)$ in (30b) after simplifications reduces the equation $g\left(s_1\right)=0$ to the following (see the comment below Equation (22)):

$$p_s(\sqrt{2}-1)s_1-\sqrt{s_1}+(1-p_s)(\sqrt{2}-1)=0,$$

(37)

whose solution in terms of $s_1$ on $0<s_1<1$ equals $s_1^{min}$, according to the reasoning behind Equation (33). Solving (37) as a quadratic equation in terms of $\sqrt{s_1}$ produces a unique root on $(0,1)$, which yields

$$s_1^{min}\left(p_s\right){|}_{ϵ\to 0}={\left(\frac{1-\sqrt{1-4p_s(1-p_s){(\sqrt{2}-1)}^2}}{2p_s(\sqrt{2}-1)}\right)}^2.$$

(38)

Result of (38) is plotted in Figure 4 with red dashed lines: in panel (a) as a function of $p_s$, and in panel (b) as horizontal lines whose vertical position is the result of (38), and horizontal span denotes the estimated applicability range (36b) (note that condition (36a) also applies, and becomes stronger than (36b) when $p_s<1/2$).

6. Comparison of Integrated Information Measures

In this Section we compare the outcome of two versions of empirical integrated information measures available in the literature, one being the “all-minus-sum” effective information ${\Phi }_{\mathrm{eff}}$ (3) from [13] which is used elsewhere in this study, and the other “decoder based” information ${\Phi }^{*}$ as introduced in [16] and expressed by Equations (5a–c). We calculate both measures by their respective definitions using the one- and two-time probabilities from Equations (6a,b) and (11a–d) for the spiking–bursting model with $N=6$ bits, assuming no spatial correlations among bits in spiking activity, with same spike probability P in each bit. In this case

$$s_x=P^{m\left(x\right)}{(1-P)}^{N-m\left(x\right)},P=s_1^{\frac{1}{N}},$$

(39)

where $m\left(x\right)$ is the number of ones in the binary word x.

We consider only a symmetric bipartition with subsystems A and B consisting of $N/2=3$ bits each. Due to the assumed equal spike probabilities in all bits and in the absence of spatial correlations of spiking, this implies complete equivalence between the subsystems. In particular, in the notations of Section 5 we get

$$s_1=s_A{s}_B,s_A=s_B=\sqrt{s_1}.$$

(40)

This choice of the bipartition is firstly due to the fact that the sign of effective information for this bipartition determines the sign of the resultant “whole minus sum” II (although the actual value of II is determined by the minimal information bipartition, which may be different). This has been established in Section 5 (see reasoning behind Equations (27)–(30) and further on); moreover, the function $g\left(s_1\right)$ introduced in Equation (30b) expresses effective information for this particular bipartition

$${\Phi }_{\mathrm{eff}}\left(AB\right)=g\left(s_1\right),$$

(41)

thus the analysis of effective information sign in Section 5 applies to this symmetric bipartition.

Moreover, the choice of the symmetric bipartition is consistent with available comparative studies of II measures [18], where it was substantiated by the conceptual requirement that highly asymmetric partitions should be excluded [2], and by the lack of a generally accepted specification of minimum information bipartition; for further discussion, see [18].

We have studied the dependence of the mentioned effective information measures ${\Phi }_{\mathrm{eff}}$ and ${\Phi }^{*}$ upon spiking activity, which is controlled by $s_1$, at different fixed values of the parameters $p_s$ and $ϵ$ characterizing the bursting component. Typical dependence of ${\Phi }_{\mathrm{eff}}$ and ${\Phi }^{*}$ upon $s_1$, taken at $p_s=0.6$ with several values of $ϵ$, is shown in Figure 5, panel (a).

The behavior of the “whole minus sum” effective information ${\Phi }_{\mathrm{eff}}$ (41) (blue lines in Figure 5) is found to agree with the analytical findings of Section 5:

• ${\Phi }_{\mathrm{eff}}$ transitions from negative to positive values at a certain threshold value of $s_1=s_1^{min}$, which is well approximated by the formula (38) when $ϵ$ is small, as required by (36a,b); the result of Equation (38) is indicated in each panel of Figure 5 by an additional vertical grid line labeled $s_1^{min}$ on the abscissae axis—cf. Figure 4;
• ${\Phi }_{\mathrm{eff}}$ reaches a maximum on the interval $s_1^{min}<s_1<1$ and tends to zero (from above) at $s_1\to 1$;
• ${\Phi }_{\mathrm{eff}}$ scales with $ϵ$ as ${ϵ}^2$, when (36a,b) hold.

To verify the scaling observation, we plot the scaled values of both information measures ${\Phi }_{\mathrm{eff}}/{ϵ}^2$, ${\Phi }^{*}/{ϵ}^2$ in the panels (b)–(d) of Figure 5 for several fixed values of $p_s$ and $ϵ$. Expectedly, the scaling fails at $p_s=0.7$, $ϵ=0.4$ in panel (d), as (36b) is not fulfilled in this case.

Furthermore, the “decoder based” information ${\Phi }^{*}$ (plotted with red lines in Figure 5) behaves mostly the same way, apart from being always non-negative (which was one of key motivations for introducing this measure in [16]). At the same time, the sign transition point $s_1^{min}$ of the “whole minus sum” information associates with a rapid growth of the “decoder based” information. When $s_1$ is increased towards 1, the two measures converge. Remarkably, the scaling as ${ϵ}^2$ is found to be shared by both effective information measures.

7. Discussion

In general, the spiking–bursting model is completely specified by the combination of a full single-time probability table $s_x$ (consisting of $2^N$ probabilities of all possible outcomes, where N is the number of bits) for the time-uncorrelated spontaneous activity, along with two independent parameters (e.g., $p_s$ and $ϵ$) for the dichotomous component. This combination is, however, redundant in that it admits a one-parameter scaling (12) which leaves the resultant stochastic process invariant.

Condition (30) was derived assuming that spiking activity in individual bits (i.e., nodes, or neurons) constituting the system is independent among the bits, which implies that the probability table $s_x$ is fully determined by N spike probabilities for individual nodes. The condition is formulated in terms of $p_s$, $ϵ$ and a single parameter $s_1$ (system-wide spike probability) for the spontaneous activity, agnostic of the “internal structure” of the system, i.e., the spike probabilities for individual nodes. This condition provides that the “whole minus sum” effective information is positive for any bipartition, regardless of the mentioned internal structure. Moreover, in the limit (36) of weak correlations in time, the inequality (30a) can be explicitly solved in terms of $s_1$, producing the solution (33), (38).

In this way, the inequality (33) together with the asymptotic estimate (38) supplemented by its applicability range (36) specifies the region in the parameter space of the system, where the “whole minus sum” II is positive regardless of the internal system structure (sufficient condition). The internal structure (though still without spike correlations across the system) is taken into account by the necessary and sufficient condition (27) for positive II.

The mentioned conditions were derived under the assumption of absent correlation between spontaneous activity in individual bits (24). If correlation exists and is positive, then $s_1>s_A{s}_B$, or $s_B<s_1/s_A$. Then comparing the expressions for ${\Phi }_{\mathrm{eff}}$ (23) (general case) to (25) (space-uncorrelated case), and taking into account that $I_0\left(p_s\right)$ is an increasing function, we find ${\Phi }_{\mathrm{eff}}<f\left(s_A\right)$, cf. (25a). This implies that any necessary condition for positive II remains as such. Likewise, in the case of negative correlations we get ${\Phi }_{\mathrm{eff}}>f\left(s_A\right)$, implying that a sufficient condition remains as such.

8. Conclusions

The present study substantiates, refines and quantifies qualitative observations in regard to II in the spiking–bursting model which were initially made in [19]. The existence of lower bounds in spiking activity (characterized by $s_1$) required for positive “whole minus sum” II which was noticed in [19] is now expressed in the form of an explicit inequality (33) with the estimate (38) for the bound $s_1^{min}$. The observation of [19] that typically $s_1^{min}$ is mostly determined by burst probability and weakly depends upon time correlations of bursts also becomes supported by the quantitative result (33), (38). In particular, there is a range of spiking activity intensity $s_1\gtrsim 0.17$, where the “whole minus sum” information is positive regardless of other system parameters, provided the spiking activity is spatially uncorrelated or negatively correlated across the system. When the burst probability is decreased (which implies less frequent activation of the astrocyte subsystem), the threshold value for spiking activity $s_1^{min}$ also decreases.

We found that II scales as ${ϵ}^2$, where $ϵ$ is proportional (as per Equation (17)) to the Pearson’s time delayed correlation coefficient of the bursting component (which essentially characterizes the duration of bursts), for $ϵ$ small (namely, within (36)), when other parameters (i.e., $p_s$ and spiking probability table $s_x$) are fixed. For the “whole minus sum” information, this is an analytical result. Note that the reasoning behind this result does not rely upon the assumption of spatial non-correlation of spiking activity (between bits), and thus applies generally to arbitrary spiking–bursting systems. According to a numerical calculation, this scaling approximately holds for the “decoder based” information as well.

Remarkably, II can not exceed the time delayed mutual information for the system as a whole, which in case of the spiking–bursting model in its present formulation is no greater than 1 bit.

The model provides a basis for possible modifications in order to apply integrated information concepts to systems exhibiting similar, but more complicated behavior (in particular, to neuronal [26,27,28,29] and neuron–astrocyte [24,30] networks). Such modifications might incorporate non-trivial spatial patterns in bursting, and causal interactions within and between the spiking and bursting subsystems.

The model can also be of interest as a new discrete-state test bench for different formalizations of integrated information, while available comparative studies of II measures mainly focus on Gaussian autoregressive models [17,18].