The Fundamental Equations of Change in Statistical Ensembles and Biological Populations

Abstract

A recent article in Nature Physics unified key results from thermodynamics, statistics, and information theory. The unification arose from a general equation for the rate of change in the information content of a system. The general equation describes the change in the moments of an observable quantity over a probability distribution. One term in the equation describes the change in the probability distribution. The other term describes the change in the observable values for a given state. We show the equivalence of this general equation for moment dynamics with the widely known Price equation from evolutionary theory, named after George Price. We introduce the Price equation from its biological roots, review a mathematically abstract form of the equation, and discuss the potential for this equation to unify diverse mathematical theories from different disciplines. The new work in Nature Physics and many applications in biology show that this equation also provides the basis for deriving many novel theoretical results within each discipline.

Nicholson et al. [1] recently published a unification of time-information uncertainty relations in thermodynamics. Their results follow from a single equation that describes the rate of change of the first moment, $\langle A\rangle$, of (thermodynamic) random variables,

$$\frac{\mathrm{d}\langle A\rangle }{\mathrm{d}t}=-\mathrm{Cov}\left(A,\dot{I}\right)+\langle \frac{\mathrm{d}A}{\mathrm{d}t}\rangle ,$$

(1)

with $I_i=-ln{p}_i$, as the “surprisal” of state i, which has the Shannon entropy as its mean value. Accordingly, $A_i$ is a property of state i. Nicholson et al. [1] illustrate how various insights about irreversible thermodynamics and its relations with information theory follow directly from this equation of motion, unifying several aspects of thermodynamics. We read their work with great interest and with admiration for their steps toward theoretical unification.

In this note, we point to the importance of Equation (1) in evolutionary biology, where it is known as the “Price equation” after George Price [2,3]. A recent volume of the Philosophical Transactions of the Royal Society B collected numerous articles to celebrate the 50th anniversary of Price’s first publication of his famous equation [4].

To show the connections between Nicholson et al.’s work, the Price equation, and a potentially broader unification of various disciplines, we start with the classic evolutionary theory interpretation of the Price equation [5]. We then extend to a more abstract version of the Price equation, which potentially unifies many seemingly distinct results in biology, physics, probability, and statistics [6,7].

Evolutionary theory provides understanding of the changes in frequencies of things that replicate themselves. For simplicity, we may consider the DNA sequence of a bacterial cell as a single replicator, because the whole sequence often gets replicated when it passes to offspring.

Replicators compete for resources and mutate, which is why they change in frequency. The frequency of a replicator increases when it makes more offspring than a less fit replicator. Here, a replicator’s “fitness” is thought of as its performance. For example, a replicator’s gene may code for a protein that influences how well the organism fits to its environment—how well its traits are adapted to the conditions of life. The Price equation relates all those processes.

Accordingly, in Equation (1), $p_i$ refers to the frequency of a replicator i that changes in time due to evolutionary processes such as competition and mutation. Differential success in competition (selection) influences fitness. The random variable $A_i$ corresponds to the value of a trait associated with a replicator of type i. The population’s average value of A increases if values of A covary with the tendency for its associated replicator to increase in frequency. That covariance is captured by the first term in Equation (1).

The second term of the Price equation captures the changes in trait values uncorrelated with changes in replicator frequency. For example, if the environment improves, then the measured value for the trait expressed by all individuals may increase. That increase has nothing to do with differential success between replicators and so lands in the second term. Often, one can think of the second term as a change in the frame of reference that alters measurements, and the first term as direct forces that change the frequency of particular measurements [6].

Because of the simple way that the Price equation relates change in trait values to change in replicator frequencies, it makes sense that this equation has been recognized as a foundational expression of evolutionary theory [4,8].

Returning to Nicholson et al., they use Equation (1) to derive a variety of interesting and general results about the change in the observable quantities, A, in thermodynamics, information, and statistics. For example, in their section Information fluctuations and intrinsic speed, they show that the time evolution of the distance moved by a probability distribution over states is the Fisher information metric.

A similar result follows in evolutionary biology from the Price equation (Equation (1)), which illustrates that the Fisher information path length between probability distributions describes the evolutionary change in biological populations [9]. That prior result also noted the close relation between the Fisher information metric, which arose from the Price equation, and the foundations of information geometry [10].

To illustrate the connections between the Price equation and a wide variety of fundamental equations in different scientific disciplines, we consider an abstract version of the Price equation [5,6]. We use z for measured values, instead of Nicholson et al.’s a, to avoid a notational clash with our earlier publications, which used a for a different variable. We closely follow the derivation given in Frank [7].

The equation describes the change in the average value of some property between two populations. A population is a set of things. Each thing has a property indexed by i. Those things with a common property index comprise a fraction, $p_i$, of the population and have average value, $z_i$, for whatever we choose to measure by z.

Write $\mathbf{p}$ and $\mathbf{z}$ as the vectors over all i. The population average value is $\langle z\rangle =\mathbf{p}·\mathbf{z}=\sum p_i{z}_i$, summed over i.

A second population has matching vectors ${\mathbf{p}}^{\prime }$ and ${\mathbf{z}}^{\prime }$. Here, $p_i^{\prime }$ is the fraction of the second population derived from entities with index i in the first population. Similarly, $z_i^{\prime }$ is the average value in the second population of members derived from entities with index i in the first population. This notation expresses an abstract mapping relation between two sets, in which the common dynamical notion of frequency change and value change over time is a special case.

Let $\Delta$ be the difference between the derived population and the original population, $\Delta \mathbf{p}={\mathbf{p}}^{\prime }-\mathbf{p}$ and $\Delta \mathbf{z}={\mathbf{z}}^{\prime }-\mathbf{z}$. The difference in the averages is $\Delta \langle z\rangle ={\mathbf{p}}^{\prime }·{\mathbf{z}}^{\prime }-\mathbf{p}·\mathbf{z}$. By using the definitions for $\Delta \mathbf{p}$ and $\Delta \mathbf{z}$, we can write the change in the average as the abstract form of the Price equation

$$\Delta \langle z\rangle =\Delta \mathbf{p}·\mathbf{z}+{\mathbf{p}}^{\prime }·\Delta \mathbf{z}.$$

(2)

The first term, $\Delta \mathbf{p}·\mathbf{z}$ is the partial difference of $\mathbf{p}$ holding $\mathbf{z}$ constant. The second term, ${\mathbf{p}}^{\prime }·\Delta \mathbf{z}$, is the partial difference of $\mathbf{z}$ holding $\mathbf{p}$ constant. In the second term, we use ${\mathbf{p}}^{\prime }$ as the constant value because, with discrete differences, one of the partial change terms must be evaluated in the context of the second set.

The dot product is mathematically equivalent to the statistical covariance function, $\Delta \mathbf{p}·\mathbf{z}=\mathrm{Cov}(\mathbf{\varphi },\mathbf{z})$, in which we define ${\varphi }_i=\Delta p_i/p_i$. We abuse standard notation to write $\mathbf{\varphi }=\Delta \mathbf{p}/\mathbf{p}$, defining the division of vectors as an elementwise division of elements.

The term $\mathbf{\varphi }$ is Fisher’s average excess in fitness in biological theory. In our prior work, $\mathbf{\varphi }\equiv \mathbf{a}$, clashing with the notation of Nicholson et al. (We use “≡” to mean equivalence under the specified conditions.) Note that in the limit of small changes, $\mathbf{\varphi }\to \mathrm{d}\mathbf{p}/\mathbf{p}=\mathrm{d}ln\mathbf{p}=-\dot{I}$, linking Nicholson et al.’s notation to ours. After matching differences to differentials with respect to time, $\Delta \equiv \mathrm{d}/\mathrm{d}t$, Equation (1) follows, noting in Equation (2) that the second term in the differential limit is ${\mathbf{p}}^{\prime }·\Delta \mathbf{z}\to \mathbf{p}·\mathrm{d}\mathbf{z}/\mathrm{d}t=\langle \mathrm{d}z/\mathrm{d}t\rangle$.

Because $\mathbf{z}$ is any arbitrary value associated with entities, we can take $\mathbf{z}\equiv \mathbf{\varphi }$, yielding

$$\Delta \langle \mathbf{\varphi }\rangle =\Delta \mathbf{p}·\mathbf{\varphi }+{\mathbf{p}}^{\prime }·\Delta \mathbf{\varphi }=0.$$

(3)

Importantly, $\langle \mathbf{\varphi }\rangle$ is always zero, because $\langle \varphi \rangle ={\sum }_i\Delta p_i$, the total probability change, which is zero. Because we always standardize total probability to be one, we may say that Equation (3) expresses the conservation law for total probability.

The conservation of total probability may seem at first glance to be a trivial standardization. However, that conservation is what gives common form to seemingly different disciplines. This particular conservation associates with a shift-invariance for probability values, a symmetry that constrains geometric properties for the dynamics of probability distributions and shapes the common forms of observed probability distributions [6,7,11,12].

The conserved system in Equation (3) leads to a variety of deep results and connections between disciplines [6,7]. We can always proceed to nonconserved systems by the transformation $\mathbf{\varphi }\mapsto \mathbf{z}$, in which $\langle z\rangle$ is not necessarily conserved.

Given the limited space here, we confine ourselves to one example that links the Price equation, biological fitness, information theory, and a generalized notion of d’Alembert’s separation between direct and inertial forces [13].

To start, note that the most fundamental measure of biological force concerns fitness, the relative success of replicators in competition with other replicators. That force drives evolutionary change through natural selection. Working with differentials to match Nicholson et al., we noted above that Fisher’s average excess in fitness is equivalent to minus the differential of the surprisal, $\mathbf{\varphi }\to \mathrm{d}\mathbf{p}/\mathbf{p}=\mathrm{d}ln\mathbf{p}=-\dot{I}$. This observation shows the essential equivalence in the mathematics of biological fitness and the mathematics of information theory and, as in Nicholson et al., the close association between the dynamics of population statistics and information.

For differentials, the first term of the Price equation with $\mathbf{z}\equiv \mathbf{\varphi }$ is

$$\mathrm{d}\mathbf{p}·\mathbf{\varphi }={\parallel \frac{\mathrm{d}\mathbf{p}}{\sqrt{\mathbf{p}}}\parallel }^2=\mathcal{F},$$

in which $\mathbf{\varphi }$ is an abstract notion of direct force acting on frequency, $\parallel ·\parallel$ is the Euclidean length of a vector, and $\mathcal{F}$ is an abstract, nondimensional expression of the Fisher information distance metric.

To match inertial forces, $\mathbf{N}$, to the second Price equation term, $\mathbf{p}·\mathrm{d}\mathbf{\varphi }$, we define $\mathbf{N}=\mathrm{d}ln\mathbf{\varphi }$, and thus

$$\mathbf{p}·\mathrm{d}\mathbf{\varphi }=\mathrm{d}\mathbf{p}·\mathbf{N}.$$

We can also write

$$\mathbf{N}=\mathrm{d}ln\mathbf{\varphi }=\mathrm{d}ln(\mathrm{d}ln\mathbf{p})=\mathrm{d}{ln}^2\mathbf{p}.$$

The relative differential, $\mathrm{d}ln$ describes relative change. The second relative differential, $\mathbf{N}=\mathrm{d}{ln}^2\mathbf{p}$, describes the relative acceleration in frequency changes. Thus, the inertial forces acting on the frame of reference can be related to an acceleration.

Combining the expressions for the direct and inertial forces yields an alternative expression for the special Price equation form of Equation (3) as

$$\left(\mathbf{F}+\mathbf{N}\right)·\mathrm{d}\mathbf{p}=\left(\mathrm{d}ln\mathbf{p}+\mathrm{d}{ln}^2\mathbf{p}\right)·\mathrm{d}\mathbf{p}=0.$$

This expression is an abstract, nondimensional generalization of d’Alembert’s principle for probability distributions that conserve total probability [6,14,15].

D’Alembert’s principle of physical mechanics generalizes Newton’s second law, force equals mass times acceleration [13]. In one dimension, Newton’s law is $F=-mN$, for force, F, and mass, m, times acceleration, $-N$. In our abstract nondimensional expressions, m drops out, so that $F+N=0$.

D’Alembert generalizes Newton’s law to a statement about motion in multiple dimensions such that, in conservative systems, the total work for a displacement, $\mathrm{d}\mathbf{p}$, and total forces, $\mathbf{F}+\mathbf{N}$, is zero. Work is the distance moved multiplied by the force acting in the direction of motion. Probability distributions are conservative in total probability. Extensions to nonconservative systems arise in some cases.

Many connections arise between this abstract expression of direct and inertial forces and fundamental equations in biology, information, thermodynamics, probability, and statistics [6]. In biology, the Price equation transformed understanding for how natural selection shapes the competitive and cooperative tendencies of organisms. Variants of the Price equation also provide the foundation for animal breeding and the analysis of traits influenced by many genes, each with a small effect [4].

Finally, recent work has extended the general form of the Price equation to describe the change in population statistics that cannot be expressed as moments. The harmonic mean provides an example [16].

In summary, Nicholson et al.’s article shows how independent lines of thought from different disciplines are now converging on a common and more general understanding of change in statistical ensembles and biological populations. Their work provides the basis for a significant conceptual unification of the sciences.