The Maximum Entropy Rate Description of a Thermodynamic System in a Stationary Non-Equilibrium State

Abstract

In this paper we present a simple model to describe a rather general system in a stationary non-equilibrium state, which is an open system traversed by a stationary flux. The probabilistic description is provided by a non-homogeneous Markov chain, which is not assumed on the basis of a model of the microscopic interactions but rather derived from the knowledge of the macroscopic fluxes traversing the system through a maximum entropy rate principle.

1. Introduction

In a recent (2006, [1]) survey on the Maximum Entropy Production principle, reference is made to an early attempt (1967, [2,3]) to extend (a pioneering paper dealing with this extension problem was written by E.T. Jaynes himself, see [4]) the celebrated Jaynes’ Maximum Entropy principle to thermodynamic systems in a stationary non-equilibrium. Informally stated, assumed that the dynamic is described by a Markov chain, the authors want to find a stochastic transition matrix such that: (i) it has a prescribed probability distribution as a stationary distribution, (ii) the associated chain evolution satisfies given constraints on the macroscopic scale (admissible microscopic evolution) and (iii) the selected transition matrix generates the maximum number of equally probable microscopic evolution paths. Motivated by their derivation, we proceed here to a self-contained independent approach to the same problem with, we think, more far–reaching consequences.

To state the main ideas, let us begin by recalling a restricted version of the ergodic theorem for Markov chains ([5]). Let us suppose that the system has a finite state space $\chi =\{1,\dots ,n\}$ and that its statistical description is given by a stationary, time-homogeneous Markov chain.

Theorem 1 Let $P\in Mat\left(n\right)$ be a stochastic matrix with positive entries and denote with $P^N$ the $N^{th}$ power of P. Then there exists a unique distribution π which is stationary for P, i.e., $P^{*}\pi =\pi$ where $P^{*}$ denotes the transposed matrix of P. Moreover,

$$\underset{N\to +\infty }{lim}{P}_{ij}^N={\pi }_j>0\forall i,j=1,\dots ,n$$

and, for every probability distribution ν we have, setting $\left|\right|\nu -\mu \left|\right|=\frac{1}{2}{\sum }_{i=1}^n|{\nu }_i-{\mu }_i|$,

$$\underset{N\to +\infty }{lim}\left|\right|{\left(P^N\right)}^{*}\nu -\pi \left|\right|=0$$

It is immediate to realize from the above Theorem that the stochastic matrix P determines its stationary distribution, while the contrary is false. In this paper we put forth a selection criterion to choose among all stochastic matrices admitting a fixed distribution π as stationary. In a sense, we want to select a preferred dynamics for the approach to equilibrium π. To this, recall that if we take the stationary distribution π as the initial distribution, then the Markov chain $(P,\pi )$ defines a discrete-time, finite-states stationary stochastic process. Let $X_i,i\ge 0$ be the χ-valued random variable which describe the state of the system at time i. For a stationary process, we can define its entropy rate (see [6]) as

$$\mathcal{H}=\underset{N\to +\infty }{lim}\frac{1}{N}H(X_0,\dots ,X_{N-1})$$

where $\Omega ={\chi }^N$ and

$$H(X_0,\dots ,X_{N-1})=-\sum _{\omega \in \Omega }p\left(\omega \right)lnp\left(\omega \right)$$

In a sense, the entropy rate $\mathcal{H}$ is the thermodynamic N-limit of the entropy of the system formed by all realizations of length N of the process. Therefore, we will call it the system entropy rate in the sequel. Moreover, for the stationary process associated to $(P,\pi )$ it holds that (see [5],[6])

$$\mathcal{H}=\mathcal{H}\left(P\right)=-\sum _{i,j}{\pi }_i{P}_{ij}ln{P}_{ij}$$

Furthermore, in agreement with (ii) above, we assume that the knowledge of the macroscopic fluxes acting on the system in a stationary non-equilibrium state will force constraints on the two-dimensional probability distribution

$${\mathcal{P}}_{ij}=Prob(X_{t+1}=j,X_t=i)={\pi }_i{P}_{ij}$$

in the form of constraints $P\in \Lambda$ on the associated stochastic matrix (see Section 2 below). We can now state our selection criterion, which we call the Maximum Entropy Rate principle and amounts to the following constrained extremum problem:

M.E.R.P. Given a positive probability distribution π, find the stochastic matrix $\widehat{P}\in \Lambda$ which admits π as a stationary distribution and that has maximum entropy rate $\mathcal{H}\left(\widehat{P}\right)$.

This principle can be seen as an instance of a Maximum Entropy principle for the case of constraints on two-dimensional distributions. Its justification is provided in [7] (see also [8],[9]) using a large deviation theory type estimate on the empirical second order distributions $\widehat{\mathcal{P}}$, in the same spirit of Sanov’ Theorem for first order empirical distributions.

Remark. In the case that the only constraints on P are the stationarity and normalization ones, ${\widehat{P}}_{ij}={\pi }_j,i,j=1,\dots ,n.$ The answer to this problem can be easily found by using the elementary inequality, see [6]

$$\mathcal{H}\left(P\right)=-\sum _{i,j}{\pi }_i{P}_{ij}ln{P}_{ij}\le H\left(\pi \right)=-\sum _i{\pi }_{i}ln{\pi }_i$$

(1.1)

with equality if and only if ${\widehat{P}}_{ij}={\pi }_j,i,j=1,\dots ,n.$ (all rows are equal). This result is a simple consequence of the fact that the entropy $H\left(\pi \right)$ can be seen as the entropy rate of a stationary process with i.i.d. random variables $X_i\sim \pi$ and that conditioning reduces the entropy rate. It is immediate to verify that $\widehat{P}$ admits π as a stationary distribution. By recalling the ergodic theorem, we see also that $P^N=\widehat{P}$ for every N.

The starting point of the above M.E.R. principle is the probability distribution π, which is, apart from being positive, completely arbitrary. It affords the statistical description of our system at equilibrium. In case we have some macroscopic information on the system, for example its average energy, that we represent as

$$e={\mathbb{E}}_{\pi }\left(E\right)=\sum _{i=1}^n{\pi }_i{E}_i$$

where $E_i$ is the energy of the system in state i, then this information can be used to select a probability distribution $\widehat{\pi }$ among all these satisfying the above constraint. By applying the Maximum Entropy principle one gets

$${\widehat{\pi }}_i=\frac{e^{-\beta E_i}}{Z\left(\beta \right)}$$

(1.2)

the Gibbs probability distribution. Here the inverse temperature $\beta =\frac{1}{kT}$ is uniquely determined by the value e.

2. Model of the System in a Stationary Non-Equilibrium State

We begin with the model of the energy exchange between our system χ and another system $\mathcal{A}$. Let us suppose that the system is described by our stationary Markov chain $(P,\pi )$. We suppose also that the whole system $\mathcal{A}\cup \chi$ is energetically insulated, that is if $X_t=i$ and $X_{t+1}=j$, then the microscopic energy conservation law holds

$$\Delta E^{\chi }=E_j-E_i=-\Delta E^{\mathcal{A}}\forall i,j=1,\dots ,n$$

Let us denote with

$${\mathcal{E}}_{ij}=E_j-E_i$$

the skew-symmetric matrix of the difference of energy between states and with

$${\mathcal{P}}_{ij}=Prob(X_{t+1}=j,X_t=i)={\pi }_i{P}_{ij}$$

the joint probability at one time step. The average energy transfer between $\mathcal{A}$ and χ is

$${\mathbb{E}}_{\mathcal{P}}[\Delta E^{\chi }]=\mathcal{E}·\mathcal{P}=\sum _{i,j}{\mathcal{P}}_{ij}{\mathcal{E}}_{ij}=\sum _{i,j}{\pi }_i{P}_{ij}{E}_j-\sum _{i,j}{\pi }_i{P}_{ij}{E}_i={\mathbb{E}}_{P^{*}\pi }\left(E\right)-{\mathbb{E}}_{\pi }\left(E\right)$$

and it follows easily that if π is stationary for P, and if our microscopic energy conservation law holds, then

$${\mathbb{E}}_{\mathcal{P}}[\Delta E^{\chi }]=-{\mathbb{E}}_{\mathcal{P}}[\Delta E^{\mathcal{A}}]=0$$

hence the stationary distribution π describes the system χ at macroscopic equilibrium with $\mathcal{A}$.

2.1. Coupling of the system with two environments

We now want to model the coupling of the system with two environments $\mathcal{A}$ and $\mathcal{B}$. As before, the statistical description of the system is given by a Markov chain. As a simplifying assumption, we suppose that at every time step along the evolution of the chain, the system χ is in contact (i.e., the microscopic energy conservation holds) with only one of the environments, alternatively $\mathcal{A}$ and $\mathcal{B}$. Therefore, now the system is described by the non-homogeneous Markov chain ($m\in \mathbb{N}$)

$$P_t=\left\{\begin{array}{c}At=2m,\hfill \\ Bt=2m+1\hfill \end{array}\right.$$

(2.1)

We will see that it is not restrictive to suppose that the stochastic matrices $A,B$ have positive entries. Also, notice that the chain “observed” at even times $t=2m$ is a time-homogeneous chain described by the positive stochastic matrix $AB$, hence the ergodic theorem applies. Let us denote with π the unique stationary distribution for $AB$. If π is the distribution describing the system χ at $t=0$, then we have

$$\pi \left(0\right)=\pi ,\pi \left(1\right)=A^{*}\pi ,\pi \left(2\right)=B^{*}{A}^{*}\pi ={\left(AB\right)}^{*}\pi =\pi ,\pi \left(3\right)=A^{*}\pi ,\dots$$

Therefore, the probability distribution describing the chain switches between π (at even times) and $A^{*}\pi$ (at odd times). Let us compute as before the joint probability at one and two time-steps starting from an even time $t=2m$ . We have

$$\begin{array}{c}{\mathcal{P}}_{ij}(2m,2m+1)=Prob(X_{2m+1}=j,X_{2m}=i)={\pi }_i{A}_{ij}\hfill \\ \\ {\mathcal{P}}_{jk}(2m+1,2m+2)=Prob(X_{2m+2}=k,X_{2m+1}=j)=\sum _i{\pi }_i{A}_{ij}{B}_{jk}\hfill \\ \\ {\mathcal{P}}_{ijk}(2m,2m+2)=Prob(X_{2m+2}=k,X_{2m+1}=j,X_{2m}=i)={\pi }_i{A}_{ij}{B}_{jk}\hfill \end{array}$$

Reasoning as before, we can compute the averaged energy differences

$$\begin{array}{c}{\mathbb{E}}_{\mathcal{P}}[\Delta E^{\chi }](2m,2m+1)=-{\mathbb{E}}_{\mathcal{P}}[\Delta E^{\mathcal{A}}]={\mathbb{E}}_{A^{*}\pi }\left(E\right)-{\mathbb{E}}_{\pi }\left(E\right)\hfill \\ \\ {\mathbb{E}}_{\mathcal{P}}[\Delta E^{\chi }](2m+1,2m+2)=-{\mathbb{E}}_{\mathcal{P}}[\Delta E^{\mathcal{B}}]={\mathbb{E}}_{{\left(AB\right)}^{*}\pi }\left(E\right)-{\mathbb{E}}_{A^{*}\pi }\left(E\right)\hfill \\ \\ {\mathbb{E}}_{\mathcal{P}}[\Delta E^{\chi }](2m,2m+2)={\mathbb{E}}_{{\left(AB\right)}^{*}\pi }\left(E\right)-{\mathbb{E}}_{\pi }\left(E\right)\hfill \end{array}$$

Modifications of the above formulae for the case that we start the observation of the chain at an odd time are straightforward. By a simple inspection of the above equalities we can draw the following conclusions:

Proposition 1 If π is a stationary distribution for $AB$, then

$${\mathbb{E}}_{\mathcal{P}}[\Delta E^{\mathcal{A}}]=-{\mathbb{E}}_{\mathcal{P}}[\Delta E^{\chi }](2m,2m+1)={\mathbb{E}}_{\mathcal{P}}[\Delta E^{\chi }](2m+1,2m+2)=-{\mathbb{E}}_{\mathcal{P}}[\Delta E^{\mathcal{B}}]$$

(2.2)

and also

$${\mathbb{E}}_{\mathcal{P}}[\Delta E^{\chi }](2m,2m+2)=0.$$

(2.3)

Moreover, if π is a stationary distribution for $AB$ and A, then it is a stationary distribution for B also and

$${\mathbb{E}}_{\mathcal{P}}[\Delta E^{\mathcal{A}}]=-{\mathbb{E}}_{\mathcal{P}}[\Delta E^{\mathcal{B}}]={\mathbb{E}}_{\mathcal{P}}[\Delta E^{\chi }](2m,2m+2)=0$$

Therefore, with a distribution π stationary for $AB$ and A we model a system at equilibrium with the two environments, and with a probability distribution π stationary for $AB$ but not for A, we model a system in a stationary non equilibrium state. Moreover, the condition of stationarity of π with respect to $AB$, which is sufficient (but not necessary in the general case) for the balance of energy to hold, is the one that will allow to compute the entropy rate. The value of the macroscopic energy flux can be specified by the macroscopic constraint introduced by the first equality in (2.2)

$${\mathbb{E}}_{\mathcal{P}}[\Delta E^{\chi }](2m,2m+1)=-{\mathbb{E}}_{\mathcal{P}}[\Delta E^{\mathcal{A}}]={\mathbb{E}}_{\mathcal{P}}[\Delta E^{\mathcal{B}}]={\mathbb{E}}_{A^{*}\pi }\left(E\right)-{\mathbb{E}}_{\pi }\left(E\right)=q\ge 0$$

Note that, as before, we made no assumption on the distribution π. We have to choose matrices A and B satisfying the following constraints (here $\mathbf{1}={(1,\dots ,1)}^{*}$)

$$\left\{\begin{array}{c}A·\mathbf{1}=B·\mathbf{1}=\mathbf{1}i.e.\sum _j{A}_{ij}=\sum _j{B}_{ij}=1\forall i(\text{normalization of }A \text{and} B)\hfill \\ \\ {\mathbb{E}}_{A^{*}\pi }\left(E\right)-{\mathbb{E}}_{\pi }\left(E\right)=q,i.e.\sum _{ij}{\pi }_i{A}_{ij}{\mathcal{E}}_{ij}=q,(\text{specify} \text{inflow})\hfill \\ \\ {\mathbb{E}}_{{\left(AB\right)}^{*}\pi }\left(E\right)-{\mathbb{E}}_{\pi }\left(E\right)=0,i.e.\sum _{ij}{\pi }_i{A}_{ij}{B}_{jk}={\pi }_k\forall k(\pi \text{is} \text{stationary} \text{for} AB)\hfill \end{array}\right.$$

(2.4)

The above constraints do not specify in the general case the matrices A and B. Therefore, we need to invoke a selection criterion, which will be our M.E.R. principle. We are led to investigate the existence of an entropy rate for the stochastic process described by the non-homogeneous Markov chain (2.1) with initial distribution π stationary for $AB$. Before turning to this, we look at the solution of our problem in case of zero flux, $q=0$. It is immediate to see that if we limit ourselves to the case $A=B$, then the chain is ergodic and

$$A=B=\widehat{P},where{\widehat{P}}_{ij}={\pi }_j$$

is the maximum entropy rate solution.

2.2. Computation of the entropy rate

It is easy to see that the chain described by (2.1) with initial distribution π stationary for $AB$ but not for A describes a non-stationary stochastic process. Moreover, the chain is not strongly ergodic, but, with the non restricting assumption that A and B have positive entries it is weakly ergodic (see e.g., [10] for the notion of strong and weak ergodicity and [11],[12] and the bibliography therein for the study of convergence of Markov processes using information theoretic tools.) We now show that the entropy rate is well defined and finite for the process at hand. This is not surprising since the inhomogeneity of the chain is very mild and the chain will become homogeneous with a suitable time-reparameterization, However, we will proceed with the chain as it is for simplicity’ sake. Recall that the probability of a typical sequence $\omega \in \Omega ={\chi }^N$ of length N of the chain with initial distribution π is

$$p\left(\omega \right)=p(i_0,i_1,\dots ,i_{N-1})={\pi }_{i_0}{A}_{i_0{i}_1}{B}_{i_1{i}_2}{A}_{i_2{i}_3}\dots$$

To compute $H(X_0,\dots ,X_{N-1})$ we use the well known chain rule (see [6]):

$$H(X_0,\dots ,X_{N-1})=\sum _{k=0}^{N-1}H\left(X_k\right|X_0,\dots ,X_{k-1})=H\left(X_0\right)+H\left(X_1\right|X_0)+H\left(X_2\right|X_1,X_0)+\dots .$$

Therefore, we have

$$\begin{array}{c}H\left(X_0\right)=H\left(\pi \right)=-\sum _{i_0}{\pi }_{i_0}ln{\pi }_{i_0}\hfill \\ \\ H\left(X_1\right|X_0)=-\sum _{i_0,i_1}p(i_0,i_1)lnp\left(i_1\right|i_0)=-\sum _{i_0,i_1}{\pi }_{i_0}{A}_{i_0{i}_1}ln{A}_{i_0{i}_1}\hfill \\ \\ H\left(X_2\right|X_1,X_0)=-\sum _{i_0,i_1,i_2}p(i_0,i_1,i_2)lnp\left(i_2\right|i_0,i_1)=-\sum _{i_0,i_1,i_2}{\pi }_{i_0}{A}_{i_0{i}_1}{B}_{i_1{i}_2}ln{B}_{i_1{i}_2}\hfill \end{array}$$

Hence, if the condition of stationary ${\left(AB\right)}^{*}\pi =\pi$ holds, the terms following $H\left(X_0\right)$ alternate between the form of

$$H\left(X_1\right|X_0)=H(\pi ,A)=-\sum _{i_0,i_1}{\pi }_{i_0}{A}_{i_0{i}_1}ln{A}_{i_0{i}_1}$$

and

$$H\left(X_2\right|X_1,X_0)=H(A^{*}\pi ,B)=-\sum _{i_0,i_1,i_2}{\pi }_{i_0}{A}_{i_0{i}_1}{B}_{i_1{i}_2}ln{B}_{i_1{i}_2}$$

Hence,

$$\mathcal{H}=\underset{N\to +\infty }{lim}\frac{1}{N}H(X_0,\dots ,X_{N-1})=\frac{1}{2}(H(\pi ,A)+H(A^{*}\pi ,B))$$

or

$$\mathcal{H}=\mathcal{H}(A,B)=-\frac{1}{2}(\sum _{i,j=1}^n{\pi }_i{A}_{ij}ln{A}_{ij}+\sum _{i,j,k=1}^n{\pi }_i{A}_{ij}{B}_{jk}ln{B}_{jk})$$

(2.5)

2.3. Application of the M.E.R. principle and solution

In this section we solve the constrained extremum problem for the objective function $\mathcal{H}(A,B)$ subject to the constraints (2.4) using the Lagrange multipliers method. The Lagrange function is

$$G(A,B)=\mathcal{H}(A,B)-\sum _{ij}{\gamma }_i{A}_{ij}--\sum _{jk}{\lambda }_j{B}_{jk}-\sum _{ijk}{\mu }_k{\pi }_i{A}_{ij}{B}_{jk}-\beta \sum _{ij}{\pi }_i{A}_{ij}{\mathcal{E}}_{ij}$$

(2.6)

and the necessary condition for the extremum are

$$\frac{\partial G}{\partial A_{ij}}=0,\frac{\partial G}{\partial B_{jk}}=0,\forall i,j,k.$$

By simple computations we get the expression for the solution A and B in terms of the unknown multipliers as

$$\begin{array}{c}{\widehat{A}}_{ij}=e^{\frac{{\lambda }_j}{{\left(A^{*}\pi \right)}_j}}{e}^{-\frac{{\gamma }_i}{{\pi }_i}}{e}^{-\beta {\mathcal{E}}_{ij}}=:x_j\left(\lambda \right)y_i\left(\gamma \right)Q_{ij}\left(\beta \right)\hfill \\ \\ {\widehat{B}}_{jk}=e^{-\frac{{\lambda }_j}{{\left(A^{*}\pi \right)}_j}}{e}^{{\mu }_k+1}=:x_j^{-1}\left(\lambda \right)z_k\left(\mu \right)\hfill \end{array}$$

(2.7)

In the following we use $x,y,z,Q$ as unknowns in place of the Lagrange multipliers. By using the normality constraint on A and B respectively we get

$$y_i=\frac{1}{{\sum }_j{Q}_{ij}{x}_j},x_j=\sum _k{z}_k:=z$$

By using the stationarity constraint of π with respect to $AB$ we get the following equations for $z_k$, $k=1,\dots ,n$,

$${\pi }_k=\sum _{ij}{\pi }_i\frac{Q_{ij}}{{\sum }_j{Q}_{ij}}\frac{z_k}{z}:=\sum _{ij}{\pi }_i{\tilde{Q}}_{ij}\frac{z_k}{z}\text{where}{\tilde{Q}}_{ij}=\frac{Q_{ij}}{{\sum }_j{Q}_{ij}}$$

Since the above introduced matrix $\tilde{Q}$ is a stochastic one with positive entries ${\sum }_j{\tilde{Q}}_{ij}=1$, it is easy to see that the solution of the above equation for $z_k$ is $z_k={\pi }_k$. Hence, $x_j=1$ and from (2.7)

$${\widehat{A}}_{ij}={\tilde{Q}}_{ij},{\widehat{B}}_{jk}={\pi }_k.$$

Before turning to the inflow constraint to determine the multiplier β, we note that the matrix $\tilde{Q}$ admits a simpler form using the definition ${\mathcal{E}}_{ij}=E_j-E_i$

$${\tilde{Q}}_{ij}=\frac{Q_{ij}}{{\sum }_j{Q}_{ij}}=\frac{e^{-\beta {\mathcal{E}}_{ij}}}{{\sum }_j{e}^{-\beta {\mathcal{E}}_{ij}}}=\frac{e^{-\beta E_j}}{{\sum }_j{e}^{-\beta E_j}}=:{\tilde{\pi }}_j$$

Therefore ${\widehat{A}}^{*}\pi =\tilde{\pi }$, since

$${\left({\widehat{A}}^{*}\pi \right)}_i=\sum _j{\tilde{Q}}_{ji}{\pi }_j=\sum _j{\tilde{\pi }}_i{\pi }_j={\tilde{\pi }}_i$$

Now the inflow constraint can be rewritten as

$$q=\sum _{ij}{\pi }_i{\widehat{A}}_{ij}{\mathcal{E}}_{ij}=\sum _{ij}{\pi }_i{\tilde{Q}}_{ij}{\mathcal{E}}_{ij}=\sum _{ij}{\pi }_i{\tilde{\pi }}_j{\mathcal{E}}_{ij}={\mathbb{E}}_{\tilde{\pi }}\left(E\right)-{\mathbb{E}}_{\pi }\left(E\right)$$

Hence, for any given π and q, setting $e={\mathbb{E}}_{\pi }\left(E\right)$, the Lagrange multiplier $\beta =\beta (e+q)$ is uniquely determined by the equation

$${\mathbb{E}}_{\tilde{\pi }}\left(E\right)=-\frac{\partial lnZ\left(\beta \right)}{\partial \beta }=\sum _j{E}_j\frac{e^{-\beta E_j}}{Z\left(\beta \right)}=q+e$$

(2.8)

We conclude by noting that, if the relation $e={\mathbb{E}}_{\pi }\left(E\right)$ is seen as a constraint for the unknown probability distribution π, then the maximum entropy assignation for π is the Gibbs distribution

$${\pi }_i=\frac{e^{-\beta E_i}}{Z\left(\beta \right)}where\beta =\beta \left(e\right).$$

(2.9)

We have found that the maximum entropy rate assignation for the non-stationary stochastic process described by the non-homogeneous Markov chain (2.1) defined by the stochastic matrices A and B and by the probability distribution π which is stationary for $AB$ is

$$A_{ij}={\tilde{\pi }}_j,B_{jk}={\pi }_k$$

(2.10)

where $\tilde{\pi }$ is the Gibbs distribution for $\beta =\beta (q+e)$ and π is the Gibbs distribution for $\beta =\beta \left(e\right)$. As expected, the solution depends only on the macroscopic information supplied: the equilibrium energy e and the flow q.

2.4. Entropy rate and entropy production

By a direct computation from (1.1), (2.8), (2.10), the entropy rate of the process is the sum of two terms of the type

$$H\left(\pi \right)=lnZ\left(\beta \right)+e\beta =S\left(e\right)$$

Hence, from (2.5)

$$\mathcal{H}(A,B)=\mathcal{H}(q,e)=\frac{1}{2}(H\left(\tilde{\pi }\right)+H\left(\pi \right))=\frac{1}{2}(S(e+q)+S\left(e\right))$$

(2.11)

Since the chain spends “half of its time” in a state with average energy e and the remaining half in a state with average energy $e+q$, the above formula proposes that the entropy rate be the time average of its “instantaneous” entropies.

Which is the relation between the entropy rate of the stochastic process and the thermodynamic entropy of the system in a stationary non-equilibrium state? If q is small, we can consider the Taylor expansion of $\mathcal{H}(q,e)$ w.r.t. q and get

$$\mathcal{H}(q,e)=\mathcal{H}(0,e)+\frac{\partial \mathcal{H}}{\partial q}(0,e)q+\mathcal{O}\left(q^2\right)=S\left(e\right)+\frac{1}{2}\frac{\partial S}{\partial e}\left(e\right)q+\mathcal{O}\left(q^2\right)$$

Hence, by the well known identification $\beta =1/kT$,

$$\mathcal{H}(q,e)=S\left(e\right)+\frac{1}{2}\beta \left(e\right)q+\mathcal{O}\left(q^2\right)=S\left(e\right)+\frac{1}{2}\frac{q}{kT\left(e\right)}+\mathcal{O}\left(q^2\right)$$

(2.12)

In the above formula, the entropy rate is the sum of two terms, one of which $S\left(e\right)$ is non negative while the other has the sign of q. It is appealing to interpret the first as the source term an the other as the flux term.

Moreover, from the relation between the average energy e and the related multiplier $\beta \left(e\right)$, we have that, if q is small

$$e+q=e\left(\beta \right)+\frac{\partial e}{\partial \beta }\left(\beta \right)d\beta +\mathcal{O}\left(2\right)$$

Hence, up to $\mathcal{O}\left(2\right)$ order terms,

$$q=\frac{\partial e}{\partial \beta }\left(\beta \right)d\beta =-\frac{{\partial }^{2}lnZ\left(\beta \right)}{\partial {\beta }^2}d\beta =-k{T}^2{C}_{v}d\left(\frac{1}{kT}\right)=C_{v}dT$$

2.5. Entropy production of the $\mathcal{A}\cup \chi \cup \mathcal{B}$ system

Let us consider the insulated system $\mathcal{A}\cup \chi \cup \mathcal{B}$ and let us suppose that the two environments $\mathcal{A}$ and $\mathcal{B}$ are two thermostats respectively at temperatures $\tilde{T}=1/k\beta (e+q)$ and $T=1/k\beta \left(e\right)$. The net effect of putting the system χ alternatively in contact with $\mathcal{A}$ and with $\mathcal{B}$ is the flow in two time steps of the chain of an average energy amount $q>0$ from a reservoir at higher temperature $\tilde{T}$ to a reservoir a lower temperature T leaving the system χ unchanged, since the equilibrium distribution π is stationary for $AB$. By a standard non-equilibrium thermodynamics formula (see e.g. [13]) the entropy production in the $\pi \stackrel{A}{\to }\tilde{\pi }\stackrel{B}{\to }\pi$ cycle is

$$S_{pr}=d{S}_A+d{S}_B=\frac{-q}{\tilde{T}}+\frac{q}{T}=kq(\beta -\tilde{\beta })\ge 0$$

If we now compute the information divergence (also called relative entropy, see [6]) of the Gibbs distribution $\tilde{\pi }$ with respect to Gibbs distribution π we find

$$D(\tilde{\pi }\parallel \pi )=\sum _i{\tilde{\pi }}_{i}ln\frac{{\tilde{\pi }}_i}{{\pi }_i}=ln\frac{Z\left(\beta \right)}{Z\left(\tilde{\beta }\right)}+(\beta -\tilde{\beta }){\mathbb{E}}_{\tilde{\pi }}\left(E\right)\ge 0$$

The information divergence it is not a symmetric function of the two probability distributions $p,q$ while the symmetrized information divergence (see [14]) of p and q is a symmetric and non negative one

$$\Delta (p,q):=D(p\parallel q)+D(q\parallel p)=\sum _i(p_i-q_i)ln\frac{p_i}{q_i}$$

(2.13)

The standard interpretation (see again [14]) of the symmetrized information divergence is a measure of the difficulty of assessing which is the statistical description (p or q) of the system on the basis of the observations of the system state. Since

$$\Delta (\pi ,\tilde{\pi }):=D(\tilde{\pi }\parallel \pi )+D(\pi \parallel \tilde{\pi })=(\beta -\tilde{\beta })({\mathbb{E}}_{\tilde{\pi }}\left(E\right)-{\mathbb{E}}_{\pi }\left(E\right))=(\beta -\tilde{\beta })q\ge 0.$$

we have the following

Proposition 2 The entropy production of the closed system $\mathcal{A}\cup \chi \cup \mathcal{B}$ is equal to the symmetrized information divergence between the probability distributions π, $\tilde{\pi }$

$$S_{pr}=k\Delta (\pi ,\tilde{\pi })\ge 0$$

(2.14)

Remark. In the literature (see e.g the books [15],[16] or the papers [17], [18]) there is a well established notion of entropy production rate for a stationary Markov chain with countable state space. Let $(P,\pi )$ be the transition matrix and its unique stationary distribution respectively. Then the entropy production rate, also called the information gain of the stationary chain with respect to its time reversal, is defined as

$$e_P=\sum _{i,j}({\pi }_i{P}_{ij}-{\pi }_j{P}_{ji})ln\frac{{\pi }_i{P}_{ij}}{{\pi }_j{P}_{ji}}.$$

We immediately see that if the the chain satisfy the detailed balance condition ${\pi }_i{P}_{ij}-{\pi }_j{P}_{ji}=0$, then the entropy production rate is zero. Let us show that this is the case for our χ system described by the maximum entropy rate matrices A and B defined in (2.10). Indeed, the chain observed at even times is described by the transition matrix $AB$ with entries ${\left(AB\right)}_{ij}={\pi }_j$, whose stationary distribution is the same π in (2.9), and this stationary Markov chain satisfy trivially the detailed balance condition. Therefore, the entropy production of the overall system $\mathcal{A}\cup \chi \cup \mathcal{B}$ is exclusively due to the energy exchange between the two reservoirs $\mathcal{A}$ and $\mathcal{B}$ as computed above.

3. Generalizations

In this section, we generalize the previous results to the case of a open system with k macroscopic observables $E^{\alpha }$ with possibly different speed of relaxation to equilibrium and k stationary fluxes $q^{\alpha }$.

3.1. The case of k isochronous fluxes and Onsager’ reciprocity relations

The considerations in Section 2 and Section 2.1 developed for E apply without changes for every macroscopic observable $E^{\alpha }$. We consider therefore our M.E.R. problem with k constraints of the form (2.4). Since the constraints specifying the values $q^{\alpha }$ of the k inflows are linear and independent, the last term in the r.h.s. of the Lagrange function (2.6) becomes a sum over the index α

$$\sum _{ij\alpha }{\beta }_{\alpha }{\pi }_i{A}_{ij}{E}_{ij}^{\alpha }$$

By a simple inspection of the previous computations, one finds that the matrix $\tilde{Q}$ has the form

$${\tilde{Q}}_{ij}=\frac{e^{-{\sum }_{\alpha }{\beta }_{\alpha }{E}_j^{\alpha }}}{{\sum }_j{e}^{-{\sum }_{\alpha }{\beta }_{\alpha }{E}_j^{\alpha }}}=:{\tilde{\pi }}_j$$

where the k Lagrange multipliers ${\beta }_{\alpha }$ are uniquely determined by the k equations

$${\mathbb{E}}_{\tilde{\pi }}\left(E^{\alpha }\right)=-\frac{\partial lnZ\left(\beta \right)}{\partial {\beta }_{\alpha }}=\sum _r{E}_r^{\alpha }\frac{e^{-{\sum }_{\sigma }{\beta }_{\sigma }{E}_r^{\sigma }}}{Z\left(\beta \right)}=q^{\alpha }+{\mathbb{E}}_{\pi }\left(E^{\alpha }\right)=q^{\alpha }+e^{\alpha }$$

The considerations and the results of Section 2.4. can be reformulated without changes for the vector of fluxes $q=(q^1,\dots ,q^k)$ and equilibrium values $e=(e^1,\dots ,e^k)$. In particular, the linear approximation of small fluxes formula (2.12) becomes

$$\mathcal{H}\left(q\right)=S\left(e\right)+\frac{1}{2}\sum _{\alpha }{\beta }_{\alpha }\left(e\right)q^{\alpha }+\mathcal{O}\left(q^2\right)$$

(3.1)

and, by a computation like the one done before in Section 2.4., we get

$$\frac{\partial e_r}{\partial {\beta }_l}=\frac{{\partial }^{2}lnZ\left(\beta \right)}{\partial {\beta }_r\partial {\beta }_l}=\frac{\partial e_l}{\partial {\beta }_r}$$

which are Onsager reciprocity relations.

3.2. The case of different speed of relaxation to equilibrium

In this section we sketch how to model a system where there are two observables $X,Y$ with different speed of relaxation to equilibrium. As before, let the system χ be alternatively in contact with the two environments $\mathcal{A}$ and $\mathcal{B}$ but now we suppose that for the observable X its average value switches from e to $e+q$ and back to e upon the contact with $\mathcal{A}$ and $\mathcal{B}$, while for the observable Y its average value switches from u to $u+r$ and back to u upon the contact with $\mathcal{A}$, $\mathcal{B}$, $\mathcal{A}$, $\mathcal{B}$ (two cycles). Therefore, the system returns (relaxes) to equilibrium with respect to observable X in a two-step cycle $A\to B$ or $C\to D$ while, with respect to observable Y relaxes to equilibrium in a full (four step) cycle $A\to B\to C\to D$. Can we infer average values of slow quantity Y at intermediate states B, C from knowledge of average values of fast observable X ?

To this we introduce a periodic Markov chain with four stochastic matrices A, B, C, D generalizing (2.1) and the following set of constraints generalizing (2.4)

$$\left\{\begin{array}{c}A·\mathbf{1}=B·\mathbf{1}=C·\mathbf{1}=D·\mathbf{1}=\mathbf{1}\left(\text{normalization}\right)\hfill \\ \\ {\mathbb{E}}_{A^{*}\pi }\left(X\right)-{\mathbb{E}}_{\pi }\left(X\right)=q,(\text{specify} \text{inflow} \text{of} X \text{in} A)\hfill \\ \\ {\mathbb{E}}_{{\left(AB\right)}^{*}\pi }\left(X\right)-{\mathbb{E}}_{\pi }\left(X\right)=0,(\text{specify} \text{outflow} \text{of} X \text{in} AB)\hfill \\ \\ {\mathbb{E}}_{{\left(ABC\right)}^{*}\pi }\left(X\right)-{\mathbb{E}}_{\pi }\left(X\right)=q,(\text{specify} \text{inflow} \text{of} X \text{in} ABC)\hfill \\ \\ {\mathbb{E}}_{{\left(AB\right)}^{*}\pi }\left(Y\right)-{\mathbb{E}}_{\pi }\left(Y\right)=r,(\text{specify} \text{inflow} \text{of} Y \text{in} AB)\hfill \\ \\ {\left(ABCD\right)}^{*}\pi =\pi ,(\pi \text{is} \text{stationary} \text{for} ABCD)\hfill \end{array}\right.$$

(3.2)

By a computation entirely analogous to the previous one, we find the expression of the entropy rate

$$\mathcal{H}=\frac{1}{4}[H(\pi ,A)+H(A^{*}\pi ,B)+H({\left(AB\right)}^{*}\pi ,C)+H({\left(ABC\right)}^{*}\pi ,D)]$$

Moreover, if the equilibrium state is defined by the macroscopic values $e={\mathbb{E}}_{\pi }\left(X\right)$ and $u={\mathbb{E}}_{\pi }\left(Y\right)$, one can show that then system switches between three Gibbsian states

$$\pi (e,u)\stackrel{A}{\to }\pi (e+q)\stackrel{B}{\to }\pi (e+q,u+r)\stackrel{C}{\to }\pi (e+q)\stackrel{D}{\to }\pi (e,u)$$

Then, the inferred average value of Y at the intermediate states is

$${\left(Y\right)}_{inf}={\mathbb{E}}_{\pi (e+q)}\left[Y\right]$$

and also

$$\frac{d{\left(Y\right)}_{inf}}{dq}dq=-\frac{Co{v}_{\pi (e+q)}(X,Y)}{Co{v}_{\pi (e+q)}(X,X)}dq$$

4. Conclusions

In this paper we have dealt with a simple model describing a system in a stationary non-equilibrium state. The probabilistic description is provided by a non-homogeneous Markov chain, which is not assumed on the basis of a model of the microscopic interactions but rather derived from the knowledge of the macroscopic fluxes traversing the system through a maximum entropy rate principle. With respect to existing applications of the M.E.R. principle, here we have introduced a Markov chain defined by two or more stochastic matrices. In this way we are able to describe a system in a stationary non-equilibrium state that may exhibit different speed of relaxation to equilibrium. We made reference to physical quantities such as the energy for the ease of interpretation of the results, but this is not necessary and the same model can be applied in other domains, such as economy or biology, as it is sufficiently general to take into account macroscopic constraints of different nature.