Stochastic Entropy Production for Classical and Quantum Dynamical Systems with Restricted Diffusion

Abstract

Modeling the evolution of a system using stochastic dynamics typically implies increasing subjective uncertainty in the adopted state of the system and its environment as time progresses, and stochastic entropy production has been developed as a measure of this change. In some situations, the evolution of stochastic entropy production can be described using an Itô process, but mathematical difficulties can emerge if diffusion in the system phase space happens to be restricted to a subspace of a lower dimension. This situation can arise if there are constants of the motion, for example, or more generally when there are functions of the coordinates that evolve without noise. More simply, difficulties can emerge if there are more coordinates than there are independent noises. We show how the problem of computing the stochastic entropy production in such a situation can be overcome. We illustrate the approach using a simple case of diffusion on an ellipse. We go on to consider an open three-level quantum system modeled within a framework of Markovian quantum state diffusion. We show how a nonequilibrium stationary state of the system, with a constant mean rate of stochastic entropy production, can be established under suitable environmental couplings.

1. Introduction

Entropy quantifies subjective uncertainty in the configuration of a system when limited information is available. If the evolution of such a system is modeled using stochastic dynamics, for example, representing the effects of coupling to an underspecified environment, then uncertainty in the configuration of the world (the system plus its environment) increases with time, corresponding to growth in the total entropy. Entropy growth does not control the evolution of the world in a causal sense: how could a subjective uncertainty do such a thing? But, it does characterize its evolution. The second law of thermodynamics arises naturally when the world is perceived at a coarse-grained level but governed by underlying equations of motion with a certain degree of deterministic chaos [1].

For system coordinates that evolve continuously according to a set of Markovian stochastic differential equations (SDEs) or Itô processes, it may be shown that the associated stochastic entropy production [2,3,4] can also be described using an SDE, but only if the noise terms satisfy certain requirements [5]. Problems arise from constraints imposed on the dynamics, for example, the existence of constants of the motion. In such cases, the matrix describing the diffusion of system coordinates in their phase space becomes singular. There are directions in the space in which diffusion does not take place and mathematical difficulties in the evaluation of the stochastic entropy production arise as a consequence.

The central aim of this paper is to show how to take into account such constraints on diffusion when computing the stochastic entropy production. In Section 2, we briefly discuss how a Markovian SDE for stochastic entropy production may be derived from Markovian SDEs describing the evolution of a set of system coordinates. The mathematical treatment of cases where diffusion is restricted is discussed in Section 3. Application to a simple example case involving the stochastic evolution of two (classical) coordinates under the influence of just one noise is covered in Section 4. Situations where the number of stochastic variables exceeds the number of independent noises are commonplace in dynamical modeling and can lead to singular diffusion matrices.

In Section 5, we consider the stochastic dynamics of the reduced density matrix of a more complicated open three-level quantum system subjected to environmental disturbance that promotes transitions between the levels. The theoretical background to such situations has been developed recently and is discussed at greater length in [6,7]. The essential details are provided in Appendix B. The goal of such modeling is to identify the increase in uncertainty of the quantum state of a system and its environment arising from an uncontrolled and therefore indeterminate process of evolution, and stochastic entropy production provides such a characterization. The three-level system is one of the simplest that can sustain a nonequilibrium stationary state, a feature that we are interested in exploring. The diffusion matrix in the relevant parameter space is singular, but we demonstrate how the stochastic entropy production can still be evaluated. We go on to compute the environmental component of mean stochastic entropy production numerically in order to characterize the equilibrium and nonequilibrium stationary states of the system. Our conclusions are outlined in Section 6.

2. Stochastic Entropy Production for Itô Processes

We consider a set of N coordinates $\mathit{x}\equiv (x_1,x_2,\cdots ,x_N)$ that specify the configuration of a system and model their evolution using Markovian stochastic differential equations, or Itô processes, of the form

$$d{x}_i=A_i(\mathit{x})dt+\sum _{j=1}^M{B}_{ij}(\mathit{x})d{W}_j,$$

(1)

where $d{W}_j$ are independent Wiener increments. The coefficients $A_i$ and $B_{ij}$ are functions of the coordinates, but, in the interest of compactness, this notation is suppressed hereafter. For simplicity, we confine the discussion to coordinates with even parity under time reversal.

Defining an $N\times N$ diffusion matrix $\mathit{D}=\frac{1}{2}\mathit{B}{\mathit{B}}^{\mathsf{T}}$, the Fokker–Planck equation for the probability density function (PDF) $p(\mathit{x},t)$ is given by

$$\begin{array}{cc}\hfill {\displaystyle \frac{\partial p}{\partial t}}& =-\sum _i{\displaystyle \frac{\partial }{\partial x_i}}\left(A_{i}p\right)+\sum _{ij}{\displaystyle \frac{\partial }{\partial x_i\partial x_j}}\left(D_{ij}p\right),\hfill \\ \hfill & =-\sum _i{\displaystyle \frac{\partial }{\partial x_i}}\left(C_{i}p-D_{ij}{\displaystyle \frac{\partial p}{\partial x_j}}\right),\hfill \end{array}$$

(2)

with

$$C_i=A_i-\sum _j{\displaystyle \frac{\partial D_{ij}}{\partial x_j}}.$$

(3)

The stochastic entropy production of the system and its environment, associated with the stochastic motion described by Equation (1), is a measure of irreversibility. It is the difference in probability between pairs of time-reversed sequences of events and defined by [2]

$$\Delta s_{\mathrm{tot}}=ln\left({\displaystyle \frac{p(\mathit{x},t)T(\mathit{x}\to {\mathit{x}}^{\prime },\Delta t)}{p({\mathit{x}}^{\prime },t+\Delta t)T({\mathit{x}}^{\prime }\to \mathit{x},\Delta t)}}\right),$$

(4)

where $T(\mathit{x}\to {\mathit{x}}^{\prime },\Delta t)$ is a probability that the system should take a (forward) path from $\mathit{x}$ to ${\mathit{x}}^{\prime }$ in a period $\Delta t$. Note that the PDF in the denominator is specified at time $t+\Delta t$: the time-reversed counterpart of the forward path is considered to start after the forward path has been completed.

The total stochastic entropy production is usually separated into system and environmental contributions, with increments given by

$$d\Delta s_{\mathrm{tot}}=d\Delta s_{\mathrm{sys}}+d\Delta s_{\mathrm{env}},$$

(5)

with $d\Delta s_{\mathrm{sys}}=-dlnp(\mathit{x},t)$. The evolution of the environmental stochastic entropy production is governed by the SDE [8],

$$\begin{array}{cc}\hfill d\Delta s_{\mathrm{env}}& =\sum _{ij}{D}_{ij}^{-1}{C}_{j}d{x}_i+\sum _{ijk}{D}_{ij}{\displaystyle \frac{\partial \left(D_{ik}^{-1}{C}_k\right)}{\partial x_j}}dt,\hfill \\ \hfill & =\sum _{ij}{D}_{ij}^{-1}{C}_j\circ d{x}_i.\hfill \end{array}$$

(6)

The final form, employing Stratonovich rules of stochastic calculus, is compact, although it is more practical to employ the Itô form given in the first line. A more general expression suitable for systems with odd as well as even parity coordinates is given in Appendix A.

It may be shown that the average of $d\Delta s_{\mathrm{sys}}$ over all possible trajectories is related to the incremental change in Gibbs entropy of the system: $d\langle \Delta s_{\mathrm{sys}}\rangle =d{S}_G$, where $S_G(t)=-\int p(\mathit{x},t)lnp(\mathit{x},t)d\mathit{x}$, to which boundary terms should be added in certain circumstances [7]. Computing the environmental stochastic entropy production, on the other hand, presents particular difficulties if the diffusion matrix is singular since the inverse matrix ${\mathit{D}}^{-1}$ is required in the above expression. This is the problem we wish to address here.

3. Defining Dynamical and Spectator Variables

3.1. Preliminaries

A singular diffusion matrix may be regarded as a consequence of having fewer independent noise terms than the number of coupled Itô processes. For example, diffusion might occur on a two-dimensional surface within a three-dimensional phase space of system coordinates if the motion is described by three Itô processes with only two independent Wiener increments. There is a direction at each point in the phase space in which there is no diffusive current, which makes the $3\times 3$ diffusion matrix singular. These directions lie parallel to spatially dependent eigenvectors of the diffusion matrix with zero eigenvalues, to be referred to as null eigenvectors.

The obvious solution is to derive a reduced set of stochastic differential equations that describe the random evolution of, in this example, two coordinates on the surface with the third coordinate related deterministically to the other two. We shall denote the stochastically evolving coordinates as ‘dynamical variables’ and the remaining coordinates as ‘spectator variables’. It might be possible in simple cases to form such a reduced set of dynamical variables, perhaps by identifying a constant of the motion. However, as we increase the dimensionality of the phase space and hence the size of the diffusion matrix, the difficulties in doing so may become insurmountable. We therefore require a more general treatment of situations with a singular diffusion matrix.

So, let us consider a system described by N coordinates $x_i$, each of which evolves stochastically according to Equation (1). According to Itô’s lemma [9], the differential of a function f of these coordinates can be written

$$df=\sum _{i=1}^N{\displaystyle \frac{\partial f}{\partial x_i}}d{x}_i+\sum _{i,j=1}^N{\displaystyle \frac{{\partial }^{2}f}{\partial x_i\partial x_j}}{D}_{ij}dt,$$

(7)

where the elements of the (symmetric) $N\times N$ diffusion matrix are $D_{ij}=\frac{1}{2}{\sum }_{k=1}^M{B}_{ik}{B}_{jk}$. Let us now consider a special case of a function where the stochastic terms in Equation (7) vanish, i.e.,

$$\sum _{i=1}^N\sum _{j=1}^M{\displaystyle \frac{\partial f}{\partial x_i}}{B}_{ij}d{W}_j=0,$$

(8)

such that f evolves deterministically. Taking the square and using $d{W}_{i}d{W}_j={\delta }_{ij}dt$, we find that

$$\begin{array}{cc}\hfill \sum _{ijkl}{\displaystyle \frac{\partial f}{\partial x_i}}{\displaystyle \frac{\partial f}{\partial x_k}}{B}_{ij}{B}_{kl}{\delta }_{jl}dt& =\sum _{ijk}{\displaystyle \frac{\partial f}{\partial x_i}}{\displaystyle \frac{\partial f}{\partial x_k}}{B}_{ij}{B}_{kj}dt\hfill \\ \hfill & =2\sum _{ik}{\displaystyle \frac{\partial f}{\partial x_i}}{\displaystyle \frac{\partial f}{\partial x_k}}{D}_{ik}dt=0,\hfill \end{array}$$

(9)

showing that there is deterministic (noise-free) evolution of f if

$${\left(\nabla f\right)}^{\mathsf{T}}\mathit{D}\nabla f=0.$$

(10)

Such an outcome can arise if $\nabla f$ is an eigenvector of $\mathit{D}$ with an eigenvalue equal to zero. A matrix is singular if one or more of its eigenvalues are zero, so we have established that $\mathit{D}$ is singular if there exists a function f of the $x_i$ that evolves deterministically. For dynamics evolving in an N-dimensional phase space under the influence of M noises where $N>M$, we conjecture that there will be $L=N-M$ such functions, each associated with one of the null eigenvectors of $\mathit{D}$. We assume that these functions and their associated null eigenvectors are mutually independent. We shall see that this allows us to recast the calculation of the stochastic entropy production to overcome the problematic singularity of $\mathit{D}$.

3.2. Identifying Constants of Motion

If the deterministic as well as stochastic terms in Equation (7) vanish, then $df=0$, and the function f is a constant of the motion; the evolution of the coordinates is constrained to a contour of constant f. We can therefore write

$$\nabla f·d\mathit{x}=0,$$

(11)

meaning that the infinitesimal vector $d\mathit{x}$ specified by Equation (1) is tangential to a contour of f. Since $\nabla f$ is also a null eigenvector of the diffusion matrix, this constraint on $d\mathit{x}$ is conveniently identified by evaluating the eigenvectors of $\mathit{D}$.

If there are $L=N-M$ constant functions of the coordinates under the dynamics, we can remove L coordinates from the original N-dimensional phase space, leaving a reduced M-dimensional phase space. Without loss of generality, the first M coordinates in the set $\left\{x_m\right\}$ with $m=1,\cdots ,M$ will be taken to be dynamical variables, and the remaining L coordinates $\left\{x_l\right\}$ with $l=M+1,\cdots ,N$ are designated spectator variables. The L constants of the motion mean we should be able to write the spectator variables as functions of the dynamical variables, namely as $x_l\left(\left\{x_m\right\}\right)$. The division into dynamical and spectator variables is arbitrary, up to a point to be discussed shortly, but, as we shall see in Section 4, some choices are more convenient for computing stochastic entropy production than others.

In this labeling scheme, the top left $M\times M$ block of the diffusion matrix $\mathit{D}$ remains relevant to the stochastic entropy calculation and will be nonsingular, allowing us to use Equation (6) to compute $\Delta s_{\mathrm{env}}$ associated with the evolution, but with i and j ranging only between 1 and M rather than 1 and N. If the elements of this matrix block depend on spectator variables, we need to take this into account when computing derivatives with respect to the dynamical variables. To emphasize this point, we write the components of $\mathit{A},\mathit{D}$ and ${\mathit{D}}^{-1}$ that appear in Equation (6) to show their dependence on the dynamical and spectator variables explicitly as follows:

$$\begin{array}{c}\hfill A_i(\left\{x_m\right\},\left\{x_l(\left\{x_m\right\})\right\})\\ \hfill D_{ij}(\left\{x_m\right\},\left\{x_l(\left\{x_m\right\})\right\})\\ \hfill D_{ij}^{-1}(\left\{x_m\right\},\left\{x_l(\left\{x_m\right\})\right\}),\end{array}$$

(12)

and we note that Equation (6) requires us to evaluate derivatives of $A_i$, $D_{ij}$, and $D_{ij}^{-1}{A}_j$ with respect to the dynamical variables $\left\{x_m\right\}$.

We consider derivatives of $D_{ij}$ in the following, but the argument is easily extended to other expressions. We begin by noting that Equation (11) can be separated according to dynamical and spectator variables such that

$$\sum _{m=1}^M{\alpha }_{km}d{x}_m+\sum _{l=M+1}^N{\alpha }_{kl}d{x}_l=0,$$

(13)

where ${\alpha }_{km}$ and ${\alpha }_{kl}$ are the dynamical and spectator components, respectively, of the kth null eigenvector of $\mathit{D}$, with $k=1,\cdots ,L$. Therefore,

$$\sum _{l=M+1}^N{\alpha }_{kl}d{x}_l=-\sum _{m=1}^M{\alpha }_{km}d{x}_m,$$

(14)

and, arranging the ${\alpha }_{km}$ as elements of a rectangular $L\times M$ matrix $Q_{km}$ and the ${\alpha }_{kl}$ as elements of a square $L\times L$ matrix $P_{kl}$, we have

$$d{x}_l=-\sum _{k,m}{P}_{lk}^{-1}{Q}_{km}d{x}_m=\sum _{m=1}^M{R}_{lm}d{x}_m,$$

(15)

where $R_{lm}$ is an element of the $L\times M$ matrix $\mathit{R}=-{\mathit{P}}^{-1}\mathit{Q}$. It should be noted that the invertibility of the $\mathit{P}$ matrix would be threatened if the spectator components of the L null eigenvectors happen not to be independent, but this is not a situation we have encountered in example cases studied so far.

Next, we write

$$d{D}_{ij}=\sum _{m=1}^M{\displaystyle \frac{\partial D_{ij}}{\partial x_m}}d{x}_m+\sum _{l=M+1}^N{\displaystyle \frac{\partial D_{ij}}{\partial x_l}}d{x}_l,$$

(16)

and, by substituting $d{x}_l$ from Equation (15) into Equation (16), we arrive at the following expression for the required derivative of $D_{ij}$ with respect to the dynamical coordinate $x_m$:

$${\displaystyle \frac{d{D}_{ij}}{d{x}_m}}={\displaystyle \frac{\partial D_{ij}}{\partial x_m}}+\sum _{l=M+1}^N{\displaystyle \frac{\partial D_{ij}}{\partial x_l}}{R}_{lm}.$$

(17)

The second term on the right-hand side provides an additional contribution to the dependence of $D_{ij}$ on $x_m$.

To summarize, the framework allows the computation of stochastic entropy production in cases where the diffusion matrix is singular due to constraints on the dynamics from the L constants of motion. The method employs these constants of motion to reduce the dimensionality of the phase space through which the system evolves such that the appropriately reduced diffusion matrix is nonsingular. Having carried out this transformation, Equation (6) can be used to compute the environmental component of stochastic entropy production with derivatives determined according to Equation (17).

3.3. Identifying Deterministically Evolving Functions

We have considered a function f of the stochastic variables $\left\{x_i\right\}$ that evolves according to

$$df=\sum _i{\displaystyle \frac{\partial f}{\partial x_i}}d{x}_i+\sum _{i,j}{\displaystyle \frac{{\partial }^{2}f}{\partial x_i\partial x_j}}{D}_{ij}dt,$$

(18)

and looked at situations where both the deterministic and stochastic terms in Equation (18) vanish, making the function f a constant of motion of the dynamics. However, it is only necessary for the stochastic terms to vanish for there to be a restriction on the diffusive motion. We now consider the more general case where f evolves deterministically according to

$$df=\left(\sum _i{\displaystyle \frac{\partial f}{\partial x_i}}{A}_i+\sum _{i,j}{\displaystyle \frac{{\partial }^{2}f}{\partial x_i\partial x_j}}{D}_{ij}\right)dt.$$

(19)

Following earlier arguments, as a consequence of Equation (8), $\nabla f$ is still a null eigenvector of $\mathit{D}$, and we now take each null eigenvector to correspond to a deterministically evolving function. To illustrate this, consider a system evolving stochastically in two dimensions ($x_1$,$x_2$) with a diffusion matrix that possesses a single null eigenvector. A function $f(x_1,x_2)$ evolving deterministically according to Equation (19) allows us to reduce the number of variables needed to describe the motion from two to one dynamical variable. The other becomes a spectator variable.

To understand this better, imagine that, at time $t_0$, the function f is given by

$$f(x_1(t_0),x_2(t_0))=c_0,$$

(20)

where $c_0$ is a constant defining a contour of f in the space. Equation (20) allows us to express spectator variable $x_2$ as a function of dynamical variable $x_1$ at time $t_0$.

Between times $t_0$ and $t_1=t_0+dt$, f changes deterministically by an amount $df$ to define a new contour

$$f(x_1(t_1),x_2(t_1))=c_1=c_0+df,$$

(21)

according to which we can again express $x_2$ as a function of $x_1$ at the later time. The evolution of the system is confined to a sequence of contours of the deterministically evolving function f such that we can always express $x_2$ in terms of $x_1.$ We can parameterize this evolution with a single coordinate and thereby employ a reduced diffusion matrix to compute a stochastic entropy production. The argument easily generalizes to an arbitrary number of dimensions.

As in the previous section, we need to consider how derivatives are modified when we reduce the dimensionality of the phase space. We write $\nabla f=g\mathit{\alpha }$, where g is equal to $|\nabla f|$ and $\mathit{\alpha }$ is a normalized null eigenvector of $\mathit{D}$, with $\left|\mathit{\alpha }\right|=1$, that lies perpendicular to the contour of f. Infinitesimal changes in coordinates $\mathit{x}$ over a timestep $dt$ are given by $d\mathit{x}$, and $\mathit{x}$ is constrained to move between points on specified contours of f at specified times, hence making only certain $d\mathit{x}$ allowed. The situation is illustrated in Figure 1.

The component of $d\mathit{x}$ in the direction normal to f is given by

$$d{\mathit{x}}_{\perp }=\left|d\mathit{x}\right|cos\theta =\left|\mathit{\alpha }\right|\left|d\mathit{x}\right|cos\theta =\mathit{\alpha }·d\mathit{x},$$

(22)

where the angle $\theta$ is shown in Figure 1. We can also write

$$df=|\nabla f|d{\mathit{x}}_{\perp }=|\nabla f|\mathit{\alpha }·d\mathit{x}=g\mathit{\alpha }·d\mathit{x},$$

(23)

where $g=|\nabla f|$, such that

$$g\mathit{\alpha }·d\mathit{x}=\left(\sum _i{\displaystyle \frac{\partial f}{\partial x_i}}{A}_i+\sum _{i,j}{\displaystyle \frac{{\partial }^{2}f}{\partial x_i\partial x_j}}{D}_{ij}\right)dt.$$

(24)

For notational convenience, we rewrite the term in brackets on the right-hand side as $gG$ so that

$$\mathit{\alpha }·d\mathit{x}=Gdt.$$

(25)

For a set of null eigenvectors ${\mathit{\alpha }}_k$ of $\mathit{D}$ labeled by $k=1,\dots ,L$, there would be L deterministically evolving functions of the coordinates, and we can specify relationships between increments in dynamical and spectator variables such that

$$\sum _{m=1}^M{\alpha }_{km}d{x}_m+\sum _{l=M+1}^N{\alpha }_{kl}d{x}_l=G_{k}dt.$$

(26)

Following the reasoning in Equation (15), we write

$$\begin{array}{cc}\hfill d{x}_l& =-P_{lk}^{-1}{Q}_{km}d{x}_m+P_{lk}^{-1}{G}_{k}dt=R_{lm}d{x}_m+S_{l}dt,\hfill \end{array}$$

(27)

with implied summation over repeated indices, and, substituting Equation (27) into Equation (16), we obtain

$$d{D}_{ij}=\sum _{m=1}^M{\displaystyle \frac{\partial D_{ij}}{\partial x_m}}d{x}_m+\sum _{l=M+1}^N{\displaystyle \frac{\partial D_{ij}}{\partial x_l}}\left(R_{lm}d{x}_m+S_{l}dt\right).$$

(28)

As before, the required derivatives of the relevant elements of the diffusion matrix take the form

$${\displaystyle \frac{d{D}_{ij}}{d{x}_m}}={\displaystyle \frac{\partial D_{ij}}{\partial x_m}}+\sum _l{\displaystyle \frac{\partial D_{ij}}{\partial x_l}}{R}_{lm}.$$

(29)

We therefore find that contributions to derivatives with respect to dynamical variables, where the expression in question also depends on spectator variables, are the same whether f is a constant of the motion or a function evolving without noise. This is useful since, in general, we are unlikely to be able to determine if the singularity of a diffusion matrix is due to the existence of a constant function or one that evolves deterministically.

4. Example of Restricted Diffusive Evolution

To illustrate the above reasoning, consider the two SDEs

$$\begin{array}{c}d{x}_1=(1-x_1^2)dW\hfill \\ d{x}_2=-{\displaystyle \frac{1}{2}}{x}_{2}dt-x_1{x}_{2}dW,\hfill \end{array}$$

(30)

which represent a case of restricted diffusive evolution since there are two Itô processes but only one noise. The $2\times 2$ diffusion matrix for the $(x_1,x_2)$ phase space is

$$\begin{array}{cc}\hfill & {\displaystyle \frac{1}{2}}\left(\begin{array}{cc}1-x_1^2& 0\\ -x_1{x}_2& 0\end{array}\right)\left(\begin{array}{cc}1-x_1^2& -x_1{x}_2\\ 0& 0\end{array}\right)\hfill \\ \hfill & ={\displaystyle \frac{1}{2}}\left(\begin{array}{cc}{\left(1-x_1^2\right)}^2& -x_1{x}_2(1-x_1^2)\\ -x_1{x}_2(1-x_1^2)& x_1^2{x}_2^2\end{array}\right),\hfill \end{array}$$

(31)

which is singular since the determinant is zero. We now compute the stochastic entropy production using Equations (4) and (5). The deterministic terms are $A_1=0$ and $A_2=-\frac{1}{2}{x}_2$. We proceed first by regarding $x_1$ as the dynamical variable and $x_2$ as a spectator and write

$$\begin{array}{cc}& d\Delta s_{\mathrm{tot}}=-dln{p}_1(x_1,t)+{\displaystyle \frac{1}{D_{11}}}{A}_{1}d{x}_1-{\displaystyle \frac{1}{D_{11}}}{\displaystyle \frac{d{D}_{11}}{d{x}_1}}d{x}_1\hfill \\ \hfill & +\left[D_{11}{\displaystyle \frac{d}{d{x}_1}}\left({\displaystyle \frac{1}{D_{11}}}{A}_1\right)-D_{11}{\displaystyle \frac{d}{d{x}_1}}\left({\displaystyle \frac{1}{D_{11}}}{\displaystyle \frac{d{D}_{11}}{d{x}_1}}\right)\right]dt,\hfill \end{array}$$

(32)

where $p_1(x_1,t)=\int p(x_1,x_2,t)d{x}_2$ and $p(x_1,x_2,t)$ satisfies the Fokker–Planck equation associated with Equation (30). Since $A_1=0$ and $D_{11}=\frac{1}{2}{(1-x_1^2)}^2$, we can take derivatives unencumbered by implicit dependence on $x_1$ arising from dependence on $x_2$. The result is

$$\begin{array}{cc}\hfill d\Delta s_{\mathrm{tot}}& =-dln{p}_1(x_1,t)-{\displaystyle \frac{1}{D_{11}}}{\displaystyle \frac{d{D}_{11}}{d{x}_1}}d{x}_1\hfill \\ \hfill & -D_{11}{\displaystyle \frac{d}{d{x}_1}}\left({\displaystyle \frac{1}{D_{11}}}{\displaystyle \frac{d{D}_{11}}{d{x}_1}}\right)dt\hfill \\ \hfill & =-dln{p}_1(x_1,t)+2(1+x_1^2)dt+4x_{1}dW.\hfill \end{array}$$

(33)

However, we could just as well decide to compute the stochastic entropy production by regarding $x_2$ as the dynamical variable and $x_1$ as the spectator and write

$$\begin{array}{cc}& d\Delta s_{\mathrm{tot}}=-dln{p}_2(x_2,t)+{\displaystyle \frac{1}{D_{22}}}{A}_{2}d{x}_2-{\displaystyle \frac{1}{D_{22}}}{\displaystyle \frac{d{D}_{22}}{d{x}_2}}d{x}_2\hfill \\ \hfill & +\left[D_{22}{\displaystyle \frac{d}{d{x}_2}}\left({\displaystyle \frac{1}{D_{22}}}{A}_2\right)-D_{22}{\displaystyle \frac{d}{d{x}_2}}\left({\displaystyle \frac{1}{D_{22}}}{\displaystyle \frac{d{D}_{22}}{d{x}_2}}\right)\right]dt,\hfill \end{array}$$

(34)

with $p_2(x_2,t)=\int p(x_1,x_2,t)d{x}_1$, $A_2=-\frac{1}{2}{x}_2$ and $D_{22}=\frac{1}{2}{x}_1^2{x}_2^2$. Since $D_{22}$ depends on the (currently chosen) spectator variable $x_1$, we have to employ the derivative

$${\displaystyle \frac{d{D}_{22}}{d{x}_2}}={\displaystyle \frac{\partial D_{22}}{\partial x_2}}+R{\displaystyle \frac{\partial D_{22}}{\partial x_1}},$$

(35)

and identify the coefficient R using the (single) null eigenvector of $\mathit{D}$, which may be shown to be given by ${(x_1/x_2^2,(1-x_1^2)/x_2^3)}^{\mathsf{T}}$.

In this example, the dynamics preserve the value of the function $f(x_1,x_2)=\left(x_1^2+K{x}_2^2-1\right)/\left(2x_2^3\right)=0$ with arbitrary constant K (i.e., the motion is confined to an elliptical contour centered on the origin). The phase space for the restricted diffusion is illustrated in Figure 2 for $K=2$, together with a set of local null eigenvectors to show that they lie perpendicular to the contours. The stochastic dynamics take the system towards fixed point attractors at $x_1=\pm 1$ and $x_2=0$.

We can use Equation (14) in the form ${\alpha }_{1}d{x}_1+{\alpha }_{2}d{x}_2=0$, where ${\alpha }_i$ is the ith component of the null eigenvector. We then obtain

$$x_1{x}_{2}d{x}_1=-\left(1-x_1^2\right)d{x}_2,$$

(36)

such that $R=d{x}_1/d{x}_2=-(1-x_1^2)/\left(x_1{x}_2\right)$ and

$${\displaystyle \frac{d{D}_{22}}{d{x}_2}}=x_1^2{x}_2-{\displaystyle \frac{(1-x_1^2)}{x_1{x}_2}}{x}_1{x}_2^2=2x_1^2{x}_2-x_2.$$

(37)

The computation of $\Delta s_{\mathrm{tot}}$ using Equations (34) and (35) can then proceed, yielding

$$\begin{array}{cc}\hfill d\Delta s_{\mathrm{tot}}=& -dln{p}_2(x_2,t)+{\displaystyle \frac{1}{2x_1^2}}\left(1+x_1^2+4x_1^4\right)dt\hfill \\ \hfill & -{\displaystyle \frac{1}{x_1}}\left(1-4x_1^2\right)dW.\hfill \end{array}$$

(38)

The two SDEs, (33) and (38), describing the evolution of the total stochastic entropy production look different but will yield the same statistics. This can be demonstrated by writing $p_1(x_1,t)d{x}_1=p_2(x_2,t)d{x}_2$; hence,

$$-dln{p}_1(x_1,t)=-dln{p}_2(x_2,t)+dln\left|R\right|,$$

(39)

and then we use the rules of stochastic calculus to show that $dln\left|R\right|=\frac{1}{2}\left(x_1^{-2}-3\right)dt-x_1^{-1}dW$.

The point we are making is that, in cases of restricted diffusive evolution, with more coordinates than independent noises, we can divide the variables arbitrarily into dynamical and spectator sets and use the former to calculate the stochastic entropy production. The implication is that some choices regarding the division might be more convenient than others. In the case just considered, it is more sensible to regard $x_1$ as the dynamical variable and $x_2$ as the spectator in order to avoid mathematical difficulties such as the divergence in the right-hand side of Equation (38) at $x_1=0$ or equivalently at $x_2=\pm \sqrt{K}$. Furthermore, the Fokker–Planck equation for the PDF $p_1(x_1,t)$ is easy to construct,

$${\displaystyle \frac{\partial p_1}{\partial t}}={\displaystyle \frac{1}{2}}{\displaystyle \frac{{\partial }^2(1-x_1^2)p_1}{\partial x_1^2}},$$

(40)

while that for $p_2(x_2,t)$ is much more difficult to analyze.

Having said this, it would not be sensible to choose as dynamical variables a set that evolves deterministically, without noise. The reduced diffusion matrix for such a set would vanish, and the calculation of the stochastic entropy production would not then be possible.

5. An Open Three-Level Quantum System with Restricted Diffusion

5.1. SDEs and Selection of Spectator Variables

We have developed the present framework for computing stochastic entropy production because there are physical systems of interest where some of the stochastically evolving variables are spectators. In this section, we consider the dynamics of an open quantum system characterized by the stochastic evolution of its reduced density matrix $\rho$, which is the appropriate representation of the state [10,11,12,13]. The stochasticity of the evolution can be interpreted as a consequence of the coupling to the environment. The stochastic entropy production associated with such evolution of the reduced density matrix has recently received attention [6,7,14,15]. It provides a measure of the irreversibility of open quantum system evolution and also characterizes the evolving uncertainty of the quantum state of a system and its environment under incompletely specified dynamics.

In Appendix B, it is shown how a Markovian stochastic Lindblad equation for the evolution of the reduced density matrix of an open system can be derived, starting from the Lindblad operators that specify the dynamical effects of the environment on the system. Using this formalism, we consider a three-level bosonic system with environmental coupling characterized by the three raising ($c_{1-3}$) and three lowering ($c_{4-6}$) Lindblad operators given by

$$\begin{array}{cc}& c_1=\omega \left(\begin{array}{ccc}0& 0& 0\\ 1& 0& 0\\ 0& 0& 0\end{array}\right)c_2=\left(\begin{array}{ccc}0& 0& 0\\ 0& 0& 0\\ 1& 0& 0\end{array}\right)c_3=\omega \left(\begin{array}{ccc}0& 0& 0\\ 0& 0& 0\\ 0& 1& 0\end{array}\right)\hfill \\ \\ \hfill & c_4=\left(\begin{array}{ccc}0& 1& 0\\ 0& 0& 0\\ 0& 0& 0\end{array}\right)c_5=\omega \left(\begin{array}{ccc}0& 0& 1\\ 0& 0& 0\\ 0& 0& 0\end{array}\right)c_6=\left(\begin{array}{ccc}0& 0& 0\\ 0& 0& 1\\ 0& 0& 0\end{array}\right)\hfill \end{array}$$

(41)

in a basis of kets $|1\rangle ,|2\rangle$, and $|3\rangle$ corresponding to the three levels. The role of the constant $\omega$ is to induce and control a nonequilibrium stationary state in the system by the breakage of detailed balance, as will be described in Section 5.2. The SDE describing the dynamics of the system is given by the stochastic Lindblad equation

$$\begin{gathered} \begin{array}{c}d\rho =\sum _{i=1}^6\left[\left(c_i\rho c_i^{†}-{\displaystyle \frac{1}{2}}\rho c_i^{†}{c}_i-{\displaystyle \frac{1}{2}}{c}_i^{†}{c}_i\rho \right)dt \\ +(\rho c_i^{†}+c_i\rho -\mathrm{Tr}[\rho (c_i+c_i^{†})\left]\rho \right)d{W}_i\right],\end{array} \end{gathered}$$

(42)

and a sketch of the inter-level transitions brought about by the $c_i$ is given in Figure 3.

The reduced density matrix $\rho$ for this system is a complex $3\times 3$ Hermitian matrix with a unit trace, representing eight degrees of freedom. We therefore parameterize $\rho$ in terms of an eight-dimensional vector $\mathit{x}$ with components $x_i$, evolving as

$$d{x}_i=A_{i}dt+\sum _j{B}_{ij}d{W}_j,$$

(43)

in terms of an eight-dimensional vector $\mathit{A}$, an $8\times 6$ matrix $\mathit{B}$, and a six-dimensional vector $d\mathit{W}$ of independent Wiener increments. Since there are fewer noise terms than SDEs, the diffusive motion will be restricted in some way.

In order to proceed, we represent the density matrix using the eight Gell–Mann matrices given by

$$\begin{array}{cc}& {\lambda }_1=\left(\begin{array}{ccc}0& 1& 0\\ 1& 0& 0\\ 0& 0& 0\end{array}\right){\lambda }_2=\left(\begin{array}{ccc}0& -i& 0\\ i& 0& 0\\ 0& 0& 0\end{array}\right){\lambda }_3=\left(\begin{array}{ccc}1& 0& 0\\ 0& -1& 0\\ 0& 0& 0\end{array}\right)\hfill \\ \\ \hfill & {\lambda }_4=\left(\begin{array}{ccc}0& 0& 1\\ 0& 0& 0\\ 1& 0& 0\end{array}\right){\lambda }_5=\left(\begin{array}{ccc}0& 0& -i\\ 0& 0& 0\\ i& 0& 0\end{array}\right){\lambda }_6=\left(\begin{array}{ccc}0& 0& 0\\ 0& 0& 1\\ 0& 1& 0\end{array}\right)\hfill \\ \\ \hfill & {\lambda }_7=\left(\begin{array}{ccc}0& 0& 0\\ 0& 0& -i\\ 0& i& 0\end{array}\right){\lambda }_8={\displaystyle \frac{1}{\sqrt{3}}}\left(\begin{array}{ccc}1& 0& 0\\ 0& 1& 0\\ 0& 0& -2\end{array}\right).\hfill \end{array}$$

(44)

It is convenient to work with eight real variables $s,r,u,v,w,x,y$, and z, defined by $\mathrm{Tr}\left({\lambda }_1\rho \right)=s$, $\mathrm{Tr}\left({\lambda }_2\rho \right)=r$, $\mathrm{Tr}\left({\lambda }_3\rho \right)=u$, etc. We can then write $\rho$ as

$$\rho =\left(\begin{array}{ccc}\frac{u}{2}+\frac{\sqrt{3}z}{6}+\frac{1}{3}& \frac{s}{2}-\frac{ir}{2}& \frac{v}{2}-\frac{iw}{2}\\ \frac{s}{2}+\frac{ir}{2}& -\frac{u}{2}+\frac{\sqrt{3}z}{6}+\frac{1}{3}& \frac{x}{2}-\frac{iy}{2}\\ \frac{v}{2}+\frac{iw}{2}& \frac{x}{2}+\frac{iy}{2}& -\frac{\sqrt{3}z}{3}+\frac{1}{3}\end{array}\right).$$

(45)

The SDEs describing the evolution of the eight stochastic variables with, for simplicity, weighting factor $\omega =1$, are given in Appendix C.

We choose y and z as spectator variables, allowing us to focus instead on the six SDEs given by

$$\left(\begin{array}{c}ds\\ dr\\ du\\ dv\\ dw\\ dx\end{array}\right)=\left(\begin{array}{c}-2s\\ -2r\\ -3u\\ -2v\\ -2w\\ -2x\end{array}\right)dt+\left(\begin{array}{cccccc}-sv+x& -s^2-u+\frac{\sqrt{3}z}{3}+\frac{2}{3}& -sx+v& -sv& -s^2+u+\frac{\sqrt{3}z}{3}+\frac{2}{3}& -sx\\ -rv-y& -sr& -rx+w& -rv& -sr& -rx\\ v(1-u)& s(1-u)& -x(u+1)& -uv& -s(u+1)& -ux\\ -v^2-\frac{2\sqrt{3}z}{3}+\frac{2}{3}& -sv+x& -vx& u-v^2+\frac{\sqrt{3}z}{3}+\frac{2}{3}& -sv& s-vx\\ -vw& -sw+y& -wx& -vw& -sw& r-wx\\ -vx& -sx& -x^2-\frac{2\sqrt{3}z}{3}+\frac{2}{3}& s-vx& -sx+v& -u-x^2+\frac{\sqrt{3}z}{3}+\frac{2}{3}\end{array}\right)\left(\begin{array}{c}d{W}_1\\ d{W}_2\\ d{W}_3\\ d{W}_4\\ d{W}_5\\ d{W}_6\end{array}\right).$$

(46)

The reduced diffusion matrix that arises from Equation (46) is too elaborate to obtain an analytical expression for its inverse. It is possible, however, to compute an inverse numerically and then to append terms to any derivatives of the matrix elements according to the procedure described in Section 3. In actual fact, with this choice in spectator variables, no such additional terms are required, making this route very convenient.

5.2. Equilibrium and Nonequilibrium Stationary States

We have solved the SDEs for the dynamics of the reduced density matrix numerically (using a simple Euler–Maruyama scheme) and computed the evolution of the environmental component $\Delta s_{\mathrm{env}}$ of the stochastic entropy production, introduced in Equation (5). Calculating the system component $\Delta s_{\mathrm{sys}}$ requires a solution of a Fokker–Planck equation, which is computationally demanding, and doing so does not add to the understanding we aim to develop regarding the stationary states of the system, so we concentrate our attention instead on the environmental component. We comment further on this point later.

The computational demands of solving Equation (6) are considerable for the system under investigation, so only limited ensembles of trajectories were generated. Runs for environmental stochastic entropy production were typically executed with a timestep of $dt={10}^{-6}$, which proved to be small enough not to have a noticeable effect on the statistics of the resulting ensemble of trajectories. The reduced density matrix was initiated in the condition where all variables s, etc., are set to $0.1$.

Figure 4 illustrates the range of environmental stochastic entropy production for 25 runs with $\omega =1$. The average environmental stochastic entropy production is nearly always within one standard deviation of zero, which, given the limited statistics, provides a strong indication that a zero mean rate of environmental stochastic entropy production has been established in the stationary state. This is precisely what is to be expected of an equilibrium state. The mean rate of system stochastic entropy production is expected to be zero as well since it corresponds in typical situations to the rate of change of the Gibbs entropy of the system [7], which characterizes the uncertainty in its adopted state, and which does not change in a stationary state.

It is worth mentioning here that we take the view that the system (either on its own or in association with the environment) does indeed adopt a particular quantum state at all times. We further regard the measurement of a system observable as a dynamical process whereby the state is driven towards specific stationary outcomes, the eigenstates of the observable [6,7,10,14,15,16]. The uncertainty in the outcome under ‘projective’ measurement for a given density matrix is the von Neumann entropy, which we discuss in Section 5.3. In contrast, the Gibbs entropy arising from our application of stochastic thermodynamics to the quantum system characterizes instead the uncertainty in the pre-measurement state. It therefore has quite a different scale since the range of possibilities is greater.

It is also worth mentioning that the treatment of stochastic entropy production in open quantum system dynamics described here differs from other approaches where instantaneous projective measurement is explicitly included in the dynamical scheme and where initial and final states for trajectories are restricted to measured eigenstates [17,18,19,20,21]. Our approach considers more general evolutions and a more general entropy, as already mentioned.

We now investigate a nonequilibrium stationary state with a probability current passing through the system phase space. Our interest in the three-level system arises precisely because it is the simplest quantum system in which such a situation might be possible. We create a nonequilibrium stationary state by breaking detailed balance and favoring a pattern of transitions $|3\rangle \to |2\rangle \to |1\rangle \to |3\rangle$ around the system. We achieve this by reducing the coupling strength associated with the Lindblad operators linked to the opposite pattern. Specifically, we set the weighting factor $\omega <1$ in Lindblad operators $c_1$, $c_2$, and $c_5$ in (41) and derive a modified set of dynamical equations. Such a nonequilibrium stationary state is expected to be associated with a positive mean rate of stochastic entropy production.

Figure 5 illustrates the environmental stochastic entropy production for an ensemble of 10 runs with $\omega =0.1$. In line with expectations, we see a constant mean rate of environmental stochastic entropy production associated with the system being in a nonequilibrium stationary state. The mean system component of stochastic entropy production is not expected to make a contribution in a stationary state since the Gibbs entropy of the system is constant in time for such a situation. There is a clear relationship between the breakage of detailed balance and the mean rate of environmental stochastic entropy production. Figure 5 provides strong support that our approach can quantify the irreversibility of open quantum systems with spectator variables.

5.3. Stochastic Entropy Production in Quantum Mechanics

The mean stochastic entropy production in the example just considered is the change in subjective uncertainty with regard to the quantum state adopted by the system and its environment (the world). We are unable to make exact predictions when the dynamics are not specified in detail: under an evolution such as Equation (42), our knowledge of the adopted state is therefore reduced with time. We should contrast this with the von Neumann entropy $S_{\mathrm{vN}}=-\mathrm{Tr}\rho ln\rho$, a commonly employed expression for entropy in quantum mechanics.

Von Neumann entropy is the uncertainty with regard to the outcomes of projective measurement of a system. It is a Shannon entropy $-{\sum }_i{P}_{i}ln{P}_i$, where $P_i$ is the probability of projection into eigenstate i of the observable. For an N-level system, the number of such outcomes is N, so the von Neumann entropy has an upper limit of $lnN$.

In contrast, there is no upper limit of the mean stochastic entropy production under general dynamics since there is a continuum of potentially adoptable quantum states of the world. The nonzero mean rate of production of stochastic entropy associated with the nonequilibrium stationary state considered in the last section represents this progressively greater uncertainty.

It is worth drawing this distinction since different probability density functions for the reduced density matrix of a system could have the same probabilities of projective measurement and hence the same von Neumann entropy and yet would express different uncertainties with regard to the adopted state prior to measurement, and hence different Gibbs entropies over the phase space of reduced density matrix elements.

Note also that the stochastic entropy production we have been considering has no connection with heat transfer or work. The three-level system under consideration does not possess an internal Hamiltonian H, and there is no change in system internal energy. Stochastic entropy production is not necessarily associated with the dissipation of potential energy into heat. In both classical and quantum settings, the purpose of entropy is to specify the degree of configurational uncertainty of a system. In classical mechanics, the configurations are described by sets of classical coordinates. In quantum mechanics, they are specified by collections of (reduced) density matrix elements.

6. Conclusions

We have employed Itô processes to model the dynamics and thermodynamics of a system interacting with an environment when detailed information about the exact configuration of either is lacking. Such an approach has frequently been used in situations described by classical dynamics [3], and also for evolving quantum systems [6,7,17,19]. In both cases, difficulties can arise when there are fewer independent sources of noise than dimensions of the system phase space. Diffusion is restricted and the diffusion matrix becomes singular, which complicates the calculation of stochastic entropy production.

The solution to the problem is simply to eliminate degrees of freedom from the entropy calculation to account for the existence of functions of the coordinates that evolve without noise. We have described a general method for doing so and illustrated it for a simple system of particle diffusion on an ellipse with two fixed points. We divide the stochastic variables into dynamical and spectator sets and construct a reduced nonsingular diffusion matrix and the associated dynamics of stochastic entropy production.

We have gone on to consider a three-level quantum system, thermalized by an environment that brings about transitions between the levels. The dynamics take place in an eight-dimensional phase space with only six noise terms, and the $8\times 8$ diffusion matrix is singular.

We have shown that an equilibrium state of the system may be established, characterized by a zero mean rate of environmental stochastic entropy production (and implicitly by a zero mean rate of system stochastic entropy production as well). More interestingly, we have adjusted the probabilities of the transitions induced by the environment to create a nonequilibrium stationary state, where the particle is made to cycle, on average, through the levels in a particular order. This state is characterized by a nonzero rate of mean environmental stochastic entropy production.

Stochastic entropy production realizes Boltzmann’s aim of linking thermodynamics, and specifically entropy production, to the dynamical evolution of system coordinates [22]. Entropy production quantifies the irreversibility of open system behavior, assessing the likelihood that reversals of sequences of events might be observed. Refining this framework to accommodate special cases such as restricted diffusion adds further scope for quantifying irreversibility in open classical and quantum systems.