Minimum Information Variability in Linear Langevin Systems via Model Predictive Control

Abstract

Controlling the time evolution of a probability distribution that describes the dynamics of a given complex system is a challenging problem. Achieving success in this endeavour will benefit multiple practical scenarios, e.g., controlling mesoscopic systems. Here, we propose a control approach blending the model predictive control technique with insights from information geometry theory. Focusing on linear Langevin systems, we use model predictive control online optimisation capabilities to determine the system inputs that minimise deviations from the geodesic of the information length over time, ensuring dynamics with minimum “geometric information variability”. We validate our methodology through numerical experimentation on the Ornstein–Uhlenbeck process and Kramers equation, demonstrating its feasibility. Furthermore, in the context of the Ornstein–Uhlenbeck process, we analyse the impact on the entropy production and entropy rate, providing a physical understanding of the effects of minimum information variability control.

1. Introduction

Time-varying probability density functions (PDFs) are a preferred approach for characterising the dynamics of various complex systems. PDFs commonly feature in emergent fields, such as active inference [1] or stochastic thermodynamics [2], where their value is derived either through data analysis or by solving the Fokker–Planck (FP) equation associated with an Itô/Stratonovich stochastic differential equation.

Drawing upon control theory [3], when the dynamics of the system under consideration are governed by an FP equation, we can explore control objectives such as the regulation (setting to a constant value) or tracking (following a time-varying reference) of the systems’ time-varying PDFs [4]. While the notion of controlling PDFs may initially appear impractical using conventional control engineering methods, advancements in technologies like optical tweezers have rendered it feasible, which is particularly evident in applications such as colloidal systems [5,6,7,8], with specific implications in biomolecule evolution control [9].

Regarding the control of FP equations, seminal works [10,11] present methodologies to control the system PDF governed by FP equations [12]. Building upon this foundation, ref. [13] discusses a bilinear optimal control problem where the control function depends on time and space. In [14], the authors prove the existence of optimal controls by considering first-order necessary conditions in the optimisation problem.

In applications like Brownian motion, we can control FP equations via reverse-engineering approaches such as the engineered swing equilibration (ESE) method [15,16]. The ESE protocol imposes a solution to the FP equation to obtain the corresponding PDF’s control parameters that provide a shortcut for time-consuming relaxations (for further details, refer to the methods section of [17]). Ref. [8] offers a comprehensive review of similar reverse-engineered approaches.

Since FP equations frequently serve as mathematical descriptions of mesoscopic systems (for further details, see [12]), i.e., systems at the nano/microscale such as molecular motors, the time evolution of the system’s PDF may not only need to adhere to time constraints but also to multiple “thermodynamic” constraints to be deemed “efficient”. For instance, the system may require the minimisation of the entropy production [18,19], a reduction in information variability [20], or an enhancement of self-organisation [21]. The incorporation of these “thermodynamic constraints” into the optimisation process implies an extension of the current literature results on the control of FP equations.

A theory that could offer insights into addressing these optimisation challenges stems from information geometry, a field resulting from the fusion of information theory and differential geometry [22]. As an evolving field, information geometry proposes novel solutions to tasks, such as maximum likelihood estimation [23], state prediction [24,25], the quantification of causality [26,27,28], or maximum work extraction [4,29]. In stochastic thermodynamics [2,30], information geometry aids in obtaining time-varying descriptions of the aforementioned constraints. For instance, based on the well-known Cauchy–Schwartz inequality [31], ref. [32] presents an inequality between the Fisher divergence [33] and the information length (IL) [25,34] to quantify the disorder in irreversible decay processes. Ref. [35] introduces an inequality describing the information rate as a speed limit on the evolution of any observable. In [4], the geodesic of the IL describes the path with the least statistical variation connecting initial and final probability distributions of the system dynamics (for further details, see [20]). Consequently, information geometry can be employed in a control protocol to enforce geodesic dynamics on the system’s PDF time evolution, achieving the minimum “geometric information variability” [21] and thereby establishing an optimal speed limit for the system’s observables. The primary challenge addressed in this study revolves around devising and applying a technique to achieve this minimum geometric information variability.

Developing an optimal protocol for the time evolution of the system’s PDFs under multiple constraints requires an exploration of existing control algorithms. The literature presents a significant amount of control procedures, spanning from classical PID control [36] to more sophisticated algorithms, like data-driven, model-free, or fractional-order controls (for instance, see [37,38,39]). However, we require an algorithm capable of handling intricate optimisation problems while remaining a feasible option for implementation in future experimental setups. One of the most popular optimisation-based control techniques is the model predictive control (MPC) scheme [40]. Generally, MPC is an online optimisation algorithm designed for constrained control problems, whose advantages have been observed in applications within robotics [41], solar energy [42], or bioengineering [43]. Furthermore, MPC can be easily implemented thanks to packages such as CasADi [44] or the Hybrid Toolbox [45].

Based on the presented discussion, this work offers a solution to an optimisation problem that integrates the concepts of the information length and the quadratic regulator (QR) [46] to guide the system’s PDF time evolution along the path with the minimum geometric information variability (the geodesic of the information length) using MPC. Although we could implement the IL, QR, or MPC in more complex scenarios, we only study their application to stochastic processes described by linear stochastic differential equations. Hence, the system’s PDF will consistently maintain a Gaussian distribution over time, assuming that the system’s initial conditions follow a Gaussian distribution. Constraining the analysis to linear stochastic dynamics allows us to use a set of deterministic differential equations to describe the evolution of the mean and covariance of the Gaussian distribution within the MPC method. The applicability of such a prediction model favours MPC over other alternatives, such as reinforcement learning [47], which could have been explored to determine the geodesic of the information length within a similar context. Additionally, MPC provides a low offline complexity, mature stability–feasibility–robustness theory, and good constraint handling [47].

The algorithm is applied to the Ornstein–Uhlenbeck process [48] and the Kramers equation [25], which both describe a particle over a heath reservoir (mesoscopic stochastic dynamics). In practice, changes in temperature and optical tweezers can manipulate the dynamics of both the noise amplitude and mean value in such systems, respectively [2,7,49]. Using previous findings from [50], the effects of the MPC method on the Ornstein–Uhlenbeck process are analysed by comparing IL values with the entropy production and entropy rate in the closed-loop (feedback-controlled) system. As a rigorous closed-loop stability analysis is beyond the scope of this work, we provide only a brief description of the BIBO stability conditions that are considered to constrain the control actions proposed by the MPC. Finally, we give a set of concluding remarks and discuss future work.

To help readers, in the following, we summarise our notations. $\mathbb{R}$ is the set of real numbers. $\mathbf{x}$ $\in {\mathbb{R}}^n$ represents a column vector $\mathbf{x}$ of real numbers of dimension n, $\mathbf{A}\in {\mathbb{R}}^{n\times n}$ represents a real matrix of dimension $n\times n$ (bold-face letters are used to represent vectors and matrices), $Tr\left(\mathbf{A}\right)$ corresponds to the trace of the matrix $\mathbf{A}$, and $A_{ij}$ is the element at row i and column j of the matrix $\mathbf{A}$. $\left|\mathbf{A}\right|$, $\mathbf{vec}\left(\mathbf{A}\right)$, ${\mathbf{A}}^{\top }$, and ${\mathbf{A}}^{-1}$ are the determinant, vectorisation, transpose, and inverse of matrix $\mathbf{A}$, respectively. The value ${\mathbf{I}}_n$ denotes the identity matrix of order n. Newton’s notation is used for the partial derivative concerning the variable t (i.e., $\frac{\partial y}{\partial t}=\dot{y}$). In addition, the average of a random vector $\mathrm{\zeta }$ is denoted by $\mu :=\langle \mathrm{\zeta }\rangle$, with the angular brackets representing the average. Finally, in

Table 1. Description of the different variables used throughout this work.

Symbol	Description
$\mathrm{\zeta }$	Random vector variable
$\mathbf{x}$	Spatial variable of the PDF
$\xi$	Gaussian stochastic variable
$u\left(t\right)$	Bounded smooth external input (any time-dependent function)
$\mathbf{D}\left(t\right)$	Time-dependent noise amplitude matrix
$\mu$	Mean value vector of the random vector $\mathrm{\zeta }$
$\mathrm{\Sigma }$	Covariance matrix of the random vector $\mathrm{\zeta }$
$p(\mathbf{x};t)$	Probability density function (PDF)
$\mathcal{L}\left(t\right)$	Information length
$\mathrm{\Gamma }\left(t\right)$	Information rate
$\dot{S}$	Entropy rate
$\mathrm{\Pi }$	Entropy production
$\mathrm{\Phi }$	Entropy flow
$\mathbf{Y}\left(t\right)$	Vector state composed by the elements of $\mu$ and $\mathrm{\Sigma }$ at time t. The vector describes the current PDF at time t.
${\mathbf{Y}}_d$	Desired vector state. The vector describes the desired Gaussian PDF.
$\mathbf{c}\left(t\right)$	Vector of controls including u and the elements $D_{ij}$ of the amplitude noise matrix $\mathbf{D}$
$\mathbf{Q}$	Weight matrix regulating the error between $\mathbf{Y}\left(t\right)$ and ${\mathbf{Y}}_d$
$\mathbf{R}$	Weight matrix regulating the control action
$I_L$	Weight factor of the error between the current information rate $\mathrm{\Gamma }\left(t\right)$ at time t and the initial information rate $\mathrm{\Gamma }\left(0\right)$ to keep it constant at all t
$\widehat{\mathrm{\Gamma }}$	Predicted information rate. The ${ }^{^}$ symbol implies prediction.
$\mu \left[k\right]$	Discrete time mean value vector. The brackets $\left[k\right]$, where $k=0,1,2,\dots$, refers to the discrete time sampled at time period $T_s$.
$\gamma$	Damping constant
$\omega$	Undamped natural frequency constant
$T_s$	Sampling period
N	Prediction horizon length

, we provide a brief description of the variables used throughout this paper.

2. Preliminaries

As mentioned in Section 1, this work considers systems whose dynamics are governed by linear non-autonomous stochastic differential equations described in the form of the following set of Langevin equations:

$$\dot{\mathrm{\zeta }}\left(t\right)=\mathbf{A}\mathrm{\zeta }\left(t\right)+\mathbf{B}u\left(t\right)+\xi \left(t\right),$$

(1)

where $\mathbf{A}$ and $\mathbf{B}$ are $n\times n$ and $n\times 1$ real time-invariant matrices, respectively; $u\left(t\right)$ is a (bounded smooth) external input, and $\xi \in {\mathbb{R}}^n$ is a Gaussian stochastic noise given by an n dimensional vector of $\delta$-correlated Gaussian noises ${\xi }_i$ ($i=1,2,\dots n$), with the following statistical property:

$$\langle {\xi }_i\left(t\right)\rangle =0,\langle {\xi }_i\left(t\right){\xi }_j\left(t_1\right)\rangle =2D_{ij}\left(t\right)\delta (t-t_1),D_{ij}\left(t\right)=D_{ji}\left(t\right),\forall i,j=1,\dots ,n.$$

(2)

Readers seeking a comprehensive study of systems governed by Equation (1) can refer to [51]. Additionally, for a concise introduction to linear time-varying and time-invariant systems, ref. [52] provides valuable insights.

The Fokker–Planck equation of (1) can also be utilised to depict the system’s PDF dynamics, characterised by both time t and the spatial variable vector $\mathbf{x}={[x_1,x_2,\dots ,x_n]}^{\top }$. This is given as follows [53,54]:

$$\frac{\partial p(\mathbf{x};t)}{\partial t}=-\sum _{i,j=1}^n\frac{\partial }{\partial x_i}\left[\left(a_{ij}{x}_j+b_{i}u\left(t\right)\right)p(\mathbf{x};t)\right]+\sum _{i,j=1}^n{D}_{ij}\frac{{\partial }^2}{\partial x_i\partial x_j}p(\mathbf{x};t).$$

(3)

If the initial PDF is Gaussian, a solution to (3) for $p(\mathbf{x};t)$ at any time t is given by [53]

$$p(\mathbf{x};t)=\frac{1}{\sqrt{det\left(2\pi \mathrm{\Sigma }\right)}}{e}^{-\frac{1}{2}{\left(\mathbf{x}-\mu \right)}^{\top }{\mathrm{\Sigma }}^{-1}\left(\mathbf{x}-\mu \right)}.$$

(4)

Equation (4) describes the dynamics of the probability distribution of the random variable $\mathrm{\zeta }$ governed by (1). The value of the mean $\mu \left(t\right)$ and covariance $\mathrm{\Sigma }\left(t\right)$ in a linear stochastic process can be computed from the solution of the following set of differential equations [51]:

$$\begin{array}{ccc}\hfill \dot{\mu }\left(t\right)& =& \mathbf{A}\mu \left(t\right)+\mathbf{B}u\left(t\right),\hfill \end{array}$$

(5a)

$$\begin{array}{ccc}\hfill \dot{\mathrm{\Sigma }}& =& \mathbf{A}\mathrm{\Sigma }\left(t\right)+\mathrm{\Sigma }\left(t\right){\mathbf{A}}^{\top }+2\mathbf{D}\left(t\right),\hfill \end{array}$$

(5b)

where $\mathbf{D}\in {\mathbb{R}}^{n\times n}$ is the matrix with elements $D_{ij}\left(t\right)$. In this work, Equations (5a) and (5b) will be used to predict the behaviour of the probability distribution of the random variable $\mathbf{x}$ in our control method.

2.1. Information Length (IL)

To obtain a minimum geometric information variability in the time evolution of the PDF of Equation (1), we need to investigate the concept of IL in more detail. In mathematical terms, given the joint PDF $p(\mathbf{x};t)$ of the n-th order random variable $\mathrm{\zeta }$, we define its IL $\mathcal{L}$ as follows:

$$\mathcal{L}\left(t\right)={\int }_0^t\left(\sqrt{{\int }_{{\mathbb{R}}^n}p(\mathbf{x};\tau ){\left[{\partial }_{\tau }lnp(\mathbf{x};\tau )\right]}^2{\mathrm{d}}^{n}x}\right)\mathrm{d}\tau ={\int }_0^t\mathrm{\Gamma }\left(\tau \right)\mathrm{d}\tau .$$

(6)

The value $\mathrm{\Gamma }{\left(\tau \right)}^2={\int }_{{\mathbb{R}}^n}p(\mathbf{x};\tau ){\left[{\partial }_{\tau }lnp(\mathbf{x};\tau )\right]}^2{\mathrm{d}}^{n}x$ in (6) corresponds to the square of the Fisher information by considering the time as a parameter, and it is commonly called the information rate [21]. The dimension of $1/\mathrm{\Gamma }\equiv \tau$ is time, which means that it serves as a dynamical time unit for information change. Hence, as shown in Figure 1, the integration of $\mathrm{\Gamma }$ between time 0 and t gives the total information change in that time interval; i.e., $\mathcal{L}$ quantifies the number of statistically different states that the system passes through in time from an initial $p(\mathbf{x};0)$ to a final $p(\mathbf{x};t)$ [55]. Furthermore, the IL is a model-free tool that proved to be a true metric between two PDFs in the statistical space [26]. In [33], $\tau$ was shown to provide a universal bound on the timescale of transient dynamical fluctuations, independent of the physical constraints on the stochastic dynamics or their function.

For Gaussian distributions, according to [25], we know that $\mathrm{\Gamma }$ can be rewritten in terms of $\mu$ and $\mathrm{\Sigma }$ as follows:

$$\mathrm{\Gamma }{\left(\tau \right)}^2=\dot{\mu }{\left(t_1\right)}^{\top }{\mathrm{\Sigma }}^{-1}\left(t_1\right)\dot{\mu }\left(t_1\right)+\frac{1}{2}Tr\left({\left({\mathrm{\Sigma }}^{-1}\left(t_1\right)\dot{\mathrm{\Sigma }}\left(t_1\right)\right)}^2\right).$$

(7)

Equation (7) simplifies the analysis of the IL when studying systems like (1).

2.2. Information–Thermodynamic Relation

However, while we recognise the utility of the IL as a tool for analysing the dynamics of time-varying PDFs, its physical significance remains unclear. Therefore, establishing a connection to a physical quantity is crucial.

In this context, consider the value of the entropy rate defined as follows: [56]

$$\dot{S}\left(t\right)=-{\int }_{{\mathbb{R}}^n}\dot{p}(\mathbf{x};t)\ln \left(p(\mathbf{x};t)\right)\mathrm{d}\mathbf{x}=\mathrm{\Pi }-\mathrm{\Phi },$$

(8)

where $\mathrm{\Pi }$ is the entropy production due to irreversible processes occurring inside the system and $\mathrm{\Phi }$ is the entropy flux from the system to the environment. Figure 2 shows a system subject to some boundary conditions to avoid matter exchange (i.e., a closed system). The system produces entropy $\mathrm{\Pi }>0$$\forall t$ and exchanges entropy $\mathrm{\Phi }$ with the environment (hence, it shares energy). Specifically, $\mathrm{\Phi }>0$ ($\mathrm{\Phi }<0$) when the entropy flows from the system (environment) to the environment (system). A system with a minimum entropy production $\mathrm{\Pi }$ produces the highest amount of free energy (useful work) [21,57]. We refer the reader to [58,59] for a complete review of thermodynamics. In a first-order linear stochastic differential equation, the value of the information rate $\mathrm{\Gamma }$ can be related to $\dot{S}$ and $\mathrm{\Pi }$ as follows (for further details, see [50]):

$${\mathrm{\Gamma }}^2=\frac{D}{\mathrm{\Sigma }}\mathrm{\Pi }+{\dot{S}}^2\forall D,\mathrm{\Sigma }>0,$$

(9)

where D and $\mathrm{\Sigma }$ are the noise amplitude and the variance of the first-order Langevin equation.

Equation (9) can be used to describe the physical effects of a minimum variability control in a dynamical system. For instance, as will be discussed later in Section 4, to obtain a minimum information variability while being out of the equilibrium (i.e., $\mathrm{\Pi }>0$), the control will exert entropy to the environment, creating a small but negative value for $\dot{S}$. This means that a minimum information variability would not necessarily lead to a maximum free energy but to an optimal path where $\mathrm{\Pi }$ is limited by the value of $\mathrm{\Gamma }$ according to (9). The complete derivation of how to compute the values of $\dot{S},\mathrm{\Pi }$, and $\mathrm{\Phi }$ for a first-order Langevin equation is shown in Appendix C.

2.3. Minimum Information Variability Problem

Now that we understand the meaning of the IL, let us explain in more detail how the IL can be used to minimise deviations from the geodesic of the system’s PDF time evolution. In [32], the authors use the inequality $\mathcal{J}\left(t_F\right)\ge \mathcal{L}{\left(t_F\right)}^2$ where $\mathcal{J}=\tau {\int }_{t_0}^{t_f}{\mathrm{\Gamma }}^2\left(t\right)\mathrm{d}t={\int }_{t_0}^{t_f}\mathrm{d}t\int \mathrm{d}x\frac{1}{p(x;t)}{\left[\frac{\partial p(x;t)}{\partial t}\right]}^2$ (Fisher divergence) with $\tau =t_f-t_0$ and $\mathcal{L}$ given by (6). As mentioned in Section 1, such inequality follows from the Cauchy–Schwartz inequality $\int {\mathrm{\Gamma }}^2\mathrm{d}t\int u^2\mathrm{d}t\ge {\left(\int \mathrm{\Gamma }u\mathrm{d}t\right)}^2$ with $u=1$. But, most importantly, the equality holds for the minimum path where $\mathrm{\Gamma }$ is constant (see, e.g., [19,32]), and the deviation from this equality is said to quantify the amount of the disorder in an irreversible process [32].

From [20], such a statement can be clarified by the following procedure. Let us define the time-average for a function $f\left(t\right)$ as $\mathbb{E}\left[f\left(t\right)\right]=\frac{1}{\tau }{\int }_{t_0}^{t_f}f\left(t\right)\mathrm{d}t$. Then, we can define the time-averaged variance

$${\sigma }^2=\frac{\mathcal{J}-{\mathcal{L}}^2}{{\tau }^2}=\mathbb{E}\left[{\mathrm{\Gamma }}^2\right]-\mathbb{E}{\left[\mathrm{\Gamma }\right]}^2\ge 0.$$

(10)

Equation (10) describes an accumulative deviation from the geodesic connecting the initial and final distributions of the system dynamics. Thus, we can conclude that by setting $\mathrm{\Gamma }$ as a constant, we obtain a geodesic that defines a path where the process has the minimum geometric information variability.

2.4. BIBO Stability of the Linear Stochastic Process

As we address a control problem, it is essential to examine the bounded-input, bounded-output (BIBO) stability of (1). For this purpose, we revisit the BIBO stability conditions for Equations (5a) and (5b), which will be instrumental in our discussion of MPC stability.

Theorem 1.

The mean (5a) and covariance (5b) dynamics of (1) are BIBO-stable if and only if the eigenvalues ${\lambda }_i$ of the matrix $\mathbf{A}$ satisfy the following inequality:

$$\Re \left[{\lambda }_i\right]<0,$$

(11)

where $\Re \left[s\right]$ stands for the real part of the complex value $s\in \mathbb{C}$.

Proof. 

For a detailed proof of this result, please refer to [60,61]. □

Remark 1.

Theorem 1 is considered to be satisfied throughout our examples; i.e., the control method is applied to stable systems only. Furthermore, the control actions are constrained to finite values.

3. Main Results

To guide the system’s PDF time evolution along the geodesic of the IL while achieving a desired set point at time $t=t_f$, we propose the following cost function:

$$J={\int }_0^{t_f}\left(I_L{({\mathrm{\Gamma }}^2\left(\tau \right)-{\mathrm{\Gamma }}^2\left(0\right))}^2+{\left(\mathbf{Y}\left(\tau \right)-{\mathbf{Y}}_d\left(\tau \right)\right)}^{\top }\mathbf{Q}\left(\mathbf{Y}\left(\tau \right)-{\mathbf{Y}}_d\left(\tau \right)\right)+{\mathbf{c}}^{\top }\left(\tau \right)\mathbf{R}\mathbf{c}\left(\tau \right)\right)\mathrm{d}\tau ,$$

(12)

where $I_L\in \mathbb{R},\mathbf{Q}\in {\mathbb{R}}^{(n+n^2)\times (n+n^2)}$; $\mathbf{Y}:={[\mu ,\mathbf{vec}\left(\mathrm{\Sigma }\right)]}^{\top }\in {\mathbb{R}}^{n+n^2}$ is the vector of states $\mu$ and $\mathbf{vec}\left(\mathrm{\Sigma }\right)$ that define the time evolution of $p(\mathbf{x};t)$; ${\mathbf{Y}}_d={[{\mu }_d,\mathbf{vec}\left({\mathrm{\Sigma }}_d\right)]}^{\top }$ is the desired position of the $n+n^2$ states defined by ${\mu }_d$; and ${\mathrm{\Sigma }}_d$ at time t, and $\mathbf{c}\in {\mathbb{R}}^m$ (such that $m\le 1+n^2$) is the vector of controls defined by $\mathbf{c}={[c_1,c_2,\dots ,c_m]}^{\top }:=[u\left(t\right),\mathbf{w}:={\left\{\left(D_{ij}\right|D_{ij}\ne 0\in \mathbf{D}\forall i,j=1,2,\dots ,n\}\right]}^{\top }$. Therefore, $\mathbf{R}\in {\mathbb{R}}^{m\times m}$. In this work, we call Equation (12) the Information Length Quadratic Regulator(IL-QR). As will be discussed in Section 3, MPC enables us to obtain the solution of (12) via a numerical scheme, circumventing analytic complexities while being applicable to practical scenarios. Analytically finding the geodesic dynamics involves using the solution of the Euler–Lagrange equations of the IL. The steps of such an approach are discussed and successfully applied in [4] for a first-order stochastic differential equation. In Appendix A, we give the details of the procedure when considering a more generalised scenario.

From (12), we observe that the first term on the right-hand side imposes a constant ${\mathrm{\Gamma }}^2$ (needed to minimise the deviations from the geodesic [4]). The term involving $\mathbf{Q}$ guides the system towards a specified PDF defined by ${\mathbf{Y}}_d$. The third term on the right-hand side of (12) regularises the control action $\mathbf{c}$ to prevent abrupt changes in the inputs. Furthermore, $\mathbf{Q}$ and $\mathbf{R}$ are matrices that penalise the error $ϵ=\mathbf{Y}-{\mathbf{Y}}_d$ and the control input u, respectively. Additionally, $I_L$ penalises the square of the error $({\mathrm{\Gamma }}^2\left(t\right)-{\mathrm{\Gamma }}^2\left(0\right))$, aiming to maintain ${\mathrm{\Gamma }}^2\left(t\right)$ close to ${\mathrm{\Gamma }}^2\left(0\right)$ over time.

In our approach, the control of the dynamics for $\mu$ is governed by $u\left(t\right)$, while the dynamics of $\mathrm{\Sigma }\left(t\right)$ are adjusted by controlling the noise width using a time-dependent vector $\mathbf{w}\left(t\right)$, where its elements replace the non-zero constant values of the matrix $\mathbf{D}$ in (5b). As discussed in the numerical examples, the noise width can be altered by modifying a macroscopic observable such as temperature (for further details, refer to the Brownian motion models presented in [12]).

Model Predictive Control

As discussed in Section 1, the solution of our optimisation problem will be computed through the MPC method. Hence, the following discrete optimal control problem encoding the MPC formulation is required:

$$\begin{array}{cc}\hfill \mathbf{c}=\arg \underset{\tilde{\mathbf{c}}}{min}& \left(J_N=\sum _{k=0}^N\left(I_L{\left({\widehat{\mathrm{\Gamma }}}^2\left[k\right]-{\mathrm{\Gamma }}^2\left[0\right]\right)}^2\right.\right.\hfill \\ \hfill & \left.\left.+{(\widehat{\mathbf{Y}}\left[k\right]-{\mathbf{Y}}_d)}^{\top }\mathbf{Q}(\widehat{\mathbf{Y}}\left[k\right]-{\mathbf{Y}}_d)+{\tilde{\mathbf{c}}}^{\top }\left[k\right]\mathbf{R}\tilde{\mathbf{c}}\left[k\right]\right)\right)\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& {\widehat{\mathrm{\Gamma }}}^2\left[k\right]=f(\widehat{\mathbf{Y}}\left[k\right],\tilde{\mathbf{c}}\left[k\right])\hfill \\ \hfill & \widehat{\mathbf{Y}}\left[k\right]={[\mu [\tilde{\mathbf{c}},k],\mathbf{vec}\left(\mathrm{\Sigma }[\tilde{\mathbf{c}},k]\right)]}^{\top }\hfill \\ \hfill & \mu [\tilde{\mathbf{c}},k]={\mathbf{A}}_d\mu [k-1]+{\mathbf{B}}_{d}u[\tilde{\mathbf{c}},k-1]\hfill \\ \hfill & \mathrm{\Sigma }[\tilde{\mathbf{c}},k]={\mathbf{A}}_d\mathrm{\Sigma }[k-1]+\mathrm{\Sigma }[k-1]{\mathbf{A}}_d^{\top }+2\mathbf{D}[\tilde{\mathbf{c}},k-1]\hfill \\ \hfill & \mu \left[0\right]=\mathbf{m},\mathrm{\Sigma }\left[0\right]=\mathbf{S}\forall \mathbf{m}\in {\mathbb{R}}^n,\mathbf{S}\in {\mathbb{R}}^{n\times n}\hfill \\ \hfill & \tilde{\mathbf{c}}\left[k\right]={[u\left[k\right],\mathbf{w}\left[k\right]]}^{\top }\hfill \\ \hfill & c_{l,i}\le c_i\le c_{u,i}{c}_{l,i},c_{u,i}\in \mathbb{R}\forall i=1,2,\dots ,m.\hfill \end{array}$$

(13)

In Equation (13), the ${ }^{^}$ symbol over $\mathbf{Y}$ and $\mathrm{\Gamma }$ refers to their predicted values over the influence of the control $\tilde{\mathbf{c}}$ throughout the optimisation process in the finite horizon of length N. Note that the value of $\widehat{\mathbf{Y}}$ is constrained by the discretised version of the set of Equations (5a) and (5b) given by

$$\begin{array}{ccc}\hfill \mu \left[k\right]& =& {\mathbf{A}}_d\mu [k-1]+{\mathbf{B}}_{d}u[k-1],\hfill \end{array}$$

(14)

$$\begin{array}{ccc}\hfill \mathrm{\Sigma }\left[k\right]& =& {\mathbf{A}}_d\mathrm{\Sigma }[k-1]+\mathrm{\Sigma }[k-1]{\mathbf{A}}_d^{\top }+2\mathbf{D}[k-1],\hfill \end{array}$$

(15)

where ${\mathbf{A}}_d=\mathbf{I}+T_s\mathbf{A}$ and ${\mathbf{B}}_d=T_s\mathbf{B}$ if a first-order approximation of the time derivative considering the sample period $T_s$ is applied (we apply a fourth-order Runge Kutta instead of a first-order approximation to compute $\widehat{\mathbf{Y}}$ in our simulations). Note that we have added the argument $\tilde{\mathbf{c}}$ in Equation (13) when describing Equations (14) and (15) to emphasise the application of the control during the optimisation procedure. The initial conditions $\mu \left[0\right],\mathrm{\Sigma }\left[0\right]$ of (14) and (15) change every time an optimal control $\mathbf{c}$ solution has been computed and they are subject to the measurements $\mathbf{m},\mathbf{S}$ of the real/simulated process. Every element $c_i$ of the control vector $\tilde{\mathbf{c}}$ is constrained by a lower and an upper limit denoted as $c_{l,i}$ and $c_{u,i}$, respectively. Finally, f is the function describing the predicted value ${\widehat{\mathrm{\Gamma }}}^2$, which is defined as follows:

$$\begin{array}{c}f(\widehat{\mathbf{Y}}\left[k\right],\tilde{\mathbf{c}}\left[k\right])={\left(\mathbf{A}\mu \left[k\right]+\mathbf{B}u[\tilde{\mathbf{c}},k]\right)}^{\top }\mathrm{\Sigma }{\left[k\right]}^{-1}\left(\mathbf{A}\mu \left[k\right]+\mathbf{B}u[\tilde{\mathbf{c}},k]\right)\hfill \\ \hfill \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}+\frac{1}{2}Tr\left({\left(\mathrm{\Sigma }{\left[k\right]}^{-1}\left(\mathbf{A}\mathrm{\Sigma }[k-1]+\mathrm{\Sigma }[k-1]{\mathbf{A}}^{\top }+2\mathbf{D}[\tilde{\mathbf{c}},k-1]\right)\right)}^2\right).\end{array}$$

(16)

For a clearer grasp of the MPC method’s application in solving the IL-QR problem in real-world scenarios, Figure 3a,b show the control diagram and the functionality of the MPC’s optimiser when considering a second-order stochastic system, respectively.

Figure 3a illustrates the real-time operation of the MPC algorithm. As the process evolves, the MPC algorithm receives input parameters, including a given set point ${\mathrm{\Gamma }}^2\left[0\right],{\mathbf{Y}}_d$, the prediction values $\tilde{\mathbf{Y}}$ of $\mu$ and $\mathrm{\Sigma }$ from a prediction model, a set of constrains, the cost function $J_N$, and the current system dynamics $\mathbf{Y}$, to solve the optimisation problem given in (13). Afterwards, the optimal control $\mathbf{c}$ solution of Equation (13) is applied to the system. The MPC method determines the optimal control $\mathbf{c}$ by utilising the differential equations of $\dot{\mu }$ and $\mathrm{\Sigma }$ as a prediction model in a finite horizon of length N. Figure 3b provides a brief overview of the working principle of the MPC’s optimiser block.

In the case of a stochastic process described by a bivariate time-varying PDF p with random variables $x_1$ and $x_2$, the MPC optimiser method initiates the optimisation process using the measured system’s PDF output (depicted in black). The optimisation process involves extrapolating the values of the PDF p within a finite horizon of length N and comparing them with the reference trajectory described by the PDF $p_d$. The optimisation problem is solved using the interior point method with CasADi [44]. In this study, the use of deterministic descriptions of the first two statistical moments over time has facilitated the control, prediction, and simulation of the PDF, owing to the nature of the Langevin equations under consideration. However, in scenarios involving pure data or more complex stochastic differential equations, estimating time-varying PDFs may require inference methods [62] or stochastic simulations [63].

4. Simulation Results

To demonstrate the numerical implementation of MPC for solving the IL-QR cost function (13), the subsequent subsections delve into applying the minimum information variability to both the Ornstein–Uhlenbeck process and the Kramers equation. Throughout the simulations, we utilise various parameters, which are conveniently summarised in

Table 2. Parameters used in the simulation results for the O-U and Kramers systems. The table includes figure numbers showing the simulation where the set of parameters was used.

System	Experiment	Figure	Y(0)			Yd(t)			${\mathbf{\Gamma }}^2$(0)
O-U	1	5, 6	${[5/6,1/(2\times 0.3)]}^{\top }$			${[1/30,1/(2\times 0.3)]}^{\top }$			2.4
	2	7, 8	${[5/6,1/(2\times 0.3)]}^{\top }$			${[1/30,1/(2\times 3)]}^{\top }$			0.41
Kramers	1	10	${[5/6,5/6,1/(2\times 0.3),0,1/(2\times 0.3)]}^{\top }$			${[0,0,1/(2\times 3),0,1/(2\times 3)]}^{\top }$			6.16667
	2	11	${[5/6,5/6,1/(2\times 0.3),0,1/(2\times 0.3)]}^{\top }$			${[-5/6,0,1/(2\times 3),0,1/(2\times 3)]}^{\top }$			6.16667
System	Experiment	Figure	γ	ω	Ts	N	IL	R	Q
O-U	1	5, 6	1	-	$1\times {10}^{-3}$	50	$1\times {10}^3$	$1\times {10}^{-2}{\mathbf{I}}_2$	$Q_{12}=Q_{21}=0,Q_{11}=1\times {10}^2$ and $Q_{22}=5\times {10}^2$
	2	7, 8	1	-	$1\times {10}^{-3}$	50	$1\times {10}^4$	$1\times {10}^{-2}{\mathbf{I}}_2$	$Q_{12}=Q_{21}=0,Q_{11}=1\times {10}^2$ and $Q_{22}=5\times {10}^2$
Kramers	1	10	2	1	$1\times {10}^{-1}$	50	$5\times {10}^3$	$1\times {10}^{-5}{\mathbf{I}}_3$	$1\times {10}^2{\mathbf{I}}_5$
	2	11	2	1	$1\times {10}^{-1}$	50	$5\times {10}^3$	$1\times {10}^{-5}{\mathbf{I}}_3$	$1\times {10}^2{\mathbf{I}}_5$

. Each system undergoes a pair of simulations or “experiments”, as detailed in Table 2.

4.1. The O-U Process

First, let us consider the Ornstein–Uhlenbeck (O-U) process (see Figure 4) defined by the following Langevin equation:

$$\dot{\mathrm{\zeta }}=-\gamma (\mathrm{\zeta }\left(t\right)-u\left(t\right))+\xi \left(t\right),$$

(17)

where $\mathrm{\zeta }\left(t\right)$ is a random variable, $u\left(t\right)$ is a deterministic force, $\gamma$ is a damping constant, and $\xi \left(t\right)$ is a short correlated random forcing such that $\langle \xi \left(t\right)\xi \left(t_1\right)\rangle =2D\delta (t-t_1)$ with $D\ge 0$ and $\langle \xi \left(t\right)\rangle =0$.

The results of the MPC implementation are shown in Figure 5, Figure 6, Figure 7 and Figure 8. In the following, it is important to note that, in the figures, black colour is used to represent values labelled on the y-left axis, while blue colour is used for values labelled on the y-right axis, unless otherwise specified. Figure 5 illustrates the scenario where the desired state ${\mathbf{Y}}_d$ of the O-U process is ${\mathbf{Y}}_d={[1/30,1/(2\times 0.3)]}^{\top }$. It also shows the time evolution of the states $\mathbf{Y}\left(t\right)=[\mu ,\beta (t\left)\right]$ and controls $\mathbf{c}\left(t\right)={[u\left(t\right),D\left(t\right)]}^{\top }$. From the results, we see that the method finds a geodesic motion (solution to the IL-QR) from the initial to the final state in less than $0.4$ seconds. The geodesic motion is corroborated by the constant value of ${\mathrm{\Gamma }}^2\left(t\right)\approx {\mathrm{\Gamma }}^2\left(0\right)=2.4$ and the plot of the information length $\mathcal{L}$ whose shape is a line with a slope of $1.5526$. ${\mathrm{\Gamma }}^2\left(0\right)$ is computed by considering that $u(t=0)=D(t=0)=0$. It is noteworthy that the value of $\mathrm{\Sigma }$ is observed to vary slightly over time compared to the hyperbolic analytical solution in [4], which is provided for a non-constant $\mathrm{\Sigma }$ (refer to Appendix A).

When analysing the stochastic thermodynamics of the O-U system controlled by the MPC technique, Figure 6 presents the plot of the entropy rate $\dot{S}$ in comparison with the entropy production $\mathrm{\Pi }$, along with a plot of the value of ${\mathrm{\Gamma }}^2$ using Equation (9). The analytical expressions for $\dot{S}$ and $\mathrm{\Pi }$, along with their derivations, are given in Appendix C. In the closed-loop system, we can see that the MPC method induces slight changes in both the entropy production $\mathrm{\Pi }$ and the entropy rate $\dot{S}$ during the process. Since the values of $\mathrm{\Sigma }$ and $\mu$ in the desired state ${\mathbf{Y}}_d$ are close to, and far from, their initial conditions at state $\mathbf{Y}\left(0\right)$, respectively, the balance between $\dot{S},\mathrm{\Pi }$, and ${\mathrm{\Gamma }}^2$ as given by (9) is kept by maintaining an almost constant $D\left(t\right)$ and a $u\left(t\right)$ with a nearly constant velocity.

Under conditions almost similar to those in Figure 5, Figure 7 shows the behaviour of the closed-loop system PDF, the states $\mu$ and $\mathrm{\Sigma }$, and the controls D and u, as well as the behaviour of ${\mathrm{\Gamma }}^2$ for ${\mathbf{Y}}_d\left(t\right)={[1/30,1/(2\times 3)]}^{\top }$. Here, ${\mathrm{\Gamma }}^2\left(0\right)$ is computed by considering that $u(t=0)=0$ and $D(t=0)=1/(2\times 0.3)$. The final state ${\mathbf{Y}}_d$ is achieved at approximately $t=2.8$. Once again, the geodesic behaviour is supported by the constant value of ${\mathrm{\Gamma }}^2\left(t\right)\approx {\mathrm{\Gamma }}^2\left(0\right)=0.41$ and the plot of the information length $\mathcal{L}$, which depicts a line with a slope of $0.64759$.

In comparison to the stochastic thermodynamics shown in Figure 6, Figure 8 exhibits minor changes in the entropy production $\mathrm{\Pi }$ and significant variations in the entropy rate $\dot{S}$ of the closed-loop system, given that the values of $\mathrm{\Sigma }$ and $\mu$ in the desired state ${\mathbf{Y}}_d$ differ from its initial condition $\mathbf{Y}\left(0\right)$. This disparity also leads to slight fluctuations in both the $D\left(t\right)$ and $u\left(t\right)$. Additionally, Figure 8 demonstrates the validity of the balance in Equation (9).

4.2. Kramers Equation

To study the solution of the IL-QR problem in a higher-order system via the MPC method, let us consider the so-called Kramers equation. The Kramers equation describes the Brownian motion of particles in an external field [12]. The non-autonomous version of the Kramers equation is given by the following set of Langevin equations:

$$\begin{array}{c}\hfill \left[\begin{array}{c}{\dot{\mathrm{\zeta }}}_1\\ {\dot{\mathrm{\zeta }}}_2\end{array}\right]=\left[\begin{array}{cc}0& 1\\ -{\omega }^2& -\gamma \end{array}\right]\left[\begin{array}{c}{\mathrm{\zeta }}_1\\ {\mathrm{\zeta }}_2\end{array}\right]+\left[\begin{array}{c}0\\ 1\end{array}\right]\xi \left(t\right)+\left[\begin{array}{c}0\\ 1\end{array}\right]u\left(t\right)\end{array}$$

(18)

Here, $\xi$ is a short correlated Gaussian noise with a zero mean $\langle \xi \rangle =0$ and a strength $D\ge 0$ with the following property:

$$\langle \xi \left(t\right)\xi \left(t^{\prime }\right)\rangle =2D\delta (t-t^{\prime }).$$

(19)

In practice, as shown in Figure 9, the Kramers Equation (18) is also a good first approximation to describe the dynamics in one-dimension of a particle in an optical trap [5]. The Kramers equation control $\mathbf{c}$ and state $\mathbf{Y}$ vectors are defined by

$$\begin{array}{ccc}\hfill \mathbf{c}& =& {[u,D]}^{\top },\hfill \end{array}$$

(20)

$$\begin{array}{ccc}\hfill \mathbf{Y}& =& {[{\mu }_1,{\mu }_2,{\mathrm{\Sigma }}_{11},{\mathrm{\Sigma }}_{12},{\mathrm{\Sigma }}_{22}]}^{\top }.\hfill \end{array}$$

(21)

Here, ${\mu }_1$ and ${\mu }_1$ are the mean values of the random variables ${\mathrm{\zeta }}_1$ and ${\mathrm{\zeta }}_2$, respectively. ${\mathrm{\Sigma }}_{11}$, ${\mathrm{\Sigma }}_{12}$, and ${\mathrm{\Sigma }}_{22}$ are the values describing the covariance matrix $\mathrm{\Sigma }$.

Figure 10 and Figure 11 show the simulation results of the closed-loop Kramers equation when considering the desired states ${\mathbf{Y}}_d\left(t\right)={[0,0,1/(2\times 3),0,1/(2\times 3)]}^{\top }$ and ${\mathbf{Y}}_d\left(t\right)={[-5/6,0,1/(2\times 3),0,1/(2\times 3)]}^{\top }$, respectively. These figures include the time evolution plots of the mean values ${\mu }_1$ and ${\mu }_2$ and the covariance matrix values ${\mathrm{\Sigma }}_{11}$, ${\mathrm{\Sigma }}_{12}$, and ${\mathrm{\Sigma }}_{22}$ of the Kramers equation random variables ${\mathrm{\zeta }}_1$ and ${\mathrm{\zeta }}_2$. In addition, they include the time evolution of the bivariate PDF $p(\mathbf{x};t)$ with the spatial vector $\mathbf{x}={[x_1,x_2]}^{\top }$, with $x_1$ and $x_2$ representing the position and velocity of the system dynamics, respectively. The remaining parameters used in the simulations are listed in Table 2. Again, in the figures, black colour is used to represent values labelled on the y-left axis, while blue colour is used for values labelled on the y-right axis, unless otherwise specified.

For the first numerical experiment, Figure 10 demonstrates that the MPC method is capable of maintaining ${\mathrm{\Gamma }}^2$ as constant through time while reaching the desired state ${\mathbf{Y}}_d$. Here, the value of $\mathrm{\Gamma }\left(0\right)$ in (13) is obtained as follows:

$$\begin{array}{c}{\mathrm{\Gamma }}^2\left(0\right)={\left(\mathbf{A}\mu \left(0\right)+\mathbf{B}u\left(0\right)\right)}^{\top }{\mathrm{\Sigma }}^{-1}\left(0\right)\left(\mathbf{A}\mu \left(0\right)+\mathbf{B}u\left(0\right)\right)\hfill \\ \hfill \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}+\frac{1}{2}Tr\left({\left({\mathrm{\Sigma }}^{-1}\left(0\right)(\mathbf{A}\mathrm{\Sigma }\left(0\right)+\mathrm{\Sigma }\left(0\right){\mathbf{A}}^{\top }+2\mathbf{D}\left(0\right))\right)}^2\right)=6.16667,\end{array}$$

(22)

where $u\left(0\right)=0$ and $\mathbf{vec}\left(\mathbf{D}\left(0\right)\right)={[0,0,0,1/(2\times 0.3)]}^{\top }$, while $\mu \left(0\right),\mathrm{\Sigma }\left(0\right)$ and $\mathbf{A}$ are taken from the corresponding $\mathbf{Y}\left(0\right)$ and the mathematical model (18), respectively. The geodesic dynamics give a behaviour with slow oscillations in the state $\mathbf{Y}$. The controls u and D present high oscillations as the system reaches the desired state ${\mathbf{Y}}_d$. The system reaches ${\mathbf{Y}}_d$ at $t\approx 7$ with an error of $1\times {10}^{-3}$. The geodesic behaviour is supported by the linear behaviour of the information length $\mathcal{L}$ compared to the fitted equation $y=24.8332t$.

Figure 10 shows a second numerical experiment where ${\mathbf{Y}}_d$ is even farther from the system’s equilibrium. Yet, the MPC method can maintain ${\mathrm{\Gamma }}^2$ as constant through time while reaching ${\mathbf{Y}}_d$. Like the example of Figure 10, in this case ${\mathrm{\Gamma }}^2\left(0\right)=6.16667$. Small oscillations persists in the time evolution of ${\mu }_1$, ${\mu }_2$, ${\mathrm{\Sigma }}_{11}$, c${\mathrm{\Sigma }}_{12}$, and ${\mathrm{\Sigma }}_{22}$. The system reaches the desired state ${\mathbf{Y}}_d$ at $t\approx 8.5$. Thus, the farther the desired state ${\mathbf{Y}}_d$ is from the initial state $\mathbf{Y}$, the longer the time it takes to reach it while following the geodesic path. The geodesic behaviour is evidenced by the plot of the information length $\mathcal{L}$, whose behaviour is compared to the fitted equation $y=24.8336t$. Similar to the example in Figure 10, the controls exhibit highly oscillatory behaviour as the system reaches ${\mathbf{Y}}_d$.

5. Conclusions

In this work, we demonstrated the application of the MPC method, an online optimisation algorithm for constrained control problems, to achieve the minimum information variability in systems governed by linear stochastic differential equations. The linearity of the system results in time-varying Gaussian PDFs, with statistical moments determined by a set of deterministic differential equations. Our simulations demonstrate that the MPC method is a practical approach for determining the geodesic of the information length between the initial and desired probabilistic states through the solution of the proposed IL-QR loss function.

From a thermodynamic standpoint, simulations of the MPC in the O-U process reveal that the MPC directly influences the amount of entropy production generated by the system to meet all optimisation requirements. Future work will involve extending this approach to nonlinear stochastic dynamics, such as toy models [64], systems with uncertain physical parameters [65], Brownian motion [66], or diffusion [67,68]. This extension aims to maximise the free energy [21] by minimising the entropy production and to analyse closed-loop stochastic thermodynamics for higher-order systems. It is worth noting that extending the method to nonlinear dynamics can be achieved through the Laplace assumption [69] and/or by employing Kalman/particle filter methods [70,71].