Bayesian Inference of Recurrent Switching Linear Dynamical Systems with Higher-Order Dependence

Abstract

Many complicated dynamical events may be broken down into simpler pieces and efficiently described by a system that shifts among a variety of conditionally dynamical modes. Building on switching linear dynamical systems, we develop a new model that extends the switching linear dynamical systems for better discovering these dynamical modes. In the proposed model, the linear dynamics of latent variables can be described by a higher-order vector autoregressive process, which makes it feasible to evaluate the higher-order dependency relationships in the dynamics. In addition, the transition of switching states is determined by a stick-breaking logistic regression, overcoming the limitation of a restricted geometric state duration and recovering the symmetric dependency between the switching states and the latent variables from asymmetric relationships. Furthermore, logistic regression evidence potentials can appear as conditionally Gaussian potentials by utilizing the Pólya-gamma augmentation strategy. Filtering and smoothing algorithms and Bayesian inference for parameter learning in the proposed model are presented. The utility and versatility of the proposed model are demonstrated on synthetic data and public functional magnetic resonance imaging data. Our model improves the current methods for learning the switching linear dynamical modes, which will facilitate the identification and assessment of the dynamics of complex systems.

1. Introduction

Complex systems frequently exhibit multiple levels of abstraction in their descriptions [1]. For example, a computer program can be characterized by the collection of functions it calls, the sequence in which it carries out statements or the assembly instructions it sends to the CPU. This assertion is true for a multitude of natural systems. Brain activity can be classified based on either broad psychological states or the activation of individual ion channels. The necessary amount of specificity may differ based on the particular task being performed [2]. Through the identification of these behavioral units and their interdependence, we can obtain a deeper understanding of the intricate mechanisms that give rise to complex natural occurrences. Furthermore, modern machine learning offers powerful tools to help model the dynamics of complex systems. The toolbox has recently been improved to incorporate more versatile elements, such as Gaussian processes [3] and neural networks [4], into probabilistic time series models.

Time series analysis encompasses several methodologies and models, such as stationary process models, spectral models, state space models, and non-linear models [5]. Among them, the state space methods are regarded as more versatile and adept at addressing a broader range of problems compared to the other models. Hidden Markov models (HMM) and switching linear dynamical systems (SLDS) are two well-known state space models. In numerous real-world time series situations, the present condition of a dynamic system is connected to its condition in prior time intervals. Both HMM and SLDS can address such problems and have been widely used in several problem fields. Examples include human motion [6,7], computer vision [8], speech recognition [9], econometrics [10], machine learning [11] and neuroscience [12,13].

HMM is a discrete Markov process with doubly embedded stochastic models, namely, (i) an unobservable hidden process characterized by a Markov chain and (ii) an observation process determined by hidden states. HMM-based models have garnered significant interest and demonstrated their utility in the signal processing community for accurately simulating the intricate temporal progression of signals. More precisely, HMM have a lengthy track record in signal processing, particularly in the field of speech processing where they have achieved notable success [14]. The SLDS allows for the modeling of nonlinear time series data by dividing it into subsequences controlled by linear dynamics. Indeed, the generative model of SLDS is slightly different from HMM, where there is an additional set of latent variables between the switching states and the observations. In addition, a SLDS differs from a HMM by choosing from a collection of linear Gaussian dynamics that evolve continuously, instead of a standard Gaussian mixture density as in HMM. The SLDS can be viewed as an expansion of the HMM, where each HMM state, or mode, is linked to a linear dynamical process. One further benefit of SLDS is its ability to handle time series data with high dimensionality. From a generative model perspective, it can be claimed that high-dimensional time series can be cheaply represented by a dynamical process specified on a low-dimensional manifold. This representation benefits from the model structure of SLDS. Nevertheless, employing HMM directly on time series with high dimensionality is likely to result in overfitting. [15]. Therefore, SLDS offers the potential for increased descriptive capability [1].

Although SLDS offers advanced predictive models, the dynamics from the data are only stated under certain conditions, which poses some shortcomings of the generative model and may limit its application. First, the continuously evolving linear Gaussian dynamics of latent variables is essentially equivalent to a first-order switching vector autoregressive (VAR) process, i.e., VAR(1) process, which may not be suitable for the case when there are higher-order dependency relationships in the time series. Higher-order autoregressive interactions are more common in time series data [16,17]. Furthermore, determining an appropriate autoregressive order must trade off the model likelihood and complexity, constituting a non-trivial task in the applications [18]. Indeed, a restricted VAR(1) process involved in the linear dynamics of latent variables limits the ability of SLDS to recover the higher-order dependency relationships in dynamical phenomena.

Second, the switching state transition in SLDS follows the Markov assumption, which means that the duration time (i.e., time interval spent in a certain state) is restricted to a geometric distribution [19]. Thus, more weight is given to shorter consecutive time periods within a certain state, which means states switch frequently. This may not be appropriate when the system spends a long time in one state [20]. In addition, the state transitions are based only on the previous state and are unrelated to the latent variables. If a discrete switch happens when the system enters a particular area of the state space, the SLDS cannot recognize this relationship. While hidden semi-Markov model (HSMM) based on the semi-Markov assumption is an extension of the HMM that alleviates this issue, it should be noted that state transitions in HSMM still have no associations with the previous observations [20].

To address the mentioned limitations, an extension of SLDS, the recurrent switching linear dynamical systems with higher-order dependence (HO-rSLDS), is presented. The HO-rSLDS provides a new method that improves the modeling of dynamics from complex systems, by enabling a higher-order dependence in the linear Gaussian dynamics of latent variables and allowing the switching state transition probabilities to hinge on the preceding values of the latent variables. Specifically, to address the first limitation, we enhance the generative process of latent variables by directly making the current latent variable depend on the state values at the previous several time steps, rather than only the previous one time step. Therefore, the proposed model is feasible for capturing the higher-order autoregressive relationships in the data. To address the second limitation, we utilize the stick-breaking logistic regression to determine the switching state transition [21]. In addition, we leverage a Pólya-gamma augmentation approach [22] to improve block inference updating. This enhancement transforms specific logistic regression evidence potentials into conditionally Gaussian potentials, facilitating efficient Bayesian inference procedures. In this way, the transition of switching states is directly associated with the previous switching state and latent variable. Moreover, since the switching states and the latent variables are mutually interdependent in HO-rSLDS, the symmetric dependency among them is recovered, enhancing the generative process of both switching states and latent variables.

Similar to SLDS, the exact inference in an HO-rSLDS is intractable, which impedes efficient model estimation and parameter learning [8]. In this study, the HO-rSLDS is learned through variational inference, which is an approximate scheme to maximize the evidence lower bound, i.e., minimize the Kullback–Leibler divergence between a restricted family of functions of the model parameters and the actual posterior distribution [23]. Moreover, we propose message-passing algorithms about switching states and latent variables in HO-rSLDS to evaluate sufficient statistics for further parameter updates.

The primary contributions of the novel approach suggested in this study are outlined below:

• HO-rSLDS assumes the linear dynamics of latent variables can be described by a higher-order VAR process, which makes it feasible to dig out and evaluate the long-term dependency relationships from dynamical phenomena;
• Stick-breaking logistic regression is applied to determine the switching state transition in HO-rSLDS. By this means, the transition probabilities are time-varying and can be adjusted according to the previous latent variable, which overcomes the limitation of restricted geometric state duration time by Markov assumption and recovers the symmetric dependency between the switching states and the latent variable;
• The Pólya-gamma augmentation strategy permits efficient Bayesian inference algorithms. In addition, we propose message-passing algorithms, including a forward Kalman filter and backward Kalman smoother for HO-rSLDS, which facilitates the parameter update in variational inference.

The rest of this study is organized as follows. In Section 2.3, we discuss the related models. In Section 3, we present the model learning algorithms in detail. In Section 4, we demonstrate the performance of the proposed HO-rSLDS on simulated data. Our method is applied to public functional magnetic resonance imaging (fMRI) data as well, and the results are described in Section 5. Finally, we wrap up our conclusions in Section 6.

2. Methods

2.1. HMM and HSMM

HMMs are mathematical models that analyze dynamic systems by using random processes with hidden states. A HMM is constructed using a Markov chain and is composed of two layers: a layer representing hidden states and a layer representing observations. The observed sequences are allocated to their respective hidden states according to the observation probability distribution. In the context of HMM, the observations are solely influenced by the current state and are not influenced by any preceding states [20]. The entire state sequence can be presented as $z_{1:T}={\left\{z_t\right\}}_{t=1}^T$, which can be viewed as a sequence over a finite set $Ƶ$ with cardinality $\left|Ƶ\right|$. The state transition probability from state i to j is defined as ${\pi }_{ij}=p(z_{t+1}=j|z_t=i)$. The distribution of observations $y_t\in {\mathcal{R}}^n$ given a specific state is denoted by $p\left(y_t\right|z_t,{\theta }_j)$, where ${\theta }_j$ denotes the emission parameters of state j. Above all, the HMM can be described as:

$$\begin{array}{cc}\hfill & z_t|z_{t-1}\sim {\pi }_{z_{t-1}},\hfill \end{array}$$

(1)

$$\begin{array}{cc}\hfill & y_t|z_t\sim F\left({\theta }_{z_t}\right),\hfill \end{array}$$

(2)

where $F(·)$ denotes emission distribution.

Compared to HMM, the HSMM formalism improves upon the standard HMM by incorporating a random state length time, chosen from a state-specific distribution, into the generative process. The state remains constant until the period elapses, at which juncture there is a Markov transition to a fresh state. By using this approach, the distribution of state durations is not limited to a geometric form [24,25]. The random variable $D_t$ represents the length of time that a state is entered at time t. The associated probability mass function is denoted as $p\left(D_t\right|z_t=j)$ and the HSMM can be described as

$${\nu }_s|{\nu }_{s-1}\sim {\pi }_{{\nu }_{s-1}},$$

(3)

$$D_s\sim g\left({\omega }_s\right),$$

(4)

$$z_{t_s^1:t_s^{D_s+1}}={\nu }_s,$$

(5)

$$y_{t_s^1:t_s^{D_s+1}}|z_{t_s^1:t_s^{D_s+1}}\sim F({\theta }_{{\nu }_s},D_s),$$

(6)

where ${\nu }_s$ indexes the state shared by state segment s. A graphical illustration of HMM and HSMM is shown in Figure 1. HMM or HSMM typically uses a Gaussian mixture as the emission distribution, which fails to capture potential dependencies in observed time series. An effective approach to tackle this problem is to employ the VAR emission model, which describes the behavior of time series through linear historical interactions among the observed time series. In this way, the emission model in HMM or HSMM can be specifically expressed as:

$$\begin{array}{c}\hfill y_t|z_t=k\sim \mathcal{N}(\sum _{l=1}^p{W}_{k,l}{y}_{t-l},{\Sigma }_k),\end{array}$$

(7)

where p denotes the maximum lag order, $W_{k,l}$ are $n\times n$ dimensional matrices representing the k-th state autoregressive coefficient matrices for lag l and the covariance matrix is ${\Sigma }_k$. Subsequently, we refer to the H(S)MM with VAR(p) emission model as the AR(p)-H(S)MM.

2.2. SLDS

A SLDS Markov process differs from a hidden Markov model (HMM) by choosing from a collection of linear Gaussian dynamics that evolve continuously, instead of a standard Gaussian mixture density as in HMM. Therefore, SLDS offers the potential for increased descriptive capability.

Similar to HMM, at each time $t=1,2,\dots ,T$ there is a discrete switching state $z_t\in \left\{1,2,\dots ,K\right\}$ that evolves according to Markov dynamics:

$$\begin{array}{c}\hfill z_{t+1}|z_t,{\left\{{\pi }_k\right\}}_{k=1}^K\sim {\pi }_{z_t},\end{array}$$

(8)

${\left\{{\pi }_k\right\}}_{k=1}^K$ constitutes the Markov transition matrix and ${\pi }_k\in {\left[0,1\right]}^K$ is its k-th row. The $z_1$ is assumed to be sampled from an initial distribution ${\pi }_0$ parameterized by prior parameter ${\alpha }_0$, i.e., $z_1|{\alpha }_0\sim {\pi }_0\left({\alpha }_0\right)$. Furthermore, a continuous latent variable $x_t\in {\mathbb{R}}^m$ follows conditionally linear dynamics depend on the state values with at the previous time step, and the switching state $z_t$ determines the linear dynamical system at time t by:

$$\begin{array}{c}\hfill x_{t+1}=A_{z_{t+1}}^{\left(0\right)}{x}_t+v_t,v_t\stackrel{i.i.d}{\sim }\mathcal{N}(0,Q_{z_{t+1}}^{\left(0\right)}),\end{array}$$

(9)

where $A_{z_{t+1}}^{\left(0\right)}\in {\mathbb{R}}^{m\times m}$ denotes the coefficient matrix and $Q_{z_{t+1}}^{\left(0\right)}\in {\mathbb{R}}^{m\times m}$ is covariance matrix. At each time t, a linear Gaussian observation $y_t\in {\mathbb{R}}^n$ is produced from the associated latent continuous variable:

$$\begin{array}{c}\hfill y_t=C^{\left(0\right)}{x}_t+w_t,w_t\stackrel{i.i.d}{\sim }\mathcal{N}(0,S^{\left(0\right)}),\end{array}$$

(10)

where $C^{\left(0\right)}\in {\mathbb{R}}^{n\times m}$, $S^{\left(0\right)}\in {\mathbb{R}}^{n\times n}$. Usually, n is much larger than m, such that the dependence structure of high-dimensional observation data can be evaluated based on the low-dimensional latent variable space. A graphical illustration of SLDS is given in Figure 2.

In classical form, however, there are some shortcomings in the generative model of SLDS, which may limit its application. First, the continuously-evolving linear Gaussian dynamics of $x_t$ is essentially equivalent to a first-order switching vector autoregressive (VAR) process, i.e., the VAR(1) process, which may not be suitable for the case when there are higher-order dependency relationships in $x_t$ or $y_t$. Substitute Equation (9) into Equation (10) and assume ${C^{\left(0\right)}}^{\top }{C}^{\left(0\right)}=I$ and $w_t$ approximately zero, one can immediately obtain the conclusion that SLDS describes the dependency relationships by a VAR(1) process since $y_t=C^{\left(0\right)}{A}_{z_t}^{\left(0\right)}{C^{\left(0\right)}}^{\top }{y}_{t-1}+C^{\left(0\right)}{v}_t$. Second, the state transition of $z_t$ follows the Markov assumption, which means that the duration time is restricted to a geometric distribution [19]. Thus, more weight is given to shorter consecutive time periods within a certain state, which means states switch frequently. This may not be appropriate when the system spends a long time in one state [20]. In addition, the state $z_{t+1}$ transitions based only on the previous state $z_t$, and $z_{t+1}|z_t$ is unrelated to the latent variable $x_t$. If a discrete switch happens when the system enters a particular area of the state space, the SLDS will not be able to recognize this relationship. To address these issues, the proposed HO-rSLDS enhances the generative process of $x_t$ and the switching state transition, which is demonstrated in the following subsection.

2.3. Recurrent Switching Linear Dynamic System with Higher-Order Dependence

In this section, we describe the proposed HO-rSLDS comprising two main components that are switching linear dynamic systems with higher-order dependence (HO-SLDS, without recurrence) and the stick-breaking logistic regression as well as the Pólya-gamma augmentation for HO-SLDS. Each component is demonstrated below.

2.3.1. HO-SLDS

The switching state transition in HO-SLDS is the same as that in SLDS. However, the continuous latent variable $x_t\in {\mathcal{R}}^m$ follows conditionally linear dynamics depending on the state values at the previous p time steps, rather than only the previous one time step, and the switching state $z_t$ determines the linear dynamical system at time t by:

$$\begin{array}{c}\hfill x_{t+1}=A_{z_{t+1}}{\overline{x}}_t+v_t,v_t\stackrel{i.i.d}{\sim }\mathcal{N}(0,Q_{z_{t+1}}),\end{array}$$

(11)

where ${\overline{x}}_t={[x_{t-1}^{\top },\dots ,x_{t-p}^{\top }]}^{\top }\in {\mathcal{R}}^{pm}$, $A_{z_{t+1}}\in {\mathcal{R}}^{m\times pm}$ and $Q_{z_{t+1}}\in {\mathcal{R}}^{m\times m}$. Indeed, the evolution of the latent variable dynamics in $x_t$ can be considered as a $VAR\left(p\right)$ process. The initial ${\overline{x}}_0$ given the initial discrete state $z_0$ is supposed to be normally distributed with mean ${\mu }_0$ and covariance ${\Sigma }_0$, i.e., ${\overline{x}}_0\sim \mathcal{N}({\mu }_0,{\Sigma }_0)$. Similarly, at each time t, a linear Gaussian observation $y_t\in {\mathcal{R}}^n$ is produced from the associated latent continuous variable:

$$\begin{array}{c}\hfill y_t=C_{z_t}{x}_t+w_t,w_t\stackrel{i.i.d}{\sim }\mathcal{N}(0,S_{z_t}),\end{array}$$

(12)

where $C_{z_t}\in {\mathcal{R}}^{n\times m}$, $S_{z_t}\in {\mathcal{R}}^{n\times n}$. Note that here we assume $C_{z_t}$ and $S_{z_t}$ are related to $z_t$. Usually, n is much larger than m, such that the dependence structure of high-dimensional observation data can be evaluated based on the low-dimensional latent variable space. The system parameters consist of the discrete Markov transition matrix and the collection of linear dynamical system matrices, denoted as:

$$\begin{array}{c}\hfill \theta ={\pi }_0\cup {\left\{{\mu }_0,{\Sigma }_0,{\pi }_k,A_k,Q_k,C_k,S_k\right\}}_{k=1}^K.\end{array}$$

(13)

2.3.2. Stick-Breaking Logistic Regression and Pólya-Gamma Augmentation for HO-SLDS

Another component included in HO-rSLDS is a stick-breaking logistic regression. We utilize a Pólya-gamma augmentation technique to improve block inference updates [21]. This approach allows certain logistic regression evidence potentials to transform to conditionally Gaussian potentials in an augmented distribution, which facilitates the Bayesian inference algorithms.

Consider a logistic regression model with regressors $x\in {\mathcal{R}}^m$ that maps to a categorical distribution over the switching state $z\in \left\{1,2,\dots ,K\right\}$, denoted as:

$$\begin{array}{c}\hfill z|x\sim {\pi }_{SB}\left(\nu \right),\nu =Rx+r,\end{array}$$

(14)

where $R\in {\mathcal{R}}^{K-1\times M}$ represents a weight matrix and $r\in {\mathcal{R}}^{K-1}$ denotes a bias vector. We use a stick-breaking link function ${\pi }_{SB}:{\mathcal{R}}^{K-1}\to {[0,1]}^K$, which transforms a real vector into a normalized probability vector based on the stick-breaking process:

$$\begin{array}{cc}\hfill & {\pi }_{SB}\left(\nu \right)=\left({\pi }_{SB}{\left(\nu \right)}^{\left(1\right)},\dots ,{\pi }_{SB}{\left(\nu \right)}^{\left(K\right)}\right),\hfill \end{array}$$

(15)

$$\begin{array}{cc}\hfill & {\pi }_{SB}^{\left(k\right)}=\sigma \left({\nu }_k\right)\prod _{j<K}\sigma (-{\nu }_j),\hfill \end{array}$$

(16)

for $k=1,\cdots ,K-1$ and ${\pi }_{SB}^{\left(K\right)}={\prod }_{k=1}^{K-1}\sigma (-{\nu }_k)$ where ${\nu }_k$ represents the kth component of $\nu$ and $\sigma \left(\nu \right)={(1+e^{-\nu })}^{-1}$ is the logistic function. The probability mass function $p\left(z\right|x)$ is given by

$$\begin{array}{c}\hfill p\left(z\right|x)=\prod _{k=1}^{K-1}\sigma {\left({\nu }_k\right)}^{\mathbb{I}(z=k)}\sigma {(-{\nu }_k)}^{\mathbb{I}(z>k)},\end{array}$$

(17)

where $\mathbb{I}(·)$ is the indicator function.

The posterior density $p\left(x\right|z)$ is non-Gaussian and does not allow straightforward Bayesian inference for this regression model due to the likelihood $p\left(z\right|x)$ not aligning with a Gaussian prior density $p\left(x\right)$. To solve this problem, we introduce Pólya-gamma auxiliary variables $\omega ={\left\{{\omega }_k\right\}}_{k=1}^K$ so that the conditional density $p\left(x\right|z,\omega )$ becomes Gaussian [22]. Specifically, the conditional density of ${\omega }_k$ is distributed according to a Pólya-Gamma distribution. A random variable X has a Pólya-Gamma distribution with parameters $b>0$ and $c\in \mathcal{R}$, denoted as $X\sim PG(b,c)$, if

$$\begin{array}{c}\hfill X\stackrel{D}{=}\frac{1}{2{\pi }^2}\sum _{k=1}^{\infty }\frac{g_k}{{(k-1/2)}^2+c^2/\left(4{\pi }^2\right)},\end{array}$$

(18)

where the $g_k\sim Gamma(b,1)$ are independent Gamma random variables, and where $\stackrel{D}{=}$ indicates equality in distribution. In particular, by choosing ${\omega }_k|x,z\sim PG(\mathbb{I}(z>k),{\nu }_k)$, we have the following:

$$\begin{array}{c}\hfill x|z,\omega \sim \mathcal{N}({\Omega }^{-1}\kappa ,{\Omega }^{-1}),\end{array}$$

(19)

where the mean vector ${\Omega }^{-1}\kappa$ and covariance matrix ${\Omega }^{-1}$ are calculated based on the following:

$$\begin{array}{cc}\hfill & \Omega =diag\left(\omega \right),\hfill \end{array}$$

(20)

$$\begin{array}{cc}\hfill & {\kappa }_k=\mathbb{I}(z=k)-\frac{1}{2}\mathbb{I}(z>k).\hfill \end{array}$$

(21)

Thus allowing efficient block updates while maintaining Gaussian posterior distribution $p\left(x\right|z)$ in Bayesian inference.

The HO-rSLDS splits the latent space into K sections, with each section following its own linear dynamics by including the recurrence (Equation (17)) in the transition density of $z_t$, which can be described as follows:

$$\begin{array}{cc}\hfill & z_{t+1}|z_t,x_t,R_{z_t},r_{z_t}\sim {\pi }_{SB}\left({\nu }_t\right),\hfill \end{array}$$

(22)

$$\begin{array}{cc}\hfill & {\nu }_t=R_{z_t}{x}_t+r_{z_t}.\hfill \end{array}$$

(23)

Moreover, we place Gaussian priors on $R_{z_t}$ and $r_{z_t}$. In this way, the probability mass function $p\left(z_{t+1}\right|x_t)$ can be expressed as:

$$\begin{array}{cc}\hfill p\left(z_{t+1}\right|x_t)& =\prod _{k=1}^{K-1}\sigma {\left({\nu }_k\right)}^{\mathbb{I}(z_{t+1}=k)}\sigma {(-{\nu }_k)}^{\mathbb{I}(z_{t+1}>k)}\hfill \\ \hfill & =\prod _{k=1}^{K-1}\frac{{\left(e^{{\nu }_{t,k}}\right)}^{\mathbb{I}(z_{t+1}=k)}}{{(1+e^{{\nu }_{t,k}})}^{\mathbb{I}(z_{t+1}>k)}}.\hfill \end{array}$$

(24)

The Pólya-gamma augmentation focuses on exactly the above densities, utilizing the following integral identity:

$$\begin{array}{c}\hfill \frac{{\left(e^{\nu }\right)}^a}{{(1+e^{\nu })}^b}=2^{-b}{e}^{\kappa \nu }{\int }_0^{\infty }{e}^{-\omega {\nu }^2}{p}_{PG}\left(\omega \right|b,0)d\omega ,\end{array}$$

(25)

where $\kappa =a-b/2$ and $p_{PG}\left(\omega \right|b,0)$ refers to the density of the Pólya-gamma distribution, $PG(b,0)$, which is independent of $\nu$ [22]. Combing Equations (24) and (25), $p\left(z_{t+1}\right|x_t)$ can be expressed as the marginalization of a component in the probability distribution $p\left(z_{t+1}\right|x_t,{\omega }_t)$, where ${\omega }_t\in {\mathbb{R}}^{K-1}$ represents a vector of auxiliary variables. Dependent on ${\nu }_t$, there is

$$\begin{array}{c}\hfill p(z_{t+1},x_t,{\omega }_t)\propto \prod _{k=1}^{K-1}exp\left\{-\frac{1}{2}{\omega }_{t,k}{\nu }_{t,k}^2+{\kappa }_{t+1,k}{\nu }_{t,k}\right\},\end{array}$$

(26)

where ${\kappa }_{t+1,k}=\mathbb{I}(z_{t+1}=k)-\frac{1}{2}\mathbb{I}(z_{t+1}>k)$. Hence,

$$\begin{array}{c}\hfill p(z_{t+1},x_t,{\omega }_t)\propto \mathcal{N}\left({\nu }_t\right|{\Omega }_t^{-1}{\kappa }_{t+1},{\Omega }_t^{-1}),\end{array}$$

(27)

where ${\Omega }_t=diag\left({\omega }_t\right)$ and ${\kappa }_{t+1}=({\kappa }_{t+1,1},\dots ,{\kappa }_{t+1,K-1})$. After augmentation, the conditional density on $x_t$ becomes virtually Gaussian, simplifying the evaluation of the integrals needed for message passing during the Bayesian inference. Finally, we summarize the proposed HO-rSLDS as

$$\begin{array}{cc}\hfill & z_1|\theta \sim {\pi }_0\left({\alpha }_0\right),\hfill \\ \hfill & {\overline{x}}_0|\theta \sim \mathcal{N}\left({\overline{x}}_0\right|{\mu }_0,{\Sigma }_0),\hfill \\ \hfill & z_{t+1}|z_t,x_t,\theta \sim {\pi }_{SB}\left({\nu }_t\right),{\nu }_t=R_{z_t}{x}_t+r_{z_t},\hfill \\ \hfill & x_{t+1}|{\overline{x}}_t,z_{t+1},\theta \sim \mathcal{N}\left(x_{t+1}\right|A_{z_{t+1}}{\overline{x}}_t,Q_{z_{t+1}}),\hfill \\ \hfill & y_t|x_t,z_t,\theta \sim \mathcal{N}\left(y_t\right|C_{z_t}{x}_t,S_{z_t}),\hfill \\ \hfill & \theta =\left\{{\alpha }_0,{\mu }_0,{\Sigma }_0\right\}\cup {\left\{A_k,Q_k,C_k,S_k,R_k,r_k\right\}}_{k=1}^K.\hfill \end{array}$$

(28)

To learn the HO-rSLDS utilizing Bayesian inference, we assign conjugate priors to each component of the model parameters $\theta$, given by

$$\begin{array}{cc}\hfill & {\pi }_0|{\alpha }_0\sim Dir\left({\alpha }_0\right),\hfill \\ \hfill & (A_k,Q_k)|\eta \sim MNIW\left(\eta \right),\hfill \\ \hfill & (C_k,S_k)|\lambda \sim MNIW\left(\lambda \right),\hfill \\ \hfill & R_k|\zeta \sim \prod _{i=1}^{K-1}\mathcal{N}\left(R_{k,i}\right|{\zeta }_i),\hfill \\ \hfill & r_k|\rho \sim \mathcal{N}\left(r_k\right|\rho ),\hfill \end{array}$$

(29)

where Dir and MNIW denote Dirichelet and matrix normal inverse Wishart distribution, respectively, $R_{k,i}$ denotes the ith row of $R_k$. A graphical illustration of HO-rSLDS is given in Figure 3.

3. Variational Inference of HO-rSLDS

Now we describe the model learning inference of HO-rSLDS, which is fulfilled by variational inference. In variational inference, a parameter is inferred by maximizing the evidence’s lower bound, hence minimizing the Kullback–Leibler divergence as well as the variational free energy. Specifically, we denote $\mathcal{Y}:=y_{1:T}$, $\Theta :=\left\{z_{1:T},({\overline{x}}_0,x_{1:T}),{\omega }_{1:T},\theta \right\}$, then the following decomposition holds:

$$\begin{array}{c}\hfill lnp\left(\mathcal{Y}\right)=\mathcal{L}\left(q(\Theta )\right)+KL\left(q(\Theta )\left|\right|p(\Theta |\mathcal{Y})\right),\end{array}$$

(30)

where

$$\begin{array}{cc}\hfill & \mathcal{L}\left(q(\Theta )\right)=\int q(\Theta )log\frac{p(\mathcal{Y},\Theta )}{q(\Theta )}d\Theta ,\hfill \end{array}$$

(31)

$$\begin{array}{cc}\hfill & KL\left(q(\Theta )\left|\right|p(\Theta |\mathcal{Y})\right)=-\int q(\Theta )log\frac{p(\Theta |\mathcal{Y})}{q(\Theta )}d\Theta .\hfill \end{array}$$

(32)

The exact evaluation of $p(z_{1:T},x_{0:T},{\omega }_{1:T}|y_{1:T})$ where $x_{0:T}:={\overline{x}}_0\cup x_{1:T}$ in HO-rSLDS is intractable. In this study, we leverage an approximate variational inference method to assess the approximate posterior $q(z_{1:T},x_{0:T},{\omega }_{1:T}|y_{1:T})$ in the proposed HO-rSLDS. In order to obtain a tractable inference, a structured mean field approximation is adopted on the generative model, given by

$$\begin{array}{c}\hfill q(z_{1:T},x_{0:T},{\omega }_{1:T},\theta |y_{1:T})=q\left(z_{1:T}\right)q\left(x_{0:T}\right)q\left({\omega }_{1:T}\right)q\left(\theta \right).\end{array}$$

(33)

Moreover, $q\left({\omega }_{1:T}\right)$ further factorizes as:

$$\begin{array}{c}\hfill q\left({\omega }_{1:T}\right)=\prod _{t=1}^T\prod _{k=1}^{K-1}q\left({\omega }_{t,k}\right).\end{array}$$

(34)

Given the factorized forms in Equations (33) and (34), we can obtain the optimal solution of the factorized variational posteriors by maximizing the evidence lower bound in Equation (31), i.e., $q\left(z_{1:T}\right),q\left(x_{0:T}\right),q\left({\omega }_{1:T}\right)$ and $q\left(\theta \right)$, given by

$$\begin{array}{cc}\hfill & lnq\left(z_{1:T}\right)\propto {\mathbb{E}}_{-q\left(z_{1:T}\right)}lnp(z_{1:T},x_{0:T},{\omega }_{1:T},\theta ,y_{1:T}),\hfill \end{array}$$

(35)

$$\begin{array}{cc}\hfill & lnq\left(x_{0:T}\right)\propto {\mathbb{E}}_{-q\left(x_{0:T}\right)}lnp(z_{1:T},x_{0:T},{\omega }_{1:T},\theta ,y_{1:T}),\hfill \end{array}$$

(36)

$$\begin{array}{cc}\hfill & lnq\left({\omega }_{1:T}\right)\propto {\mathbb{E}}_{-q\left({\omega }_{1:T}\right)}lnp(z_{1:T},x_{0:T},{\omega }_{1:T},\theta ,y_{1:T}),\hfill \end{array}$$

(37)

$$\begin{array}{cc}\hfill & lnq\left(\theta \right)\propto {\mathbb{E}}_{-q\left(\theta \right)}lnp(z_{1:T},x_{0:T},{\omega }_{1:T},\theta ,y_{1:T}),\hfill \end{array}$$

(38)

where ${\mathbb{E}}_{-q\left(z_{1:T}\right)}(·):={\mathbb{E}}_{q\left(x_{0:T}\right)q\left({\omega }_{1:T}\right)q\left(\theta \right)}(·)$. Hence the joint log-likelihood is

$$\begin{array}{c}\hfill lnp(z_{1:T},x_{0:T},{\omega }_{1:T},\theta ,y_{1:T})=lnp(z_{1:T},x_{0:T},{\omega }_{1:T},y_{1:T}|\theta )+lnp\left(\theta \right).\end{array}$$

(39)

The conditional log-likelihood $lnp(z_{1:T},x_{0:T},{\omega }_{1:T},y_{1:T}|\theta )$ can be written as

$$\begin{array}{cc}\hfill & lnp(z_{1:T},x_{0:T},{\omega }_{1:T},y_{1:T}|\theta )=\hfill \\ \hfill & lnp\left(z_1\right|\theta )+lnp\left({\overline{x}}_0\right|\theta )+\sum _{t=1}^T\left[lnp\left(z_t\right|x_{t-1},z_{t-1},{\omega }_{t-1},\theta )\mathbb{I}(t>1)+ln\left(x_t\right|{\overline{x}}_{t-1},z_t,\theta )\right]\hfill \\ \hfill & +\sum _{t=1}^{T}p\left(y_t\right|z_t,x_t,\theta )\hfill \end{array}$$

(40)

$$\begin{array}{cc}\hfill & \propto lnp\left(z_1\right|\theta )-\frac{1}{2}{({\overline{x}}_0-{\mu }_0)}^{\top }{\left({\Sigma }_0\right)}^{-1}({\overline{x}}_0-{\mu }_0)\hfill \\ \hfill & \phantom{\propto }-\frac{1}{2}\sum _{t=1}^T[{({\nu }_{t-1}-{\Omega }_{t-1}^{-1}{\kappa }_t)}^{\top }{\Omega }_{t-1}({\nu }_{t-1}-{\Omega }_{t-1}^{-1}{\kappa }_t)\mathbb{I}(t>1)\hfill \\ \hfill & \phantom{-\frac{1}{2}\sum _{t=1}^T[aaa}+{(x_t-A_{z_t}{\overline{x}}_{t-1})}^{\top }{Q}_{z_t}^{-1}(x_t-A_{z_t}{\overline{x}}_{t-1})]\hfill \\ \hfill & \phantom{\propto }-\frac{1}{2}\sum _{t=1}^T{(y_t-C_{z_t}{x}_t)}^{\top }{S}_{z_t}^{-1}(y_t-C_{z_t}{x}_t).\hfill \end{array}$$

(41)

The exact optimal solutions for $q\left(z_{1:T}\right)$ and $q\left(x_{0:T}\right)$ will not be explicitly evaluated; rather, we use message passing algorithms to obtain the sufficient statistics with respect to these factors. In the following subsections, we present the involved message passing algorithms and the detailed derivations of update formula for each factor.

3.1. Update for $q\left(x_{0:T}\right)$

Since ${\mathbb{E}}_{-q\left(x_{0:T}\right)}lnp\left(\theta \right)$ is not a function of $x_{0:T}$, Equation (36) can be written as

$$\begin{array}{cc}\hfill lnq\left(x_{0:T}\right)& \propto {\mathbb{E}}_{-q\left(x_{0:T}\right)}lnp(z_{1:T},x_{0:T},{\omega }_{1:T},y_{1:T}|\theta ),\hfill \\ \hfill & \propto {\mathbb{E}}_{-q\left(x_{0:T}\right)}\left[-\frac{1}{2}{({\overline{x}}_0-{\mu }_0)}^{\top }{\left({\Sigma }_0\right)}^{-1}({\overline{x}}_0-{\mu }^0)\right]\hfill \\ \hfill & \phantom{\propto }+{\mathbb{E}}_{-q\left(x_{0:T}\right)}\left[-\frac{1}{2}\sum _{t=2}^T{({\nu }_{t-1}-{\Omega }_{t-1}^{-1}{\kappa }_t)}^{\top }{\Omega }_{t-1}({\nu }_{t-1}-{\Omega }_{t-1}^{-1}{\kappa }_t)\right]\hfill \\ \hfill & \phantom{\propto }+{\mathbb{E}}_{-q\left(x_{0:T}\right)}\left[-\frac{1}{2}\sum _{t=1}^T{(x_t-A_{z_t}{\overline{x}}_{t-1})}^{\top }{Q}_{z_t}^{-1}(x_t-A_{z_t}{\overline{x}}_{t-1})\right]\hfill \\ \hfill & \phantom{\propto }+{\mathbb{E}}_{-q\left(x_{0:T}\right)}\left[-\frac{1}{2}\sum _{t=1}^T{(y_t-C_{z_t}{x}_t)}^{\top }{S}_{z_t}^{-1}(y_t-C_{z_t}{x}_t)\right].\hfill \end{array}$$

(42)

From Equation (42) it can be observed that the structure of the variational posterior $lnq\left(x_{0:T}\right)$ resembles the joint log-likelihood function of a time-varying linear dynamical system with higher-order dependence (HO-LDS). Thus, the $lnq\left(x_{0:T}\right)$ can be re-expressed as

$$\begin{array}{cc}\hfill lnq\left(x_{0:T}\right)& \propto -\frac{1}{2}{({\overline{x}}_0-{\widehat{\mu }}_0)}^{\top }{\left({\widehat{\Sigma }}_0\right)}^{-1}({\overline{x}}_0-{\widehat{\mu }}_0)\hfill \\ \hfill & \phantom{\propto }-\frac{1}{2}\sum _{t=1}^T{(x_t-{\widehat{A}}_t{\overline{x}}_{t-1})}^{\top }{\widehat{Q}}_t^{-1}(x_t-{\widehat{A}}_t{\overline{x}}_{t-1})\hfill \\ \hfill & \phantom{\propto }-\frac{1}{2}\sum _{t=1}^T{(y_t-{\widehat{C}}_t{x}_t)}^{\top }{\widehat{S}}_t^{-1}(y_t-{\widehat{C}}_t{x}_t),\hfill \end{array}$$

(43)

and the set of variational parameters involved in Equation (43) are defined as

$$\begin{array}{c}\hfill {\widehat{\theta }}_{x_{0:T}}:=\left\{{\widehat{\mu }}_0,{\widehat{\Sigma }}_0,{\left\{{\widehat{A}}_t\right\}}_{t=1}^T,{\left\{{\widehat{Q}}_t\right\}}_{t=1}^T,{\left\{{\widehat{C}}_t\right\}}_{t=1}^T,{\left\{{\widehat{S}}_t\right\}}_{t=1}^T\right\}.\end{array}$$

(44)

Each component in the ${\widehat{\theta }}_{x_{0:T}}$ can be precisely determined by matching the coefficients of each term in Equations (42) and (43). Algorithm 1 outlines an efficient scheme for this purpose.

Algorithm 1: Obtaining ${\widehat{\theta }}_{x_{0:T}}$

Remark 1. 

$A_{z_t}{\overline{x}}_{t-1}={\sum }_{j=1}^p{A}_{z_t}\left(j\right)x_{t-j}$ where $A_{z_t}\left(j\right)$ is defined as the submatrix of $A_{z_t}$ associated with $x_{t-j}$ in $A_{z_t}{\overline{x}}_{t-1}$.

Thus, we perform message passing on a time-varing HO-LDS parameterized by ${\widehat{\theta }}_{x_{0:T}}$ to obtain the posterior distribution $q\left(x_{0:T}\right)$ and sufficient statistics used in the update for the other factors. To be concrete, the message passing comprises the generalized Kalman filter and Kalman smoother for the time-varing HO-LDS, which are demonstrated by Theorems 1 and 2. To prove the theorems, we need the following lemmas [26]:

Lemma 1. 

If random variables X and Y follow the Gaussian probability distributions:

$$\begin{array}{cc}\hfill & X\sim \mathcal{N}(m,P),\hfill \end{array}$$

(45)

$$\begin{array}{cc}\hfill & Y|X\sim \mathcal{N}(HX+u,R),\hfill \end{array}$$

(46)

then the joint distribution of X and Y, as well as the marginal distribution of Y, are provided by

$$\begin{array}{cc}\hfill & \left(\begin{array}{c}X\\ Y\end{array}\right)\sim \mathcal{N}\left(\left(\begin{array}{c}m\\ Hm+u\end{array}\right),\left(\begin{array}{cc}P& P{H}^{\top }\\ HP& HP{H}^{\top }+R\end{array}\right)\right),\hfill \\ \hfill & Y\sim \mathcal{N}\left(Hm+u,HP{H}^{\top }+R\right).\hfill \end{array}$$

(47)

Lemma 2. 

If the random variables X and Y follow the joint Gaussian probability distribution

$$\begin{array}{c}\hfill \left(\begin{array}{c}X\\ Y\end{array}\right)\sim \mathcal{N}\left(\left(\begin{array}{c}a\\ b\end{array}\right),\left(\begin{array}{cc}A& C\\ C^{\top }& B\end{array}\right)\right),\end{array}$$

(48)

then the marginal and conditional distributions of X and Y are as follows:

$$\begin{array}{cc}\hfill & X\sim \mathcal{N}(a,A),\hfill \\ \hfill & Y\sim \mathcal{N}(b,B),\hfill \\ \hfill & X|Y\sim \mathcal{N}\left(a+C{B}^{-1}(Y-b),A-C{B}^{-1}{C}^{\top }\right),\hfill \\ \hfill & Y|X\sim \mathcal{N}\left(b+C^{\top }{A}^{-1}(X-a),B-C^{\top }{A}^{-1}C\right).\hfill \end{array}$$

(49)

Now we demonstrate Theorems 1 and 2 in turn, which provides the Kalman filter and smoother equations about $q\left(x_{0:T}\right)$ in the time-varying HO-LDS.

Theorem 1. 

The Kalman filter equations for the time-varing HO-LDS (Equation (43)) parameterized by ${\widehat{\theta }}_{x_{0:T}}$ can be evaluated in closed form, which results in Gaussian distributions:

$$\begin{array}{cc}\hfill & p\left(x_t\right|y_{1:t-1})=\mathcal{N}\left(x_t\right|{\mu }_t^{\left(p\right)},{\Sigma }_t^{\left(p\right)}),\hfill \end{array}$$

(50)

$$\begin{array}{cc}\hfill & p\left(x_t\right|y_{1:t})=\mathcal{N}\left(x_t\right|{\mu }_t^{\left(u\right)},{\Sigma }_t^{\left(u\right)}),\hfill \end{array}$$

(51)

$$\begin{array}{cc}\hfill & p\left({\overline{x}}_t\right|y_{1:t})=\mathcal{N}\left({\overline{x}}_t\right|{\mu }_t^{\left(m\right)},{\Sigma }_t^{\left(m\right)}).\hfill \end{array}$$

(52)

The distribution parameters can be calculated using the following Kalman filter prediction, update and merging steps detailed in the following proof. The recursion begins with the prior mean ${\mu }_0^m$ and covariance ${\Sigma }_0^m$ (i.e., ${\widehat{\mu }}_0$ and ${\widehat{\Sigma }}_0$).

Proof. 

The Gaussian filter distributions (Equations (50)–(52)) can be obtained by the following steps:

1. Prediction step. By Lemma 1, the joint distribution of $x_t$ and ${\overline{x}}_{t-1}$ given $y_{1:t-1}$ is

$$\begin{array}{cc}\hfill p(x_t,{\overline{x}}_{t-1}|y_{1:t-1})& =p\left(x_t\right|{\overline{x}}_{t-1})p\left({\overline{x}}_{t-1}\right|y_{1:t-1})\hfill \\ \hfill & =\mathcal{N}\left(x_t\right|{\widehat{A}}_t{\overline{x}}_{t-1},{\widehat{Q}}_t)\mathcal{N}\left({\overline{x}}_{t-1}\right|{\mu }_{t-1}^{\left(m\right)},{\Sigma }_{t-1}^{\left(m\right)})\hfill \\ \hfill & =\mathcal{N}\left(\left.\left(\begin{array}{c}{\overline{x}}_{t-1}\\ x_t\end{array}\right)\right|{\mu }^{\left(1\right)},{\Sigma }^{\left(1\right)}\right),\hfill \end{array}$$

(53)

where

$$\begin{array}{c}\hfill {\mu }^{\left(1\right)}=\left(\begin{array}{c}{\mu }_{t-1}^{\left(m\right)}\\ {\widehat{A}}_t{\mu }_{t-1}^{\left(m\right)}\end{array}\right),{\Sigma }^{\left(1\right)}=\left(\begin{array}{cc}{\Sigma }_{t-1}^{\left(m\right)}& {\Sigma }_{t-1}^{\left(m\right)}{\widehat{A}}_t^{\top }\\ {\widehat{A}}_t{\Sigma }_{t-1}^{\left(m\right)}& {\widehat{A}}_t{\Sigma }_{t-1}^{\left(m\right)}{\widehat{A}}_t^{\top }+{\widehat{Q}}_t\end{array}\right),\end{array}$$

(54)

and by Lemma 2 the marginal distribution of $x_t$ is given by

$$\begin{array}{c}\hfill p\left(x_t\right|y_{1:t-1})=\mathcal{N}\left(x_t\right|{\mu }_t^{\left(p\right)},{\Sigma }_t^{\left(p\right)}),\end{array}$$

(55)

where

$$\begin{array}{c}\hfill {\mu }_t^{\left(p\right)}={\widehat{A}}_t{\mu }_{t-1}^{\left(m\right)},{\Sigma }_t^{\left(p\right)}={\widehat{A}}_t{\Sigma }_{t-1}^{\left(m\right)}{\widehat{A}}_t^{\top }+{\widehat{Q}}_t.\end{array}$$

(56)

2. Update step. By Lemma 1, the joint distribution of $y_t$ and $x_t$ given $y_{1:t-1}$ is

$$\begin{array}{cc}\hfill p(x_t,y_t|y_{1:t-1})& =p\left(y_t\right|x_t)p\left(x_t\right|y_{1:t-1})\hfill \\ \hfill & =\mathcal{N}\left(y_t\right|{\widehat{C}}_t{x}_t,{\widehat{S}}_t)\mathcal{N}\left(x_t\right|{\mu }_t^{\left(p\right)},{\Sigma }_t^{\left(p\right)})\hfill \\ \hfill & =\mathcal{N}\left(\left.\left(\begin{array}{c}{x}_t\\ y_t\end{array}\right)\right|{\mu }^{\left(2\right)},{\Sigma }^{\left(2\right)}\right),\hfill \end{array}$$

(57)

where

$$\begin{array}{c}\hfill {\mu }^{\left(2\right)}=\left(\begin{array}{c}{\mu }_t^{\left(p\right)}\\ {\widehat{C}}_t{\mu }_t^{\left(p\right)}\end{array}\right),{\Sigma }^{\left(2\right)}=\left(\begin{array}{cc}{\Sigma }_t^{\left(p\right)}& {\Sigma }_t^{\left(p\right)}{\widehat{C}}_t^{\top }\\ {\widehat{C}}_t{\Sigma }_t^{\left(p\right)}& {\widehat{C}}_t{\Sigma }_t^p{\widehat{C}}_t^{\top }+{\widehat{S}}_t\end{array}\right).\end{array}$$

(58)

By Lemma 2 the conditional distribution of $x_t$ given $y_{1:t}$ is

$$\begin{array}{cc}\hfill p\left(x_t\right|y_t,y_{1:t-1})& =p\left(x_t\right|y_{1:t})\hfill \\ \hfill & =\mathcal{N}\left(x_t\right|{\mu }^{\left(u\right)},{\Sigma }^{\left(u\right)}),\hfill \end{array}$$

(59)

where

$$\begin{array}{cc}\hfill & {\mu }^{\left(u\right)}={\mu }_t^{\left(p\right)}+{\Sigma }_t^{\left(p\right)}{\widehat{C}}_t^{\top }{({\widehat{C}}_t{\Sigma }_t^{\left(p\right)}{\widehat{C}}_t^{\top }+{\widehat{S}}_t)}^{-1}(y_t-{\widehat{C}}_t{\mu }_t^{\left(p\right)}),\hfill \end{array}$$

(60)

$$\begin{array}{cc}\hfill & {\Sigma }^{\left(u\right)}={\Sigma }_t^{\left(p\right)}-{\Sigma }_t^{\left(p\right)}{\widehat{C}}_t^{\top }{({\widehat{C}}_t{\Sigma }_t^{\left(p\right)}{\widehat{C}}_t^{\top }+{\overline{S}}_t)}^{-1}{\widehat{C}}_t{\Sigma }_t^{\left(p\right)}.\hfill \end{array}$$

(61)

3. Merging step. By Lemma 1, the joint distribution of ${\overline{x}}_{t-1},x_t,y_t$ given $y_{1:t-1}$ is

$$\begin{array}{cc}\hfill p({\overline{x}}_{t-1},x_t,y_t|y_{1:t-1})& =p\left(y_t\right|x_t)p(x_t,{\overline{x}}_{t-1}|y_{1:t-1})\hfill \\ \hfill & =\mathcal{N}({\widehat{C}}_t{x}_t,{\widehat{S}}_t)\mathcal{N}\left(\left.\left(\begin{array}{c}{\overline{x}}_{t-1}\\ x_t\end{array}\right)\right|{\mu }^{\left(1\right)},{\Sigma }^{\left(1\right)}\right),\hfill \\ \hfill & =\mathcal{N}\left(\left.\left(\begin{array}{c}{\overline{x}}_{t-1}\\ x_t\\ y_t\end{array}\right)\right|{\mu }^{\left(3\right)},{\Sigma }^{\left(3\right)}\right),\hfill \end{array}$$

(62)

where

$$\begin{array}{cc}\hfill & {\mu }^{\left(3\right)}=\left(\begin{array}{c}{\mu }_1^{\left(3\right)}\\ {\mu }_2^{\left(3\right)}\\ {\mu }_3^{\left(3\right)}\end{array}\right)=\left(\begin{array}{c}{\mu }_{t-1}^{\left(m\right)}\\ {\widehat{A}}_t{\mu }_{t-1}^{\left(m\right)}\\ {\widehat{C}}_t{\widehat{A}}_t{\mu }_{t-1}^{\left(m\right)}\end{array}\right),\hfill \end{array}$$

(63)

$$\begin{array}{cc}\hfill {\Sigma }^{\left(3\right)}& =\left(\begin{array}{ccc}{\Sigma }_{11}^{\left(3\right)}& {\Sigma }_{12}^{\left(3\right)}& {\Sigma }_{13}^{\left(3\right)}\\ {\Sigma }_{21}^{\left(3\right)}& {\Sigma }_{21}^{\left(3\right)}& {\Sigma }_{23}^{\left(3\right)}\\ {\Sigma }_{31}^{\left(3\right)}& {\Sigma }_{32}^{\left(3\right)}& {\Sigma }_{33}^{\left(3\right)}\end{array}\right),\hfill \\ \hfill & =\left(\begin{array}{ccc}{\Sigma }_{t-1}^{\left(m\right)}& {\Sigma }_{t-1}^{\left(m\right)}{\widehat{A}}_t^{\top }& {\Sigma }_{t-1}^{\left(m\right)}{\widehat{A}}_t^{\top }{\widehat{C}}_t^{\top }\\ {\widehat{A}}_t{\Sigma }_{t-1}^{\left(m\right)}& {\widehat{A}}_t{\Sigma }_{t-1}^{\left(m\right)}{\widehat{A}}_t^{\top }+{\widehat{Q}}_t& ({\widehat{A}}_t{\Sigma }_{t-1}^{\left(m\right)}{\widehat{A}}_t^{\top }+{\widehat{Q}}_t){\widehat{C}}_t^{\top }\\ {\widehat{C}}_t{\widehat{A}}_t{\sigma }_{t-1}^{\left(m\right)}& {\widehat{C}}_t({\widehat{A}}_t{\Sigma }_{t-1}^{\left(m\right)}{\widehat{A}}^{\top }+{\widehat{Q}}_t)& {\widehat{C}}_t({\widehat{A}}_t{\Sigma }_{t-1}^{\left(m\right)}{\widehat{A}}_t^{\top }+{\widehat{Q}}_t){\widehat{C}}_t^{\top }+{\widehat{S}}_t\end{array}\right).\hfill \end{array}$$

(64)

By Lemma 2, the conditional distribution of ${\overline{x}}_{t-1}$ and $x_t$ given $y_{1:t}$ is

$$\begin{array}{cc}\hfill p({\overline{x}}_{t-1},x_t|y_t,y_{1:t-1})& =p({\overline{x}}_{t-1},x_t|y_{1:t})\hfill \\ \hfill & =\mathcal{N}({\overline{x}}_{t-1},x_t|{\mu }^{\left(4\right)},{\Sigma }^{\left(4\right)}),\hfill \end{array}$$

(65)

where

$$\begin{array}{cc}\hfill & {\mu }^{\left(4\right)}=\left(\begin{array}{c}{\mu }_1^{\left(3\right)}\\ {\mu }_2^{\left(3\right)}\end{array}\right)+\left(\begin{array}{c}{\Sigma }_{13}^{\left(3\right)}\\ {\Sigma }_{23}^{\left(3\right)}\end{array}\right){\left({\Sigma }_{33}^{\left(3\right)}\right)}^{-1}(y_t-{\mu }_3^{\left(3\right)}),\hfill \end{array}$$

(66)

$$\begin{array}{cc}\hfill & {\Sigma }^{\left(4\right)}={\Sigma }_{11}^{\left(3\right)}-\left(\begin{array}{c}{\Sigma }_{13}^{\left(3\right)}\\ {\Sigma }_{23}^{\left(3\right)}\end{array}\right){\left({\Sigma }_{33}^{\left(3\right)}\right)}^{-1}\left(\begin{array}{cc}{\Sigma }_{13}^{\left(3\right)}& {\Sigma }_{23}^{\left(3\right)}\end{array}\right).\hfill \end{array}$$

(67)

Finally, the marginal distribution of ${\overline{x}}_t$ is

$$\begin{array}{cc}\hfill p\left({\overline{x}}_t\right|y_{1:t})& =\mathcal{N}\left({\overline{x}}_t\right|{\mu }_t^{\left(m\right)},{\Sigma }_t^{\left(m\right)})\hfill \\ \hfill & =\mathcal{N}\left({\overline{x}}_t\right|{\mu }^{\left(4\right)}\left({\overline{x}}_t\right),{\Sigma }^{\left(4\right)}\left({\overline{x}}_t\right)),\hfill \end{array}$$

(68)

where ${\mu }^{\left(4\right)}\left({\overline{x}}_t\right)$ and ${\Sigma }^{\left(4\right)}\left({\overline{x}}_t\right)$ are marginal mean vector and covariance matrix of ${\overline{x}}_t$.    □

The Kalman filter for the time-varying HO-LDS in Theorem 1 calculates the estimates based on measurements collected up to and including time step t. After obtaining the filtering posterior state distributions, Theorem 2 outlines the results for calculating the marginal posterior distributions for each time step based on all data up to the final time step T, which is described as the following:

Theorem 2. 

The Kalman smoother equations for the time-varing HO-LDS (Equation (43)) parameterized by ${\widehat{\theta }}_{x_{0:T}}$ can be evaluated in closed form, which results Gaussian distributions:

$$\begin{array}{cc}\hfill & p\left(x_t\right|y_{1:T})=\mathcal{N}\left(x_t\right|{\mu }_t^{\left(s_1\right)},{\Sigma }_t^{\left(s_1\right)}),\hfill \end{array}$$

(69)

$$\begin{array}{cc}\hfill & p\left({\overline{x}}_t\right|y_{1:T})=\mathcal{N}\left({\overline{x}}_t\right|{\mu }_t^{\left(s_2\right)},{\Sigma }_t^{\left(s_2\right)}),\hfill \end{array}$$

(70)

$$\begin{array}{cc}\hfill & p({\overline{x}}_{t-1},x_t|y_{1:T})=\mathcal{N}\left(({\overline{x}}_{t-1},x_t)\right|{\mu }_t^{\left(s_3\right)},{\Sigma }_t^{\left(s_3\right)}).\hfill \end{array}$$

(71)

The smoothing distribution and filtering distribution of the last time step are identical, allowing for the recursive computation of the smoothing distribution for all time steps by beginning from the last step and moving backward.

Proof. 

Firstly by Lemma 1, the joint distribution of ${\overline{x}}_{t-1}$ and $x_t$ given $y_{1:t-1}$ is

$$\begin{array}{cc}\hfill p(x_t,{\overline{x}}_{t-1}|y_{1:t-1})& =p\left(x_t\right|{\overline{x}}_{t-1})p\left({\overline{x}}_{t-1}\right|y_{1:t-1})\hfill \\ \hfill & =\mathcal{N}\left(x_t\right|{\widehat{A}}_t{\overline{x}}_{t-1},{\widehat{Q}}_t)\mathcal{N}\left({\overline{x}}_{t-1}\right|{\mu }_{t-1}^{\left(m\right)},{\Sigma }_{t-1}^{\left(m\right)})\hfill \\ \hfill & =\mathcal{N}\left(\left.\left(\begin{array}{c}{\overline{x}}_{t-1}\\ x_t\end{array}\right)\right|{\mu }^{\left(1\right)},{\Sigma }^{\left(1\right)}\right),\hfill \end{array}$$

(72)

where

$$\begin{array}{c}\hfill {\mu }^{\left(1\right)}=\left(\begin{array}{c}{\mu }_{t-1}^{\left(m\right)}\\ {\widehat{A}}_t{\mu }_{t-1}^{\left(m\right)}\end{array}\right),{\Sigma }^{\left(1\right)}=\left(\begin{array}{cc}{\Sigma }_{t-1}^{\left(m\right)}& {\Sigma }_{t-1}^{\left(m\right)}{\widehat{A}}_t^{\top }\\ {\widehat{A}}_t{\Sigma }_{t-1}^{\left(m\right)}& {\widehat{A}}_t{\Sigma }_{t-1}^{\left(m\right)}{\widehat{A}}_t^{\top }+{\widehat{Q}}_t\end{array}\right).\end{array}$$

(73)

According to the Markov property of the states, we obtain

$$\begin{array}{c}\hfill p\left({\overline{x}}_{t-1}\right|x_t,y_{1:T})=p\left({\overline{x}}_{t-1}\right|x_t,y_{t-1}).\end{array}$$

(74)

Therefore, we have the conditional distribution as

$$\begin{array}{cc}\hfill p\left({\overline{x}}_{t-1}\right|x_t,y_{1:T})& =p\left({\overline{x}}_{t-1}\right|x_t,y_{t-1}),\hfill \\ \hfill & =\mathcal{N}({\overline{x}}_{t-1}|{\tilde{\mu }}^{\left(2\right)},{\tilde{\Sigma }}^{\left(2\right)}),\hfill \end{array}$$

(75)

where

$$\begin{array}{cc}\hfill & G_t={\Sigma }_{t-1}^{\left(m\right)}{\widehat{A}}_t^{\top }{({\widehat{A}}_t{\Sigma }_{t-1}^{\left(m\right)}{\widehat{A}}_t^{\top }+{\widehat{Q}}_t)}^{-1},\hfill \\ \hfill & {\tilde{\mu }}^{\left(2\right)}={\mu }_{t-1}^{\left(m\right)}+G_t(x_t-{\widehat{A}}_t{\mu }_{t-1}^{\left(m\right)}),\hfill \\ \hfill & {\tilde{\Sigma }}^{\left(2\right)}={\Sigma }_{t-1}^{\left(m\right)}-G_t{\widehat{A}}_t{\Sigma }_{t-1}^{\left(m\right)}.\hfill \end{array}$$

(76)

Then, the joint distribution of ${\overline{x}}_{t-1}$ and $x_t$ given all the data is

$$\begin{array}{cc}\hfill p({\overline{x}}_{t-1},x_t|y_{1:T})& =p\left({\overline{x}}_{t-1}\right|x_t,y_{1:T})p\left(x_t\right|y_{1:T})\hfill \\ \hfill & =\mathcal{N}\left({\overline{x}}_{t-1}\right|{\tilde{\mu }}^{\left(2\right)},{\tilde{\Sigma }}^{\left(2\right)})\mathcal{N}\left(x_t\right|{\mu }_t^{\left(s_1\right)},{\Sigma }_t^{\left(s_1\right)})\hfill \\ \hfill & =\mathcal{N}\left(\left.\left(\begin{array}{c}{\overline{x}}_{t-1}\\ x_t\end{array}\right)\right|{\mu }_t^{\left(s_3\right)},{\Sigma }_t^{\left(s_3\right)}\right),\hfill \end{array}$$

(77)

where

$$\begin{array}{cc}\hfill & {\mu }_t^{\left(s_3\right)}=\left(\begin{array}{c}{G}_t{\mu }_t^{\left(s_1\right)}+{\mu }_{t-1}^{\left(m\right)}-G_t{\widehat{A}}_t{\mu }_{t-1}^{\left(m\right)}\\ {\mu }_t^{\left(s_1\right)}\end{array}\right),\hfill \end{array}$$

(78)

$$\begin{array}{cc}\hfill & {\Sigma }_t^{s3}=\left(\begin{array}{cc}{G}_t{\Sigma }_t^{\left(s_1\right)}{G}_t^{\top }+{\tilde{\Sigma }}^{\left(2\right)}& {\Sigma }_t^{\left(s_1\right)}{G}_t^{\top }\\ G_t{\Sigma }_t^{\left(s_1\right)}& {\Sigma }_t^{\left(s_1\right)}\end{array}\right).\hfill \end{array}$$

(79)

Thus, the marginal distribution of ${\overline{x}}_{t-1}$ is given as

$$\begin{array}{c}\hfill p\left({\overline{x}}_{t-1}\right|y_{1:T})=\mathcal{N}\left({\overline{x}}_{t-1}\right|{\mu }_t^{\left(s_2\right)},{\Sigma }_t^{\left(s_2\right)}),\end{array}$$

(80)

where

$$\begin{array}{cc}\hfill & {\mu }_t^{\left(s_2\right)}=G_t{\mu }_t^{\left(s_1\right)}+{\mu }_{t-1}^{\left(m\right)}-G_t{\widehat{A}}_t{\mu }_{t-1}^{\left(m\right)},\hfill \end{array}$$

(81)

$$\begin{array}{cc}\hfill & {\Sigma }_t^{\left(s_2\right)}=G_t{\Sigma }_t^{\left(s_1\right)}{G}_t^{\top }+{\tilde{\Sigma }}^{\left(2\right)}.\hfill \end{array}$$

(82)

The $p\left(x_{t-1}\right|y_{1:T})$ will be directly obtained from $p\left({\overline{x}}_{t-1}\right|y_{1:T})$. Therefore, we have derived the concrete expressions of the Gaussian smoother distributions (Equations (69)–(71)).    □

Thus, we can obtain the posterior distributions of $q\left(x_{0:T}\right|y_{1:T})$ as well as the expectations involved in the update for the other factors, which are ${\mathbb{E}}_{q\left(x_{0:T}\right)}\left(x_t\right),{\mathbb{E}}_{q\left(x_{0:T}\right)}\left(x_t{x}_t^{\top }\right)$ and ${\mathbb{E}}_{q\left(x_{0:T}\right)}\left({\overline{x}}_t{x}_{t+1}^{\top }\right)$.

3.2. Update for $q\left({\omega }_{1:T}\right)$

$q\left({\omega }_{1:T}\right)$ can be updated using a closed-form expression by the Pólya-gamma augmentation. Equations (37) and (41) results in the following update for $q\left({\omega }_{1:T}\right)={\prod }_{t=1}^T{\prod }_{k=1}^{K-1}q\left({\omega }_{t,k}\right)$:

$$\begin{array}{cc}\hfill lnq\left({\omega }_{t,k}\right)& \propto {\mathbb{E}}_{-q\left({\omega }_{1:T}\right)}lnp(z_{t+1},{\omega }_t|x_t)\hfill \\ \hfill & \propto {\mathbb{E}}_{-q\left({\omega }_{1:T}\right)}lnp\left(z_{t+1}\right|{\omega }_t,x_t)+{\mathbb{E}}_{-q\left({\omega }_{1:T}\right)}lnPG\left({\omega }_{t,k}\right|\mathbb{I}(z_{t+1}>k),0).\hfill \end{array}$$

(83)

Since ${\mathbb{E}}_{-q\left({\omega }_{1:T}\right)}lnp\left(z_{t+1}\right|{\omega }_t,x_t)\propto -\frac{1}{2}{\mathbb{E}}_{-q\left({\omega }_{1:T}\right)}\left({\nu }_{t,k}^2\right){\omega }_{t,k}$, we have

$$\begin{array}{c}\hfill q\left({\omega }_{t,k}\right)=PG\left({\omega }_{t,k}\right|{\mathbb{E}}_{q\left(z_{1:T}\right)}\mathbb{I}(z_{t+1}>k),{\mathbb{E}}_{-q\left({\omega }_{1:T}\right)}\left({\nu }_{t,k}^2\right)).\end{array}$$

(84)

Moreover, the expectation of the posterior Pólya-gamma distribution is available in closed form [22]:

$$\begin{array}{c}\hfill \mathbb{E}\left({\omega }_{t,k}\right)=\frac{{\mathbb{E}}_{q\left(z_{1:T}\right)}\mathbb{I}(z_{t+1}>k)}{2{\mathbb{E}}_{q\left(z_{1:T}\right)q\left(x_{1:T}\right)q\left(\theta \right)}\left({\nu }_{t,k}^2\right)}tanh\left[\frac{1}{2}{\mathbb{E}}_{q\left(z_{1:T}\right)q\left(x_{1:T}\right)q\left(\theta \right)}\left({\nu }_{t,k}^2\right)\right].\end{array}$$

(85)

3.3. Update for $q\left(z_{1:T}\right)$

Since ${\mathbb{E}}_{-q\left(z_{1:T}\right)}lnp\left(\theta \right)$ is not a function of $z_{1:T}$, Equation (35) can be written as

$$\begin{array}{cc}\hfill lnq\left(z_{1:T}\right)& \propto {\mathbb{E}}_{-q\left(z_{1:T}\right)}lnp(z_{1:T},x_{0:T},{\omega }_{1:T},y_{1:T}|\theta )\hfill \\ \hfill & \propto {\mathbb{E}}_{-q\left(z_{1:T}\right)}\left[lnp\left(z_1\right|\theta )+\sum _{t=2}^{T}lnp\left(z_t\right|x_{t-1},z_{t-1},{\omega }_{t-1},\theta )+\sum _{t=1}^{T}lnp\left(x_t\right|z_t,\theta )\right].\hfill \end{array}$$

(86)

From Equation (86) it is evident that the resulting factor $q\left(z_{1:T}\right)$ has the form of an HMM parameterized by $q(x_{1:T},{\omega }_{1:T},\theta )$, and the expected sufficient statistics required for updating the other factors can be obtained by message passing algorithms. We define $ln{\tilde{\pi }}_{z_1}:={\mathbb{E}}_{-q\left(z_{1:T}\right)}lnp\left(z_1\right|\theta )$, $ln{\tilde{a}}_{z_t{z}_{t+1}}:={\mathbb{E}}_{-q\left(z_{1:T}\right)}lnp\left(z_{t+1}\right|z_t,x_t,{\omega }_t,\theta )$, $ln{\tilde{b}}_{z_t}\left(x_t\right):={\mathbb{E}}_{-q\left(z_{1:T}\right)}$$lnp\left(x_t\right|z_t,\theta )$ and the set of the HMM parameters is denoted as ${\widehat{\theta }}_{z_{1:T}}$. Thus, we have

$$\begin{array}{c}\hfill lnq\left(z_{1:T}\right)\propto ln{\tilde{\pi }}_{z_1}+\sum _{t=2}^{T}ln{\tilde{a}}_{z_{t-1}{z}_t}(t-1)+\sum _{t=1}^{T}ln{\tilde{b}}_{z_t}\left(x_t\right).\end{array}$$

(87)

Note that the densities $p\left(z_1\right|\theta ),p\left(z_{t+1}\right|z_t,x_t,{\omega }_t,\theta ),p\left(x_t\right|z_t,\theta )$ belong to the exponential family, these expectations can be evaluated in closed form. The computation of the HMM parameters by Equation (86) is summarized in Algorithm 2.

We apply a standard forward-backward algorithm for an HMM where the forward recursion is:

$$\begin{array}{c}\hfill {\alpha }_t\left(i\right):=p(z_t=i|x_{1:t})\propto {\tilde{b}}_i\left(x_t\right)\sum _{j=1}^K{\alpha }_{t-1}\left(j\right),\end{array}$$

(88)

where ${\alpha }_1\left(i\right)\propto {\tilde{\pi }}_i{\tilde{b}}_i\left(x_1\right)$ and the backward recursion is:

$$\begin{array}{c}\hfill {\beta }_t\left(i\right):=p\left(x_{t+1:T}\right|z_t=i)\propto \sum _{j=1}^K{\beta }_t\left(j\right){\tilde{b}}_j\left(x_t\right){\tilde{a}}_{ij}\left(t\right).\end{array}$$

(89)

Algorithm 2: Obtaining ${\widehat{\theta }}_{z_{1:T}}$

Then we can obtain the sufficient statistics used in the update for the other factors using the following equations:

$$\begin{array}{cc}\hfill & {\gamma }_t\left(i\right):=p(z_t=i|x_{1:T})\propto {\alpha }_t\left(i\right){\beta }_t\left(i\right),\hfill \end{array}$$

(90)

$$\begin{array}{cc}\hfill & {\xi }_t(i,j):=p(z_t=i,z_{t+1}=j|x_{1:T})\propto {\alpha }_t\left(i\right){\tilde{a}}_{ij}\left(t\right){\tilde{b}}_j\left(x_{t+1}\right){\beta }_{t+1}\left(j\right).\hfill \end{array}$$

(91)

3.4. Update for $q\left(\theta \right)$

Given the sufficient statistics of $q\left(z_{1:T}\right)$ and $q\left(x_{1:T}\right)$ obtained by message passing, the conjugate posterior distribution of the model parameters can be updated in closed form by evaluating Equation (38). For each parameter, we present the optimized variational distribution in the following:

• Update $q\left({\pi }_0\right)$. Simplifying Equation (38) and extracting the terms related to ${\pi }_0$, we have

  $$\begin{array}{c}\hfill lnq\left({\pi }_0\right)\propto {\mathbb{E}}_{q(-\theta )}lnp\left(z_1\right|\theta )+lnp\left({\pi }_0\right),\end{array}$$

  (92)

  which results in

  $$\begin{array}{c}\hfill q\left({\pi }_0\right)\sim Dir\left({\pi }_0\right|{\alpha }_0^{\prime }),\end{array}$$

  (93)

  where ${\alpha }_0^{\prime }=\left[{\alpha }_{0,1}^{\prime },\dots ,{\alpha }_{0,k}^{\prime },\dots ,{\alpha }_{0,K}^{\prime }\right]\in {\mathbb{R}}^K$ and

  $$\begin{array}{c}\hfill {\alpha }_{0,k}^{\prime }={\alpha }_{0,k}+{\gamma }_1\left(k\right),\end{array}$$

  (94)
  ${\alpha }_{0,k}$ are prior parameters of $p\left({\pi }_0\right)$.
• Update $({\mu }_0,{\Sigma }_0)$. The posterior $q({\mu }_0,{\Sigma }_0)$ can be obtained by the expectations of $q\left(x_{0:T}\right)$, given by

  $$\begin{array}{cc}\hfill & {\mu }_0\leftarrow {\mathbb{E}}_{q\left(x_{0:T}\right)}\left({\overline{x}}_0\right),\hfill \end{array}$$

  (95)

  $$\begin{array}{cc}\hfill & {\Sigma }_0\leftarrow {\mathbb{E}}_{q\left(x_{0:T}\right)}\left({\overline{x}}_0{\overline{x}}_0^{\top }\right)-{\mu }_0{\left({\mu }_0\right)}^{\top }.\hfill \end{array}$$

  (96)
• Update $q(A_k,Q_k)$. Simplifying Equation (38) and extracting the terms related to $(A_k,Q_k)$, we have

  $$\begin{array}{cc}\hfill & lnq(A_k,Q_k)\propto \hfill \\ \hfill & {\mathbb{E}}_{-q\left(\theta \right)}\left[-\frac{{\sum }_{t=1}^T{\gamma }_t\left(k\right)}{2}ln\left|Q_k\right|-\frac{1}{2}\sum _{t=1}^T{\gamma }_t\left(k\right){(x_t-A_k{\overline{x}}_{t-1})}^{\top }{Q}_k^{-1}(x_t-A_k{\overline{x}}_{t-1})\right]\hfill \\ \hfill & +lnp(A_k,Q_k),\hfill \end{array}$$

  (97)

  which results in

  $$\begin{array}{c}\hfill q(A_k,Q_k)\sim MNIW(A_k,Q_k|{\eta }_k^{\prime }),\end{array}$$

  (98)

  where ${\eta }_k^{\prime }:=\left\{M_k^{\left({\eta }^{\prime }\right)},{\mathrm{\Phi }}_k^{\left({\eta }^{\prime }\right)},{\iota }_k^{\left({\eta }^{\prime }\right)},{\Delta }_k^{\left({\eta }^{\prime }\right)}\right\}$,

  $$\begin{array}{cc}\hfill q(A_k,Q_k)& \propto |Q_k{|}^{-\frac{{\iota }_k^{\left({\eta }^{\prime }\right)}+(p+1)m+1}{2}}exp\{-\frac{1}{2}Tr\left({\Delta }_k^{\left({\eta }^{\prime }\right)}{Q}_k^{-1}\right)\hfill \\ \hfill & -\frac{1}{2}Tr\left[{\left({\mathrm{\Phi }}_k^{\left({\eta }^{\prime }\right)}\right)}^{-1}{(A_k-M_k^{\left({\eta }^{\prime }\right)})}^{\top }{Q}_k^{-1}(A_k-M_k^{\left({\eta }^{\prime }\right)})\right]\}.\hfill \end{array}$$

  (99)
  The variational posterior parameters can be optimized as

  $$\begin{array}{cc}\hfill & {\mathrm{\Phi }}_k^{\left({\eta }^{\prime }\right)}={\left({\left({\mathrm{\Phi }}_k^{\left(\eta \right)}\right)}^{-1}+\sum _{t=1}^T{\mathbb{E}}_{q\left(x_{0:T}\right)}\left({\gamma }_t\left(k\right){\overline{x}}_{t-1}{\overline{x}}_{t-1}^{\top }\right)\right)}^{-1},\hfill \\ \hfill & M_k^{\left({\eta }^{\prime }\right)}=\left(M_k^{\left(\eta \right)}{\left({\mathrm{\Phi }}_k^{\left(\eta \right)}\right)}^{-1}+\sum _{t=1}^T{\mathbb{E}}_{q\left(x_{0:T}\right)}\left({\gamma }_t\left(k\right)x_t{\overline{x}}_{t-1}^{\top }\right)\right){\mathrm{\Phi }}_k^{\left({\eta }^{\prime }\right)},\hfill \\ \hfill & {\Delta }_k^{\left({\eta }^{\prime }\right)}={\Delta }_k^{\left(\eta \right)}+\sum _{t=1}^T{\mathbb{E}}_{q\left(x_{0:T}\right)}\left({\gamma }_t\left(k\right)x_t{x}_t^{\top }\right)+M_k^{\left(\eta \right)}{\left({\mathrm{\Phi }}_k^{\left(\eta \right)}\right)}^{-1}{\left(M_k^{\left(\eta \right)}\right)}^{\top }-M_k^{\left({\eta }^{\prime }\right)}{\left({\mathrm{\Phi }}_k^{\left({\eta }^{\prime }\right)}\right)}^{-1}{\left(M_k^{\left({\eta }^{\prime }\right)}\right)}^{\top },\hfill \\ \hfill & {\iota }_k^{\left({\eta }^{\prime }\right)}={\iota }_k^{\left(\eta \right)}+\sum _{t=1}^T{\gamma }_t\left(k\right),\hfill \end{array}$$

  (100)

  where $M_k^{\left(\eta \right)},{\mathrm{\Phi }}_k^{\left(\eta \right)},{\iota }_k^{\left(\eta \right)},{\Delta }_k^{\left(\eta \right)}$ are prior parameters of $p(A_k,Q_k)$.
• Update $q(C_k,S_k)$. Simplifying Equation (38) and extracting the terms related to $(C_k,S_k)$, we have

  $$\begin{array}{cc}\hfill & lnq(C_k,S_k)\propto \hfill \\ \hfill & {\mathbb{E}}_{-q\left(\theta \right)}\left[-\frac{{\sum }_{t=1}^T{\gamma }_t\left(k\right)}{2}ln\left|S_k\right|-\frac{1}{2}\sum _{t=1}^T{(y_t-C_k{x}_t)}^{\top }{S}_k^{-1}(y_t-C_k{x}_t)\right]\hfill \\ \hfill & +lnp(C_k,S_k),\hfill \end{array}$$

  (101)

  which results in

  $$\begin{array}{c}\hfill q(C_k,S_k)\sim MNIW(C_k,S_k|{\lambda }_k^{\prime }),\end{array}$$

  (102)

  where ${\lambda }_k^{\prime }:=\left\{M_k^{\left({\lambda }^{\prime }\right)},{\mathrm{\Phi }}_k^{\left({\lambda }^{\prime }\right)},{\iota }_k^{\left({\lambda }^{\prime }\right)},{\Delta }_k^{\left({\lambda }^{\prime }\right)}\right\}$,

  $$\begin{array}{cc}\hfill q(C_k,S_k)& \propto |S_k{|}^{-\frac{{\iota }_k^{\left({\lambda }^{\prime }\right)}+(p+1)m+1}{2}}exp\{-\frac{1}{2}Tr\left({\Delta }_k^{\left({\lambda }^{\prime }\right)}{S}_k^{-1}\right)\hfill \\ \hfill & -\frac{1}{2}Tr\left[{\left({\mathrm{\Phi }}_k^{\left({\lambda }^{\prime }\right)}\right)}^{-1}{(C_k-M_k^{\left({\lambda }^{\prime }\right)})}^{\top }{Q}_k^{-1}(C_k-M_k^{\left({\lambda }^{\prime }\right)})\right]\}.\hfill \end{array}$$

  (103)
  The variational posterior parameters can be optimized as

  $$\begin{array}{cc}\hfill & {\mathrm{\Phi }}_k^{\left({\lambda }^{\prime }\right)}={\left({\left({\mathrm{\Phi }}_k^{\left(\lambda \right)}\right)}^{-1}+\sum _{t=1}^T{\mathbb{E}}_{q\left(x_{0:T}\right)}\left({\gamma }_t\left(k\right)x_t{x}_t^{\top }\right)\right)}^{-1},\hfill \\ \hfill & M_k^{\left({\lambda }^{\prime }\right)}=\left(M_k^{\left(\lambda \right)}{\left({\mathrm{\Phi }}_k^{\left(\lambda \right)}\right)}^{-1}+\sum _{t=1}^T{\mathbb{E}}_{q\left(x_{0:T}\right)}\left({\gamma }_t\left(k\right)y_t{x}_t^{\top }\right)\right){\mathrm{\Phi }}_k^{\left({\lambda }^{\prime }\right)},\hfill \\ \hfill & {\Delta }_k^{\left({\lambda }^{\prime }\right)}={\Delta }_k^{\left(\lambda \right)}+\sum _{t=1}^T{\gamma }_t\left(k\right)y_t{y}_t^{\top }+M_k^{\left(\lambda \right)}{\left({\mathrm{\Phi }}_k^{\left(\lambda \right)}\right)}^{-1}{\left(M_k^{\left(\lambda \right)}\right)}^{\top }-M_k^{\left({\lambda }^{\prime }\right)}{\left({\mathrm{\Phi }}_k^{\left({\lambda }^{\prime }\right)}\right)}^{-1}{\left(M_k^{\left({\lambda }^{\prime }\right)}\right)}^{\top },\hfill \\ \hfill & {\iota }_k^{\left({\lambda }^{\prime }\right)}={\iota }_k^{\left(\lambda \right)}+\sum _{t=1}^T{\gamma }_t\left(k\right),\hfill \end{array}$$

  (104)

  where $M_k^{\left(\lambda \right)},{\mathrm{\Phi }}_k^{\left(\lambda \right)},{\iota }_k^{\left(\lambda \right)},{\Delta }_k^{\left(\lambda \right)}$ are prior parameters of $p(C_k,S_k)$.
• Update $q\left(R_k\right)$. Simplifying Equation (38) and extracting the terms related to $R_k$, we have

  $$\begin{array}{cc}\hfill & lnq\left(R_k\right)\propto \hfill \\ \hfill & {\mathbb{E}}_{-q\left(\theta \right)}\left[-\frac{1}{2}\sum _{t=1}^{T-1}{\gamma }_t\left(k\right){(R_k{x}_t+r_k-{\Omega }_t^{-1}{\kappa }_{t+1})}^{\top }{\Omega }_t(R_k{x}_t+r_k-{\Omega }_t^{-1}{\kappa }_{t+1})\right]\hfill \\ \hfill & +\sum _{i=1}^{K-1}lnp\left(R_{k,i}\right),\hfill \end{array}$$

  (105)

  which results in

  $$\begin{array}{c}\hfill q\left(R_k\right)\sim \prod _{i=1}^{K-1}\mathcal{N}\left(R_{k,i}\right|{\zeta }_{k,i}^{\prime }),\end{array}$$

  (106)

  where ${\zeta }_{k,i}^{\prime }:=\left\{{\mu }_{k,i}^{\left({\zeta }^{\prime }\right)},{\Sigma }_{k,i}^{\left({\zeta }^{\prime }\right)}\right\}$, which can be optimized as

  $$\begin{array}{cc}\hfill & {\Sigma }_{k,i}^{\left({\zeta }^{\prime }\right)}={\left[{\left({\Sigma }_{k,i}^{\left(\zeta \right)}\right)}^{-1}+\sum _{t=1}^T{\mathbb{E}}_{q({\omega }_{1:T},x_{0:T})}\left(x_t{\gamma }_t\left(k\right){\Omega }_{t,i}{x}_t^{\top }\right)\right]}^{-1},\hfill \end{array}$$

  (107)

  $$\begin{array}{cc}\hfill & {\mu }_{k,i}^{\left({\zeta }^{\prime }\right)}={\Sigma }_{k,i}^{\left({\zeta }^{\prime }\right)}\left[\sum _{t=1}^{T-1}{\mathbb{E}}_{q\left({\omega }_{1:T}\right)q\left(x_{0:T}\right)}\left({\gamma }_t\left(k\right){\Omega }_{t,i}{H}_{t,k,i}{x}_t\right)+{\left({\Sigma }_{k,i}^{\left(\zeta \right)}\right)}^{-1}{\mu }_{k,i}^{\left(\zeta \right)}\right],\hfill \end{array}$$

  (108)

  $$\begin{array}{cc}\hfill & H_{t,k}={[H_{t,k,1},\cdots ,H_{t,k,i},\cdots ,H_{t,k,K-1}]}^{\top }={\mathbb{E}}_{q\left(r_k\right)q\left({\omega }_{1:T}\right)}\left(r_k-{\Omega }_t^{-1}{\kappa }_{t+1}\right),\hfill \end{array}$$

  (109)

  and ${\mu }_{k,i}^{\left(\zeta \right)},{\Sigma }_{k,i}^{\left(\zeta \right)}$ are prior parameters of $p\left(R_{k,i}\right)$. To derive Equation (107), just note that $R_k^{\top }{\Omega }_t{R}_k={\sum }_{i=1}^{K-1}{R}_{k,i}{\Omega }_{k,i}{R}_{k,i}^{\top }$ and use the matrix trace.
• Update $q\left(r_k\right)$. Simplifying Equation (38) and extracting the terms related to $r_k$, we have

  $$\begin{array}{cc}\hfill & lnq\left(r_k\right)\propto \hfill \\ \hfill & {\mathbb{E}}_{-q\left(\theta \right)}\left[-\frac{1}{2}\sum _{t=1}^{T-1}{\gamma }_t\left(k\right){(R_k{x}_t+r_k-{\Omega }_t^{-1}{\kappa }_{t+1})}^{\top }{\Omega }_t(R_k{x}_t+r_k-{\Omega }_t^{-1}{\kappa }_{t+1})\right]\hfill \\ \hfill & +lnp\left(r_k\right),\hfill \end{array}$$

  (110)

  which results in

  $$\begin{array}{c}\hfill q\left(r_k\right)\sim \mathcal{N}\left(r_k\right|{\rho }_k^{\prime }),\end{array}$$

  (111)

  where ${\rho }_k^{\prime }:=\left\{{\mu }_k^{\left({\rho }^{\prime }\right)},{\Sigma }_k^{\left({\rho }^{\prime }\right)}\right\}$, which can be optimized as

  $$\begin{array}{cc}\hfill & {\Sigma }_k^{\left({\rho }^{\prime }\right)}={\left[{\left({\Sigma }_k^{\left(\rho \right)}\right)}^{-1}+\sum _{t=1}^{T-1}{\mathbb{E}}_{q\left({\omega }_{1:T}\right)}\left({\gamma }_t\left(k\right){\Omega }_t\right)\right]}^{-1},\hfill \end{array}$$

  (112)

  $$\begin{array}{cc}\hfill & {\mu }_k^{\left({\rho }^{\prime }\right)}={\Sigma }_k^{\left({\rho }^{\prime }\right)}\left[{\left({\Sigma }_k^{\left(\rho \right)}\right)}^{-1}{\mu }_k^{\left(\rho \right)}+\sum _{t=1}^{T-1}{\mathbb{E}}_{q\left(R_k\right)q\left(x_{0:T}\right)q\left({\omega }_{1:T}\right)}\left({\gamma }_t\left(k\right){\Omega }_t\left(R_k{x}_t-{\Omega }_t^{-1}{\kappa }_{t+1}\right)\right)\right],\hfill \end{array}$$

  (113)

  where ${\mu }_k^{\left(\rho \right)},{\Sigma }_k^{\left(\rho \right)}$ are prior parameters of $p\left(r_k\right)$.

3.5. Initialization

To obtain a reliable variational inference, we initialized the model parameters and latent states with reasonable values by using the following initialization procedure:

• Probabilistic Principal Component Analysis (PPCA) [27] is conducted on the data, to initialize the continuous latent variables, $x_{0:T}$ and the parameters C;
• Fit an AR(p)-HMM to initialize the switching states, $z_{1:T}$ and the parameters, $\left\{A_k,Q_k\right\}$. The autoregressive order, p, is determined by AIC, BIC, and HQ criterion;
• To alleviate the possible and undesirable dependence on ordering that arises from the stick-breaking formulation during the inference, we adopt a strategy of greedily fitting a decision list [21] to identify the most suitable permutation of the switching states for the stick-breaking process. Specifically, we start by performing a greedy search on permutations by creating a decision list based on $(x_t,z_t),z_{t+1}$ pairs, given by

  $$\begin{array}{c}\hfill z_{t+1}=\left\{\begin{array}{cc}{o}_1\hfill & if{p}_1\hfill \\ o_2\hfill & if\neg p_1\cap p_2\hfill \\ ⋮\hfill \\ o_K\hfill & if{\cap }_{i=1}^{K-1}(\neg p_i)\hfill \end{array}\right.,\end{array}$$

  (114)

  where $(o_1,\dots ,o_K)$ is a permutation of $(1,\dots ,K)$, and $p_1,\dots ,p_k$ are predicates that rely on $(x_t,z_t)$ and provide a true or false. In our framework, these predicates are defined by logistic functions:

  $$\begin{array}{c}\hfill p_j=\sigma \left(R_{0,j}^{\top }{x}_t\right)>\epsilon ,\end{array}$$

  (115)

  where $\epsilon \in (0,1)$ should be predetermined. To determine $o_1$ and $r_1$, we used the maximum a posteriori estimate of the model for each of the K potential states. For the kth logistic regression, the inputs are $x_{0:T}$ and the outputs are $y_t=\mathbb{I}(z_{t+1}=k)$. We chose the logistic regression model with the largest likelihood as the first output. Then we excluded time points where $z_{t+1}=o_1$ from the data and proceeded to $K-1$ logistic regressions to predict the subsequent output, $o_2$, and so on. After cycling through all K results, we obtained the permutation of $z_{1:T}$. In addition, the ${\left\{R_{0,j}\right\}}_{j=1}^{K-1}$ served as an initialization of the recurrence weights (R).

4. Numerical Experiments

To investigate the performance of the proposed HO-rSLDS, we generated synthetic datasets through the following steps:

• Benchmark model settings.We take AR-HSMM as a benchmark model to generate synthetic time series. Specifically, the dimensionality of the synthetic time series is $n=5$, and the total length is $T=1000$. The state number is $K=3$, and the autoregressive order p takes values $\left\{1,2,3\right\}$.
• Generating $(A_k,Q_k)$. $(A_k,Q_k)$ corresponds to the emission parameters involved in the VAR process of latent variables $x_t$ (Equation (11)). The autoregressive coefficient matrices $A_k$ are generated with 50% sparsity (defined as the proportion of the zero elements), i.e., 50% of the elements in $A_k$ are 0. The non-zero elements of $A_k$ are generated to be positive or negative with equal probability, and their absolute value is sampled uniformly between 0.2 and 0.5. The covariance matrices $Q_k$ are generated as $Q_k=P_k^{\top }diag\left({\sigma }_1,\dots ,{\sigma }_d\right)P_k$, with $P_k$ being an orthogonal matrix and ${\sigma }_i$ assumed positive. The matrix $P_k$ is constructed by orthogonalizing a random matrix whose entries are simulated from a standard Normal, while each ${\sigma }_i$ is uniformly sampled in the interval $[1,3]$. Additionally, a rejection step is done to check that the sampled $(A_k,Q_k)$ constituted a stable VAR process.
• Generating switching state sequence $z_{1:T}$. To generate $z_{1:T}$, we set the state transition probabilities ${\pi }_{jk}$ to 0.5, for $j\ne k=1,2,3$. Note that in an HSMM, the self-transition probability is 0. Furthermore, we simulate the state duration time in two cases. In the first case, the state duration time is sampled from a geometric distribution (the between-state transition probabilities are set to 0.1). In the second case, the state duration time is directly sampled from $\left\{1,\dots ,10\right\}$ with specified sampling probabilities. Since the second case does not correspond to a parametric distribution, we denote these two cases as geometric and nonparametric, respectively. In addition, the initial state is uniformly sampled from $\left\{1,\dots ,10\right\}$.
• Generating $(C_k,S_k)$. $(C_k,S_k)$ corresponds to the emission parameters generating the Gaussian observations $y_t$ (Equation (12)). We generate $(C_k,S_k)$ in the same manner as $(A_k,Q_k)$.
• Repeat the above procedure 100 times to obtain 100 synthetic data by setting different random seeds when generating $(A_k,Q_k,C_k,S_k)$.

Figure 4 displays all dimensions and the true state sequence of one synthetic dataset.

We investigated the performance of HO-rSLDS by comparing with the competing models, while the estimation accuracy is evaluated by the normalized mean squared error (NMSE) between the true and estimated emission model parameters and the signal-to-noise ratio (SNR) of the inferred optimized switching state sequence (${\widehat{z}}_{1:T}$), given by

$$\begin{array}{cc}\hfill & NMSE=\frac{1}{K}\sum _{k=1}^K\frac{\parallel {\widehat{C}}_k-C_k{\parallel }_F}{\parallel C_k{\parallel }_F},\hfill \end{array}$$

(116)

$$\begin{array}{cc}\hfill & SNR=\frac{{\sum }_{t=1}^T\mathbb{I}(z_t={\widehat{z}}_t)}{T},\hfill \end{array}$$

(117)

where ${\parallel ·\parallel }_F$ denotes Frobenius norm, ${\widehat{C}}_k$ and ${\widehat{z}}_{1:T}$ are estimated values. For AR-HMM, the NMSE is evaluated based on the true and estimated VAR emission parameters. To obtain ${\widehat{z}}_{1:T}$, we set ${\widehat{z}}_t=\underset{k}{argmax}{\gamma }_t\left(k\right)$. In addition, we avoid the optimization about discrete state number (K) and the dependence order (p) in HO-rSLDS by directly setting them to the true value.

The results of estimated NMSE and SNR for the three models are summarized in

Table 1. NMSE derived from AR-HMM, SLDS, and HO-rSLDS based on the synthetic time series with different state duration distribution and autoregressive order in the generative model. All the results are the average of the 100 experiments (mean ± std).

NMSE		$\mathit{p}=1$	$\mathit{p}=2$	$\mathit{p}=3$
Geometric	AR-HMM	0.25 ± 0.06	0.33 ± 0.07	0.37 ± 0.09
	SLDS	0.23 ± 0.05	0.39 ± 0.09	0.44 ± 0.11
	HO-rSLDS	0.16 ± 0.04	0.17 ± 0.05	0.20 ± 0.06
Nonparametric	AR-HMM	0.32 ± 0.07	0.41 ± 0.09	0.48 ± 0.11
	SLDS	0.31 ± 0.09	0.44 ± 0.11	0.49 ± 0.13
	HO-rSLDS	0.18 ± 0.06	0.20 ± 0.05	0.21 ± 0.06

and

Table 2. SNR derived from AR-HMM, SLDS, and HO-rSLDS based on the synthetic time series with different state duration distribution and autoregressive order in the generative model. All the results are the average of the 100 experiments (mean ± std).

SNR		$\mathit{p}=1$	$\mathit{p}=2$	$\mathit{p}=3$
Geometric	AR-HMM	0.79 ± 0.08	0.76 ± 0.07	0.69 ± 0.08
	SLDS	0.85 ± 0.06	0.68 ± 0.09	0.64 ± 0.09
	HO-rSLDS	0.93 ± 0.04	0.92 ± 0.05	0.90 ± 0.05
Nonparametric	AR-HMM	0.73 ± 0.08	0.65 ± 0.07	0.59 ± 0.10
	SLDS	0.80 ± 0.06	0.60 ± 0.09	0.54 ± 0.11
	HO-rSLDS	0.91 ± 0.04	0.87 ± 0.05	0.86 ± 0.05

, respectively. It is observed that HO-rSLDS outperforms AR-HMM and SLDS with lower average NMSE and higher average SNR for all the autoregressive orders and state duration distribution combinations. In addition, when $p=1$, for both geometric and nonparametric state duration, SLDS outperforms AR-HMM slightly, which may be because the synthetic data are generated by a simulated extensive SLDS structure. However, when $p>1$ and there is long-term dependence involved in the latent variables or observation data, it is just the opposite, since AR-HMM with higher p can capture the temporal relationships in the observation data while SLDS cannot. Moreover, when the transition of the underlying discrete states is characterized by a nonparametric duration distribution, the estimation accuracy of AR-HMM and SLDS is reduced, while HO-rSLDS remains stable with good estimation accuracy. This is because state transitions in both AR-HMM and SLDS are characterized by a Markov assumption and restricted to geometric duration distribution. Thus, more weight is given to shorter consecutive time periods within a certain state, which means the regimes switch frequently, which may not be appropriate in cases without sufficient prior information about a geometric state duration. By comparison, HO-rSLDS exhibits robustness to the state duration distributions since it improves the state transition by leveraging the stick-breaking logistic regression. The current state $z_t$ no longer depends on only the preceding state $z_{t-1}$, but also the latent variable $x_{t-1}$, and, furthermore, the previous observations.

A more exhaustive comparison is illustrated in Figure 5 where HO-rSLDS outperforms the competing models with better estimation accuracy and statistical significance in all the cases (t-test, ** $p<0.01$).

5. Dynamic Functional Connectivity Analysis in fMRI Data

To show that HO-rSLDS can also work well with real-world data, we evaluate the performance of HO-rSLDS by comparing it with standard SLDS based on public functional magnetic resonance imaging (fMRI) data. The data are collected in the Human Connectome Project (HCP) 1200 Parcellation Timeseries Netmats (HCP1200-TPN) dataset (Available at https://www.humanconnectome.org, (accessed on 6 March 2024)). Please see [28] for a detailed explanation of the entire acquisition protocol. In this study, we apply HO-rSLDS to data from 20 unrelated subjects with dimension $d=15$ and times series length $T_0=4800$ for each subject included in the HCP1200-PTN release to perform dynamic functional connectivity (DFC) analysis, which can be used to analysis the dynamical temporal coherence among endogenous fluctuations in distributed brain regions [29,30]. In general, HO-rSLDS and SLDS assume that there are metastable states with characteristic connectivity pattern in the brain, while the connectivity pattern can be represented by the statistical correlations directly. Specifically, by Lemma 1, the functional connectivity pattern of state k in HO-rSLDS and SLDS can be obtained as

$$\begin{array}{c}\hfill {DFC}_k=\frac{{\sum }_{t=1}^T{\gamma }_t\left(k\right)\left[\mathbb{E}{C}_k\left(\mathbb{E}{x}_t{x}_t^{\top }-\mathbb{E}{x}_t\mathbb{E}{x}_t^{\top }\right)\mathbb{E}{C}_k^{\top }+\mathbb{E}{S}_k\right]}{{\sum }_{t=1}^T{\gamma }_t\left(k\right)}.\end{array}$$

(118)

In addition, there are two unknown parameters: the state number K and the dependence order p. We use the mixture minimum description length (MMDL) considering optimal code lengths for each state in a mixed model [31], given by

$$\begin{array}{c}\hfill MMDL\left(K\right):=-ln\mathcal{L}\left(K\right)+\frac{1}{2}ln\left(T\right)K(K-1)+\sum _{k=1}^K\frac{1}{2}ln\left(T{\overline{\gamma }}_k\right)d^2,\end{array}$$

(119)

where $\mathcal{L}\left(K\right)$ is the variational lower bound evaluated during the model inference and ${\overline{\gamma }}_k:={\sum }_{t=1}^T{\gamma }_t\left(k\right)$. The concept of MMDL may be elucidated by the minimal description length principle in [32]. The state number K is determined by minimizing the MMDL on a specified list of K. In addition, we fit VAR model on the fMRI time series and the optimal dependence order p will be determined by AIC, BIC and HQ criteria [33], given by

$$\begin{array}{cc}\hfill & AIC:=ln\left(det\left(\widehat{\Sigma }\right)\right)+\frac{2p{d}^2}{T},\hfill \end{array}$$

(120)

$$\begin{array}{cc}\hfill & BIC:=ln\left(det\left(\widehat{\Sigma }\right)\right)+\frac{ln\left(T\right)p{d}^2}{T},\hfill \end{array}$$

(121)

$$\begin{array}{cc}\hfill & HQ:=ln\left(det(\Sigma )\right)+\frac{2ln\left(lnT\right)p{d}^2}{T},\hfill \end{array}$$

(122)

where $\widehat{\Sigma }$ is the estimated covariance matrix of noise in VAR model. To avoid the model order too large to permit feasible computation, we choose the lag order on the condition when AIC/BIC/HQ show no further substantial decreases at higher orders [34]. As illustrated in Figure 6, we set $K=4$ according to the minimal MMDL and $p=4$ since the downward trend of AIC, BIC and HQ tends to be flat with higher order.

In Figure 7, we present the DFC patterns derived from the proposed HO-rSLDS and the standard SLDS. The 15 brain regions producing the fMRI time series are indexed by numbers 1 to 15. The states are ordered by their fractional occupancy (defined as the percentage of time allocated to that specific state); it is evident that the 4 states from SLDS have a more average fractional occupancy distribution. Moreover, it is observed that DFC patterns derived from SLDS are more similar than those from HO-rSLDS, where there is only a slight difference in DFC patterns between State4 and the other three states from SLDS. We use cosine similarity to measure the difference of the DFC patterns characterized by each state, given by

$$\begin{array}{c}\hfill cos(F_1,F_2)=\frac{Tr\left(F_1{F}_2\right)}{\parallel F_1{\parallel }_F{\parallel F_2\parallel }_F},\end{array}$$

(123)

where $F_1$ and $F_2$ are square matrices. By this means, the averaged cosine similarity for SLDS is 0.9189 while that for HO-rSLDS is 0.6731. Although the real DFC patterns of this fMRI data are unknown, compared to standard SLDS, HO-rSLDS can indeed dig out more information about the DFC patterns by the model superiority. For example, the State4 in HO-rSLDS, with the lowest fractional occupancy, can be considered as a strongly connected state where the absolute functional connectivity is significantly larger than that of the other three states. Moreover, compared with standard SLDS, there is significantly positive or negative functional connectivity in the four states derived from HO-rSLDS. In addition, there is similar functional connectivity between State1 and State2, which results in a higher cosine similarity than the averaged value for HO-rSLDS (0.7776). The similar functional connectivity is not exhibited obviously in State3 (such as the negative connectivity between region 8 and regions 11, 12, 13). However, these findings cannot be obtained by standard SLDS.

6. Conclusions

In this study, we have established a new method named HO-rSLDS as an extension of SLDS that improves the statistical modeling of dynamics from complex systems. First, HO-rSLDS addresses the problem of discovering a higher-order temporal dependence involved in the latent variables or observation data by allowing the current latent variable to be associated with the measurements of several previous time steps as in AR-HMM. Second, HO-rSLDS improves the switching state transition by utilizing stick-breaking logistic regression and Pólya-gamma augmentation, making the transition dependent on the latent variable and thus overcoming the limitations of the Markov assumption, leading to a geometric distribution of state durations. Moreover, the symmetric dependency between the switching states and the latent variable is recovered from the asymmetric relationships as in standard SLDS. We have presented the detailed inference algorithms of the HO-rSLDS for message passing and parameter learning. In numerical experiments, we concluded that HO-rSLDS outperforms the competing models with higher estimation accuracy and robustness. The utility and versatility of the developed HO-rSLDS have been demonstrated on real-world data as well. The application of fMRI data for DFC analyses indicates the superiority of the method in discovering abundant information about DFC patterns. A potential limitation of the proposed HO-rSLDS is that the state number and dependency order need to be predetermined in applications, while the hyperparameter optimization is always a non-trivial task. Future research will improve the proposed model by utilizing Bayesian nonparametric methods so that the complicated hyperparameter optimization can be efficiently avoided [35]. The proposed model improves the current methods for learning the switching linear dynamical modes, which will facilitate the identification and assessment of the dynamics of complex systems.