A Sparse Multiwavelet-Based Generalized Laguerre–Volterra Model for Identifying Time-Varying Neural Dynamics from Spiking Activities

Abstract

Modeling of a time-varying dynamical system provides insights into the functions of biological neural networks and contributes to the development of next-generation neural prostheses. In this paper, we have formulated a novel sparse multiwavelet-based generalized Laguerre–Volterra (sMGLV) modeling framework to identify the time-varying neural dynamics from multiple spike train data. First, the significant inputs are selected by using a group least absolute shrinkage and selection operator (LASSO) method, which can capture the sparsity within the neural system. Second, the multiwavelet-based basis function expansion scheme with an efficient forward orthogonal regression (FOR) algorithm aided by mutual information is utilized to rapidly capture the time-varying characteristics from the sparse model. Quantitative simulation results demonstrate that the proposed sMGLV model in this paper outperforms the initial full model and the state-of-the-art modeling methods in tracking performance for various time-varying kernels. Analyses of experimental data show that the proposed sMGLV model can capture the timing of transient changes accurately. The proposed framework will be useful to the study of how, when, and where information transmission processes across brain regions evolve in behavior.

1. Introduction

Modeling of information transmission processes across brain regions, including spike generation and synaptic transmission, is challenging because neurobiological processes can be nonstationary [1,2]. For example, studies establish that the synaptic plasticity, with specific forms of long-term potentiation (LTP) and long-term depression (LTD), occur in response to specific input patterns, which can be indicated as the system nonstationarity. However, tracking of the changes for such time-varying functional connectivity on the level of neural ensembles is still challenging [3] due to the potential nonstationary properties.

Generally, three major classes of system identification methods have been applied to the study of the time-varying neural system. The first class of the nonstationary systems modeling method is the data-driven artificial neural networks (ANNs), which have been widely applied to various fields, including speech recognition [4], forecasting [5], as well as nonstationary or nonlinear system modeling [6]. However, the multiple hidden layers and the nodes in the networks structure are usually difficult to interpret. The second class of time-varying parametric estimation approaches is based on the adaptive filtering strategy, such as the steepest descent algorithm [7], stochastic gradient algorithms [8], Kalman filter and its variants [9]. The adaptive filter algorithms for the spike train data update the model parameters iteratively according to the difference between the occurrence of a real spike and the estimated spiking firing probability [10], which has an advantage in online estimation of a model that represents the best fit to an unknown neural system. However, if the model coefficients changed abruptly, adaptive filters might be incapable of capturing the transient properties of a time-varying system due to the convergence time taken in the adaptive filters method [11].

The third class is the basis function expansion method where the unknown time-varying parameters are projected onto a set of predefined basis functions. By this means, the time-varying identification problem can be transformed into an expansion-based time-invariant parametric modeling scheme, with remarkable reduction in the number of the coefficients to be estimated. A variety of basis functions can be utilized under this strategy, including the Walsh function [12], Chebshev basis [13], Laguerre polynomials [14], Legendre series [15], and wavelets [16,17]. Each family of basis functions should be considered based on its unique fit to the kernel shape if the number of basis functions to be used is limited. For example, the Chebshev basis and the Legendre polynomials both perform well in tracking smoothly or slowly changing parameters [13], while the Walsh and Haar basis functions are appropriate for tracking abrupt time-varying parameters [18]. Towards the applicability to a dynamical system with spike train inputs and outputs, the Chebshev basis expansion technique has been tested [19]. However, an unknown neural system can involve multitudes of time-varying dynamics, where both smooth and abrupt changes can be found simultaneously. The Chebshev basis expansion may be insufficient to capture the potential diverse time-varying properties effectively. Instead, we propose the use of a wavelet basis function, which is localized both in the time and frequency domain [20], can be effectively utilized to represent a sharp or slow variation vanishing outside a short time interval [21,22]. Li et al. previously combined the cardinal B-splines basis function with a block least mean square algorithm to track both slow and rapid changes of coefficients effectively in time-varying linear systems [23,24]. The team later extended this method to the identification of nonlinear time-varying systems and obtained satisfactory results of tracking time-varying variations from EEG nonstationary data [25].

A biological neural network commonly exhibits small-world properties where the neurons are not fully connected [26,27]. Commonly, the direct utilization of the basis function expansion method based on all input spiking signals can easily suffer from an overfitting problem due to the spurious input connectivity. Regularization is crucial for achieving functional sparsity at the global levels, which has been increasingly applied to solve the ill-posed or overfitting problems in the statistics and machine learning field [28]. In the context of the sparsity in neural connectivity, the regularization method is also very popular to process the issue of the sparse connectivity [29]. Various regularization approaches, such as the least absolute shrinkage and selection operator (LASSO) [30], the smoothly clipped absolute deviation (SCAD) [31], group LASSO [32], and group ridge [33] have been employed to obtain sparse representations. For example, Song et al. implemented a group LASSO to capture the sparsity of a neural system [34]. However, existing studies only focused on a time-invariant system, while there is a lack of reliable method for the regularized estimation of the time-varying system in nature. In addition, our previous work indicated that the multiwavelet-based time-varying generalized Laguerre–Volterra model can be used to track both slow and rapid changes of time-varying parameters simultaneously for the spiking train data [24], while it may fall into the dilemma of overfitting in the case that the neural system is rather sparsely connected. Therefore, we may integrate both the group LASSO and the multiwavelet basis expansion method in a unified framework to include their advantages (e.g., the group LASSO algorithm can eliminate the spurious input and capture the sparsity of a neural system, while the multiwavelet basis expansion method can effectively track both slow and rapid changes of time varying parameters simultaneously) and achieve an efficient and accurate estimation for the nonstationary spiking system.

In this paper, we propose a sparse multiwavelet-based generalized Laguerre–Volterra (sMGLV) modeling framework to identify the underlying nonstationary sparse neuronal connectivity by observing the input and output spike trains only. The contributions of this paper are threefold. First, we address the problem of effective input selection by using a group LASSO regularization method in the analysis of the time-varying dynamics within a neural system. Second, in order to estimate the time-varying kernels quickly and accurately, we project the time-varying parameters in the sparse model onto a set of multiwavelet-based basis functions so that the time-varying parameters estimation problem can be tackled within the time-invariant regression scheme. Third, we utilize an effective forward orthogonal regression (FOR) aided by the mutual information (MI) criterion (FOR-MI) algorithm to determine and refine the model structure. The redundant model terms or regressors will be removed, and thus a compact model with good generalization performance is achieved. Particularly, the proposed sMGLV modeling approach can produce an optimal model structure to characterize the time-varying dynamical processes underlying the spiking system. The novel sMGLV framework integrates the group LASSO regularization algorithm and the multiwavelet basis expansion method in a unified framework to obtain an efficient and accurate estimation for the nonstationary spiking system. An obvious advantage of the proposed framework lies in its ability to track rapidly and even sharply time-varying processes on the condition that the neurons are not fully connected. Extensive simulation results demonstrate that the sMGLV model outperforms the existing state-of-the-art modeling methods in the identification of sparse time-varying neural connectivity. Redundant inputs can be effectively eliminated while more reliable kernel estimations can be obtained from a group of neural spiking signals. Experimental results show that the proposed method can track the changes of the functional connectivity in a real retinal neural spike train signals accurately.

2. Methodology

The schematic diagram for the proposed sMGLV framework is shown in Figure 1. The sMGLV model is implemented by using the Laguerre expansion of the kernel technique, the group LASSO regularization method, and the multiwavelet expansion scheme based on the time-varying Volterra series and generalized linear model, respectively. Furthermore, an effective FOR-MI algorithm provides an optimal model structure, where the expansion parameters are obtained with a maximum likelihood estimation method. Finally, the time-varying Volterra kernels are reconstructed for a biologically plausible description of the neural dynamics.

2.1. Time-Varying Generalized Volterra Model

Neural spiking activities can be viewed as point processes, represented as binary signals [35]. That is, 1 in the time series corresponds to when the spike is observed at the predefined time interval, and 0 otherwise [36]. The spiking activity of an output neuron depends on not only the current activities of other neurons, but also the spiking histories, which requires a dynamical model to capture a causal relationship [34]. Given the history of the input and output signals, the conditional probability of the action potential of the output signal can be expressed as follows [34]:

$$\theta \left(t\right)=\mathrm{Prob}\left(y\left(t\right)=1\left|X\left(t-\tau \right),0\le \tau \le M\right.\right)=\mathrm{E}\left(y\left(t\right)\left|X\left(t-\tau \right),0\le \tau \le M\right.\right),$$

(1)

where y denotes the output signal, X represents the set of all the equivalent input signals and M is the memory length in the system. Note that $X\left(t\right)={\left(x_1,x_2,\cdots ,x_{N_0}\right)}^T$, which contains the output signal y itself for the purpose of including the autoregressive effect of the preceding activities, namely, the total number for the input signals is $N_0-1$, and the autoregressive effect from the preceding output can be written as $x_{N_0}$ for simplicity (

Table 1. Table of notations.

Symbols	Meanings
y	Output signal
X	Equivalent input signals set
M	Memory length of signals
$g(·)$	Link function of the model
$k_0$	Zeroth-order kernel function
$k^{\left(n\right)}(t,\tau )$	First-order kernel function of the n-th input at lag $\tau$
$c_l^{\left(n\right)}\left(t\right)$	Time-Varying Laguerre expansion coefficient of the l-th basis function of the n-th kernel
$b_l$	The l-th order Laguerre basis function
L	Number of Laguerre basis functions
$\beta$	Laguerre parameter
P	Composite penalty function
$\lambda$	Regularization parameter
${\alpha }_{l,j,k}^{r_n,m}$	Time-invariant multiwavelet expansion coefficients
$B_m$	The m-th order Bspline basis function
j	The scale parameter of Bspline basis function
m	The order parameter of Bspline basis function
p	The number of selected significant model terms

).

The input–output dynamics is represented by a multi-input, multi-output (MIMO) framework, which is decomposed into a series of structurally plausible multi-input, single-output (MISO) models. The structure for the time-varying MIMO and MISO neural systems for spiking data can be given in Figure 2. Note that a population of neurons in the network tend to be sparsely connected. An output is typically innervated by only a small subset of neurons in its input region and thus exhibits a high level of sparsity in nature.

The Voterra series expression has been widely used to represent the feedforward and feedback kernels in the context of the spiking neural system [34]. To describe the causal relationship between the input and output spiking probability, the generalized linear model with time-varying Volterra series is utilized as follows:

$$g\left(\theta \left(t\right)\right)=k_0\left(t\right)+{\displaystyle \sum _{n=1}^{N_0}}{\displaystyle \sum _{\tau =0}^M}{k}^{\left(n\right)}\left(t,\tau \right)x_n\left(t-\tau \right),$$

(2)

where $k_0$ is a scalar zeroth-order kernel function, and $k^{\left(n\right)},n=1,2,\cdots ,N$ are first-order kernel functions describing the relationship between the output spiking probability and the n-th input. Here, $g\left(·\right)$ is a known probit link function, which can be interpreted as a noisy threshold function that transforms the pre-threshold membrane potential to output spikes [36]. With the probit link function, Equation (2) can be rewritten in the form of the normal cumulative distribution function as follows [34]:

$$\theta \left(t\right)=0.5-0.5\times \mathrm{erf}\left\{\frac{-k_0\left(t\right)-{\displaystyle \sum _{n=1}^{N_0}}{\displaystyle \sum _{\tau =0}^M}{k}^{\left(n\right)}\left(t,\tau \right)x_n\left(t-\tau \right)}{\sqrt{2}}\right\},$$

(3)

where the function erf is the Gaussian error function that can be defined by

$$\mathrm{erf}\left(q\right)=\frac{2}{\sqrt{\pi }}{\displaystyle \underset{0}{\overset{q}{\int }}}{e}^{-t^2}dt.$$

(4)

2.2. Time-Varying Generalized Laguerre–Volterra Model

The memory length in the neurons may last for hundreds of milliseconds. If M is large, thousands of unknown coefficients have to be estimated for each input kernel in Equation (2). To alleviate computational complexity, Laguerre expansion decomposes the kernel functions $k^{\left(n\right)}\left(t,\tau \right)$ into a set of weighted summation of the predefined orthogonal Laguerre basis functions:

$$k^{\left(n\right)}\left(t,\tau \right)\approx {\displaystyle \sum _{l=1}^L}{c}_l^{\left(n\right)}\left(t\right)b_l\left(\tau \right),0\le \tau \le M,$$

(5)

where $b_l(l=1,2,\cdots ,L$) represents the l-th order Laguerre basis function, and $c_l^{\left(n\right)}$ denote time-varying Laguerre expansion coefficients of the n-th kernels. The total number of basis functions L has been chosen in the range of 3 to 15 based on previous experimental analyses [37]. L also determines the complexity for the Laguerre expansion model. Specially, a larger L can give rise to more model terms and increase the computationally complex, while a more accurate and good generalization model will be produced. The Laguerre basis function b employed here can be obtained iteratively by [38]:

$$b_l\left(\tau \right)=\left\{\begin{array}{c}\sqrt{{\beta }^T\left(1-\beta \right)},l=0,\tau \ge 0,\hfill \\ \sqrt{\beta }{b}_{l-1}\left(0\right),l>0,\tau =0,\hfill \\ \sqrt{\beta }\left[b_l\left(\tau -1\right)+b_{l-1}\left(\tau \right)\right]-b_{l-1}\left(\tau -1\right),l>0,\tau >0,\hfill \end{array}\right.$$

(6)

where $\beta$ is the Laguerre parameter ($0<\beta <1$), which determines the exponential decaying rate of the Laguerre basis functions. The bigger the parameter $\beta$ is, the slower the Laguerre basis function decays, and a longer spread of significant non-zero values can be seen in the basis function [37]. The system kernels with longer memory may require a larger $\beta$ for efficient representation. Therefore, the proper selection of the parameter $\beta$ and L can effectively represent the time-varying system coefficients. Figure 3 shows the first 15 Laguerre basis functions, where the Laguerre parameters L equals 15 and $\beta$ equals 0.7. The basis function expands the entire system memory $[0,M]$.

With input spike trains in X convolved with b, the convolved products v are represented by

$$v_l^{\left(n\right)}\left(t\right)=b_l\left(t\right)\ast x_n\left(t\right).$$

(7)

The time-varying generalized Volterra model in Equation (2) can thus be rewritten by

$$g\left(\theta \left(t\right)\right)\approx c_0\left(t\right)+{\displaystyle \sum _{n=1}^{N_0}}{\displaystyle \sum _{l=0}^L}{c}_l^{\left(n\right)}\left(t\right)v_l^n\left(t\right),$$

(8)

where $\left\{c_0\left(t\right),c_1^{\left(1\right)}\left(t\right),\cdots ,c_L^{\left(1\right)}\left(t\right),\cdots ,c_L^{\left(N_0\right)}\left(t\right)\right\}$ represent the coefficients of the Laguerre expansion of the respective kernels, as functions of time. The model described in Equation (8) forms the time-varying generalized Laguerre–Volterra (TVGLV) model. Given that the order of the Laguerre basis functions L is usually much smaller than the memory length M, the number of time-varying coefficients to be estimated can be largely reduced after the Laguerre expansion procedure.

2.3. Functional Sparsity Based on a Group Regularization Method

Nodes in biological neural network are not fully connected [26]. Namely, only a small subset of the given multiple input signals may connect to the signal recorded from the output neuron. However, a full Volterra kernel model constructed with all the observed signals described in Equation (2) or (8) does not capture the sparsity yet. Therefore, functional sparsity is introduced to the time-varying generalized Laguerre–Volterra model. Using a regularization approach, the functional sparsity can select the significant inputs by enforcing the estimated kernels of the redundant inputs to be zeroes.

Regularization approaches, including LASSO [30], SCAD [31], group LASSO [32] and group bridge [33], have been widely applied to simultaneous parameter estimation and variable selection. The former two methods can achieve individual coefficient selection, while the latter two can produce a sparse set of groups. In the TVGLV model, the Laguerre approximation coefficients can be grouped with regard to the associated input signals, for example, $\{c_1^{\left(n\right)}\left(t\right),c_2^{\left(n\right)}\left(t\right),\cdots ,c_L^{\left(n\right)}\left(t\right)\}$ can be assigned to the same group because they all belong to the n-th input. Given this group setting, functional sparsity then refers to the occurrence that the estimation for several groups of Laguerre expansion coefficients all remain zero-valued. Therefore, a group LASSO regularization method based on the minimization of the sum of squared errors is utilized for a sparse TVGLV model. The group LASSO penalty term can be consisted as a hybrid of $L_2$-penalty and $L_1$-penalty: within the groups, it shrinks (but does not select) the coefficients with the $L_2$-norm regularization; across groups, it selects the groups in a $L_1$-norm regularization fashion. Specifically, the expression for the group LASSO penalty utilized here can be given as:

$$P_2^1={\displaystyle \sum _{n=1}^{N_0}}{{∥{\overline{c}}_l^{\left(n\right)}∥}_2^1={\displaystyle \sum _{n=1}^{N_0}}\left({\displaystyle \sum _{l=1}^L}{\left({\overline{c}}_l^{\left(n\right)}\right)}^2\right)}^{\frac{1}{2}},$$

(9)

where the subscript “2” is the $L_2$-norm regularization within the groups, and the super-script “1” represent the $L_1$-norm regularization across the groups. ${\overline{c}}_l^{\left(n\right)}$ represents the mean value of time-varying Laguerre expansion coefficients during the whole data length, and can be estimated by using the maximum likelihood estimation method in Equation (8). The group LASSO regularization criterion can thus be represented by:

$$C=arg{\displaystyle \underset{\lambda }{min}}\left\{{\displaystyle \sum _{t=1}^T}{\left[g\left(\theta \left(t\right)\right)-c_0-{\displaystyle \sum _{n=1}^{N_0}}{\displaystyle \sum _{l=1}^L}{\overline{c}}_l^{\left(n\right)}{v}_l^n\left(t\right)\right]}^2+\lambda P_2^1\right\},$$

(10)

where $\lambda >0$ is a regularization parameter that controls the sparsity degree of the model, i.e., a larger value of $\lambda$ yields sparser results of the coefficients. A wide range of $\lambda$ from ${10}^{-5}$ to ${10}^{-1}$ is explored. For each $\lambda$, deviance is obtained from a cross-validation strategy. The $\lambda$ with the smallest deviance and the corresponding set of C are used to find out the spurious inputs. For example, a redundant input can be determined if all the Laguerre expansion coefficients belonging to its group equal to zeroes.

Suppose that the selected Laguerre expansion coefficients are $\{c_0\left(t\right),c_1^{\left(r_1\right)}\left(t\right),\cdots ,c_L^{\left(r_1\right)}\left(t\right),\cdots ,c_L^{\left(r_N\right)}\left(t\right)\}$, $\left\{r_1,r_2,\cdots ,r_N\right\}\subset \left\{1,2,\cdots ,N_0\right\}$, the associated inputs can thus be expressed as $x_{r_1},x_{r_2},\cdots ,x_{r_N}$. According to Equation (8), the sparse TVGLV model can be written as follows:

$$g\left(\theta \left(t\right)\right)\approx c_0\left(t\right)+{\displaystyle \sum _{n=1}^N}{\displaystyle \sum _{l=0}^L}{c}_l^{\left(r_n\right)}\left(t\right)v_l^{r_n}\left(t\right).$$

(11)

By means of the group LASSO regularization method, the functional global sparsity can be achieved and the neural connectivity can thus be accurately represented. Furthermore, time-varying parameters can be efficiently estimated by using the following multiwavelet-based expansion scheme.

2.4. Sparse Multiwavelet-Based Generalized Laguerre–Volterra Model

Previous work has illustrated that expanding time-varying coefficients onto a set of basis functions can largely improve the tracking performance of the time-varying model and reduce the model complexity because the TVGLV model can be transformed into a time-invariant regression model [19]. The multiwavelet basis functions are usually employed to approximate the time-varying parameters because of their good approximation performance in tracking both fast and slowly varying trends regardless of the a priori knowledge of nonstationary processes. Thus, we can project time-varying Laguerre expansion coefficients in the sparse TVGLV model onto a set of multiwavelet basis functions, which can be yielded the following formula:

$$\begin{array}{c}{c}_0\left(t\right)={\displaystyle \sum _m}{\displaystyle \sum _{k\in {\Gamma }_m}}{\alpha }_{j,k}^{0,m}{\psi }_{j,k}^m\left(\frac{t}{T}\right),\hfill \\ c_l^{\left(r_n\right)}\left(t\right)={\displaystyle \sum _m}{\displaystyle \sum _{k\in {\Gamma }_m}}{\alpha }_{l,j,k}^{r_n,m}{\psi }_{j,k}^m\left(\frac{t}{T}\right),\hfill \end{array}$$

(12)

where ${\psi }_{j,k}^m\left(\frac{t}{T}\right),t=1,2,\cdots ,T$ are the wavelet basis functions, with the shift indices $k\in {\Gamma }_m$, ${\Gamma }_m=\left\{k:-m\le k\le 2^j-1\right\}$, wavelet scale j and the wavelet order m, respectively. $\left\{{\alpha }_{j,k}^{0,m},{\alpha }_{l,j,k}^{r_n,m}\right\}$, with $n=1,2,\cdots ,N,l=1,2,\cdots ,L$ are the time-invariant coefficients in the multiwavelet-based model expansion. T is the length of observed data. Multiwavelet basis functions are combined with a series of wavelet basis functions from different orders. It should be noted that wavelet basis functions of lower orders tend to perform well on the change of signals with sharp transients, while the wavelet basis functions of higher orders tend to perform well on the change of signals smoothly. Multiwavelet basis functions can thus be adopted to approximate time varying coefficients with both rapid and slow variations. Considering the cardinal B-spline basis functions to be completely supported with excellent approximation properties [17], ${\psi }_{j,k}^m\left(·\right)$ can take the form of the B-spline basis function with the m-th order ($B_m$):

$${\psi }_{j,k}^m\left(x\right)=2^{j/2}{B}_m\left(2^{j}x-k\right),$$

(13)

where the variable $x=t/\phantom{tT}T$ is normalized to $[0,1]$ and shift indices k satisfies $-m\le k\le 2^j-1$. The scale j determines the resolution, namely, a larger j leads to a higher resolution, and also a bigger of the computation complexity. For most nonlinear dynamical modelling problems for the neural spiking data, scale j of 3, and the wavelets of order selected from $\left\{{\psi }_{j,k}^m:m=2,3,4,5\right\}$ are usually adequate, which can be graphically illustrated in Figure 4. The detailed discussions on the selection of proper scale j and the associated wavelet order can be found in [39].

By substituting Equation (13) into Equation (11), the sparse multiwavelet-based generalized Laguerre- Volterra model is given below:

$$g\left(\theta \left(t\right)\right)\approx {\displaystyle \sum _m}{\displaystyle \sum _{k\in {\Gamma }_m}}{\alpha }_{j,k}^{0,m}{\psi }_{j,k}^m\left(\frac{t}{T}\right)+{\displaystyle \sum _{n=1}^N}{\displaystyle \sum _{l=1}^L}{\displaystyle \sum _m}{\displaystyle \sum _{k\in {\Gamma }_m}}{\alpha }_{l,j,k}^{r_n,m}{\psi }_{j,k}^m\left(\frac{t}{T}\right)v_l^{\left(r_n\right)}\left(t\right),$$

(14)

where $\left\{{\alpha }_{j,k}^{0,m},{\alpha }_{l,j,k}^{r_n,m}\right\}, n=1,2,\cdots ,N,l=1,2,\cdots ,L$, are the time-invariant coefficients to be estimated. For simplification, new variables can be defined by:

$$\begin{array}{c}{V}_{j,k}^{0,m}\left(t\right)={\psi }_{j,k}^m\left(\frac{t}{T}\right),\hfill \\ V_{l,j,k}^{r_n,m}\left(t\right)={\psi }_{j,k}^m\left(\frac{t}{T}\right)v_l^{\left(r_n\right)}\left(t\right).\hfill \end{array}$$

(15)

Substituting Equation (15) into Equation (14) obtains

$$g\left(\theta \left(t\right)\right)\approx {\displaystyle \sum _m}{\displaystyle \sum _{k\in {\Gamma }_m}}{\alpha }_{j,k}^{0,m}{V}_{j,k}^{0,m}\left(t\right)+{\displaystyle \sum _{n=1}^N}{\displaystyle \sum _{l=1}^L}{\displaystyle \sum _m}{\displaystyle \sum _{k\in {\Gamma }_m}}{\alpha }_{l,j,k}^{r_n,m}{V}_{l,j,k}^{r_n,m}\left(t\right),$$

(16)

where $\left\{{\alpha }_{j,k}^{0,m},{\alpha }_{l,j,k}^{r_n,m}\right\},n=1,2,\cdots ,N,l=1,2,\cdots ,L$ are the full model terms in the sparse mutliwavelet-based TVGLV expansion model. Hence, the original time-varying model described in Equation (11) is now transformed into a time-invariant regression model, which enables the coefficient estimation problem to be solved within the maximum-likelihood framework.

2.5. Model Structure Selection

The multiwavelet-based basis function expansion model in Equation (16) may involve myriad candidate terms or regressors if the model parameters L, J, m and N are large, which can easily result in an overfitting problem. Thus, determination of the significant terms or regressors to be included in the final identified model is crucial prior to estimating time-invariant expansion coefficients.

LASSO-related penalized regression methods are popular for variables or terms selection studies [30,40,41]. However, the LASSO algorithm cannot guarantee consistency in the model term selection, and noisy terms can be occasionally selected [42]. Nonetheless, the computation of such methods is demanding considering that the number for the candidate terms in the expansions is large [42]. By contrast, the orthogonal-least-squares-type methods can obtain unbiased estimates [43]. The forward orthogonal regression (FOR) algorithm aided by mutual information criterion (MI) (FOR-MI) is an efficient and widely-used method in respect of model structure selection [44]. Thus, the FOR-MI method has been adopted to select significant model terms in the neural dynamical system from spike train data, which can effectively achieve local sparsity for the model structure. Consider two random discrete variables $X_0$ and $Y_0$, the MI criterion is used to measure the reduction in the uncertainty of $Y_0$ due to the knowledge of $X_0$, which can be defined as [45]:

$$I\left(X_0,Y_0\right)={\displaystyle \sum _{x\in X}}{\displaystyle \sum _{y\in Y}}\mathrm{P}\left(X_0,Y_0\right)\log \left(\frac{P\left(X_0,Y_0\right)}{P\left(X_0\right)P\left(Y_0\right)}\right),$$

(17)

where $P\left(X_0\right)$ and $P\left(Y_0\right)$ represent the marginal probability density functions, and $P\left(X_0,Y_0\right)$ denotes the joint probability density function.

Given the binary characteristics of input and output data, the model structure detection algorithm based on the full model in Equation (16) can be described as follows:

Step 1. Combine the output observation y and the full model candidates $\left\{V_{j,k}^{0,m}\left(t\right),V_{l,j,k}^{r_n,m}\left(t\right)\right\}$, the maximum likelihood estimator ${\widehat{\alpha }}_{mle}$ can be estimated by using a generalized linear model fitting method. The output spike probability $g\left(\theta \left(t\right)\right)$ can further be obtained using Equation (16). Assume $G\left(t\right)=g\left(\theta \left(t\right)\right),t=1,2,\cdots ,T$, to be a vector of the measured outputs with T time instants.

Step 2. The model terms $v_i\left(t\right)\in V_1=\left\{V_{j,k}^{0,m}\left(t\right),V_{l,j,k}^{r_n,m}\left(t\right)\right\},i=1,2,\cdots ,M$ are candidates for the first selected significant terms ${\phi }_1\left(t\right)$. Let ${\phi }_1^{\left(i\right)}\left(t\right)=v_i\left(t\right),t=1,2,\cdots ,T,$ and

$${\ell }_1=\arg {\displaystyle \underset{1\le i\le M}{\max }}\left\{I\left(G,v_i\right)\right\}.$$

(18)

The first significant term can be selected as $v_{{\ell }_1}\left(t\right)$, and ${\phi }_1\left(t\right)={\phi }_1^{\left({\ell }_1\right)}\left(t\right)$ is selected with both $G_1\left(t\right)=G\left(t\right)-\frac{G\left(t\right){\phi }_1\left(t\right)}{{\phi }_1^2\left(t\right)}{\phi }_1\left(t\right)$ and $M{I}_1=I\left(G\left(t\right),v_{{\ell }_1}\left(t\right)\right)$. The error-to-signal ratio (ESR) is given as the search termination criterion below:

$${∥r_1∥}^2=\frac{{∥G_1∥}^2}{{∥G∥}^2}=\left({∥G∥}^2-\frac{{\left(G{\phi }_1\right)}^2}{\left({\phi }_1^2\right)}\right)/{∥G∥}^2.$$

(19)

Step 3. The rest of the model terms $V_2=v_i\left(t\right),i=1,2,\cdots ,M,i\ne {\ell }_1$ form the candidate terms for ${\phi }_2\left(t\right)$. Let

$${\phi }_2^{\left(i\right)}\left(t\right)=v_i\left(t\right)-\frac{{\phi }_1\left(t\right)v_i\left(t\right)}{{\phi }_1^2\left(t\right)}{\phi }_1\left(t\right),i=1,2,\cdots ,M,i\ne {\ell }_1,$$

(20)

and we can obtain

$${\ell }_2=\arg {\displaystyle \underset{\genfrac{}{}{0pt}{}{1\le i\le M\hfill }{i\ne {\ell }_1\hfill }}{\max }}\left\{I\left(G_1,v_i\right)\right\}.$$

(21)

Then, the second term can be selected as $v_{{\ell }_2}\left(t\right)$, and ${\phi }_2\left(t\right)={\phi }_2^{\left({\ell }_2\right)}\left(t\right)$ is selected with both $G_2\left(t\right)=G_1\left(t\right)-\frac{G_1\left(t\right){\phi }_2\left(t\right)}{{\phi }_2^2\left(t\right)}{\phi }_2\left(t\right)$ and $M{I}_2=I\left(G_1\left(t\right),v_{{\ell }_2}\left(t\right)\right)$. ESR is represented by

$${∥r_2∥}^2=\frac{{∥G_2∥}^2}{{∥G∥}^2}=\left({∥G_1∥}^2-\frac{{\left(G_1{\phi }_2\right)}^2}{\left({\phi }_2^2\right)}\right)/{∥G∥}^2.$$

(22)

Step 4. Subsequent significant terms can be selected in the same way step by step until terminated at the p-th step when some certain criteria is satisfied. A generalized cross-validation (GCV) is used to determine model size, which can be expressed as [44]:

$$\mathrm{GCV}\left(p\right)={\left(\frac{T}{T-p}\right)}^2\mathrm{MSE}\left(p\right),$$

(23)

where $\mathrm{MSE}\left(p\right)={∥r_p∥}^2/\phantom{{∥r_p∥}^{2}p}p$ is the mean square error (MSE) corresponding to the optimal model with the p significant terms selected. The effective number of model terms can thus be obtained with a minimum GCV value. Through the model structure detection procedure, the significant model terms can be selected from a large number of redundant model terms and therefore a parsimonious model can be obtained.

2.6. Parameter Estimation and Tuning Parameter Optimization

Assuming that the significant model terms selected using the FOR-MI algorithm can be given as $\varphi =\left\{{\phi }_1,\cdots ,{\phi }_p\right\}$, and the associated time-invariant expansion coefficients $\alpha =\left\{{\alpha }_1,\cdots ,{\alpha }_p\right\}$ can be easily estimated by using a maximum-likelihood method. The log likelihood function can be expressed as follows:

$$\begin{array}{c}L\left(y|X\right)={\displaystyle \prod _{t=0}^T}\mathrm{Prob}\left(y\left(t\right)|X\left(t-\tau \right),0\le \tau \le M\right)={\displaystyle \prod _{t=0}^T}\theta {\left(t\right)}^{y\left(t\right)}{\left(1-\theta \left(t\right)\right)}^{1-y\left(t\right).}\hfill \end{array}$$

(24)

The log-likelihood can thus be defined by:

$$l\left(y|X\right)={\displaystyle \sum _{t=1}^T}\left\{y\left(t\right)log\theta \left(t\right)+\left(1-y\left(t\right)\right)log\left(1-\theta \left(t\right)\right)\right\}.$$

(25)

Plugging the selected model terms $\varphi$ to approximate $\theta \left(t\right)$ in the above equation yields an approximation of l. The criterion is a function of basis function coefficients $\alpha =\left\{{\alpha }_1,\cdots ,{\alpha }_p\right\}$ and denoted by $l\left(\alpha \right)$. The maximizer of $l\left(\alpha \right)$ is the maximum likelihood estimator (MLE) that is represented as $\left\{{\tilde{\alpha }}_1,\cdots ,{\tilde{\alpha }}_p\right\}$, which can be calculated using the standard iterative reweighted least-squares method.

The optimization of the tuning parameters, including the $\beta ,L,J,m$, are critical for achieving an accurate and efficient model. The well-known Bayesian information criterion (BIC) is employed to determine the proper tuning parameters, which can be calculated as follows:

$$\mathrm{BIC}\left(\beta ,L,J,m\right)=-2l\left(\alpha \left(\beta ,L,J,m\right)\right)+\frac{ln\left(T\right)}{T}p,$$

(26)

where $\alpha \left(\beta ,L,J,m\right)$ is the estimated coefficients corresponding to a certain set of tuning parameters. The optimal tuning parameters of the proposed sparse multiwavelet-based generalized Laguerre–Volterra model can be determined with the minimum of the BIC value.

2.7. Kernel Reconstruction and Interpretation

With the proper model terms selected and the corresponding time-invariant expansion coefficients estimated, a parsimonious model can therefore be achieved through a kernel reconstruction and normalization procedure.

The Laguerre expansion coefficients $\{{\tilde{c}}_0\left(t\right),{\tilde{c}}_1^{\left(r_1\right)}\left(t\right),\cdots ,{\tilde{c}}_L^{\left(r_1\right)}\left(t\right),\cdots ,{\tilde{c}}_L^{\left(r_N\right)}\left(t\right)\}$ can be calculated using Equation (12). The Voterra kernels for the significant inputs can thus be reconstructed as follows:

$$\begin{array}{c}{\tilde{k}}_0\left(t\right)={\tilde{c}}_0\left(t\right),\hfill \\ {\tilde{k}}^{\left(n\right)}\left(t,\tau \right)={\displaystyle \sum _{l=1}^L}{\tilde{c}}_l^{\left(n\right)}\left(t\right)b_l^{\left(n\right)}\left(\tau \right),n=r_1,r_2,\cdots ,r_N.\hfill \end{array}$$

(27)

Towards the interpretation, the kernel functions can further be normalized by the zeroth-order kernel as

$${\widehat{k}}^{\left(n\right)}\left(t,\tau \right)={\tilde{k}}^{\left(n\right)}\left(t,\tau \right)/\phantom{{\tilde{k}}^{\left(n\right)}\left(t,\tau \right){\tilde{k}}_0\left(t\right)}{\tilde{k}}_0\left(t\right),t=1,2,\cdots ,T,$$

(28)

where the threshold for the spike generating is set to be 1, and the ${\widehat{k}}_0\left(t\right)$ was set to be −1.

As to the interpretation for the reconstructed model, $\theta \left(t\right)$ is the conditional probability of the action potential of the output signal, and $g\left(\theta \left(t\right)\right)$ with a probit link function can be regarded as the membrane potential of the output neuron. ${\widehat{k}}^{\left(n\right)}\left(t,\tau \right)$ is the postsynaptic potential in the output neuron elicited by the n-th input neuron when n is within $[0,N_0-1]$, and the output neuron itself when n equals to $N_0$. Inputs with all zero-valued kernels is identified as the redundant inputs.

3. Simulation Studies

Synthetic spike train data was generated from known systems to study the tracking performance of both the rapid and gradual changing kernels using the proposed sMGLV approach. Simulated system shown in Figure 5, Figure 6 and Figure 7 is an 8-input and 1-output spiking neural system where the first four kernel functions ($k^{\left(1\right)},k^{\left(2\right)},k^{\left(3\right)},k^{\left(4\right)}$) undergo step changes and the other two kernels ($k^{\left(5\right)},k^{\left(6\right)}$) undergo slower gradual changes concurrently. To make the tracking task more difficult, the last two kernels ($k^{\left(7\right)},k^{\left(8\right)}$) are designed to be zeroes as redundant inputs to the spiking neural system. The transient changes for all step changing kernels occur at 400 s. In particular, the amplitudes of $k^{\left(1\right)}$ and $k^{\left(2\right)}$ are doubled, while the amplitudes of $k^{\left(3\right)}$ and $k^{\left(4\right)}$ are decreased by half. To test the concurrent tracking performance in identifying gradual changes, $k^{\left(5\right)}$ and $k^{\left(6\right)}$ are linear time-varying kernels with different slopes, as shown in Figure 6 and Figure 7. Additionally, the feedback kernel from the preceding output signal can be denoted by $k^{\left(9\right)}$, and is set to be time-invariant in this simulation. The zeroth-order kernel, i.e., the baseline, is set as a constant 0. Specific shapes for different kernels are shown in Figure 5. The input signals are eight independent Poisson random spike trains with a mean firing rate of 10 Hz. The simulation length is 800 s with a bin size of 2 ms. Given the input spike trains and the known kernels defined in this simulation, the conditional probability of the output signal can be generated by using Equation (3), which can be used to produce the output spike trains at different times through a Bernoulli process.

Using the input and output spike train data, kernel functions can be estimated by the proposed sMGLV method. As a comparison, the tracking result of a direct multiwavelet-based modelling method (dMGLV) without removing the redundant inputs is also provided to estimate the time-varying dynamical system in order to verify the effectiveness of the proposed approach. The tuning parameters utilized are the same for two compared methods according to Equation (27). Figure 5 illustrate the recovered kernel functions during different time courses using the dMGLV model and the proposed sMGLV model. The x-axis in the figure represents the time lag $\tau$ between the previous input events and the present time, and the y-axis indicates the amplitude of kernels. The true kernels are shown in black lines, and the estimated kernels over different time evolution can be displayed in different colors. Specifically, Figure 5a represent the estimated kernels in the first half of the simulation time course (the first 400 s) using two different methods, while Figure 5b represent the estimated kernels in the second half of the simulation time evolution (the last 400 s) using two different methods. The experiment results clearly indicate that the dMGLV model cannot capture global sparsity, that is, unable to recognize the redundant inputs. The dMGLV modeling method estimates spurious inputs even for zero-valued kernel functions, which is shown in Figure 5. In contrast, the proposed sMGLV model can accurately recognize all redundant inputs. As for the non-zero kernel functions, both dMGLV and sMGLV modeling methods can estimate the accurate kernel shapes accurately. Particularly, the proposed sMGLV model can effectively eliminate the redundant inputs, and therefore achieve a more reliable and accurate kernel estimation for the neural spiking system.

We have also compared the tracking performance for both the rapid and gradual changing kernels in the whole simulation data length. As shown in Figure 6 and Figure 7, apart from the proposed method, the stochastic state point process filtering algorithm (SSPPF), SSPPF with the group LASSO algorithm (sSSPPF), and the dMGLV model are also utilized for comparison. Specifically, Figure 6 shows the actual and estimated kernels over the time evolution, where the x-axis denotes the simulation time, the y-axis represents the time lag $\tau$ lasting from the previous input events to the present time, and the amplitudes of the kernel functions are color-coded. Additionally, Figure 7 illustrates the tracking results of the peak values of the estimated nonstationary feedforward and time-invariant feedback kernels using four different methods, where the x-axis represents simulation time course, and the y-axis indicates the kernel amplitudes for different spike train signals. The goodness-of-fit is quantitatively evaluated by comparing the mean absolute error (MAE) and normalized root mean square error (RMSE). The MAE and RMSE of the kernel estimates for the simulation spike train data, with respect to the corresponding true values, are estimated and shown in

Table 2. A performance comparison of kernel estimates for the simulation examples using different approaches.

Performance Indicators	Estimated Kernels	Identification Methods
		SSPPF	sSSPPF	dMGLV	sMGLV
MSE	${\widehat{k}}^{\left(1\right)}$	0.1409	0.1310	0.0592	0.0536
	${\widehat{k}}^{\left(2\right)}$	0.0453	0.0435	0.0330	0.0259
	${\widehat{k}}^{\left(3\right)}$	0.0980	0.0922	0.0483	0.0416
	${\widehat{k}}^{\left(4\right)}$	0.0911	0.0882	0.0521	0.0408
	${\widehat{k}}^{\left(5\right)}$	0.0525	0.0529	0.0260	0.0217
	${\widehat{k}}^{\left(6\right)}$	0.0405	0.0387	0.0222	0.0162
	${\widehat{k}}^{\left(7\right)}$	0.0431	0.0000	0.0250	0.0000
	${\widehat{k}}^{\left(8\right)}$	0.0467	0.0000	0.0371	0.0000
	${\widehat{k}}^{\left(9\right)}$	0.0317	0.0288	0.0078	0.0078
NRMSE	${\widehat{k}}^{\left(1\right)}$	0.1987	0.1894	0.0847	0.0791
	${\widehat{k}}^{\left(2\right)}$	0.2315	0.2243	0.1472	0.1187
	${\widehat{k}}^{\left(3\right)}$	0.2337	0.2177	0.1101	0.0991
	${\widehat{k}}^{\left(4\right)}$	0.2307	0.2229	0.1212	0.0952
	${\widehat{k}}^{\left(5\right)}$	0.2848	0.2821	0.1270	0.1137
	${\widehat{k}}^{\left(6\right)}$	0.2370	0.2316	0.1284	0.0875
	${\widehat{k}}^{\left(7\right)}$*	-	-	-	-
	${\widehat{k}}^{\left(8\right)}$*	-	-	-	-
	${\widehat{k}}^{\left(9\right)}$	0.1634	0.1533	0.0425	0.0384

* The NRMSE does not exist considering that the actual kernel values are designed to be zeros.

. Compared with the SSPPF, sSSPPF and the dMGLV model, the MAE and RMSE for the proposed sMGLV are much smaller. The MAE and RMSE of the peak amplitudes for the kernel are defined by

$$\mathrm{MAE}=\frac{1}{T}{\displaystyle \sum _{t=1}^T}|\widehat{K}\left(t\right)-K\left(t\right)|,$$

(29)

$$\mathrm{NRMSE}=\sqrt{\frac{1}{T}{\displaystyle \sum _{t=1}^T}\frac{{∥\widehat{K}\left(t\right)-K\left(t\right)∥}^2}{{∥K\left(t\right)∥}^2,}}$$

(30)

where $K\left(t\right)$ and $\widehat{K}\left(t\right)$ represent the peak amplitudes of the actual and estimated kernels at different times, respectively. Table 2 statistically confirms the better performance of the proposed sparse multiwavelet-based TVGLV model. Compared with the conventional SSPPF, the SSPPF with the group LASSO and the direct multiwavelet-based modeling method, the above simulation results indeed show that the proposed approach possesses much higher accuracy in kernel estimates.

Experiment results show that four estimation methods can all track the nonstationary kernels with step or linear changes. Although the SSPPF can roughly track the changes of the significant kernels, its estimates for the insignificant inputs are poor due to its inability in identifying the redundant model inputs. By contrast, the sSSPPF method can firstly remove the redundant inputs and provide relatively satisfactory tracking results of time-varying kernels by eliminating the disruptions from the redundant inputs simultaneously. However, considering the deficiency of the convergence rate for the adaptive filter methods, the sSSPPF cannot accurately capture the transient properties of the jump or abrupt time-varying processes. The dMGLV method, which established under the multiwavelet-based expansion framework, outperforms the SSPPF-type methods in the respect of tracking results of time-varying kernels with both rapid and gradual changes over time evolution. Nevertheless, the parametric modeling method without considering the sparsity condition can also produce spurious kernels generated by the redundant model inputs. The proposed sMGLV can instead provide optimal estimations and be consistent with the identification of the sparse time-varying neural connectivity, where the redundant inputs can be effectively eliminated and more reliable kernel estimations can be obtained from a group of neural spiking signals. Therefore, the proposed sMGLV framework can remarkably improve the tracking performance of time-varying kernels from time-varying dynamical systems based on spiking activities, and can be generally applied to the analysis for the real nonstationary neural functional connectivity.

The estimated model under the proposed sGLMV framework can be further validated using the correlation plot for the estimated and actual output spike train. Figure 8a shows the estimated and actual output spike trains based on identical input spiking signals. The pre-threshold membrane potential was firstly calculated from the input spike trains (X) and the estimated kernels ($\widehat{k}$) using the proposed sMGLV model. Firing probability intensity ($\theta$) for the output was then obtained using the Gaussian error function in Equation (3) (Figure 8a, first row). The estimated output spike train ($\widehat{y}$) is finally obtained through a threshold procedure (Figure 8a, second row). The corresponding actual spike train (y) is also given in the third row in Figure 8a. Considering the stochastic nature of the model, the similarities between the estimated output spike train and the actual output spike train can be quantified with correlation plot (Figure 8b). Noted that the point-process signals $\widehat{y}$ and y are firstly convolved with a Gaussian kernel with standard deviation ${\sigma }_g$, and thus smoothed to continuous signals ${\stackrel{⌢}{y}}_{{\sigma }_g}\left(t\right)$ and $y_{{\sigma }_g}\left(t\right)$, respectively. The correlation coefficient r is defined as follows [36]:

$$\begin{array}{c}\hfill r\left({\sigma }_g\right)=\left({\displaystyle \sum _{t=0}^T}{\stackrel{⌢}{y}}_{{\sigma }_g}\left(t\right)y_{{\sigma }_g}\left(t\right)\right)/\sqrt{\left({\displaystyle \sum _{t=0}^T}{\stackrel{⌢}{y}}_{{\sigma }_g}\left(t\right)y_{{\sigma }_g}\left(t\right)\right)\left({\displaystyle \sum _{t=0}^T}{y}_{{\sigma }_g}\left(t\right)y_{{\sigma }_g}\left(t\right)\right)}\end{array}$$

(31)

It shows that the proposed sMGLV model can generate output highly correlated with the actual output for a large range of standard deviation ${\sigma }_g$, and further validates the effectiveness of the proposed framework.

Results of additional intensive simulations have validated the advantages of the proposed sMGLV model over the state-of-the-art methods with respect to the identification of sparse time-varying neural connectivity. The proposed method can first select the significant inputs to avoid the estimation error produced by the redundant ones; then, both the gradual and rapid changes of the time-varying kernels can be tracked rapidly and accurately by using the sparse multiwavelet-based modeling framework.

4. Application to Functional Connectivity Analysis of Retinal Neural Spikes

The proposed sMGLV model can be generally applied to identify the functional connection from the experimental spiking train data. A set of spontaneous retinal spike train recordings has been used to illustrate and reveal the potential properties of the neural dynamics [46]. The dataset contains 512 electrodes recordings from retina of rats. The spontaneous spike trains were recorded simultaneously for 1000 s, with a sampling rate of 20 kHz and a bin size of 2 ms. In this experiment example, the system is composed of four inputs and one output. Utilizing the proposed modeling method, dynamics of four feedforward and one feedback kernels can be estimated during the time course.

Similar to the previous simulated example, the order for the B-splines basis function were adopted from 2 to 5 to construct the sMGLV model based on the spike trains. The tuning parameters are optimized using Equation (26), and thus both the optimal Laguerre parameter and the Laguerre basis number are 0.83 and 11, respectively. The regularization parameter in the group LASSO can be obtained using a cross validation method introduced in the Section C. A group LASSO with the B-splines expansion method is applied to the experimental spiking data to reveal the potential properties underlying the retinal neural system. The parsimonious model structure is produced with a FOR-MI algorithm and the expansion parameters are obtained with a maximum likelihood estimation method.

Figure 9 shows the reconstructed kernel results using the proposed sMGLV method over the time course. Specifically, the left figure in Figure 9 illustrates the estimated kernels at different time (i.e., 300 s, 400 s, 500 s, 600 s, 700 s), where the x-axis represents time lag $\tau$ between the previous input events and the present time, and the y-axis stands for the amplitudes of kernels. Experimental results show that the kernels for the first and second input spike train signals (i.e., $k^{\left(1\right)}$ and $k^{\left(2\right)}$) are nonzero, and thus are significant inputs, while the kernels of the third and fourth input spike train signals (i.e., $k^{\left(3\right)}$ and $k^{\left(4\right)}$) are yielded to be zero-valued for the whole system memory, and therefore deemed to be insignificant inputs. The sMGLV model can effectively identify the sparsity in the experimental system. Owing to eliminating the spurious inputs, the kernels for significant inputs can be estimated more accurately and are shown to keep the stable estimation results. The right figure in Figure 9 demonstrates the peak amplitudes and their corresponding stand deviation of different kernels across the experimental time in the spiking system, where the red lines represent the changes of the peak amplitudes over time and the green shade indicates the standard deviation for the amplitudes of the estimated kernels. Both of the peak amplitudes and their corresponding stand deviation can be easily derived based on the time-invariant expansion coefficients $\alpha$ through Equations (12), (27) and (28). The peak amplitudes describe the changes for the estimated kernels during the experimental time, while the standard deviation accounts for the fluctuation ranges for the peak values. The experimental results in the right figure illustrate that the peak amplitudes for the significant inputs (i.e., $k^{\left(1\right)}$ and $k^{\left(2\right)}$) keep varying over time, which indicates the significant time-varying connection strength. These results demonstrate the applicability of the proposed sMGLV model framework to the real experimental data.

5. Conclusions

A novel sMGLV modeling framework has been formulated for the identification of time-varying neural dynamics from spiking activities. The proposed framework has three distinct advantages compared to the state-of-the-art modeling methods. First, in terms of the sparse neural connectivity, the proposed sMGLV modeling framework combines the Laguerre expansion technique with the group LASSO algorithm to realize a global sparsity, where spurious inputs can be effectively eliminated by enforcing the Laguerre expansion coefficients to be zero. Second, the proposed sMGLV modeling scheme projects the time-varying coefficients onto a set of multiwavelet-based basis functions. By this means, the time-varying modeling problem can be converted into a time-invariant regression problem, and further the tracking performance of time-varying models is improved. Third, for the issue of model overfitting, the proposed sMGLV model employs a FOR-MI method to select the significant terms and removes the redundant ones efficiently, and a satisfactory sparse model with good generalization performance is finally produced. Numerical simulation verified the effectiveness of the proposed modeling scheme in identifying the changes of the time-varying kernels. Its application to a real retinal spike train recorded indicates that the proposed method can be applied to the real experimental data, and obtain a relatively faithful expression for the sparse functional connectivity among the neural population. Therefore, the proposed sMGLV model developed in this paper can provide an integrated computational framework for investigating nonstationary properties and track more general forms of dynamical changes in the sparsely connected neural network. It may better the understanding of the information transmission across brain regions during behavior.

Finally, it should be noted that the mutliwavelet-based basis expansion scheme proposed here could be generally applied to kernels with time-varying weights while using different basis functions, such as Kautz filters [47], to capture the kernel shapes.