1. Introduction
Mathematical model building, especially the construction of governing equations, is a significant problem for a large variety of dynamic systems, such as vibration systems [
1], biological systems [
2], and fluid mechanical systems [
3]. These models can be used for prediction, simulation, condition monitoring, optimization, etc. [
4]. The traditional methods to build a model are based on the basic knowledge in one or many categories. These types of model building are described as “bottom–up” [
5]. However, these methods work only with
a priori knowledge. In recent years, the rapid growth of computational capacity of computers has enabled researchers in several domains to build models with a “top–down” approach using acquired data instead of the domain knowledge [
5]. Many topics are related to this approach, such as System Identification (SI) and the development or improvement of structural models using the input and output data from real measurements [
6]. These methods are especially useful when the first principle knowledge is not discovered, for example, the macroscopic behavior of advanced materials [
7], in aerospace engineering [
8], in biology [
2,
9,
10], in economy [
11], in music [
12,
13], etc. The models of vibration systems can be identified with various approaches, such as iterative model updating [
14], the Polynomial NonLinear State Space (PNLSS) model [
15], Volterra series [
16], the Wiener and Hammerstein model [
17,
18], artificial neural networks [
19,
20], and equation-free model building [
21]. On the downside, these methods usually build “black-box” or “grey-box” models, which means the theoretical understanding of the system is totally or partially absent from the model [
4].
In the category of sparse regression, the SINDy framework (Sparse Identification of Nonlinear Dynamical Systems) shows a bright perspective in terms of interpretability. It is a technique to build the differential governing equation by extracting a few relevant terms from a library of candidate functions [
22]. The polynomial development of a time series
is assumed as candidate functions and the regression method
Least Squares method with Sequential Threshold (LSST, also called STLS [
23]) is suggested for sparse regression [
24]. In recent research, SINDy was applied to boundary value problems [
25], and advanced materials [
7]. Furthermore, SINDy is also applicable to Model Predictive Control (MPC) to reduce complexity and to enhance the interpretability of the integrated empirical model [
26,
27].
Many extensions have been adopted on the SINDy framework for different benefits. First, this framework can be modified using an enlarged library of candidate functions, such as trigonometric functions [
24], partial differential equations [
28,
29], and rational functions [
22]. Second, different methods of sparse regression can be applied. The methods, for example,
Least Absolute Shrinkage and Selection Operator (LASSO) [
30] and compressed sensing [
31,
32,
33], have been used to enforce sparsity before the development of SINDy. Recently, a broad range of regularized regression methods have been reformulated and generalized in the framework of Sparse Relaxed Regularized Regression (SR3) [
34]. Throughout this work, LSST and the
Least Squares method Post-LASSO (LSPL) [
35] will be discussed. Third, different signal processing methods are used to correct or transfer the measurement in order to make the solved problem better-conditioned, such as normalization [
29], Hankel Alternative View Of Koopman (HAVOK) [
36], and neural networks [
37].
Since the governing equation of a discrete vibration system and SINDy share the same mathematical model structure, the SINDy framework gains attention for identifying the governing equations of discrete vibration systems. The traditional governing equation of a discrete vibration system is an Ordinary Differential Equation (ODE) using factors related to mass,
m, stiffness,
k, and damping ratio,
d, which should be individually measured to build the governing equation. Nonlinear vibration systems can also be interpreted with similar concepts. For example, the Duffing oscillator has cubic stiffness in relation to displacement [
38]. Thus, a SINDy model of a vibration system not only predicts the dynamic behavior but also is interpretable with fundamental concepts. Furthermore, the results of identification can contribute to a deeper theoretical understanding of an unknown system. In summary, SINDy shows potential for identifying the governing equation of discrete vibration systems in a “top–down” and interpretable manner without individually acquiring the parameters
m,
k, or
d related to the domain knowledge of structural vibrations.
In this research, we explore the possibility of identifying the governing equation of an oscillator using the two sparse regression methods LSST and LSPL. We conduct numerical uncertainty analysis and experimental validation. The methods LSST and LSPL are introduced in detail in
Section 2. In
Section 3, we test SINDy on a numerical model of a single-mass oscillator. In order to simulate the true use case, noise signals at different levels are artificially added to the training data. At the same time, the highest polynomial order of the library varies from 1 to 5. The uncertainties of the results for both regression methods are obtained through Monte Carlo Simulation (MCS) with randomly generated parameters of the oscillator. LSST, the original sparse regression algorithm of SINDy, delivers less accurate sparsity and worse overfitting at increased noise levels. A similar statement was mentioned in a previous work, where the approximately noise-free measurement data is the requirement of SINDy [
37]. To improve the robustness of SINDy against noise, we suggest the method LSPL to replace the original LSST method. After the numerical uncertainty analysis, the modified SINDy method, SINDy-LSPL, is tested on a real test bench in
Section 4. In
Section 4 and
Section 5, we define a complete workflow from acquisition of the time series to identification and evaluation. Overall, in both numerical and experimental tests, we review how SINDy-LSPL performs at discovering the governing equation of a simple vibration system, which is fundamental for further research dealing with complex systems with more than one Degree of Freedom (DoF). Detailed conclusions and outlooks for this research are given in
Section 6.
2. Methods
The SINDy framework [
24] is based on differential polynomial and sparse regression. It is assumed that the equation of motion to be identified,
has a set of polynomial candidate functions
regarding the state vector
and a static matrix of coefficient
.
, the
i-th column of the matrix
, corresponds to the
i-th dimension of the derived state vector
. The
-dimensional vector of candidate functions
includes the constant 1 and all powers of the vector
up to a certain order [
24]. For example,
. These signals of
and
from measurements and calculation can be written in the following matrices:
Thus, the solving process of Equation (
1) can be cast into a regression task [
24]:
The model is established by identifying the coefficients because in most physical systems only a small subset of all candidate functions are relevant. The coefficient matrix
is sparse in the high-dimensional search space. In this situation, the Least-Squares (LS) method
is not suitable for identification in practice because the parameters with the true value zero will be estimated as a non-zero value due to the noise in the signals. At the same time, overfitting occurs [
39].
As one method of sparse regression, the LSST approach promotes sparseness by zeroing the estimated coefficients of the LS method below a fixed threshold
. The candidate functions corresponding to non-zero coefficients are selected for the next iteration of LS regression [
24]. After a certain number of iterations, usually 10 times as suggested by Brunton [
24], the algorithm converges to a fixed point [
40]. This process is presented in
Figure 1a. The parameter
needs to be small enough to let the correct candidate functions be selected and large enough to avoid noise being picked up [
40]. However, a suitable threshold cannot be found if the noise is so intensive that a zero coefficient is estimated to be greater than a non-zero coefficient. This situation often appears when the coefficients are of significantly different orders of magnitude.
To overcome this issue, another sparse regression method was developed, which is based on LASSO. LASSO is an optimization method for estimating the coefficients
in the objective function,
Compared with the LS method, LASSO has additionally a penalty term
, i.e., the
norm of the optimized vector. The
norm not only brings sparsity but also keeps the objective function convex, so that the
norm demands less computation time than the non-convex
problem [
41]. Following the standard of Matlab [
42], the factor
adjusts the scale of
norm the residuum and the hyperparameter
balances the weighting of
and
norms. A bigger value of
leads to more sparsity in the optimized vector
and an undesired increase of Mean Square Error (MSE), and vice versa. Thus, an appropriate value of
needs to be determined, so that the sparsity is promoted as much as possible without much increase of MSE. In order to select the hyperparameter
automatically, cross-validation [
43] and the 1SE-principle [
42] are used here. Cross-validation separates the whole data set into
small sets. In the training process,
sets of data are used for training and one for testing. Each small data set is selected as the test set in turn, so that the distribution of MSE can be obtained to evaluate the quality of fitting [
43]. This distribution of MSE depends on the hyperparameter
of LASSO, which can be calculated by running LASSO with different values of
. The biggest value of
, whose mean value of MSE after cross-validation within one Standard Error (SE) of minimum mean value of MSE of all experimented hyperparameters stays, is selected to be the hyperparameter of LASSO, because it tends to give a sparse coefficient vector without introducing too much error of fitting. This process is thus called the 1SE-principle [
42].
LASSO automates the construction of models with sparseness. However, the shrinkage of the estimated coefficients is not wanted, i.e., the absolute values of the coefficients are estimated to be smaller than the true values. To avoid this problem, the LS method after LASSO can be performed with selected candidate functions. This method is called LS-Post-LASSO (LSPL) [
35]. The procedure of LSPL is shown in
Figure 1b.
6. Conclusions and Outlook
The SINDy framework has rapidly developed in different extensions and has been applied to various dynamic systems in the past few years. In this work, we introduce a new sparse regression process for vibration systems based on the SINDy framework and Least Squares Post LASSO (LSPL) algorithms. In the numerical uncertainty analysis, we compared the two regression methods LSST and LSPL in terms of sparsity, convergence, eigenfrequency of oscillator, and coefficient of determination in Monte Carlo Simulations. Based on a through analysis and comparison with characteristic categories, the method SINDy-LSPL delivered a more sparse model and a more accurate prediction of the dynamic behavior. We designed a complete workflow to identify the governing equation of a single-mass oscillator using SINDy-LSPL in experimental tests. The problem of drift effect in the reconstructed velocity and displacement was accounted for by filtering out the low frequencies of reconstructed signals. In this evaluation, we tested two different types of excitation signals, namely impulse and sweep signals. The results reveal that the form and strength of excitation influence the identified result. A potential explanation of this phenomenon lies in the preprocess of training data. A detailed analysis should be dealt with in further research.
In future work in relation to SINDy-LSPL, we could further optimize the conditioning of the regression problem by improving the signal processing technique in the pre-processing of measurement data, such as denoising of the signals, integral processes, and normalization of the data. On the other hand, the usage of SINDy-LSPL in continuous vibration systems and discrete systems with multiple DOF is also worth exploring. We will also consider integrating SINDy-LSPL into adaptive control algorithms.