Uncertainty Analysis and Experimental Validation of Identifying the Governing Equation of an Oscillator Using Sparse Regression

Abstract

In recent years, the rapid growth of computing technology has enabled identifying mathematical models for vibration systems using measurement data instead of domain knowledge. Within this category, the method Sparse Identification of Nonlinear Dynamical Systems (SINDy) shows potential for interpretable identification. Therefore, in this work, a procedure of system identification based on the SINDy framework is developed and validated on a single-mass oscillator. To estimate the parameters in the SINDy model, two sparse regression methods are discussed. Compared with the Least Squares method with Sequential Threshold (LSST), which is the original estimation method from SINDy, the Least Squares method Post-LASSO (LSPL) shows better performance in numerical Monte Carlo Simulations (MCSs) of a single-mass oscillator in terms of sparseness, convergence, identified eigenfrequency, and coefficient of determination. Furthermore, the developed method SINDy-LSPL was successfully implemented with real measurement data of a single-mass oscillator with known theoretical parameters. The identified parameters using a sweep signal as excitation are more consistent and accurate than those identified using impulse excitation. In both cases, there exists a dependency of the identified parameter on the excitation amplitude that should be investigated in further research.

1. Introduction

Mathematical model building, especially the construction of governing equations, is a significant problem for a large variety of dynamic systems, such as vibration systems [1], biological systems [2], and fluid mechanical systems [3]. These models can be used for prediction, simulation, condition monitoring, optimization, etc. [4]. The traditional methods to build a model are based on the basic knowledge in one or many categories. These types of model building are described as “bottom–up” [5]. However, these methods work only with a priori knowledge. In recent years, the rapid growth of computational capacity of computers has enabled researchers in several domains to build models with a “top–down” approach using acquired data instead of the domain knowledge [5]. Many topics are related to this approach, such as System Identification (SI) and the development or improvement of structural models using the input and output data from real measurements [6]. These methods are especially useful when the first principle knowledge is not discovered, for example, the macroscopic behavior of advanced materials [7], in aerospace engineering [8], in biology [2,9,10], in economy [11], in music [12,13], etc. The models of vibration systems can be identified with various approaches, such as iterative model updating [14], the Polynomial NonLinear State Space (PNLSS) model [15], Volterra series [16], the Wiener and Hammerstein model [17,18], artificial neural networks [19,20], and equation-free model building [21]. On the downside, these methods usually build “black-box” or “grey-box” models, which means the theoretical understanding of the system is totally or partially absent from the model [4].

In the category of sparse regression, the SINDy framework (Sparse Identification of Nonlinear Dynamical Systems) shows a bright perspective in terms of interpretability. It is a technique to build the differential governing equation by extracting a few relevant terms from a library of candidate functions [22]. The polynomial development of a time series $\mathit{x}\left(t\right)$ is assumed as candidate functions and the regression method Least Squares method with Sequential Threshold (LSST, also called STLS [23]) is suggested for sparse regression [24]. In recent research, SINDy was applied to boundary value problems [25], and advanced materials [7]. Furthermore, SINDy is also applicable to Model Predictive Control (MPC) to reduce complexity and to enhance the interpretability of the integrated empirical model [26,27].

Many extensions have been adopted on the SINDy framework for different benefits. First, this framework can be modified using an enlarged library of candidate functions, such as trigonometric functions [24], partial differential equations [28,29], and rational functions [22]. Second, different methods of sparse regression can be applied. The methods, for example, Least Absolute Shrinkage and Selection Operator (LASSO) [30] and compressed sensing [31,32,33], have been used to enforce sparsity before the development of SINDy. Recently, a broad range of regularized regression methods have been reformulated and generalized in the framework of Sparse Relaxed Regularized Regression (SR3) [34]. Throughout this work, LSST and the Least Squares method Post-LASSO (LSPL) [35] will be discussed. Third, different signal processing methods are used to correct or transfer the measurement in order to make the solved problem better-conditioned, such as normalization [29], Hankel Alternative View Of Koopman (HAVOK) [36], and neural networks [37].

Since the governing equation of a discrete vibration system and SINDy share the same mathematical model structure, the SINDy framework gains attention for identifying the governing equations of discrete vibration systems. The traditional governing equation of a discrete vibration system is an Ordinary Differential Equation (ODE) using factors related to mass, m, stiffness, k, and damping ratio, d, which should be individually measured to build the governing equation. Nonlinear vibration systems can also be interpreted with similar concepts. For example, the Duffing oscillator has cubic stiffness in relation to displacement [38]. Thus, a SINDy model of a vibration system not only predicts the dynamic behavior but also is interpretable with fundamental concepts. Furthermore, the results of identification can contribute to a deeper theoretical understanding of an unknown system. In summary, SINDy shows potential for identifying the governing equation of discrete vibration systems in a “top–down” and interpretable manner without individually acquiring the parameters m, k, or d related to the domain knowledge of structural vibrations.

In this research, we explore the possibility of identifying the governing equation of an oscillator using the two sparse regression methods LSST and LSPL. We conduct numerical uncertainty analysis and experimental validation. The methods LSST and LSPL are introduced in detail in Section 2. In Section 3, we test SINDy on a numerical model of a single-mass oscillator. In order to simulate the true use case, noise signals at different levels are artificially added to the training data. At the same time, the highest polynomial order of the library varies from 1 to 5. The uncertainties of the results for both regression methods are obtained through Monte Carlo Simulation (MCS) with randomly generated parameters of the oscillator. LSST, the original sparse regression algorithm of SINDy, delivers less accurate sparsity and worse overfitting at increased noise levels. A similar statement was mentioned in a previous work, where the approximately noise-free measurement data is the requirement of SINDy [37]. To improve the robustness of SINDy against noise, we suggest the method LSPL to replace the original LSST method. After the numerical uncertainty analysis, the modified SINDy method, SINDy-LSPL, is tested on a real test bench in Section 4. In Section 4 and Section 5, we define a complete workflow from acquisition of the time series to identification and evaluation. Overall, in both numerical and experimental tests, we review how SINDy-LSPL performs at discovering the governing equation of a simple vibration system, which is fundamental for further research dealing with complex systems with more than one Degree of Freedom (DoF). Detailed conclusions and outlooks for this research are given in Section 6.

2. Methods

The SINDy framework [24] is based on differential polynomial and sparse regression. It is assumed that the equation of motion to be identified,

$$\begin{array}{c}\hfill \dot{\mathit{x}}\left(t\right)=\mathit{f}\left(\mathit{x}\left(t\right)\right)={\left(\mathit{\theta }\left({\mathit{x}}^{\mathrm{T}}\left(t\right)\right)\mathrm{\Xi }\right)}^{\mathrm{T}},\end{array}$$

(1)

has a set of polynomial candidate functions $\mathit{\theta }\left({\mathit{x}}^{\mathrm{T}}\left(t\right)\right)$ regarding the state vector $\mathit{x}\in {\mathbb{R}}^{n_x}$ and a static matrix of coefficient $\mathrm{\Xi }\in {\mathbb{R}}^{n_{\xi }\times n_x}$. ${\xi }_i\in {\mathbb{R}}^{n_{\xi }}$, the i-th column of the matrix $\mathrm{\Xi }$, corresponds to the i-th dimension of the derived state vector $\dot{\mathit{x}}\left(t\right)$. The $n_{\xi }$-dimensional vector of candidate functions $\mathit{\theta }\left({\mathit{x}}^{\mathrm{T}}\left(t\right)\right)$ includes the constant 1 and all powers of the vector $\mathit{x}\left(t\right)$ up to a certain order [24]. For example, $\mathit{\theta }\left([x_1,x_2]\right){|}_{Ord=2}=[1,x_1,x_2,x_1^2,x_1{x}_2,x_2^2]$. These signals of $\dot{\mathit{x}}$ and $\mathit{\theta }\left({\mathit{x}}^{\mathrm{T}}\right)$ from measurements and calculation can be written in the following matrices:

$$\begin{array}{c}\hfill \dot{\mathit{X}}=\left[\begin{array}{c}{\dot{\mathit{x}}}^{\mathrm{T}}\left(t_1\right)\\ {\dot{\mathit{x}}}^{\mathrm{T}}\left(t_2\right)\\ ⋮\\ {\dot{\mathit{x}}}^{\mathrm{T}}\left(t_{n_t}\right)\end{array}\right],\mathrm{\Theta }\left(\mathit{X}\right)=\left[\begin{array}{c}\mathit{\theta }\left({\mathit{x}}^{\mathrm{T}}\left(t_1\right)\right)\\ \mathit{\theta }\left({\mathit{x}}^{\mathrm{T}}\left(t_2\right)\right)\\ ⋮\\ \mathit{\theta }\left({\mathit{x}}^{\mathrm{T}}\left(t_{n_t}\right)\right)\end{array}\right].\end{array}$$

(2)

Thus, the solving process of Equation (1) can be cast into a regression task [24]:

$$\begin{array}{c}\hfill \dot{\mathit{X}}=\mathrm{\Theta }\left(\mathit{X}\right)\mathrm{\Xi }.\end{array}$$

(3)

The model is established by identifying the coefficients because in most physical systems only a small subset of all candidate functions are relevant. The coefficient matrix $\mathrm{\Xi }$ is sparse in the high-dimensional search space. In this situation, the Least-Squares (LS) method

$$\begin{array}{cc}\hfill {\widehat{\xi }}_i& =\underset{{\xi }_i\in {\mathbb{R}}^{n_{\xi }}}{\mathrm{argmin}}\{\parallel {\dot{\mathit{X}}}_{*,i}-\mathrm{\Theta }{\xi }_i{\parallel }_2^2\}\hfill \end{array}$$

(4)

$$\begin{array}{cc}\hfill & ={\left({\mathrm{\Theta }}^{\mathrm{T}}\mathrm{\Theta }\right)}^{-1}{\mathrm{\Theta }}^{\mathrm{T}}{\dot{\mathit{X}}}_{*,i}\hfill \end{array}$$

(5)

is not suitable for identification in practice because the parameters with the true value zero will be estimated as a non-zero value due to the noise in the signals. At the same time, overfitting occurs [39].

As one method of sparse regression, the LSST approach promotes sparseness by zeroing the estimated coefficients of the LS method below a fixed threshold $\lambda >0$. The candidate functions corresponding to non-zero coefficients are selected for the next iteration of LS regression [24]. After a certain number of iterations, usually 10 times as suggested by Brunton [24], the algorithm converges to a fixed point [40]. This process is presented in Figure 1a. The parameter $\lambda$ needs to be small enough to let the correct candidate functions be selected and large enough to avoid noise being picked up [40]. However, a suitable threshold cannot be found if the noise is so intensive that a zero coefficient is estimated to be greater than a non-zero coefficient. This situation often appears when the coefficients are of significantly different orders of magnitude.

To overcome this issue, another sparse regression method was developed, which is based on LASSO. LASSO is an optimization method for estimating the coefficients ${\xi }_i$ in the objective function,

$$\begin{array}{c}\hfill {\widehat{\xi }}_i=\underset{{\xi }_i\in {\mathbb{R}}^{n_{\xi }}}{\mathrm{argmin}}\{\frac{1}{2n_y}\parallel {\dot{\mathit{X}}}_{*,i}-\mathrm{\Theta }{\xi }_i{\parallel }_2^2+\lambda \parallel {\xi }_i{\parallel }_1\}.\end{array}$$

(6)

Compared with the LS method, LASSO has additionally a penalty term $\parallel {\xi }_i{\parallel }_1$, i.e., the $l_1$ norm of the optimized vector. The $l_1$ norm not only brings sparsity but also keeps the objective function convex, so that the $l_1$ norm demands less computation time than the non-convex $l_0$ problem [41]. Following the standard of Matlab [42], the factor $\frac{1}{2n_y}$ adjusts the scale of $l_2$ norm the residuum and the hyperparameter $\lambda$ balances the weighting of $l_1$ and $l_2$ norms. A bigger value of $\lambda$ leads to more sparsity in the optimized vector ${\widehat{\xi }}_i$ and an undesired increase of Mean Square Error (MSE), and vice versa. Thus, an appropriate value of $\lambda$ needs to be determined, so that the sparsity is promoted as much as possible without much increase of MSE. In order to select the hyperparameter $\lambda$ automatically, cross-validation [43] and the 1SE-principle [42] are used here. Cross-validation separates the whole data set into $n_{cv}$ small sets. In the training process, $n_{cv}-1$ sets of data are used for training and one for testing. Each small data set is selected as the test set in turn, so that the distribution of MSE can be obtained to evaluate the quality of fitting [43]. This distribution of MSE depends on the hyperparameter $\lambda$ of LASSO, which can be calculated by running LASSO with different values of $\lambda$. The biggest value of $\lambda$, whose mean value of MSE after cross-validation within one Standard Error (SE) of minimum mean value of MSE of all experimented hyperparameters stays, is selected to be the hyperparameter of LASSO, because it tends to give a sparse coefficient vector without introducing too much error of fitting. This process is thus called the 1SE-principle [42].

LASSO automates the construction of models with sparseness. However, the shrinkage of the estimated coefficients is not wanted, i.e., the absolute values of the coefficients are estimated to be smaller than the true values. To avoid this problem, the LS method after LASSO can be performed with selected candidate functions. This method is called LS-Post-LASSO (LSPL) [35]. The procedure of LSPL is shown in Figure 1b.

3. Numerical Uncertainty Analysis

Firstly, in order to explore the identification ability of the LSST and LSPL methods in the SINDy framework for vibrations, a simple vibration system was chosen for the validation. For this paper we use an oscillator with one DoF. These two methods were numerically compared in Monte Carlo Simulation (MCS) with known parameters of the oscillator. In each MCS the two methods identified $10,000$ randomly generated oscillators that have the normal distributed parameters mass, m, damping coefficient, d, and stiffness, k, as shown in

Table 1. Oscillator parameters in numerical uncertainty analysis.

Parameter	Value
$m\mathrm{in}\mathrm{kg}$	$\mathcal{N}(5,{0.05}^2)$
$d\mathrm{in}\mathrm{kg}/\mathrm{s}$	$\mathcal{N}(5,{0.5}^2)$
$k\mathrm{in}{\mathrm{kg}/\mathrm{s}}^2$	$\mathcal{N}(2000,{66.7}^2)$
${ϵ}_N$	$\{0,0.0025,0.005,0.0075,0.01\}$
$\mathit{x}\left(0\right)$	$[0;1]$
${\mathit{x}}_1^{*}\left(0\right)$	$[0.1;0]$
${\mathit{x}}_2^{*}\left(0\right)$	$[1;0]$

. The movement of an oscillator with the DoF q is governed by the following equation:

$$\begin{array}{c}\hfill \left[\begin{array}{c}\dot{q}\\ \ddot{q}\end{array}\right]=\left[\begin{array}{cc}0& 1\\ -\frac{k}{m}& -\frac{d}{m}\end{array}\right]\left[\begin{array}{c}q\\ \dot{q}\end{array}\right].\end{array}$$

(7)

In this simulation, the minimum absolute value of all coefficients was approximately $|-\frac{\overline{d}}{\overline{m}}|=1\frac{1}{s}$. Accordingly, the threshold $0.5$ was selected as the hyperparameter $\lambda$ in the method LSPL, so that no real non-zero parameter is falsely filtered out. Of course, the predetermination of the value of $\lambda$ is impossible for a real identification case. In this study, interest lies in the performance of the algorithm and not the process of finding a value for $\lambda$. A plausible threshold value can be found using the optimization method detailed in other research [1].

3.1. Generation of the Training and Test Data Sets

The training data can be generated by solving an Initial Value Problem (IVP). Given the governing Equation (7) and an initial state $\mathit{x}\left(0\right)={[q\left(0\right),\dot{q}\left(0\right)]}^{\mathrm{T}}$, the signals of acceleration $\ddot{q}\left(t\right)$, velocity $\dot{q}\left(t\right)$, and displacement $q\left(t\right)$ can be calculated simultaneously with a differential equation solver, such as ODE45 from Matlab [44].

The noise in a real measurement was simulated by adding the normal distributed random value $S_i\left(t\right)$ into the generated signals $\ddot{q}\left(t\right)$, $\dot{q}\left(t\right)$, and $q\left(t\right)$, as shown in the following equation:

$$\begin{array}{c}\hfill S_1\left(t\right)\sim \mathcal{N}(0,{({ϵ}_{N}max\left(q\left(t\right)\right))}^2),\end{array}$$

(8)

$$\begin{array}{c}\hfill S_2\left(t\right)\sim \mathcal{N}(0,{({ϵ}_{N}max\left(\dot{q}\left(t\right)\right))}^2),\end{array}$$

(9)

$$\begin{array}{c}\hfill S_3\left(t\right)\sim \mathcal{N}(0,{({ϵ}_{N}max\left(\ddot{q}\left(t\right)\right))}^2).\end{array}$$

(10)

The noise factor ${ϵ}_N$ regulates the intensity of a noise vector $S\left(t\right)$. In Table 1, five levels of noise are selected. The intensities of these five noise levels can also be interpreted with a Signal-to-Noise Ratio (SNR) as $\{\infty ,40.1,34.1,30.5,28.0\}$ dB.

For training purposes, time series with the initial state $\mathit{x}\left(0\right)$ are generated. In the training data set, 10,000 examples of numerical oscillators were randomly generated for each noise level. Thus, the training data set contained 50,000 examples. Furthermore, two different initial states, ${\mathit{x}}_1^{*}\left(0\right)$ and ${\mathit{x}}_2^{*}\left(0\right)$, were used to generate two test data sets. Similar to the training data set, each test data set also had 50,000 examples. The values of these initial states are shown in Table 1.

In order to test the effect of overfitting in the polynomial model, the test initial states were selected to bring varied amplitudes of vibrations into consideration. As shown in Figure 2, compared with $\mathit{x}\left(0\right)$, the vibrations that started with the initial states ${\mathit{x}}_1^{*}\left(0\right)$ and ${\mathit{x}}_2^{*}\left(0\right)$ have approximately doubled and grown 20-fold in amplitude according to velocity signals.

3.2. Evaluation of the Results

The order of an identified polynomial model was set from one to five, so that two methods of sparse regression (LSST and LSPL) are implemented with five different polynomial orders into data sets with five levels of noise intensity. For each method, 25 MCSs were executed in total for the evaluation. The identified model was analyzed regarding different aspects. First, the sparsity of the identified model was observed. The sparsity determines the complexity of a model. If a model is too complex or too simple, this model can be overfitted or underfitted, respectively [45]. Second, the convergence of the reconstructed time series indicates the usability of the identified model. The free vibration of a conservative system declines over time. An identified model can be discarded if the reconstructed signal diverges. Third, the eigenfrequency of a reconstruction characterizes the correctness of the numerical model of a vibration system. Lastly, the coefficient of determination $R^2$ denotes how well the reconstructed time series fits the measurement. In the following paragraphs, the results of MCSs will be evaluated.

3.2.1. Sparsity

Because the coefficients $-\frac{k}{m}$ and $-\frac{d}{m}$ to be identified appear only in the second row of Equation (7), the results corresponding to the second row of Equation (7) were selected for the evaluation. That means that the real NNZC equals two and the signal $\ddot{q}\left(t\right)$ is used to deliver the spectrum for further evaluation.

To compare the NNZC with these two methods, the results from the data set $Ord=3$ and ${ϵ}_N=0.005$ were used at first as an example to present the distribution of NNZC. As shown in Figure 3, 8847 out of 10,000 examples identified by LSST had six or seven coefficients. By contrast, 9606 NNZCs identified by the LSPL method had exactly two coefficients.

The median, 0.25 quantile, and 0.75 quantile were chosen to represent the NNZCs of all 25 MCSs with five different noise factors and five polynomial orders in Figure 4. The total numbers of candidate functions for each polynomial order are marked by horizontal lines with corresponding colors, which are also the maximum possible values of NNZC. The classic combination of mean value and standard deviation is not appropriate under this circumstance because the distribution of NNZC is strongly asymmetric, as shown in Figure 3. On the left side of Figure 4, the NNZC of the LSST method has a clear dependency on the noise factor ${ϵ}_N$ and polynomial order $Ord$. Except for the identification with $Ord=1$, more coefficients were identified with a higher noise factors. From left to right, the curves for each order $Ord>1$ increase steeply on the low noise side and then flatten on the high noise side, which indicates a saturated character.

On the contrary, the NNZC of the LSPL method does not rise substantially with the noise factor ${ϵ}_N$; see Figure 4b. The medians of all MCSs do not vary from the true number of coefficients. Only in the MCSs with $Ord\ge 3$ and ${ϵ}_N\ge 0.0075$, a change of the 0.75 quantile occurs up to 3. Compared to LSST, the LSPL method identifies the sparsity of the numerical model with a significant advantage of precision, which means that LSPL has a lower risk of overfitting.

3.2.2. Convergence

To evaluate the quality of the reconstructed time series from the aspect of convergence, the divergence ratio ${\eta }_{div}$ was used, which is defined as the ratio between the number of divergent reconstructions $n_{div}$ and the total number of experiments $n_{all}$ in one MCS,

$$\begin{array}{c}\hfill {\eta }_{div}=\frac{n_{div}}{n_{all}}.\end{array}$$

(11)

The divergence ratio ${\eta }_{div}$ varies from 0 to 1. A lower ${\eta }_{div}$ means a higher portion of the MCS represents the conservative character of the single-mass oscillator. The subsequent analysis is separated into two parts using different initial values, ${\mathit{x}}_{\mathbf{1}}^{*}\left(0\right)$ and ${\mathit{x}}_{\mathbf{2}}^{*}\left(0\right)$.

With the initial value ${\mathit{x}}_{\mathbf{1}}^{*}\left(0\right)$ and the LSST method, the reconstructed signals converge up to the polynomial order $Ord=4$ with a low divergence ratio ${\eta }_{div}$, as presented in Figure 5. A tendency can be identified that the divergence ratio ${\eta }_{div}$ rises with higher $Ord$ and ${ϵ}_N$. The maximum value in this figure equals $0.24$ from the simulation, where ${ϵ}_N=0.01$, $Ord=4$. In Figure 6a the divergence ratio ${\eta }_{div}$ with $Ord=5$ increases proportionally with the noise factor up to $0.4$, which makes the curves of the first four polynomial orders in this figure unrecognizable. On the contrary, with ${\mathit{x}}_{\mathbf{1}}^{*}\left(0\right)$ and LSPL, ${\eta }_{div}$ equals zero in all MCSs; see Figure 6b.

The second test initial value ${\mathit{x}}_{\mathbf{2}}^{*}\left(0\right)$ is more challenging for the identified model than ${\mathit{x}}_{\mathbf{1}}^{*}\left(0\right)$ due to its ten times larger amplitude; see Figure 2. In Figure 6c,d, the divergence ratio ${\eta }_{div}$ of both methods up to a polynomial order of two remains zero. From $Ord=3$, the divergence ratios of LSST increase to higher values. In particular, more than $80\%$ of the reconstructions do not converge for ${ϵ}_N\ge 0.0025$. On the other hand, the divergence ratio of LSPL rises up to only $7.2\%$ for ${ϵ}_N>0.0025$.

3.2.3. Natural Frequency

In terms of the natural frequency of the single-mass oscillator, the difference between the identified and the true value is used to evaluate the correctness of a trained model. The true eigenfrequency of an oscillator from the training data set can be calculated using the following equation:

$$\begin{array}{c}\hfill f_{true}=\frac{1}{2\pi }\sqrt{\frac{k}{m}}.\end{array}$$

(12)

Since the mass, m, and stiffness, k, are randomly generated, the true eigenfrequency from the training set shows a symmetrical distribution in the histogram Figure 7.

The eigenfrequencies $f_{rec}$ of the reconstructed data were obtained from the peaks of the spectra of the acceleration signals. The natural frequency difference reads

$$\begin{array}{c}\hfill \mathrm{\Delta }f=f_{rec}-f_{true}.\end{array}$$

(13)

The distribution of $\mathrm{\Delta }f$ with the initial state ${\mathit{x}}_{\mathbf{1}}^{*}\left(0\right)$ was evaluated in order to identify the dependency of natural frequency on the polynomial order of the candidate functions and noise intensity, because most reconstructions of the signals with ${\mathit{x}}_{\mathbf{1}}^{*}\left(0\right)$ succeed, as discussed in Section 3.2.2. In Figure 8, the natural frequencies of reconstructed accelerations with both methods have similar distributions up to polynomial order 4. With stronger noise, the error in the natural frequency increases continuously without a significant change in the range between quantiles. In the simulation with ${ϵ}_N=0.01$ and $ord=4$, the medians of $\mathrm{\Delta }f$ for the LSST and LSPL methods are equal to −0.0035 and −0.0034, respectively. The results of the LSST method in polynomial order 5 differ obviously from the other groups of data, with a larger error in the median and wider range between the 0.75 and 0.25 quantiles. Its median reaches −0.0158 with ${ϵ}_N=0.01$. This phenomenon corresponds well with Figure 6a,b regarding the divergence ratio of both methods with the initial state ${\mathit{x}}_{\mathbf{1}}^{*}\left(0\right)$, which means the missing samples of divergent reconstructions cause an additional error to the distribution of the natural frequency.

In order to reduce the complexity of the detailed comparison of the natural frequency, in the following analysis, only the results of the training data sets $Ord=3$ and ${ϵ}_N=0.005$ are shown as an example. The natural frequency differences $\mathrm{\Delta }f$ of both methods show a similar distribution for the initial condition ${\mathit{x}}_{\mathbf{1}}^{*}\left(0\right)$ as shown in Figure 9a,b. The maximum deviation corresponds to the resolution of the frequency vector after the fast Fourier transformation, i.e., $0.01$ Hz. With the initial value ${\mathit{x}}_{\mathbf{2}}^{*}\left(0\right)$, the distribution of the natural frequencies is flatter and wider when using LSST instead of using LSPL; see Figure 9c,d. Furthermore, the peak in Figure 9c is underestimated from the true value zero by $0.01$ Hz. Despite the missing reconstructed signals with the LSSL method and the initial state ${\mathit{x}}_{\mathbf{2}}^{*}\left(0\right)$, a conclusion can be made that the LSPL method has a more precise and general prediction of the natural frequency of oscillators than LSST, especially with a large initial state.

3.2.4. Coefficient of Determination $R^2$

The coefficient of determination $R^2$ describes the proportion of variability in training data that can be explained using a model [46]. To calculate $R^2$ with the noise-free training data x and the reconstructed time series $\widehat{x}$, we use the following definition:

$$\begin{array}{cc}\hfill R^2& =1-\frac{RSS}{TSS},\hfill \end{array}$$

(14)

$$\begin{array}{cc}\hfill TSS& =\mathrm{\Sigma }{(x_i-\overline{x})}^2,\hfill \end{array}$$

(15)

$$\begin{array}{cc}\hfill RSS& =\mathrm{\Sigma }{(x_i-{\widehat{x}}_i)}^2,\hfill \end{array}$$

(16)

where TSS means the total sum of squares and RSS is the residual sum of squares. The value of $R^2$ varies in the interval [0, 1]. The better a reconstruction fits the training data, the higher $R^2$ becomes. In the ideal case, $R^2$ is equal to 1.

As for the training process, the analysis of the coefficient of determination $R^2$ is only applied to the reconstruction using the same initial condition ${\mathit{x}}_0\left(0\right)$ as in the training process. The distributions of $R^2$ in Figure 10a,b show group characteristics for different polynomial orders. The first group of both LSST and LSPL methods with $Ord=\{1,2\}$ have similar medians and quantiles; for example with ${ϵ}_N=0.01$, their medians are equal to 0.9985, and the 0.75 and 0.25 quantiles are 0.9993 and 0.9971, respectively. The second group of the LSST method has two orders, $Ord=\{3,4\}$. However, the LSPL method has three orders, $Ord=\{3,4,5\}$, in its second group of $R^2$ with a higher median and smaller distribution quantiles. The $R^2$ of the LSST method in the fifth polynomial order decreases more rapidly with the increasing noise factor than the other groups of MCSs.

For the purpose of comparing the distribution of $R^2$ in detail, the boxplots as an example with $Ord=3$ and ${ϵ}_N=0.005$ in Figure 11 and

Table 2. Distribution of $R^2$ of LSST and LSPL with $Ord=3$ and ${ϵ}_N=0.005$.

Method	LSSL	LSPL
Median	0.9996	0.9997
Upper quartile	0.9998	0.9999
Lower quartile	0.9992	0.9994
Maximum	1	1
Minimum	0.9936	0.9962
Number of points	10,000	10,000
Number of outliers	474	476

indicate that LSPL has a slightly higher median value and a smaller interval between both quantiles compared to LSST. This means that the time series from the models trained by LSPL are more consistent with the noise-free true signals.

3.2.5. Comparison between LSST and LSPL

In comparison with the LSST method, LSPL tends to identify a more sparse model and has a lower overfitting risk according to the analysis of sparsity. The model trained by LSPL generally has a lower divergence ratio, ${\eta }_{div}$, than the one trained by LSST, which means LSPL delivers more usable models, especially with a large initial amplitude. In terms of natural frequency, LSPL is advantageous when the initial amplitude is large, which means that LSPL gives more accurate predictions with smaller distributions. The coefficient of determination $R^2$ indicates that LSPL suffers from less noise interference in the training data. Thus, it can be concluded that the LSPL method is a more suitable regression method for identifying the governing equation of a single-mass oscillator in the SINDy framework.

4. The Test Bench

In this section, we introduce a test bench of a single-mass oscillator used to generate measurement time series for model building. The theoretical values of the parameters are calculated “bottom–up” from the known value of each basic element. These values will be compared with the “top–down” identified parameters, i.e., those obtained from SINDy.

4.1. Experimental Setup

The test bench shown in Figure 12a contains an isolated oscillator, an excitation, and measuring instruments. This single-mass oscillator was originally designed to represent a vehicle suspension leg, whose parameters, such as stiffness, are adjustable, in order to build a basis for further study of passive and active vibration isolation [47,48]. The moving mass and two Carbon Fiber Reinforced Polymer (CFRP) beams of the oscillator shown in Figure 12b are mounted on a rigid base that is suspended by four elastic ropes with low stiffness and a low damping coefficient, so that the natural frequency of the oscillator can be distinguished from the vibration of the suspension. The signal of the excitation is derived from a signal generator. Then, the signal is amplified via an amplifier and led to the shaker. The excitation force is brought to the rigid base of the oscillator by a coupling between the shaker and the base. The accelerometers in Figure 12c,d measure the accelerations at the moving mass and the base. The excitation force is measured using a force sensor between the shaker and the base. All signals are recorded simultaneously by a data acquisition system.

4.2. Governing Equation and Parameters

The stiffness of the oscillator is determined by the position of the clamps in the guide rails and the material damping causes energy dissipation during the oscillation. The base is hung with ropes and therefore movable [47,48]. Thus, the oscillator can be simplified theoretically as in Figure 13. There are two DoFs in the oscillator: $x_{abs}$ for the moving mass and $x_{base}$ for the base. The parameters m, d, and k are the mass, damping coefficient, and stiffness, respectively.

The motion of the oscillator can be described mathematically by the following equation:

$$\begin{array}{c}\hfill m{\ddot{x}}_{abs}+d({\dot{x}}_{abs}-{\dot{x}}_{base})+k(x_{abs}-x_{base})=0,\end{array}$$

(17)

because the excitation is only carried out on the base. Introducing the relation DoF $x_{rel}=x_{abs}-x_{base}$, Equation (17) is:

$$\begin{array}{c}\hfill m{\ddot{x}}_{abs}+d{\dot{x}}_{rel}+k{x}_{rel}=0.\end{array}$$

(18)

Dividing Equation (18) by m and setting ${\omega }_0^2=\frac{k}{m}$ and $2D{\omega }_0=\frac{d}{m}$ yields:

$$\begin{array}{c}\hfill {\ddot{x}}_{abs}+2D{\omega }_0{\dot{x}}_{rel}+{\omega }_0^2{x}_{rel}=0.\end{array}$$

(19)

The calculated values of the parameters are listed in

Table 3. Theoretical parameters of the oscillator, modified from [47,48].

Parameter	Value	Description
m	0.755 kg	moving mass
k	$3.298\times {10}^4$ N/m	stiffness
D	4%	damping ratio
${\omega }_0^2$	$43,682{\mathrm{s}}^{-2}$	coefficient for $x_{rel}$ in Equation (19)
${\omega }_0$	$209{\mathrm{s}}^{-1}$	angular eigenfrequency
$f_0$	$33.3\mathrm{Hz}$	eigenfrequency
$2D{\omega }_0$	$16.7{\mathrm{s}}^{-1}$	coefficient for ${\dot{x}}_{rel}$ in Equation (19)

. Since errors accumulate into the theoretical values through the calculation, the listed theoretical values could vary to some degree from the true value. However, the numbers in Table 3 are useful to evaluate the order of magnitude of the identified parameters.

4.3. Excitation Signals

A vibration system can be excited in a wide frequency range with an impulse excitation. The shaker gives a proper frequency range but not an ideal impulse excitation. For example, the force at the excitation point in Figure 14 shows multiple peaks. Because of the coupling between the shaker and the base, an interaction exists during the vibration. Fortunately, the impulse excitation by the shaker does not affect the results of system identification, which is due to the fact that only kinematic data are relevant for SINDy.

Compared to the impulse excitation, the vibration excited by a sine sweep signal allows more data to be collected in a single measurement. Furthermore, the recorded signals have a larger amplitude, which is a benefit for the Signal-to-Noise Ratio (SNR). In this work, the frequency of the excitation ranges from $0\mathrm{Hz}$ up to $100\mathrm{Hz}$ in approximately 60 s, as shown in Figure 15.

5. Data Analysis and Results

Due to the advantages of LSPL over LSST mentioned in Section 3, LSPL was selected to experimentally identify the oscillator using the signals from the measurement. The identified models were then compared with the theoretical linear model in Equation (19) to evaluate the performance of this identification method.

5.1. System Identification with the SINDy-LSPL Method

The total force on the moving mass depends only on the relative kinematic force between the mass and the base. With Newton’s second law of motion, it can be assumed that the absolute acceleration of a moving mass ${\ddot{x}}_{abs}$ is a function related to speed ${\dot{x}}_{rel}$ and displacement $x_{rel}$. This assumption is also supported by Equation (19), the theoretical governing equation of the oscillator. For the purpose of practical implementation, the signals of the relative kinematic position are collected in the following matrix:

$$\begin{array}{c}\hfill {\mathit{X}}_{rel}=\left[\begin{array}{c}{\mathit{x}}_{rel}{\dot{\mathit{x}}}_{rel}\end{array}\right].\end{array}$$

(20)

Cast into a sparse regression, the problem to be solved can be posed as:

$$\begin{array}{c}\hfill {\ddot{\mathit{x}}}_{abs}=\mathrm{\Theta }\left({\mathit{X}}_{rel}\right)\xi .\end{array}$$

(21)

In Equation (21), $\mathrm{\Theta }\left({\mathit{X}}_{rel}\right)$ denotes the polynomial candidate functions and $\xi$ the coefficient vector. The highest order of the models was set up with $Ord\in \{3,5\}$, in order to validate the nonlinear identification with middle and high polynomial orders.

5.2. Reconstruction of the Signals in State Space

The absolute acceleration ${\ddot{\mathit{x}}}_{abs}$ is directly obtained from the measurement. However, the relative speed ${\dot{\mathit{x}}}_{rel}$ and displacement ${\mathit{x}}_{rel}$ need to be reconstructed from the relative acceleration ${\ddot{\mathit{x}}}_{rel}={\ddot{\mathit{x}}}_{abs}-{\ddot{\mathit{x}}}_{base}$ through integration. This problem can be solved with the integration property of Fourier transform [49]

$$\begin{array}{c}\hfill \mathcal{F}\{{\int }_{-\infty }^{t}f\left(\tau \right)d\tau \}=\frac{1}{\mathrm{j}\omega }F\left(\omega \right)+\pi F\left(0\right)\delta \left(\omega \right).\end{array}$$

(22)

In Equation (22), $F\left(\omega \right)=\mathcal{F}\left\{f\right(t\left)\right\}$ is the Fourier transform of the function $f\left(t\right)$. Corresponding to this practical situation, $f\left(t\right)$ equals ${\ddot{\mathit{x}}}_{rel}$ and ${\dot{\mathit{x}}}_{rel}$ in the first and second integrations, respectively. In the first integration, the amplitude at zero frequency in the spectrum $\mathcal{F}\left\{{\ddot{\mathit{x}}}_{rel}\right\}$ indicates a translation of the moving mass with a constant acceleration. Since the oscillator vibrates only around the neutral position, theoretically, $\mathcal{F}\left\{{\ddot{\mathit{x}}}_{rel}\right\}{|}_{\omega =0}$ equals zero. Due to the same reason, there is also $\mathcal{F}\left\{{\dot{\mathit{x}}}_{rel}\right\}{|}_{\omega =0}=0$ in the second integration. Furthermore, in order to avoid the drift effect, the spectrum under $10\mathrm{Hz}$ or $20\pi \mathrm{rad/s}$ is forced to be zero [50,51], so that

$$\begin{array}{cc}\hfill {\dot{\mathit{x}}}_{rel}& ={\mathcal{F}}^{-1}\{\frac{1}{\mathrm{j}\omega }\mathcal{F}\left\{{\ddot{\mathit{x}}}_{rel}\right\}\}{|}_{\omega >20\pi \mathrm{rad/s}},\hfill \end{array}$$

(23)

$$\begin{array}{cc}\hfill {\mathit{x}}_{rel}& ={\mathcal{F}}^{-1}\{\frac{1}{\mathrm{j}\omega }\mathcal{F}\left\{{\dot{\mathit{x}}}_{rel}\right\}\}{|}_{\omega >20\pi \mathrm{rad/s}}\hfill \end{array}$$

(24)

$$\begin{array}{cc}\hfill & ={\mathcal{F}}^{-1}\{{(\frac{1}{\mathrm{j}\omega })}^2\mathcal{F}\left\{{\ddot{\mathit{x}}}_{rel}\right\}\}{|}_{\omega >20\pi \mathrm{rad/s}}.\hfill \end{array}$$

(25)

In experiments with impulse excitations, the amplitudes of signals decay over time. To obtain a sufficient ratio of signals against noise, the portion of a signal with a large amplitude is used for identification, i.e., the first half of a signal. With zeroing the spectrum under $10\mathrm{Hz}$, the reconstructed responses of velocity and displacement with low frequency excitation can be interfered. Therefore, only the data in the interval from $20\%\mathrm{to}100\%$ of the time span were used for identification. The selected portions of signals are visualized in Figure 16.

5.3. Results Using Impulse Excitation

The identified coefficients and their deviations from five experiments using impulse excitation are summarized in

Table 4. Identified coefficients by using impulse excitation and their errors compared with the theoretical values from Table 3.

No.	${\mathit{F}}_{\mathit{max}}$ in N	Coefficient			Relative Error		${\mathbf{R}}^2$
		${\mathbf{\xi }}_{\mathbf{2}}$	${\mathbf{\xi }}_{\mathbf{3}}$	${\mathbf{\xi }}_{\mathbf{6}}$	$|\frac{-{\mathit{\xi }}_{\mathbf{2}}-{\mathit{\omega }}_{\mathbf{0}}^{\mathbf{2}}}{{\mathit{\omega }}_{\mathbf{0}}^{\mathbf{2}}}|$	$|\frac{-{\mathit{\xi }}_{\mathbf{3}}-\mathbf{2}\mathit{D}{\mathit{\omega }}_{\mathbf{0}}}{\mathbf{2}\mathit{D}{\mathit{\omega }}_{\mathbf{0}}}|$
1	3.39	−47,360	$-21.6$	$1498.5$	$8.4\%$	$29.3\%$	0.9957
2	3.36	$-47,270$	$-22.0$	$2381.0$	$8.2\%$	$31.5\%$	0.9957
3	6.62	$-47,300$	$-21.4$	$777.7$	$8.3\%$	$28.4\%$	0.9970
4	6.72	$-47,305$	$-21.4$	$705.3$	$8.3\%$	$28.1\%$	0.9957
5	13.11	$-47,325$	$-20.8$	0	$8.3\%$	$24.4\%$	0.9959
Mean	-	$-47,312$	$-21.4$	$1072.5$	$8.3\%$	$28.3\%$	$0.9960$

. The model identified by LSPL provides identical sparseness with polynomial orders 3 and 5. Except for the fifth experiment, the coefficients ${\xi }_2$, ${\xi }_3$ and ${\xi }_6$ are non-zero values, i.e., the factors of the candidate functions $x_{rel}$, ${\dot{x}}_{rel}$, and ${\dot{x}}_{rel}^2$ respectively. In the fifth experiment, the model was even more compact because the candidate function ${\dot{x}}_{rel}^2$ was excluded. The magnitude of the excitations has three levels, so the maximum excitation forces are about $3.4\mathrm{N}$, $6.7\mathrm{N}$, and $13.1\mathrm{N}$.

The coefficients ${\xi }_2$ and ${\xi }_3$ are consistent in the five experiments. Converted form Table 4,

Table 5. Mean value, standard deviation, and coefficient of variation of the identified coefficients by using impulse excitation.

Coefficient	Mean ${\overline{\mathit{\xi }}}_{\mathit{i}}$	Standard Deviation $\mathit{s}\left({\mathit{\xi }}_{\mathit{i}}\right)$	Coefficient of Variation $\mathit{s}\left({\mathit{\xi }}_{\mathit{i}}\right)/{\overline{\mathit{\xi }}}_{\mathit{i}}$
${\xi }_2$	−47,312	33.3	$0.07\%$
${\xi }_3$	$-21.4$	0.43	$2\%$
${\xi }_6$	$1072.5$	903.66	$84\%$

shows coefficients of variation of $0.07\%$ and $2\%$ for ${\xi }_2$ and ${\xi }_3$, respectively. In comparison with the theoretical values, the identified coefficients ${\xi }_2$ and ${\xi }_3$ have consistent offsets by approximately $8.3\%$ and $28.3\%$, respectively, in Table 4.

The coefficient ${\xi }_6$ varies by $84\%$. It can be seen that the identified coefficient ${\xi }_6$ decreases to zero as the excitation force increases, as shown in Figure 17. The coefficient ${\xi }_6$ thus depends strongly on the setup of the experiment.

5.4. Results Using Sweep Signal

System identification with a sweep signal provides compact SINDy models, each with two identified parameters, ${\xi }_2$ and ${\xi }_3$. The excitations have three levels with the peaks 5.4 N, 10.6 N, and 21.6 N in forces. Under these different settings, the identified coefficients match well with the results from the experiments with impulse excitations, as shown in

Table 6. Identified coefficients by using sweep signal excitation and their errors compared with theoretical values.

No.	${\mathit{F}}_{\mathit{max}}$ in N	Coefficient		Relative Error		${\mathbf{R}}^2$
		${\mathit{\xi }}_{\mathbf{2}}$	${\mathit{\xi }}_{\mathbf{3}}$	$\frac{-{\mathit{\xi }}_{\mathbf{2}}-{\mathit{\omega }}_{\mathbf{0}}^{\mathbf{2}}}{{\mathit{\omega }}_{\mathbf{0}}^{\mathbf{2}}}$	$\frac{-{\mathit{\xi }}_{\mathbf{3}}-\mathbf{2}\mathit{D}{\mathit{\omega }}_{\mathbf{0}}}{\mathbf{2}\mathit{D}{\mathit{\omega }}_{\mathbf{0}}}$
1	5.4	−46,556	$-21.1$	$6.6\%$	$26.6\%$	0.9998
2	10.6	−46,790	$-21.3$	$7.1\%$	$27.7\%$	0.9997
3	21.6	−47,385	$-21.7$	$8.5\%$	$29.7\%$	0.9996

.

Associating the identified coefficients ${\xi }_2$ and ${\xi }_3$ with the excitation force, it was found that with higher excitation strength the identified coefficients ${\xi }_2$ and ${\xi }_3$ decrease slightly. At the same time, the coefficient of determination ${\mathrm{R}}^2$ decreases and the relative errors increase compared to theoretical values. Figure 18a,b shows the linear dependencies of the coefficients ${\xi }_2$ and ${\xi }_3$ on the maximum excitation strength $F_{max}$. The lines through the data points were analyzed using linear regression. The estimated linear relationship is given by the following equations:

$$\begin{array}{cc}\hfill {\xi }_2& =-(46263+52F_{max}){\mathrm{s}}^{-2},\hfill \end{array}$$

(26)

$$\begin{array}{cc}\hfill {\xi }_3& =-(20.90+0.037F_{max}){\mathrm{s}}^{-1}.\hfill \end{array}$$

(27)

A possible explanation for this lies in the numerical integration, because zeroing out a data point in a Fourier series corresponds to subtracting a sine signal with the same frequency, amplitude, and phase from the time domain. The time series of velocity and displacement can thus be compressed. Another potential reason is the absence of terms with higher polynomial orders. The verification of these hypotheses and the improvement of the numerical integration can be investigated in further work. Overall, compared with the impulse excitation, the identified governing equation using sweep signals shows a more consistent result.

6. Conclusions and Outlook

The SINDy framework has rapidly developed in different extensions and has been applied to various dynamic systems in the past few years. In this work, we introduce a new sparse regression process for vibration systems based on the SINDy framework and Least Squares Post LASSO (LSPL) algorithms. In the numerical uncertainty analysis, we compared the two regression methods LSST and LSPL in terms of sparsity, convergence, eigenfrequency of oscillator, and coefficient of determination in Monte Carlo Simulations. Based on a through analysis and comparison with characteristic categories, the method SINDy-LSPL delivered a more sparse model and a more accurate prediction of the dynamic behavior. We designed a complete workflow to identify the governing equation of a single-mass oscillator using SINDy-LSPL in experimental tests. The problem of drift effect in the reconstructed velocity and displacement was accounted for by filtering out the low frequencies of reconstructed signals. In this evaluation, we tested two different types of excitation signals, namely impulse and sweep signals. The results reveal that the form and strength of excitation influence the identified result. A potential explanation of this phenomenon lies in the preprocess of training data. A detailed analysis should be dealt with in further research.

In future work in relation to SINDy-LSPL, we could further optimize the conditioning of the regression problem by improving the signal processing technique in the pre-processing of measurement data, such as denoising of the signals, integral processes, and normalization of the data. On the other hand, the usage of SINDy-LSPL in continuous vibration systems and discrete systems with multiple DOF is also worth exploring. We will also consider integrating SINDy-LSPL into adaptive control algorithms.