4.1. Datasets
Our experimental data are mainly from the stock exchange of China, CSMAR, MorningStar database, KMV default database and China Stock Market and Accounting Research Database. CSMAR is a research-oriented and accurate database in the field of economics and finance, which was developed from the needs of academic research and the professional standards of famous international databases, such as CRSP, Compustat, TAQ and Thomson of the University of Chicago and combined with the actual national conditions of China.
After more than 20 years of continuous improvement and accumulation, the CSMAR database has covered the green economy, stocks, companies, China securities, futures, foreign exchanges, macro finance, industry and other significant areas of economic and financial research-oriented databases, including 200+ databases, 4000+ tables and 60,000+ fields, which can be time, code and other indicators data for the charts in the database; can be exported to Excel with CSV (comma separated values) and other data formats. The stock exchange of China is an essential tool for investment and empirical research.
The MorningStar database is designed to provide investors with professional financial information, fund and stock analysis, ratings and convenient, practical and functional analysis application tools, such as Rating, Investment Style Box and Category Rating [
34]. These tools are professional and easy to use and can help investors make informed investment decisions. The archived data is extracted from the Morningstar database, is easily accessible and reproducible and provides a quick way to extract financial data. There are approximately 500,000 investment products with raw data in the Morningstar database. The Morningstar database is now widely used in the literature across multiple disciplines. This paper uses all available data from the database of public companies.
To forecast the credit risk of listed companies, we propose a benchmark for assessing the credit risk of SCF China SMEs through 18 indices [
35], which are also the initial independent variables of the CNN-LSTM-AM model. The 18 independent variables are classified into five categories: liquidity, leverage, profitability, activity and non-financial. Among them, the dependent variable is the credit risk status of the listed companies as risky or risk-free. The dependent variable is assigned a value of 1 when the data sample of listed companies shows non-risky but a value of 0 when the data sample release shows a negative signal.
For these 18 independent variables, we define and describe them as shown in
Table 2.
4.3. Experimental Results and Analysis
As shown in
Figure 5, we conducted comparison experiments to compare the inference speed of the CNN-LSTM-AM model with the traditional logistic and KMV models, and the results show that our method exhibits faster inference speed in different datasets, and the advantage of the CNN-LSTM-AM model becomes more apparent when the dataset increases.
This is because, when the number of inference cases is small, most of the time overhead is spent on data loading, while as the number of inference cases increases, most of the time overhead is spent on inference. It can be seen that the inference speed of the CNN-LSTM-AM model is significantly due to the other models. In contrast, with the increasing number of inference cases, the equipment appears to heat up, and the time cost rises accordingly. Therefore, our model is more economical than traditional logistic and KMV models.
In
Figure 6, we compare the speed of the different models tested in complex cases. In finance, it is most important to deal with some problematic case situations, and, in corporate credit evaluation, these data are critical. It can be seen from the figure that, in dealing with complex cases, our method has clear advantages compared to the new neural network model, which is an essential inspiration for future credit evaluation models.
We verified the accuracy of different models for predicting the credit risk of listed companies and then presented it graphically to make it more intuitive. In
Figure 7, the shades of color are responsible for different degrees with different models’ heat maps, and higher values indicate higher correctness.
Figure 7 shows that the CNN-LSTM-AM model had higher accuracy than the KMV and SMV models [
38].
In
Figure 8, to verify the applicability of our model, we demonstrate the generalization of different models on different datasets. These data sets are the stock exchange of China, CSMAR, MorningStar database and KMV default database. The performance of the CNN-LSTM-AM model in the four data sets is better than other models, showing a wide range of applicability. This makes up for the problem of KVM in dealing with nonlinear data sets and has made significant improvements and breakthroughs.
In what follows, the training process Algorithm 1 of this paper is described in detail. Compared with other neural-network models, our training process has removed many weak phase factors in the CNN model. Therefore, compared with Tree, SVM, the training process of the neural network model is more straightforward, and so the iteration speed is faster, and the calculation time and accuracy are improved.
Algorithm 1: Algorithmic representation of the training process in this paper. |
![Electronics 12 01643 i001]() |
As can be seen from
Figure 9, our model has clear advantages in the amount of calculation, mainly because we first perform convolution processing through CNN, which can extract feature vectors well and simplify indicators. Then, we introduce the attention mechanism; the attention mechanism can match the weights more reasonably and effectively so that the deep learning network can focus more energy on more efficient operations [
39].
In
Figure 10, to verify the prediction accuracy of different models in the CSMAR dataset, we divided the CSMAR dataset into five groups for experiments. As seen in the figure, our method is always more accurate than other models in the five grouping experiments, and all of them have shown high predictive ability. It effectively predicts the credit risk of listed companies in the CSMAR data set and has a more significant advantage than other models.
In
Figure 11, we conduct experiments on the CSMAR dataset, comparing our model with the ZPP model for AUC. As the number of training continues to increase, the precision of the model accuracy also increases, and our model’s prediction accuracy and increase are always higher than that of the ZPP model [
40]. Compared with the ZPP model, the learning ability of the CNN-LSTM-AM model is more vital than that of the ZPP model.
Errors in the forecasting model are essential to validating our public company risk forecasting model. In
Table 3, we selected the data of the CSMAR and MorningStar databases to verify the prediction errors of different models. It can be seen from the table that the prediction errors of our model in the two data sets are smaller than those of other models.
In
Table 4, we first simplify the operation process by CNN convolutional processing and use the CNN model to find the six indicators with the most significant correlation with the credit risk of listed companies. Then, through the attention mechanism model, we perform deep learning and assign the weight ratio by ourselves to improve the machine-learning efficiency, which makes the LSTM operation process more accessible, and operations target more critical influencing factors, thus, improving the operation efficiency. Finally, by predicting these six indicators, we compare the prediction results of different models for the feature parameters. The results show that our model has a significant advantage in the prediction of these six indicators.
As shown in
Figure 12, we compared the parameters of AM [
41], CNN-LSTM, LSTM and our model. The results show that the parameter cost of our model is significantly smaller than other models, which benefits from the CNN network and attention mechanism on LSTM model optimization.
As shown in
Figure 13, we compared the computational load of different models to test our model’s performance further. The results show the reasonable distribution of weight and computational load in the feature processing and attention mechanism of the CNN network of our model, compared with the other CNN [
42], LSTM [
43] and CNN-LSTM [
44].
In
Figure 14, we compare the training time of the CNN, CNN-LSTM, LSTM and our model, which is also one of the essential indicators to measure the model’s performance. The results show that, in different amounts of data processed, the training time of our model time, is better than for the other models.
In
Figure 15, to further test the performance of our model, we compared the inference time of different models in complex data. The results show that our model has clear advantages in the performance of more complex data sets [
45].
In
Figure 16, to verify the generalization of our model, we compared the prediction accuracy of different models in four different data sets. These are the stock exchange of China, CSMAR, MorningStar database and KMV default database. The results show that, whether in linear or nonlinear data sets, in the prediction of our model, the accuracy is higher than in other models, showing good generalization [
46].
Table 5 compares the accuracy, computation and parameter size of the models mentioned in the paper with our model. The table shows that our model has significant advantages in these aspects.