Toward a Comprehensive Evaluation of Student Knowledge Assessment for Art Education: A Hybrid Approach by Data Mining and Machine Learning

Abstract

By analyzing students’ understanding of a certain subject’s knowledge and learning process, and evaluating their learning level, we can formulate students’ learning plans and teachers’ curricula. However, the large amount of data processing consumes a lot of manpower and time resources, which increases the burden on educators. Therefore, this study aims to use a machine learning model to build a model to evaluate students’ learning levels for art education. To improve the prediction accuracy of the model, SVM was adopted as the basic model in this study, and was combined with SSA, ISSA, and KPCA-ISSA algorithms in turn to form a composite model. Through the experimental analysis of prediction accuracy, we found that the prediction accuracy of the KPCA-ISSA-SVMM model reached the highest, at 96.7213%, while that of the SVM model was only 91.8033%. Moreover, by putting the prediction results of the four models into the confusion matrix, it can be found that with an increase in the complexity of the composite model, the probability of classification errors in model prediction gradually decreases. It can be seen from the importance experiment that the students’ achievements in target subjects (PEG) have the greatest influence on the model prediction effect, and the importance score is 9.5958. Therefore, we should pay more attention to this characteristic value when evaluating students’ learning levels.

1. Introduction

The rapid development of internet technology has brought a lot of convenience to people’s lives, study, and work. With the continuous improvement of people’s living standards and the rapid updating of new technologies, society has developed increasingly strong requirements for individual learning ability. Under such high requirements and fierce competitive pressure, the evaluation standard of the traditional art education model has been unable to meet the requirements of students’ knowledge level evaluation [1]. Therefore, how to evaluate students’ knowledge level more accurately, find out the strengths of students’ learning ability, and improve students’ learning ability more effectively has become a research hotspot in the global art education community [2,3,4].

In past studies, students’ knowledge level has often been assessed by academic performance or test scores. Marks [5] used socioeconomic background as a characteristic indicator to assess the differences in student performance between in-school and out-of-school students. The comparison results show that these differences are mainly affected by students’ abilities. This contradicts the assumption that differences are determined by socioeconomic background, and supports official rationalizations used to track other forms of educational disparity. This can be used to assign students to different school types and academic positions within the school. Choudhury [6] evaluated students’ knowledge levels by establishing correlations in the students’ performances in the environmental control system course during the undergraduate period. The results prove that there is a positive correlation between students’ knowledge level and academic performance.

Therefore, increasing the evaluation parameters in the evaluation system can effectively improve the evaluation accuracy and reliability of the system. Through the correlation analysis and mining of different parameters, the relevant factors affecting students’ knowledge levels are found, and the evaluation results can help teachers make unique teaching plans for students with different grades [7,8], and to provide constructive suggestions for students’ learning and teaching decisions. Secondly, by analyzing the relevant factors that affect students’ knowledge levels and inputting them into the machine learning model, data modeling is established to predict students’ knowledge levels. Combined with the characteristic variables analyzed, this helps teachers identify the weaknesses of students and intervene in the learning process in advance, to effectively improve the learning level of students [9].

The most important part of this is the mining of student data. Data mining refers to the discovery of potential relationships and knowledge in massive student learning data. This process generally includes data acquisition, preprocessing, mining, and evaluation. To process data more quickly and accurately, artificial intelligence machine learning is used to achieve the corresponding goals. The machine learning model is trained and adjusted through the limited data collected, and seeks to discover internal relationships amongst the input parameters in the evaluation system and establish an evaluation system with high precision and reliability. It then seeks to provide more information to guide the development of students [10,11].

At present, some scholars have used machine learning models to evaluate students’ learning abilities. Zine et al. [12] assessed the applicability of e-learning to developing strategies to improve students’ readiness. Combining the ADKAR model with a machine learning-based feature importance recognition method, an effective method to assess the readiness of online learning is proposed. Two machine learning algorithms, random forest (RF) and decision tree (DT), are used to capture nonlinear relationships between data and improve the flexibility of data-driven models to effectively identify ADKAR factors that have the greatest impact on e-learning readiness. It is confirmed that ability and knowledge are important factors affecting e-learning readiness through experimental testing. Mahmud et al. [13] focused on statistics learning ability at the graduate level in order to change traditional assessment methods. Using the Rasch analysis method to assess students’ cognitive ability and logscale measurement based on the Rasch model can help us to accurately measure students’ learning in statistics courses. Bakar et al. [14] believed that students’ ability to learn mathematics should be evaluated and predicted, and appropriate interventions should be made to solve students’ learning problems. However, the adaptive neuro-fuzzy inference system (ANFIS) has some shortcomings, and has not been applied in the field of educational measurement and evaluation. Therefore, the AGES model based on neuroscience and the ANFIS formula are combined to construct a prediction model for evaluating students’ mathematical learning ability.

However, the above scholars only use a single model for modeling and forecasting, which entails limitations. First, a single machine learning model cannot be generalized and can only solve a specific range of problems. Overfitting or underfitting can occur when the model is faced with unencountered data. Therefore, when encountering different problems, scholars need to choose the most appropriate prediction model according to their own experience. On the other hand, single-model prediction is highly affected by noise and outliers in datasets, which reduces the prediction accuracy [15,16]. Therefore, it is necessary to preprocess the data. The pre-processing process generally includes data cleaning, integration, selection, and conversion. This process consumes a lot of time, so we introduce another learning model to perform the initial sorting and collection of data, and use machine learning-related classification algorithms to statistically classify parameters related to students’ learning ability, thus reducing the impact of extreme data [17,18,19]. Therefore, in this study, three additional models, sparrow search algorithm (SSA), improved sparrow search algorithm (ISSA), and kernel principal component analysis–ISSA (KPCA-ISSA) were used to process the data. The composite algorithm can effectively optimize the influences of single-peak function and multi-peak function on the prediction results, and has a strong local search ability, making it easy to search for the optimal solution. This helps to improve the accuracy and generalization ability of the support vector machine (SVM) machine learning model used to analyze students’ learning ability. In addition, compared with previous studies, the following innovations are made in this study:

-

Through the importance analysis of the input characteristic values, the most serious characteristic values affecting the assessment of students’ learning levels are discovered. This provides theoretical support for the use of the system to evaluate learning abilities in the future, and helps in the development of programs guiding students’ learning;

-

Traditional evaluation models have limitations, so this study uses artificial intelligence as a forecasting tool to build machine learning models. Compared with the traditional model, it can detect the internal relationships between different variables more effectively, and effectively improve the generalization ability of the prediction model, so that the model is not limited to the unilateral assessment of students’ learning ability;

-

It precludes the hyperparameter selection problem that arises in the machine learning model and the impact of extreme data on model prediction performance. In this study, three algorithms, SSA, ISSA, and KPCA-ISSA, are introduced to combine with the machine learning model. Thus, the reliability of the prediction results of the machine learning model is improved.

The first chapter firstly discusses the background and practical significance of studying students’ learning ability levels, and introduces the research status of evaluations of students’ learning ability at home and abroad. A solution to the problem is proposed, and a machine-learning model is introduced. The innovations of this study are then introduced. Finally, the overall structure of this paper is described. The second chapter provides the source of the data and the description of the input features, analyzes the correlation between the features, and further modifies the input features with the analysis results. The third chapter discusses various principles and work related to the analysis and research of students’ learning abilities. It mainly includes the method of data preprocessing, the additional algorithm used, and the machine learning model adopted in this study. In the fourth part, the correlation coefficients between input variables are shown, and the specific experimental results of the three composite models used in this study are compared. Through experimental comparisons, the influence of the composite algorithm on the machine learning model is analyzed, the prediction results of students’ learning ability are analyzed, and relevant conclusions are drawn.

2. Materials

To improve the prediction accuracy of the model used in this study, the data are preprocessed first. A correlation analysis for each attribute in the dataset is carried out to understand the linear relationship between the parameters and their degrees of influence on the experimental results.

The primary data utilized in this study were gathered by previous researchers [20]. These collected datasets are employed for training the aforementioned machine learning classification model, and the specific predictive performance of the model is evaluated by comparing the disparity between the predicted outcomes and the actual dataset. The dataset comprises five distinct input variables and an evaluative indicator to assess students’ knowledge levels. The attributes of input characteristics can be roughly divided into three categories: the attributes of students’ behavior, the attributes related to exam results, and the objective attributes of knowledge level. The details are presented in

Table 1. Data description.

Attribute Type	Evaluation Characteristics	Attribute Description
Individual behavioral	Length of study for target subjects	The amount of time students spend on learning objectives
	Target subject study times	The number of times students repeat the learning objective
	Length of study in relevant subjects	The amount of time students spend learning information related to their learning objectives
Exam score related	Target subject test scores	The students’ test scores in other subjects related to the learning objectives
	Relevant subject test results	Student test scores in target subjects
Knowledge level	User knowledge state	Student learning level assessment indicators

.

The dataset contains 403 items of data, each of which has values corresponding to the attributes shown in Table 1. Among these parameters, STG and STR represent the learning time spent by students on learning objectives and information related to learning objectives, respectively; STR and LPR represent students’ achievements in terms of learning objectives and relevant information, respectively; SCG is the number of times a student repeats a learning goal; UNS is an evaluation index based on the above five parameters.

Because there may be nonlinear relations between different data that are difficult to find, the model may be inclined to change one or two features when outputting prediction results, resulting in deviations in the model’s prediction results [21,22,23]. Therefore, the original data are initially assessed before being used in the learning process of the model. Through the correlation analysis of different parameters, we determine whether there is a too-high correlation coefficient between different parameters. Thus, some input parameters can be excluded or modified effectively, and the prediction accuracy of the model can be improved. The following figure shows the experimental results of correlation analysis.

The correlation coefficient represents the strength of the linear relationship between the two input parameters. As depicted in Figure 1, a correlation factor of “1” is observed for the same parameters. When one parameter changes, the other parameter also changes [24,25]. As can be seen from Figure 1, the overall correlation is below 0.3, which means that the selected input parameters are not highly correlated and can be effectively used in the learning and prediction of machine learning models. Consequently, it can be inferred that the selected input parameters satisfy the model requirements, and excessively influencing factors will not compromise the predictive performance of the model.

3. Methodology

To find a convenient way to predict students’ learning abilities, this paper uses a machine learning model to evaluate students’ learning abilities according to their professional-level data. After the data collection is completed, the dataset is split to enable the model to fully learn and detect its learning results; 70% of the data are used to build and learn the model, and the other 30% are used to check whether the model’s prediction results meet the requirements. The data from the training set are then used to create machine learning models that classify students according to their learning abilities. In this study, three different composite algorithms are proposed to combine with the SVM model, so as to provide three different data processing methods. After the comparison, the model with the highest accuracy can be evaluated by different evaluation techniques. The following sections provide more details about the model.

3.1. Data Pre-Processing

In the data collection stage, potential defects such as sample bias, selection bias and information bias may have a significant impact on the model training and testing stage, and then affect the prediction accuracy of the model. Therefore, the proper pre-processing of the original dataset is essential before building a model with the collected dataset and conducting a preliminary analysis.

To mitigate these issues, we plan to process the data with the following strategies to reduce their potential impact on the model’s predictive power.

Resampling: Through the implementation of resampling technology, the aim is to improve the representativeness of the sample and reduce the sample bias.

Data enhancement: The use of data enhancement techniques to balance the distribution of different categories and alleviate the problem of the accumulation of characteristic data.

Sensitivity analysis: The sensitivity analysis of key feature parameters is carried out to assess their impact on the research results and provide theoretical bases for identifying and selecting feature factors with significant impacts [26].

After implementing the above measures, we will verify the effectiveness of bias mitigation measures in the following ways.

Cross-validation: Cross-validation is used to evaluate the stability and generalization ability of the model on different data subsets.

External dataset comparison: Comparisons are made with external datasets to verify that the predictive power of the model has been substantially improved.

In addition, we will use the test results to further adjust the model parameters to optimize the model performance. Through this series of comprehensive measures, we expect to be able to significantly improve the prediction accuracy of the model and ensure the reliability and validity of the research results.

3.2. Data Classification and Evaluation Methods

The present study combines three distinct processing algorithms with the SVM model, comparing the prediction results of these composite models to highlight performance disparities among different algorithms. Specifically, the SSA, ISSA, and KPCA-ISSA algorithms are employed in developing the knowledge evaluation system.

The main purpose of this study is to find a more effective prediction model, in order to provide technical support for evaluating students’ learning levels. To evaluate classification algorithms, unsupervised learning techniques, mixed matrix approaches, and hyperparameter tuning iterations are utilized. Further elaboration on the various algorithms and evaluation measures employed in this study is provided within this section and 70% of the data collected in this study were used for the training phase of the model, while the other 30% were used to test the model’s predictions.

3.3. Tuning Algorithms and Machine Learning Models

The purpose of this study is to evaluate the predictive performance of a series of support vector machine (SVM)-based composite models. SVM was adopted as the basic model. Sparrow Search Algorithm (SSA), Improved Sparrow Search Algorithm (ISSA) and Kernel Principal Component Analysis are gradually integrated, and the Improved Sparrow Search Algorithm (KPCA-ISSA) is used to construct composite models of different complexity.

The SSA-SVM model introduces the SSA algorithm on the basis of the traditional SVM to perform the task of feature selection and parameter optimization. The complexity of the model is enhanced by introducing additional parameters in the training process, and the optimal solution is determined by an iterative search process.

The ISA-SVM model is optimized on the basis of SSA-SVM, and the search strategy and parameter adjustment mechanism of the composite model are improved by introducing adjustment parameters. This improvement effectively reduces the time and computing resource consumption of model training.

KPCA-ISA-SVM model further integrates KPCA as a means of feature extraction, thus enhancing the ability of the model to deal with nonlinear problems. The introduction of KPCA as a feature extraction step significantly improves the complexity of the model and correspondingly increases the operation time. Especially when dealing with large-scale datasets, the feature extraction of KPCA and the training process of SVM require more computing resources, which may lead to a significant increase in computing costs. Despite the complexity of the KPCA-ISA-SVM model, feature extraction through KPCA is expected to improve the generalization performance of the model.

In summary, with the gradual integration of algorithms in the composite model, the complexity of the model gradually increases, which is not only reflected in the model structure, but also reflected in the training process and computing requirements. However, this increase in complexity is intended to improve the overall predictive performance of the model through finer feature selection and parameter tuning, as well as more powerful feature extraction capabilities.

The following offers an introduction to each component.

3.3.1. Support Vector Machine (SVM)

The support vector machine (SVM) algorithm is utilized for both classification and regression problems. The core principle of SVM is to find a hyperplane in the dataset space that can effectively separate different types of data samples [27]. This hyperplane should ideally be positioned in the center of the various samples, maximizing the distance between them. In the face of unprocessed datasets, this method significantly improves the generalization ability of the model. Moreover, when confronted with nonlinear problems, SVM can employ kernel function techniques to address them successfully. By employing dummy variables to represent each classification attribute, SVM efficiently classifies data by mapping the original dataset into a higher-dimensional feature space where linear separability is achieved.

Firstly, a position hypothesis of the data sample in the data space is laid out. $T=\left\{(x_1,y_1),(x_2,y_2),\dots (x_i,y_i)\right\}$. X represents the I-th eigenvector in this space, and y represents the class of that eigenvector. $x_i\in R^D$, $y_i\in \left\{-1,+1\right\}$.

The main purpose of SVM is to find a hyperplane that can maximize the separation of data types, so let this plane be:

$${\omega }^T\mathrm{x}+b=0$$

(1)

The position of the hyperplane must be in the geometric interval of the sample, so the search for the optimal hyperplane can be turned into the search for the common line farthest from each feature point, where the distance from the point to the optimal plane can be defined as:

$$r=\frac{\left|{\omega }^T\mathrm{x}+b\right|}{‖\omega ‖}$$

(2)

We define the distance of the function as

$$y_i\left({\omega }^T{x}_i+b\right),i=1,\dots ,m$$

(3)

By combining Formula (2) with Formula (3), the expression of geometric distance between the dataset and the hyperplane can be obtained.

$$min\frac{y_i\left({\omega }^T{x}_i+b\right)}{‖\omega ‖},\mathrm{}i=1,\dots ,m$$

(4)

To improve the fault tolerance of the model, two planes parallel to the optimal plane are added.

$$\left\{\begin{array}{c}{\omega }^{T}x+b\ge 1,y_i=+1\\ {\omega }^{T}x+b\le -1{,y}_i=-1\end{array}\right.$$

(5)

Finding the maximum interval between the two planes is beneficial to fully partitioning the eigenvalues in the dataset.

$$\gamma =\frac{2}{‖\omega ‖}$$

(6)

The Lagrange formula is established by combining the constraint conditions with the objective function.

$$L\left(\omega ,b,\alpha \right)=\frac{1}{2}\cdot {‖\omega ‖}^2+{\sum }_{i=1}^m{\alpha }_i\left[{1-y}_i\right({\omega }^T\cdot x_i+b\left)\right]$$

(7)

$$f\left(x\right){=\omega }^T\mathrm{x}+b={\sum }_{i=1}^m{\alpha }_i{y}_i{x_i}^T\cdot x_i+b$$

(8)

3.3.2. Sparrow Search Algorithm (SSA) and Improved Sparrow Search Algorithm (ISSA)

The SSA seeks the optimal solution within a dataset by simulating the survival behavior of a population of sparrows. In captive sparrow populations, there are two distinct roles: discoverers and entrants. The main job of the discoverers is to find the optimal solution from the data space, which is the source of food, and provide other sparrows with specific information about the location of food. Entrants rely on this information to obtain food resources. Additionally, when confronted with a predator, one or more members of the group emit warning signals, prompting the entire group to move away from danger and search for food in a new area. These individuals who issue warnings are referred to as alerts [28,29].

The location of the discoverer changes as the number of iterations is updated.

$$x_{i,j}\left(t+1\right)=\left\{\begin{array}{c}{x}_{i,j}\left(t\right)·exp\left(-\frac{i}{\alpha ·{item}_{max}}\right), R_2<ST\\ x_{i,j}\left(t\right)+Q·L,R_2\ge ST\end{array}\right.$$

(9)

The t in the formula represents the current iteration number of the model, and the accuracy of the model also changes with the increase in the iteration number. ${item}_{max}$ is a constraint on the model created by humans and represents the maximum number of iterations the model can run. We can then reduce the time wasted by excessive model iteration. The information provided by $x_{i,j}\left(t\right)$ is the sparrow’s current position in the data space. α is a random number that prevents the model from falling into a locally optimal solution. $R_2$ and ST are warning values and safety values, respectively, which are used to evaluate the safety degree of the current area where the sparrow is, so as to judge the next action. L is a matrix of 1 × d, and the contents of the matrix are all 1.

When the discoverers find the food, the position of the entrants is updated, as follows:

$$x_{i,j}\left(t+1\right)=\left\{\begin{array}{c}Q·exp\left(\frac{{x_{wrost}-x}_{i,j}\left(t\right)}{i^2}\right), i>\frac{n}{2}\\ x_p\left(t+1\right)+\left|x_{i,j}\left(t\right)-x_p\left(t+1\right)\right|·A^{+}·L, i\le \frac{n}{2}\end{array}\right.$$

(10)

$x_p$ is the local best position found by the discoverer, and $x_{wrost}$ is the global worst position. A is a 1 × d matrix where each element is randomly assigned to either 1 or −1, $A^{+}={A^{\mathrm{T}}\left(A{A}^{\mathrm{T}}\right)}^{-1}$. When b, this indicates that i entrant with low fitness has not obtained food and needs to update its position again to find new food.

$x_p$ and $x_{wrost}$ represent the best and worst positions found by the finder. A is a 1 × d matrix randomly assigned 1 or −1.

The alert’s location is updated below.

$$x_{i,j}\left(t+1\right)=\left\{\begin{array}{c}{x}_{bestj}\left(t\right)+\beta \left|x_{i,j}\left(t\right)-x_{bestj}\left(t\right)\right|, f_i>f_g\\ x_{i,j}\left(t\right)+K\left(\frac{\left|x_{i,j}\left(t\right)-x_{wrostj}\left(t\right)\right|}{\left(f_i{-f}_w\right)+e}\right),f_i=f_g\end{array}\right.$$

(11)

Although the SSA model has the ability to search relatively quickly, it may fall into the local optimal solution due to lacking the ability to jump out of the local limit value when facing large datasets. To solve this problem, the algorithm is improved and the levy flight strategy is introduced into the formula. This increases the global search capability of the model [30].

$$x_{i,j}\left(t+1\right)=\left\{\begin{array}{c}Levy\left(d\right)·x_{bestj}\left(t\right)+\beta \left|x_{i,j}\left(t\right)-Levy\left(d\right)·x_{bestj}\left(t\right)\right|, f_i>f_g\\ x_{i,j}\left(t\right)+K\left(\frac{\left|x_{i,j}\left(t\right)-x_{wrostj}\left(t\right)\right|}{\left(f_i{-f}_w\right)+e}\right),f_i=f_g\end{array}\right.$$

(12)

$$Levy\left(d\right)=0.01·\frac{r_1·\sigma }{{\left|r_2\right|}^{\frac{1}{\beta }}}$$

(13)

$$\sigma ={\left\{\frac{\mathsf{\Gamma }\left(1+\beta \right)·\sin \left(\frac{\pi \beta }{2}\right)}{\mathsf{\Gamma }\left(\left(\frac{1+\lambda }{2}\right)\beta ·2^{\frac{\left(\beta -1\right)}{2}}\right)}\right\}}^{\frac{1}{\beta }}$$

(14)

Using the above methods, the ISSA model greatly reduces the risk of sparrows falling into local optimality, while still maintaining the capability for sufficient local searching. Both $r_1$ and $r_2$ are random numbers of [0, 1].

3.3.3. Kernel Principal Component Analysis (KPCA)

The utilization of KPCA is employed to address the issue of linear indivisibility in numerous datasets, thereby reducing their dimensionality and transforming them into linearly separable sets through nonlinear mapping. KPCA processes logarithmic data points, and transfers them into the feature space to improve the regularity of data and facilitate model processing and analysis (Figure 2). This approach significantly diminishes computational complexity while enabling seamless handling by linear classifiers or regressors. Moreover, kernel functions within KPCA can be selectively adjusted based on specific requirements, enhancing the model’s flexibility and generalization capabilities through real-time adaptation to diverse data scenarios [31,32,33].

3.4. Evaluation Index

The confusion matrix is an effective tool for evaluating classifier performance. Its main function is to compare the difference between the model classification results and the actual data, so as to realize the accuracy of the model classification results [34]. Through images, the misclassified and correctly classified data are visually represented in order to effectively determine the prediction accuracy of the model, as shown in

Table 2. Confusion matrix.

		Predicted
		Positive	Negative
Actual	Positive	True positive (TP)	False negative (FN)
	Negative	False positive (FP)	True negative (TN)

.

We use confusion matrices to develop performance metrics that help us understand the performance of each classifier. The above data classification approach is used to evaluate the prediction results of the model. Accuracy is used as the measurement standard in this study, and its calculation formula is as follows:

$$Precision=\frac{TP}{TP+FP}$$

(15)

To effectively compare the gap between model training results and actual data, root mean square error (RMSE) was used as a measurement index in this study.

$$RMSE=\sqrt{\frac{1}{m}{\sum }_{i=1}^m{\left({py}_i-{my}_i\right)}^2}$$

(16)

where m is the test size, ${py}_i$ is the predicted value, and ${my}_i$ is the mean value of the actual value.

4. Results and Discussion

4.1. Hyperparameter Adjustment for the Student’s Knowledge Level Prediction

Figure 3 shows iterative curves of the SVM model under different hyperparameter adjustment algorithms, including the SSA-SVM, ISSA-SVM, and KPCA-ISSA-SVM. With the increase in the number of iterations, fitness can be significantly reduced when the three hyperparameter adjustment algorithms are combined with the SVM model, indicating that all three hyperparameter tuning algorithms can effectively classify SVM machine learning models and predict students’ knowledge level, which shows that all three algorithms are effective in classifying SVM models. Also, the mixed models of ISSA-SVM and KPCA-ISSA-SVM can significantly reduce fitness compared with SSA-SVM, which reflects their advantages. Among these, ISSA-SVM obtains the smallest fitness, at about 0.0245.

4.2. Hyperparameter Adjustment for the Student’s Knowledge Level Prediction

Figure 4 gives the predicted UNS for the training and testing datasets using the SSA-SVM hybrid model. As mentioned earlier, 70% of the data in this study were used for the training set and 30% were used for the test set to predict the students’ knowledge level predictions. It can be seen that when the training set and the test set are used to predict the students’ knowledge level, their accuracy values are 96.0854% and 91.8033%, respectively. The accuracy of the training set is significantly higher than that of the test set, which may cause under-fitting in the prediction process of students’ knowledge levels.

When comparing four different models, as shown in

Table 3. Predicted UNS for the training datasets and testing datasets.

Accuracy Rate (%)	SVM	SSA-SVM	ISSA-SVM	KPCA-ISSA-SVM
Training Datasets	96.0584	96.4413	95.7295	96.7972
Testing Datasets	91.8033	92.623	95.082	96.7213

, we see that the accuracy of the training set did not improve significantly. The data show that the prediction accuracy of the SSA-SVM model in the training set is higher than that of the ISSA-SVM model. However, by comparing the results of the test set, it can be seen that the SSA-SVM model displays an overfitting phenomenon in the training stage. That is, the results of the training set differ greatly from those of the test set. However, the test results show that with an increase in complexity, the test results become increasingly close to the results of the training set. The accuracy values of the ISSA-SVM model for the training and test sets are 95.7295% and 95.082%, respectively, and those of the KPCA-ISSA-SVM model are 96.7972% and 96.7213%, respectively. This suggests that additional algorithms and information processing methods can effectively solve the underfitting problem when predicting students’ knowledge levels. Therefore, before the prediction of students’ learning level, data processing methods or data search algorithms can be used to pre-process the initial dataset, in order to effectively improve the prediction accuracy of the model. This reduces forecast errors due to underfitting or overfitting [35,36].

4.3. Hyperparameter Adjustment for the Student’s Knowledge Level Prediction

To further analyze the model’s ability to predict students’ knowledge level, and evaluate the composite model’s ability to improve the model, we performed analyses via a confusion matrix. The blue part of Figure 5 indicates that the predicted data match the actual data, while the brown part means that the predicted structure of the model deviates from the actual data, and does not compound the type of the actual data. Therefore, the greater the amount of data in the blue section, the more accurate the prediction of the model.

The second type of prediction is taken as the observation object, and the accuracy of the prediction in the training set increases with the increase in the complexity of the model. The prediction results of SSA-SVM were similar to those of the KPCA-ISSA-SVM model, and the prediction accuracy of the second type of data reached 92.7%. This is possible because the SSA-SVM model predicts overfitting during the training stage.

Observing the results of the test set, it is obvious that the accuracy of the composite model’s prediction of students’ knowledge level increases with the increase in the complexity of the model. The prediction results yielded by the training sets of the ISSA-SVM and KPCA-ISSA-SVM models were similar to those for the test sets, and the second type of prediction accuracy for the ISSA-SVM model was 89.9% in the training set and 92.3% in the test set. The second type of prediction accuracy for the KPCA-ISSA-SVM model was 92.7% for the training set and 94.9% for the test set.

Through the analysis of the above data and pictures, we can conclude that the integration of the SSA algorithm and the KPCA data processing method with the initial machine learning model SVM can effectively improve the prediction accuracy of the model. In this study, by comparing the prediction results and prediction accuracy of the SVM, SSA-SVM, ISSA-SVM, and KPCA-ISSA-SVM models, a composite machine learning model that can be developed that can accurately predict students’ knowledge level. This model can effectively analyze the influences of input parameters such as STG and SCG on the UNS results, and explore the internal relationships among them. Thus, a prediction model with high accuracy is constructed, which can effectively help to show students’ knowledge level, and further provide powerful informational support for students’ development and individualized teaching.

4.4. Importance Analysis of Factors Related to Students’ Knowledge Level

To more accurately assess the impact of each input parameter on the prediction of students’ knowledge level, an importance analysis of the input parameters was conducted. As depicted in Figure 6, PEG and LPR exhibit the highest importance scores, namely, 9.5958 and 4.2821, respectively, while STR, STG, and SCG only possess importance scores of 0.1344, 0.0856, and 0.0328, respectively. A higher importance score indicates that the prediction model will be more inclined towards parameters with greater significance when forecasting students’ knowledge levels; consequently, any changes in these parameters will significantly influence the final prediction outcome. PEG represents a student’s performance in learning objectives, whereas LPR signifies their performance in other goals related to those objectives. However, factors such as time allocation (STR), study time (STG), and self-control ability (SCG) have minimal impacts on the predictive results of this model. This substantial deviation from expected outcomes may potentially compromise both accuracy and reliability in predicting students’ knowledge levels, thus warranting further investigation and discussion.

4.5. Discussion

In this paper, we utilized the collected dataset on students’ learning levels to develop a prediction model for evaluating these levels. The dataset is divided into two parts in this study, with 70% used for training the model and the remaining 30% utilized to assess the accuracy of prediction results. To ensure that the model maintained high precision in predicting different datasets, we enhanced its generalization ability. Additionally, we enhanced the complexity of the model by combining SSA, ISSA, and KPCA algorithms with an initial classification model SVM. Consequently, three composite models (SSA-SVM, ISSA-SVM, and KPCA-ISSA-SVM) have been constructed. Each classifier underwent training and testing using the dataset to compare predictions and determine which composite model exhibits superior prediction accuracy.

It is worth noting that this study only predicts and evaluates students’ learning levels through five commonly used parameters, namely, PEG, LPR, STR, SCG, and STG. Other factors, such as students’ participation in learning goals, interest level, and learning enthusiasm, also have certain impacts on students’ learning levels. This study used machine learning models to fully analyze the presented information on students’ learning to determine their learning level. Although the input parameters for the predictive analysis are lacking, this approach provides a method to automatically evaluate students’ learning levels, and provides a theoretical basis for accurately evaluating students’ learning levels in the future, as well as proposing corresponding teaching programs.

From the experimental data presented in this paper, it can be seen that different algorithms have different effects on the hyperparameter tuning of the SVM model. The ISSA algorithm had the best tuning effect, and its fitness reached a minimum value of 0.0245 when the number of iterations reached four. The fitness of the more complex KPCA-ISSA-SVM integrated model was only 0.035, although this was still lower than that of the SSA-SVM model (0.067). This means that, regarding the prediction accuracy of the model, hyperparameter adjustment can effectively improve the degree of fitness of the model to the dataset, and thus improve the prediction accuracy of the model.

The prediction accuracies of four models were compared to facilitate a comprehensive comparison and analysis of their predictive performances on UNS. Figure 4 illustrates that both the SVM and SSA-SVM models exhibited overfitting during the training stage. In other words, the prediction accuracy of the training stage was much higher than that of the test stage, which may have been caused by the insufficient complexity of the model. By observing the prediction results of ISSA-SVM and KPCA-ISSA-SVM, it can be seen that, with increases in the complexity of the model, the gap between the prediction accuracy of the model in the training stage and the test stage gradually narrowed. This means that the construction of two of the models, ISSA-SVM and KPCA-ISSA-SVM, was effective. Improving the complexity of the model can improve the ability of the model to predict students’ learning levels.

A confusion matrix has been used to visualize the test results. From the matrix, it can be clearly inferred that there are some different inaccuracies in the prediction results of the four models. Taking the second type of prediction as an example, from the point of view of the data, the accuracy of the second type of prediction increased with the increase in the complexity of the model. In the second type of prediction, the SVM model showed 15.6% wrong prediction results, while the KPCA-ISSA-SVM only gave 7.7% wrong prediction results.

The influences of different input parameters on the model are analyzed by analyzing a large amount of student performance data. The key variables affecting the assessment of students’ learning levels are pointed out. As can be seen from the importance scores of each input parameter in Figure 6, PEG and LPR, that is, students’ scores on learning goals and goals related to learning, had the greatest influence on UNS. Their importance scores were 9.5958 and 4.2821, respectively. The other three factors scored much lower.

From the above three experiments, some potential problems can be inferred. First of all, the types of experimental data in this study are not rich enough to take into account other factors that affect students’ achievement and knowledge, such as students’ learning interests, family economic background, and previous knowledge mastery. On the other hand, the model used in this study greatly favors two input factors when predicting students’ learning levels, and the other factors have little influence on the model’s prediction results. This can lead to a lack of reliability in the model’s predictions.

5. Conclusions

To accurately assess the levels of learning mastery by students after systematic learning, the key variables that have the greatest impact on performance have been identified. This provides a theoretical basis for educators to customize teaching plans and allocate teaching resources. This study developed and improved upon a predictive model based on machine learning systems. The prediction model was constructed using the data from various aspects of a student’s learning process, and three algorithms with increasing complexity were proposed to optimize the SVM model. The collected student learning information was used to construct a dataset, 70% of which was used for model training, and the other 30% of which was used to test the prediction effect of the model. By comparing the prediction effects of the four models, the advantages and disadvantages of the four models can be accurately evaluated. According to the research results, the following conclusions can be drawn:

• Experiments show that the SSA, ISSA, and KPCA-ISSA compound algorithms can effectively improve the adaptability of the SVM model to data after multiple iteration optimization. According to the data, the ISSA algorithm has the lowest adaptation value of 0.0245 after four iterations;
• The prediction accuracies of the four models can be found. The results of the SVM model training stage produced an overfitting phenomenon, and the accuracy values of the training stage and test stage were 96.0584 and 91.8033, respectively. However, as the complexity of the composite model increased, the problem of overfitting was solved. The accuracy values of the KPCA-ISSA-SVM model in the training stage and test stage were 96.7972 and 96.7213, respectively. Therefore, improving the complexity of the model can effectively reduce the phenomenon of underfitting or overfitting;
• The importance experiment was used to explore the key variables that had the greatest influence on the prediction of students’ learning levels. Regarding the data, the importance scores of PEG and LPR were 9.5958 and 4.2821, respectively; far greater than the other three input variables (STR, STG, and SCG). This means that the model will be biased towards PEG and LPR changes when predicting and evaluating students’ learning levels, thus affecting the predicted results. Therefore, more attention should be paid to the PEG and LPR parameters of students when using UNS to evaluate students and specify study plans.

The main purpose of this research was to effectively evaluate the learning levels of students by developing a system based on machine learning models. Through the integration and analysis of a large amount of data on student performance, educators can effectively classify students and make relevant teaching plans based on the model prediction results. This study is helpful in rationally allocating learning resources and making targeted intervention measures for students, helping to achieve individualized teaching. However, in general, this study still has some shortcomings. One is that the diversity of the dataset used is not high enough. On the other hand, there is a lack of comparison between the models. This study only used the SVM model for optimization, so several research directions remain to be addressed in future research. The first is to expand students’ data sources and improve the diversity of data types. Other factors affecting the learning process, such as economic conditions, learning motivation, and knowledge system, are included in the construction data employed by the learning model. On the other hand, a horizontal comparison of different models has been performed to find a more appropriate prediction model that will improve the prediction accuracy and reliability of the model.