Data Mining of Online Teaching Evaluation Based on Deep Learning

Abstract

With the unprecedented growth of the Internet, online evaluations of teaching have emerged as a pivotal tool in assessing the quality of university education. Leveraging data mining technology, we can extract invaluable insights from these evaluations, offering a robust scientific foundation for enhancing both teaching quality and administrative oversight. This study utilizes teaching evaluation data from a mathematics course at a university in Beijing to propose a comprehensive data mining framework covering both subjective and objective evaluations. The raw data are first cleaned, annotated, and preprocessed. Subsequently, for subjective evaluation data, a model combining Bidirectional Encoder Representations from Transformers (BERT) pre-trained models and Long Short-Term Memory (LSTM) networks is constructed to predict sentiment tendencies, achieving an accuracy of 92.76% and validating the model’s effectiveness. For objective evaluation data, the Apriori algorithm is employed to mine association rules, from which meaningful rules are selected for analysis. This research effectively explores teaching evaluation data, providing technical support for enhancing teaching quality and devising educational reform initiatives.

1. Introduction

As education informatization continues to progress, numerous universities and educational institutions have embraced online evaluation systems to gather student feedback on teaching, making these data a pivotal basis for assessing and enhancing teaching quality. These evaluations often encompass a wealth of information, spanning student ratings of various course aspects, their overall course evaluations, and suggestions for teachers. However, the sheer volume of these data, coupled with the lack of effective technical tools, has resulted in their utilization being limited to basic query and statistical operations. This underutilization of the latent insights within these data is disadvantageous for the future development of colleges and universities [1]. Therefore, the imperative arises to harness the vast assessment data to extract valuable information that can bolster teaching quality and refine teaching methodologies.

Data mining represents the intricate process of uncovering, retrieving, and meticulously analyzing latent patterns, connections, and insights buried within vast datasets. This interdisciplinary field fuses multiple domains, encompassing statistics, machine learning, database management, data visualization, and high-performance computing, with the ultimate objective of extracting meaningful and valuable information from data to empower decision-making and forecast future trends. As a profound data analysis technique, it is invaluable in analyzing teaching evaluations and thoroughly exploring the intricate relationships between teaching quality and its myriad influencing factors, surpassing the limitations of traditional methodologies [2].

Sentiment analysis, also referred to as opinion mining, harnesses techniques from text mining and computational linguistics to identify, analyze, process, summarize, and reason about subjective text that carries emotional undertones [3]. Text sentiment analysis aims to extract valuable information regarding users’ emotional states from such subjective texts [4]. It involves extracting key terms from the text, utilizing appropriate algorithms to analyze these keywords, and subsequently determining the overall emotional tendency of the text.

With the advent of the big data era and the exponential growth of data in education and teaching, manually managing the categorization of sentiments from a vast number of comments has become increasingly challenging for teachers and administrators. This challenge underscores the significant potential for applying sentiment analysis and opinion mining techniques. In education, sentiment analysis can unveil and analyze emotional tendencies and opinions expressed in student evaluations. It goes beyond mere grading statistics by leveraging deep learning and natural language processing techniques such as Convolutional Neural Networks (CNNs) and Long Short-Term Memory Network (LSTM) to delve into student feedback texts with greater depth. Academic research has demonstrated promising outcomes in this area. Specific sentiment analysis techniques have proven to be invaluable tools for institutions to tackle learning issues effectively. They also enhance the overall quality of higher education by evaluating the teaching–learning process and faculty performance. This capability not only aids in addressing specific educational challenges but also fosters continuous improvement within educational institutions [5].

Association rule mining, a pivotal aspect of data mining, falls under the domain of unsupervised learning, enabling the extraction of valuable insights from data without explicit guidance. One of the most renowned applications of association rule mining is “shopping basket analysis”, wherein consumer purchase records are scrutinized to unveil potential correlations between products. This analysis informs strategies such as product bundling or targeted recommendations, ultimately bolstering product sales and enhancing consumer satisfaction. Association rule mining is currently widely applied in the field of education. Mao et al. [6] obtained basic evaluations of the effectiveness of ideological and political education courses among college students through questionnaire surveys. They employed an improved Apriori-Gen algorithm to mine association rules from these evaluations. The refined Apriori-Gen algorithm corrected biases in the teaching process, thereby enhancing the effectiveness of ideological and political education. Liu et al. [7] developed an optimal Apriori algorithm model for assessing the effectiveness of classroom activities in higher education. They constructed an optimal evaluation function model using the support–confidence joint estimation method and optimized Apriori algorithm, enabling effective assessment and multidimensional parameter estimation of classroom activities in higher education.

Through in-depth research and the application of data mining techniques, student evaluation data can be effectively utilized to provide accurate feedback and decision support for colleges and universities, thereby fostering continuous improvement and innovation in educational quality. To address this, this study collects final evaluation text data from mathematics courses at a university in Beijing and focuses on the following key areas:

• Detailed introduction of dictionary-based sentiment analysis methods, the BERT-LSTM model, and association rule mining.
• Calculation of sentiment scores for unannotated text data using sentiment lexicon methods, determining sentiment tendencies through calculated scores combined with manual judgment.
• Development of a teaching evaluation sentiment analysis model using BERT-LSTM for predicting and analyzing sentiment trends.
• Application of the Apriori algorithm to mine association rules from objective teaching evaluation data, followed by analysis based on the mining results.

The structure of this paper is organized as follows: Section 2 discusses related work on the application of sentiment analysis and association rule mining in education. Section 3 elaborates on the approaches used for sentiment analysis and association rule mining. Section 4 presents the results of sentiment analysis utilizing the BERT-LSTM model. Section 5 showcases the findings from association rule mining using the Apriori algorithm. Finally, Section 6 provides a comprehensive summary and outlines future directions.

The flow of this paper for analyzing the assessment data is shown in Figure 1.

2. Related Work

2.1. Sentiment Analysis

Research methods for text sentiment analysis can be broadly categorized into three types: lexicon-based methods, machine learning-based methods, and deep learning-based methods [8].

2.1.1. Lexicon-Based Methods

Dictionary-based sentiment analysis represents one of the simplest approaches for assessing text polarity or emotional content [9]. Its primary advantage lies in its independence from training data, making it classified as unsupervised by some experts [10]. This method operates by utilizing a sentiment lexicon comprising opinion words, which are matched against the text to ascertain polarity. Opinion words in the lexicon are assigned sentiment scores that denote their positivity, negativity, or objectivity [11]. However, due to its reliance on the lexicon’s construction and selection, this method exhibits limited adaptability across diverse domains and struggles with complex or ambiguous words alone. Consequently, it is commonly employed as a complementary technique in conjunction with other methods in practical applications [12].

2.1.2. Machine Learning Methods

Machine learning approaches employ algorithms to learn and extract features from the text to determine its emotional polarity. Commonly utilized methods include Naive Bayes (NB), Support Vector Machine (SVM), and Logistic Regression (LR), among others.

Ortigosa, Martin, and Carro [13] proposed alternative sentiment analysis methods tailored for e-learning environments, combining a Spanish lexicon with machine learning techniques. Their experiments revealed that combining lexicon-based techniques with SVMs achieved the highest accuracy (83.27%). Dsouza et al. [14] utilized machine learning algorithms such as Random Forests, Polynomial Naive Bayesian Classifiers, and SVMs to analyze student feedback for sentiment analysis. They found that Polynomial Naive Bayesian Classifiers outperformed other methods with an accuracy of 80%.

However, the performance of machine learning methods in sentiment categorization is constrained by complex feature design and limited adaptability across different domains. Thus, the focus is shifting towards discovering effective feature combinations for future advancements [12].

2.1.3. Deep Learning Methods

Deep learning can establish neural networks that simulate the human brain for analysis and learning, capable of capturing more complex semantic and contextual information, thus finding widespread application in sentiment analysis. Deep learning-based sentiment analysis methods integrate feature extraction and text tendency judgment, eliminating the need for manual feature extraction and achieving higher accuracy, albeit at the cost of longer model training times.

Kandhro et al. [15] used the LSTM network model to predict and analyze the text of English student evaluation, which confirms that the deep network learning method can obtain a better classification accuracy than the traditional machine learning method. Zhang et al. [16] introduced a deep learning-based sentiment analysis method for the traditional teaching evaluation session and compared the traditional SVM model with the deep learning model, and they found that the accuracy of CNN and simple recurrent neural network (RNN) and LSTM was significantly improved compared to the traditional classification prediction methods. Cabada et al. [17] proposed several deep learning architectures for educational sentiment analysis with CNN using long and short-term memory, achieving 84.32% classification accuracy.

2.2. Association Rule Mining

In terms of association rule mining, Zhang et al. [18] mined the association between teachers’ teaching effectiveness and teaching assessment results, and found the association between teaching assessment results and the characteristics and learning basis of the class itself. Li [19] analyzed the association between each evaluation subitem of the objective evaluation data to obtain strong rules, and based on this, they put forward teaching improvement suggestions for teachers’ reference to improve teaching quality. Xu et al. [1] used the Apriori algorithm to dig deeper into students’ grades in each subject in the teaching management system, and obtained correlations between courses, which provided a reference for teaching administrators to design teaching plans.

3. Method

3.1. Sentiment Analysis Based on Sentiment Lexicon

3.1.1. Text Preprocessing

Data cleaning. The text of teaching evaluations often contains arbitrary and subjective elements, including non-standardized terms such as homophones, misspelled words, and comments expressed in pinyin. Moreover, the corpus may include redundant and irrelevant comments. If left unprocessed, these noisy data can lead to errors in corpus segmentation and lexical labeling, thereby affecting the accuracy of analysis results. Therefore, prior to segmentation, it is essential to perform denoising on the original data [20]. This involves eliminating worthless and redundant evaluations. Since the data originate from multiple datasets with overlapping entries, we conduct data integration and remove duplicate values to ensure data cleanliness.

Text segmentation. In Chinese text, words are inherently continuous and must be segmented into separate units for analysis. Utilizing tools or algorithms for segmentation is essential. This process enhances the accuracy of sentiment analysis, reduces the dimensionality of the feature space, and extracts key information from the text, enabling a better understanding and analysis of its sentiment tendencies. In this study, we employ the jieba segmentation package (version 0.42.1) to segment the subjective assessment text data.

Removing deactivated words. After segmenting the subjective assessment text data, the next step involves removing common words that lack clear sentiment tendencies in sentiment analysis, known as stopwords. These stopwords typically include functional words like prepositions, pronouns, and conjunctions, which contribute minimally to sentiment analysis. By eliminating stopwords, the accuracy and efficiency of the analysis can be improved. In this study, we utilize the Harbin Institute of Technology stopwords list to filter out stopwords from the segmented data.

3.1.2. Calculating Sentiment Scores Using a Sentiment Lexicon

To compute sentiment scores, we utilize the sentiment dictionary from the China National Knowledge Infrastructure, which is the most widely used and authoritative sentiment lexicon in China [20]. It categorizes words into degree-level words, positive and negative sentiment words, and positive and negative evaluation words. Prior to calculating the sentiment scores, we establish the score constants for degree-level words, as depicted in

Table 1. Score constants for degree-level words.

Type	Score
most	8
very	6
more	4
over	2
ish	0.6
insufficiently	−1.5

, and for positive and negative words, as shown in

Table 2. Score constants for positive and negative words.

Type	Score
positive	1
negative	−1
no attitude	0

.

Degree words are classified into six levels, each corresponding to a distinct score. Both positive sentiment words and evaluative words are assigned a score of 1, while negative sentiment words and evaluative words carry a score of −1. When traversing the provided list of positive and negative words, we determine whether each word is present in the positive or negative lexicon.

If the word is found in the positive lexicon, we assess the lexical properties of the preceding word to assign different positive scores accordingly. For instance, if the preceding word is “most”, the score is multiplied by the highest score constant of 8 and the positive score constant of 1. If the preceding word does not have any special lexical property, a positive score constant of 1 is directly applied.

Similarly, if the word is located in the negative lexicon, the score is determined based on the lexical properties of the preceding word, and the corresponding negative score is assigned. Words not found in either the positive or negative lexicon are treated as non-attitudinal words and are assigned a constant score of 0.

The positive and negative scores from each evaluation are summed to derive the final sentiment score for that evaluation. For instance, the content and score of a single positive evaluation in the subjective assessment are illustrated in

Table 3. Example of sentiment score calculation.

Evaluation	Segment	Score
A very responsible teacher, Chaoxing and MOOC have his own time to record microclasses, as well as for the examination and non-examination students of different levels of difficulty of the knowledge points of the explanation, the class will also be uploaded to the Chaoxing so that we can review after class, on the homework of the explanation is also very important, he will also record the course for us to speak, Mr. Tan is the most responsible teacher I met in the university.	[‘very’, ‘responsible’, ‘class’, ‘time’, ‘record’, ‘micro classes’, ‘different ‘, ‘difficult’, ‘knowledge’, ‘explanation’, ‘learning’, ‘lesson’, ‘will’, ‘homework’, ‘explanation’, ‘very’, ‘important’, ‘university’, ‘met’, ‘the most’, ‘responsible’]	23

below.

In this evaluation, the degree-level words are “very”, “very much”, and “most”, while positive evaluation words include “responsible”, “ important”, and “would”. Specifically, “very” corresponds to 6 points, “very much” and “most” correspond to 8 points, and all three positive evaluation words correspond to 1 point. Combinations of intensity words and positive evaluation words include “very responsible”, “important”, “most responsible”, and “would”. Therefore, the score for the phrase “very responsible” is 6 × 1 = 6, “very important” and “most responsible” are both 8 × 1 = 8, and “would” without an intensity word in front of it is 1 point. Applying the method outlined for calculating sentiment scores, we obtain 6 × 1 + 8 × 1 + 8 × 1 + 1 = 23. This evaluation yields a sentiment score of 23.

3.2. Sentiment Analysis Based on BERT-LSTM Model

3.2.1. BERT

The BERT pre-training model employs a bidirectional transformer to analyze the relationship between each word and others within a sentence. Through this process, it comprehends the contextual dependencies within the text and extracts intricate language features, which are subsequently utilized by the model [21]. BERT has two pre-training tasks: Masked Language Modeling (MLM) and Next Sentence Prediction (NSP) [22].

In Masked Language Modeling, a sentence is given with one or more words randomly masked. The model is then tasked with predicting the masked words based on the remaining context. For the masked words, 80% of the time, they are replaced with a special token [MASK], 10% of the time with a random word, and in the remaining 10%, they are left unchanged [12]. Importantly, during fine-tuning on downstream tasks, the [MASK] token is not used. Another benefit of this approach is that during word prediction, the model does not know if the input word at that position is the correct word, forcing it to rely more on contextual information, thereby enhancing its error-correction capabilities.

Next Sentence Prediction involves providing two sentences from a document and requires the model to predict whether the second sentence follows the first one in the text. This task enhances the model’s understanding of context, enabling it to better grasp the logical relationships between sentences.

3.2.2. LSTM

LSTM, a type of temporal RNN, is well suited for processing and predicting significant events within time series data characterized by long intervals and delays. It effectively addresses the limitations of traditional RNNs, making it the most popular choice among RNN architectures today [23]. One of LSTM’s key features is its ability to utilize the output state from the previous time step as input for the next time step. This mechanism enables LSTM to capture long-range dependencies within sequences by leveraging gate control units, thus preserving crucial rule-compliant information [24].

In an LSTM cell, depicted in Figure 2, several components are crucial: a forgetting gate, an input gate, an output gate, a memory cell, and a hidden state. The forgetting gate determines whether information from the previous memory cell should be preserved or forgotten. It evaluates the previous cell’s output $h_{t-1}$ and the current input $x_t$, producing an output vector $f_t$ ranging between 0 and 1 through a sigmoid function. A value of 1 indicates complete retention, while 0 signifies complete discardment. The specific formula is represented in Equation (1), where $\sigma$ denotes the sigmoid function, $W_f$ represents the forgetting gate weight, and $b_f$ denotes the forgetting gate bias.

$$f_t=\sigma \left(W_f·\left[h_{t-1},x_t\right]+b_f\right)$$

(1)

The input gate determines which new information should be incorporated into the memory cell. It comprises a sigmoid layer and a tanh layer. The sigmoid layer regulates which information will be stored in the candidate state, as illustrated in Equation (2). Meanwhile, the tanh layer generates the candidate state itself, as depicted in Equation (3).

$$i_t=\sigma \left(W_i·\left[h_{t-1},x_t\right]+b_i\right),$$

(2)

$$\widetilde{C_t}=\tanh \left(W_C·\left[h_{t-1},x_t\right]+b_C\right).$$

(3)

After information passes through the forgetting gate and the output gate, the current memory cell undergoes an update. The previous state is multiplied by $f_t$, where $f_t$ determines the portion of information to forget. The filtered candidate state is then added to form the new memory unit. The calculation formula is presented in Equation (4).

$$C_t=f_t\times C_{t-1}+i_t\times \widetilde{C_t}.$$

(4)

The output gate controls which feature information the current memory cell should output. Initially, it decides the information to be passed through via a sigmoid layer, as depicted in Equation (5). Subsequently, it processes the memory cell using the tanh layer and multiplies this result by the output from the sigmoid layer, yielding the final output information. This process is detailed in Equation (6).

$$o_t=\sigma \left(W_o·\left[h_{t-1},x_t\right]+b_o\right),$$

(5)

$$h_t=o_t·\tanh \left(C_t\right).$$

(6)

3.2.3. BERT-LSTM

When constructing the model, we opt to incorporate the LSTM layer following the BERT layer, aiming to amalgamate the contextual comprehension prowess of BERT with the sequence modeling capability of LSTM. This strategic arrangement enables the entire model to acquire multi-level features from textual information, facilitating the extraction and discernment of crucial implicit features influencing the emotional tendencies within students’ comments. The complete model comprises an input layer, a BERT layer, an LSTM network, a fully connected layer, and an output layer, as illustrated in Figure 3.

The input layer receives preprocessed sample features and forwards them to the BERT layer. Through the BERT pre-training model, input text sequences are transformed into their respective word embedding representations, which are then passed to the LSTM layer. Leveraging its feedback memory capability, the LSTM network extracts feature relationships between the context and words, thereby capturing sequential information in text by learning both long-term and short-term dependencies. Subsequently, the fully connected layer amalgamates the previously extracted features and maps them to a higher-dimensional feature space, enhancing their suitability for classification or regression tasks. Lastly, the output layer generates the final prediction results, utilizing the softmax function to convert the output values of the multiclassification into a probability distribution ranging from 0 to 1, summing up to 1 [25]. This distribution represents the probability of the different categories.

With this deep network model structure, the BERT-LSTM model enables us to conduct sentiment analysis of students’ evaluations effectively. By leveraging this model, vital implicit features can be extracted from the evaluations, thereby enhancing the ability to analyze and predict the sentiment tendencies therein.

3.3. Association Rule Mining Based on the Apriori Algorithm

Association rules highlight the correlations between individual transactions and other transactions, making them a valuable tool for uncovering interrelationships among data items within extensive datasets. This technique is commonly employed to unearth potential associations or connections between items, thereby facilitating recommendations based on patterns identified in the data [26].

3.3.1. Basic Concept of Association Rules

Support represents the proportion of itemsets that contain both X and Y simultaneously, quantifying the likelihood of co-occurrence of these items within the dataset. It essentially measures the probability of encountering both X and Y within an itemset:

$$Support\left(X\to Y\right)=P\left(X,Y\right).$$

(7)

The minimum support serves as the threshold that a user-defined association rule must surpass.

Confidence denotes the ratio of the number of itemsets containing both X and Y to the number of itemsets containing only X.

$$Confidence\left(X\to Y\right)={\displaystyle \frac{Support\left(X,Y\right)}{Support\left(X\right)}},$$

(8)

$$Confidence\left(X\to Y\right)=P\left(Y|X\right).$$

(9)

The minimum confidence signifies the smallest confidence threshold set by the user that an association rule must exceed, indicating the minimum level of reliability required for the association rule to be considered significant.

Lift is computed as the ratio of the conditional probability of the term before and after the association rule to the probability of the term after it, as illustrated below:

$$Lift\left(X\to Y\right)={\displaystyle \frac{Confidence\left(X\to Y\right)}{P\left(Y\right)}}={\displaystyle \frac{P\left(Y|X\right)}{P\left(Y\right)}}.$$

(10)

When Lift = 1, it indicates that the prior and posterior terms are statistically independent.

When Lift > 1, it suggests a positive correlation between the prior and posterior terms, implying that the likelihood of both X and Y occurring together is higher than if they were independent.

When Lift < 1, it signifies a negative correlation between the two terms.

Lift serves as a measure of the association strength between the antecedent and consequent terms in an association rule [27]. Typically, when Lift exceeds 1, it signifies a meaningful relationship between the antecedent and consequent terms. Consequently, filtering association rules based on a Lift threshold greater than 1 is often straightforward and effective [19].

Frequent itemsets refer to non-empty sets of items with support greater than or equal to the minimum support threshold. When a frequent itemset contains k elements, it is termed a k-frequent itemset.

An association rule that meets both the minimum support and minimum confidence criteria is referred to as a strong association rule [19].

3.3.2. Apriori Algorithm

Association rule mining is used to mine the rules with support greater than the minimum support and confidence greater than the minimum confidence from the transaction set, so association rule mining can be divided into two steps. The first step is to discover frequent itemsets, and the second step is to generate association rules.

The Apriori algorithm stands as one of the most classic association rule algorithms. Leveraging the downward closure property of frequent itemsets, it systematically identifies all rules meeting the minimum support and minimum confidence criteria. By iteratively generating and pruning candidate itemsets, it sifts through potential associations, ultimately retaining those that satisfy the screening criteria [27]. This algorithm addresses limitations present in models like discrete choice, which lack the ability to quantitatively compare and analyze result variations across groups. Its versatility and scalability make it widely applicable [28].

The Apriori algorithm operates based on the principle that any subset of a frequent itemset must also be frequent, while a superset of an infrequent itemset must also be infrequent. Employing an iterative approach termed layer-by-layer search, it progresses from k-itemsets to (k + 1)-itemsets. Initially, it identifies the set of frequent 1-itemsets, denoted as set L1. L1 serves as the foundation for discovering the set L2 of frequent 2-itemsets, which, in turn, leads to the identification of L3, and so forth, until no further frequent k-itemsets can be found. This iterative nature of the Apriori algorithm efficiently compresses the search space, enhancing the generation of frequent itemsets layer by layer.

The execution flow of the Apriori algorithm is depicted in Figure 4 below:

4. Subjective Evaluation of Teaching Analysis

4.1. Experimental Data Preparation

4.1.1. Preprocessing of Subjective Evaluation Data

After preprocessing the text data, the essential vocabulary can be extracted from the subjective assessment data. By conducting a simple word frequency count, the top 20 words with the highest occurrences are identified, as depicted in Figure 5.

Subsequently, a word cloud is generated based on these extracted words and their respective frequencies. The word cloud visually presents an overview of the evaluation by depicting words in varying sizes and colors based on their frequency and importance, as illustrated in Figure 6.

It is evident that words with high frequencies are predominantly positive. Words such as “good”, “serious”, “conscientious and responsible”, “patience”, and “humor” have notably high occurrence frequencies. This suggests that the majority of teachers are diligent and responsible in their teaching, and their lessons are well received by students.

After completing the preprocessing of the teaching evaluation texts, a dataset comprising 13,026 subjective teaching evaluations was obtained. Calculating sentiment scores for these data reveals 4933 positive evaluations, 7955 neutral evaluations, and 138 negative evaluations. Figure 7 illustrates the distribution of teaching evaluation data across various emotional tendencies.

It is evident that neutral evaluations constitute the largest proportion, representing 61.07% of all subjective evaluation data. This predominance may stem from many students not giving sufficient attention to teaching evaluations, particularly subjective ones. Subjective teaching evaluations often entail using non-emotional vocabulary and straightforward responses. The following neutral evaluations are positive ones, comprising 37.87% of all subjective evaluation data. Conversely, negative evaluations make up only 1.06%. This indicates that the majority of students who provided detailed evaluations had a positive learning experience from the courses. They acquired knowledge, enhanced their skills, and were generally content with the courses.

After calculating the sentiment scores for each subjective evaluation, those with a score greater than 0 are categorized as positive and labeled as 1. Evaluations with a score less than 0 are deemed negative and labeled as 2. Evaluations with a score equal to 0 are regarded as neutral and labeled as 0.

Table 4. Sentiment tendency and labeling of teaching evaluations.

Index	Evaluation	Score	Tendency	Label
1	(1) The teacher lectures seriously, carefully, can make full use of time, the image of the organization, the key knowledge of the explanation is very clear and easy to understand, so that the students on the knowledge of easy to understand, the teacher’s passion for lectures will infect us, the classroom atmosphere is very good. (2) The teacher is good at mobilizing the enthusiasm of students, the classroom atmosphere is very active. Teaching carefully and kindly. Attitude serious and responsible, extremely patient, is our heart of the teacher’s dear. (3) The teacher is very serious about lecturing, content outline, organized and strong, and especially good at giving examples, so that students theory and practice, learning is very easy, and impressive, received good results. The teacher is kind, classroom interaction with students, creating a warm classroom atmosphere. (4) The teacher is rigorous, strict requirements, can deeply understand the students’ learning and living conditions, and follow the good advice, approachable. Attention to inspire and mobilize students’ enthusiasm, the classroom atmosphere is more active.	49	positive	1
2	The instruction was clearly delineated and organized, it just got in the way of the overall level being a little easy for me, but that’s not the teacher’s problem.	0.4	positive	1
3	Some particularly simple topics can be briefly summarized and less homework can be assigned.	−12	negative	2
4	Classes are slightly boring.	−0.6	negative	2

below presents the evaluations with the highest and lowest scores.

4.1.2. Addressing Category Imbalance

In university evaluation data, the number of positive and neutral evaluations typically far exceeds that of negative evaluations. Directly applying traditional machine learning sentiment classification methods to such unbalanced sentiment category corpora can lead to biased classification results favoring the majority class [29], thereby impacting the model’s training and prediction. To address this issue, the positive and neutral evaluations are downsampled by randomly selecting 150 samples from each tendency. Subsequently, the negative evaluations are upsampled by augmenting them to 150, fine-tuning the inflectional order to ensure consistency in sample numbers across the three types of evaluations. Finally, 20% of the samples from each evaluation type are randomly selected as test samples, with the remainder utilized as training samples.

4.2. Performance Metrics

To effectively assess the model’s performance, commonly used evaluation metrics such as accuracy, precision, recall, and $\mathrm{F}1$ Score are chosen to gauge the predictive effectiveness in classification tasks [30]. The confusion matrix for the predicted results is shown in

Table 5. Confusion matrix.

	Positive Simple in Forecast	Negative Simple in Forecast
Positive simple in reality	TP	FN
Negative simple in reality	FP	TN

.

Accuracy represents the ratio of correctly predicted samples to the total number of samples, serving as a comprehensive evaluation metric for the entire model. However, it may not provide an accurate assessment of model performance when there is significant class imbalance. The formula for calculating accuracy is

$$Accuracy={\displaystyle \frac{TP+TN}{TP+TN+FP+FN}}.$$

(11)

The precision rate measures the proportion of true positive samples among all instances predicted as positive by the model. It is calculated by the following formula:

$$Precision={\displaystyle \frac{TP}{TP+FP}}.$$

(12)

Recall, also referred to as sensitivity, quantifies the percentage of samples correctly predicted as positive by the model out of all samples that actually have a positive true value. It is calculated using the following formula:

$$Recall={\displaystyle \frac{TP}{TP+FN}}.$$

(13)

The $F1 Score$ is a harmonized average of precision and recall, given equal weight in its calculation. It is computed as follows:

$$F1 Score={\displaystyle \frac{2\times precision\times recall}{precision+recall}}.$$

(14)

4.3. Experimental Environment and Parameter Settings

The experiment is based on the TensorFlow 2.12.0 (TensorFlow Team, Mountain View, CA, USA) deep learning framework and coded in Python 3.11.5 (Python Software Foundation, Beaverton, OR, USA). The BERT-LSTM model’s hyperparameter settings are detailed in

Table 6. Hyperparameter settings.

Hyperparameter	Conversion Result
learning rate	0.00002
max_length	75
batch_size	16
LSTM units	128
dropout rate	0.2
optimizer	Adam
loss function	SparseCategoricalCrossentropy
epochs	30

below. The model’s input layer utilizes BERT-base-Chinese, an open-source Chinese pre-trained language model provided by Hugging Face, as the pre-trained model.

4.4. Results

Following model training, we derive the prediction accuracy values for both training and validation datasets, showcased in Figure 8a. Additionally, Figure 8b illustrates the prediction loss values for the training and validation datasets. The training accuracy of the model shows an initial sharp increase followed by a gradual leveling off, reaching a peak of 100%. Similarly, the validation accuracy demonstrates rapid growth in the early stages, peaking at 98.33% by the fourth iteration. The validation loss value fluctuates within a certain range after an initial rapid decrease. Considering both the training accuracy and the magnitude of the loss value, we designate the network model parameters from the 14th iteration as the optimal parameters.

For the test samples, utilizing the BERT-LSTM model for sentiment analysis yields a prediction accuracy of 92.76%, achieved within a prediction time of only 43 s. This not only attains high prediction accuracy but also significantly reduces sentiment analysis time compared to manual judgment. A summary of the model performance is shown in

Table 7. Model performance summary.

Item	Label	Precision	Recall	Accuracy
Neutral	0	100.00%	92.50%	92.76%
Positive	1	87.80%	88.89%
Negative	2	90.48%	92.68%

.

As illustrated in Figure 9, only four positive samples with a true label of 1 are incorrectly predicted as label 2, two negative samples with a true label of 2 are misclassified as label 1, and three neutral samples with a true label of 0 are misclassified as label 1. The recall rate for positive samples reaches 90.00%, 95.00% for negative samples, and 92.50% for neutral samples.

5. Objective Evaluation of Teaching Analysis

5.1. Objective Evaluation Data Preprocessing

5.1.1. Data Cleaning

The objective evaluation data encompass multiple datasets, totaling 146,760 records. Upon integration, considering potential overlaps and missing values within individual datasets, the following processing steps are executed:

Identification and removal of duplicate data: A total of 33,076 duplicate entries are detected. Only the initial entry is retained, while subsequent duplicates are deleted.

Elimination of entries with missing values: Due to the dataset’s size, missing values are relatively infrequent. Therefore, entries containing missing values are directly removed, exerting minimal impact on the analytical outcomes.

Following data cleaning procedures, 130,205 data entries persist, representing 88.72% of the original dataset.

5.1.2. Data Conversion

The Apriori algorithm, employed for association rule mining, is designed exclusively for Boolean data. Consequently, numerical objective assessment data must be converted into a Boolean format.

The objective assessment of teaching comprises 20 questions, each referenced by a keyword for streamlined analysis, detailed in

Table 8. Keywords used to refer to questions.

Index	Item	Word
1	At the beginning of the semester, the teacher clearly told us the course requirements and assessment methods.	Information
2	I can grasp the teacher’s main points.	Emphasis
3	The teacher’s content is moderate in difficulty and ease.	Moderate
4	The teacher can explain the knowledge points through practical cases.	Case
5	I feel the teacher is organized in class.	Orderliness
6	The teaching method adopted by the teacher works well for me.	Effectivity
7	Teachers can inspire me to think.	Enlighten
8	I think the teacher makes good use of the class time.	Efficiency
9	The teacher’s explanation made it easier for me to understand and master the course content.	Explain
10	The teacher’s lectures are full of spirit and not boring.	Interesting
11	I have the opportunity to participate in classroom interaction.	Interaction
12	The teacher often assigns us homework.	Homework
13	Teachers timely comment homework, answer questions carefully.	Answer
14	Teachers respect us and care about us.	Respect
15	The teacher is strict with his students.	Strictness
16	If I need to, I can contact the teacher to communicate.	Communication
17	The teachers are well-prepared and proficient in their lessons.	Preparation
18	Through the teacher’s guidance, I have mastered the main content of this course.	Guidance
19	The teacher’s words and deeds give me inspiration in life.	Edification
20	I think the teacher is one of the teachers teaching this semester.	Evaluation

below:

In the objective assessment data, scores for each item fall within the range of [3,5]. Consequently, the grades are categorized based on these scores. Taking question 1 as an example, the conversion is illustrated in

Table 9. Data conversion rules.

Item	Score	Level	Conversion Result
Information	5	1	Information1
Information	4.5	2	Information2
Information	4	3	Information3
Information	3.5	4	Information4
Information	3	5	Information5

. Similarly, the remaining 19 objective assessment questions are also transformed into categorical data using this approach.

5.2. Association Rule Mining for Objective Assessment Data

5.2.1. Association Rule Mining Results

The minimum support is established as 0.8 and the minimum confidence as 0.8, filtering association rules with a Lift greater than 1. This process yields a total of 8643 association rules. Among them, 22 are deemed to possess practical significance for further analysis. The filtered association rules are displayed in

Table 10. Association rule mining results.

Index	Antecedents	Consequents	Support	Confidence	Lift
1	Orderliness1, Preparation1, Case1	Explain1	0.83	0.98	1.08
2	Enlighten1, Preparation1	Explain1	0.84	0.98	1.09
3	Preparation1	Enlighten1, Orderliness1	0.84	0.90	1.05
4	Orderliness1, Preparation1	Efficiency1, Explain1	0.82	0.93	1.09
5	Information1, Preparation1	Emphasis1	0.85	0.96	1.08
6	Preparation1, Case1	Orderliness1	0.85	0.97	1.07
7	Moderate1	Explain1, Emphasis1	0.81	0.95	1.11
8	Respect1	Answer1	0.81	0.96	1.12
9	Homework1	Emphasis1	0.85	0.92	1.02
10	Interesting1, Preparation1, Orderliness1	Emphasis1	0.80	0.98	1.10
11	Information1, Preparation1	Guidance1	0.81	0.91	1.09
12	Homework1, Information1	Answer1	0.80	0.92	1.07
13	Information1, Preparation1, Orderliness1	Emphasis1	0.84	0.97	1.09
14	Answer1, Explain1	Enlighten1	0.80	0.97	1.10
15	Preparation1	Efficiency1, Emphasis1, Orderliness1	0.83	0.90	1.06
16	Orderliness1, Case1	Effectivity1	0.85	0.97	1.07
17	Interesting1	Emphasis1	0.82	0.97	1.08
18	Enlighten1	Efficiency1, Explain1, Emphasis1	0.82	0.93	1.10
19	Orderliness1, Case1	Efficiency1, Explain1, Emphasis1	0.81	0.93	1.11
20	Enlighten1, Case1, Preparation1	Emphasis1	0.81	0.98	1.09
21	Case1	Interesting1, Explain1	0.81	0.89	1.07
22	Answer1	Explain1, Emphasis1	0.81	0.94	1.09

below:

5.2.2. Association Rule Analysis

Due to the abundance of association rules obtained, this paper will only select a subset for analysis.

According to the analysis of Rules 3, 6, and 15, thorough lesson preparation and using practical examples in explanations are likely crucial factors for organized teaching, inspiring student reflection, focusing on key course elements, and ensuring effective teaching methods. The professionalism and practical skills of teachers in lesson planning and delivery directly impact teaching effectiveness and student learning experiences.

According to the analysis of Rule 7, teaching content of moderate difficulty can prevent students from feeling overly challenged or bored, while also stimulating their interest and deepening their thinking. This balance helps teachers impart knowledge more effectively, enabling students to follow and understand the teaching content more easily, thereby enhancing teaching effectiveness and learning experience.

According to Rule 14, timely grading of assignments, thorough responses to questions, and clear explanations by teachers not only help students better grasp course material but also promote their cognitive development and deepen their learning.

Rule 18 indicates that teachers not only impart knowledge but also stimulate students’ critical thinking abilities. Through effective classroom management and clear explanations, they assist students in better understanding course material and mastering key concepts.

According to Rule 19, when teachers conduct classes in an organized manner, their teaching is usually more structured and logically coherent, which helps students utilize classroom time more effectively. Explaining theoretical concepts through practical examples makes abstract content more concrete and applicable, thereby making it easier for students to understand and master course materials.

6. Conclusions and Future Work

This study employs sentiment analysis and association rule mining techniques to analyze the teaching evaluations of mathematics courses at a university in Beijing, presenting an effective method for mining teaching evaluation data. Firstly, a BERT-LSTM model is constructed to predict the sentiment orientation of subjective evaluations, achieving an accuracy of 92.76% as validated by experimental results, confirming the model’s efficacy. Subsequently, the Apriori algorithm is applied to mine association rules from objective evaluations. From numerous generated rules, 22 significant rules are selected for in-depth analysis. These rules help elucidate the impact of specific teaching behaviors within defined parameter ranges on students’ learning outcomes, while also revealing potential interrelationships among them.

This research underscores the practical utility of text mining and association rule mining methods in extracting actionable insights from teaching evaluation data, thereby providing technical support for educational reform decisions. Regrettably, due to privacy concerns, information regarding evaluated teachers’ gender, age, and title could not be obtained. Access to such information might uncover additional valuable association rules.

Furthermore, this study identifies several issues. Firstly, there exists a high degree of similarity among objective evaluation items, such as “The teaching methods used by the instructor were very effective for me” and “The instructor’s explanations made it easier for me to understand and grasp the course content”, both addressing core aspects of teaching effectiveness. Redundant indicators not only diminish the efficiency of the evaluation system but also hinder subsequent data mining and analysis efforts. Secondly, many subjective evaluations provided by students are overly brief or incomplete, impeding the analysis of teaching evaluations. Research indicates that students are more likely to participate actively in evaluations if they perceive them as beneficial for course improvement. However, skepticism regarding the seriousness of evaluation outcomes or concerns about the time-consuming nature of the process may lead students to opt out [31]. Universities could consider strategies such as providing timely feedback on evaluation results and demonstrating tangible improvements in teaching methods based on evaluation outcomes to inform students that their feedback contributes to meaningful reforms and empowers them to provide thoughtful evaluations.

Lastly, it is essential to acknowledge the limitations of the Apriori algorithm, particularly its efficiency constraints when dealing with larger datasets. Enhancing algorithm efficiency should be a priority in future research efforts to streamline the association rule mining process effectively. This may involve exploring alternative algorithms or optimizing the existing Apriori algorithm to handle large datasets more efficiently.