Improving Low-Resource Chinese Named Entity Recognition Using Bidirectional Encoder Representation from Transformers and Lexicon Adapter

Abstract

Due to their individual advantages, the integration of lexicon information and pre-trained models like BERT has been widely adopted in Chinese sequence labeling tasks. However, given their high demand for training data, efforts have been made to enhance their performance in low-resource scenarios. Currently, certain specialized domains, such as agriculture, the industrial sector, and the metallurgical industry, suffer from a scarcity of data. Consequently, there is a dearth of effective models for entity relationship recognition when faced with limited data availability. Inspired by this, we constructed a suitable small balanced dataset and proposed a based-domain-NER model. Firstly, we construct a domain-specific dictionary based on mine hoist equipment and fault text and generate a dictionary tree to obtain word vector information. Secondly, we use a Lexicon Adapter to obtain the vector information of the domain-specific dictionary feature words matched using the characters and calculate the weights between their word vectors, integrating position encoding to enhance the positional information of the word vectors. Finally, we incorporate word vector information into the feature extraction layer to enhance the boundary information of domain entities and mitigate the semantic loss problem caused via using only character feature representation. Experimental results on a manually annotated dataset of mine hoist fault texts show that this method outperforms BiLSTM, BiLSTM-CRF, BERT, BERT-BiLSTM-CRF, and LEBERT, effectively improving the accuracy of named entity recognition (NER) for mine hoist faults.

1. Introduction

Named Entity Recognition (NER) [1] is a tagging task aimed at identifying and categorizing named entities in a text according to predefined semantic types, including but not limited to individuals, organizations, and geographical locations. In the field of complex mechanical equipment, NER can be used to identify entities such as equipment, components, and types of faults. With NER technology, helpful information can be extracted more quickly and accurately from a large amount of text data, supporting fault diagnosis and repair. Given the specificity and industry-specific nature of the complex mechanical equipment field, it is necessary to develop specific NER models for this domain. Nonetheless, the availability of substantial amounts of labeled training data is vital for their triumph. Regrettably, the process of constructing extensive labeled datasets for every new domain of interest proves to be costly and time-consuming, necessitating a well-educated manpower for annotation tasks. Therefore, effectively addressing the problem of named entity recognition under low-resource [2] conditions has become an important research topic.

While significant progress has been made in Named Entity Recognition (NER) tasks in resource-rich scenarios and low-resource scenarios, deep learning models struggle to train effective feature extraction capabilities due to the lack of annotated datasets. Therefore, enhancing model performance in low-resource NER tasks has recently become a research hotspot. Some researchers have adopted cross-lingual transfer methods [3], primarily transferring knowledge from resource-rich to low-resource languages. For instance, Xie et al. [4] proposed a translation method based on bilingual word vectors to improve the mapping of cross-language text, replacing a Bidirectional Long Short-Term Memory (BiLSTM) [5] with a Self-Attention [6] mechanism as the encoder to enhance the robustness of different language word order differences. Wu et al. [7] proposed a meta-learning algorithm to fine-tune a source language model using a few test examples of the target language to enhance the model’s generalization ability across different languages. Bari et al. [8] achieved an unsupervised cross-language NER model through word-level adversarial learning, parameter sharing, and feature-enhancing fine-tuning.

In addition to cross-lingual transfer learning, some researchers have proposed cross-domain transfer [9], transferring knowledge from resource-rich to low-resource domains. Liu et al. [10] proposed a domain NER model without external resources, enhancing zero-resource domain adaptability’s robustness through multi-task learning and a hybrid entity expert framework. Lin and Lu [11] introduced an adaptive layer on top of an existing neural network structure, eliminating the need to retrain the source domain data. Wang et al. [12] proposed a cross-disciplinary label-aware dual transfer system, mainly through two label-aware rules to achieve feature and parameter transfer.

Unlike the two research directions above, some researchers have proposed cross-task transfer methods, attempting to improve NER task’s effect through utilizing related information from other tasks, such as part-of-speech or boundary information. Kruengkrai et al. [13] proposed a multi-classification model, co-training sentence-level labels, and word-level NER. Sanh et al. [14] used a model to co-train entity extraction and detection, relation extraction, and NER tasks.

The above research revolves around transfer learning, but due to differences between different domains, it is challenging to ensure the effectiveness of knowledge transfer. Some researchers have adopted data augmentation [15,16] methods to mitigate the impact of scarce annotated data to enhance NER performance. However, injecting lexical information into the model for lexical enhancement has become an effective way to improve Chinese entity recognition performance. Liu et al. [17] proposed a model, LEBERT, to solve the Chinese sequence labeling task, mainly through injecting lexical features into the BERT (Bidirectional Encoder Representation from Transformers) model [18] to enhance words. Li et al. [19] proposed a method for constructing a multimodal domain knowledge graph based on LEBERT to solve the problem of vast and dispersed knowledge systems in computer science. Wu [20] et al. introduced an external dictionary to enhance BERT features and the adversarial training entity recognition model LEBERT-BCF, solving the problems of lack of word information, waste of entity boundary information, and poor model robustness in Chinese medical record entity recognition. Most of the above entity recognition research focuses on general domain entity recognition. Due to the presence of a large number of domain entities in the domain dataset, the above methods do not apply to the entity recognition task in the complex mechanical equipment domain without the assistance of domain knowledge.

In recent years, several noteworthy studies have emerged, offering innovative approaches to enhancing low-resource NER. Sabane et al. [21] proposed a method using various adaptations of BERT to enhance the performance of low-resource NER. Chen et al. [22] introduced a Translation-and-fusion framework, which translates a low-resource language text into a high-resource language for annotation using fully supervised models before fusing the annotations back into the low-resource language. Zhou et al. [23] proposed a two-stage learning pipeline to tackle oncological NER task in Chinese language, which is a typical task lacking training resources. Wang et al. [24] proposed SeqUST, a novel uncertain-aware self-training framework for NSL to address the labeled data scarcity issue and to effectively utilize unlabeled data. Chen et al. [25] proposed a CP-NER model to solve the problem of cross-domain resource scarcity in practical scenarios. Experiments show that the model has good performance on both single-source and multi-source cross-domain tasks. Ghosh et al. [26] proposed a novel data augmentation framework, BioAug, for low-resource biomedicine. It is used to solve new text reconstruction tasks based on selective masking and knowledge enhancement. Ghosh et al. [27] proposed Attention-map aware keyword selection for Conditional Language Model fine-tuning, a new data augmentation method based on conditional generation to address the data scarcity problem in low-resource complex NER. Mehta and Varma [28] proposed a multilingual complex named entity recognition method using XLM-RoBERTa. This method utilizes a pre-trained language model for task transfer learning to achieve cross-language and cross-domain entity recognition. Wang et al. [29] proposed a named entity recognition method called GPT- NER in 2023 using a large language model for training, which bridges the gap through transforming a sequence labeling task to a generation task that can be easily adapted by LLMs.

In response to the problem of entity recognition for complex mechanical equipment under low-resource conditions, this study proposes a method for recognizing named entities in Chinese for complex mechanical equipment faults, incorporating dictionary information based on the LEBERT model proposed by Liu et al. [17]. The difference between this method and that of Liu et al. is that, in order to integrate domain knowledge better, this study uses the Word2Vec [30,31] method in the open-source library Gensim [32] to train a domain vocabulary for mine hoist equipment and construct a domain dictionary for mine hoist equipment and faults. A dictionary adapter is embedded between the Transformer encoders in the BERT model to match characters with the domain dictionary and obtain word set information, which is integrated into the character representation to enhance lexical information. This addresses the issues of blurred entity boundaries and semantic loss in domain character representation.

The specific contributions of this study are as follows:

(1)

We have constructed a domain-specific dictionary for complex mechanical equipment to enhance the accuracy of NER. We collected and organized texts on mine hoist equipment and faults, analyzed the fault ontology and related entity types, and built a domain-specific dictionary for mine hoist equipment faults. We integrated this dictionary into the BERT pre-training model to alleviate the issue of blurred entity boundaries in the domain of mine hoist fault texts. The experimental results show that our method can effectively improve the accuracy of NER.

(2)

We propose a simple and effective NER method based on domain adaptation technology for the complex mechanical equipment fault domain, which needs more datasets. This method uses a dictionary adapter to obtain the vector of the word set matched using characters and integrates position encoding to enhance the recognition of entity boundaries. At the same time, we use a Conditional Random Field (CRF) [33,34] as a model classifier to overcome the imbalance problem among samples in the mine hoist domain, thereby improving the model’s recognition performance.

2. Building Domain Dictionaries and Dictionary Matching Methods

2.1. Domain Dictionary Construction

This study collected a large amount of the literature on mine hoist equipment faults from the field maintenance logs, operation manuals, safety procedures, and checklists of a state-owned large non-ferrous metal group, as well as from CNKI, Wanfang Data, and Baidu Encyclopedia. These resources extracted domain-specific terms related to mine hoist equipment faults. The specific extraction methods are as follows:

2.1.1. Statistical Method

Firstly, each corpus was pre-processed. Then, in each corpus, the top 50 words with the highest Term Frequency-Inverse Document Frequency (TF-IDF) [35,36] values were selected. These words were grouped into a set $W=\left\{W_i | 1\le i\le 50\right\}$, where $W_i$ represents the i-th word among these words. Subsequently, the normalized word frequency $t{f}_i$ of each word was calculated. If the condition $t{f}_i\ge n$ is met (where n is a predefined threshold), the word $W_i$ is added to the domain dictionary. Finally, these domain words were subjected to manual screening and validation by experts in the respective domains to ensure the creation of a more accurate domain-specific dictionary.

2.1.2. Manual Method

Using the collected corpus related to mine hoist equipment and its faults from CNKI, Wanfang Data, and Baidu Encyclopedia, professional words with blurred boundaries in the corpus were extracted manually.

This study collected 1295 professional terms related to mine hoist equipment and equipment faults, with an average word length of seven. Examples of these terms are shown in

Table 1. Examples of words in the field of mine hoisting fault.

Entity Type	Examples
Failure phenomenon	The contact tip on the rectifier bar is melted off by heat, Severe speed reducer oscillation…
Fault Location	Hoist spindle bearing, Hydraulic system reducer…
Cause of fault	Spindle bearing breakage, Improper gear meshing…
Fault Impact	Hoist emergency brake, Hydraulic system light up alarm…
Repair measures	Optimization of the lubrication station cooling system, Smooth out the rectifier…

.

As shown in Table 1, domain-specific words are composed of different combinations of words. The boundaries of combined words are blurred in entity recognition. For example, “齿轮啮合不当 (improper gear meshing)” is composed of “齿轮(gear)” and “啮合不当 (improper meshing)”. In conventional entity recognition, it would be recognized as entity types of fault location and fault phenomenon, but in reality, it is a fault cause type.

The extracted domain-specific words were added to the word segmentation tool [37] to avoid incorrect segmentation of words in the corpus. The Word2Vec word vector model in the open-source library Gensim was used to train the mine hoist fault corpus to obtain a domain dictionary composed of 2856-word vectors with a word vector dimension of 200.

2.2. Dictionary Tree

A prefix tree [38,39] is an efficient data structure. It can store vocabulary in the domain of mine hoist equipment, thereby constructing a domain dictionary tree. For instance, we can store the three strings “矿井提升机 (Mine hoist)”, “矿井主提升机 (main mine hoist)”, and “矿井副立井提升机 (auxiliary mine hoist)” in the prefix tree, as shown in Figure 1.

In this prefix tree, each node represents a character and the path from a root node to a leaf node represents a string. For example, the path from the root node “矿 (mine)” to the leaf node “hoist” represents the string “矿井提升机 (mine hoist)”. Storing strings in a prefix tree effectively reduces storage space because it saves space through sharing common prefixes. Moreover, string retrieval using a prefix tree is very efficient because we only need to search along the link that matches the first letter of the target string. This minimizes the number of string comparisons, thereby enhancing retrieval efficiency. The specific steps of the dictionary tree construction Algorithm 1 are as follows:

Algorithm 1: Dictionary tree construction algorithm

Input: $s_c=\left.\{c_1,c_2,\dots ,c_n\right\}$, a sentence with n characters $s_c$

Output: $s_{cw}=\left\{\left(c_1,w{s}_1\right),\left(c_2,w{s}_2\right),\dots ,\left(c_n,w{s}_n\right)\right\}$

Initialize the word list $W{S}_i=\left\{w{s}_1,w{s}_2,\dots ,w{s}_n\right\}$, which is derived from a pre-compiled list of domain-specific terms, where each ws represents a unique term. Initialize the dictionary tree root node: $root$ for $i\leftarrow 1$ to n do Initialize tree node: $node$ repeat   Iterate through the word list $WS$ to get the words matching the matching words $w{s}_i$   Take $\left(c_i,w{s}_i\right)$ Constructing dictionary tree sub-nodes: $sub$   $node\to next=sub,node=sub$ until the traversal is complete $WS$ stop $root\to next=node$ end for

2.3. Lexicon Adapter

Dictionary matching involves embedding a dictionary adapter between the BERT Transformer encoders. Therefore, we designed a dictionary matching algorithm to match character-level feature vectors with word-level feature vectors through the constructed domain word dictionary, thereby obtaining more fusion weights than character-level feature vectors. As shown in Figure 2, the “矿 (mine)” in “矿井提升机 (mine hoist)” matches the word set “矿井 (mine shaft)” and “矿井提升 (mine hoist)” through the domain dictionary tree. If a character does not match any word in the domain dictionary, the word set of that character is defined as “None”. The Algorithm 2 steps are as follows:

Algorithm 2: Dictionary matching algorithm

Input:$WS=\left\{w{s}_1,w{s}_2,\dots ,w{s}_n\right\}, Y=\left\{y_1^c,y_2^c,\dots ,y_n^c\right\}$, $w{s}_1$ represents the i-th character assigned to match the word, where $y_1^c$ is a character vector.

Output: $H=\left\{h_1,h_2,\dots ,h_n\right\}$

1. for $i\leftarrow 1$ to n do   for $j\leftarrow 1$ to n do    $w_{ij}=w{s}_{ij}$   $x_{ij}^{ws}=e^w\left(w_{ij}\right)$, where $e^w$ is pre-trained word embedding lookup table and $w_{ij}$ is the j-th word in $w{s}_i$.   end for  end for $X_{ij}^{ws}=\left\{x_{i1}^w,x_{i2}^w,\dots ,x_{im}^w\right\}$ 2. A nonlinear transformation $v_{ij}^w=W_2\left(\tanh \left(W_i{x}_{ij}^w+b_1\right)\right)+b_2$ of the word embedding vector $X_{ij}^{ws}$, yields $V_i=\left\{v_{i1}^w,v_{i2}^w,\dots ,v_{im}^w\right\}$ 3. for $i\leftarrow 1$ to n do 4. Calculating word and domain feature word correlation $a_i=softmax\left(y_i^c{W}_{attn}{V}_i^T\right)$ 5. Calculate the weighted sum of all words $z_i^w={{\displaystyle \sum }}_{j=1}^m{a}_{ij}{v}_{ij}^w$ 6. Weighted lexical information injection character vector $\widetilde{h_i}=y_i^c+z_i^w$ 7. end for

Figure 2. Diagram of the complex machinery and equipment entity recognition model incorporated into the lexicon. In this example, we used “矿井提升机 (mine hoist)” as input data and extracted the domain words: “矿 (Ore)”, “矿井 (Mines)”, “提升机 (Hoister)”, “矿井提升 (Mine Hoisting)”, and “矿井提升机 (Mine hoist)”. Taking the word “矿 (Ore)” as an example, we matched the domain words “矿井 (Mines)” and “矿井提升 (Mine Hoisting)” from the domain dictionary tree to obtain their relative word vectors and complete the dictionary matching. Among them, TrieRoot is the root node of the domain dictionary tree.

Figure 2. Diagram of the complex machinery and equipment entity recognition model incorporated into the lexicon. In this example, we used “矿井提升机 (mine hoist)” as input data and extracted the domain words: “矿 (Ore)”, “矿井 (Mines)”, “提升机 (Hoister)”, “矿井提升 (Mine Hoisting)”, and “矿井提升机 (Mine hoist)”. Taking the word “矿 (Ore)” as an example, we matched the domain words “矿井 (Mines)” and “矿井提升 (Mine Hoisting)” from the domain dictionary tree to obtain their relative word vectors and complete the dictionary matching. Among them, TrieRoot is the root node of the domain dictionary tree.

3. Method

This study proposes an entity recognition method for complex mechanical equipment that incorporates a dictionary. The model framework, as shown in Figure 2, includes the following three parts:

(1)

Input Layer: This layer represents the input text in terms of individual characters, which serves as the data for subsequent character-to-domain dictionary matching.

(2)

Character Encoding Layer: This layer uses the Transformer encoder and dictionary adapter to obtain vector representations of the characters in the sentence sequence of mine hoist faults input into the model.

(3)

Label Prediction Layer: This layer uses a Conditional Random Field (CRF) to predict the entity labels corresponding to each character in the text of mine hoist faults.

3.1. BERT-Based Character Vector Representation

For a given text, it is considered as a sequence of characters $s_c=\left\{c_1,c_2,\dots ,c_n\right\}$, which serves as the input to the Input Embedding of BERT to get the output $E=\left\{e_1,e_2,\dots ,e_n\right\}$. The $E$ is then fed into the Transformer encoder in BERT, and each Transformer encoder is represented in the following form:

$$G=Layernormalization\left(H^{l-1}+Multiheadattention\left(H^{l-1}\right)\right)$$

(1)

$$H^l=Layernormalization\left(G+FFN\left(G\right)\right)$$

(2)

where $H^l=\left\{h_1^l,h_2^l,\dots ,h_n^l\right\}$ denotes the output of the l-th layer and $H^0=E$; FFN is a two-layer feed-forward network with ReLU as the hidden activation function.

3.2. Lexicon Adapter

In Figure 2, we obtain a set of words through matching characters with a specialized dictionary tree. We use the dictionary matching algorithm to compute the word-level feature vectors with a higher fusion weight than the character-level feature vectors. We then combine this set of word vectors with the character vectors.

3.2.1. Char-Words Pair Sequence

Given a dictionary tree of feature words in the mine hoist domain and a sentence $s_i=\left\{w_1,w_2,\dots ,w_n\right\}$ containing n characters. First, all the character subsequences of the sentence are traversed, and all the character subsequences are matched with the dictionary tree using a lexicon fusion adaptation algorithm to obtain potential matching words, where each word corresponds to at most three words and less than three words are filled with <PAD>. For example, Figure 1 depicts a case in which there are four feature words in the trie: “矿井 (Mines)”, “矿井提升 (Mine hoisting)”, “提升机 (Lifter)”, and “矿井提升机 (Mine hoist)”. When the input sentence is “矿井提升机 (Mine hoist)”, its characters “矿 (Ore)”, “井 (The wells)”, “提 (Lift)”, “升 (Liter)”, and “机 (Machine)” match the feature words containing their characters in the trie, and through doing so, they can obtain the corresponding feature words and accomplish the matching of characters to feature words.

3.2.2. Lexicon Enhanced BERT

Lexicon Enhanced BERT is a combination of Lexicon Adapter and BERT, in which the Lexicon Adapter is applied to a certain layer of BERT as shown in Figure 3. For a given Chinese $s_c=\left\{c_1,c_2,\dots ,c_n\right\}$, it is constructed as a word–word pair form.

$$s_{cw}=\left\{\left(c_1,w{s}_1\right),\left(c_2,w{s}_2\right),\dots ,\left(c_n,w{s}_n\right)\right\}$$

(3)

After obtaining the corresponding Transformer encoder from Equations (1) and (2), the lexical information is injected between the k-th and (k+1)-th Transformer layers through the Lexicon Adapter. The output of the k-th Transformer layer is $H^k=\left\{h_1^k,h_2^k,\dots ,h_n^k\right\}$. Each pair $\left(h_i^k,x_i^{ws}\right)$ of these is transformed using the Lexical Adapter (LA) to obtain:

$$\tilde{h}{x}_i^k=LA\left(h_i^k,x_i^{ws}\right)$$

(4)

Then, the feature vectors injected with lexical information are output to the remaining (L-k) Transformer layers.

3.3. Entity Label Prediction Layer

As shown in Figure 1, the probability of predicting the label sequence $y=\left\{y_1,y_2,\dots ,y_n\right\}$ by text $s=\left\{c_1,c_2,\dots ,c_n\right\}\in V_c$ is calculated as shown in Equation (5).

$$\left\{\begin{array}{c}{P}_i=W_P+b_P\\ p\left(y|s\right)=\frac{exp\left({{\displaystyle \sum }}_{i=1}^n\left(P_{i,y_i}+T_{y_{i-1},y_i}\right)\right)}{{{\displaystyle \sum }}_{y\in Y_c}exp\left(P_{i,y_i}+T_{y_{i-1},y_i}\right)}\end{array}\right.$$

(5)

where $W_p$, $b_p$ are the parameters for calculating the score matrix P and T is a transfer matrix.

4. Experiments and Results

4.1. Datasets

This study focuses on entity recognition tasks in the field of mine hoist equipment faults. Since no public dataset exists in this field, we extracted fault text information from the on-site maintenance logs, operation manuals, safety procedures, and equipment checklists of a large state-owned non-ferrous metal group. We collected the literature related to mine hoists from CNKI, Wanfang Data, Baidu Encyclopedia, and other websites. The collected corpus was preprocessed, such as removing stop words, deleting unrecognizable special symbols, and useless spaces between strings, resulting in a total of 7732 pieces of text corpus in the field of mine hoists. The generated corpus was manually annotated using a Label-Studio annotation platform, with the annotation method being BIO (Begin Inside Outside). For example, in the sentence “齿轮啮合不当导致油温过高 (Improper gear meshing causes high oil temperature)”, “齿轮 (gear)” is the “Fault Location”, annotated as FLN; “啮合不当 (improper meshing)” is the “Fault Cause”, annotated as FCE; “油温过高 (high oil temperature)” is the “Fault Phenomenon”, annotated as FPN; and information unrelated to the entity is annotated as O. An annotation example is shown in Figure 4. During the process of manual annotation, our team discovered that Chinese sentences possess a diverse range of expression patterns that prevent traditional Named Entity Recognition (NER) models from accurately identifying the desired entities. To adapt to the various ways of expressing Chinese sentences, we adopted a dictionary embedding approach to solve this problem.

Table 2. Corpus segmentation comparison with added dictionary.

Datasets	Not Added to Dictionary	Add to the Dictionary
减速器传动轴弯曲 (The drive shaft of the reducer is bent)	减/速/器/传/动/轴/弯/曲 (The drive shaft of the reducer is bent)	减速器/传动轴/弯曲 (The drive shaft of the reducer is bent)
齿轮啮合不当导致油温过高 (Improper gear meshing causes high oil temperature)	齿/轮/啮/合/不/当/导/致/油/温/过/高 (Improper gear meshing causes high oil temperature)	齿轮/啮合不当/导致/油温过高 (Improper gear meshing causes high oil temperature)

shows the corpus with and without dictionary entries, allowing for a better understanding of the effectiveness of our method.

This study follows the current national standards for mine hoists and divides entity types into five categories: Fault Phenomenon (FPN), Fault Cause (FCE), Fault Location (FLN), Fault Effect (FET), and Repair Measures (RMS). Based on the corpus annotation, a mine hoist dataset was built, and according to a ratio of 80%, 10%, and 10%, 6182 training sets, 775 validation sets, and 775 test sets were obtained.

4.2. Experiment Parameter Settings

The experimental model was built using the PyTorch framework, the underlying encoder was pre-trained with the BERT-wwm [40] language model, and the model parameters were learned adaptively using the Adam optimizer. The experimental environment setup is shown in

Table 3. Experimental environment setup.

Experimental Environment	Configuration
Operating System	CentOS 7.5 64bit
GPU	NVIDIA Tesla P100
Python	3.7
PyTorch	1.5.1 + cudaa92

.

The validation set was used to determine the values of the parameters, and all the experimental parameters were set as shown in

Table 4. Experiment parameter settings.

Parameter Name	Value
Word2vec word vector dimensionality	200
Learning Rate	1 × 10−5
Batch Size	16
Random Discard Rate	0.1
Hidden layer dimension	768
Number of attention mechanism heads	12
Maximum sentence length	128
Maximum fusion of vocabulary information per Chinese character	3

when the model achieved the optimal F1 value on the validation set.

4.3. Evaluation Indicators

For the method proposed in this study, an entity was considered correctly identified when and only when its boundary and class were correctly identified. To facilitate comparison with traditional models, the precision rate Precision (P), recall rate Recall (R), and F1 value (F1-score) were used as quantitative evaluation metrics of the model.

$$P=\frac{TP}{TP+FP}\times 100\%$$

(6)

$$R=\frac{TP}{TP+FN}\times 100\%$$

(7)

$$F_1=\frac{2\times P\times R}{P+R}\times 100\%$$

(8)

where TP (True Positives) indicates the number of entities in the test set for which the category was correctly identified, FP (False Positives) indicates the number of entities for which the category was incorrectly identified, and FN (False Negatives) represents the number of entities that were not identified.

4.4. Experiment Results and Analysis

To validate the effectiveness of the proposed method, we conducted comparative experiments with existing entity recognition methods. We set up ablation experiments to verify the impact of the domain dictionary on model performance.

4.4.1. Relevant Comparison Experiments

We conducted comparative experiments with five methods published in recent years, as follows:

(1)

In 2015, Wang et al. [41] proposed a solution based on word embedding and bidirectional LSTM recursive neural networks for unified tagging.

(2)

In 2015, Baidu-Research [5] proposed a solution for text annotation using bidirectional Long Short-Term Memory (LSTM) networks and Conditional Random Fields (CRF).

(3)

In 2018, Devlin et al. [18] proposed BERT, a deep bidirectional pre-training language model based on Transformer.

(4)

In 2020, Xie et al. [42] proposed a research method based on the BERT-BiLSTM-CRF model.

(5)

In 2021, Liu et al. [17] proposed LEBERT, a model for solving Chinese sequence labeling through integrating lexical information into the underlying encoding process of BERT.

To verify the practicality of the method proposed in this study, we selected BiLSTM, BiLSTM-CRF, BERT, BERT-BiLSTM-CRF, and LEBERT as baseline models for comparative experiments. The results of the comparative experiments are shown in

Table 5. Relevant comparison experiment results.

Model Name	P	R	F1
BiLSTM	38.45%	40.33%	39.37%
BiLSTM-CRF	68.27%	64.24%	66.20%
BERT	83.27%	91.98%	83.27%
BERT-BiLSTM-CRF	89.22%	94.37%	91.73%
LEBERT	97.01%	98.20%	97.60%
Method of this study	97.33%	98.39%	97.86%

.

As shown in Table 5, the entity recognition performance of the method proposed in this study on the mine hoist dataset is significantly superior to that of models such as BiLSTM, BiLSTM-CRF, BERT, and BERT-BiLSTM-CRF. This is primarily due to the specificity of the text describing mine hoist faults, which can easily lead to entities belonging to different types due to varying contexts. Consequently, the BiLSTM method’s lack of perception of local features and information loss becomes more apparent. However, the introduction of BERT has led to a particular improvement in the accuracy, recall, and F1 score of entity recognition compared to the BiLSTM and BiLSTM-CRF models, effectively addressing the issue of entity polysemy. After the LEBERT model incorporated word features into text embedding, compared to BERT on the mine hoist dataset, the precision of entity recognition increased by 13.74%, and the F1 score increased by 14.33%. Through constructing a mine hoist domain dictionary, the method proposed in this study is more sensitive to semantic changes at the word-level brought about via domain-specific terminology. Compared to BERT, the accuracy of entity recognition increased by 14.06%, and the F1 score increased by 14.59%. In comparison with LEBERT, the accuracy of entity recognition only increased by 0.32%, and the F1 score increased by 0.26%. The domain dictionary can effectively enhance the entity recognition effect in domains with scarce datasets.

4.4.2. Ablation Experiments

In order to validate the effectiveness of incorporating domain-specific dictionaries into our method, we conducted three experiments: one without dictionary incorporation, one with integrating Tencent’s public dictionary, and one with integrating a domain-specific dictionary.

As depicted in Figure 5, with an increase in the number of iterations, a significant improvement in the F1 scores was observed across all three experiments. This is attributed to the model’s ability to learn more feature information through continuous iterations.

A comparison was made between the F1 scores at the final iteration. The experiment incorporating Tencent’s public dictionary showed a 4.87% increase in the F1 score compared to the experiment without dictionary incorporation. This improvement is due to the semantic information in Tencent’s public dictionary, underscoring the importance of dictionary incorporation for the semantic representation of text. In the experiment incorporating the domain-specific dictionary, the F1 score improved by 1.16% compared to the experiment incorporating Tencent’s public dictionary. This is because the domain-specific dictionary contains semantic information and entity boundary information specific to the field of mine hoist faults, resulting in better recognition of entities related to mine hoist faults.

5. Conclusions

To address the structural features of fault text entities in complex mechanical equipment, this study proposes a method for recognizing fault entities in mechanical equipment through incorporating a domain-specific dictionary. This dictionary, which covers fault causes, locations, impacts, and repair measures, was constructed. An adapter for dictionary integration was embedded between two Transformer layers of the BERT model, enabling character matching with the set of words in the domain-specific dictionary. The weighted word vector information was then injected into the character vector, facilitating a more effective integration of the domain-specific dictionary into the character vector representation. The experimental results indicate that our method offers specific improvements in Chinese NER fault texts of complex mechanical equipment. As a future direction, we plan to investigate the nested relationships between entities in the domain of complex mechanical equipment faults. This will allow us to extract more domain-specific words, construct a more extensive professional vocabulary, and address the issue of poor entity recognition results due to words not appearing in the domain-specific dictionary.