Knowledge Embedding Relation Network for Small Data Defect Detection

Abstract

In industrial vision, the lack of defect samples is one of the key constraints of depth vision quality inspection. This paper mainly studies defect detection under a small training set, trying to reduce the dependence of the model on defect samples by using normal samples. Therefore, we propose a Knowledge-Embedding Relational Network. We propose a Knowledge-Embedding Relational Network (KRN): firstly, unsupervised clustering and convolution features are used to model the knowledge of normal samples; at the same time, based on CNN feature extraction assisted by image segmentation, the conv feature is obtained from the backbone network; then, we build the relationship between knowledge and prediction samples through covariance, embed the knowledge, further mine the correlation using gram operation, normalize the power of the high-order features obtained by covariance, and finally send them to the prediction network. Our KRN has three attractive characteristics: (I) Knowledge Modeling uses the unsupervised clustering algorithm to statistically model the standard samples so as to reduce the dependence of the model on defect data. (II) Covariance-based Knowledge Embedding and the Gram Operation capture the second-order statistics of knowledge features and predicted image features to deeply mine the robust correlation. (III) Power Normalizing suppresses the burstiness of covariance module learning and the complexity of the feature space. KRN outperformed several advanced baselines in small training sets on the DAGM 2007, KSDD, and Steel datasets.

1. Introduction

Industrial quality inspection is one of the important application directions of computer vision in smart factories. However, in the industrial environment, defect samples are very scarce, and there may even be zero samples for some defect categories [1]. This makes small training sets a key constraint on the industrial implementation of many data-driven algorithms [2].

In the context of small training datasets, industry scholars have explored two main approaches: increasing the amount of data and reducing the dependency of algorithms on data. The former approach mainly generates new data through augmentation or introduces new data from other datasets [3,4], while the latter focuses on model improvement [5] and algorithm optimization [6] to enhance the feature extraction capabilities of small training sets. However, these methods from other machine learning applications in industrial manufacturing may not be directly applicable to industrial computer vision problems. For instance, in defect detection, defects may not be consistently present, or only a limited number of samples may be available over an extended period. Publicly available defect datasets also suffer from a scarcity of defect samples: the KolektorSDD dataset [7] contains only 52 defect samples out of 399 total, the AITEX dataset [8] has only 105 defect samples, and each category in the RSDDs dataset [9] has only 300 samples. This makes it challenging to effectively apply the aforementioned methods in these contexts.

Therefore, we believe that the small training set of industrial vision requires the introduction of external knowledge. Crucially, we noticed that there is an important difference between defect detection and target detection, that is, in addition to defect samples in the support set of defect detection, standard samples of products are also given, but target detection only has a small number of object samples [10].

Most of the existing automatic inspection equipment manufacturers do not use data-driven AI algorithms for automatic optical inspection, and their equipment generally still uses artificially designed features, although these thresholds test the engineer’s ability to adjust parameters [11]. In fact, the deep learning algorithm can capture some latent features so that it far exceeds the ability of traditional algorithms in a single dataset, but in the actual industrial production environment, it cannot mine the correct features from limited samples, that is, the dataset External defect characteristics. The traditional “statistical modeling + similarity matching” algorithm can be more robust.

Inspired by this, we propose an idea to perform statistical modeling on standard samples as an auxiliary knowledge representation. This standard sample will participate in prediction and assist in identifying product quality inspection. Unlike other algorithms, we use standard samples as a priori knowledge rather than an input. Specifically, we introduce the Gram Matric for Knowledge Modeling. The Gram Matric has achieved outstanding results in the field of style transfer [12]. Some scholars have introduced it into few-shot classification and showed amazing results [13]. We believe that the covariance operation in the Gram Matric can deeply mine pixel-level feature correlations, which can also be considered to be related to texture features. Therefore, we introduce it into Knowledge Modeling and use it as an adjunct to enhance the identification of known and unknown defects.

The main contributions of this paper are as follows:

• An aim at the lack of defect samples in industrial quality inspection scenarios, starting from external knowledge, using statistical modeling methods to build standard templates, using them as prior knowledge, and designing a defect detection network enhanced by prior knowledge;
• In order to measure the difference between the standard sample and the predicted sample, as well as to embed this difference into the feature for subsequent head defect identification, a Knowledge-Embedding module based on self-attention was designed;
• In order to obtain the relationship between features in the vector space and mine weak clues, we designed an eccentric covariance matrix to extract the characteristics of each dimension of the statistic, we automatically adjusted the unnecessary information in the extraction process avoiding interference from the cluttered background.
• We demonstrated the effectiveness of this method on the public DAGM2007 [14], KolektorSDD [15], and Severstal Steel defect detection datasets.

2. Literature Review

Small Training Sets. Small training sets has always been a huge challenge in the application practice of deep machine vision. Even small training sets leads to a machine learning task of “decomposing the dataset into different meta tasks to understand the generalization ability of the model when the category changes”—otherwise known as Few-Shot Learning [16]. There are three methods of small training sets: data augmentation, model improvement, and algorithm optimization [17]: Data augmentation is used to expand the training data through various image methods to achieve the effect of increasing training samples, such as mixup [18], adding noise [19], and generating samples based on GANs [20]. In recent years, Pseudo-Labeling [21,22] has also become an effective method to improve performance points. Model improvement refers to adjusting the model structure to enhance the feature extraction ability [23]. The optimization algorithm is used to adjust learning strategies to improve algorithm performance. Semisupervised and unsupervised have also become a popular idea (to solve small training set problems) [24]. This paper focuses on introducing a priori knowledge and improving the model to reduce the dependence of the algorithm on defect data.

Active Shape Model. The statistical model of the PCB standard board has a strong positive impact on defect detection. The statistical shape modeling technology was proposed by Cootes in his paper [25] in 1995. It is a deformable model in computer vision, which is used to model the shape in the image. This method only needs to establish a flexible mathematical model and only needs to compare each time. Using this method, the debugging efficiency of the AOI is accelerated, and the misjudgment rate is reduced. Inspired by this, this paper encodes the standard image through CNN, makes statistical analysis on the standard samples by using the clustering algorithm, obtains representative standard samples, and constructs the standard template as the representation of a priori knowledge.

Self-Attention Modules. They have been successfully applied in NLP [26] and physical system modeling [27]. The self-attention mechanism can capture the relationship between the original sentence and the target sentence in natural language processing, and replace the recurrent neural network with an attention model, so as to realize parallel implementation and more efficient learning. These works inspire us to deduce the variant of knowledge embedding based on correlation mining. We converted the original elements from words to conv features and employed the knowledge model of the predicted image. We used this mechanism to establish the knowledge embedding method in the feature mapping from the low dimension to the high dimension.

Gram Matrix. In fact, it can be regarded as an eccentric covariance matrix between features, that is, a covariance matrix without mean subtraction. Second-order statistics have been studied in the context of texture recognition through so-called regional covariance descriptors (RCDs), which were further applied to object class recognition [28]. Co-occurrence patterns can also be used in the CNN setting. A recent approach [29] extracted feature vectors at two separate locations in a feature map and performed an outer product to form a CNN co-occurrence layer. Higher-order statistics have also been used for fine-grained image classification [30] and domain adaptation [31]. SoSN utilizes second-order information and power normalization for end-to-end training with one- or few-shot learning. Based on the second-order statistics applied to these matrics, we designed a multi-relational feature descriptor that captures deep relationships between proposals before being passed to the classification network for defect identification.

3. Research Methods

Below, we introduce our deep template matching defect detector network and then describe its individual components.

3.1. Overview

In this paper, this method operates on the so-called small training set defect detection, which is essentially a classification task. However, in different scenarios, defects are detected, and segmentation is also a task requirement. Taking classification as the main goal, we evaluate the segmentation and detection of defects.

Different from some defect detection schemes that simply add negative samples, we use standard samples as a priori knowledge to identify defects by mining feature relationships. Our Knowledge-Embedding Relational Network (KRN) consists of (i) an encoding network, (ii) Knowledge Modeling, (iii) Knowledge Embedding, (iv) a Gram Operation, and (v) a Prediction Network. Figure 1 shows an example of an architecture that supports images.

The role of the Encoding Network is to generate image-level convolutional feature vectors (descriptors), and our Encoding Network includes the segmentation part. The task of the Knowledge Modeling part is to perform statistical modeling on multiple standard samples in order to obtain a knowledge representation that can assist in enhancing defect detection. The knowledge association embedding module is an operation of mining the relationship between prior knowledge and predicted samples, aiming to promote the fusion of prior knowledge and predicted samples. The task of the Gram Operation is to use the Gram Matrix to mine the latent relationship between each feature vector so as to make the defect salient. Finally, the Predictive Network learns and recognizes this knowledge-embedded relation mining feature.

3.2. Encoding Network

The feature encoding network is responsible for generating convolutional feature vectors, which serve as image descriptors. To address the challenges of sample concentration, high resolution, and small target scenes in industrial visual defect detection tasks, this paper utilizes a convolutional neural network architecture based on ESDN [32]. Specifically, we employed the Segmentation Network and Decision Network, which perform downsampling by a factor of 32, as the feature encoding network. It is important to note that the segmentation component was used as an auxiliary module for feature extraction.

The Encoding Network can be described as $f:({\mathbb{R}}^{W\times H};{\mathbb{R}}^{\left|\mathcal{F}\right|})\to {\mathbb{R}}^{W\times H}$, where W and H represent the width and height of the input image.The Encoding Network f is a convolutional neural network specifically designed for feature extraction in industrial visual defect detection tasks. It takes an input image of size $W\times H$ and produces a feature map of the same spatial dimensions. This network includes multiple convolutional layers, which downsample the input by a factor of 8. After downsampling, the output is split into two branches: one branch undergoes segmentation using a 1 × 1 convolution, and the result is concatenated with the original downsampled feature map. This architecture is optimized to retain defect details by operating at a middle scale, balancing the trade-off between computational efficiency and the preservation of important features.

3.3. Correlation Knowledge Embedding

3.3.1. Knowledge Modeling

Defect detection is a small training sets recognition task, and a large number of normal samples can be used as a reference. Therefore, we specially designed the Knowledge Modeling module, which aims to perform knowledge mining on normal samples for the reference of defect recognition.

Specifically, we used resnet50 to map standard samples to high-dimensional features and convert them into tensors. The weight is trained on Imagenet and has extensive classification ability. Then, n representative images are selected from a large number of standard samples by a clustering algorithm, and then a multi-dimensional image composed of overlapping standard images is defined as $X_{norm}\in {\mathbb{R}}^{W\times H}$, that is, the constructed statistical knowledge.

3.3.2. Knowledge Embedding

Then, we designed a knowledge embedding method based on self-attention, as shown in Figure 2.

The simplest knowledge fusion is concat and add operations, but we hope that this fusion can excavate the correlation between the two to a certain extent for fusion. Referring to the self-attention mechanism, we designed a knowledge embedding method based on relationship mining. In our Knowledge Embedding module, the input consists of conv feature ${\Phi }_{pre}$ and knowledge feature ${\Phi }_{norm}$; ${\Phi }_{pre}$ comes from the last convolution output feature of the Encoding Network. ${\Phi }_{norm}$ is the shallow tensor obtained by $X_{norm}$ through three maxpoling and three groups of convolution sampling, as shown in Formula (1).

$$\begin{array}{c}\hfill {\Phi }_{norm}=f_{norm}\left(X_{norm};F\right),{\Phi }_{norm}\in {\mathbb{R}}^{K\times N}\end{array}$$

(1)

where F are the parameters to learn of three convolution layers in the Knowledge Model.

A dot product is performed between the conv features and knowledge features to obtain their correlation. A softmax function is applied to obtain the weights on the values. Given matrices ${\varphi }_{norm}$ (by flattening ${\Phi }_{norm}$) and matrices ${\varphi }_{pre}$ (by flattening ${\Phi }_{norm}$), their correlation is computed as follows:

$$\begin{array}{c}\hfill M=s\left({\Phi }_{norm},{\Phi }_{pre}\right)=softmax\left({\varphi }_{norm}{\varphi }_{pre}^T\right)\end{array}$$

(2)

This correlation will be embedded into the original conv feature through multiplication. The output is computed as

$$\begin{array}{c}\hfill {\varphi }_{out}=mul\left(M,{\varphi }_{pre}\right)=softmax\left(dot\left({\varphi }_{norm},{\varphi }_{pre}^T\right)\right){\varphi }_{pre}\end{array}$$

(3)

Finally, the feature matrix after Knowledge Embedding will be reshape as a feature maps.

In addition, after visiting an electronic factory, we speculated that adding the standard template would help the consumer electronics industry with some wrong parts and defects. They had no appearance damage defects, but the welded components were inconsistent with the design drawings. Having been limited by this dataset, we may test it in future practice.

3.4. Gram Operation

The Gram Matrx is an operation to deeply mine the correlation between features, as shown in Figure 3. We used it as a feature mining tool for defect textures and normal textures. Its input is a feature vector from Knowledge Embedding, which we define as $\Phi ={\left\{{\varphi }_n\right\}}_{n\in \mathcal{N}}$. Then, we used $\varphi {\varphi }^T=\uparrow {\otimes }_2\varphi$ to denote the covariance operation of the eigenvectors. Taking $\Phi$ for example,

$$\begin{array}{c}\hfill \Psi \left({\Phi }_n\right)={\displaystyle \frac{1}{N}}\sum _{n\in {\mathcal{N}}_s}\uparrow {\otimes }_r{\varphi }_n=\Psi \left({\left\{{\varphi }_n\right\}}_{n\in {\mathcal{N}}_s}\right)={\displaystyle \frac{1}{N}}\sum _{n\in {\mathcal{N}}_s}{\varphi }_n{\varphi }_n^T\end{array}$$

(4)

KE and GO capture feature correlation through covariance, which essentially introduces second-order statistics. Second-order statistics have to deal with the so-called burstiness, which is “the property that a given visual element appears more times in an image than a statistically independent model would predict”. Power Normalization [13] is known to suppress this burstiness and has been extensively studied and evaluated in the context of Bag-of-Words and Few-Shot Learning. Therefore, we adopted SigmE PN which is defined as

$$\begin{array}{c}\hfill {\mathcal{G}}_{SigmE}(M,\eta )={\displaystyle \frac{2}{1+e^{\frac{-{\eta }^{\prime }M}{Tr\left(M\right)+\lambda }}}}-1\end{array}$$

(5)

where $1\le \eta \approx N$ interpolates between counting and detection, $\lambda \approx l{e}^{-6}$ is a regularization constant, and the trace $Tr$ stops the diagonal from exceeding 1.

After calculating the Gram Matrix, it is replicated 16 times in the channel dimension to ensure compatibility with the subsequent predict network layers, which are designed to process a specific number of input channels. While using a single-channeled Gram Matrix could reduce computational redundancy, this would require significant architectural changes to the predict network, potentially affecting its performance and stability. The replication maintains the continuity and integrity of the convolutional bottleneck structure without altering the network’s existing architecture.

3.5. Predictive Network

The function of the Prediction Network is to mine and judge the features of the Knowledge Embedding, as well as realize the detection of the target image. We did not directly predict the features of the Knowledge Embedding but used three Conv Blocks: the specific parameters are two Conv $1\times 1$ with channel 32 and one Conv $3\times 3$ with channel 16, implemented using convolution operations to facilitate feature mining between standard templates and predicted samples.

3.6. Loss Function

The loss function of the Knowledge-Embedding Relation Network (DKER) consists of two parts: the loss $L_{seg}$ for segmentation that is assisted and the classification loss $L_{cls}$. The total loss can be denoted as $L=\lambda Lseg+\delta \left(1-\lambda \right)L_{cls}\left(M\right)$, where $\lambda$ is a simple linear function, and $\delta$ is a weight coefficient with a small value. The detailed sets were defined as in [32].

4. Experiments

Below, we experimentally demonstrate the merits of our Knowledge-Embedding Relation Network. Our method was mainly evaluated on the DAGM2007, KolektorSDD, and Severstal Steel datasets. We compared with other advanced algorithms, designed a small training sets test, and conducted ablation experiments.

4.1. Datasets

DAGM2007 contains texture data of 10 categories, and each category contains 1000 negative samples and 150 positive samples saved in grayscale 8-bit PNG format. The training set and test set of each category were allocated in a proportion of 1:1, and the size was a $512\times 512$ image. In addition, we explored small training sets scenarios of 5, 10, 15, 20, and 25 positive samples in proportion. It should be pointed out that the standard sample modeling in all training was established from the standard samples of the complete dataset.

The KolektorSDD dataset includes eight non-overlapping images collected from each commutator surface of 50 defective electronic commutators, and a total of 399 images were obtained, including 52 defective images and 347 defect-free images. All data settings refer to dagm, except that the image size was $1408\times 512$ for smaller datasets. In addition, our ablation experiment was evaluated on the KSDD dataset with five positive samples.

The Severstal Steel dataset is from the Kaggle Challenge, which contains 12,568 images and involves four kinds of defects. In the effectiveness demonstration of this method, we adopted the scheme of 1000 positive samples, and the size of the input image was $256\times 1600$. In the small training sets scenario, our data settings were the same as above, but the number of test sets was not changed (in order to be more consistent with the real scenario).

4.2. Implementation Details

All codes were implemented in PyTorch. All experiments were tested in the PyTorch framework under the Ubuntu system, and two Titan Xs were used for GPU acceleration. For the learning rate, we followed the learning scheme [32], and DAGM adopted $LR=0.01$ and $\delta =1$. KSDD adopted $LR=0.5$ and $\delta =0.01$, as well as $LR=0.1$ and $\delta =0.1$. However, for the number of learning iterations, due to the introduction of high-order moments, the high-dimensional mapping of features dragged down the convergence speed to a certain extent at the beginning of training, so we adjusted the number of iterations of the experiment: in the small training sets of 5, 10, 15, 20, and 25 positive samples, we trained 350, 190, 170, 150, and 140 epochs, respectively. In the complete experiment, the three data trained 150 epochs.

We conducted less exploration on training tricks and paid more attention to less sample training. In the following experiments, we compare the research results with several advanced methods, and we report the commonly used matrices for KRN, such as AP, FP, and FN.

4.3. Comparison with the State of the Art

4.3.1. DAGM2007

The proposed KRN was evaluated on the DAGM 2007 dataset, and the obtained true positive rate (TPR) and true negative rate (TNR) are shown in

Table 1. mAP on four methods (DAGM, 150 positive samples).

Surface	Our		ESDN		Racki et al.		Kim et al.		Scholz et al. [35]
	TPR	TNR	TPR	TNR	TPR	TNR	TPR	TNR	TPR	TNR
1	100	100	100	100	100	98.8	99.8	100	99.7	99.4
2	100	100	100	100	100	99.8	100	100	80.0	94.3
3	100	100	100	100	100	96.3	100	100	100	99.5
4	100	100	100	100	98.5	99.8	99.9	100	96.1	92.5
5	100	100	100	100	100	100	100	100	96.1	96.9
6	100	100	100	100	100	100	100	100	96.1	100
7	100	100	100	100	100	100	-	-	-	-
8	100	100	100	100	100	100	-	-	-	-
9	100	100	100	100	100	99.9	-	-	-	-
10	100	100	100	100	100	100	-	-	-	-

. Our method achieved 100% TPR and TNR on all folds, which means its completely solved this dataset. Practically, the ESDN had achieved this goal before that. Some other explorations also achieved high scores, such as Racki et al. [33], who obtained nine 100% outcomes and a 98.5% in a ten fold, and Kim et al. [34], who obtained 100%, except fold 1 and fold 4 Dagm, as a classic dataset of material texture, whih has sufficient data samples. We tested it above to prove that the KRN guarantees a high score on this complete dataset. We visualized some results in Figure 4.

4.3.2. KolektorSDD

The proposed KRN is compared with the ESDN, SDN, and EfficientNet in

Table 2. mAP on four methods (KSDD, 33 positive samples).

Method	AP/%	FP + FN
EfficientNet	-	-
SDN	99.00	1 + 0
ESDN	99.49	1 + 2
Ours	100.00	0 + 0

. The KSDD is a totally industrial few-shot dataset with only 53 positive samples. Its author defines it as a classification problem and especially adds small-scale segmentation annotation as an auxiliary. Assisted by segmentation, the SDN obtained a higher score of 99.00%. After a series of optimizations such as dynamic balance loss and gradient adjustment, the ESDN became an end-to-end defect detection model, and it achieved a 99.49% AP and 1 + 2 (FP + FN) in our experiments. According to Table 2, Our KRN achieved a further performance of 100.00% AP and realized 0 + 0 (FP + FN).

4.3.3. Severstal Steel

Table 3. mAP on four methods (Steel, 1000 positive samples).

Method	AP/%	FP + FN
EfficientNet	91.56	-
SDN	92.35	-
ESDN	98.45	68 + 85
Ours	99.73	40 + 32

compares our KRN with the ESDN, SDN, and EfficientNet on Severstal steel for 1000 positive samples. As shown in the table, in contrast, the KRN was the best among all methods. Its performance on the AP was 1.18%, 7.38%, and 8.17% higher than the ESDN, SDN, and EfficientNet, respectively.

4.4. Ablation Study

Below we analyze the effectiveness of each component of the proposed KRN approach. We designed six groups of experimental variants (except for the ablation experiment of each component), including variants used by all components and variants not used by all components. The following ablation studies were based on the KSDD dataset with the five positive samples setting. For some components, we further designed comparative experiments for in-depth analysis. For example, the Knowledge Embedding part also adopted add, concat, and mat operations.

Knowledge Modeling (KM): We conducted KM on flawless samples. In this part of the ablation experiment, we changed the KM to predict the conv feature of the image to simulate the effect of no KM. The experimental results show that KM has a great impact on the KRN (97.39% vs. 94.88%). For KSDD settings, it shows that without statistical modeling in

Table 4. mAP on four methods (ablation study on KSDD Dataset with 5 positive samples).

Fold 0	Fold 1	Fold 2	Mean	Knowledge Model	Knowledge Embedding	Gram Operation	Power Normalizing
94.90	90.01	98.47	94.46
93.18	94.70	96.75	94.88		√	√	√
99.43	91.58	8.47	96.49	√		√	√
94.28	95.00	96.08	95.12	√	√		√
96.84	91.21	96.31	94.79	√	√	√
99.38	97.98	94.80	97.39	√	√	√	√

, the AP scores of our KRN on fold 0 and fold 1 decreased by 6.2% and 3.28%, respectively. The analysis shows that KM brings additional knowledge and enhances the recognition performance of the model.

Knowledge Embedding (KE): In the ablation experiment of KE, our KRN performed the variant (97.39% vs. 96.49%), which levers the regular concat fusion in Table 4. concat had the best AP value on fold 0 and fold 2, but fold 1 had only a 91.58% AP. The KE module has the ability to capture knowledge features and predict the relationship between sample features in design, because the module has been significantly enhanced in the defect identification of fold 1 (97.39% vs. 91.58%).

We additionally analyzed the impact of concat, add, and mul operations on KE. The experimental results are shown in

Table 5. mAP on four variants of KE (KSDD, 5 positive samples).

Method	AP/%	FP + FN
Add	95.26	2 + 1
Concat	96.49	1 + 1
Mul	97.26	1 + 1
KE	97.39	0 + 1

. Among them, the add operation was the roughest for feature fusion through simple addition, with the AP score being the lowest, which was 95.26%. The concat operation retained the original features and knowledge features, which was 1.23% higher than add. Mul enlarged the local difference, and the multiplication between the conv feature and knowledge feature was conducive to mining the relationship between the two features; its ap score was close to KE. Our KE not only retains the original features, but also constructs the relationship between the quasi-sample and the predicted sample, deeply excavates the potential differences, and shows the best performance (97.39% vs. 95.26%, 96.49%, and 97.26%).

Gram Operation (GO): The GO is a supplement to KE, mining the relationship between features after KE. As shown in Table 4, the performance of the GO defect detection algorithm had a 2.27% drop (97.39% vs. 95.12%). GO mines the relationship between features of the final total features. Based on the experimental results of three folds (94.28%, 95.00%, and 96.08%), the GO was one of the main components to improve the performance of the model. The results in fold 0 show that its contribution to the KRN is second only to KM.

Power Normalizing (PN): We also analyzed the improvement brought by the PN, which considers a Burst suppressor on second-order statistics. It can be seen from the previous experiments that the KE and GO introduced negative effects. The analysis shows that it was due to the burst of high-order statistics. In

Table 6. mAP on Two PNs (KSDD, 5 positive samples).

Method	AP/%	FP + FN
None	94.79	2 + 2
AsinhE	96.89	0 + 2
SigmE	97.39	0 + 1

, it can be seen that after adding the PN operation, the negative effects brought by KE and GO were suppressed. The AP on fold 0 and fold 1 increased by 2.54% and 6.77%, respectively, and the average AP increased by 2.60%.

In particular, we explored two different PN strategies: Asinhe and SigmE. Without any power normalization, the AP score was only 94.79%, and FP + FN was 2 + 2. After adding Power Normalization, the performance was optimized, the false detection and missed detection were reduced from 4 to 2, and the AP increased to 96.89% and 97.39%, respectively. Our knowledge model relationship detector with sigma pooling is beneficial for small training sets defect detection.

In summary, KM, the GO, and PN have a significant impact on the three folds. Among them, KM and GO have a greater gain on fold 0, KE and PN have a greater impact on fold 1, and Ke seems to have a negative effect on fold 2. KM introduces external knowledge, KE embeds and fuses the knowledge, and GO and PN further promote the integration. The four modules cooperate with each other to form gain, mine potential features, and improve the performance of the small training sets defect detection model.

4.5. Small Training Sets

We paid special attention to the scene, where it is difficult to obtain defect samples in the industrial scene, that is, a small number of positive samples. We used positive samples of 5, 10, 15, 20, and 25 for each dataset. The results are shown in

Table 7. mAP on Three Methods (DAGM, KSDD, STEEL).

Model	Dataset	Num of Positive Samples
		5		10		15		20		25
		AP/%	FP + FN	AP/%	FP + FN	AP/%	FP + FN	AP/%	FP + FN	AP/%	FP + FN
baseline	DAGM	85.58	43 + 12	91.02	43 + 5	98.09	2 + 3	99.27	2 + 2	99.18	1 + 2
Our		90.12	23 + 16	96.23	3 + 7	98.04	2 + 3	99.65	1 + 1	99.11	1 + 1
baseline	KSDD	95.82	2 + 2	97.25	2 + 2	97.96	1 + 2	98.74	1 + 1	99.78	0 + 1
Our		97.35	1 + 2	97.84	1 + 2	99.15	1 + 1	99.29	0 + 1	100.00	0 + 0
baseline	STEEL	58.45	490 + 25	54.27	508 + 18	64.81	334 + 65	62.87	419 + 56	67.04	401 + 61
Our		63.28	467 + 31	60.13	438 + 53	65.57	346 + 70	68.13	349 + 48	75.46	315 + 83

.

For the DAGM2007 dataset, originally based on 150 positive samples, EfficientNet and ESDN could have good performance, but when the number of positive samples decreased, the detection performance of all algorithms showed different decline, as shown in the figure. The details of the decline can be seen in Table 7. When the number of positive samples decreased from 150 to 25 and then to 5, the AP value of the baseline was 100%–90.11%–82.39%; the AP value of the KRN 100%–99.11%–82.39%. When the training samples were sufficient, their AP scores were very high. When the number of positive samples decreased to 25, ESDN decreased by 3%, and KRN decreased by 1.47%; when the number of positive samples turned to five, the AP of the baseline decreased significantly (14%), and the KRN performed better, which only decreased by 9%.

For the KSDD dataset, all three methods achieved good scores. After analysis, we believe that this is because KSDD itself is a small dataset, and the test set is not complex, which makes it possible to obtain a good recognizer with only a small number of samples. However, five positive samples still brought differences to the performance of the various methods. According to Table 7, when the number of positive samples was greater than 20, the AP of each of the three methods was close to 100%. However, with the decrease in the number of positive samples to five, the baseline decreased by 4.18%, while the KRN was the least affected by the standard template KE and relationship mining module, which was only 2.65%.

Regarding the performance for the Steel dataset, like DAGM, all methods were greatly reduced: in the scenario of five positive samples, the AP of the baseline decreased to 58.45%, and there were false detection and missed detection values of 490 + 25; the AP performance of the KRN was the best, which was 63.28%, and the false detection and missing detection came out to only 467 + 34. Although these benefits are lower than the DAGM and KSDD, they are consistent and significant considering that the Steel dataset is more challenging in terms of complexity and dataset size.

We visualized the experimental results, as shown in Figure 5. The AP and FP + FN increased by varying degrees with the increase in positive samples. It reveals the generalization fragility of the baseline under the small training sets: without adequate training images, it detected poorly defected from input images. In contrast, for our KRN, the KE and GO demonstrated superior performance on small training sets defect detection.

4.6. Visualization of Detection Results

The following shows some test images on the DAGM, KSDD, Steel, and other datasets. We output the segmentation results, including the original image, ground truth, baseline, and KRN. Our model is more classification-based, and segmentation is the auxiliary part of conv feature extraction. Therefore, segmentation only needs shallow segmentation in the small and medium scale. Based on the visual results, the defect part in the segmented image will be larger than ground truth, which is normal. In addition, from Figure 5, in “the first sample of DAGM, the third sample of KSDD and the second sample of Steel”, it can also be seen that the difference was amplified by the KRN after introducing the second-order moment of covariance. But at the same time, the higher-order feature increased the burst (the noise in the upper right of the second sample of Steel).

5. Conclusions

In this paper, we proposed a Knowledge-Embedding Relation Network (KRN) for the small training sets Defect Detection to address few-shot defect detection. Our model extends the ESDN through embedding standard templates and second-order statistics into CNN features of segmentation excitation. The standard template provides external knowledge for defect samples, while KE and GO provide high-latitude potential relationship features. In order to demonstrate the effectiveness of the KRN, we have conducted extensive quantitative and qualitative experiments on several datasets. In particular, we simulated the industrial detection scene of small training sets and carried out relevant experiments.