Auto-Probabilistic Mining Method for Siamese Neural Network Training

Abstract

Training deep learning models for classification with limited data and computational resources remains a challenge when the number of classes is large. Metric learning offers an effective solution to this problem. However, it has its own shortcomings due to the known imperfections of widely used loss functions such as contrastive loss and triplet loss, as well as sample mining methods. This paper address these issues by proposing a novel mining method and metric loss function. Firstly, this paper presents an auto-probabilistic mining method designed to automatically select the most informative training samples for Siamese neural networks. Combined with a previously proposed auto-clustering technique, the method improves model training by optimizing the utilization of available data and reducing computational overhead. Secondly, this paper proposes the novel cluster-aware triplet-based metric loss function that addresses the limitations of contrastive and triplet loss, enhancing the overall training process. To evaluate the proposed methods, experiments were conducted with the optical character recognition task using the PHD08 and Omniglot datasets. The proposed loss function with the random-mining method achieved $82.6\%$ classification accuracy on the PHD08 dataset with full training on the Korean alphabet, surpassing the known baseline. The same experiment, using a reduced training alphabet, set a new baseline of $88.6\%$ on the PHD08 dataset. The application of the novel mining method further enhanced the accuracy to $90.6\%$ (+2.0%) and, combined with auto-clustering, achieved $92.3\%$ (+3.7%) compared with the new baseline. On the Omniglot dataset, the proposed mining method reached $92.32\%$, rising to $93.17\%$ with auto-clustering. These findings highlight the potential effectiveness of the developed loss function and mining method in addressing a wide range of pattern recognition challenges.

1. Introduction

Deep learning methods have been widely used in recent years, transforming industries such as natural language processing and computer vision. Convolutional neural networks (CNNs), capable of efficiently building hierarchical representations from raw inputs, have become a cornerstone of many applications. Optical character recognition (OCR) is one such application where CNNs have shown outstanding performance. OCR systems are essential for digitizing and extracting text from documents, images and other scanned materials, simplifying tasks like text-based analysis and automated data entry [1,2].

The ability of CNNs to automatically learn discriminative features straight from raw pixel data accounts for their effectiveness in OCR tasks [3]. CNNs are capable of capturing the details necessary for precise character identification by using convolutional layers to extract hierarchical structures and local patterns. For example, in OCR tasks, the number of neurons in the final layer of a neural network typically matches the number of distinct characters or symbols being recognized. But handling a huge alphabet (Figure 1) in OCR is a problem.

Under such circumstances, the number of neurons in the last layer increases dramatically, increasing the volume of data and requiring a large amount of processing power and training data. As a result, training these networks becomes ineffective.

Metric learning aids in avoiding this issue [4]. Metric learning operates by matching a point in the metric space to the input vector, ensuring that the point is positioned as close as possible to points of the same class and as far as possible from points of other classes [5].

There is a metric approach named Siamese neural networks (SNN). SNNs, initially introduced by Bromley et al. [6], have attracted considerable attention for their capability to learn robust representations in scenarios with limited labeled data. Unlike traditional CNN architectures that aim to directly predict class labels, SNNs acquire a similarity metric between inputs, allowing them to distinguish subtle differences and similarities even without sufficient labeled data.

An SNN consists of a few CNN branches with shared weights. The Euclidean norm is used to calculate the distances between the outputs of the branches, and the resulting vectors are then sent to the loss function.

The primary contributions of this paper are as follows:

• The development of a mining method for training SNNs that enhances data representation and reduces computational costs.
• The proposal of a novel metric loss function that combines the advantages of contrastive loss and triplet loss while introducing a new mechanism to ensure cluster compactness.
• An experimental evaluation of the proposed methods’ effectiveness in OCR tasks using the PHD08 and Omniglot datasets, demonstrating significant improvements in classification accuracy and computational efficiency.

The remainder of this paper is organized as follows: Section 2 provides an overview of existing mining methods and loss functions in metric learning tasks. Section 3 is dedicated to the considered datasets. Section 4 describes the proposed methods, including the mining approach and the novel metric loss function. Section 5 focuses on the metric network architecture, training and experimental evaluation. The results tables are presented in Section 6. Finally, Section 7 presents the conclusions and directions for future research.

2. Related Work

OCR involves classifying characters and associating them with their respective classes. While some languages, like English, have achieved high recognition rates [7], the task becomes increasingly challenging as the size of the alphabet exceeds 10,000, requiring novel training and recognition methodologies.

2.1. Contrastive Loss

The contrastive loss function [8,9] (Figure 2) that was initially proposed by Chopra et al. for face verification tasks has gained popularity in metric learning experiments, as shown by its use in studies like [10,11,12].

In summary, contrastive loss (Equation (1)) uses pairwise distance and pulls positive pairs as close to each other as possible and pushes negative pairs to a distance from each other of no less than the selected margin ($\alpha$).

$$L^{\alpha }(x^i,x^j;f)=y_{ij}{d}_{ij}^2+(1-y_{ij})·max{(0,\alpha -d_{ij})}^2,$$

(1)

where $d_{ij}={\parallel f\left(x_i\right)-f\left(x_j\right)\parallel }_2$ is the Euclidean distance between the pair, and $y_{ij}$ returns 1 if $y_i=y_j$ and 0 otherwise. $x^i,x^j$ respresents the inputs of the SNN f.

The absence of a natural stopping mechanism is one of the notable aspects of contrastive loss. This loss function encourages the minimization of distance between a positive pair to zero. In practice, the neural network may use computational resources to refine pairs that are already close to convergence, preventing it from exploring and optimizing other examples.

The recent work [13] mentions a different interpretation of a contrastive loss function that has similar meaning but is formulated differently. In our work, we refer to the classic version of contrastive loss, proposed in [8,9].

2.2. Triplet Loss

The triplet loss function (Equation (2)), as proposed and described in [14], provides an alternative approach to learning embeddings in metric learning tasks. Unlike contrastive loss, which deals with pairs of samples, triplet loss operates on triplets comprising an anchor, a positive sample from the same class as the anchor and a negative sample from a different class.

$$L^{\alpha }(x_a,x_p,x_n,f)=max(0,{∥f\left(x_a\right)-f\left(x_p\right)∥}_2-{∥f\left(x_a\right)-f\left(x_n\right)∥}_2+\alpha )$$

(2)

where $x_a,x_p,x_n$ represents the anchor and the positive and negative inputs of the SNN f.

The objective (Figure 3) of triplet loss is to train the network to reduce the distance between the anchor and positive samples while simultaneously increasing the distance between the anchor and negative samples by at least a specified margin ($\alpha$).

However, it is important to note that triplet loss introduces cubic complexity [15], meaning that there are significantly more possible triplets than pairs. Not all triplets are equally informative for training, and including all possible triplets can lead to slower convergence and computational inefficiency. Therefore, selecting “hard” triplets that are particularly challenging [16] and informative is crucial for effectively training the model and achieving faster convergence and improved performance.

The proposed mining method (Section 4) in this work introduces a strategy to solve the “hard” triplets problem.

Another challenge associated with triplet loss is related to the absence of a direct requirement for closeness between the anchor and positive instances. Rather, the emphasis lies on ensuring that the distance between the anchor and positive samples is smaller than the distance between the anchor and negative samples by a predefined margin.

The proposed metric loss (Section 4.4) offers an effective solution to this challenge by addressing key limitations of classic loss functions, such as contrastive and triplet loss, improving the model’s ability to learn discriminative features and achieve better performance in metric learning tasks.

2.3. Quadruplet Loss

There exists a quadruplet loss function [17], which extends traditional triplet loss by introducing an additional negative sample. This approach enforces a stricter constraint, ensuring that the anchor-positive pair is not only closer than the first negative but also closer than the second negative, thereby improving feature discrimination. While this method enhances generalization in metric learning, it also increases computational complexity and requires more sophisticated mining strategies for effective sample selection. Despite these challenges, quadruplet loss has demonstrated improved performance in representation learning tasks.

2.4. Character Decomposition

Moreover, certain languages permit character decomposition, enabling segmentation into individual components for recognition and subsequent composition of the final result [10,18]. This approach obviates the need for neural networks with tens and even hundreds of thousands of outputs. However, it suffers from drawbacks such as vulnerability to image distortions and heavy reliance on segmentation quality. Some researchers [19] choose deep neural networks with numerous trainable parameters, which offer exceptional quality but demand significant computational resources and are unsuitable for mobile applications and so on.

2.5. Mining Methods

Furthermore, training metric networks presents challenges in managing the sequence of pair/triplet mining. While random data selection is common [20], some works prioritize addressing this issue by focusing on similarity. For instance, in [21], an aggressive hard-mining strategy is employed, mining pairs with the highest error for backpropagation to train the network on challenging examples exclusively. The method’s ability to tolerate noise in data can make it difficult to reach the local minimum.

Another interesting point is the mathematical justification [22] for the effectiveness of hard-mining in metric learning, using the Isometric Approximation Theorem. The authors show that hard-mining is equivalent to minimizing the Hausdorff distance between the neural network and its ideal function, which explains the empirical success of this approach. In [23], the authors propose Easy Positive Triplet Mining (EPTM) to address the instability of traditional hard-mining methods. Hard Positive Mining often selects outliers or mislabeled samples, leading to noisy optimization and poor generalization. EPTM mitigates this issue by selecting easier yet informative positives, ensuring better clustering while maintaining stable training. Combined with Semi-Hard Negative Mining, which avoids extreme negatives, this approach leads to more robust embeddings and improved convergence.

Additionally, distance-based pairs mining methods, such as the approach proposed by [11], demonstrate effective results by generating samples based on creating a vector of distances for all possible impostor pairs. While promising in terms of training quality, this type of method is significantly time-consuming.

The auto-clustering [24] method partially solves the mining problem in SNNs. This method is based on creating clusters, i.e., groups consisting of classes similar from a network point of view. The usage of clusters allows the network to pay more attention to classes that are difficult to differentiate. According to the auto-clustering method the negative sample is selected from the same cluster as the positive sample. Let us consider how to effectively choose a positive sample.

In the process of training the neural network, we can figure out which classes are far from their cluster in the metric space and try to manually increase the probability of their choice when generating a positive class. However, automatic methods are recommended as the most impactful and convenient solutions for such a task.

3. Datasets

3.1. Korean Hangul Recognition

3.1.1. PHD08 Dataset

The Korean alphabet (Hangul) is known for its large set of characters and the high visual similarity between many of them, making it particularly challenging for OCR tasks. These characteristics align well with the goals of our research, as they test the robustness and discriminative power of the proposed methods. Thus, to evaluate the suggested methods, the printed Hangul character dataset called PHD08 [25] was chosen. Moreover the same dataset was used in the previous works [10,11,24].

The dataset contains 2350 classes, and each class has 2187 images of Korean characters. There are a total of 5,139,450 binary images that have different sizes, rotations and distortions. Examples of the dataset images are shown in Figure 4.

3.1.2. Synthetic Korean Alphabet Training Data

To assess of objectivity of the proposed approach, synthetic data were generated using the same approach as in [24] for training the network in the same way as in the works [10,11,24] using an average of eight fonts per each class. The total number of classes was 11,172 characters of the Korean language. Moreover, in our experiments, we also decided not to generate all possible classes and make the number of classes equal to the evaluation dataset size. This decision was made to more accurately evaluate the proposed auto-probabilistic mining method (Section 4.2), because the network could train on some classes, that do not exist in the evaluation dataset. And this fact, in combination with the considered approach, could significantly increase such character probabilities.

3.2. Omniglot Dataset

The Omniglot dataset was collected by Brenden Lake and his collaborators at MIT via Amazon’s Mechanical Turk to produce a standard benchmark for learning from few examples in the handwritten character recognition domain [26].

Omniglot contains 1623 characters from 50 different alphabets (Figure 5). It consists of not only international languages like Korean and Latin, but also lesser known local dialects and fictional character sets such as Futurama and Klingon. Each of these was hand drawn by 20 different people. The number of letters in each alphabet varies considerably from about 15 to upwards of 40 characters. All characters across these alphabets were produced a single time by each of 20 individuals. The dataset is composed of two subsets: images_background, containing 19,289 images, and images_evaluation, containing 13,180 images.

3.3. Augmentation

Data augmentation [24] was applied to the images during the training process, with a probability of $0.7$ for each sample and the following distortions:

• Projective transformation—Each point of the image is transformed, and the values of the shift on axes x and y are chosen randomly in the range $[0.0,1.0]$, where the minimum and maximum shares of offset are presented on the x-axis and y-axis from the width of the image.
• Rotation—Rotating an image by an angle in the range $[-5,5]$ in degrees.
• Scale—Scaling the image to a given size and then scaling the result to the original size with $min=0.7$ and $max=0.9$ scaling coefficients of the original image in width and height to make the image pixelated.

3.4. Experimental Setup

For all experiments in this paper, the images were resized to $37\times 37$ pixels and converted to grayscale to align with the preprocessing standards of previous studies for objective comparative analysis.

During each epoch, 50 iterations were performed with 10,240 elements generated per iteration. For contrastive loss, pairs were dynamically sampled (3072 genuine pairs and 7168 impostor pairs), while CATML utilized triplets. A total of 10,240 elements were generated for both the training and validation sets during every epoch. These settings were applied uniformly to all datasets used in the experiments, including PHD08 and Omniglot, to keep the training conditions consistent and allow for an unbiased evaluation of the proposed methods.

4. Suggested Method

4.1. Auto-Probabilistic Mining Method

Assigning specific probabilities to classes during neural network training can, in fact, improve the model’s capacity to identify certain classes. Prioritizing some classes with greater probability encourages the network to produce them more frequently, which enhances the network’s ability to recognize these symbols.

Manually assigning probability raises practical issues in a variety of situations. Firstly, the cognitive strain of maintaining comprehensive knowledge of several varied classes is beyond human capacity when their number is in the thousands. Furthermore, while automatic probability assignment is possible for some organized data, such as languages with symbol decompositions based on keys, many real-world settings lack such inherent structure, making hand assignment problematic. An automated technique can be applied to a wider range of items, regardless of their type or complexity.

As a result, the auto-probabilistic mining (APM) method provides a more flexible approach. The network gains the capacity to dynamically adapt and prioritize classes according to their significance for learning by automatically computing the probabilities for each class during training. This makes it possible for the network to determine which classes need more attention and to produce them more frequently, which improves recognition performance overall and optimizes the training process. Therefore, the APM method offers a promising way to enhance neural network training efficiency.

Using computed probabilities, the algorithm in this method determines whether each pair belongs to a positive or negative class. This approach differs from uniform distribution techniques and it looks for classes with the greatest average distances between each example of class and corresponding cluster.

4.2. Automatic Calculation of Class Appearance Probabilities

To enhance the training process using the APM method, we calculate a class probability vector $P^e$ at each epoch e. The method leverages the average distances between samples and their corresponding class centers to define class probabilities.

Let $f\left(x_{i,j}\right)$ be the feature representation of the input image $x_{i,j}$ (i-th class, j-th sample) obtained from the model f. Define d as the distance (L2 norm of difference) between $f\left(x_{i,j}\right)$ and the center $c_i$ of the i-th class, where $c_i$ represents the mean feature vector of samples from class i. The class probability vector $P^e$ is calculated as

$$P_i^e={\displaystyle \frac{{\left(\frac{1}{M_i}{\sum }_{j=1}^{M_i}d(f\left(x_{i,j}\right),c_i)\right)}^{\gamma }}{{\sum }_{k=1}^N{\left(\frac{1}{M_k}{\sum }_{j=1}^{M_k}d(f\left(x_{k,j}\right),c_k)\right)}^{\gamma }}},i=1,\dots ,N$$

where

• $M_i$ is the number of samples of i-th class;
• $x_{i,j}$ is j-th sample of the i class;
• N is the total number of classes;
• e is the number of epochs;
• $\gamma$ is the amplification factor controlling the impact of distances to emphasize dominant classes.

To adapt probabilities across epochs, the class probability vector $P^e$ on epoch e is updated recursively as

$$P^e=(1-w)·P^e+w·P^{e-1},$$

(3)

where

• $P^{e-1}$ is the class probability vector from the previous epoch;
• w ($0\le w\le 1$) is the weight factor controlling the contribution of the last epoch probabilities;
• $\gamma$ amplifies the updated probabilities to emphasize dominant classes.

Finally, $P_e$ is normalized to form a discrete probability distribution, which is utilized to guide the mining of positive pairs during training. Specifically, this distribution ensures that positive pairs are sampled more effectively by prioritizing classes with larger distances from the current samples.

4.3. Auto-Clustering Method Improvements

The base auto-clustering method contains the following steps: calculate all norms between ideal vectors of every class, sort them and, since there can be many of them, select some of them for analysis.

In the previous version of the auto-clustering method, there were lots of confusing hyperparameters required for network training. And such an approach is not convenient for experiments. In this stage, it was decided to simplify but retain the essence and leave just two parameters:

• The probability of selecting a class from a cluster for a negative pair/triplet ($\theta$);
• The number of the lowest norms considered for cluster generation ($\eta$).

This method is used for the mining of negative samples in pairs/triplets during training.

4.4. Cluster-Aware Triplet-Based Metric Loss

We propose the novel cluster-aware triplet-based metric loss (CATML) function (Equation (4)) combining the contrastive and triplet loss functions’ advantages.

$$CATML=\rho \ast g_1+\tau \ast g_2+\xi \ast g_3$$

(4)

where

• $\rho \ast g_1$ is the contrastive loss contribution, where $\rho$ is responsible for reducing the distance between the anchor and positive;
• $\tau \ast g_2$ is the triplet loss contribution, where $\tau$ regulates the principle that the distance between the anchor and positive should be less than the distance between the anchor and negative by a selected value equal to or greater than $\alpha$;
• $\xi \ast g_3$ is the cluster contribution, where $\xi$ is responsible for stability so that the clusters created during training do not fall apart and receive a gradient to stay in the same place.

In this work, we let $\rho =0.1$, $\tau =1.0$, $\xi =1.0$ and $\alpha =10$.

The components $g_1$, $g_2$ and $g_3$ are defined as follows:

• $g_1=d_{AP}={∥s\left(f\left(x_{Anc}\right)\right)-s\left(f\left(x_{Pos}\right)\right)∥}_2$;
• $g_2=s(d_{AP}-d_{AN}+\alpha )=s({∥s\left(f\left(x_{Anc}\right)\right)-s\left(f\left(x_{Pos}\right)\right)∥}_2-{∥s\left(f\left(x_{Anc}\right)\right)-s\left(f\left(x_{Neg}\right)\right)∥}_2)$, where
  $\alpha$ is the margin value;
• $g_3={\displaystyle \frac{1}{M_i}}{\sum }_{j=1}^{M_i}d(f\left(x_{i,j}\right),c_i),i=1,\dots ,N$, where $x_{i,j}\in (x_{Anc},x_{Pos},x_{Neg})$, $c_i$ is an ideal vector corresponding to a certain class (the average vector of the samples of the given class).

Here, f denotes the deep learning model used for feature extraction, and $s\left(x\right)=softrelu\left(x\right)$ is an activation function.

5. Experiments

5.1. Model

We chose the architecture (

Table 1. A list of the layers for the metric network.

Layers
N	Layer Type	Parameters	Output Size	Activation Function
1	conv	16 filters $3\times 3$, stride $1\times 1$, no padding	$35\times 35\times 16$	softsign
2	conv	16 filters $5\times 5$, stride $2\times 2$, padding $2\times 2$	$18\times 18\times 16$	softsign
3	conv	16 filters $3\times 3$, stride $1\times 1$, padding $1\times 1$	$18\times 18\times 16$	softsign
4	conv	24 filters $5\times 5$, stride $2\times 2$, padding $2\times 2$	$9\times 9\times 24$	softsign
5	conv	24 filters $3\times 3$, stride $1\times 1$, padding $1\times 1$	$9\times 9\times 24$	softsign
6	conv	24 filters $3\times 3$, stride $1\times 1$, padding $1\times 1$	$9\times 9\times 24$	softsign
7	fully connected	25 outputs	$1\times 1\times 25$	-

) from the works [10,11,24] for consistency in our comparison. A more illustrative scheme is represented in Figure 6.

The activation function softsign (Equation (5)) is described in detail in the works [28,29] and is recommended as the most suitable for this kind of problems. Softsign is a bounded activation function that promotes stable convergence and eliminates any numerical inconsistencies. This function was also used in the works [10,11,24].

$$\mathit{softsign}\left(x\right)={\displaystyle \frac{x}{1+\left|x\right|}}$$

(5)

5.2. Training

5.2.1. Synthetic Korean Characters

We performed experiments where the network was trained on synthetic images of Korean characters with the new CATML function. We considered two sizes of training alphabet: full (11,172 classes) and reduced (2350 classes). In the evaluated PHD08 dataset, there are only 2350 classes, so the reduced approach enables the APM method to have a more objective impact and avoids focusing on unnecessary classes.

Firstly, we trained CATML on the full alphabet with the random-mining method. Secondly, it was trained on the reduced alphabet with random-mining, auto-clustering and auto-probabilistic mining. Moreover, the specificity of two last methods allows us to use them both, where the APM method is used for positive sample mining and auto-clustering is responsible for negative samples.

In our experiments, the APM method only showed the best accuracy with $\gamma =1$, but for the APM + auto-clustering method, $\gamma =2$ showed the best result. The auto-clustering method demonstrated high accuracy, with $\theta =0.5$ and $\eta =1000$, and the APM method achieved the best result, with w equal to 0, which means that there was no dependency on the previous probability distribution, but this weight factor can also be considered for future experiments.

5.2.2. Omniglot Dataset

For the Omniglot dataset, we trained our network and set aside $60\%$ of the total data for training. The test and validation parts were set the same as in [3], both equal to $20\%$. We fixed a uniform number of training examples per alphabet so that each alphabet receives equal representation during optimization, although this is not guaranteed for the individual character classes within each alphabet.

All parameters used for the suggested methods were the same as for the experiments on the Korean alphabet.

5.3. Evaluation

The accuracy ($Acc$) for classification was defined as

$$Acc={\displaystyle \frac{N_{correct}}{N_{total}}}·100\%,$$

(6)

where $N_{total}$ is the size of the dataset and $N_{correct}$ is the number of images correctly classified by the network.

6. Results

All experiments followed a common evaluation protocol with data partitioned into three distinct sets: training, validation and test sets. The validation set was exclusively used for hyperparameter tuning and model selection through epoch-wise performance monitoring, while the test set remained strictly isolated for final evaluation. The final metrics were computed through a single evaluation on the test set using the optimal model checkpoint, selected based on the validation performance. To ensure reproducibility and prevent data leakage, no parameter updates or architecture decisions were influenced by test set observations during training.

For each experimental configuration, we provide the following:

• Epoch-wise validation curves demonstrating convergence patterns and training dynamics.
• A results table reporting both the training and test accuracies for the selected model. The training set metrics are included specifically to verify model behavior; the close alignment between training and test performance provides empirical evidence against overfitting.

6.1. PHD08

The results of the experiments with a large number of classes are presented in

Table 2. Classification accuracy for the full alphabet with 11,172 classes.

Loss	Mining Method	Train $\mathit{Acc}$	Test $\mathit{Acc}$
Contrastive loss	Random-mining [20]		$63.7\%$
	Hard-mining [21]		$64.5\%$
	Distance-based mining [11]		$69.7\%$
	Auto-clustering [24]	$79.3\%$	$76.1\%$
CATML	Random-mining	$86.2\%$	82.6%

. The results of random-mining, hard-mining and distance-based mining were taken from the work [11]. The CATML function showed notable accuracy (Figure 7) compared to all previous results.

We can also see in

Table 3. Classification accuracy for the reduced alphabet with 2350 classes.

Loss	Mining Method	Train $\mathit{Acc}$	Test $\mathit{Acc}$
CATML	Random-mining	$89.1\%$	$88.6\%$
	Auto-probabilistic	$94.8\%$	$90.6\%$
	Auto-clustering	$93.7\%$	$91.1\%$
	Auto-probabilistic + Auto-clustering	$95.1\%$	92.3%

that the proposed APM method improves classification accuracy. Moreover, this method is supposed to be an efficient and convenient solution regardless of the type of object being recognized. Additionally, it can be successfully applied both with and without the previously suggested auto-clustering method (Figure 8).

6.2. Omniglot

As we can see in the

Table 4. Classification accuracy for the Omniglot dataset.

Loss	Mining Method	Train $\mathit{Acc}$	Test $\mathit{Acc}$
CATML	Random-mining	$93.33$%	$91.25$%
	Auto-clustering	$95.21$%	$91.60$%
	Auto-probabilistic	$94.78$%	$92.32$%
	Auto-probabilistic + Auto-clustering	$95.89\%$	93.17%

, using the proposed methods improves the classification accuracy. Also, it is worth noting that the auto-clustering method does not improve accuracy significantly (Figure 9). This could be attributed to the dataset’s diversity, as it contains multiple alphabets, including Korean.

7. Conclusions

We have proposed the auto-probabilistic mining method, which enhances the accuracy of Siamese neural networks, making them more effective for classification tasks like OCR. This method can be successfully used both in combination with and without the auto-clustering method. It enhances network training, achieving high accuracy, as demonstrated on the PHD08 dataset, surpassing the previous baseline, and on the Omniglot verification task. The developed method does not require any knowledge of the object’s nature and can be used to train networks for any type of object, such as characters, feature point descriptors, faces, and so on. In addition to this, the proposed cluster-aware triplet-based metric loss function combines the benefits of contrastive and triplet loss functions and improves the accuracy of training.

In future research, the proposed methods, including the novel loss function and the auto-probabilistic mining strategy, could be explored in the context of one-shot learning and re-identification tasks. These problems present unique challenges, such as limited data and the necessity for robust discrimination between highly similar instances, which align well with the strengths of metric learning. By applying the proposed techniques to these tasks, we aim to investigate their potential to enhance generalization, reduce dependency on large-scale datasets and achieve competitive performance in scenarios requiring fine-grained feature representations and adaptability.