Leveraging Deep Features Enhance and Semantic-Preserving Hashing for Image Retrieval

Abstract

The hash method can convert high-dimensional data into simple binary code, which has the advantages of fast speed and small storage capacity in large-scale image retrieval and is gradually being favored by an increasing number of people. However, the traditional hash method has two common shortcomings, which affect the accuracy of image retrieval. First, most of the traditional hash methods extract many irrelevant image features, resulting in partial information bias in the binary code produced by the hash method. Furthermore, the binary code made by the traditional hash method cannot maintain the semantic similarity of the image. To find solutions to these two problems, we try a new network architecture that adds a feature enhancement layer to better extract image features, remove redundant features, and express the similarity between images through contrastive loss, thereby constructing compact exact binary code. In summary, we use the relationship between labels and image features to model them, better preserve the semantic relationship and reduce redundant features, and use a contrastive loss to compare the similarity between images, using a balance loss to produce the resulting binary code. The numbers of 0s and 1s are balanced, resulting in a more compact binary code. Extensive experiments on three commonly used datasets—CIFAR-10, NUS-WIDE, and SVHN—display that our approach (DFEH) can express good performance compared with the other most advanced approaches.

1. Introduction

In recent years, with the huge increase in the amount of network data, tens of thousands of pictures have been submitted online every minute. It is very difficult for users to look for the pictures they need according to their various requirements. It is difficult to describe similar images by language and find them—perhaps in an attempt to find pictures that are similar to the content of the pictures, pictures that are semantically similar, or images that are similar in terms of their background. Similarity is chiefly divided into two types: one is visual similarity, and the other is semantic similarity. How to quickly process so much data is very important. Among the methods used, the approximate nearest neighbor search [1,2,3]—that is, searching for the closest and most similar data to the query data—has become a very important research topic. Traditional image retrieval methods manually extract the features of the picture and convert them into real values and accomplish an approximate nearest neighbor search by computing a certain distance between two image representations to obtain the most similar images. At present, the database contains a large amount of data; if the traditional image retrieval method is used, it will consume considerable energy and storage space. To resolve the difficulties of the low efficiency of traditional retrieval means, a hash method [4] is proposed. Hash retrieval technology generates a hash function through calculation, which maps the high-dimensional image data matrix after data processing into binary codes of a certain length without changing their original structure [5,6,7]. It maps the high-dimensional characteristic representation of the graphics to a compact two-dimensional code, realizes data dimensionality reduction, and can carry out measurement operations in low-dimensional space [8]. In this way, the retrieval results are obtained by approximate matching by calculating the difference [9] between the binary codes of the pictures and using this difference as a measure of the similarity of the images. Because binary representation is used instead of real value representation, the storage stress is greatly reduced, and binary representation is easier to query than real value representation, thus improving the efficiency of the search [10]. However, the retrieval efficiency of most hashing approaches today depends heavily on the characteristics they use, which are largely unsupervised and thus more suitable for processing semantically similar images than visually similar images.

In today’s era of data explosion, the traditional method of extracting image features by hand is no longer applicable. In addition, the manually extracted image features are only suitable for limited parameters [11,12], and the depicted image features are relatively rough, which may lead to poor experimental results. New approaches based on deep learning and convolutional neural networks can automatically perform feature extraction and self-learning [13], effectively reducing the task of developing and optimizing new feature extractors. In various projects, the convolutional neural network can be seen as a characteristic collector [14], led by an overall function specifically devised for a single task. The convolutional neural network is a deep neural network model, and its convolutional layer uses several groups of convolution kernels to map the image to realize the dimensionality reduction operation of the image and extract the image features more accurately. During the model training process, the CNN learns through the backpropagation algorithm [15,16]. This backpropagation algorithm was first accomplished based on the learning mechanism of the human brain. By continuously optimizing the relevant parameters through the back-propagation algorithm, the data model can achieve a better fitting state. The birth of more optimization algorithms and the creation of various neural network architectures have brought artificial intelligence to a new frontier.

Most of the current hashing methods are combined with the deep neural network to analyze the semantic structure of the image from various angles, considering how to maintain the semantic structure and better combine the neural network with the extracted image features to capture various aspects [17]. In recent years, the methods of image enhancement [18] and image feature space optimization [19] have also achieved success. Inspired by these methods, we consider whether we can optimize the image features from the perspective of features to remove feature redundancy and achieve deep hashing. In addition, thanks to the rise of contrastive learning [20], we are inspired to obtain similarities between images by contrast. Therefore, we propose a deep network architecture with feature enhancement layer as shown in Figure 1 to learn hash codes better. First, the feature enhancement layer constructs the mapping from labels to image features by establishing the relationship between labels and image features and optimizes the features. Then, the image features are further extracted and fused through the feature extraction layer to obtain the most important image features. The network output is converted to a binary code through a hash layer by applying regularization [21] to better approximate the expected discrete values. The loss function makes similar images attract each other, shortens the distance between them, and makes dissimilar images repel each other, increasing the distance between them. This can make the learned Hamming structure more closely approximate the semantic structure of the images.

Other parts of the paper are arranged as follows. In Section 2, we introduce the related work and briefly illustrate the implementation principles of some hashing methods. In Section 3, we detail the network structure of our approach and introduce our loss function. In Section 4, we describe the relevant environment of our machine and its configuration, detailing how our method compares with other methods on three datasets. Finally, Section 5 summarizes our approach, summarizing our conclusions from our experiments.

2. Related Work

Early image retrieval was based on retrieving the surface visual characteristics of pictures, and its characteristics were the basic rough features of pictures extracted by hand [3]. In image retrieval, the approximate nearest neighbor search method, as a more practical method, has attracted increasing attention due to its high efficiency [22]. The purpose of approximate neighbor search technology is to search a piece of data in the database—the data point that is closest to the querier on a certain index. In the case of large databases or high computational cost of ranging, the computational cost of an accurate nearest neighbor search can be very high. As a representative of the retrieval algorithm, the hash method requires only a small amount of space and a low time complexity, which can greatly improve the efficiency of the approximate neighbor search. Earlier hashing algorithms, such as LSH [23], were based on the principle that two adjacent points in the initial space would still be adjacent in the new data space after being mapped the same way, while neighboring points would not always be adjacent after being mapped. Since increasing the bits of the hash code can help the Hamming distance to be closer to the feature space, the accuracy of LSH depends more on the length of the hash code and requires more memory. MLH [24] learns the hash function by optimizing the hinge loss. KSH [25] studies hash functions by minimizing or maximizing the distance with a kernel function. FashH [26] trains to enhance the decision tree to learn the hash function. SDH [27] optimizes hash functions by classification combined with the learning of binary codes. COSDISH [28] learns hash codes through an iterative algorithm based on column sampling. SPLH [29] learns hash functions sequentially according to the number of bits in the hash code, and each function is used to optimize the error generated by the previous function.

Because a deep convolutional neural network has a great ability to conduct self-learning and characteristic expression, it can be used to describe image content more accurately and precisely. By using DCNN’s powerful learning ability and nonlinear mapping ability, combined with hashing algorithms, researchers have presented many deep hashing approaches, which have obvious advantages compared to traditional hashing methods. Among the classical deep hashing techniques, CNNH [30] uses the similarity matrix factorization technique to learn the hash function, constructs the hash value of the training sample, and then uses the neural network to reversely study the picture characteristics and hash function corresponding to the hash value. However, since it is a two-part frame structure, the learned characteristics cannot be fed back to the former to optimize the resulting binary code. To solve the defect of CNNH, NINH [31] is proposed, which changes the network structure of the two parts so that characteristic learning and hash function learning can promote each other. DSRH [32] learns hash functions by extracting semantic similarity information hidden by multiple class labels. DQN [33] learns the representation of image features and hash codes on the basis of controlling quantization loss. DPSH [34] uses the corresponding information of images and paired tags to study characteristics and hash functions, respectively. DSH [35] draws on the Siamese network architecture and constructs a loss function to learn the hash function of similarity, which improves the discrimination ability of the network. DTSH [36] extends labels on the basis of DPSH and uses triple tag-related information to study the objective function. DHN [37] studies the hash function by optimizing the quantization loss function of image pairs and the cross entropy loss function of image pairs. HashNet [38] presents a new deep learning architecture to learn hash codes from imbalanced similarity-like data. SSDH [39] assumes that the semantic label has hidden attribute control and learns the hash function by optimizing the classification loss. These deep learning-based methods can be classified into unsupervised approaches [7,40], supervised approaches [24,41] and semisupervised approaches [24,42]. Unsupervised approaches use unlabeled datasets as training samples to study hash codes, such as LSH. For the first time, LSH associates an image with a hash and randomly projects the data points in its feature space to produce a binary code. The disadvantage is that only a longer binary code can be generated to improve the retrieval performance. The semisupervised method uses some data with label marks and some data without label marks to study the hash function, such as SSH [43]. SSH uses the supervision information to simulate the hash function to improve the precision of the binary code. By reducing the variance of the tag data point pair, the hash code is as balanced and independent as possible, thereby avoiding overfitting. The supervised hash method uses labeled data to learn the hash function, such as KSH [25], and trains the hash function bit by bit using the equivalent characteristics of the Hamming distance and the scalar product of the code, thereby generating a more effective binary code. Although most hashing methods first relax the code in the process of optimizing the objective function and then quantize the result to produce binary codes, it is not certain that the code obtained after quantization is optimal. To solve this difficulty, novel approaches such as DGH [44] and SDH [27] are proposed, which introduce auxiliary variables to optimize the regularization problem, remove the defects of relaxation, and improve the accuracy.

Algorithms of traditional hashing methods result in an insufficient description of low-level image features and find it difficult to express rich semantic information of images [45]. Most of the hashing methods proposed in recent years are from the perspective of keeping the global semantics of the image unchanged. By using the cross-entropy loss and the quantization loss [46] to optimize the binary encoding, the similarity between the original space and the binary space can be better maintained. Our work differs from other methods in some ways. (1) We consider starting from the local feature direction of the images, hoping to obtain more essential features through the relationship between labels and features, thereby reducing the deviation caused by irrelevant features. (2) By using the contrast loss to measure the similarity between images, similar images are close to each other, and dissimilar images are far away from each other. (3) We obtain a more compact binary code by adding regularization to the network output and adding balance constraints.

3. Proposed Method

We try to produce a tighter binary code by adding network structure and using three kinds of constraints. (a) The distance between binary codes produced by similar pictures is much smaller than the distance between binary codes of different pictures. Similar images attract each other and dissimilar images repel each other, as shown in Figure 2. (b) Images with the same tag attribute should be mapped to similar hash values, and images with no shared tag attributes should be mapped to different hash values. (c) Zeroes and ones are distributed in the binary code as evenly as possible.

3.1. Deep Network Framework

This consists of a feature enhancement layer, three convolutional pooling layers, two fully connected layers, a hash layer, and a loss layer. The feature enhancement layer is responsible for processing image features and marking image features. The convolution pooling layer transfers the extracted features to the fully connected layer by reducing the dimensionality of the picture characteristics. In addition, the fully connected layer is responsible for integrating image features and converting the image into binary code through the hash layer. During training, the loss function in the loss layer continuously optimizes the parameters.

3.1.1. Feature Enhancement Layer

Picture tags are not only convenient for image sorting but also important information for learning hash functions. We consider that since image labels can represent the classification of images and classification is associated with image features, we assume that image labels hide some features of images that can distinguish their categories. Based on this assumption, we assume that the matrix $W^F\in {\mathbb{R}}^{K\times C}$ is a mapping from labels to image features, where K represents the dimension of the hash code and C represents the number of categorical labels.

3.1.2. Feature Extractor

Most image retrieval and image classification methods work in a similar way, and they both belong to a way of processing images. The extraction of image features by the two is mostly consistent, and both can determine the classification of the image and determine the characteristics of the image from the given label. Inspired by the Siamese network, we learned to study the hash function by learning the similarity between the two images, not from the direction of image classification, to quantify the output to obtain a good binary code. Because the hash code is binary, 1 in the binary code indicates that the feature exists, and 0 indicates that the feature does not exist.

3.1.3. Hash Layer

In traditional hashing methods, the main task of most hash layers is to encode features for output. Although our hash layer is similar to a traditional hash layer, our hash layer is mainly used for feature mapping and generating more compact binary codes. Traditional hashing methods may not care about the semantic information of image features, but we found that different neurons in the hash layer control whether different features exist, and using these findings, we can better maintain semantic similarity. We set the matrix $W^H\in {\mathbb{R}}^{P\ast K}$ as the weights of the hash layer. P is the number of elements in the output $z_i^f$ of the previous layer, and k is the number of binary code bits. $z_i^h\in {\mathbb{R}}^K$ represents the output of the input data $x_i$ through the hash layer. Then,

$$\begin{array}{c}\hfill z_i^h=\mathrm{relu}({\left(W^H\right)}^{\top }\ast z_i^f+b^H)\end{array}$$

(1)

where $b^H$ is the bias of the hash layer, and relu$\left(\right)$ is represented as

$$\begin{array}{c}\hfill \mathrm{relu}\left(x\right)=\left\{\begin{array}{cc}x\hfill & x>0\hfill \\ 0\hfill & x\le 0\hfill \end{array}\right.\end{array}$$

(2)

Therefore, the hash codes can be expressed as $b_i=\mathrm{sgn}\left(z_i^h\right)$; if $x\ge 0$, $\mathrm{sgn}\left(x\right)=1$; otherwise, $\mathrm{sgn}\left(x\right)=-1$.

3.1.4. Loss Layer

The loss layer optimizes parameters through a loss function. The similarity between pictures is described by the contrastive loss, the error of the quantization process is reduced by the quantization loss, and the numbers of 0s and 1s in the binary code are balanced by the balanced loss.

3.2. Objective Function

We divide the overall objective function into three parts: contrastive loss, quantization loss, and balanced loss. Let $\mathcal{X}=\left\{x_n\right\}$ be N pictures and $\mathcal{Y}=\{y_n\in {\left\{0,1\right\}}^C\}$ be the tag attribute marked with each image, where C is defined as the number of categories of label attributes. Our approach is to learn the mapping from image $\mathcal{X}$ to k-bit binary codes: $\mathcal{F}$:$\mathcal{X}$→${\{0,1\}}^k$ while maintaining the similarity from the picture structure to the Hamming structure.

3.2.1. Contrastive Loss

The purpose of the loss is to make the difference between the binary codes of similar pictures smaller and the difference between the hash codes of different pictures larger. As the so-called “things cluster together, people are divided into groups”, people with similar characters will attract each other, and different people will repel each other, as shown in Figure 3. Therefore, for a pair of images $x_1$,$x_2$∈${\left\{x_n\right\}}^N$ and the corresponding binary code $b_1$,$b_2$∈${\{+1,-1\}}^k$, if the two pictures are similar, then $Y=1$; otherwise, $Y=0$. The loss function of both is defined as

$$\begin{array}{cc}\hfill L_c(W,(x_1,x_2,Y))& =\frac{1}{2}(1-Y)\max {(M-D(x_1,x_2),0)}^2\hfill \\ & +\frac{1}{2}Y{D}^2(x_1,x_2)\hfill \end{array}$$

(3)

where W represents the weight of the network structure, and $x_1$ and $x_2$ represent two random images in the dataset. where $D(·,·)$ indicates the Euclidean distance between the two pictures and M is the boundary value of contrast loss. The first term of Formula (3) maps binary codes of different images. When the distance between them is greater than the m value, the loss between them will be ignored. The latter term is a computation that describes similar images.

We choose to randomly select image pairs from the training images ${\left\{x_n\right\}}^N$. Assuming a total of $N_p$ image pairs, the overall contrastive loss of the images is as follows:

$$\begin{array}{cc}\hfill {\mathcal{L}}_c=& \sum _{i=1}^{N_p}{L}_c(W,(x_{i,1},x_{i,2},Y_i))\hfill \\ \hfill & s.t.x_{i,j}\in {\left\{x_n\right\}}^N,j\in \{1,2\},\forall i\hfill \end{array}$$

(4)

3.2.2. Quantization Loss

On the one hand, to improve the quality of binary codes and produce more accurate binary codes, we consider adding constraints to the hash layer. On the other hand, to reduce the quantization error caused by the mapping of feature representations to binary hash codes, we consider increasing the quantization loss in an effort to keep the binary-like values closer to 1 or −1. The quantization loss is described as

$$\begin{array}{c}\hfill {\mathcal{L}}_q=\sum _{i=1}^N\parallel |z_i^h{|-0.5e\parallel }_t^t\end{array}$$

(5)

where e represents a vector with all elements equal to 1 and $t\in \{1,2\}$.

3.2.3. Balanced Loss

On the other hand, considering the problem of binary code balance, we decide to add a constraint to keep the numbers of 0s and 1s in the binary code balanced, each accounting for 50%. For the input image $x_i$, the corresponding $z_i^h$ forms a discrete distribution on $\{0,1\}$. Then, our balanced loss is defined as

$${\mathcal{L}}_b=\sum _{i=1}^N{|\mathrm{mean}\left(z_i^h\right)-0.5|}^t$$

(6)

where $t\in \{1,2\}$. $\mathrm{mean}(·)$ is represented as follows:

$$\mathrm{mean}\left(z_i^h\right)=\frac{1}{K}\sum _{i=1}^k{z}_i^h$$

(7)

3.2.4. Total Loss

The total objective function we constructed is

$$\mathcal{L}={\mathcal{L}}_c+\theta {\mathcal{L}}_q+\eta {\mathcal{L}}_b$$

(8)

where $\theta$ and $\eta$ are the weight parameters for these terms.

3.3. Optimization

Due to the difference between the Hamming distance and Euclidean distance, ignoring the binary constraint results in inaccurate binary code. With the relaxation scheme, the threshold is approximated mainly by a saturated nonlinear function, but this will slow down the convergence of the network. Finally, instead of the binary constraint, we apply regularization to change Formula (3) to

$$\begin{array}{cc}\hfill L_c^o(W,(x_1,x_2,Y))& =\frac{1}{2}(1-Y)\max (M-\parallel x_1-x_2{{\parallel }_2^2,0)}^2\hfill \\ & +\frac{1}{2}Y\parallel x_1-x_2{\parallel }_2^2+\lambda {\parallel W\parallel }_2^2\hfill \end{array}$$

(9)

where ${\parallel ·\parallel }_2$ denotes the L2 norm, and $\lambda$ is the value parameter that controls the intensity of regularization. Here, the L2 norm is used to narrow the output range of the network and control the complexity of the model to reduce the risk of the structural method. Then, Formula (3) can be rewritten as

$$\begin{array}{cc}\hfill {\mathcal{L}}_c=& \sum _{i=1}^N\{\frac{1}{2}(1-Y_i)\max (M-\parallel x_{i,1}-x_{i,2}{{\parallel }_2^2,0)}^2\hfill \\ & +\frac{1}{2}{Y}_i\parallel x_{i,1}-x_{i,2}{\parallel }_2^2+{\lambda \parallel W\parallel }_2^2\}\hfill \end{array}$$

(10)

On the basis of this function, the backward propagation algorithm with gradient descent is used to train the network. When $Y=1$ and the function is $L_c=\frac{1}{2}{\sum }_{i=1}^{N}Y{D}^2$, calculate the gradient $\frac{\partial L_c}{\partial W}$—that is, calculate the partial derivative of $x_1$ and $x_2$, respectively:

$$\frac{\partial L_c}{\partial W}=\left\{\begin{array}{cc}\frac{\partial L_c}{\partial x_1}=D\frac{\partial D}{\partial x_1}=x_1-x_2,\hfill & D>M\hfill \\ \frac{\partial L_c}{\partial x_2}=D\frac{\partial D}{\partial x_2}=-(x_1-x_2),\hfill & D<M\hfill \end{array}\right.$$

(11)

When $Y=0$, $L_c=\frac{1}{2}{\sum }_{i=1}^N(1-Y)\max {(M-D,0)}^2$, the calculated gradient is

$$\frac{\partial L_c}{\partial W}=\left\{\begin{array}{cc}-(M-D)\frac{\partial D}{\partial W}\hfill & ,D<M\hfill \\ 0\hfill & ,D>M\hfill \end{array}\right.$$

(12)

$$\frac{\partial L_c}{\partial W}=\left\{\begin{array}{c}\frac{\partial L_c}{\partial x_1}=-(M-D)\frac{\partial D}{\partial x_1}=-(M-D)\frac{x_1-x_2}{D}\hfill \\ \frac{\partial L_c}{\partial x_2}=-(M-D)\frac{\partial D}{\partial x_2}=-(M-D)\frac{-(x_1-x_2)}{D}\hfill \end{array}\right.$$

(13)

3.4. Image Retrieval

Finally, the image to be retrieved is used as input, the corresponding binary code is compared with the binary code of other images, the distance is calculated, and the pictures in the database are sorted according to the size of the Hamming distance. An image with a smaller distance is more similar to the query image, as shown in Figure 4.

4. Experiments

We conducted experiments and evaluations on the model presented by our approach on three standard datasets and compared these with other scholars’ methods to demonstrate the feasibility of our approach. These experiments were implemented through Python’s deep learning framework TensorFlow. The operating system used by our computer was Windows 10, the memory was 16G, and it was equipped with an NVIDIA GeForce RTX 3060 laptop GPU.

In the training process, we used a small part of the gradient descent algorithm for optimization, set the batch size to 200, the momentum to 0.9, the weight decay to 0.001, the learning rate to 0.001, and used a total of 150,000 iterations.

4.1. Datasets Used in the Experiment

(1)

CIFAR-10 [47]. The dataset has 60,000 color images divided into 10 classes in total, each with 6000 pictures. There are 50,000 training batches and 10,000 pictures in each batch. The other 10,000 images are used for testing and form a separate batch.

(2)

NUS-WIDE [48]. The dataset is a multilabel dataset containing 269,648 pictures acquired from Flickr. Each picture receives one or more of the 81 category tags. Our experiments selected the most common 21 categories, including 195,834 images, with 5000 images in each category as the training set, and other pictures as the test dataset.

(3)

SVHN [49]. The SVHN dataset contains 73,257 32 × 32 digital images, each containing a set of “0–9” Arabic numerals. However, the number of samples per class is very different. To solve this problem, we used class-aware sampling during the training phase and randomly selected 20 images from each class in each iteration.

During our experiments, the similarity between images was determined by the image tags, and pictures with the same kind of labels were regarded as similar, and vice versa. In addition, if the images contained multiple kinds of labels, the intersection of their label information was considered to be similar if the intersection was not an empty set; otherwise, they were regarded as dissimilar.

4.2. Evaluation Index

We followed the practice of our predecessors [30,31] and adopted three evaluation metrics to assess the efficiency of our approach, including the mean average precision (mAP) value of the hash code under different bits, the precision-recall curve, and the precision within Hamming radius r. Here, we set $r=2$.

mAP [50] is a performance indicator that reflects the overall performance. Its calculation formula is described as

$$\begin{array}{c}\hfill mAP=\frac{1}{N}\sum _{i=1}^N\frac{1}{N^{*}}\sum _{j=1}^{N^{*}}Precision\end{array}$$

(14)

where N is the quantity of query sets, and $N^{*}$ denotes the number of cases related to positive examples in the retrieval sets. The precision [50] predicts how many positive samples are real, and the recall rate [50] accurately predicts how many positive samples are in the sample. The formulas for precision and recall are defined as

$$\begin{array}{c}\hfill Precision=\frac{TP}{TP+FP}\end{array}$$

(15)

$$\begin{array}{c}\hfill Recall=\frac{TP}{TP+FN}\end{array}$$

(16)

where $TP$, $FP$, and $FN$ in the formula represent the positive samples in the correct classification, the positive samples in the incorrect classification, and the negative samples in the incorrect classification, respectively.

The precision within the Hamming radius r represents the Hamming distance between all images and a given query image, given the proportion of the number of images with a Hamming distance less than r in all retrieval results.

4.3. Experimental Results

Our experiments compare the accuracy of our present approach and other hashing methods on three different large datasets. Our experimental results show that adding a neural network layer to the method can significantly improve the accuracy of the method. This shows that the features obtained by using the neural network can not only obtain low-level characteristics but also higher-level semantic characteristics, thereby improving the accuracy of the method.

Table 1. Time (seconds) to train our method with different datasets.

Method	CIFAR-10	NUS-WIDE	SVHN
DFEH	2389.211	9362.316	11,043.457

shows the training time of our method on different datasets, which is largely spent on the extraction of high-quality image features.

Table 2. mAP scores of various hashing approaches on CIFAR-10.

Method	12 bit	24 bit	36 bit	48 bit
LSH [7]	0.1319	0.1367	0.1407	0.1492
SH [7]	0.1319	0.1278	0.1364	0.1320
ITQ [51]	0.1080	0.1088	0.1117	0.1184
CCA-ITQ [51]	0.1653	0.1960	0.2085	0.2176
MLH [24]	0.1844	0.1994	0.2053	0.2094
BRE [41]	0.1576	0.1624	0.1684	0.1717
KSH [25]	0.2956	0.3732	0.4019	0.4167
CNNH [30]	0.5326	0.5613	0.5631	0.5563
DLBHC [52]	0.5504	0.5810	0.5769	0.5883
DNNH [31]	0.5711	0.5868	0.5892	0.5911
DPSH [34]	0.6534	0.6546	0.6610	0.6632
DSH [35]	0.6776	0.7213	0.7465	0.7568
DFEH	0.6753	0.7216	0.7641	0.7864

and

Table 3. mAP for different hashing techniques for different bits on NUS-WIDE.

Method	12 bit	24 bit	36 bit	48 bit
LSH [7]	0.3329	0.3392	0.3450	0.3474
SH [7]	0.3401	0.3374	0.3343	0.3332
ITQ [51]	0.3425	0.3464	0.3522	0.3576
CCA-ITQ [51]	0.3874	0.3977	0.4146	0.4188
MLH [24]	0.3829	0.3930	0.3959	0.3990
BRE [41]	0.3556	0.3581	0.3549	0.3592
KSH [25]	0.4331	0.4592	0.4695	0.4692
CNNH [30]	0.4315	0.4358	0.4451	0.4332
DLBHC [52]	0.4663	0.4728	0.4921	0.4916
DNNH [31]	0.5471	0.5367	0.5258	0.5248
DPSH [34]	0.5652	0.5743	0.5876	0.5881
DSH [35]	0.5601	0.5783	0.5814	0.5877
DFEH	0.5674	0.5788	0.5863	0.5921

respectively show the comparison of our method with traditional methods and recent methods in different hash code lengths. In Figure 5 and Figure 6, we can see the trend of the impact of different hash code lengths on mAP values on different datasets. From Figure 7, we can see the relationship between training loss, test loss and the number of iterations. The loss convergence is fast, because we can extract more useful information from better features. Compared with the traditional hashing method, it is obvious that our method has a small improvement. After all, the traditional hashing method usually uses the features extracted by hand, which has great redundancy, and it is not easy to obtain high-level image features with higher accuracy. Our method uses an optimized CNN network to improve the efficiency of feature extraction. In addition, in Figure 5, we can see that our method has a small improvement compared with some deep hashing approaches, and the accuracy is improved by approximately 1%, which reflects the feasibility of our presented approach.

4.4. Ablation Study

In this module, we study the effect of certain components or variables on the experimental results.

4.4.1. Ablation Study On Regularization

Because adjacent points may be mapped to different binary codes during quantization, we added regularization to reduce the variation from the original image space to Hamming space, thus improving retrieval precision. To prove the availability of regularization, we trained the model with 12 hash code bits and set $M=24$, training the models when $\lambda$ was 0, 0.1, 0.01, and 0.001, and compared their experimental results.

The experimental results obtained with different $\lambda$ values are shown in

Table 4. The effect of different $\lambda$ values on the mAP value.

$\mathit{\lambda }$	CIFAR-10	NUS-WIDE	SVHN
0	0.5436	0.5371	0.7966
0.1	0.2543	0.4376	0.7841
0.01	0.6874	0.5763	0.8624
0.001	0.6047	0.5576	0.8362

. Clearly, different regularization weight parameters can lead to different mAP values. Not using regularization and bad weight parameters can lead to bad experimental results. Experiments show that a value of k between 0.01 and 0.001 will produce better results. This part proves that adding regularization still helps to improve the accuracy of our method.

4.4.2. Ablation Study on Loss Function

To obtain a more compact binary code, we add two constraints: one is a quantization constraint to reduce the error of the quantization process, and the other is a balance constraint to ensure that the numbers of 0s and 1s in the binary code are balanced. Next, we analyze the effect of constraints through experiments and verify whether the accuracy of our method can be improved by adding two constraints.

Table 5. Different experimental results under different constraints.

Loss	CIFAR-10	NUS-WIDE	SVHN
${\mathcal{L}}_c$	0.6253	0.5174	0.7862
${\mathcal{L}}_c+{\mathcal{L}}_q$	0.6654	0.5534	0.8068
${\mathcal{L}}_c+{\mathcal{L}}_b$	0.6546	0.5364	0.7934
${\mathcal{L}}_c+{\mathcal{L}}_q+{\mathcal{L}}_b$	0.6774	0.5604	0.8242

shows the indicators under the influence of each part of the loss function. It can be seen from the table that using only one loss function produces the worst effect, and using three losses will achieve better results. It can be found from the table that the impact of loss measurement on our method is greater than balance loss. In this way, we can infer that the two constraints still have a great effect on improving the efficiency of our method.

4.4.3. Ablation Study on Deep Network

In contrast to the usual network architecture, we add feature enhancement layers, hash layers, and loss layers to better compute the performance of each layer. As shown in Figure 8, compared with the initial network framework, the performance of our network on each dataset has been slightly improved, which reflects the feasibility of our network architecture. We attribute the improvement to image feature augmentation, which emphasizes label information to connect with image features and is more able to preserve semantic information when extracting image characteristics.

5. Conclusions

This paper proposes a new network architecture for image retrieval that not only preserves the semantic structure of images but also fully captures high-quality features of images. Our framework structure consists of four parts: a feature enhancement layer, feature extraction layer, hash layer, and loss layer. The framework starts from the perspective of image label features, not from the overall image, and adds a feature enhancement layer that specializes in processing and optimizing the high-quality features of the images, reducing the acquisition of a large number of useless and repeated features and reducing the redundant information brought by feature information. A great deal of time is spent in exchange on high-quality image features, so that the resulting binary code carries more primary feature information but will lack some secondary feature information. By optimizing the contrastive loss function and imposing constraints on the hash code, a more concise and compact binary code can be obtained. The experimental results show that starting from the feature orientation of the image also achieves good performance. In future work, we will further explore the influence of primary and secondary features on image retrieval and explore methods to obtain the best features of images.