Approximate Nearest Neighbor Search Using Enhanced Accumulative Quantization

Abstract

Approximate nearest neighbor (ANN) search is fundamental for fast content-based image retrieval. While vector quantization is one key to performing an effective ANN search, in order to further improve ANN search accuracy, we propose an enhanced accumulative quantization (E-AQ). Based on our former work, we introduced the idea of the quarter point into accumulative quantization (AQ). Instead of finding the nearest centroid, the quarter vector was used to quantize the vector and was computed for each vector according to its nearest centroid and second nearest centroid. Then, the error produced through codebook training and vector quantization was reduced without increasing the number of centroids in each codebook. To evaluate the accuracy to which vectors were approximated by their quantization outputs, we realized an E-AQ-based exhaustive method for ANN search. Experimental results show that our approach gained up to 0.996 and 0.776 Recall@100 with eight size 256 codebooks on SIFT and GIST datasets, respectively, which is at least 1.6% and 4.9% higher than six other state-of-the-art methods. Moreover, based on the experimental results, E-AQ needs fewer codebooks while still providing the same ANN search accuracy.

1. Introduction

With the rapid growth of image data in daily life, large-scale image retrieval has attracted much attention in the research communities for image processing, computer vision and machine learning [1]. In particular, nearest neighbor search has been widely studied for fast image retrieval [2]. Compared with exact nearest neighbor (NN) search, approximate nearest neighbor (ANN) search tasks have lower complexity and higher feasibility [3]. The goal of ANN search is to find the sample in the coding space that is closest in probability to the query with some given metric methods [4]. Normally, to perform an efficient search, the vectors in a dataset need to be stored in memory, which is a key challenge for large-scale datasets [5]. Therefore, ANN-related methods usually apply compact codes to represent each vector in the dataset. According to the current research focus and achievements, there are mainly two categories for ANN search, which are vector quantization and binary hashing [6]. Vector quantization adopts the Euclidean distance to measure the similarity between vectors, while binary hashing usually uses the Hamming distance.

Binary hashing algorithms commonly learn hashing functions to map the image features from high-dimensional space to low-dimensional Hamming space [7,8,9,10,11,12,13,14], matching query and database vectors by their hashing codes [15,16]. However, the representational ability of binary hashing codes is limited by the number of binary codes. As a result, the accuracy of ANN hashing-related methods is generally inferior to that of vector quantization methods [17].

In recent years, many effective quantization methods have been proposed in some research. Among them, product quantization (PQ) [18] was first proposed and applied in ANN search. This method segments the vectors into sub-vectors in the primitive dimension space and quantizes them separately to reduce the complexity of the algorithm. Then it constitutes the approximate representation of feature vectors through the Cartesian product of sub-quantization codes. In this way, PQ can significantly control the quantized storage increment. Many variants based on PQ [5,19,20,21,22,23,24] further improved the performance of ANN search. In addition, PQ can be applied to construct an index for un-exhaustive ANN search [25], which can improve ANN search time efficiency.

Besides PQ and its evolutions, some excellent methods have also been proposed to figure out the optimal scheme of the feature vector compression problem. These different methods were also effectively applied in ANN search. Iterative quantization (ITQ) [26], which has higher precision compared with product quantization, uses the quantization rotation matrix to obtain an optimal result. In addition, some analogous works introduced tree dynamics [27], orthogonality constraints [28] and the additive quantization scheme [29], respectively, to optimize quantization performance. Residual vector quantization (RVQ) [30] has multiple different low-complexity quantizers to reduce the quantization error sequentially. Moreover, to optimize the effect of quantization, some significant methods [31,32,33] have been proposed to provide modified strategies. On the other hand, an analogous method called accumulative quantization (AQ) [34] also constructs approximations by addition.

Recently, some research [1,35,36,37,38,39,40,41] has proposed deep quantization by combining classical quantization methods with a deep neural network. Different from the above methods using artificial image features, deep quantization realizes the end-to-end learning process from image feature extraction to quantization encoding and also works effectively in some applications.

In addition, vector quantization is also used in image compression to reduce the memory requirements for storing an image. The representative works include [42,43], where the codebooks design was treated as an optimization task to minimize the error between an image and its reconstruction. Under bio-inspired metaheuristics, FA-LBG [44] was improved by allowing greater influence of the training set in the codebook design [42]. Additionally, an FSS-LBG [43] was presented, in which a new breeding operation based on fish school search [45] was further proposed to favor exploration ability.

In this paper, we focus on further improving the accuracy of ANN search. We propose an enhanced accumulative quantization that combines accumulative quantization with the idea of the quarter point. Based on our former work of accumulative quantization, after decomposing vectors into several partial vectors, the quarter vector was computed and used as the quantization output instead of the nearest centroid for both training codebooks and quantizing vectors. Due to the lower error produced by the quarter vector than the nearest centroid, the codebooks were used to approximate the vectors more precisely. In addition, we designed a corresponding method of efficiently computing the Euclidean distance between the query vector and a vector in the dataset.

2. Related Works

2.1. Quarter-Point Product Quantization

Like PQ, quarter-point product quantization (QPQ) [24] divides the vectors into M independent sub-vectors, whose dimension is d/M. Then, a dictionary $C$ consisting of M sub-codebooks $\left\{C_1,C_2,\cdots ,C_M\right\}$ is obtained by performing k-means on sub-vectors, where $C_i=\left\{c_{i,1},c_{i,2},\cdots ,c_{i,K}\right\}$, ($i=1,\dots M$ and $K$ are the number of centroids in each sub-codebook).

After dividing a vector y into M sub-vectors {$y(i), i=1,\dots ,M$}, each $y(i)$ is quantized by corresponding codebook $C_i$. Instead of using the nearest centroid as the quantization output, QPQ uses the quarter point $c_i^{\prime }(y(i))$, which can be generated by the nearest centroid $c_i{}^1(y(i))$ and the second nearest centroid $c_i{}^2(y(i))$. The $c_i^{\prime }(y(i))$ can be computed by the Formula (1):

$$c_i^{\prime }(y(i))=\frac{3}{4}{c}_i{}^1(y(i))+\frac{1}{4}{c}_i{}^2(y(i))$$

(1)

Then, the Euclidean distance d(,) between query vector q and vector y can be formalized with the Equation (2):

$$d\left(q,y\right)={\displaystyle \sum _{i=1}^{M}d\left(q(i),c_i^{\prime }\left(y(i)\right)\right)}$$

(2)

2.2. Accumulative Quantization

In our former work [34], after decomposing vectors into partial-vectors of the same dimension, each codebook was trained on partial-vectors. Then, we used accumulative quantization (AQ) to approximate a vector according to the sum of its partial vectors, which we quantized with the corresponding codebook.

Taking the partial vector of vector y as $y(i),(i=1,\dots ,M)$, the quantization output of $y(i)$ is denoted as $c_i(y(i))$. Thus, y can be approximated by the Equation (3):

$$y\approx {\displaystyle \sum _{i=1}^M{c}_i\left(y(i)\right)}$$

(3)

where $c_i(y(i))$ is the nearest centroid in codebook $C_i$ according to the Euclidean distance.

The asymmetric distance between query vector q and the vector y can be computed with the following Equation (4):

$$y \approx {\displaystyle \sum _{i=1}^M{c}_i\left(y(i)\right)}$$

(4)

where d(,) denotes the Euclidean distance.

The QPQ approach divides the vectors into several low dimensional sub-vectors. It is based on the assumption that the components in a vector are independent of each other, which is not applicable for real data. AQ is performed on the whole vector with several codebooks so that the dependence among the components in a vector is preserved. Although the accuracy of ANN search can be improved by increasing the number K of codebooks or the number M of centroids in each codebook, this brings more costs. Under this background, we tried to combine the AQ and the idea of the quarter point for the purpose of both preserving the dependence among the vector components and improving the accuracy of ANN search without increasing K or M.

3. Enhanced Accumulative Quantization

Generally, for accumulative quantization, the ANN search accuracy can be improved by increasing the number K of centroids when training codebooks. However, larger K results in more training costs. Built on AQ, we propose an enhanced accumulative quantization (E-AQ) to reduce the errors during training codebooks under the same K, so that ANN search accuracy can be improved.

E-AQ consists of codebook training and vector quantization, whereas codebook training includes initial codebook training and codebook optimization.

3.1. Codebook Training

By combining the idea of the quarter point with AQ, with E-AQ we aimed to train M codebooks consisting of K*M centroids to quantize and approximate vectors. Instead of the nearest centroid, the quarter point was applied and used to quantize vectors.

3.1.1. Initial Codebook Training

Let $X=\{x_n\in {\mathrm{R}}^{\mathrm{D}}\}, n=1,\dots ,N,$ be the training vectors set. Similar to AQ, E-AQ decomposes X into M training sets where the dimension of vectors stays unchanged. Each vector in X is equal to the sum of its corresponding M partial vectors. As shown in Figure 1, each vector was first divided into M D/M-dimension sub-vectors. Then, each sub-vector was padded with 0 until the vector dimension was restored to D. Consequently, X was partitioned into M sets $X(1),\dots ,X(M)$.

Towards each training set $X(i)$, K-means was performed to generate an initial codebook $C_i$ that contained K centroids $\{c_{i,1},c_{i,2},\cdots ,c_{i,K}\}$.

Then the training error was measured by the mean squared error (MSE) between vectors and their reconstructions, shown in Formula (5):

$$MSE=\frac{1}{N}\left({\displaystyle \sum _{n=1}^N{‖x_n-{\widehat{x}}_n‖}^2}\right)$$

(5)

where ${\widehat{x}}_n$ is the sum of M quantization outputs after decomposing x_n into M partial vectors.

Inspired by the idea of the quarter point, in order to reduce the quantization error, the nearest centroid $c_i{}^1(x_n(i))$ and the second nearest centroid $c_i{}^2(x_n(i))$ in each codebook $C_i$ were obtained to compute the quarter vector, which was used as the quantization output ${\widehat{\mathbf{x}}}_n(i)$ of the partial vector $x_n(i)$.

Then, ${\widehat{\mathbf{x}}}_n$ was formulized by the Equation (6):

$$\begin{array}{l}{\widehat{\mathbf{x}}}_n={\displaystyle \sum _{i=1}^M\left[{\widehat{\mathbf{x}}}_n(i)\right]}\\ \hspace{1em}={\displaystyle \sum _{i=1}^M\left[\frac{3}{4}{c}_i^1\left(x_n(i)\right)+\frac{1}{4}{c}_i^2\left(x_n(i)\right)\right]}\end{array}$$

(6)

However, each codebook $C_i$ was trained independently so as to minimize the error between $x_n(i)$ and ${\widehat{\mathbf{x}}}_n(i)$. Although the quarter point mechanism was adopted, how to minimize the overall error between $x_n$ and ${\widehat{\mathbf{x}}}_n$ was still not considered. Therefore, a codebook optimization was needed for this purpose.

3.1.2. Codebooks Optimization

After training initial codebooks, the error vector $e_n$ association to each training vector $x_n$ can be denoted as the Formula (7):

$$e_n=x_n-{\widehat{\mathbf{x}}}_n (1\le n\le N)$$

(7)

The M initial codebooks were optimized sequentially in an iterative manner, which is detailed in algorithm 1. When all the M codebooks were updated once, an iteration was performed. Then the MSE was computed, and the MSE between two iterations was compared to determine whether to end codebook optimization.

Algorithm 1: Codebook optimization pseudocode.
Inputs: initial codebooks {$C_i$, i = 1, …, M}, quantization outputs {${\widehat{\mathbf{x}}}_n(i)$, i = 1, …, M; n = 1, …, N}, overall error vectors {$e_n$, n = 1, …, N}, the number K of centroids in each codebook
Outputs: optimized codebooks {$C_i$, i = 1, …, M}
1.	while true do
2.	for i from 1 to M
3.	compute input vectors by $x_n{}^{\prime }(i)={\widehat{\mathbf{x}}}_n(i)+e_n$ (n = 1, …, N)
4.	find the nearest centroid in codebook $C_i$ for each input vector
5.	for j from 1 to K
6.	collect the input vectors whose nearest centroid is $c_{i,j}$ in $C_i$
7.	a mean mechanism is designed in which the mean vector of the above     selected input vectors is computed as the new $c_{i,j}$
8.	end for
9.	with updated $C_i$, recompute the quarter point for each input vector and    take it as new quantization output ${\widehat{\mathbf{x}}}_n(i)$
10.	update each error vector by $e_n=x_n{}^{\prime }(i)-{\widehat{\mathbf{x}}}_n(i)$
11.	end for
12.	if the MSE computed by Formula (5) converges
13.	terminate the algorithm
14.	end if
15.	end while

3.2. Quantizing Vector Based on E-AQ

To quantize a vector y, we generally followed AQ, including initial vector quantization and quantization output optimization. However, instead of the nearest centroid, we computed the quarter point using E-AQ, which was considered the quantization output so that y could be approximated more accurately.

(1) Initial vector quantization. After decomposing y into M partial vectors $y(m) (m=1,\dots ,M)$, each $y(m)$ was quantized with the corresponding codebook $C_m$, where the nearest centroid $c_m^1\left(y(m)\right)$ and second nearest centroid $c_m^2\left(y(m)\right)$ were found in order to compute the quarter vector according to formula (1). Then, M initial quantization outputs $\widehat{\mathbf{y}}(m) (m=1,\dots ,M)$ were obtained; meanwhile, the overall quantization error vector $e$ was computed by Formulas (6) and (7).

(2) Quantization output optimization. Like AQ, the M quantization outputs were optimized sequentially for the purpose of minimizing the quantization error with fixed codebooks. The detail of optimizing is shown in algorithm 2. When the M quantization outputs were optimized completely, an iteration was performed. Then the new generated quantization outputs were compared with those in the last iteration, and the optimization ended if the quantization outputs did not change. Figure 2 displays that the curve of optimizing quantization outputs tended to be flat in small numbers of iterations. We concluded that this process converged rapidly.

Algorithm 2: Quantization output optimization pseudocode.
Inputs: codebooks $\{C_i$, i = 1, …, M}, initial quantization outputs $\{\widehat{\mathbf{y}}(m)$, m = 1, …, M} of vector y, overall error vector e
Outputs: optimized quantization outputs $\{\widehat{\mathbf{y}}(m)$, m = 1, …, M},
1.	while true do
2.	for m from 1 to M
3.	compute the input vector by $T=\widehat{\mathbf{y}}(m)+e$
4.	compute the quarter vector and take it as new quantization output $\widehat{\mathbf{y}}(m)$
5.	compute the residual vector by $e=T-\widehat{\mathbf{y}}(m)$
6.	end for
7.	if quantization outputs do not change
8.	terminate the algorithm
9.	end if
10.	end while

4. E-AQ-Based Exhaustive ANN Search Method

To evaluate the accuracy of approximating vectors by E-AQ, an exhaustive search for ANN was implemented. Then the Euclidean distance d(,) between query vector q and vector y in the dataset was approximated using the Equation (8):

$$d(q,y)\approx d(q,\widehat{\mathbf{y}})$$

(8)

Suppose the M quantization outputs of y are $\widehat{\mathbf{y}}(m), m=1,\dots ,M$. By applying Formula (6), Formula (8) can be transformed into the following Formula (9):

$$\begin{array}{l}d{\left(q,\widehat{\mathbf{y}}\right)}^2\\ ={‖q-\widehat{\mathbf{y}}‖}^2={‖q-{\displaystyle \sum _{m=1}^M\widehat{\mathbf{y}}(m)}‖}^2\\ ={‖q-{\displaystyle \sum _{m=1}^M\left[\frac{3}{4}{c}_m^1\left(y(m)\right)+\frac{1}{4}{c}_m^2\left(y(m)\right)\right]}‖}^2\\ ={‖q‖}^2+{‖\widehat{\mathbf{y}}‖}^2-2{\displaystyle \sum _{i=1}^M〈q,\left(\frac{3}{4}{c}_m^1\left(y(m)\right)+\frac{1}{4}{c}_m^2\left(y(m)\right)\right)〉}\\ ={‖q‖}^2+{‖\widehat{\mathbf{y}}‖}^2-\frac{3}{2}{q}^T\cdot c_m^1(y(m))-\frac{1}{2}{q}^T\cdot c_m^2(y(m))\end{array}$$

(9)

The vectors q and y were both assumed to be column vectors; subsequently, the items $q^T\cdot c_m^1(y(m))$ and $q^T\cdot c_m^2(y(m))$ were identical to the inner product between q and the centroids in the mth codebook, which was calculated and stored in a look-up table when q was submitted. The term ${‖\widehat{\mathbf{y}}‖}^2$ was pre-computed when quantizing it completely. The term ${‖q‖}^2$ was constant for all database vectors and did not affect the ANN search, so it did not need to be computed.

5. Experiments

5.1. Datasets

Two publicly available datasets, the SIFT descriptor dataset and the GIST descriptor dataset, were used to evaluate performance.

Both the SIFT and GIST datasets had three subsets: learning set, database set and query set. The learning set was used to train stage codebooks, and the database and query sets were used for evaluating quantization performance and ANN search performance. For the SIFT dataset, the learning set was extracted from Flicker images [46] and the database and query vectors were extracted from INRIA holidays images [47]. For GIST dataset, the learning set consisted of the tiny image set of [48]. The database set was the holidays image set combined with Flicker 1M. The query vectors were extracted from the holidays image queries. All the descriptors were high-dimensional float vectors. The details of the datasets are given in

Table 1. Summary of the used datasets.

Information	SIFT-1M	GIST-1M
Dimension	128	960
Size of learning set	100,000	500,000
Size of database set	1,000,000	1,000,000
Size of query set	10,000	1000

.

5.2. Convergence of Training Codebooks

For easy implementation, the total number of iterations (the total number was 10) was set as the convergence condition instead of the preset threshold when optimizing codebooks. The training error produced in training codebooks was adopted to measure the convergence.

The number of centroids in each codebook was denoted as the parameter K, which ranged {64, 256}. The number of codebooks was denoted as the parameter M, which ranged {2, 4, 8, 16}.

Figure 3 and Figure 4 show the convergence of training codebooks under different parameters of K and M on SIFT-1M and GIST-1M, respectively. When the iteration number is 0, the training error represents the error produced while training the initial codebooks.

The training error was measured by the MSE, which was computed according to Formula (5).

With larger K or M, the final training error became smaller. Compared to the error from training initial codebooks, the training error reduced significantly after performing codebook optimization. Moreover, the process of optimizing codebooks could converge rapidly in a few number of iterations, which can be observed from Figure 3 and Figure 4. It took only 5 iterations on SIFT-1M, and only 7 iterations on GIST-1M.

5.3. Comparison of Training Error and Quantization Error

Figure 5 shows the comparison of training error produced by training codebooks between E-AQ and AQ [34] under different parameters on SIFT-1M and GIST-1M. The training error was measured by the MSE, computed according to Formula (5). The parameter K denotes the number of centroids in each codebook, ranged {64, 256}. The parameter M denotes the number of codebooks, ranged {2, 4, 8, 16}.

It can be seen from Figure 5 that the training error of both AQ and E-AQ reduced with increasing values of K and M. Meanwhile, under the same K or M, E-AQ had a smaller training error than AQ on both SIFT-1M and GIST-1M. We may draw the conclusion that E-AQ can further improve the discrimination of codebooks by combining with the quarter point mechanism.

Figure 6 compares the errors produced through quantizing 1M database vectors in SIFT and GIST datasets under different K and M. The overall quantization error was measured with the MSE between vectors and their reconstructions. It can be seen that E-AQ had lower overall quantization error than AQ under the same K and M. This means that vectors can be approximated more accurately by E-AQ than AQ.

5.4. ANN Search Performance

Exhaustive ANN search was implemented on SIFT-1M and GIST-1M to evaluate the accuracy to which vectors were approximated by their quantization output.

This section mainly discusses the impacts of K and M on the accuracy of ANN search, which was measured with Recall@R. This was defined as the proportion of query vectors for which the nearest neighbor was ranked in the first R position. The larger the Recall@R was, the better the accuracy of the ANN search was.

Figure 7 and Figure 8 show the Recall@R (R = 1, 2, 5, 10, 20, 50, 100) when performing E-AQ for exhaustive ANN search on SIFT-1M and GIST-1M respectively. The parameter K was set as $K\in \{64,128,256,512\}$, and the parameter M was set as $M\in \{2,4,8,16\}$.

Each curve shows that the ANN search accuracy was improved with increasing values of K under the same M. By comprehensively observing the (a), (b), (c) and (d) in Figure 7 and Figure 8, larger M brought better ANN search accuracy under the same K. Comparing Figure 5 with Figure 7 and Figure 8, larger K and larger M made for lower training error, as well as a more accurate ANN search. Therefore, the conclusion can be drawn that E-AQ based ANN search can be improved by reducing the error in training codebooks.

5.5. Comparison with the State-of-the-Arts

We compared our method with six state-of-the-art methods regarding the accuracy of exhaustive ANN search. These methods included PQ [18], OPQ [19], RVQ [30], QPQ [24], NOCQ [28] and AQ [34].

The parameters in these 6 methods were typically set as K = 256 and M = 8 in the corresponding references. For PQ, OPQ and QPQ, these approaches divided the vectors into 8 sub-vectors and each sub-vector was quantized with a codebook of 256 centroids. For RVQ, NOCQ and AQ, these approaches quantized the vectors with 8 codebooks where each codebook possessed 256 centroids. Hence, we set the parameters K and M to be the same in this experiment for consistency.

Figure 9 compares the exhaustive ANN search accuracy between E-AQ and these 6 approaches on SIFT-1M and GIST-1M, where the accuracy was measured with Recall@R.

From Figure 9a, it can be seen that E-AQ was superior to PQ, OPQ, RVQ, QPQ, NOCQ and AQ under the same scale of codebooks. Due to the larger dimension of the GIST vector compared to the SIFT vector, the Recall@R of all the methods decreased on the GIST-1M dataset, especially for PQ and QPQ. However, E-AQ still had the best ANN search accuracy.

Comparing E-AQ with AQ, E-AQ had better accuracy than AQ on both the SIFT-1M and GIST-1M datasets, demonstrating that E-AQ can further improve AQ by combining with the idea of the quarter point so that a vector can be approximated by its quantization outputs more accurately.

Table 2. Comparison of exhaustive ANN searches on SIFT-1M.

Methods	Recall@1	Recall@10	Recall@100
PQ	0.228	0.603	0.922
OPQ	0.245	0.639	0.940
RVQ	0.257	0.659	0.952
NOCQ	0.290	0.715	0.970
QPQ	0.315	0.745	0.975
AQ	0.300	0.740	0.980
E-AQ (M = 6)	0.284	0.737	0.963
E-AQ (M = 7)	0.368	0.811	0.989
E-AQ (M = 8)	0.401	0.852	0.996

and

Table 3. Comparison of exhaustive ANN searches on GIST-1M.

Methods	Recall@1	Recall@10	Recall@100
PQ	0.054	0.121	0.319
OPQ	0.128	0.344	0.696
RVQ	0.113	0.325	0.676
NOCQ	0.140	0.378	0.730
QPQ	0.063	0.152	0.377
AQ	0.110	0.350	0.740
E-AQ (M = 6)	0.121	0.332	0.669
E-AQ (M = 7)	0.123	0.360	0.744
E-AQ (M = 8)	0.145	0.410	0.776

also show the comparison between E-AQ and the other 6 methods of exhaustive ANN search, in which the search result of E-AQ with M = 6 and M = 7 is further detailed.

There is a scenario where the dimension of a vector cannot be divided by M when E-AQ decomposes the vector into M partial vectors, such as M = 7 on a SIFT vector. We set the number of non-zero components in each of the first 6 partial vectors as floor(D/M) and the number of non-zero components in the last partial vector as floor(D/M) + (D%M).

From Table 2 and Table 3, we can see that E-AQ outperformed the other methods on the condition of the same K and M. Moreover, E-AQ with M = 7 had better Recall@R than PQ, OPQ, RVQ, NOCQ and QPQ and comparable Recall@R with AQ on both SIFT-1M and GIST-1M. This demonstrates that AQ needs fewer codebooks than the other methods under the condition of the same search accuracy.

6. Conclusions

By combining AQ with the idea of the quarter point, we propose E-AQ for improving the accuracy of ANN search. After decomposing vectors into partial vectors, the quarter vector of each input vector is computed as the corresponding quantization output instead of using the nearest centroid in both training codebooks and quantizing vectors; as a result, vectors can be approximated by their quantization outputs more accurately. The experimental results show that E-AQ obtained better ANN search accuracy than state-of-the-art methods. Furthermore, under the condition of gaining the same ANN search accuracy, E-AQ needed fewer codebooks than other methods. This demonstrates that our approach was effective in further improving the accuracy of exhaustive ANN search.

Our approach achieved an improvement compared to AQ without increasing K and M. However, the process was performed based on the original dimension of the vectors. Therefore, the vector dimensions are still a restriction on performance. We will further investigate a method that performs in low dimensions while preserving the dependence among vector components. The k-means method was used to train the initial codebooks in our approach. It is simple and effective, but has some limitations, such as pre-setting the cluster number and only using the Euclidean distance to measure similarity. We will try to study bio-inspired metaheuristics for training codebooks. Additionally, as exhaustive ANN search needs to compute the distance between the query vector and all the vectors in a dataset, all the distances are sorted; as a result, the search time efficiency mostly depends on the scale of the dataset. In the future, we will investigate an efficient non-exhaustive ANN search with E-AQ.