Identifying All Matches of a Rigid Object in an Input Image Using Visible Triangles

Abstract

It has been suggested that for objects identifiable by their corners, every triangle formed by these corner points can serve as a reference for detecting other corner points. This approach enables effective rigid object detection, including partial matches. However, when there are many corner points, the implementation becomes impractical due to excessive memory requirements. To overcome this, we propose a new algorithm that leverages Delaunay triangulation, considering only the triangles generated by the Delaunay triangulation to reduce the complexity of the original approach. Our algorithm is significantly faster and requires significantly less memory, offering a viable solution for large problem instances. Moreover, it excels at identifying all matches of a queried object in an image when visible triangles of the object are present. A triangle formed by an object’s vertices is considered visible if a matching triangle is detected, and no vertices from any other object lie within its circumcircle. Recent AI-based methods have revolutionized rigid object matching, providing impressive accuracy with deep learning techniques. However, these methods require extensive training and specialized hardware like GPUs. In contrast, our approach requires no training or specialized hardware, making it a lightweight and efficient solution that maintains strong matching capabilities without the overhead of AI-based methods. Our study of the geometric features, combined with Delaunay triangulation, offers new mathematical insights.

1. Introduction

Rigid object identification in 2D involves recognizing and matching objects under transformations such as translation, rotation, and scaling while preserving the object’s geometric properties. This problem is fundamental in computer vision and has applications in fields ranging from robotics to medical imaging. Early approaches relied on corner detection techniques, such as the Harris Corner Detector [1], to identify robust features in an image. Traditional methods, such as the Viola–Jones algorithm [2], used the boosted cascades of simple features for object detection. Advances in feature-based methods, particularly the Scale-Invariant Feature Transform (SIFT) [3] and the Speeded-Up Robust Features (SURF) [4], revolutionized the field by enabling robust matching under varying scales and orientations. Shape-based approaches, such as the Shape Context method [5], provided additional tools for identifying objects using their contours and spatial relationships. These methods often rely on principles from computational geometry, including the Hausdorff distance [6], to compare and match geometric configurations. Geometric-based descriptors have also been explored for facial expression recognition, demonstrating their effectiveness in capturing structural features [7]. Textbooks such as Multiple View Geometry in Computer Vision [8] and Computer Vision: Algorithms and Applications [9] offer comprehensive overviews of these techniques and their integration into modern vision systems. Together, these methods and references form the foundation for accurate and efficient rigid object identification in 2D.

Rigid object detection and identification in complex scenarios have undergone significant advancements, particularly with the integration of deep learning techniques. The field saw a transformative shift with deep learning-based methods, beginning with the YOLO series [10], which reformulated object detection as a single regression problem, enabling real-time performance. RetinaNet [11] improved accuracy by introducing focal loss to address class imbalance, while Detection Transformer (DETR) [12] leveraged transformer architectures to capture global relationships within images, simplifying detection pipelines. Specialized methods further refined rigid object detection, including viewpoint adaptation techniques [13], which enhanced robustness across varying perspectives, and rigidity-aware detection for 6D object pose estimation [14], which improved detection accuracy in cluttered scenes. Additionally, application-specific advancements, such as rigid object tracking for augmented reality [15], explored dense tracking methods to enhance the performance of Augmented Reality (AR) devices. These developments illustrate how deep learning has revolutionized rigid object detection, achieving improved accuracy, efficiency, and adaptability across diverse applications.

In this work, we adopt a traditional approach by leveraging geometric features and introducing a new algorithm. Our motivation is as follows: While recent AI-driven methods have significantly advanced rigid object matching with impressive accuracy using deep learning, these techniques require extensive training and specialized hardware, such as GPUs. In contrast, our method eliminates the need for training or dedicated hardware, providing a lightweight and efficient solution that maintains strong matching performance without the computational overhead of AI-based approaches. Furthermore, our algorithm offers new insights into the concept of object identification, potentially inspiring further advancements in this domain.

A triangle-based geometric feature has been introduced as a valuable tool for object detection, utilizing the corners of rigid objects within images [16]. By employing a reference triangle and calculating distances to corners, other corner positions can be predicted and verified for object detection, even in partially occluded images. However, the original algorithm’s complexity increases significantly with the number of corner points, leading to high time and space requirements. To overcome this, we propose an alternative that uses the results of Delaunay triangulation as input, enhancing time and space efficiency while preserving accuracy.

For the method proposed in [16], the key step is to locate a triangle of query Q in the searched image I. The locations of all other corners can then be predicted using the distances from this triangle. The method in Ref. [16] considers all possible triangles as reference candidates. We propose a new approach that utilizes triangles obtained from Delaunay triangulations for both Q and I. The performance of the new method depends on successfully locating a triangle of Q within I. The correct detection of each instance is guaranteed if a certain condition is met. The new method improves both time and space efficiency. In particular, the bottleneck due to the space requirement is removed.

For a given set of vertices (corners), when no four distinct corner points are circular, the set of triangles obtained from a Delaunay triangulation is unique. This property has been well established and applied in computer vision [17] for tracing correspondences in video. The precision of our method does not rely on a unique Delaunay triangulation. It suffices for the Delaunay triangulations of Q and I to contain at least one pair of similar triangles corresponding to each instance of Q in I. For this condition to hold, the instance of Q in I must include a triangle from the Delaunay triangulation of I that is similar to a triangle in the Delaunay triangulation of Q and does not enclose any point from an overlapping object, including points from another instance of Q. We refer to such a triangle as a visible triangle of Q in the image I.

The outline of the paper is as follows: In Section 2, we briefly summarize the key ideas of the object detection method presented in [16]. Next, in the same section, we introduce a new algorithm that leverages the same geometric feature while incorporating the results of Delaunay triangulation for more efficient object detection. In Section 3, we present our test results obtained using the new algorithm. We discuss implementation details and the advantages of our new algorithm in Section 4. Finally, we provide our concluding remarks in Section 5.

2. Methods

Arslan [16] introduces a geometric feature that associates objects with identifiable corners. By considering any triangle, the unique coordinates of its corner points can be derived based on the distances to these corners. The process involves predicting all corner points by matching triangles and checking all possible triangles.

We summarize the geometric feature, the definitions, and the algorithm in [16]. These provide the foundation for the new ideas and the algorithm we propose in this work.

The key observation in this approach is that the Euclidean distances from the corners of a triangle uniquely identify a point in 2D. Figure 1 illustrates a query object and the search image in which this object is being searched. Only the corner points are extracted and considered. The images are repeated in the figure beneath the original images, now also displaying the detected corner points. All possible triangles and their pairwise matches are considered in object detection. In the case shown in Figure 1, one triangle (among many possible ones) is shaded, serving as a reference. A different corner point is selected to illustrate that it can be identified by the distances to the corners (vertices) of the triangle in the query object. These are shown by the dashed lines in the figure. Similarly, the corresponding point in the search image can be identified by using a matching triangle in the search image. There are two instances of the query image in the search image, labeled as 1 and 2. This occurs in both cases. The described process for one corner point applies similarly to all other corner points. Collectively, all corners, and hence all instances 1 and 2 are identified in this example.

Let ${▵}_{i,j,k}$ denote the triangle whose vertices are indexed by $i,j,k$, and whose internal angles are $\angle ijk,\angle jki,\angle kij$ in clockwise order. For any two triangles $t_1$ and $t_2$, let $t_1\sim t_2$ indicate that $t_1$ and $t_2$ are similar ( they have the same internal angles and proportional side lengths).

A triangle feature descriptor of Q with triangle ${▵}_{i,j,k}$ is defined as $D_{Q,{▵}_{i,j,k}}=(i,j,k,$$\angle ijk,\angle jki,\angle kij,\left|ij\right|,P=\left\{\right(dist(i,x),dist(j,x),dist(k,x)\left)\right|x\in Q\setminus \{i,j,k\}\}$, where $dist$ denotes the Euclidean distance between two points. We use $D_{Q,{▵}_{i,j,k}}.P$ to refer to the set P in this definition.

A triangle feature descriptor is a local feature descriptor for Q that predicts and yields a match for Q. A set is defined for the predicted and verified points for a potential match of Q in I. For ${▵}_{a,b,c}\in I\sim {▵}_{i,j,k}\in Q$, let $S_{D_{Q,{▵}_{i,j,k},I,{▵}_{a,b,c}}}$ denote the set of points predicted based on ${▵}_{a,b,c}$ in I, and triangle feature descriptor $D_{Q,{▵}_{i,j,k}}$ for Q. More precisely, this can be expressed as follows:

Definition 1.

For ${▵}_{a,b,c}\in I$ and ${▵}_{i,j,k}\in Q$ such that ${▵}_{a,b,c}\sim {▵}_{i,j,k}$, $S_{D_{Q,{▵}_{i,j,k},I,{▵}_{a,b,c}}}=$ $\left\{x^{\prime }\right|x^{\prime }$ is identified by $(r·dist(i,x),r·dist(j,x),r·dist(k,x\left)\right)$ for some $\left(dist\right(i,x),dist(j,x),$$dist(k,x))\in D_{Q,{▵}_{i,j,k}}.P,r=\left|ab\right|/\left|ij\right|\}\bigcup \{a,b,c\}$. For ${▵}_{a,b,c}\nsim {▵}_{i,j,k}$, $S_{D_{Q,{▵}_{i,j,k},I,{▵}_{a,b,c}}}=\varnothing$.

In other words, $S_{D_{Q,{▵}_{i,j,k},I,{▵}_{a,b,c}}}$ is a set of predicted points, each of which corresponds to a point in Q for a potential match of Q in I. If the calculated ratio $|S_{D_{Q,{▵}_{i,j,k}},I,{▵}_{a,b,c}}\cap I|/|Q|$ is greater than or equal to a given threshold ratio H in $(0,1]$, then, the subset $S_{D_{Q,{▵}_{i,j,k}},I,{▵}_{a,b,c}}\cap I$ is considered an instance of Q in I.

The objective is to find all instances of Q in I, which can be described as follows: $M(Q,I,H)=\left\{({▵}_{a,b,c},{▵}_{i,j,k})\right|{▵}_{a,b,c}$$\in I,\exists {▵}_{i,j,k}\in Q$ such that $|S_{D_{Q,{▵}_{i,j,k}},I,{▵}_{a,b,c}}\cap I|/|Q|\ge H\}$.

We summarize Algorithm 1 for the object detection problem. It takes as input a set Q of corner points for the query object and a set I containing corner points belonging to objects in the searched image. The threshold H is a parameter in $(0,1]$ used to distinguish significant matches from insignificant findings.

Algorithm 1: Algorithm for finding all matches of Q in I, summarized from [16].

In Lines 1–3, the algorithm calculates all internal angles $(\alpha ,\beta ,\gamma )$ for the triangles in Q and I and assigns them to bins $A(\alpha ,\beta ,\gamma )$, which are created using discretization parameters. These bins contain triangles from Q and I, referred to as $A(\alpha ,\beta ,\gamma ).Q$ and $A(\alpha ,\beta ,\gamma ).I$, respectively.

In Lines 4–9, the algorithm iterates over all discrete angle values. For each set of angles, all matching pairs of triangles in the Cartesian product of $A(\alpha ,\beta ,\gamma ).Q$ and $A(\alpha ,\beta ,\gamma ).I$ are considered. The corresponding geometric feature, as defined in Definition 1, is then computed in Line 5. Based on this feature, the predicted position of each corner point in I is determined. If a sufficient number of point matches are found in Line 6, a match for Q in I is confirmed and reported in Line 7.

The total time complexity of Algorithm 1 is $\Theta (N^3+C^3+N^2{N}_{\sim })$, where $N_{\sim }$ is the number of similar triangle pairs in Q and I. We note that the resulting time complexity is sensitive to the output size. The values of C and N are predefined for the algorithm, and the maximum value of $N_{\sim }$ depends on C and N. Consequently, the time requirement can be adjusted by tuning the parameters C and N. The parameter C controls the granularity of angles, where $\pi /C$ represents the size of an angular bin within which all angles are treated as equivalent. The parameter N sets an upper bound on the number of points considered in the computation. In the experiments reported, the typical values are $C=17$, $N=10$ for the query and $N=40$ for the searched image. The value of $N_{\sim }$ is often in the order of several thousand. However, the actual time complexity, $\Theta (N^3+C^3+N^2{N}_{\sim })$, can be significant, as $N_{\sim }$ may grow as large as $\Omega \left(N^6\right)$. To see this, note that there can be $O\left(N^3\right)$ similar triangles in both Q and I, implying $N_{\sim }=\Omega \left(N^6\right)$ in the worst case. We encountered such instances during our tests.

A careful study of the space requirement of Algorithm 1 shows that its space complexity is $\Theta \left(C^3{N}^3\right)$, as there can be $\Theta \left(N^3\right)$ triangles from Q and I in each of the $\Theta \left(C^3\right)$ bins. While this complexity can be mitigated by imposing limits on corner-laden objects, its practicality diminishes when dealing with images containing numerous objects with many corners, exacerbating both time and space complexities. In particular, the space requirement can become prohibitive. Although in many cases these parameters are not very large, the space complexity remains the most limiting factor when a high number of corners needs to be detected and processed in input images. It is not surprising that we encountered instances where Algorithm 1 failed to complete due to insufficient memory.

To address these challenges, our paper proposes a novel approach that leverages Delaunay triangulation, considering only the triangles generated by the Delaunay triangulation, unlike in [16]. The number of triangles considered is linear with respect to the number of corner points, offering a substantial reduction in complexity. We demonstrate that focusing solely on the triangles within the Delaunay triangulation suffices for many practical cases in which a certain condition holds. The result is a more feasible and efficient object recognition algorithm, as validated by our empirical results. There are examples where our new method produces correct results efficiently, while Algorithm 1 cannot be executed due to insufficient memory.

Another contribution of our new algorithm is its efficient handling of rigid object transformations by considering all vertex permutations of all triangles in Q. This approach does not impact the asymptotic time or space complexity of the algorithm, as the number of considered triangles remains small relative to the input size.

For a given set of discrete points P, a Delaunay triangulation, denoted by $Del\left(P\right)$ [18], is a triangulation in which no point in P lies inside the circumcircle of any triangle in $Del\left(P\right)$. For any triangle in $Del\left(P\right)$, its circumcircle does not contain any other point in P. $Del\left(P\right)$ satisfies several interesting properties. For example, in $Del\left(P\right)$ for any P, the minimum angle among all the triangles is maximized. The Delaunay triangulation of N points has $O\left(N\right)$ (approximately $2N-4$) triangles.

Figure 2 illustrates an example problem instance in which a query object and a search image contain two matches to this query, labeled as 1 and 2. The Delaunay triangulations for both the query and the image are shown beneath them. Algorithm 1 examines all possible triangles and their pairwise matches to identify all matches for the query. In contrast, the newly proposed algorithm focuses only on triangles within the Delaunay triangulations and their pairwise matches. In this case, the query matches are successfully identified as long as at least one correct triangle is found and used as a reference for each query match. The Euclidean distances from the corners of a triangle uniquely identify a point in 2D. In Figure 2, after aligning each common pair of triangles indicated by the dashed lines, we calculate the Euclidean distances from the query’s vertices to the corners of their reference triangle. Using these distances, along with the scaling factor between the matching triangles, we determine the positions of all other vertices in a potential match within the search image. In Figure 2, each of the two shaded triangles yields instances labeled as 1 and 2 of the query in the search image.

For a set of points in the plane, the Delaunay triangulation is unique if and only if no four points are cocircular, meaning that the circumcircle passing through any three points does not contain any other points from the set. In such cases, the Delaunay triangulation is well-defined and unique.

Let Q be the set of corner points for a query object and I be the set of corner points obtained from an input image.

A triangle t is a visible triangle of Q in I if t is in $Del\left(I\right)$ and there exists a matching triangle in $Del\left(Q\right)$. Two triangles are considered a match if they are similar (i.e., if they have the same angles). A triangle t is a yielding triangle for an instance of Q in I if t is a visible triangle of Q in I, and the vertices predicted and verified using the geometric feature, with t as a reference, yield an instance of Q in I. In Figure 2, two triangles in $Del\left(I\right)$ are highlighted with shaded interiors. The dashed lines connect these triangles to their corresponding matching triangles in $Del\left(Q\right)$. The highlighted triangles in $Del\left(I\right)$ are yielding triangles. It is important to note that other yielding triangles exist, but only two are shown for simplicity of illustration.

If an instance of Q in I contains all vertices of Q and does not overlap with any other object, then all triangles of Q can be found as visible triangles in I. In our proposed method, any of these triangles will yield a correct match. However, if the Delaunay triangulation for Q is not unique, there can still be a visible triangle in an instance of Q in I. A necessary and sufficient condition for detecting this instance is the existence of a yielding triangle from $Del\left(I\right)$ whose circumcircle does not encompass any point belonging to another (overlapping) object, including another instance of Q. This assumption is reasonable, as the absence of such a visible triangle precludes the possibility of detecting object instances within the given image. Furthermore, without this assumption, it is unclear whether any method could reliably determine or suggest the presence of an object instance.

We note that in Q, every triangle formed by three vertices whose circumcircle does not contain any other vertex will be part of $Del\left(Q\right)$ according the definition of Delaunay triangulation. For a given triangle in $Del\left(Q\right)$, if a similar triangle is detected in $Del\left(I\right)$, it may indicate a local feature that could lead to a match with Q in I. At least some local features must be preserved in images when a match exists. More formally, a matching pair of triangles in $Del\left(I\right)$ and $Del\left(Q\right)$ can serve as a potential reference for an instance of Q in I.

The concepts of visible and yielding triangles are used solely for analyzing the performance of our proposed method. The existence of a yielding triangle in $Del\left(I\right)$ for an instance of Q in I implies that we find this instance if we examine all matching pairs of triangles in $Del\left(Q\right)$ and $Del\left(I\right)$ as references. Our proposed algorithm is based on comparing triangles in $Del\left(Q\right)$ and $Del\left(I\right)$ and determining whether they yield matches. By using a visible triangle in I and a matching triangle in Q as a reference, the proposed method can predict all other corners of the queried object for a potential instance of Q in I. Under certain practical assumptions, this approach identifies the same matches while requiring significantly less time and space. In some cases, the proposed method produces results even when Algorithm 1 fails to run due to its excessive memory requirements.

Let $Q_1$ and $Q_2$ be two rigid objects in 2D, represented as sets of points. We consider transformations on rigid objects [19]. We say that $Q_1$ and $Q_2$ in 2D are geometrically isomorphic if and only if there exists a rigid transformation T such that $Q_2=T\left(Q_1\right)$, where T is one of the following: translation, a shift of all points in $Q_1$ by a fixed vector v; rotation, a rotation of all points in $Q_1$ around a fixed point (e.g., the origin) by an angle $\theta$; reflection, a mirroring of all points in $Q_1$ across a line of reflection, or a combination of these, for example, a reflection followed by a translation. These transformations preserve the geometry of rigid objects, ensuring that the distances and angles between all points remain unchanged. We also extend T to include the transformation resize, which scales $Q_1$ by a positive real factor r. After applying operations in T on $Q_1$, the angles and distance ratios are preserved. For any three points $p_1,p_2,p_3\in Q_1$, the angles between the vectors in $Q_1$ are preserved in $Q_2$ between the vectors for the corresponding points $p{’}_1,p{’}_2,p{’}_3\in Q_2$, respectively. For any two points $p_1,p_2\in Q_1$, and the corresponding points $p{’}_1,p{’}_2\in Q_2$, respectively, the ratio of the distance between $p_1$ and $p_2$ to the distance between $p{’}_1$ and $p{’}_2$ is preserved. This holds for all pairs of points in the same object.

If $Q_1$ and $Q_2$ are geometrically isomorphic, then $Del\left(Q_1\right)$ and $Del\left(Q_2\right)$ contain the same triangles, with the corresponding vertices, identical angles, and proportional side lengths.

Definition 2.

Compute ${\bigcup }_{f\in T}$ M(f(Q),I,H), where $f\left(Q\right)$ denotes applying the transformation f from $T=\{translation,rotation,reflection,resizing,composition\}$ to Q.

The formulation in Definition 2 considers all possible transformations of Q. Any triangle in Q can be chosen as a reference. Applying the transformation $T\left(Q\right)$ to Q is equivalent to transforming the reference triangle and computing the corresponding points to match with respect to this transformed triangle. We compute this using the set of triangles obtained from the Delaunay triangulation and their possible transformations. Algorithm 1 utilizes the geometric feature defined in Definition 1 and implements an approach that is invariant to both scaling and rotation. Scaling invariance is achieved by computing the scaling factor between two matching triangles and uniformly applying this factor to all distances.

To achieve invariance under all transformations in T, we consider all permutations of a matching triangle ${▵}_{i,j,k}$ in $Del\left(Q\right)$ with a given triangle ${▵}_{a,b,c}$ in $Del\left(I\right)$ whenever a new pair of matching triangles is identified.

Applying T to all possible triangles in Q collectively results in the same effect as applying T directly to Q. A rigid transformation T composed of translation, rotation, and reflection preserves distances and angles. When T is applied to the set of points Q, it moves each point in Q while maintaining the geometric relationships among them. The resulting set of points is $T\left(Q\right)$. If we consider all possible triangles formed by the points in Q, these triangles are subsets of Q. Applying T to each triangle involves applying T to the three vertices of the triangle, resulting in a new triangle that has undergone the same transformation as the individual points. When T is applied to every triangle ${▵}_{i,j,k}$ in Q, each triangle undergoes the same rigid transformation, resulting in the transformed set $\left\{T\right(\{i,j,k\}\}$ of triangles. The “collective total effect” of applying T to all possible triangles in Q is equivalent to transforming Q as a whole. Since the triangles are subsets of Q, transforming all triangles corresponds to transforming all points in Q. Thus, $\left\{T\left({▵}_{i,j,k}\right)\right|{▵}_{i,j,k}\in Q\}\equiv T\left(Q\right)$. Applying T to all possible triangles in Q collectively is equivalent to applying T directly to Q. The transformation of the points in Q inherently transforms all the triangles formed by those points, preserving the geometric structure.

Our observation is that any transformation based on a reference triangle can be applied to the entire Q using this reference triangle and the geometric feature in Definition 1, as all angles and length ratios are preserved under these transformations. Comparing Q and $f\left(Q\right)$ for $f\in T$ is equivalent to comparing Q and the portion of Q associated with the reference triangle g found in I after applying f to g. This comparison can also be achieved by accounting for all possible configurations of g.

For a given $\{i,j,k\}$, let ${\pi }_{i,j,k}$ be the permutation set $\left\{\right(i,j,k),(i,k,j),(j,i,k),(j,k,i),$$(k,i,j),(k,j,i)\}$. We note that for any rigid transformation of $f\in T$, and for any $\{i,j,k\}$, where ${▵}_{i,j,k}\in Del\left(Q\right)$, $f\left(\right\{i,j,k\left\}\right)$ is in ${\pi }_{i,j,k}$. Based on this, we reformulate the objective in Definition 2 as follows:

Definition 3.

Compute $\left\{({▵}_{a,b,c},{▵}_{i^{\prime },j^{\prime },k^{\prime }})\right|{▵}_{a,b,c}$$\in Del\left(I\right),(i^{\prime },j^{\prime },k^{\prime })\in {\pi }_{i,j,k},\exists {▵}_{i,j,k}\in Del\left(Q\right)$ such that $|S_{D_{Q,{▵}_{i^{\prime },j^{\prime },k^{\prime }},I,{▵}_{a,b,c}}}\cap I|/|Q|\ge H\}$.

We perform preprocessing before searching for a rigid object in an image. To extract the corner points Q and I from the query and search input images, respectively, we use the Shi–Tomasi Corner Detector [20] implemented in the following Python version 3.11 function: cv.goodFeaturesToTrack(). This function applies a slight modification to the Harris Corner Detector and produces better results. It is based on computing the minimum eigenvalue of the structure tensor (second-moment matrix). The dominant operations involve computing image gradients and eigenvalue calculations, both of which run in linear time. Thus, its time and space complexities are $O\left(M\right)$, where M is the number of pixels. From the detected corner points, we use the Delaunay triangulation implementation in Python [21] to generate Delaunay triangulations. This implementation is based on an incremental convex hull algorithm for triangulation. For N input points in 2D, the Delaunay triangulation is computed in expected time $O(NlogN)$ and in worst-case time $O\left(N^2\right)$ using $O\left(N\right)$ space. Corner detection is a crucial step in both Algorithm 1 and our newly proposed Algorithm 2. Although Delaunay triangulation is an additional preprocessing step for Algorithm 2, its computational cost is not significant in the overall complexity of our approach. We propose an object matching algorithm that takes as input the Delaunay triangulations $Del\left(Q\right)$ and $Del\left(I\right)$, generated from Q and I, respectively.

By applying Delaunay triangulation, we obtain a list of vertices along with their neighboring vertices. Using this information, we construct a list of triangles, generating $Del\left(Q\right)$ and $Del\left(I\right)$. As in Algorithm 1, we assume that both $\left|Q\right|$ and $\left|I\right|$ are $O\left(N\right)$, and therefore, $\left|Del\right(Q\left)\right|$ and $\left|Del\right(I\left)\right|$ are also $O\left(N\right)$.

Algorithm 2 takes Q, I, $Del\left(Q\right)$, and $Del\left(I\right)$ as input. It finds and returns all matches of Q in I based on the assumption that $Del\left(Q\right)$ defines the object Q. Every match of $Del\left(Q\right)$ found in $Del\left(I\right)$ corresponds to an instance of Q in I.

The input threshold H is a parameter in $(0,1]$ that is used to separate significant matches from other insignificant partial matches.

The loop in Lines 1–10 considers each triangle ${▵}_{a,b,c}$ in I, and compares it with all permutations ${▵}_{i^{\prime },j^{\prime },k^{\prime }}$ of every triangle ${▵}_{i,j,k}$ in $Del\left(Q\right)$ in the loop in Lines 2–9. Line 4 computes the set $S_{D_{Q,{▵}_{i^{\prime },j^{\prime },k^{\prime }},I,{▵}_{a,b,c}}}$ of predicted points in I, as described in Definition 1, using the dynamic programming approach which was also used in Algorithm 1. This computation is performed by first matching the two triangles, ${▵}_{a,b,c}$ and ${▵}_{i^{\prime },j^{\prime },k^{\prime }}$, and then extending the match by one verified point at a time. The position of each new point in I is predicted based on the triangle feature descriptor and subsequently verified. This iterative process is a key aspect that characterizes our algorithm as a dynamic programming algorithm.

Algorithm 2: Algorithm for finding all matches of Q in I.

From the similar triangles ${▵}_{i^{\prime },j^{\prime },k^{\prime }}$ and ${▵}_{a,b,c}$, the scale ratio is computed as $r=\left|ab\right|/|i^{\prime }{j}^{\prime }|$. For each point $x\in Q$, the transformed coordinates $x^{\prime }$ are determined using the scaled distance triplet $(r·dist(i^{\prime },x^{\prime }),r·dist(j^{\prime },x^{\prime }),r·dist(k^{\prime },x^{\prime }))$ along with the vertices of ${▵}_{a,b,c}$. This transformation is achieved by finding the intersection of three circles centered at $a,b,c$, with respective radii $r·dist(i^{\prime },x),r·dist(j^{\prime },x)$, and $r·dist(k^{\prime },x)$. This is done by creating a system of three quadratic equations and solving them. In this work, we improved the solution of the resulting system of equations by developing a different sequence of intermediate calculations, as shown in Appendix A, making the solutions more numerically robust compared to those in Algorithm 1. For equality tests involving real numbers, we consistently applied tolerance values throughout our implementation.

The coordinates of $x^{\prime }\in I$ are computed with an error tolerance, adjusted dynamically based on the scaling factor, for each pair of similar triangles. Each computed $x^{\prime }$ is then checked within I to determine if there exists a point in I within the tolerance distance. If such a point is found, it is considered a corresponding match for x in a potential alignment. Initially, every similar triangle pair contributes three point matches due to the alignment of their corner points.

Line 5 first evaluates the total number of matched points, including additional $(x,x^{\prime })$ pairs beyond the triangle corners, and verifies whether the condition $|S_{D_{Q,{▵}_{i,j,k}},I,{▵}_{a,b,c}}\cap I|/|Q|\ge H$ is satisfied. We note that since verification is done in I, the computed set is described as the intersection of $S_{D_{Q,{▵}_{i,j,k}},I,{▵}_{a,b,c}}$, and I. If the tested condition holds, Line 6 reports this set as a new match. It is easy to see that all elements of the objective set in Definition 3 will be reported in this manner.

The loop in Lines 1–10 iterates $O\left(\right|Del\left(I\right)\left|\right)=O\left(N\right)$ times. Similarly, the loop in Lines 2–9 iterates $O\left(\right|Del\left(Q\right)\left|\right)=O\left(N\right)$ times, too. The loop in Lines 3–8 iterates 6 times. In Line 4, solving a system of three quadratic equations for each point requires constant time. Therefore, Line 4, as well as Lines 5 and 6, each run in time $O\left(\right|Q\left|\right)=O\left(N\right)$. Thus, we conclude that the overall time complexity of the algorithm is $O\left(N^3\right)$. The space requirement of the algorithm is $O\left(N\right)$ since the number of triangles considered from Q and I is $O\left(N\right)$. The pairwise comparisons and matching-related computations are performed using $O\left(N\right)$ space. This represents a significant improvement over Algorithm 1, which has a space complexity of $\Theta \left(C^3{N}^3\right)$, where C is a discretization factor for angles (e.g., $C=17$).

A post-processing of this set can be performed to report all instance of Q in I. In this work, we follow the same objective. However, we propose a method that does not consider all possible triangles in Q and I.

Unlike Algorithm 1, the new algorithm does not consider every possible triangle from Q and I. Provided that a triangle in $Del\left(Q\right)$ is visible in I and included in $Del\left(I\right)$, the new algorithm will not miss any instances of Q in I. This holds true for all practical purposes. For this to fail, objects would need to overlap at every triangle in Q formed by adjacent neighbors. Under such conditions, it is unclear whether an instance of Q exists in I at all.

3. Results

The new algorithm we propose improves time requirement to $\Theta \left(N^3\right)$ and space requirement to $\Theta \left(N\right)$.

We also study the accuracy of our algorithm. Let R be a subset of points in I that corresponds to an instance of Q in I. We consider the question of whether there exists a yielding triangle $p^{\prime }\in Del\left(I\right)$. To analyze this, suppose that R matches a sufficiently large subset of $T\left(Q\right)$. Let t be a triangle in $Del\left(Q\right)$. If the circumcircle defined by the corner points of t does not contain any other points, the triangle t will be part of every Delaunay triangulation of the set Q. Therefore, in most cases, different Delaunay triangulations of Q and any transformed version $T\left(Q\right)$ that appears in I will share many triangles. The probability that two such triangulations have no triangles in common is negligible. In other words, they will “almost certainly” share a triangle that is a yielding triangle. If any of these triangles is visible in $R\subseteq I$, it will yield a match.

For a query object Q, triangles in small components of the object can serve as effective identifiers. For example, in the 2D image of an arrow, the triangular head of the arrow can define such a component. Experimental results reported in Figure 3 and Figure 4 illustrate this. The execution times shown under the images were obtained by running Algorithms 1 and 2 on a computer with a 2.3 GHz AMD EPYC 7501 processor.

In Figure 3, we present results from Algorithm 2 for the same cases examined in [16] and provide comparisons. In parts (a) and (b), all possible matches have been identified. In part (b), our new results from Algorithm 2 outperform those obtained by Algorithm 1, a difference we attribute to the improved corner-finding algorithm introduced in this study. In this case, Algorithm 2 demonstrates superior performance.

In Figure 4, we report results on completely new cases. Although the objects in parts (a) and (b) are not rigid, the results remain accurate. When shapes are sufficiently preserved and contain visible triangles, our algorithm performs well, even when the objects are not rigid. The objects in parts (c) and (d) are rigid, yielding excellent results. In part (c), the queried object has one perfect match and one near-perfect match. Both were accurately detected by the algorithm, reporting 100% and 79% matches, respectively. In part (d), all matches have been accurately found. Most matches are perfect $100\%$ matches. Partial matches near the edges of the rectangular frame are correctly identified, with match ratios starting at $87\%$. In our tests, Algorithm 1 failed to solve this case due to a space bottleneck encountered during runtime. In contrast, Algorithm 2 handled it efficiently, as shown in part (d).

4. Discussion

Algorithm 1 introduced a novel idea for detecting rigid objects. However, there are cases where the algorithm’s high complexity makes it impractical. The method’s precision relies on finding matching triangles and using them as references to predict and verify all corner points in a potential match. The new algorithm uses the same geometric feature, however, it reduces the search complexity for a matching pair of triangles significantly. This is achieved through the use of Delaunay triangulation. When there is an instance of the searched object Q in an image I, the probability that no common pair of triangles from $Del\left(Q\right)$ and $Del\left(I\right)$ exists is virtually zero. This guarantees finding the match while simultaneously saving time and space. If $Del\left(Q\right)$ and $Del\left(I\right)$ do not share any similar triangles, then, for every triangle in $Del\left(Q\right)$, a vertex belonging to another object (or another instance of Q) appears in the circumcircle of each similar triangle in $Del\left(I\right)$. Consequently, we can conclude that no detectable instance of Q exists in I in this scenario.

We summarize the similarities and differences between Algorithms 1 and 2 as follows: Both algorithms use the same geometric feature and take as input the sets Q, I, and a threshold value H. They follow the same fundamental approach—first locating a reference triangle and then constructing a match by iteratively predicting and verifying the next vertex in a dynamic programming manner. With standard line lengths and spacing in C++ code, Algorithms 1 and 2 were implemented in 595 and 466 lines, respectively. Except for a few simple common functions, they implement significantly different algorithms. Algorithm 2 differs from Algorithm 1 in several key aspects. It also takes $Del\left(Q\right)$ and $Del\left(I\right)$ as input, utilizing the Delaunay triangulations of Q and I to restrict the search space. Instead of considering all possible triangles, it only examines those within these triangulations, significantly reducing complexity. Additionally, Algorithm 2 solves the resulting system of quadratic equations differently when computing predicted vertex positions, as shown in Appendix A. Another major distinction is that Algorithm 2 accounts for rigid transformations by considering all permutations of triangle vertices. This approach, in conjunction with the geometric feature in Definition 1, inherently handles transformations of the entire object, including translation and reflection, without requiring additional space.

Algorithm 2 can be further improved to reduce both time and space complexities through better implementation. For instance, similar triangles from $Del\left(Q\right)$ and $Del\left(I\right)$ could be grouped into the same bins. Then, only the pairs (and their permutations) within the same bins would need to be considered in Line 4 of Algorithm 2, rather than all possible pairs.

In Algorithm 2, we generated the positions of bounding boxes (for inclusion relations, the largest bounding boxes). This approach was sufficient for our purpose of obtaining results for comparison. We manually added the directed lines to indicate the positions of the detected matches and included the match percentages in Figure 3 and Figure 4.

We have limited our comparisons to Algorithm 1, which have demonstrated the effectiveness of the geometric feature in comparison to the fully affine-invariant ASIFT algorithm [22], as shown in [23]. AI-based algorithms belong to a different class, as they require training on the query and rely on substantial computational resources, including extensive processing power. Naturally, these methods produce more robust, accurate, and precise results than ours. However, our algorithm is specifically designed for rigid objects without any prior training, utilizing lightweight computation. This makes it particularly well-suited for real-time search applications on untrained queries. Moreover, our study of the proposed geometric feature and its integration with Delaunay triangulation provides new mathematical insights into this method.

5. Conclusions

We propose an improved object detection algorithm for rigid objects using a previously introduced geometric feature. The algorithm leverages triangles from Delaunay triangulation as reference points. Based on these triangles, potential matches are identified through a dynamic programming approach to handle various rigid object transformations. Experiments showed results that were equal to or better than those reported in [16]. Additionally, we presented a test case that could not be handled efficiently using the same geometric feature in the past, and this was accurately and efficiently solved by the new algorithm. Our exploration of this geometric feature and its integration with Delaunay triangulation provides fresh mathematical insights into the method.