1. Introduction
Detecting objects in images is crucial in multiple applications, including surveillance, autonomous navigation, augmented reality, and so on [
1,
2,
3,
4]. While AI-based approaches such as Convolutional Neural Networks (CNNs) have demonstrated high performance in object detection [
5,
6], in scenarios where object/scene being recognized is unknown it is difficult to achieve accurate recognition. i.e., without prior training data on that specific object, traditional AI methods reach poor performance.
On the other hand, feature-based approaches like SIFT [
7], SURF [
8], and ORB [
9] offer capability to search any template but have limitations under complex visual variations. These methods extract distinctive features from an image and use them to detect objects [
10,
11]. By carefully selecting and combining these feature-based techniques, developers can achieve efficient object detection solutions without compromising performance.
Another approach to object detection involves high-level abstraction descriptors such as those based on edges and shapes. These methods combine the advantages of both CNNs and feature-based techniques [
1,
12]. They aim to be more robust than feature-based approaches, akin to CNNs, and at the same time they can maintain relatively simple complexity allowing to search any template, akin to feature-based methods.
In this work, we present an edge-based object/scene recognition method. We propose that the utilization of shapes obtained from edge points should achieve high performance given any type of template. Our contribution lies in the introduction of a novel directional pixel voting descriptor using image segments. The experimental results demonstrated high performance under complex visual variations. Furthermore, our methodology enables both recognition of both objects and scenes. This dual capability opens up promising avenues for future exploration and practical real-world applications.
1.1. The Object Recognition Problem
The object recognition problem is a key challenge in computer vision, with applications in several domains, such as robotics, surveillance, and augmented reality. Object recognition involves identifying and categorizing objects within digital images or video frames. This task is particularly challenging due to the variability in object appearance, including differences in viewpoint, scale, lighting conditions, occlusions, and so on.
Addressing the object recognition problem typically involves developing algorithms and techniques capable of extracting discriminative features from visual data, followed by classification or matching processes to determine the identity or category of detected objects. Traditional approaches, such as SIFT, SURF, or ORB, were designed to capture distinctive characteristics of objects, as shown in
Figure 1. However, these methods offer poor accuracy under complex visual variations and required extensive tuning for optimal performance across different scenarios [
13,
14].
1.2. The Scene Recognition Problem
Scene recognition is a key issue in several computer vision applications such as robotics and autonomous driving. In recent years, deep learning-based approaches have demonstrated high performance for scene recognition. In particular, convolutional Neural Networks (CNNs) demonstrated remarkable capabilities in learning representations of scenes directly from raw pixel data, enabling accurate scene categorization across diverse datasets and scenarios [
15,
16,
17,
18].
Unlike object recognition, which focuses on identifying individual objects within images, scene recognition involves understanding and categorizing entire scenes or environments depicted in images or video frames. This task presents its own set of challenges, primarily regarding the interaction on numerous objects, textures, structures, and spatial arrangements within a scene, see
Figure 2.
Scene recognition algorithms must consider a wide range of factors, including variations in scene composition, lighting conditions, viewpoints, occlusions, and clutter. In addition, scenes often exhibit semantic context and spatial relationships between objects, further complicating the recognition process. Overcoming these challenges involves the development of complex computational models capable of extracting robust visual features from scene representations and effectively capturing the contextual information suitable for accurate scene categorization.
In the current literature, scene recognition approaches consist of techniques such as bag-of-visual-words, spatial pyramid matching, and deep learning architectures like Convolutional Neural Networks (CNNs). These methods aim to encode both local and global spatial information, enabling robust recognition across diverse scene categories. However, achieving reliable scene recognition performance is not possible, with opportunities for innovation in feature representation, model design, and training strategies [
19,
20,
21,
22].
4. Results
In this section, we present experimental results for the proposed algorithm compared with previous approaches. Since the use of standard datasets has become common in the computer vision area (for learning-based approaches), we looked for some proper datasets. However, since there is a lack of standardized datasets for image-based algorithms, previous works such as [
13,
14,
27] used their own images, and this is the approach that we will follow here. Following this idea, we compare our algorithms with other canonical algorithms in the current literature, which have served as the basis for multiple works. These are SIFT [
7], SURF [
36], FAST [
37], and ORB [
9]. Multiple tests for object/scene detection and processing speed were carried out. For practical purposes, only some examples will be presented in this manuscript. In all cases, MatLab 2023b and a PC with an Intel(R) Core(TM) i9-9900K CPU 3.60 GHz, 32 GB RAM DDR3 were used (Intel, Santa Clara, CA, USA).
In
Figure 10, results for object detections are shown. In all cases, accuracy is defined as the percentage of template features successfully detected. Since accuracy is higher than 70%, the object being recognized is found in all cases. Different rotations and scale variations between the template and query were tested. In all cases, accurate detection can be reached. In
Figure 10a, two mismatches (outliers) were generated: in one case, the texture on the wall matched with a small pattern on the bottle. On the other hand, a line in the template was matched with a line in the query that is different from the line in the template. There is no global spatial information, which could be expected, since, following our descriptor, both lines’ matches present a similar orientation, length, and gradient (as both present similar contrasts between background and object, of which the feature owns). Furthermore, we can see that the “U” character was found two times. This is because we have not applied any non-maximal matched, and therefore, one feature in the template can be matched multiple times in the query. In this case, each “U” character in the template can be matched with each “U” character in the query, as expected. In
Figure 10b,c, only one outlier was generated each. Finally, no outliers are presented in
Figure 10d. In all cases, the square/mesh pattern on the bottle generated mismatches, but again, given our algorithm undertakes feature matching based on shape and color contrast against the features (gradients), it is expected that repetitive patterns can generate wrong matches. These problems can be addressed by adding global information so that each feature can be the same in terms of gradient or shape but different, for example, with respect to the Euclidean distance to the origin. Of course, this is only a hypothesis and will be explored in future work. In any case, these results demonstrate, at a high level, relatively accurate detection for object recognition. Regarding performance under complex visual variations in In
Figure 11, results for object detection are shown. In all cases, accuracy is defined as the percentage of template features that are successfully detected. Accuracy higher than 70% is reached, and therefore, accurate/robust detection is reached for objects. In
Figure 11a, changes in rotation and objects with similar shapes were tested. In all cases, accurate detection can be reached. Even with rotations such as the one presented in the black cylinder and the wood box, our descriptor can properly match these objects. This demonstrates the rotational invariance of our descriptor. In
Figure 11b, the complex texture scenario was tested, which demonstrated the robustness of the proposed method; shapes such as circles, cuboids, and so on and complex segments such as a book cover were correctly detected. Again, rotational invariance is demonstrated by the algorithm’s ability to properly match segments inside the book cover.
In
Figure 12, the performance for other methods in the literature, in particular, the original SIFT [
7] with non-maxima suppression (i.e., one feature on the template (left side), which can have multiple matches in the query (right side)) and original SURF [
36], is shown. As can be seen, by comparing only feature points or blobs, low performance is achieved. In both cases, several mismatches or multiple matches for the same feature are generated. This demonstrates the low performance that point information can generate. On the other hand, our approach in
Figure 10 and
Figure 11, which is based on shape and gradient, involves higher robustness for the matching process. This is shown in
Table 1, in which average accuracy values by considering multiple test for several templates are presented.
In
Figure 13 and
Figure 14, results for scene recognition are shown. Again, when compared to our approach (
Figure 13) with the original SIFT (
Figure 14), our approach clearly outperforms previous work. In our work, most important mismatches are generated for some cloud patterns in the template which match the tree patterns in the query. Again, this is as expected, since matched patterns have the same shape and gradient. On the other hand, SIFT generated several mismatches, and in this case, it could be difficult to determine if the scene was recognized or not.
Finally, in
Table 2 and
Table 3, processing speed comparisons are presented. Our approach, which consists of complex features (compared with points which are the features used by previous approaches), requires more processing time. However, even with this complexity, our algorithms outperform the original ORB. Considering that algorithms like ORB have been successfully used in real-time applications such as SLAM [
38], robotics [
39], and so on, we consider our approach can be successfully used. Note that our algorithm was implemented in MATLAB, without hardware acceleration or advanced programming patterns, while the other algorithms have been highly improved in terms of implementation in order to reach optimal processing speed. We believe our approach can be improved in terms of implementation, and this should increase the processing speed. This will be addressed in future work.
Limitations: Since the direction in the proposed Directional Pixel Voting Descriptor is defined in the 2D image pixel coordinate, rotations which deforms or occlude primitive shapes in template are not supported, as shown in
Figure 15a. However, even classical methods such as SIFT or SURF suffer from this in the case of textureless objects
Figure 15b.
Asa final discussion, experimental results demonstrate the effectiveness and robustness of our proposed approach for object and scene recognition tasks. We have demonstrated that our method outperforms previous approaches, particularly in scenarios with variations in rotation, scale, and texture. Despite the increased computational complexity of our method compared to traditional point-based approaches, our algorithms achieve promising performance, laying down the basis for future improvements and applications in real-world applications. Additionally, while our implementation in MATLAB provides a solid implementation, future optimizations and hardware accelerations are expected to further improve processing speed and efficiency. We believe that our work allows for improvements such as additional constraints for the descriptor previously laid down, non-maximal suppression methods (to avoid multiple matches for some features in the template and query), and so on.