Next Article in Journal
Gait Analysis Methods: An Overview of Wearable and Non-Wearable Systems, Highlighting Clinical Applications
Previous Article in Journal
A Vibration-Based MEMS Piezoelectric Energy Harvester and Power Conditioning Circuit
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Feature Optimization for Long-Range Visual Homing in Changing Environments

College of Automation, Harbin Engineering University, Harbin 150001, China
*
Author to whom correspondence should be addressed.
Sensors 2014, 14(2), 3342-3361; https://doi.org/10.3390/s140203342
Submission received: 20 December 2013 / Revised: 13 February 2014 / Accepted: 14 February 2014 / Published: 19 February 2014
(This article belongs to the Section Physical Sensors)

Abstract

: This paper introduces a feature optimization method for robot long-range feature-based visual homing in changing environments. To cope with the changing environmental appearance, the optimization procedure is introduced to distinguish the most relevant features for feature-based visual homing, including the spatial distribution, selection and updating. In the previous research on feature-based visual homing, less effort has been spent on the way to improve the feature distribution to get uniformly distributed features, which are closely related to homing performance. This paper presents a modified feature extraction algorithm to decrease the influence of anisotropic feature distribution. In addition, the feature selection and updating mechanisms, which have hardly drawn any attention in the domain of feature-based visual homing, are crucial in improving homing accuracy and in maintaining the representation of changing environments. To verify the feasibility of the proposal, several comprehensive evaluations are conducted. The results indicate that the feature optimization method can find optimal feature sets for feature-based visual homing, and adapt the appearance representation to the changing environments as well.

Graphical Abstract

1. Introduction

A usual dichotomy partitions robot visual navigation methods into quantitative and qualitative methods [1]. The estimation of metric position information is not needed in a qualitative method, and the environments are usually represented by a route. It is crucial for robots to move autonomously from their current position to the home position along the route. Numerous methods for route guidance have been proposed, including node-based [26] ones and those that do not use nodes [7,8]. Navigation models that do not separate routes into sequences of waypoints have been proposed in [7,8], following the widely neglected branch of models based on recognition-triggered response. The navigation with nodes has been modeled by local visual homing strategies [25]. Inspired by the simple principle of biological navigation, local visual homing has the ability to return to a previously visited location, and has drawn attention in robotics study as computationally cheap building blocks in a minimalistic appearance-based navigation framework.

In this paper, we present that long-range homing along a route can be achieved by implementing local visual homing strategy to sequentially visit the topological nodes. In the feature-based local visual homing in unprepared environments, the extracted visual features are ideal alternatives to artificial landmarks. However, a big challenge during long-range homing in changing environments is the management of rich feature sources [9,10]. In order to avoid irrelevant and redundant features, task-oriented selection schemes are needed, which contain additional constraints in evaluating feature usefulness for the tasks. Furthermore, in order to obtain optimal features during long-range navigation, it is necessary to maintain an up to date representation of surrounding environments. Propelled by above requirements, the main work of this paper focuses on the feature optimization orienting towards long-range feature-based visual homing in changing environments.

Several feature-based visual homing strategies have been presented in the literature, such as average displacement vector [11], average landmark vector (ALV) [12] and the included angle difference method [2]. However, they all need or imply an equal-distance assumption that all landmarks are distributed at the same distance from the snapshot location. Since the assumption is always violated in unstructured environments, the true home vector will deviate from the estimated direction [2,11,13]. Uniformly distributed features do improve the feature-based homing accuracy, especially for the ALV, which has drawn significant attention for simplicity [13]. There are adequate visual features extracted from unstructured environments to identify a goal to home, yet no attention has been given to modify the feature distribution for feature-based visual homing. A few of the widely employed feature extraction algorithms have tried to get uniformly distributed features [1416], however, none meets the requirement on feature distribution for feature-based visual homing.

The feature selection is essential to object classification [17], localization [18], robot navigation [19,20], etc. With respect to the features used in visual homing, there are several criteria for the selection process [21,22], in which the demanding task is the explicit quantitative characterization of feature properties in view of their relative importance. In the previous literature, most approaches to the characterization of feature quality focus on recognition and classification tasks, but few of them are ideally suited to feature-based visual homing. Furthermore, most previous research on visual homing was carried out under the assumption that the environments are static. The significance of rating and updating mechanisms, which are crucial to continuously evaluating the relative importance of features and discarding useless ones, is always ignored.

Motivated by the aforementioned thought, our work concerns the optimization of feature distribution, selection and updating. In particular, we focus on acquiring uniformly distributed features to fulfill the equal-distance assumption. The features are graded by the quantitatively characterized selection criteria of visual homing. When the agent retraces the environments, the importance of features is re-evaluated to update the appearance representation. In this paper, the ALV strategy is adopted as building blocks of the route for simplicity. Besides, the features are extracted by SURF algorithm [23], because of its high accuracy and less computing time. In order to improve the performance of long-range feature-based visual homing in changing environments, the work presented in this paper concentrates on maximizing the advantage of the ALV method by modifying the distribution of high quality SURF features.

The remainder of the paper is organized as follows: Section 2 outlines the extraction algorithm of well-distributed SURF features. Section 3 presents the feature selection and updating mechanisms. Section 4 shows the framework of feature optimization. Experiments in Section 5 demonstrate the performance. Section 6 draws conclusions and points out future work directions.

2. Uniformly Distributed Features

Due to their good invariance, local features have been introduced to substitute for artificial landmarks when the agent is situated in unknown environments. The Speeded Up Robust Features (SURF) [23] and Scale-Invariant Feature Transform (SIFT) [24] are widely used, however, neither controls the feature distribution. Several modified versions have focused on improving spatial distribution [1416], but none can ideally meet the need of feature-based visual homing. In addition, little work has been done to improve the distribution of SURF features. Considering that SURF burden is computationally low, we aim at improving the spatial distribution.

Considering that the SURF features are extracted in scale space and the image sizes are identical, extracting features over the scale space in a uniform way becomes very important. The SURF scale space consists of several octaves and scale layers, as shown in the left side of Figure 1. In consideration of using panoramic images, each scale layer can be divided into regular sector rings along the radial and circumferential directions (right side of Figure 1). The features should be uniformly distributed in each sector ring. Therefore, the feature distribution problem in the image can be seen as a problem in scale layers. Since the scale space is constructed by up-scaling the box filter size, the number of features decreases as the filter size increases because of the smoothing characteristics. Besides, the number of sector rings decreases progressively among octaves of the scale space.

Supposing that the number of key-points extracted by the standard SURF algorithm in the scale layer (s) of the octave (o) is Nos, the number of sector rings is nos, and the key-point number in the ith sector ring is M os i. The feature distribution in the scale layer will be relatively uniform if the key-point number in each sector ring is the same. We can define N os i in Equation (1), which denotes the target number of key-points when each sector ring has the same key-point number. For the ith sector ring, all the key-points in this sector ring will be reserved when M os i is not greater than N os i. On the contrary, if M os i is greater than N os i, the excess key-points will be discarded by their quality which is measured according to strength value and spatial dispersion. The strength value Vstr will be shown in Equation (16) and specifically explained in Section 3.1. With regard to spatial dispersion, it is computed as follows. As shown in Equation (2), Ej is the entropy of the square region used to construct the descriptor of the jth key-point, and ql is the probability of the lth gray level value over all the grey levels within the square region:

N o s i = N o s n o s , o = 1 , 2 , , O ; s = 1 , 2 , , S ; i = 1 , 2 , , n o s
E j = l q l log 2 q l

Supposing (xj, yj) is the position of the jth key-point in the ith sector ring. The center (x̄, ȳ) of all the key-points in the ith sector ring can be computed by Equation (3):

( x ¯ , y ¯ ) = ( j = 1 M o s i E j x j j = 1 M o s i E j , j = 1 M o s i E j y j j = 1 M o s i E j )
where Ej is taken as the weight of the jth key-point. Supposing distj in Equation (4) is the Euclidean distance between the jth key-point and the center, and Dist in Equation (5) is the divergence factor of the ith sector ring. Then the quality of the jth key-point in the ith sector ring can be measured by its strength value and spatial dispersion, as given in Equation (6) where WD is weight factor:
dist j = ( x j x ¯ ) 2 + ( y j y ¯ ) 2
Dist = j = 1 M o s i ( x j x ¯ ) 2 + j = 1 M o s i ( y j y ¯ ) 2 M o s i
U j = W D dist j Dist + ( 1 W D ) V str j 1 M o s i k = 1 M o s i V str k

In order to acquire uniformly distributed key-points, the points with lower quality are rejected to make sure that the key-point number is less than N os i in each sector ring. A pseudo-code description of the extraction of uniformly distributed features is given in Algorithm 1.


Algorithm 1 Algorithm of uniformly distributed feature extraction.

 1: M os i is set as the original key-point number of the ith sector ring in scale layer (s) of octave (o)
 2: N os i is set as the key-point number of the ith sector ring in scale layer (s) of octave (o) when uniform distribution
 3:for (each scale layer of each octave) do
 4: Calculate N os i according to Equation (1)
 5:for (each sector ring) do
 6:  Check whether N os i is greater than .
 7:  if TRUE then
 8:   Calculate the key-point quality according to Equation (6).
 9:   Classify the key-points by their quality in descending order.
 10:   Discard the last ( N os i− ) key-points.
 11:  else
 12:   Reserve all the key-points.
 13:  end if
 14:end for
 15:end for

3. Proposed Approaches to Feature Selection and Updating

The feature strength is crucial in feature-based visual homing in unstructured environments where adequate visual features can be extracted. With respect to the features used in the homing process, there should be several criteria of feature selection to evaluate their strength, in addition to the feature distribution. Besides, the fulfillment of long-range navigation in changing environments critically depends on the feature updating.

3.1. Feature Selection

Besides the criterion mentioned above, there are three more criteria identified for selection of features [21,22]: (1) relevance: the relevance of a feature is defined as its importance in making navigational decisions; (2) distinctiveness: features should be clearly distinguishable from their surroundings, which means the feature configurations should be unique; (3) reliability: taking the invariance of feature descriptor into account, features should cope with normal image deformations, which requires high robustness. What really counts is that the features should be widely visible, and consistently observed when the robot passes the same position every time.

The next step is to get the quantitative characterization of the criteria, in order to assign individual strength value to each feature. From the relevance aspect, individual feature evaluation is based on their relevance to the task and relationships with other features. Relevance and relationships are generally characterized in terms of correlation and mutual information. The distribution quality, discussed in Section 2, is significant for visual homing and has been taken into account during feature selection process, thus every selected feature is surely relevant to the homing task. The mutual information has been applied for navigation tasks [6,20,25,26], however, it is used in a different way in this paper. The mutual information can be used to measure the quantity of information shared by two random variables. Therefore, it can be used to evaluate the relationships among features. The more information it measures, the more relevant the feature relationships are. The mutual information is defined as follows:

M I ( X , Y ) = H ( X ) H ( X | Y )
where the entropy H(X) measures the variability of variable X. H(XY)is the conditional entropy, giving the expected information in X if a measurement is taken.

Supposing C denotes the group of all the T features in one sector ring. The mutual information MI(C,F) between group C and feature F in this group can be measured by Equation (7). Due to the lack of previous knowledge, it is supposed that p(C) has a uniform prior. Therefore, all the T features in group C have the same probability and the entropy of group C can be computed by Equation (8). In addition, the conditional entropy is given by Equation (9):

H ( C ) = log 2 T
H ( C | F ) = p ( C | F ) × log 2 ( p ( C | F ) )
where p(CF) can be computed by the Bayes rule:
p ( C | F ) = p ( F | C ) p ( C ) p ( F )

Here, p(CF) is estimated by applying a kernel density estimator [27]:

p ( F | C ) = 1 T i = 1 T K ( d ( F , F i ) )

In Equation (11)K(·); is the Gaussian Kernel function with standard deviation, and d(·)); denotes the Euclidean distance. An efficient and accurate nearest-neighbor approximation of this estimator is done as follows [27]:

p ( F | C ) = max T K ( dist ( F , F i ) )

During the navigation, the images captured at current positions are continuously matched with the image stored at the target. Supposing M is the number of times the target image has been matched, and m is the number of times that the feature F extracted from the target image has been matched. The mutual information of feature F can be computed according to Equation (13):

M I = M I ( C , F ) + j = 1 m M I ( C j , F j ) × e d j m + 1
where Fj is the matching feature of F, dj denotes the matching distance between F and Fj, and the structural principle of Cj is the same as C.

A distinctiveness selection can be on the basis that the more informative the feature is, the greater its distinctive power reduces. The feature distinctiveness is written as Equation (14):

D = M I F j = 1 m M I ( C j , F j )

The contrast value indicates the reliability property directly. In addition, the power of reliability can also be estimated by the record times that the feature has been successfully re-observed. Thus the quantitative characterization of reliability can be measured according to Equation (15):

R = W H H norm + ( 1 W H ) m M
where Hnorm denotes the normalized Hessian value (SURF key-point), and WH is the weight factor.

The feature strength value Vstr, given in Equation (16), is obtained by adding the three quantitative characterizations:

V str = W M M I norm + ( 1 W m W R ) D norm + W R R
where MInorm is the normalized mutual information, Dnorm is the normalized reliability, WM and WR are the mutual information and the reliability weight factors, respectively.

3.2. Feature Updating

As for the long-range navigation in changing environments, the feature updating is a matter of cardinal significance. The approaches close to our work are presented in [9,10]. They are both based on the multi-store model of human memory [28], which is divided into three basic stores: sensory memory (SM), short-term memory (STM) and long-term memory (LTM). The sensory memory stores all information perceived by the senses, and elementary selection measures are taken to determine what information will be transported to the STM. Through a process of rehearsal, the information in STM can be transported to LTM to be retained for longer periods of time. The approach presented in [9] is mainly used to keep feature sets updating, and the feature upgrade or degradation is based on feature “hit” or “miss” rather than quantitative characterization. An adapted memory model is adopted to update the features in [10], where the main purpose is robot localization.

In this paper, the feature updating is implemented by adopting the concepts mentioned above in a simple way. During the process of SURF feature extraction, original key-points are denoted as SM, and those with low contrast and large principal curvature will be rejected. Then the final features are classified as STM or LTM according to their strength values. For the purpose of finding the most suitable threshold Tstr to distinguish STM from LTM, an experiment described specifically in Section 5.1 is conducted to test the homing performance. When the feature strength values are re-computed, the feature sets will be updated. For STM, they have four times to be transported to LTM, on the premise that their strength values are greater than the threshold Tstr, otherwise, they will be deleted. Besides, the LTM will degrade to STM if the strength values are less than the threshold Tstr.

4. Overview of the Proposed Feature Optimization Procedure

The outline of the feature optimization procedure of long-range visual homing is shown in Figure 2. The route consisting of nodes is determined when the robot enters the environments for the first time. When the robot retraces the environments, it follows the route using the strategies in [26]. During the retracement process, the ALV homing algorithm directs the robot towards the target node. In addition, the ALV homing direction is computed with the assistance from an external compass. The two-stage procedure is detailed as follows:

(1)

Map building. The robot is manually driven in the environments and panoramic images are captured at particular places with approximately 2 m between each other. Each node in the route is a collection of optimal features, which are extracted from the panoramic image. In the first step, each panoramic image is divided into regular sector rings as shown in Figure 1, and features are extracted by standard SURF algorithm. Then the strength value is calculated for each feature, and all features are classified into STM and LTM according to their strength values.

For all the LTM features, the method proposed in Section 2 is used to improve the feature distribution, regardless of the STM features. If the LTM feature number in one sector ring is less than the number required, the STM features with higher strength values compensate for the shortage of feature number. The uniformly distributed LTM features are used to compute the ALV vectors.

(2)

Retrace phase. When the robot follows the succession of nodes by ALV homing method, the feature updating approach will be adopted.

During the navigation between two nodes, panoramic images are continuously captured and matched with the image taken at the target node before the robot arrives at the node. The strength values of features stored at the target node are re-evaluated during this process. Therefore, the feature set can be updated. It should be noticed that feature matching aims at re-evaluating the strength values rather than calculating homing direction, since there is no need of feature matching in the ALV method.

The homing direction will be changed in two cases. One case is that, if the LTM features used to compute current direction of migration change by Tnum or higher, the ALV vector at current location will be computed to obtain new homing direction. The other case is that, if the LTM features stored at the target node is updated, its ALV vector will be re-computed and the robot will move on in a new direction.

With regard to route guidance using linked local visual homing, there are two issues of crucial importance that have to be noticed: the determination of a new node and the arrival detection at a node [3,7,8]. As part of autonomous navigation, the major objective of this work is to maintain feature optimization, thus, the determinations on at which point a new node is set and whether the robot is sufficiently close to the target node are not the key points. In this paper, the locations of nodes are manually selected at a distance of approximately 2 m. Besides, the arrival detection of nodes is straight. During navigation between nodes, features extracted at current position are continuously matched with features stored at the target node. Arrival is declared once the maximum value of the bearing angle differences of matching features falls below threshold Tθ.

5. Experiments

5.1. Robot Platform and Parameter Settings for Experiments

Our experiments were performed with a crawler mobile robot (see Figure 3). A panoramic vision system, composed of a mirror with a diameter of 100 mm and a camera with a resolution of 800 × 800, was mounted on top of the robot. A 2 GHz Pentium (R) processor was dedicated to vision processing and robot control. In addition, the robot was equipped with a digital compass to acquire the global orientation.

Table 1 shows the parameter settings. Similar to the standard SURF algorithm, the scale space consists of three octaves and four scale layers (O = 3 and S = 4). Because the number of features decreases gradually with the number of octaves, the number of sector rings nos should taper off. More specifically, the sector ring number in each scale layer of the first octave is 100 (5 × 5 × 4), and the others are 64 (4 × 4 × 4) and 36 (3 × 3 × 4). The weight factor WD in Equation (6) must be determined by an appropriate compromise between the spatial dispersion and feature strength value. The optimal value of WD was determined by a local homing experiment, where we used 60 random pairs of panoramic images captured in the room and corridor. When the homing error was calculated, the Hessian value was substituted for feature strength value in Equation (6). The value assigned to WD was changed from 0 to 1 in 0.1 step. By seeking the minimum average angular error of homing, 0.3 was determined as the optimal value. The same experiment was conducted to determine the optimal value of WH, and the only difference was that the reliability value was substituted for feature strength value in Equation (6). Finally WH was set to 0.6 through the experimental analysis. Similar to the above two experiments, another experiment was performed to figure out the most suitable WM and WR values. The feature strength value measured according to Equation (16) was used. The minimum average angular error of homing was obtained when WM value was 0.5 and WR value was 0.3.

We conducted another experiment to find the optimal threshold to distinguish LTM features from STM features. The threshold Tstr must balance feature quantity and quality. A low threshold will result in redundant features, and the features with low relative importance will degrade the homing performance. On the contrary, a high threshold will prune too many features, giving rise to the lack of representativeness of environmental appearance. The STM features will be less accessible to turn into LTM ones. The experimental results are shown in Figure 4. A well-established compromise in the mutual contradiction mentioned earlier was reached when the threshold Tstr was set to 0.7. In addition, the thresholds Tnum for changing homing direction and Tθ for arrival detection were empirically set to 10% and 10°, respectively.

5.2. Evaluation Tests of the Uniformly Distributed Features

To evaluate the improvement on feature distribution using the proposed approach, an experiment was conducted in the room with a 10 m × 5 m floor space and the corridor with a 16 m × 3 m floor space. Figure 5a,c show the distribution of features extracted by the standard SURF algorithm. They manifest that the feature distribution is non-uniform, which means that the homing performance will be degraded. Figure 5b,d show the feature extraction results by the proposed modified version of SURF algorithm. The comparison shows the feasibility of the proposed approach in the improvement on feature distribution. However, this is just visual inspection. The improvement on the distribution quality should be judged in terms of visual homing. The chosen measure of quantitative characterization of distribution, similar to the one proposed in [29], is defined as:

Q dist = 1 O × S o = 1 O s = 1 S ( i = 1 n o s p i log 2 p i )
where pi is the ratio of matching features in the ith sector ring over all the ones in the scale layer (s) of the octave (o). If the correctly matching features were uniformly distributed, a maximum distribution quality would be achieved (5.94 for our experiments).

Figure 6 shows the homing error and the corresponding feature distribution quality. Homing error is the absolute angular error between correct homing vector and the computed homing vector. In addition, the homing method presented in [4] was taken for comparison, as shown in Figure 6e,f. The proposed approach performed as well as the method in [4], with slight performance differences. The promising results indicate that it is feasible to improve the homing accuracy of ALV by employing the proposed approach, which can decrease the impact of anisotropic feature distribution.

5.3. Long-Range Homing Experiments

In this section, long-range homing experiments were conducted to demonstrate the feasibility of the feature optimization approach, and the standard SURF algorithm was taken for comparison. Figure 7 shows the environment representation marked with four typical regions, where no special arrangements and modifications were made to facilitate the execution of visual homing. The robot was located upon the starting node, and sequentially passed the intermediate nodes until it arrived at the home node eventually. Notice that the initial orientations of the robot were roughly the same in the following experiments.

The experiments were classified into several groups according to the degree of change of the environmental appearance. The environments were manually changed in various manners, including modifying illumination conditions by control of curtains and lamps, rearranging object locations, adding new objects, removing existing objects and covering objects with cardboard. It should be noticed that temporary occlusions caused by people always existed. Here, five types of experiments had been conducted: in the first one, under good illumination conditions, the environmental layout remained the same as that in the first tour without modification; the second one was conducted under worse illumination conditions, compared with the first one; in the third and fourth experiments, environment changes were identical, and the illumination conditions were the same as that in the first and second, respectively; the fifth one was conducted in randomly changing environments. By means of the proposed feature optimization approach, the features stored in nodes updated five times via the five types of experiments mentioned above.

In order to demonstrate the effectiveness of each part of the feature optimization procedure, our approach was compared with three other approaches. The experimental results are presented in Figures 8,9, 10,1112. The results of the proposed approach and the standard SURF are shown by red and violet, respectively. The blue indicates the results obtained by our approach without the procedure of feature selection and updating. And the results shown by green are obtained by our approach without the procedure of feature updating. The decrease of curvature and the improvement on smoothness can be chosen as criteria to the effectiveness of the proposed approach on homing accuracy.

5.3.1. Long-Range Homing with Different Illumination Conditions in Non-Modified Environments

Figures 8 and 9 show the homing trajectories under different illumination conditions in non-modified environments. Figure 8 corresponds to the results obtained at around noon with fairly good illumination, and Figure 9 corresponds to the results obtained in the afternoon with relatively poor illumination. It can be observed that the experimental results obtained under fairly good illumination condition are better than those obtained under relatively poor illumination conditions. Observing the results of the four approaches, our approach outperforms the other three ones. The performance of the standard SURF is the worst, which may be caused by its poor control of the feature distribution and quality. The performance differences between the two experimental results are relatively bigger when using the standard SURF, compared with the other three approaches. This can be attributed to the utilization of feature selection and updating, which partially decrease the influence of large changes in illumination and people. Moreover, it is noteworthy that the homing performance in the corridor is improved apparently owing to the uniformly distributed features, compared to that in the room and hall. Although the performance is improved in the corridor, the homing error is bigger compared with that in the room and hall. Besides, a problem has been arisen that there exists significant degradation of homing performance in the junctions of two corridors, between the room and corridor, since the environmental appearance information between them is obviously different.

5.3.2. Long-Range Homing with Different Illumination Conditions in Modified Environments

The experimental results under different illumination conditions in modified environments are provided in Figures 10 and 11. Figure 10 indicates the results obtained at around noon with fairly good illumination, and Figure 11 is obtained at around afternoon with relatively poor illumination. The modifications made in the environments will result in decreasing of matching feature number, thus the ability of coping with the large changes in environments can be tested. Analyzing the homing trajectories, our approach performs well despite the changes in environmental layout. Though the homing error caused by anisotropic feature distribution decreases when the robot approaches the target node, the performance differences demonstrate that the feature optimization procedure does take effect. In a word, our approach maintains a better appearance representation of the changing environments.

5.3.3. Long-Range Homing in Randomly Changing Environments

Figure 12 shows the homing trajectories under randomly changing environmental conditions. The illumination conditions were changed randomly by controlling the curtains and lamps, and the object positions were in disorder, posing a challenge to visual homing. Comparing the experimental results with those in Sections 5.3.1 and 5.3.2, it is clear that the performance differences increase as the feature set updating times goes up. It is worth to note that the approach using standard SURF is incapable of dealing with these situations. From the experimental results, the effect of the feature optimization in our approach, which is the result of these performance differences, is evident. Given these challenging conditions, our approach performs well and demonstrates that it is of importance to get a suitable appearance representation of the environments.

5.4. Analysis of Experimental Results

From the perspective of route following, the homing trajectories in this paper are not the straightest, compared with those in [3032]. Considering the simple navigation framework, the homing error tends to decrease as the robot is guided closer to the target node. Besides, the method of selecting node locations may result in the lack of features in common between consecutive nodes. Therefore, the homing direction will be changed frequently due to the strategy of changing mobile direction. The changing environments, the extreme unevenness of feature distribution in the corridor, and the transition from one environment to another with obviously different appearance pose severe challenges to visual homing.

To quantify the performance, the mean values and standard deviations of the mobile direction angles during the long-range homing are listed in Table 2, where the smoother trajectory and better performance is indicated by lower mean value and standard deviation. From the homing trajectories and mobile direction angles, the feature optimization method improves the homing accuracy. However, it is hard to know how well the proposed method performs due to the error introduced by the digital compass. During the first tour when the route was determined, the images were captured when the robot was parallel to one side of the oblong floor tile. Therefore, the directions of translation between images stored at neighboring nodes are known. The homing errors between neighboring nodes can be computed using the feature sets obtained through the above five experiments, without impact of the digital compass. The results in Table 3 indicate that each part of the optimization procedure has a positive effect on the decrease of homing error.

Despite the homing performance improvement, the equal-distance assumption is difficult to meet completely. This is obvious in the corridor and can be explained in the way that features are projected on the panoramic images [13]. In addition, the mechanisms of arrival detection and new node determination in this paper do not perform very well. The robot does wrongly declare the node arrival in a number of cases. The set distance of 2 m between consecutive nodes does not work in every environment and results in lack of autonomy.

6. Conclusions

This paper proposes an approach to feature optimization, especially for long-range feature-based homing in changing environments. The task-dependent optimization procedure consists of feature distribution, selection and updating. The method proposed in this paper shows good homing performance, including improvement on feature distribution quality, homing accuracy and environmental adaptability. Therefore, it is feasible and suitable for long-range navigation in changing environments.

The selection of key images is significant for representing a route with fewer nodes. In addition to that, the arrival detection is closely related to homing accuracy. However, the locations of nodes are determined manually and arrival detection mechanism does not work well in this paper. As crucial parts of autonomous navigation, more efforts need to be spent on solving these two issues. Other possible improvements lie in computing the most suitable strength value threshold with an adaptive algorithm to distinguish STM features from LTM features, as well as exploring the possibility of substituting visual compass for inaccurate digital compass.

Acknowledgments

The authors extend thanks to the reviewers for their valuable comments which have improved this paper significantly. This work is partially supported by National Natural Science Foundation of China (61175089, 61203255), Fundamental Research Funds for the Central Universities (HEUCF1321001, HEUCFX41304), and the Science Foundation for Youths of HRB (NO.2013RFQXJ106).

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Möller, R.; Krzykawski, M.; Gerstmayr, L. Three 2D-warping schemes for visual robot navigation. Auton. Robots 2010, 29, 253–291. [Google Scholar]
  2. Argyros, A.A.; Bekris, K.E.; Orphanoudakis, S.C. Robot homing by exploiting panoramic vision. Auton. Robots 2005, 19, 7–25. [Google Scholar]
  3. Smith, L.; Philippides, A.; Graham, P.; Baddeley, B. Linked local navigation for visual route guidance. Adapt. Behav. 2007, 15, 257–271. [Google Scholar]
  4. Labrosse, F. Short and long-range visual navigation using warped panoramic images. Robot. Auton. Syst. 2007, 55, 675–684. [Google Scholar]
  5. Fu, Y.; Hsiang, T.R.; Chung, S.L. Multi-waypoint visual homing in piecewise linear trajectory. Robotica 2013, 31, 479–491. [Google Scholar]
  6. Dame, A.; Marchand, E. Using mutual information for appearance-based visual path following. Robot. Auton. Syst. 2013, 61, 259–270. [Google Scholar]
  7. Baddeley, B.; Graham, P.; Husbands, P.; Philippides, A. A model of ant route navigation driven by scene familiarity. Plos Comput. Biol. 2012, 8. [Google Scholar] [CrossRef]
  8. Baddeley, B.; Graham, P.; Philippides, A.; Husbands, P. Holistic visual encoding of ant-like routes: navigation without waypoints. Adapt. Behav. 2011, 19, 3–15. [Google Scholar]
  9. Dayoub, F.; Cielniak, G.; Duckett, T. Long-term experiments with an adaptive spherical view representation for navigation in changing environments. Robot. Auton. Syst. 2011, 59, 285–295. [Google Scholar]
  10. Bacca, B.; Salvi, J.; Cufí, X. Appearance-based mapping and localization for mobile robots using a feature stability histogram. Robot. Auton. Syst. 2011, 59, 840–857. [Google Scholar]
  11. Franz, M.O.; Schölkopf, B.; Mallot, H.A.; Bülthoff, H.H. Where did I take that snapshot? Scene-based homing by image matching. Biol. Cybern. 1998, 79, 191–202. [Google Scholar]
  12. Lambrinos, D.; Möller, R.; Labhart, T.; Pfeifer, R. A mobile robot employing insect strategies for navigation. Robot. Auton. Syst. 2000, 30, 39–64. [Google Scholar]
  13. Ramisa, A.; Goldhoorn, A.; Aldavert, D.; Toledo, R. Combining invariant features and the ALV homing method for autonomous robot navigation based on panoramas. J. Intell. Robot. Syst. 2011, 64, 625–649. [Google Scholar]
  14. Song, R.; Szymanski, J. Well-distributed SIFT features. Electron. Lett. 2009, 45, 308–310. [Google Scholar]
  15. Lingua, A.; Marenchino, D.; Nex, F. Performance analysis of the SIFT operator for automatic feature extraction and matching in photogrammetric applications. Sensors 2009, 9, 3745–3766. [Google Scholar]
  16. Sedaghta, A.; Mokhtarzade, M.; Ebadi, H. Uniform robust scale-invariant feature matching for optical remote sensing images. IEEE Trans. Geosci. Remote Sens. 2011, 49, 4516–4527. [Google Scholar]
  17. Arauzo-Azofra, A.; Aznarte, J.L.; Benítez, J.M. Empirical study of feature selection methods based on individual feature evaluation for classification problems. Expert Syst. Appl. 2011, 38, 8170–8177. [Google Scholar]
  18. Thrun, S. Bayesian landmark learning for mobile robot localization. Mach. Learn. 1998, 33, 41–76. [Google Scholar]
  19. Fernando, A.; Cheein, A. Covariance-based measurement selection criterion for gaussian-based algorithms. Entropy 2013, 15, 287–310. [Google Scholar]
  20. Fernando, A.; Cheein, A.; Carelli, R. Analysis of different feature selection criteria based on a covariance convergence perspective for a SLAM algorithm. Sensors 2011, 11, 62–89. [Google Scholar]
  21. Gillner, S.; Weiβ, A.M.; Mallot, H.A. Visual homing in the absence of feature-based landmark information. Cognition 2008, 109, 105–122. [Google Scholar]
  22. Basten, K.; Mallot, H.A. Simulated visual homing in desert ant natural environments: Efficiency of skyline cues. Biol. Cybern. 2010, 102, 413–425. [Google Scholar]
  23. Bay, H.; Tuytelaars, T.; Gool, L.V. SURF: Speeded up Robust Features. Proceedings of the 9th European Conference on Computer Vision, Graz, Austria, 7–13 May 2006; pp. 404–417.
  24. Lowe, D.G. Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 2004, 60, 91–110. [Google Scholar]
  25. Campos, F.M.; Correia, L.; Calado, J.M.F. Global localization with non-quantized local image features. Robot. Auton. Syst. 2012, 60, 1011–1020. [Google Scholar]
  26. Kim, P.; Szenher, M.D.; Webb, B. Entropy-Based Visual Homing. Proceedings of the IEEE International Conference on Mechatronics and Automation, Changchun, China, 9–12 August 2009; pp. 3601–3606.
  27. Boiman, O.; Shechtman, E.; Irani, M. In Defense of Nearest-Neighbor Based Image Classification. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Anchorage, AK, USA, 24–26 June 2008; pp. 1–8.
  28. Atkinson, R.; Shiffrin, R. Human Memory: A Proposed System and Its Control Processes. In In The Psychology of Learning and Motivation; Spence, K.W., Spence, J.T., Eds.; Academic Press: New York, NY, USA, 1968; Volume 2, pp. 89–195. [Google Scholar]
  29. Liu, M.; Pradalier, C.; Pomerleau, F.; Siegwart, R. The Role of Homing in Visual Topological Navigation. Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, Vilamoura, Portugal, 7–12 October 2012; pp. 567–572.
  30. Furgale, P.; Barfoot, T.D. Visual teach and repeat for long-range rover autonomy. J. Field Robot. 2010, 27, 534–560. [Google Scholar]
  31. Zhang, A.; Kleeman, L. Robust appearance based visual route following for navigation in large-scale outdoor environments. Int. J. Robot. Res. 2009, 28, 331–356. [Google Scholar]
  32. Chen, Z.; Birchfield, S. Qualitative vision-based path following. IEEE Trans. Robot. 2009, 25, 749–754. [Google Scholar]
Figure 1. The octave and division of the scale layer. The left side shows the octave consisting of scale layers. The right side shows the scale layer consisting of 16 sector rings.
Figure 1. The octave and division of the scale layer. The left side shows the octave consisting of scale layers. The right side shows the scale layer consisting of 16 sector rings.
Sensors 14 03342f1 1024
Figure 2. Outline of feature optimization procedure.
Figure 2. Outline of feature optimization procedure.
Sensors 14 03342f2 1024
Figure 3. The mobile robotic platform.
Figure 3. The mobile robotic platform.
Sensors 14 03342f3 1024
Figure 4. Average angular error versus strength value threshold.
Figure 4. Average angular error versus strength value threshold.
Sensors 14 03342f4 1024
Figure 5. The distribution of features. The red circles denote feature positions. (a,c) show the non-uniform distribution of features extracted by the standard SURF; (b,d) show the uniform distribution of features extracted by the proposed approach.
Figure 5. The distribution of features. The red circles denote feature positions. (a,c) show the non-uniform distribution of features extracted by the standard SURF; (b,d) show the uniform distribution of features extracted by the proposed approach.
Sensors 14 03342f5 1024
Figure 6. The homing error and distribution quality. The axes denote the positions where the homing errors are computed. (a,b) show the homing error in the room using features extracted by the standard SURF and the proposed approach, respectively; (c,d) show the distribution quality corresponding to the results in (a,b); (e,f) show the homing error in the room and the corridor, respectively using the method in [4]; (g,h) show the homing error in the corridor using features extracted by the standard SURF and the proposed approach, respectively; (i,j) show the distribution quality corresponding to (g,h).
Figure 6. The homing error and distribution quality. The axes denote the positions where the homing errors are computed. (a,b) show the homing error in the room using features extracted by the standard SURF and the proposed approach, respectively; (c,d) show the distribution quality corresponding to the results in (a,b); (e,f) show the homing error in the room and the corridor, respectively using the method in [4]; (g,h) show the homing error in the corridor using features extracted by the standard SURF and the proposed approach, respectively; (i,j) show the distribution quality corresponding to (g,h).
Sensors 14 03342f6 1024
Figure 7. The experimental environments. Phase 1 denotes the visual homing process in the room and the junction between the room and corridor; phase 2 corresponds to the process in the junction between two corridors; phase 3 denotes the process in the corridor; phase 4 corresponds to the process in the hall.
Figure 7. The experimental environments. Phase 1 denotes the visual homing process in the room and the junction between the room and corridor; phase 2 corresponds to the process in the junction between two corridors; phase 3 denotes the process in the corridor; phase 4 corresponds to the process in the hall.
Sensors 14 03342f7 1024
Figure 8. The homing trajectories in non-modified environments with fairly good illumination. (ad) show the results obtained in phase 1, 2, 3 and 4, respectively. Red lines correspond to the proposal. Violet lines correspond to SURF. Blue lines correspond to the proposal without selection and updating. Green lines correspond to the proposal without updating. In each figure, the robot moves from left to right, the black disks with the same size are intermediate nodes, and the bigger one is starting node or home node.
Figure 8. The homing trajectories in non-modified environments with fairly good illumination. (ad) show the results obtained in phase 1, 2, 3 and 4, respectively. Red lines correspond to the proposal. Violet lines correspond to SURF. Blue lines correspond to the proposal without selection and updating. Green lines correspond to the proposal without updating. In each figure, the robot moves from left to right, the black disks with the same size are intermediate nodes, and the bigger one is starting node or home node.
Sensors 14 03342f8 1024
Figure 9. The homing trajectories in non-modified environments with relatively poor illumination. (ad) show the results obtained in phase 1, 2, 3 and 4, respectively. Red lines correspond to the proposal. Violet lines correspond to SURF. Blue lines correspond to the proposal without selection and updating. Green lines correspond to the proposal without updating. In each figure, the robot moves from left to right, the black disks with the same size are intermediate nodes, and the bigger one is starting node or home node.
Figure 9. The homing trajectories in non-modified environments with relatively poor illumination. (ad) show the results obtained in phase 1, 2, 3 and 4, respectively. Red lines correspond to the proposal. Violet lines correspond to SURF. Blue lines correspond to the proposal without selection and updating. Green lines correspond to the proposal without updating. In each figure, the robot moves from left to right, the black disks with the same size are intermediate nodes, and the bigger one is starting node or home node.
Sensors 14 03342f9 1024
Figure 10. The homing trajectories in modified environments with fairly good illumination. (ad) show the results obtained in phase 1, 2, 3 and 4, respectively. Red lines correspond to the proposal. Violet lines correspond to SURF. Blue lines correspond to the proposal without selection and updating. Green lines correspond to the proposal without updating. In each figure, the robot moves from left to right, the black disks with the same size are intermediate nodes, and the bigger one is starting node or home node.
Figure 10. The homing trajectories in modified environments with fairly good illumination. (ad) show the results obtained in phase 1, 2, 3 and 4, respectively. Red lines correspond to the proposal. Violet lines correspond to SURF. Blue lines correspond to the proposal without selection and updating. Green lines correspond to the proposal without updating. In each figure, the robot moves from left to right, the black disks with the same size are intermediate nodes, and the bigger one is starting node or home node.
Sensors 14 03342f10 1024
Figure 11. The homing trajectories in modified environments with relatively poor illumination. (ad) show the results obtained in phase 1, 2, 3 and 4, respectively. Red lines correspond to the proposal. Violet lines correspond to SURF. Blue lines correspond to the proposal without selection and updating. Green lines correspond to the proposal without updating. In each figure, the robot moves from left to right, the black disks with the same size are intermediate nodes, and the bigger one is starting node or home node.
Figure 11. The homing trajectories in modified environments with relatively poor illumination. (ad) show the results obtained in phase 1, 2, 3 and 4, respectively. Red lines correspond to the proposal. Violet lines correspond to SURF. Blue lines correspond to the proposal without selection and updating. Green lines correspond to the proposal without updating. In each figure, the robot moves from left to right, the black disks with the same size are intermediate nodes, and the bigger one is starting node or home node.
Sensors 14 03342f11 1024
Figure 12. The homing trajectories in randomly changing environments. (ad) show the results obtained in phase 1, 2, 3 and 4, respectively. Red lines correspond to the proposal. Violet lines correspond to SURF. Blue lines correspond to the proposal without selection and updating. Green lines correspond to the proposal without updating. In each figure, the robot moves from left to right, the black disks with the same size are intermediate nodes, and the bigger one is starting node or home node.
Figure 12. The homing trajectories in randomly changing environments. (ad) show the results obtained in phase 1, 2, 3 and 4, respectively. Red lines correspond to the proposal. Violet lines correspond to SURF. Blue lines correspond to the proposal without selection and updating. Green lines correspond to the proposal without updating. In each figure, the robot moves from left to right, the black disks with the same size are intermediate nodes, and the bigger one is starting node or home node.
Sensors 14 03342f12 1024
Table 1. Parameters in the proposed approach.
Table 1. Parameters in the proposed approach.
ParametersValueParametersValueParametersValue
O3n4s16Tstr0.7
S4WD0.3Tnum10%
n1s100WH0.6Tθ10°
n2s64WM0.6
n3s36WR0.3
Table 2. Mobile direction angles. Experiments 1–5 correspond to the results shown in Figures 8,9,10,1112, respectively. E (°) denotes the mean value of mobile direction angle, and σ (°) denotes the standard deviation of mobile direction angle. Compared with the standard SURF (violet trajectory), PE and Pσ denote the percentages by which mean value and standard deviation have been lowered through the five experiments, respectively.
Table 2. Mobile direction angles. Experiments 1–5 correspond to the results shown in Figures 8,9,10,1112, respectively. E (°) denotes the mean value of mobile direction angle, and σ (°) denotes the standard deviation of mobile direction angle. Compared with the standard SURF (violet trajectory), PE and Pσ denote the percentages by which mean value and standard deviation have been lowered through the five experiments, respectively.
TrajectoriesExperiment 1Experiment 2Experiment 3Experiment 4Experiment 5PEPσ

EσEσEσEσEσ
Violet41.410.850.312.551.612.857.913.359.513.900
Blue29.36.634.77.833.97.537.28.340.58.732.4%38.6%
Green23.85.027.65.928.36.232.26.934.57.343.8%50.7%
Red18.13.920.54.521.64.623.14.825.35.258.2%63.7%
Table 3. Average homing errors. Feature sets 1–5 correspond to the features stored at the nodes and obtained during the above five types of experiments. Compared with the standard SURF (violet trajectory), P denotes the percentage by which the homing error has been lowered through the five feature sets.
Table 3. Average homing errors. Feature sets 1–5 correspond to the features stored at the nodes and obtained during the above five types of experiments. Compared with the standard SURF (violet trajectory), P denotes the percentage by which the homing error has been lowered through the five feature sets.
ApproachFeature Set 1Feature Set 2Feature Set 3Feature Set 4Feature Set 5P
Violet45.156.353.759.863.40
Blue31.235.436.138.941.534.0%
Green22.627.129.533.935.446.9%
Red17.621.123.825.426.559.0%

Share and Cite

MDPI and ACS Style

Zhu, Q.; Liu, X.; Cai, C. Feature Optimization for Long-Range Visual Homing in Changing Environments. Sensors 2014, 14, 3342-3361. https://doi.org/10.3390/s140203342

AMA Style

Zhu Q, Liu X, Cai C. Feature Optimization for Long-Range Visual Homing in Changing Environments. Sensors. 2014; 14(2):3342-3361. https://doi.org/10.3390/s140203342

Chicago/Turabian Style

Zhu, Qidan, Xue Liu, and Chengtao Cai. 2014. "Feature Optimization for Long-Range Visual Homing in Changing Environments" Sensors 14, no. 2: 3342-3361. https://doi.org/10.3390/s140203342

Article Metrics

Back to TopTop