Next Article in Journal
Full-Field Calibration of Color Camera Chromatic Aberration using Absolute Phase Maps
Previous Article in Journal
Highly Sensitive Sputtered ZnO:Ga Thin Films Integrated by a Simple Stencil Mask Process on Microsensor Platforms for Sub-ppm Acetaldehyde Detection
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

A Novel Method of Aircraft Detection Based on High-Resolution Panchromatic Optical Remote Sensing Images

1
Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun 130033, China
2
University of Chinese Academy of Sciences, Beijing 100049, China
*
Author to whom correspondence should be addressed.
Sensors 2017, 17(5), 1047; https://doi.org/10.3390/s17051047
Submission received: 27 March 2017 / Revised: 26 April 2017 / Accepted: 3 May 2017 / Published: 6 May 2017
(This article belongs to the Section Remote Sensors)

Abstract

:
In target detection of optical remote sensing images, two main obstacles for aircraft target detection are how to extract the candidates in complex gray-scale-multi background and how to confirm the targets in case the target shapes are deformed, irregular or asymmetric, such as that caused by natural conditions (low signal-to-noise ratio, illumination condition or swaying photographing) and occlusion by surrounding objects (boarding bridge, equipment). To solve these issues, an improved active contours algorithm, namely region-scalable fitting energy based threshold (TRSF), and a corner-convex hull based segmentation algorithm (CCHS) are proposed in this paper. Firstly, the maximal variance between-cluster algorithm (Otsu’s algorithm) and region-scalable fitting energy (RSF) algorithm are combined to solve the difficulty of targets extraction in complex and gray-scale-multi backgrounds. Secondly, based on inherent shapes and prominent corners, aircrafts are divided into five fragments by utilizing convex hulls and Harris corner points. Furthermore, a series of new structure features, which describe the proportion of targets part in the fragment to the whole fragment and the proportion of fragment to the whole hull, are identified to judge whether the targets are true or not. Experimental results show that TRSF algorithm could improve extraction accuracy in complex background, and that it is faster than some traditional active contours algorithms. The CCHS is effective to suppress the detection difficulties caused by the irregular shape.

1. Introduction

Remote sensing images contain large amount of geo-graphical environmental information and have widely used in different scientific fields. Target detection (e.g., bridges, ships and aircrafts) is one of the fundamental applications in remote sensing images. It is an important basis to conduct military attack and intelligent transportation, rapidly and correctly [1,2]. Among these targets, aircraft detection becomes a research hotspot due to its important military and traffic status [3,4,5,6,7,8,9,10,11,12,13,14]. With the resolution improvement of optical remote sensing image, the abundant detail information of targets makes target detection more easily. However, some intractable obstacles, including the hugeness and complication of spatial data, changeable environment and relatively low signal-to-noise ratio (SNR), cause the low detection efficiency and accuracy.
Specifically, there are two major challenges that hinder the improvement of the detection effect:
One is that how to extract targets exactly from the complex background. For this challenge, there are few systematic methods to cope with it. A suspicious target is usually extracted from the thumbnail slice containing the well-marked target. Arandjelović [3] proposed an efficient and accurate set-based registration algorithm of aerial images through constraint graphs optimization of pairwise image registration. They formulate the joint registration problem as an optimization scheme. Although this idea is very novel, it needs to be further studied before the application to the target detection area.
The other one is identifying the targets in case of irregular target shapes target due to natural conditions (low signal-to-noise ratio, illumination condition or swaying photographing) and occlusion by surrounding objects (boarding bridge, equipment). This part could be classified into two major strands. One strand is to take advantage of the shape of an aircraft. With the aid of the obvious structural features, including principal axis, contour and rigidity, the scholars extracted and detected the aircrafts by symmetry and various rotation-invariant moments of aircrafts. Some classical algorithms are combined to be used, such as principle component analysis, Zernike invariant moments, contour tracking and so on [4,5,6,7,8]. These methods always extract rotation-invariant or symmetrical features from an overall contour obtained by gray value threshold segmentation, which means the extracted contour should be perfect and regular. However, the idealized objects make these methods fail when the target is deformed, irregular or asymmetric due to low SNR, optical aberrance or swaying photographing. Arandjelović [9] proposed a gradient edge mapping algorithm for face recognition. The vertical symmetry was used in the edge extraction process, which may be applied to symmetry target detection in remote sensing image. The other strand focuses on building a target sample library to match the targets. Because of the uniqueness of aircraft structure, few false-alarm targets can be matched the aircraft structure. For example, Cheng et al. [11] extract the gradient direction of targets through the HOG operator and obtain the aircraft targets by matching the trained samples to the region of interest (ROI). The aircrafts gradient direction feature is not unique, leading to the large number of false targets in the preliminary processing, and increasing the burden of subsequent processing. Moreover, it is hard to establish the sample library that contains all the types of aircrafts. Facing the difficulty of establishment of the sample library, Liu et al. [12] match the targets by calculating an average template based on a large number of samples and identify the targets in accordance with their similarity to the template. But such method also needs a lot of samples for training, and the average template could not match all the targets. In summary, the above-mentioned methods all focus on the whole targets in the small and clean image slice. Arandjelović [13] proposed a method of object matching for smooth and untextured images using boundary descriptors, which comprises a profile of sampled boundary normals’ directions. The equidistant key points are obtained by sampling the image boundary curves. Then, the boundary normals of a local profile around one key point were utilized to describe this point. The proposed method was shown to be robust in many classical object image sets. But the local descriptor is more sensitive to local noise and incorrect shape components, which commonly exist in remote sensing images.
In this paper, we solve the above-mentioned problems in two steps. Unlike the ocean target detection, the remote sensing image of land consists of many gray levels, which causes relative difficulty to extract the candidates from the image utilizing pure threshold segmentation, e.g., the popular Otsu’s algorithm [15]. Therefore, an improved active contour method is provided for ROI extraction. To ensure fast operation and accurate segmentation, the proposed approach combines Otsu’s algorithm with active contour to segment the images. Meanwhile, the aircraft is a rigid target with fixed structure, a special shape and salient corners. According to these characteristics, we propose an algorithm named corner-convex hull based segmentation algorithm (CCHS), and it can effectively overcome the problem of irregular aircraft confirmation. The algorithm analyzes contour features of an aircraft, extracts the aircraft corners, draws the minimum convex hull via extracted corners, and divides the convex hull into several fragments. A series of new features named target-to-fragment radio (TFR) and fragment-to-hull radio (FHR) obtained from the segmented targets can help the targets identification by support vector machine (SVM) [16]. The advantages of the present approach include (1) improving the precision of the aircraft ROI extraction in the complex background; (2) improving the detection precision of the irregular shape aircraft after ROI extraction. Experiments on Google Earth images show that the proposed method is effective and robust in complex and large sense.

2. Detection Algorithm

2.1. A Whole Process

The proposed detection algorithm is divided into two steps, as shown in the Figure 1. In the stage of extracting candidate area, the input image is binarized through the TRSF algorithm, and the binary image is extracted the connected region after morphological treatment. In the stage of target identification, the CCHS algorithm checks the convex hull via the corners of connected regions, and divides the targets into fragments. At last, the fragments are analyzed to confirm the targets and output the results. Figure 2 gives an intuitive illustration by an example.

2.2. ROI Extraction

The binarization of the input image is a necessary operation before extracting the connected regions. In the Otsu’s algorithm, the segmentation threshold is obtained through the gray histogram calculation based on statistical information. Such algorithm is characterized by simple, stable and effective calculations and ensures the image segmentation quality in the case of simple background. It is usually regarded as one of the most fashionable threshold segmentation algorithms. However, considering that a remote sensing image consists of numerous grey levels, it seems a difficult task to extract the ROI from such a complex image directly through Otsu’s algorithm. It is found out that the aircraft luminance is usually very high through observation. So the suspected aircraft area can be obtained through further gray histogram segmentation based on Otsu’s algorithm pre-segmentation. The active contour algorithm, such as C-V model [17], can effectively segment a target with a weak boundary. But the conventional C-V model has low computational efficiency with many iterations, and it is unable to process the case of many gray levels. By approaching its energy function curve to the target contour, the Region-Scalable Fitting (RSF) model [18] can segment the image with uneven intensity distribution. But such model is sensitive to the contour line. Simultaneously, the initial contour has certain effects on both the number of iterations and convergence. By combining the merits of Otsu’s algorithm and RSF, this paper proposes the improved segmentation algorithm TRSF for target segmentation and extraction.
The Otsu’s algorithm characterizes the separating properties of both the targets and the background through the Interclass Variance, which is defined as:
σ 2 ( T ) = W a ( μ a μ ) 2 W b ( μ b μ ) 2 ,
where σ 2 is the variance between two classes, W a is the proportion of Class A in the image, μ a is the gray average of Class A, W b is the proportion of Class B in the image, μ b is the gray average of Class B, and μ is the overall gray average of the image. The T , which maximizes the total interclass variance σ 2 ( T ) , is just the segmentation threshold between the minimum and maximum gray values. The segmented result is defined as level set function ϕ as follows.
ϕ ( x ) = { 1   x Ω 1 1 x Ω 2 ,
where Ω 1 is the segmented quasi-target region, and Ω 2 is the quasi-background region. The energy function of RSF model in the level set is:
{ E = i = 1 2 λ i Ω Ω K σ ( x y ) | I ( y ) f i ( x ) | 2 M i ( ϕ ( y ) ) d y d x L ( ϕ ) = Ω | H ( ϕ ( x ) ) | d x P ( ϕ ) = Ω 1 2 ( | ϕ ( x ) | 1 ) 2 d x ,
Equation (3) is composed of three terms, where L ( ϕ ) is the length of closed contour line, P ( ϕ ) is the regular term of level set, and λ i is a non-negative constant representing the weighting coefficient of every energy term. M 1 ( ϕ ) = H ( ϕ ) and M 2 ( ϕ ) = 1 H ( ϕ ) . The H ( ϕ ) is approximated to Heaviside function:
H ε ( x ) = 1 2 [ 1 + 2 π arctan ( x ε ) ] ,
The introduced non-negative kernel function K σ is defined as:
K σ ( u ) = 1 2 π σ 2 exp ( | u | 2 2 σ 2 ) ,
where σ is positive scale parameter.
When the closed boundary ϕ is not located in the two homogeneous regions, the minimum of E cannot be achieved. The variable E gets its minimal value if the contour line is on the boundary between two homogeneous regions. Therefore, the two matching functions f 1 ( x ) and f 2 ( x ) are adopted to fit the image intensity of the areas on both sides of the boundary C, which is defined as:
f i ( x ) = K σ ( x ) × [ M i ( ϕ ( x ) ) I ( x ) ] K σ ( x ) × M i ( ϕ ( x ) ) ,
The introduced kernel function K σ can control the boundary scalability, which makes the most of local intensity information of an image. This model segments an image with intensity inhomogeneity and could achieve relatively good segmentation results for some objects with weak boundaries.
This paper employs in coarse-to-fine fashion to extract the ROIs. Figure 3 shows this process in detail as follows:
  • Downsample the original image by a factor of 4, and then implement initial Otsu’s algorithm segmentation for the low-resolution image. In this way, the complexity of computation is reduced.
  • After the completion of Otsu’s algorithm, weed out small areas at first, and then process the segmented region through secondary segmentation and morphological processing at a high gray level (10%). At this point, the aircraft area can be located basically.
  • Take the segmented binary image as the local region and initial condition of active contour, define the initial level set function ϕ as the piecewise function containing only two function values, namely 1 and −1, and use this function to guide the curve evolution in the image.
  • Evolve the level set function through energy functionality, and search for the level set K = { x | ϕ ( x ) 0 } . When the symbol of ϕ ( x ) changes, the corresponding energy functionality will decrease so that ϕ ( x ) = ϕ ( x ) . Otherwise, keep ϕ ( x ) unchanged.
  • Repeat the steps c and d, and stop the repetition when the energy functionality doesn’t change any more. Finally, the segmentation result can be obtained.
  • Process the segmented binary image morphologically, and weed out small areas.
For subsequent processing, we employ the searching algorithm for 8-neighborhood connected regions to extract the connected regions from a binary image [19]. Suppose the image function is f ( x , y ) , the ( p + q )-order moment is given as:
m p q = x = 1 M y = 1 N x p y q f ( x , y ) ,     p , q = 0 , 1 , 2
The area is the zero-order moment, ( x ¯ , y ¯ ) is the centroid of a connected region, and x ¯ = m 10 / m 00 , y ¯ = m 01 / m 00 .

2.3. ROI Analysis to Confirm Target

In this section, a corner-convex-hull based segmentation (CCHS) algorithm is given. Furthermore, the new features—namely TFR and FHR—are introduced into the judgment of true targets particularly.

2.3.1. Corner-Convex-Hull Based Segmentation Algorithm

After the last step, many of the connected regions, which include the connected regions of the aircraft target, have been extracted by TRSF algorithm. Then we would exclude the false targets in these connected regions. The aircraft targets should be filtered in accordance with the aircraft features that we presented below. The most fundamental features, such as area and aspect ratio of the minimum external moment, are used to eliminate a large number of regions. Since the aircraft contour takes on typical geometric features, its feature points are available for target screening. With a small calculation burden and steady results, the Harris corner operator [20] is adopted. Corners are important local features of a target because they cannot only keep important shape information of a target, but also reduce redundant data of the target effectively to relieve the computation burden. Besides, corners remain invariant during transition, zooming and rotation. Thus, corner feature is the crucial basis for aircraft detection method proposed by this paper. The response value R of Harris corner points is:
R = det ( C ) k × trace 2 ( C ) ,
where C ( x ) = [ I u 2 ( x ) I u v ( x ) I u v ( x ) I u 2 ( x ) ] , in which I u ( x ) , I v ( x ) , and I u v ( x ) are the partial derivatives and second-order mixed partial derivative of the gray value of image point x in the R and v directions; k is an empirical value, usually between 0.04~0.06 and we set 0.04. By setting the threshold T , a point whose response value R is bigger than T is considered as Harris corner. The detection result of target corners is shown in the Figure 4b.
After the extraction of corners from a segmented binary image, it is found that an aircraft has five salient corners at the aircraft nose, wings and tail respectively. Convex hull [21] of these corners reveals that the convex hull of aircraft corners is a pentagon constituting only five vertexes or corners at the aircraft nose, wings and tail respectively, while other corners are located inside the convex hull, as shown in Figure 4c. Some defective corners caused by segmentation are also found inside the convex hull. Hence, some defects arising from threshold segmentation have no effects on the extraction of convex hull. The corner extraction from a binary image can also shorten the operation time.
When multiple adjacent corners overlap, the similar corners can be merged in accordance by applying the Euclidean distance to prevent one feature point from being detected as several corners. In this way, the pentagon corners for an aircraft become extremely clear. The usage of extracted corner set can greatly reduce the calculation burden in convex closure detection in a whole connected region.
The process of convex hull detection in this section is given as follows (Algorithm 1):
Algorithm 1
Sensors 17 01047 i001
Points in the result will be listed in counter-clockwise order.

2.3.2. New Features for Target Confirmation

After drawing the convex hull of ROIs, there are still some false candidates in a pentagon shape which needs to be further wed out. Here, we propose a segmentation approach that divides a connected region into several parts so that every slice can be independently analyzed to remove the false targets.
In this section, the centroid of a connected region is connected to the vertexes of convex hull to segment the region into five fragments. The shape of each fragment is a triangle which includes a part of connected region and a part of convex hull background. The new feature TFR describes the radio of the area of each target part to each triangle and the other feature FHR describes the radio of area of each triangle to the whole hull. The definitions are as follows.
T F R i = S t a r g e t S f r a g m e n t ,
F H R i = S f r a g m e n t S c o n v e x ,
where S t a r g e t is the number of the pixels in a fragment whose value is 1, S f r a g m e n t is the number of pixels in every whole fragment, and S c o n v e x is the number of pixels in the whole fragment.
The result of fragment segmentation is shown in Figure 5. Moreover, Table 1 gives the features of aircraft fragments. Through the five features, an aircraft target can be identified.
According to Table 1, fragments 2 and 4 have a very small TFR of around 10%, followed by the TFR of fragments 1 and 5, which is about 70%. The TFR of fragment 3 is more than 80%, the highest among them. Some regularities can also be found in the distribution of FHR. By adopting these features, false alarms can be rejected.
The fragment features are applied to different aircraft models [22] shown in the Figure 6, and the corresponding data are shown in the Table 2. It is found that the two features of different aircraft models show some regularity. The pseudo color map shows the regularity more obviously in Figure 7.
If the extracted aircraft is irregular, as shown in Figure 8a, the invariant moment based on symmetry will be no longer valid. The features proposed in this paper could cope with this problem. Figure 8b illustrates that the false alarm can be rejected though it also has five vertexes.
Through the SVM learning and training for a large number of remote sensing aircraft image slices, the targets can be judged accurately [23].

3. Test Process and Results

Due to the lack of standard data sets of high resolution remote sensing images for object detection, we evaluate the proposed method on 52 images from Google Earth for aircraft detection with the resolution of 60 cm/pixel. (Since we concentrates on the aircraft detection in panchromatic image, we convert the RGB Google Earth images into grayscale images for the experiments). We collect 500 positive patches from the Google Earth to train the TFR and FHR features. Simulation is done with Visual Studio 2012 in a computer with a CPU of Intel 4 GHz and 32 G memory to evaluate the calculation time.

3.1. Datasets and Experimental Settings

As shown in Figure 9, we choose three representative images among the numerous images to show the application of our method.

3.2. ROI Extraction

In this part, we show the result of the ROI extraction using our proposed method. According to [17], we use the following parameters in this paper: σ = 3.0 , time step Δ t = 0.1 , μ = 1 , and ν = 0.001 × 255 × 255 . The results of the three images shown in the Section 3.1 are given in the Figure 10.
Then, the segmentation algorithms of Otsu’s algorithm, watershed, and C-V level set are compared with our proposed method. Figure 11 shows that the results of our method are significantly superior to some other algorithms because of its high accuracy. Table 3 shows the TRSF’s operation speed is faster than conventional C-V model algorithm. Although it runs slower than the gray value threshold algorithm and watershed algorithm, the segmentation effect is obviously better than the two previous algorithms.

3.3. Target Identification

After the ROI segmentation, the corners are detected and their convex hulls are drawn. Then the ROI is segmented into several fragments and the features are extracted. That we adopt the Harris points to calculate the convex hull can greatly improve the running speed compared with the method based on the whole contour. Table 4 shows the time and efficiency of convex closure detection based on the corner points and a whole contour respectively. It is evident that the detection efficiency based on the corners is higher.
In details, the local enlarged drawing of the Figure 11c of our method in segmentation stage is shown in the Figure 12. The irregular parts are signed by red circles. Our method can correct this defect and detect the targets successfully. Moreover, the swaying photographing image in Figure 8b can also detect accurately. The detection effect is shown in Figure 13.

3.4. Conclusion Remarks of Simulations

After the results of target detection output, we evaluate the performance metrics in terms of precision and recall, denoted as P and R [24]. The precision is aimed to maximize the target detection without missing any target, so the detection rate is the most important criterion in the detection result assessment. The recall is the percentage of false targets in the detection results. The lower the false alarm rate is, the better the algorithm is. As we can see in Table 5, A is the number of images correctly classified as having airports, B denotes the number of images wrongly classified as having airports, and C represents the number of images wrongly classified as having other objects.
P and R are respectively defined as:
P = A A + B ,
R = A A + C ,
In order to verify the proposed method has a positive impact on aircraft detection, a wide range of tests are conducted on a large number of images involving 312 aircrafts in total. In comparison, a method based on size and Hu moments multi-normal features and a method based on HOG-SVM are used as comparison. All of the methods extract the ROIs using our first stage method and then validate the results in different features. In addition, the method in [10] is also used as comparison to evaluate the proposed method. The recall and precision of the four different methods in total are listed in Table 6. According to the table, it is obvious that the algorithm proposed by this paper has a better detection effect with higher recall and precision. However, the average processing time is not small, mainly because of the iteration of energy function method used for ROI extraction. Overall, the proposed method is effective and robust. And there is room to improve the processing speed, although it is much better than traditional method. The compare result is illustrated in Figure 14 by using precision–recall curves. At last, a result image is given in Figure 15 for exhibition.

4. Conclusions

In summary, a novel method has been proposed in two steps to settle the problem of low aircraft detection precision in high-resolution panchromatic optical remote sensing images. First of the two steps, an improved RSF algorithm has been suggested to solve ROI extraction difficulty in complex background. Then, a corner-convex-hull based segmentation algorithm has been introduced to solve the aircraft shape irregularity problems. The new features, target-to-fragment radio (TFR) and fragment-to-hull radio (FHR), have been constructed to take into the false targets elimination. The performance has been evaluated and validated in terms of precision and recall using the images from Google Earth. As a conclusion, it can be observed that the proposed method is tractable to be built and implemented to improve the accuracy.
The further research is underway in the following two directions.
  • It is of interest to extend the complex remote sensing background of the target extraction from the aircraft mutatis mutandis to other targets by introducing new feature descriptors, for instance vehicles or marine vessels.
  • We are interested in how to combine the panchromatic with multi-spectral information to detect the aircraft target. It is promising to use the high resolution of panchromatic images and the different reflectivity of different spectral bands for different objects to accurate detect targets.

Acknowledgments

This work has been supported by The National 863 High Technology Research and Development Program of China (No. 863-2-5-1-13B). The authors wish to thank the associate editor and the anonymous reviewers for their valuable suggestions. The first author would like to thank Bowen Yi and Jiantao Peng for their language help. The authors wish to express their gratitude to CIOMP for its experiment and site support. A special thanks is expressed to Google Earth for the images support.

Author Contributions

Wensheng Wang proposed the idea of the method. Wensheng Wang and Ting Nie performed conceived experiment. Wensheng Wang designed and performed the experiments; Tianjiao Fu analyzed the data; Wensheng Wang wrote the paper and all the authors read and approved the final paper.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Asrar, G. Theory and Applications of Optical Remote Sensing; John Wiley and Sons: New York, NY, USA, 1989; pp. 51–53. [Google Scholar]
  2. Tomás, R.; Li, Z.H. Earth observations for geohazards: Present and future challenges. Remote Sens. 2017, 9, 194. [Google Scholar] [CrossRef]
  3. Arandjelović, O.; Pham, D.; Venkatesh, S. Efficient and accurate set-based registration of time-separated aerial images. Pattern Recognit. 2015, 48, 3466–3476. [Google Scholar] [CrossRef]
  4. Shen, H.Y.; Li, Y.E. The aircraft target recognition based on edge contour feature matching. J. Test Meas. Technol. 2013, 27, 260–265. [Google Scholar]
  5. Cai, R.Z.M.; Zhang, M.C. Aircraft recognition via structural features. Comput. Eng. Appl. 2013, 49, 174–177. [Google Scholar]
  6. Dudani, S.A.; Breading, K.J.; Mcghee, R.B. Aircraft identification by moment invariants. IEEE Trans. Comput. 1977, C-26, 39–46. [Google Scholar]
  7. Liu, F.; Yu, P.; Liu, K. Research concerning aircraft recognition of remote sensing images based on ICA Zernike invariant moments. CAAI Trans. Intell. Syst. 2011, 6, 51–56. [Google Scholar]
  8. Zhang, Y.N.; Yao, G.Q. Plane recognition based on moment invariants and neural networks. Comput. Knowl. Technol. 2009, 5, 3771–3778. [Google Scholar]
  9. Arandjelović, O. Gradient edge map features for frontal face recognition under extreme illumination changes. In Proceedings of the British Machine Vision Conference, Surrey, UK, 3–7 September 2012. [Google Scholar]
  10. Zhang, W.C.; Sun, X. Object detection in high-resolution remote sensing images using rotation invariant parts based model. IEEE Geosci. Remote Sens. Lett. 2014, 11, 74–78. [Google Scholar] [CrossRef]
  11. Cheng, G.; Han, J.W.; Li, K.M.; Hu, X.T. Object detection in remote sensing imagery using a discriminatively trained mixture model. ISPRS J. Photogramm. Remote Sens. 2013, 85, 32–43. [Google Scholar]
  12. Liu, G.; Sun, X.; Fu, K.; Wang, H. Aircraft recognition in high-resolution satellite images using coarse-to-fine shape prior. IEEE Geosci. Remote Sens. Lett. 2013, 10, 573–577. [Google Scholar]
  13. Arandjelović, O. Object matching using boundary descriptors. In Proceedings of the British Machine Vision Conference, Surrey, UK, 3–7 September 2012. [Google Scholar]
  14. Wu, Q.C.; Sun, H.; Sun, X.; Zhang, D.B.; Fu, K.; Wang, H.Q. Aircraft recognition in high-resolution optical satellite remote sensing images. IEEE Geosci. Remote Sens. Lett. 2015, 12, 112–116. [Google Scholar]
  15. Otsu, N. A threshold selection method from gray-level histogram. IEEE Trans. Syst. Man Cybern. 1979, 9, 62–66. [Google Scholar] [CrossRef]
  16. Gunn, S.R. Support Vector Machines for Classification and Regression; ISIS Technical Report; University of Notre Dame: Notre Dame, IN, USA, 1998. [Google Scholar]
  17. Chan, T.F.; Vese, L.A. Active contours without edges. IEEE Trans. Image Process. 2001, 10, 266–277. [Google Scholar] [CrossRef] [PubMed]
  18. Li, C.M.; Kao, C.R.; Gore, J.C.; Ding, Z.H. Minimization of region-scalable fitting energy for image segmentation. IEEE Trans. Image Process. 2008, 17, 1940–1949. [Google Scholar] [PubMed]
  19. Yang, X.D. An Improved Algorithm for Labeling Connected Components in a Binary Image; Cornell University Computer Science Department: Ithaca, NY, USA, 1989. [Google Scholar]
  20. Harris, C.; Stephens, M. A combined corner and edge detector. In Proceedings of the 4th Alvey Vision Conference, Manchester, UK, 31 August–2 September 1988; pp. 147–151. [Google Scholar]
  21. Barber, C.B.; Dobkin, D.P.; Huhdanpaa, H. The quickhull algorithm for convex hulls. ACM Trans. Math. Softw. 1996, 22, 469–483. [Google Scholar] [CrossRef]
  22. Hsieh, J.W.; Chen, J.M.; Chuang, C.H.; Fan, K.C. Aircraft type recognition in satellite images. Proc. Inst. Elect. Eng. 2005, 152, 307–315. [Google Scholar] [CrossRef]
  23. Huang, X.; Zhang, L. An SVM ensemble approach combining spectral, structural, and semantic features for the classification of high-resolution remotely sensed imagery. IEEE Trans. Geosci. Remote Sens. 2013, 51, 257–272. [Google Scholar] [CrossRef]
  24. Wang, Y.T.J.; Pan, L. Automatic airport recognition based on saliency detection and semantic information. ISPRS Int. J. Geo-Inf. 2016, 5, 115–133. [Google Scholar] [CrossRef]
Figure 1. Proposed aircraft detection algorithm framework.
Figure 1. Proposed aircraft detection algorithm framework.
Sensors 17 01047 g001
Figure 2. Diagrammatic sketch of target detection process for an aircraft example. In the figure, (a) is about the input, (b) is about the binarization, (c) is regarding the extracted connected regions, (d) is for determination of corner convex hull, (e) is the segmentation effect, (f) shows each fragment, and (g) gives the final results.
Figure 2. Diagrammatic sketch of target detection process for an aircraft example. In the figure, (a) is about the input, (b) is about the binarization, (c) is regarding the extracted connected regions, (d) is for determination of corner convex hull, (e) is the segmentation effect, (f) shows each fragment, and (g) gives the final results.
Sensors 17 01047 g002
Figure 3. Extraction of image target region: (a) Original image; (b) the first Otsu’s algorithm segmentation; (c) the second segmentation; (d) Initial RSF boundary; (e) RSF result; (f) Binary segment result.
Figure 3. Extraction of image target region: (a) Original image; (b) the first Otsu’s algorithm segmentation; (c) the second segmentation; (d) Initial RSF boundary; (e) RSF result; (f) Binary segment result.
Sensors 17 01047 g003
Figure 4. Corner-convex hull detection result. (a) The connected region of aircraft in the Figure 3f. (b) The Harris points of the connected region. (c) The blue points are the interior points. (d) The result of convex hull detection of these points.
Figure 4. Corner-convex hull detection result. (a) The connected region of aircraft in the Figure 3f. (b) The Harris points of the connected region. (c) The blue points are the interior points. (d) The result of convex hull detection of these points.
Sensors 17 01047 g004
Figure 5. Aircraft fragment segmentation. (a) The target is marked with corners and convex hull. The pink pentagon hull is the convex hull of these corners. The blue corners are the internal corners and the red corners are the external corners. The yellow lines are the dividing lines. (b) The target is divided into five fragments by the dividing lines.
Figure 5. Aircraft fragment segmentation. (a) The target is marked with corners and convex hull. The pink pentagon hull is the convex hull of these corners. The blue corners are the internal corners and the red corners are the external corners. The yellow lines are the dividing lines. (b) The target is divided into five fragments by the dividing lines.
Sensors 17 01047 g005
Figure 6. Different aircraft models.
Figure 6. Different aircraft models.
Sensors 17 01047 g006
Figure 7. The pseudo color map of TFR and FHR. For different types of aircraft, each feature has few changes. For TFRi, the value of TFR1, TFR3, and TFR5 are relatively large; contrarily, the value of TFR2 and TFR4 are relatively small. For FHRi, they are relatively average with a small variance.
Figure 7. The pseudo color map of TFR and FHR. For different types of aircraft, each feature has few changes. For TFRi, the value of TFR1, TFR3, and TFR5 are relatively large; contrarily, the value of TFR2 and TFR4 are relatively small. For FHRi, they are relatively average with a small variance.
Sensors 17 01047 g007
Figure 8. The example of irregular shape aircraft and false alarm from the connected region of Google Earth (a) Aircraft defect; (b) False alarm; (c) Detection results of aircraft defect; (d) Detection results of false alarm; (e) Segmentation result of aircraft defect; (f) Segmentation results of false alarm.
Figure 8. The example of irregular shape aircraft and false alarm from the connected region of Google Earth (a) Aircraft defect; (b) False alarm; (c) Detection results of aircraft defect; (d) Detection results of false alarm; (e) Segmentation result of aircraft defect; (f) Segmentation results of false alarm.
Sensors 17 01047 g008
Figure 9. The original images. (a) A normal image. (b) Two aircrafts with swaying photographing. (c) An aircraft in red rectangle is connected together with something others, which can be seen in the enlarged drawing (d). (d) The superfluous parts are shown in red circles of (c). Their sizes are respective 830 × 868, 1183 × 1631, and 2060 × 1992.
Figure 9. The original images. (a) A normal image. (b) Two aircrafts with swaying photographing. (c) An aircraft in red rectangle is connected together with something others, which can be seen in the enlarged drawing (d). (d) The superfluous parts are shown in red circles of (c). Their sizes are respective 830 × 868, 1183 × 1631, and 2060 × 1992.
Sensors 17 01047 g009
Figure 10. ROI extraction results: (ac) the initial RSF contours obtained by threshold segmentation; (df) the final RSF contours after energy function stability.
Figure 10. ROI extraction results: (ac) the initial RSF contours obtained by threshold segmentation; (df) the final RSF contours after energy function stability.
Sensors 17 01047 g010
Figure 11. Comparison of segmentation results of different methods. The column (ac) results correspond to the original images of Figure 9a–c.
Figure 11. Comparison of segmentation results of different methods. The column (ac) results correspond to the original images of Figure 9a–c.
Sensors 17 01047 g011
Figure 12. The local enlarged drawing of the Figure 11c. (a) is the original images. (b) is the local enlarged drawing. (c) marks the irregular parts by red circles.
Figure 12. The local enlarged drawing of the Figure 11c. (a) is the original images. (b) is the local enlarged drawing. (c) marks the irregular parts by red circles.
Sensors 17 01047 g012
Figure 13. Detection results (continued from Figure 9 of the Section 3.1): (ac) are convex hull results of ROIs. (df) are the detection results.
Figure 13. Detection results (continued from Figure 9 of the Section 3.1): (ac) are convex hull results of ROIs. (df) are the detection results.
Sensors 17 01047 g013
Figure 14. Precision–recall curves of different methods.
Figure 14. Precision–recall curves of different methods.
Sensors 17 01047 g014
Figure 15. The detection result of an image form Google Earth. The size of the image is 4096 × 4096.
Figure 15. The detection result of an image form Google Earth. The size of the image is 4096 × 4096.
Sensors 17 01047 g015
Table 1. Fragment features.
Table 1. Fragment features.
Features Sensors 17 01047 i002 Sensors 17 01047 i003 Sensors 17 01047 i004 Sensors 17 01047 i005 Sensors 17 01047 i006
TFR73.6%10.3%73.8%9.8%80.5%
FHR23.4%19.8%23.3%19.6%13.9%
Table 2. Fragment feature data of different aircraft models shown in Figure 6. For the two series of features of each type aircraft in longitudinal data, each of these features is stable, and there is little change among the different types of models. TFR1, TFR3 and TFR5 fluctuate about from 50% to 80%. TFR2 and TFR4 fluctuate about from 10% to 30%. Each of FHRs are also stable, about from 10% to 25%.
Table 2. Fragment feature data of different aircraft models shown in Figure 6. For the two series of features of each type aircraft in longitudinal data, each of these features is stable, and there is little change among the different types of models. TFR1, TFR3 and TFR5 fluctuate about from 50% to 80%. TFR2 and TFR4 fluctuate about from 10% to 30%. Each of FHRs are also stable, about from 10% to 25%.
FeatureTarget-to-Fragment RadioFragment-to-Hull Radio
NumberTFR1TFR2TFR3TFR4TFR5FHR1FHR2FHR3FHR4FHR5
Type 155.3%10.8%55.3%10.8%60.3%19.8%23.9%19.8%23.9%12.6%
Type 261.5%12.4%61.5%12.4%68.5%18.8%24.0%18.8%24.0%14.4%
Type 365.4%12.5%65.4%12.5%69.4%18.8%24.4%18.8%24.4%13.6%
Type 472.1%14.9%72.1%14.9%72.1%20.1%23.2%20.1%23.2%13.4%
Type 571.5%15.6%71.5%15.6%75.5%20.4%23.2%20.4%23.2%12.8%
Type 658.6%19.8%58.6%19.8%77.2%21.4%20.8%21.4%20.8%15.6%
Type 778.6%25.4%78.6%25.4%74.6%20.5%19.1%20.5%19.1%20.8%
Type 879.2%24.3%79.2%24.3%74.5%20.4%19.3%20.4%19.3%20.6%
Type 963.2%20.1%63.2%20.1%79.7%18.3%21.5%18.3%21.5%20.4%
Type 1077.8%10.2%77.8%10.2%69.3%17.2%23.5%17.2%23.5%18.6%
Table 3. Comparison of segmentation speeds (unit: s).
Table 3. Comparison of segmentation speeds (unit: s).
NumberSizeOtsu’s AlgorithmWatershed C-V Level SetProposed Algorithm
a830 × 8680.0130.89220.1261.134
b1183 × 16310.0281.54860.5742.143
c2060 × 19920.0522.056100.5623.289
Table 4. Comparison of convex hull detection speeds (unit: ms).
Table 4. Comparison of convex hull detection speeds (unit: ms).
NumberTraditional Convex HullProposed Algorithm
1530.6
21193.5
32381.2
Table 5. Relationships among the detection results.
Table 5. Relationships among the detection results.
RelatedUnrelated
DetectedA (true positives)B (false positive)
UndetectedC (false negative)
Table 6. Relationships among the detection results.
Table 6. Relationships among the detection results.
MethodPrecision/%Recall/%Average Processing Time/s
Multi-normal features81.372.12.89
HOG-SVM86.783.33.58
Method in [10]87.985.61.94
Our method90.992.82.56

Share and Cite

MDPI and ACS Style

Wang, W.; Nie, T.; Fu, T.; Ren, J.; Jin, L. A Novel Method of Aircraft Detection Based on High-Resolution Panchromatic Optical Remote Sensing Images. Sensors 2017, 17, 1047. https://doi.org/10.3390/s17051047

AMA Style

Wang W, Nie T, Fu T, Ren J, Jin L. A Novel Method of Aircraft Detection Based on High-Resolution Panchromatic Optical Remote Sensing Images. Sensors. 2017; 17(5):1047. https://doi.org/10.3390/s17051047

Chicago/Turabian Style

Wang, Wensheng, Ting Nie, Tianjiao Fu, Jianyue Ren, and Longxu Jin. 2017. "A Novel Method of Aircraft Detection Based on High-Resolution Panchromatic Optical Remote Sensing Images" Sensors 17, no. 5: 1047. https://doi.org/10.3390/s17051047

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop